Oglavlenie fajla ( /LYNX/WEBMASTER/robots-faq.txt )


  • The Web Robots FAQ
  • Neskol'ko slov o tom, kak rabotayut roboty (spiders) poiskovyh mashin
  • Standart isklyuchenij dlya robotov

  • Size: 43399 bytes
    Last-modified: Thu, 28 May 1998 14:14:26 GMT

    I kusochkami po 40Kb: [1][2]