Robots.txt: verschil tussen versies
Uit dotSearch
| Regel 1: | Regel 1: | ||
| − | + | User-agent: * | |
| + | |||
Disallow: /cgi-bin/ | Disallow: /cgi-bin/ | ||
| + | |||
Disallow: /tmp/ | Disallow: /tmp/ | ||
| + | |||
Disallow: /junk/ | Disallow: /junk/ | ||
Disallow: /~joe/junk.html | Disallow: /~joe/junk.html | ||
| + | |||
Disallow: /~joe/foo.html | Disallow: /~joe/foo.html | ||
| + | |||
Disallow: /~joe/bar.html | Disallow: /~joe/bar.html | ||
| − | |||
Versie van 17 mrt 2010 om 16:31
User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /junk/
Disallow: /~joe/junk.html
Disallow: /~joe/foo.html
Disallow: /~joe/bar.html
