Robots Exclusion Protocol
The Official GoogleBlog published a post regarding the Robots Exclusion Protocol (REP). REP allows you even more flexibility now. take a look at these highlights:
- CONTENT=”unavailable_after:” - a great way of letting the Search engines know when your content expires / is not relevant anymore
- X-Robots-Tag: noarchive, nosnippet - do you have pages that you don’t want to be archived? not shown in the sippets?
- CONTENT=”NOFOLLOW” - don’t let the Search engines follow the links
Read these three posts if you want to know more:
- Controlling how search engines access and index your website
- The Robots Exclusion Protocol
- Robots Exclusion Protocol: now with even more flexibility
admin :: Jul.30.2007 :: SEO, google, robots.txt :: No Comments »
