Robots.txt & Meta Noindex, Nofollow Tags

Robots Exclusion Standard

When primitive robots were first created, some of them would crash servers by requesting too many pages too quickly. A robots exclusion standard was crafted to allow you to tell any robot (or all of them) that you do not want some of your pages indexed or that you do not want your links followed.


Full training module available to subscribers.

Already have a premium account? Please log in using the Log In link at the top of the left column.