Noindex,Nofollow not Working

Whenever you want a page to NOT be included in a search engines index you use the following code in your page header, don't you.

But haven't you noticed this doens't always work?

Google and MSN usually understand what you mean and never include the page or even drop it from their index. Only Yahoo and several other search engines don’t remove the page from their index. They just stop updating it.

It can be rather disturbing if you just want the page to be removed from the search index and you wait, and wait and nothing happens. There is however a cure. Whenever you want a page to be removed or never indexed it’s best to restrict access to that page in the robots.txt file.

The robots.txt file is accepted by all official spiders and should be placed in the root of your folder website. In this case the robots text file should contain:

Read more about using a robots.txt file.