Crawler

A crawler is also referred to as a robot or bot from a search engine. A crawler reads the content from a page and usually stores this in a search engines database, called an index. From this index the search results are extracted when one has entered a keyword in a search engine.
There are also other crawlers, like the Wayback-machines crawler, that just stores the content(also images) from a website to create a record of the internet.  www.waybackmachine.org is usefull for seeing how websites have evolved through time.
Many other type of crawlers also scour the internet on a frequent basis.