Yahoo, Google, and other major search engine companies have estimated there are between 8 and 20 billion documents on the web. It isn't so. The truth is that the web is home to more than 500 billion web based documents that have thus far been unreachable by today's crop of search engines. There is a vast amount of data and documents stored in databases that can't be accessed by the current versions of web crawlers.
General purpose search engines have not been designed to search for information located in every single database, data caches, or data created on the fly. As a result, in order to find more obscure information, you have to know exactly where to look for it.
As an example, suppose you were trying to find information about the age at which a Great Dane puppy develops its adult second and third premolars. Using the popular search engines to search for puppy teeth will give you information on biting, teething, jaw alignment, Great Dane breeders, and human tooth development web sites. It would be necessary to go to a veterinary book on breed specific dentistry to find the information you are searching for. Although the information exists, it is not accessible through a search of the general Internet.
There is a lot of upside in this space. We are at the tip of the iceberg...
