Commercial search engines make use of information retrieval (IR) in order to determine the popularity and relevance of a website. This is because even early in the development of the search engines, IR researchers realized that these are the two critical components for search functionality. So what is popularity and relevance exactly?

Popularity ? this is the relative importance usually measured through citations of the document that matches a searcher’s query. The popularity of the particular document will increase when other documents over the internet use it as reference.

Relevance ? this is the degree by which the content matches with the user’s terms and query intentions. The relevance of the document will increase when the phrase used by the user occurs many times in the document including its title, the headlines, and the sub-headlines.

Up to this date, these two components are used in web search and it manifests itself in different ways today; usually in the form of link analysis and document analysis.

In link analysis, the search engine will measure not only what website is linking to a particular page but also what these sites are saying about the site. Search engines also have a good grasp of which sites are affiliated with each other through the site’s registration records, historical link data, and various other sources. Generally, links from “trusted” sites such as .gov and .edu pages are more valuable compared to any other domain extensions.

The other aspect is document analysis. Almost everybody is familiar with this; search engine looks at the important areas of the document such as its title, heading tags, meta data, and the body of the document. They also try to measure the content’s quality through complex algorithms beyond the scope of this article.