Search engine

According to Wikipedia, “a search engine is a program that help users to find information stored on a computer system. There are several types of search engines, which are designed to retrieve documents stored on the World Wide Web, inside a corporate or proprietary network, or in a personal computer.” ([1], accessed 17:33, 26 January 2007 (MET)) Anyway, the expression search engine usually refers to a Web search engine, which retrieves information from the public Web.

When users need some pieces of information, they just ask for web pages containing a given keyword and the search engine provides a list of items where the keyword can be found. This list is often sorted on the basis of relevance of the results.

A short history of search engines

In 1990 the very first search engine, called [2], was created. It indexed files located on public FTP (File Transfer Protocol) sites so that users could find specific files inside a searchable database. It could not search by file contents.

In 1994 the [3] search engine was created. It was the first full text search engine, i.e. users can search for any word in any web page. This feature became the standard for all major search engines since and it was the first search engine to be widely known.

In 1994 Lycos came out; soon after [4] and [5] appeared. At the moment the most popular search engines are [6], Search, and Search by Microsoft.

How search engines work

Web search engines use 'robot' applications that automatically search the Internet, visit web pages, and store data about their content which is then compiled into a huge index. Every search engine uses a proprietary algorithm to create its indixes such that only meaningful results are returned for each query. In order to find relevant web pages search engines give each document a rank, i.e. a relevance score. Relevance scores reflect the number of times a search term appears, if it appears in the title, if it appears at the beginning of the document, and if all the search terms are near each other. However, usually a short summery is more useful than a ranking.

Looking for more words narrows down, while looking for less words widens up the search. Many search engines use tools such as AND, OR and NOT to help. Some search engines have sophisticated ways of narrowing down a search to increase the chances of the user finding what they want: they can focus on a particular type of website, group their index into subjects and topics, or offer users suggestions after their initial search.

Here are the required steps to get information from a search engine: 1. users make a query typing keywords into a search engine; 2. the search engine software quickly sorts through millions of pages in its database to find matches to this query; 3. the results are ranked in order of relevancy.

How to search databases

Boolean Searching Users can type AND or NOT or just put + (plus) or – (minus) signs after search terms in order to narrow the search and find exactly what they are looking for.

Search by Field Users can specify where the search terms should be on the page to get high priority or be included in the results. This option is usually available in the 'advanced search' page of the site.

Exact Quotation Searches Users can type the expression they are looking for between double quotation marks so that they are sure the terms are in the exact right order and are together. Quotation marks allow you also to search for exact phrases.

Filters Filters can be used to narrow searches. The most common types of filters get rid of 'adult' sites.

Mind Map A search engine, [7], shows its search results as a visual mind map of connections between different sites.

Natural Language Searching When looking for specific information users can type a query in the form of an English question.

Google Search Users type the terms they are looking for, click enter and the engine will return all websites that have the search terms in them. Retrieved pages will be shown in a list based on how popular the page is on the web.

How to improve your search results

Do you always find what you were looking for when using a search engine? Many search engines find a huge amount of matches, but often one does not get what one wanted to find. On the Webreference webpage you can find many tricks that can be useful to improve the quality (and the result, of course!) of your search. The first thing you can do to improve your search results is: think about what you are looking for and be as much precise as possible about your search terms. It is more useful to put only few words-or even better keywords- and to make “title only” searches, than looking for many search words at the same time. The broadest your search, the higher the possibility of coming up with unwanted results. This tip alone can help you save time and find more practical results. Another thing you can do is: refine your searches. Even though this can really make a great difference in the amount (and quality) of pages returned, a BBC article argued that only 10% of people refine the results. Use the “advanced search” option that many search engines offer you. In fact, just specifying the language you want your results to be, you will avoid getting many useless matches. If you still don’t find what you were looking for following these tips, try being more technical. Boolean terms are a bit harder to use, but they can help you to narrow your search results. To get Boolean logic, just use the words AND, NOT, and OR. This type of search also gives one the possibility of eliminating unwanted matches by using the AND NOT operator. Moreover, keep in mind that search engines are very different, and that they offer one different services. Search engines have been tested to help one find the right search engine according to what one is looking for. Knowing which is the right one according to your tasks is a further way to help you achieve better matches. A table that clearly illustrates which search engine you can use when can be found on the Webreference page. However, an article that appeared on the BBC webpage argues that it is much better “to become as familiar as possible with one search engine and stick to it”. This comes from the fact that a study that was carried out found that “of 600 queries 60% of results returned for a particular set of terms will be the same across all search sites”.

Most used seach engines

According to Darren Waters, technology editor of the BBC News website, Google is “the world's most popular search engine.” It has been argued that its dominance of the search market, which is said to stand in the UK at 75% (followed by MSN and AskJeeves, both on 8.4%, and Yahoo with 8%), is due to the fact that Google has become an habit. In fact, Mr Elliott said that "it isn't much trouble to go to another but people increasingly have Google on their browser window and even for those that type it in each time it has become a habitual thing." It has never been proved that Google has better results than other search engines. On the contrary, Webreference, for example, tested many search engines following these parameters: “One Item Among Many Related Pages”, “Obscure Item”, “Selectivity: Apple trees NOT computers”. On the basis of these tests, it appeared that “ Lycos is the official heavy weight search engine champion of the universe”, even though, according to this webpage, it is better to “choose different engines for different tasks.” Mr Elliot supports this idea, arguing that “people are unreasonably attached to Google and the issue is that people are not experimenting with other products.” Nowadays other search engines have been trying to break the so-called “Google habit”, like Ask, WebFetch and its sister search engine in the US DogPile. Even though their position in the search market is still very limited, Jane Wakefield argued that “the movement to persuade users away from the dominant search engines such as Google and Yahoo may be small but it is gathering momentum from those with more solid radical credentials.”

The future of search

The future of search seems to look bright: many search engines have in mind to develop ambitious projects to make their service better and better. Let’s see some. According to Mr Merrill, the company Google is planning to expand activity very much in the future. Its projects include better search for mobiles, personalised search, language translation (something has already been done on mechanised translation), accessing offline information and defeating web spam, instant messaging and online mapping. Moreover, according to Spencer Kelly, Google “is also hoping to build on its online music radio service, Launchcast, which includes a personalised music selector that learns the types of music you like and streams you different tracks accordingly.” He adds that, as regards videos, Google “is planning a service that allows you to search transcripts and plotlines for videos stored on the web, and watch them.” According to what he says, as regards Yahoo, if you “create a Yahoo user ID and you not only get free e-mail, but you also get a chance to create your own homepage, to which you can add your own elements: calendar, e-mail inbox, news stories, weather, change the colour scheme, etc.” This way, your web interface is going to be really like you want it to be. The tendency seems to be that in the future, search engines will be even more personal. We are not only talking about personalised homepages, but even your search options will be personalised. According to Jane Wakefield, Jimmy Wales, founder of the online encyclopedia Wikipedia wants to offer an alternative "people-powered" search engine. She said that “his plan is to Wiki-fy the process of internet search, so that human beings decide openly how to rank and organise information, not the huge private servers of Google and Yahoo.” Jimmy Wales called this ambitious project "Search Wikia", saying that it will be "the search engine that changes everything".

References

• "A Guide to Using Search Engines." BBC - h2g2. 20 May 2003. 21 Apr. 2007 <http://www.bbc.co.uk/dna/h2g2/A1047917>

• Day, Peter. "Google searches for the future." BBC News. 15 Nov 2005. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/business/4436764.stm>

• Grossan, Bruce. Search Engines: What they Are, How They Work, and Practical Suggestions for Getting the Most Out of Them. webreference.com. 21 Feb. 1997. 21 Apr. 2007. <http://www.webreference.com/content/search/index.html>

• Kelly, Spencer. "Search engine race gets personal." BBC News. 12 Aug. 2005. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/programmes/click_online/4144646.stm>

• "Search engine." Wikipedia. 18 Apr. 2007. 21 Apr. 2007. <http://en.wikipedia.org/wiki/Search_engine>

• "Search engine secrets revealed." BBC News. 18 May. 2003. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/technology/3033385.stm>

• "Search users 'stop at page three'." BBC News. 12 Apr. 2006. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/technology/4900742.stm>

• "Search wars - which is the best?" BBC News. 12 Nov. 2004. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/uk_news/magazine/4005901.stm>

• Wakefield, Jane. "The future of search looks bright." BBC News. 3 Jun. 2005. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/business/4436764.stm>

• ---. "Trying to break the Google habit." BBC News. 12 Apr. 2007. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/technology/6546501.stm>

• Ward. Mark. "Search sites get closer to users." BBC News. 7 Sep. 2004. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/technology/4074019.stm>

• Waters, Darren. "Google to stay focused on search." BBC News. 3 Jul. 2006. 21 Apr. 2007. <http://news.bbc.co.uk/go/pr/fr/-/2/hi/technology/5140066.stm>

• "What is a search engine?" Webopedia Computer Dictionary. 21 Apr. 2007. <http://www.webopedia.com/TERM/s/search_engine.html>