Chapter 2
In This Chapter
All search engines try to make their results the most relevant. They want to make you happy, because when you get what you want, you’re more likely to use that search engine again. The more you use them, the more money they make. It’s a win/win situation. So when you do your search on classic car customization and find what you’re looking for right away instead of having to click through five different pages, you’ll probably come back and use the same search engine again.
In this chapter, you meet the major search engines and discover their similarities and differences, get familiar with the difference between organic and paid results, and gain a better understanding of how the search engines get their organic results. Plus, you find out about the search engines’ paid search programs and get help deciding whether metasearch engines have a place in your SEO campaign.
To keep their results relevant, all search engines need to understand the main subject of a website. You can help the search engines find your website by keeping in mind the three major factors they’re looking for:
You also have some control over two variables that search engines are looking at when they set the spiders on you. One is your site’s response time, which is how fast your server is and how long it takes to load a page. If you’re on a server that loads one page per second, the bots request pages at a very slow rate. A second seems fast to us, but it's an eternity for a bot that wants five to seven pages per second. If the server can't handle one page per second, imagine how long it would take the bots to go through 10,000 pages. In order not to crash the server, spiders request fewer pages; this puts a slow site at a disadvantage to sites with faster load times. Chances are bots will index sites on a fast server more frequently and thoroughly than sites on a slow server. Page speed has become very important to Google in particular and so deserves some attention. We discuss improving page and site speed in depth in Book VII, Chapter 1.
The second variable is somewhat contested. Some SEOs believe that your rank could be affected by something called bounce rate, which measures how often someone clicks on a page and immediately hits the Back button. The search engines can detect when a user clicks on a result and then clicks on another result in a short time. If a website constantly has people loading the first page for only a few seconds before hitting the Back button to return to the search results, it’s a good bet that the website is probably not very satisfying. Remember, engines strive for relevancy and user experience in their results, so they most likely consider bounce rate when they're determining rankings.
It’s time to meet the three major search engines: Google, Bing, and Yahoo. As we said earlier, they all measure relevancy a bit differently. Google might rank a page of content as more relevant than Bing does, so Google’s results pages could look quite different from Bing’s results pages for the same search query. Meanwhile, Yahoo uses Bing’s index of the web and ranking algorithm to serve organic and much of its paid search listings. For this reason, deciding which search engine is best is often subjective. It all depends on whether you find what you’re looking for.
One of the major ways search engines are differentiated is how they handle their organic versus paid results. Organic results are the web pages that the search engines find on their own using their spiders. Paid results (also called sponsored listings) are the listings that the site owners have paid for. In web searches, paid results usually appear as ads along the top or right side of the window, but they also can appear lower on the page, among or below organic listings. Paid results don’t necessarily match your search query either. Here’s how this happens.
Companies can bid on almost any keyword for which they want to get traffic (with some legal exceptions). The bid price needed to have an ad show in the SERP is based on many factors, including competition for the keyword, traffic on the keyword, and, at least in Google's case, the quality of the landing page. The better-constructed the landing page (the web page that a visitor receives after clicking an ad), the lower the minimum bid price. This doesn't have to be an exact match. Businesses often bid on keywords that are related to their products in hopes of catching more visitors. For example, if a visitor searches for tickets to Sports Team A, a sponsored (paid) link might show up advertising Sports Team B. This is what's happening below in Figure 2-1. Ticketmaster has bid on [Lakers tickets] as a keyword in order to advertise tickets for a different team, the LA Clippers, and clicking the sponsored link takes you to a page for buying Clippers tickets. The ad for Boston Celtics Tickets operates the same way. The organic links, however, should all take you to sites related to Lakers tickets.
Table 2-1 lists the major search engine players (in order of their appearance) and the attributes of each, for comparison. The following sections introduce you to each search engine in more detail and talk about its organic results and paid advertising services.
Table 2-1 U.S. Search Engine Comparison Table
Engine Name |
Organic |
Paid (Desktop) |
Paid (Mobile) |
Yahoo |
Yes. Uses Bing's index and algorithm |
Yes. Uses Bing Ads |
Yes. Yahoo Gemini |
Spider name: Googlebot |
Yes |
Yes. Google AdWords |
Yes. Google AdWords |
Bing Spider name: Bingbot |
Yes |
Yes. Bing Ads |
Yes. Bing Ads |
In 1994, two electrical engineering graduate students at Stanford University, David Filo and Jerry Yang, created Yahoo, a search engine and network of properties that has undergone much change in its 20-year history. For many years, Yahoo has outsourced its search function to other providers like Google and, today, Bing.
In 2010, Yahoo made a deal with Bing to power its search engine and pay per click program. That means that the organic search results in Yahoo use Bing’s ranking algorithm and Bing’s index of the web.
Because Yahoo no longer provides its own paid results for desktop searches, in order to advertise on Yahoo's network, you must use Microsoft Bing Ads (and get two-for-one exposure).
Mobile advertising is a different story. In 2014, Yahoo migrated to Yahoo Gemini, its own mobile ad marketplace. The strategy appears to be paying off; its third-quarter earnings place Yahoo in third place for total mobile ad revenue, behind Google and Facebook but, surprisingly, ahead of Twitter.
Google began as a research project by two other Stanford University students, Larry Page and Sergey Brin, in January 1996. They officially incorporated as Google in September 1998.
Over time, Google has developed into the powerhouse of the search engine realm. Here are just some of the reasons why Google is the king of search engines and shows no signs of giving up the crown:
http://www.comscore.com/Insights/Press_Releases/2014/3/comScore_Releases_February_2014_U.S._Search_Engine_
).Table 2-2 comScore Explicit Core Search Share Report, U.S. (August versus July 2014)*
Core Search Entity |
July 2014 |
August 2014 |
Share Change |
Total Explicit Core Search |
100.0% |
100.0% |
None |
Google Sites |
67.4% |
67.3% |
–0.1 |
Microsoft Sites |
19.3% |
19.4% |
0.1 |
Yahoo Sites |
10.0% |
10.0% |
0.0 |
Ask Network |
2.0% |
2.0% |
0.0 |
AOL, Inc. |
1.3% |
1.3% |
0.0 |
*“Explicit Core Search” excludes contextually driven searches that do not reflect specific user intent to interact with the search results.
Google has a service called Google AdWords that regulates its paid results for desktop and mobile. It’s a pay per click advertising model that lets you create your own ads, choose the keyword phrases you want your ad to appear for, and set your bid price and budget. Google ranks its ads based on the ad’s bid amount and Quality Score — a combined measure of the ad’s relevance, landing-page experience, and expected click-through rates, or how often the ad is clicked. Google AdWords can also help you create your ads if you’re stuck on how to do so.
After your AdWords ads are set up, Google matches your ads to the right audience within its network, and you pay only when your ad is clicked. Google provides gives an ever-increasing number of ways to target your ad audience, such as by demographics like gender, age group, annual household income, ethnicity, and number of children in the household. AdWords also allows you to do location-based targeting and day-parting, which limits the display of your advertisement to certain times of the day. Recently, Google introduced “In-market audiences” targeting, which lets you leverage Google’s understanding of consumer behavior patterns and find people who are actively researching products like yours.
You can potentially get a lot of exposure for your paid ads. The Google AdWords distribution network lets you advertise on Google search sites and affiliates like AOL and Ask.com. Mobile and tablet ads are also centrally managed through AdWords.
Bing (previously named MSN Search and Microsoft Live Search) is a search engine designed by Microsoft that competes with Google and Yahoo. It’s currently the second-most-used general search engine in the United States.
In addition to providing rich, blended SERP results on par with Google’s, Bing differentiates itself through features like its daily full-screen home page image, a longer list of search results per page, and a rewards program that lets people earn points that can be redeemed for gift cards just for using Bing. Bing Search also lets users easily modify the search results based upon any location they would like the search to appear from.
Microsoft’s paid program is called Bing Ads, and reports are that it offers advertisers an extremely good return on investment (ROI). Like Google, Bing ranks its ads based on keyword bid price and ad quality. Microsoft also lets you place adjustable bids based on demographic details. For example, a mortgage lead from an adult with a higher income might be worth more than an equivalent search by someone who is young and still in college.
The five biggest search engines worldwide right now are Google, Yahoo, Bing, Baidu (a Chinese search engine), and Yandex (a Russian search engine — see Book IX, Chapter 2 for more information on Baidu and Yandex), with Google taking home the lion’s share. But other smaller engines that draw a pretty respectable number of hits are still operating.
AOL has been around in some form or another since 1983. Today AOL provides some services such as email, chat, and its own search engine. AOL gets all its search engine results from Google, both organic and paid.
Ask.com (some of you may remember it as Ask Jeeves) pioneered blended search (the integration of different content types, such as images, videos, news, blogs, books, maps, and so on, onto the search results page) but failed to gain any significant market share from the larger three engines (Google, Bing, and Yahoo). Ask.com changed its market strategy and now considers itself an “answer engine,” rather than a search engine. Ask.com generates comprehensive results answering question-focused search queries, pulling information from third-party directories, feeds, Q&A forums, and Google’s organic results.
Ask.com gets most of its paid search ads from Google AdWords. Ask.com does have its own internal ad service, but it places internal ads above the Google AdWords ads only if it feels the internal ads will bring in more revenue.
We’ve been talking mostly about general search engines, whose specific purpose is to scour everyone and everything on the web and return results to you. But there’s also another type of search engine known as a vertical search engine. Vertical search engines are search engines that restrict their search either by industry, geographic area, or file type. Google has several vertical search engines listed in the upper-left corner of its home page for images, apps, and so forth. So when you type [jam] into Google’s Images search, it only returns pictures of jam instead of web pages. The three main types of vertical search engines are detailed in the following sections.
Industry-specific vertical search engines serve particular types of businesses. The real estate industry has its own search engines like Realtor.com (www.realtor.com) and Zillow.com (www.zillow.com), which provide housing listings, and companion sites like HomeAdvisor.com
, which is for home improvement contractors. If you want to conduct searches related to the medical industry, you can use WebMD.com (www.webmd.com
), a search engine devoted entirely to medical questions and services. Questions about Hollywood stars and anyone ever connected with movie-making can be answered at IMDB.com
, which stands for Internet Movie Database. If you are searching for legal services, FindLaw.com (www.findlaw.com
) and Lawyers.com (www.lawyers.com
) can help you search for an attorney by location and practice.
A local search engine is an engine specializing in websites that are tied to a limited physical area also known as a geotargeted area. Basically, this type of engine is looking for things in your general neck of the woods. In addition to their main indexes, the major search engines have local-only indexes that they integrate into their main results, like Google My Business and Bing Places. Businesses submit a local listing to the search engines, which includes information about the local business such as its name, address, city, phone number, categories the business may be searched under, and so on. That means the site could pop up if someone’s looking for that type of business within its geographic area.
A searcher might specify a ZIP code, city, or other geographic qualifier in a search query to find geotargeted results, such as [Milwaukee chiropractor]. However, Google and Bing can pretty reliably understand the intent of a search. If the engine thinks the searcher wants to find a nearby business, it displays local listings right in the search results. For example, a search for [dry cleaners] brings up website links to businesses near the searcher, even though no location was specified in the query.
Local results take up a lot of SERP space and often rank above the organic results. To be clear, these local results are still served up “organically” by the local search algorithm, but the local results may take precedence over what we have come to know in the past as the top organic results. Searches performed on a smartphone or other mobile device tend to have even more local results, since the majority of queries people speak or type into mobile devices are thought to have local intent. See Chapter 4 of this minibook for examples of several types of local results.
Certain search engines devote themselves to just one type of file — videos, photographs, or other. Rather than limiting traffic, being a file type-specific engine creates traffic because visitors know exactly what they’re going to get. The video-only engine YouTube.com
, for example, reports some astonishing statistics: More than 1 billion people visit YouTube each month, watching more than 6 billion hours of video and uploading an average of 100 hours of video per minute. In fact, YouTube’s traffic makes it the second most popular website in the world, behind only Google (http://www.youtube.com/yt/press/statistics.html). There are other video engines as well, including Blinkx.com, Metacafe.com, and Hulu.com; Hulu specializes in the video niche of television shows and movies.
Image searches performed on Google, Bing, or Yahoo return images from all over the web. But there are also many successful image-only search engines, like PicSearch.com and CC (Creative Commons) Search (search.creativecommons.org), which comb the Internet looking for images that match your criteria. Sites like Flickr.com are image-sharing sites, where users can upload their photos and specify how they may be used. Other sites have their own stock of images that users can search for and download (either for free or for a fee based on the user’s intended use), such as GettyImages.com
, iStockphoto.com
, Fotolia.com
, and Dreamstime.com
, just to name a few.
Say you’re writing an article and you need to reference something in the New York Times. The New York Times website (www.nytimes.com
) keeps an archive of online articles, but because you can’t remember the date the article was published, you’d have a long trek through the online archives. Luckily, it has its own internal site search engine that enables you to look up articles using keywords and filters. Any search engine that’s site-specific, meaning it searches just that website, is an internal site search engine.
Larger websites with thousands of pages employ these internal site search engines as an easy way of browsing their archives. A very small site probably doesn't need an internal search, but most e-commerce sites with more than a few products should consider implementing one.
Techniques that help you rank in general search engines also help your users when they need to find something on your site using an internal search. A good internal search can be the difference between making a sale and visitors leaving in frustration. To get started quickly, Google offers a hosted internal search solution as well as an enterprise-level solution. See https://www.google.com/enterprise/search/products/gss.html
for more information.
Another breed of search engine to be aware of is a metasearch engine. Metasearch engines do not maintain a database of their own, but instead combine results from multiple search engines. The advantage they tout is a twist on “bigger is better” — the more results you see in one fell swoop, the better. The sites Dogpile.com (www.dogpile.com
) and DuckDuckGo.com (www.duckduckgo.com
) top the list of metasearch engines. When you run a search on Dogpile, it pulls and displays results from four of the largest global engines (including Google and Yahoo) in one place. DuckDuckGo’s marketing proposition is that it respects searchers’ privacy by not tracking their search behavior, but pulls from many different search engines and key sites to present non-personalized search results.
After pulling results from multiple search engines, the metasearch engines retain the top ranked results from the separate search engines and present the user with the top results. This is different from applying an algorithm, as the indexed search engines do (an algorithm is a mathematical equation that weighs many specific criteria about each web page to generate its ranking result, as we discuss in Chapter 1 of this minibook). Metasearch engines take more of a filtering approach to all the indexed data gathered from the other search engines. Ads appear at the top and bottom, with organic “Web Results” in the middle of the page. We’ve found metasearch engines helpful for keeping track of which competitors buy paid results for which keywords. (You can read more about paid results in Book I, Chapter 4). Figure 2-2 shows you a results page from the Dogpile metasearch engine.
18.191.165.62