Most users don’t know the difference between a search engine and a directory. After all, both directories and search engines contain lots of web page listings, and both are searchable. What’s the diff?

The difference between a directory and a search engine is in how the listings are compiled. As you learned in Chapter 1, “Inside Google,” Google’s search engine works by sending out automated GoogleBot software to crawl the Web, and then it uses a proprietary formula to match the pages to users’ search queries. This process guarantees a huge number of results for most queries, but it’s all very sterile and automated. There’s no good way to truly judge the content or quality of a page; it’s all about numbers.

In contrast, a directory is assembled by a team of human editors. That’s right—human beings, not machines. They find and evaluate pages on the Web, annotate the page listings, and organize them into relevant categories. Unlike computers, human beings can make qualitative judgments about a page’s content and can evaluate the page’s actual meaning. It’s not about numbers; it’s about content.

All of this means that a directory is likely to have higher-quality results than a search engine. It’s also likely to have fewer results (far fewer, when compared to Google’s gargantuan search index) because of the need to closely examine each individual page before it’s added to the directory. Where the Google search index includes billions of listings, the Google Directory contains approximately 4.6 million listings—less than one-tenth of 1% of what’s in the search index.

Google’s search engine is like casting a wide net and taking everything that’s caught. Google Directory is like dropping a single fishing line in the water with the intent of catching a particular type of fish. You get lots of fish with the wide-net approach, but you get the fish you want by using a rod and reel.

The other big difference between a directory and a search engine is organization. A search index has none; those billions of pages are dumped into one big database, with no sense of order. A directory, on the other hand, is all about order; the human editors not only pick the web pages to be included, they also organize the sites into logical categories. So although you can’t browse a search index, you can browse a directory, simply by clicking through the hierarchy of categories and subcategories.

Because they look at every page included in the directory, the directory’s human editors also have the opportunity to annotate those pages. Browse through a directory’s category listings and you’re likely to see summaries, reviews, and comments about the web pages listed. These annotations are not automatically generated from the page’s content; they’re added by the editors, in what amounts to a very human touch.

A directory’s human editors also provide one other important function—they continually check for and remove dead links. This is something that search engines don’t always do well; a human being is more diligent about keeping the listings updated.

So what are the final differences between a search engine and a directory? Table 8.1 summarizes them.

Table 8.1 Search Engines Versus Directories

The Google Directory works like most other web directories, such as the Yahoo! Directory (dir.yahoo.com) or Best of the Web (www.botw.org). Thousands of human editors sort through sites submitted by users, as well as do their own web browsing, to find the sites included in the directory. As soon as a web page has been accepted for inclusion, the editors write a brief review/overview of the page and assign it to a topic category. It’s a totally manual process; there are no bots crawling the Web or linguistic programs excerpting page contents. All the work is done by hand.

But here’s the thing. Google doesn’t assemble its own directory. No, the Google Directory is actually a customized version of a third-party directory called the Open Directory Project. (Figure 8.2 shows the Open Directory home page—look familiar?)

Figure 8.2. The Open Directory—the basis of the Google Directory.

Google takes the Open Directory listings and grafts the Google interface and search engine to them. So, although the listings in the directory are assembled by Open Directory editors, they’re ranked using Google’s PageRank technology. If you compare a category in the Google Directory with the same category in the Open Directory, the listings will be the same, but arranged differently.

The good thing about Google using the Open Directory is that it’s perhaps the largest and highest-quality web directory available. The Open Directory Project is a huge undertaking, hosted and administered by Netscape (part of America Online), with more than 80,000 volunteer editors submitting reviews and rankings of websites and pages. Google made a good choice in partnering with the Open Directory. Combining Open Directory listings with Google’s interface and PageRank rankings makes the Google Directory the easiest-to-use and most useful directory on the Web.

It’s the use of Google’s PageRank technology that makes the Google Directory so easy to use. Instead of just browsing through the category listings (which you can do if you want), Google’s search technology lets you search the directory listings the same way you search Google’s search index. The listings in the directory—and thus the results of your query—are ranked according to relevance, thanks to the use of PageRank. The most relevant sites always appear near the top of the listings, which they don’t necessarily do if you access the raw Open Directory listings.

Now that you know the difference between Google’s standard search engine and the Google Directory, which should you use for your searching?

Here are some tips:

The bottom line is that if you want a lot of results, and you don’t mind wading through the chaff to find a little wheat, use the standard Google search engine. But if you’re tired of search results that aren’t quite what you’re looking for, and you want more qualified results, consider using the Google Directory. In other words, if you want quantity, use the standard Google search engine. If you want quality—or a good category overview—use the Google Directory instead.

Most users opt to use the Google Directory much as they do the regular Google search page—that is, by searching the directory, rather than browsing it. To search the Google Directory, follow these steps:

Google displays a search results page like the one shown in Figure 8.3. This page looks a lot like a standard web search results page. Each result listing also features a link to the category in which it is included; click a category link to view all the pages listed in that category.

Figure 8.3. A Google Directory search results page.

If you’re accustomed to using Google to search for information, the concept of browsing might be new to you. It’s really quite simple; it all hinges on the concept of hierarchical organization of information into topic categories and subcategories.

You start at the Google Directory home page, where 16 different categories (and a handful of subcategories) are listed. Here are the major categories:

You start your browsing by clicking a category that matches your interest. This displays a major category page that lists all the subcategories within the major category. (Sometimes a few related categories are listed also.) For example, The Home category lists 25 subcategories, such as Apartment Living, Consumer Information, Cooking, and the like.

Click a subcategory link and you see a subcategory page, like the one shown in Figure 8.4. Some subcategories include even more subcategories (sub-subcategories?), which are listed at the top of the page. You’ll also find a list of related categories; then you see the list of pages within the subcategory.

Figure 8.4. A typical subcategory page, including additional subcategories and pages listed in the subcategory.

These pages are ranked (using Google’s PageRank technology) in order of relevance. The small green bar to the left of each page listing visually indicates the relevance; the longer the green bar, the more relevant the result. Each listing also includes the page’s title (click to jump to the page), the page’s URL, and the editor’s description of the page.

When you stumble across a big category, one with lots of pages listed, it may be difficult to find exactly the page you want. To that end, Google lets you search for pages within a category. All you have to do is navigate to a category or subcategory page, enter your query in the search box at the top of the page, check the Search Only in Category option, and then click the Google Search button. Google searches the current category—and only the current category—for the keywords you entered and displays the results on a separate search results page.

Searching within a category can be particularly useful in restricting your search to a particular topic. For example, if you search the entire Google Directory for lions, Google might return pages about lions (the animal), Lions (the football team), Lions (the public service organization), or any number of other lion-related subjects. But if you first navigate to the Sports, Football, American, NFL category and then search for lions, you’ll see only results related to the Detroit Lions football team.

As you can see in Figure 8.5, the main Google Knol page is your gateway to the various articles available. You can browse through the featured articles on the home page, or search for articles using the top-of-page search box.

Figure 8.5. Searching for authoritative articles on Google Knol.

An article, or what Google calls a knol, is just what you’d expect it to be. Figure 8.6 shows a typical article, complete with a clickable table of contents at the top of the page. A user rating for this article appears at the top; author information and user reviews of the article appear along the right side.

Figure 8.6. Viewing a Google Knol article.

To rate an article, just use your mouse to click the appropriate number of stars in the Your Rating section at the top of the page. (It’s a one-to-five scale, five being tops.) To provide your two cents’ worth about the article’s contents, click the Review This Knol link and start typing.

In its own way, Google Knol is kind of a contributor to Wikipedia. Like Wikipedia, anyone can write an article; all you need is the desire to do so and a little knowledge of the topic at hand.

To create an article, click the Write a Knol button on the main Google Knol page. Assuming that you’ve signed in with your Google Account, you now see the page shown in Figure 8.7. From here you start filling in the blanks:

Figure 8.7. Writing a Google Knol article.

That’s it. Google doesn’t edit your article, nor is it checked for accuracy—officially, that is. Also like Wikipedia, Google relies on the user community to rate, comment on, and even collaborate on the submitted articles. That’s right, other users can edit your articles, just as you can edit theirs (assuming their articles are configured for open or moderated collaboration, that is). The collaborative nature of Google Knol is supposed to encourage greater accuracy in information, as well as more comprehensive articles.

So what do you do if you find an inaccuracy in an article—or simply want to elaborate on what was originally written? Unless the article was configured for Closed Collaboration, you should see a series of tabs at the top of the article. Click the Edit tab and the article is now editable. That is, you can edit the text of the article as you see fit, in your web browser. Click the Save button to save your edits.

If the article is configured for Moderated Collaboration, your edits have to be approved by the article’s writer. If the article is configured for Open Collaboration, your edits immediately appear in the version of the article that the general public sees.