Configure Databases and Host Distribution Rules
When SharePoint indexes content, it stores the addresses it has crawled and the status result along with additional tracking details in one of the available crawl databases in the farm. By default, SharePoint load-balances the distribution of these crawl entries between databases according to an internal analysis of the quantity of items that are being crawled under a given hostname. This database usage can be overridden via Host Distribution Rules.
Creating Additional Crawl Databases
The crawl database is used by SharePoint to track the content addresses, crawl schedules, and crawl history for content indexed by the search service. In large-scale indexing scenarios, the volume of content being crawled can result in a great many read and write operations to the crawl database. Contention issues may arise between crawlers accessing this data. To eliminate bottlenecks, you can create additional crawl databases to provide load balancing.
To add a new crawl database, follow these steps:
1. From the Search Administration page, under the Search Application Topology section, click the Modify button.
2. On the Manage Search Topology page, click New, and then click Crawl Database.
3. In the Add Crawl Database section, enter the name of the database server and the database name.
4. Click OK.
5. On the Manage Search Topology page, click Apply Topology Changes.
Creating a Host Distribution Rule
Normally, SharePoint 2010 identifies the most appropriate crawl database to use to store the crawl results for a given server that is being crawled. In some cases, the distribution that SharePoint creates may not be best for your needs. In these cases, you can manually associate specific server hostnames with specific crawl databases. To set up a host distribution rule, do the following:
1. From the Search Administration page, click Host Distribution Rules.
2. On the Host Distribution Rules page, click Add Distribution Rule.
3. In the Hostname field, enter the name of the content server for which the crawl results should be associated with a different database.
4. In the Select Crawl Database drop-down, select the database to use.
5. Click OK.
6. On the Host Distribution Rules page, click Apply Changes.
7. Click OK to commit the rule.