Chapter 5. Designing Highly Available Microsoft Windows Servers

Chapter 4, covered one of the main pieces of the foundation: your disk subsystem. This chapter helps to complete that foundation by providing information about configuring the true software base for Microsoft SQL Server: the operating system. This chapter covers both Microsoft Windows 2000 Server and Microsoft Windows Server 2003.

General Windows Configuration for SQL Servers

Before delving into the specific high availability options for the base operating system, there are some basic ideas you need to think about before you install the bits.

Choosing a Version of Windows

Choosing an operating system for SQL Server is no longer as clear-cut as it used to be. The operating system you choose affects the performance you get as your requirements grow. The need to make systems last longer these days makes your operating system choice that much more important.

Your operating system choice should match your SQL Server needs for processor and memory. Table 5-1 outlines the memory and processor support for editions of Windows 2000 Server and Windows Server 2003.

Table 5-1. Memory and Processors Supported per Operating System

Operating System

Memory Configuration

Processors

Windows 2000 Server (32-bit)

4 GB; 128-MB minimum; 256-MB minimum recommended

Up to 4; 133 MHz or higher, Pentium-compatible

Windows 2000 Advanced Server (32-bit)

8 GB; 128-MB minimum; 256-MB minimum recommended

Up to 8; 133 MHz or higher, Pentium-compatible

Windows 2000 Data-center Server

32 GB; 256-MB minimum

Up to 32; Pentium III Xeon or higher required

Windows Server 2003 Standard Edition (32-bit)

4 GB; 128-MB minimum; 256-MB minimum recommended

Up to 4; 133 MHz or higher, Pentium-compatible; 550 MHz or higher recommended

Windows Server 2003 Enterprise Edition (32-bit)

32 GB; 128-MB minimum; 256-MB minimum recommended

Up to 8; 133 MHz or higher, Pentium-compatible; 733 MHz or higher recommended

Windows Server 2003 Enterprise Edition (64-bit)

64 GB; 128-MB minimum; 256-MB minimum recommended

Up to 8; 733 MHz or higher, supported 64-bit processors; 733 MHz or higher recommended

Windows Server 2003 Datacenter Edition (32-bit)

64 GB; 512-MB minimum; 1-GB minimum recommended

Up to 64 (minimum of 8); 400 MHz or higher, Pentium-compatible; 733 MHz or higher recommended

Windows Server 2003 Datacenter Edition (64-bit)

512 GB; 512-MB minimum; 1-GB minimum recommended

Up to 64 (minimum of 8); 733 MHz or higher, supported 64-bit processors; 733 MHz or higher recommended

Note

Windows Server 2003 Web Edition is not recommended for use with SQL Server 2000.

Note

Windows Datacenter licenses allow you to run as many instances of Datacenter as you have physical processors through hardware or virtual partitioning. If you think you will ever need to use more than eight processors, a Datacenter version is your only choice, and the price premium is not that great when compared to the overall cost of an eight-processor system. Make sure you work with your preferred original equipment manufacturer to purchase expandable hardware or modular systems to prevent inadvertently being unable to expand the system at some later time.

32-Bit Versus 64-Bit

As you have already seen, Windows Server 2003 (in both Enterprise and Datacenter editions) can be found in both 32- and 64-bit versions. Cosmetically, all versions of Windows Server 2003 look and feel the same, but they are not the same inside because of the different architecture of the 64-bit chips.

What does this mean to you, the SQL Server consumer? There are two main areas that immediately come to mind: processor and memory. From a processor standpoint, even though a 64-bit chip might have double the amount of address bits and might be faster in terms of clock speed than the current Pentium-class machines, that does not guarantee you double the performance. A 64-bit machine should perform better than its 32-bit counterpart, as long as the application was coded to work with the 64-bit version of Windows Server 2003 and the 64-bit version of Microsoft SQL Server 2000. As with most applications, you should test against the 64-bit version of SQL Server 2000 to see what performance gains might be garnered from implementing this chip.

Important

The 64-bit operating system requires a 64-bit version of SQL Server—you cannot install the 32-bit version of SQL Server on the 64-bit editions of Windows Server 2003.

From a SQL Server perspective, the main benefit of going to 64-bit is how memory is handled: all memory is dynamic under 64-bit, and you can address more than 4 GB of memory without setting any special options in your Boot.ini file. Under 32-bit operating systems, this is not the case. Under 32-bit operating systems, any addressable memory above 4 GB can be accessed only after configuration switches are placed in a system file, and then the application must be coded to recognize that additional amount of memory (which is why you turn on Address Windowing Extensions [AWE] support within SQL Server).

More Info

Chapter 14, covers SQL Server 2000 and memory usage in greater detail in the section "Memory Management for SQL Server 2000," as it is an aspect of SQL Server administration and configuration that affects all installations.

For SQL Server 2000, when you install the 64-bit edition, you do not get all features and utilities that come with the 32-bit version. For a complete list, see the topic "Differences Between 64-bit and 32-bit Releases (64-bit)" in the 64-bit SQL Server Books Online.

For information on options, especially tools, that might not be available to you under 64-bit operating systems, see the topic "Features unavailable on 64-bit versions of the Windows Server 2003 family" in TechNet at http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/windowsserver2003/proddocs/entserver/unsupported64BitFeatures.asp or in your Windows Server 2003 documentation. If anything on this list is crucial for your environment to run on the same server as SQL Server 2000, you might not be able to use the 64-bit version of Windows Server 2003. For example, at launch time, the .NET Framework is not yet available under 64-bit.

Also, consult the Windows Catalog at http://www.microsoft.com/windows/catalog/server2 for information on 64-bit applications.

Versions of SQL Server and Windows Server 2003

For those considering upgrading or doing a new installation of a 32-bit edition of Windows Server 2003, you can only use SQL Server 2000 RTM or Release A with SQL Server 2000 Service Pack 3 or later applied immediately after installing the base SQL Server 2000 software.

Important

If you are upgrading your operating system, you must install SQL Server 2000 Service Pack 3 or later on all of the instances that will run in your server cluster or on the stand-alone server prior to upgrading the operating system.

Windows Server 2003 does not support any other versions of SQL Server—including Microsoft SQL Server 6.5, Microsoft SQL Server 7.0, and SQL Server 2000 with SQL Server 2000 Service Pack 2 or earlier. You will see the warning shown in Figure 5-1 pop up prior to the installation of SQL Server 2000. During an upgrade, you will see a message similar to that shown in Figure 5-2.

SQL Server 2000 warning prior to installation.

Figure 5-1. SQL Server 2000 warning prior to installation.

Warning during the upgrade process.

Figure 5-2. Warning during the upgrade process.

Disk Requirements for Windows

For an operating system configuration that will support applications, you need to configure your disks properly. This involves proper capacity planning, as well as configuring components like the page file and making sure the operating system can see the disks and access them without errors.

Whether you are implementing a server cluster or not, your system disk must be configured properly. It should be of sufficient size to accommodate all of the program files (and other files) you might place on it, the operating system, and potentially the page file. You might even want to make it part of a RAID stripe set to ensure you have redundancy in the event of a disk failure.

More Info

For information on configuring your page file, see the section "Page File Size and Location" in Chapter 14.

With both Windows 2000 Server and Windows Server 2003, you also have the option to boot from a storage area network (SAN). The storage vendor must support this configuration; otherwise, do not attempt this because you might invalidate your support contract. If you encounter problems booting your Windows system, the storage vendor should be the first point of contact for support. You should take into account the following issues if you want to boot from a SAN, but talk to your storage vendor to get a complete list.

  • Fibre Channel–Arbitrated Loop (FC–AL) is not supported if you want to boot from a SAN. The SAN must be in a switched environment or directly connected to the servers.

  • The server must get exclusive access to the disk it will be booting from. No other server should be able to access or see the bootable logical unit (LUN). This means that you need to use zoning, masking, or a combination of the two.

  • Redundancy can ensure that your system will always be able to see the system disk. This means that using multipathing and multiple host bus adapters (HBAs) will help greatly. However, ensure that the multipath software is on the Microsoft Hardware Compatibility List (HCL) or in the Windows Catalog for your solution.

  • In a clustered scenario, the SAN needs to be on the multicluster device list if the individual nodes will be booting from a SAN.

  • For Windows Server 2003, the HBA driver must be a Storport driver, which is new to Windows Server 2003. The manufacturer must code the driver to this.

  • Booting from a SAN might limit your scalability and recoverability options by putting all of your eggs in one proverbial basket. For example, if you have a two-node cluster and have the shared cluster disks as well as the boot partitions on the SAN, if you lose the SAN, you literally lose everything.

  • You need to ensure there will not be any I/O performance issues, especially if it is a cluster, because all disk I/O will be going through the same channels to get to the disks.

If you are implementing a server cluster, you might need to take into account the quorum disk, a disk resource for Microsoft Distributed Transaction Coordinator (MS DTC), as well as any application data (such as SQL Server data and log files).

Tip

For security, you can encrypt your file system. This is fully supported with a server cluster only on Windows Server 2003. To do this, you must enable Kerberos, and all cluster node computer accounts and the cluster server service account must be trusted.

Security

The security of any server that runs SQL Server should be of paramount concern in your planning and, ultimately, your deployment of that server. You should not put any features on your server that you will not need, and you should uninstall ones you are not using. For example, if you are not using the Web-based query features of SQL Server 2000, you should not configure—or at least you should disable—Internet Information Services (IIS) so that it does not consume resources and you will not have anyone trying to access your server.

To assess your security, you can use the Microsoft Baseline Security Analyzer, which is located at http://www.microsoft.com/technet/treeview/default.asp?url=/technet/security/tools/Tools/MBSAhome.asp. You can download this free utility and run it on your systems to scan for potential vulnerabilities.

More Info

Security is a topic that can obviously constitute a book (or books) of its own. Use the information provided in this book as a primer and then do more research on your own. See Chapter 2, for information on how to think about security in a high availability environment, as well as Chapter 14 for some SQL Server–specific topics. For an excellent resource on configuring secure applications, including securing your operating system, consult the Microsoft Press book Designing Secure Web-Based Applications for Microsoft Windows 2000. Although this was written for Windows 2000 Server, its usage can certainly be applied to Windows Server 2003. For information on securing your Windows servers, this link for Windows has many links to security-related documents: http://www.microsoft.com/technet/treeview/default.asp?url=/technet/security/prodtech/windows/secwin2k/default.asp

Here you can find such topics as Windows Security, Best Practices, Tools and Checklists, and more.

Terminal Server

Remember to consider how people will eventually access the server in a production environment. One way to do this is to configure Terminal Services, whereby people can log into your server remotely and do what they need to without physical access to the server. However, if you are going to be employing Terminal Services on a server cluster under Windows 2000 Server, there are a few points you need to consider:

  • In the event you fail over to another node, all of the information held at the local node is lost, including the Remote Desktop Connection from the client computer. Although Terminal Services and a server cluster can coexist, they are blind to each other.

  • Terminal Services must be installed only in Remote Administration mode on a cluster node.

  • Remote Administration mode only allows up to two connections.

  • Using Disk Management, you might see the following behavior:

    • If you change a drive letter, the change might not be reflected immediately.

    • If you create or remove a partition, if you view its properties you will see either the label of the partition is not accurate or the partition is claiming it is only 0 bytes.

  • If a failover occurs, you might not be able to access a shared cluster disk, or a question mark might be displayed.

Tip

To solve the display issues, log out and log back into Terminal server.

Under Windows Server 2003, you should use Remote Desktop Administration to access the nodes. You should not install Terminal Services directly on your cluster nodes when using it with SQL Server 2000 failover clustering.

Windows Server 2003 Enhancements

Any administrator will want to be aware of some new features of Windows Server 2003. These features are described next.

Server Roles

The concept of a server role, which allows your installation to be more geared to a specific use, is new to Windows Server 2003. Server roles that are configured are displayed in Manage Your Server, as shown in Figure 5-3. By default, no server roles are configured when Windows Server 2003 is installed.

Manage Your Server application.

Figure 5-3. Manage Your Server application.

More Info

For more information on server roles in Windows Server 2003, see the documentation that comes with the operating system.

Unless you have a specific need to enable server roles (such as Application Server if you need IIS for some SQL Server features you might be using), you should disable all roles not in use to ensure not only a proper configuration but also to remove any unnecessary components that could cause a security risk to the SQL Server. To disable the roles, follow these steps:

  1. Log on as someone with administrative privileges to the local server.

  2. If Don’t Display This Page At Logon has not been selected in the past, the Manage Your Server application appears (as in Figure 5-3). If this screen does not appear, select Start, then Manage Your Server, or under Start, Administrative Tools, select Manage Your Server. Select the Add Or Remove A Role option.

    If you want to bypass the Manage Your Server screen, select Start, Administrative Tools, and then Configure Your Server Wizard. When you select Add Or Remove A Role, this is where you are directed.

  3. In the Welcome To The Configure Your Server Wizard page, shown in Figure 5-4, click Next.

    Welcome To The Configure Your Server Wizard page.

    Figure 5-4. Welcome To The Configure Your Server Wizard page.

  4. In the Preliminary Steps page, shown in Figure 5-5, click Next. The wizard gathers information about your Windows Server 2003 configuration.

    Preliminary Steps wizard page.

    Figure 5-5. Preliminary Steps wizard page.

  5. In the Server Role wizard page, shown in Figure 5-6, all of the server roles that are configured are designated with a Yes next to the name of the server role. To select a server role to remove, select it and click Next.

    Server Role wizard page.

    Figure 5-6. Server Role wizard page.

  6. In the Role Removal Confirmation wizard page, shown in Figure 5-7, make sure that the Summary reflects the operations you would like to perform. When ready, select the Remove The <name of role> Role check box and click Next.

    Role Removal Confirmation wizard page.

    Figure 5-7. Role Removal Confirmation wizard page.

  7. Once the role is removed, a final wizard page confirms the removal, as shown in Figure 5-8. Click Finish to close the Configure Your Server Wizard.

    Final confirming wizard page.

    Figure 5-8. Final confirming wizard page.

  8. Repeat this process for all unnecessary server roles.

Tip

In your Windows Server 2003 installation directory, there is a directory called Debug. This directory contains a file called Configure Your Server.log that stores any changes made to Windows server roles. There are other log files in the Debug directory that you might also want to audit from time to time.

Shutdown

When you shut down or restart a Windows Server 2003 server, you are asked what you want to do (log off the user, shut down, or restart), as shown in Figure 5-9. You must now select the reason you are stopping or restarting your Windows server in the Option drop-down list, and you must enter a Comment. These will show up in Event Viewer, as shown in Figure 5-10. From a high availability and a security standpoint, these help you assess what has been done on the server. This provides you with an audit trail that can be analyzed later to improve processes and procedures and indicate where more training is needed or how you need to change how you do your work. It might seem annoying for administrators who are used to just shutting down or restarting your servers, but it helps force process into your organizations.

Windows Server 2003 Shut Down Windows dialog box.

Figure 5-9. Windows Server 2003 Shut Down Windows dialog box.

Entry in Event Viewer.

Figure 5-10. Entry in Event Viewer.

Important

It cannot be said enough: technology is only a small portion of the answer to availability. The largest portion of all possible downtime is related to how you deal with process, procedure, and people.

If you experienced an unexpected shutdown, when Windows Server 2003 restarts, it prompts you to enter what happened, as shown in Figure 5-11. The entry also appears in Event Viewer.

Shutdown Event Tracker.

Figure 5-11. Shutdown Event Tracker.

High Availability Options for Windows

Whether you are configuring Windows 2000 Server or Windows Server 2003, you are referring to different forms of Windows Clustering. With any version of a stand-alone Windows server—unless you employ RAID for the system disks—clustering or using third-party offerings is the only way to ensure that another server will start to perform the work if the server and its applications fail. Even if you configure a duplicate server with the same software and operating system options, you would need a process to switch to that server. This is where a server cluster and Network Load Balancing might be able to assist you in increasing availability to your Windows installation.

Windows Reliability Features

In addition to clustering, there are other availability and reliability features built into Windows that are better covered in other documents. Here is a list of topics you can use to research more information.

Windows 2000 Server

The following were introduced with Windows 2000 Server to increase availability and reliability:

  • Reboots minimized for some configuration tasks

  • Slipstreaming of service packs and chaining of hotfixes

  • Application failure recovery with IIS restart and out of process application protection

  • Safe mode boot

  • System Recovery Console

Windows Server 2003

The following were introduced with Windows Server 2003 to increase availability and reliability:

  • Tools to measure and audit, including the already mentioned Shutdown Tracker, enhanced logging, and system tracing

  • The overall hardware and software installation has been improved, including:

    • Driver install improvements

    • Driver rollback

    • New driver preference using .inf files

    • Hotplug Peripheral Component Interconnect (PCI)

    • Side-by-side assembly install

    • Hot add RAM

    • Signature support

  • Volume Shadow Copy (VSS) Backups and enhanced backup and defragment APIs

  • Fault tolerance system support through things like machine check architecture for 64-bit, multipath I/O and Load Balancing and Failover (LBFO)

  • Better recovery through processes like autosystem recovery, last known good enhancements, up to an eight-node server cluster, geographically dispersed clusters, Majority Node Set (MNS) clusters, process cycling for IIS out-of-proc applications, COM+ object recycling, and service restarts

  • Maintenance tasks have also been improved, such as hot patch operating system hotfixes, no reboot hotfixes, and update improvements

  • Testing and qualification

    • Driver Verifier and App Verifier

    • Device, system, and domain controller Hardware Compatibility Tests (HCTs) and qualifications improved

    • Certification improvements

  • Services "decomposed"

More Info

For basic information about server clusters and Network Load Balancing, see "Windows Clustering" in Chapter 3.

Important

You cannot combine a server cluster and Network Load Balancing on the same physical hardware. They can coexist on the same network or domain, but they cannot be configured to run together on Windows installation.

Server Clusters

Server clusters are the pure availability form of Windows Clustering, and they form the basis for a SQL Server 2000 failover cluster. If you do not configure it properly, your failover cluster installation could be built on a bed of quicksand.

More Info

For more information on server clusters, you can go to the Cluster Technologies Community Center, located at http://www.microsoft.com/windowsserver2003/community/centers/clustering/default.asp. This site covers Microsoft Windows NT 4.0, Windows 2000 Server, and Windows Server 2003. If you want direct access to white papers in the Community Center, they can be found at http://www.microsoft.com/windowsserver2003/community/centers/clustering/more_resources.asp#MSCS. There are many best practices papers, such as for security, which might cover topics that are not mentioned in this chapter.

Planning a Server Cluster

Planning your server cluster is in many ways more important than the implementation itself: most issues that eventually crop up with a server cluster configuration stem from a missed configuration point.

On the CD

To assist you in your planning, use the document Server_Cluster_Configuration_Worksheet.doc.

Types of Server Clusters

Starting with Windows Server 2003, there are two types of server clusters: a standard server cluster, which is the same technology that can be found in Windows 2000 Server, and a new type called a majority node set (MNS) cluster. Both utilize many of the same semantics behind the scenes and behave similarly, but there are a few main differences.

The first difference is the quorum resource. The quorum not only contains the definitive and up-to-date server cluster configuration, but it also is used in the event a "split-brain" scenario occurs. A split-brain scenario can happen if two or more nodes in your server cluster lose all public and private network connectivity. At that point, you might have different "partitions" of your server cluster. The node owning the quorum resource gains ownership of all clustered resources, and the other nodes not seen by the partition that can use the quorum are evicted.

Both types require a quorum, but the mechanism is different. For a standard server cluster, this disk is accessed by one server at a time and is on the shared disk array. Under an MNS cluster, this is not a disk at all—nothing is shared. This is only found on an MNS cluster under Windows Server 2003. The quorum is actually found on your system disk in %SystemRoot%ClusterQoN.%ResourceGUID%$\%ResourceGUID%$MSCS. This directory and its contents should not be modified in any way. The other nodes access the quorum through a share named \%NodeName%\%ResourceGUID%$ created with local access. Again, because all nodes of the cluster use this share, do not modify permissions with the share name, the Administrators group, or the Cluster Service account itself.

Tip

If you are implementing an MNS cluster, you should use RAID on your system disks to ensure the availability of your quorum. Do not use Integrated Device Electronics (IDE) disks.

Note

If the node owning the resources is still up and it is, say, the first node in the cluster, the cluster might appear to function until a reboot of that node. If all of the other nodes are still unavailable, the MNS cluster will not start and you might have to force the quorum. When nodes go offline, you might see a message pop up, alerting you that a delayed write to Crs.log failed.

SQL Server supports both types of server clusters. If you use an MNS cluster, you have the immediate benefit of not worrying about one more shared disk taking up a drive letter. It also gives you another possible geographic solution, assuming your hardware vendor certifies the solution. You are also protecting yourself from physical quorum disk failures bringing down the cluster. However, because losing the wrong number of nodes causes the entire solution to go down, it might not be a good choice. Table 5-2 shows the number of node failures tolerated. If your vendor builds a geographic solution based on MNS clusters, it might be a good thing, but for a local SQL Server cluster, you might be better off implementing a standard server cluster. You must weigh your options.

Table 5-2. Numbers of Nodes and Failure Tolerance in a Majority Node Set Cluster

Nodes

Maximum Node Failures Tolerated

1

0

2

0

3

1

4

1

5

2

6

2

7

3

8

3

More Info

For more information on MNS clusters, see the white paper "Server Clusters: Majority Node Set Quorum" at http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/windowsserver2003/deploy/confeat/majnode.asp.

Disk Subsystem

For up to a two-node Windows 2000 Advanced Server and Windows Server 2003 Enterprise Edition 32-bit server cluster, the shared disk subsystem can be either SCSI or fibre-based. For anything more than two nodes, all Datacenter editions, and all 64-bit editions, you must use fibre.

Cluster Service Account

The service account used to administer your server cluster does not need to be a domain administrator. In fact, you should not make the cluster administrator a domain administrator because that is an escalation of privileges. Because the cluster administrator account needs to be able to log into SQL Server, you could expose yourself if someone maliciously impersonated that account. The Cluster Service account must be a domain account that is a member of the local Administrators group on each node. During the installation of the server cluster, the account is configured with the proper rights, but if you ever need to manually re-create the account on each node, here are the privileges required:

  • Act As Part Of The Operating System

  • Back Up Files And Directories

  • Increase Quotas

  • Increase Scheduling Priority

  • Load And Unload Device Drivers

  • Lock Pages In Memory

  • Log On As A Service

  • Restore Files And Directories

Of this list, only Lock Pages In Memory, Log On As A Service, and Act As Part Of The Operating System are not granted when you place an account in the Administrators group on a server. As part of that group, the account also inherits the rights of Manage Auditing And Security Log, Debug Programs, and Impersonate A Client After Authentication (Windows Server 2003 only). Even if you restrict these privileges to other administrators, these rights must be granted to the Cluster Service account.

Tip

If you have multiple clusters, you might want to use the same service account for each to ease administration.

Tip

In some companies, security administrators lock down the rights that an account can use on a server. Without the listed privileges, your server clusters will not work. Work with your security or network administrators to ensure that the service accounts used for the server cluster and for SQL Server have the rights they need.

Server Clusters, the Hardware Compatibility List, and the Windows Catalog

Whether you are implementing a standard server cluster or an MNS cluster under Windows Server 2003, your solution must be on the Hardware Compatibility List (HCL), and going forward in the Windows Catalog. The HCL can be found at http://www.microsoft.com/hwdq/hcl, and the Windows Catalog for server products can be found at http://www.microsoft.com/windows/catalog/server2/default.aspx?subID=22. Remember that the entire solution must be on the HCL—server nodes, network cards, the SAN or direct attached storage (DAS) device, driver versions, and so on.

In the Windows Catalog, there are two subcategories that you need to check under the main Cluster Solutions category: Cluster Solution and Geographically Dispersed Cluster Solution. In the HCL, the categories that you need to check are Cluster, Cluster/DataCenter 2-Node, Cluster/DataCenter 4-Node, Cluster/Geographic (2-Node Advanced Server), Cluster/Geographic (2-Node Datacenter Server), Cluster/Geographic (4-Node Datacenter Server), and Cluster/Multi-Cluster Device. There are other cluster-related categories on the HCL, but they are for vendors only.

What is the purpose of these lists? The goal is to ensure that you have a known good platform for your server cluster solutions. Because you are combining multiple products (including drivers) to work with Windows, they need to work well together. The validation process involves low-level device tests of both the storage and the network as well as higher-level stress tests of the cluster under extreme conditions, such as many failures or heavy load. The Windows operating system has specific requirements of the hardware, particularly in a server cluster environment where data disks are physically connected to and visible to all the nodes in the cluster. Configurations that have not passed the server cluster test process are not guaranteed to operate correctly. Because disks are visible to multiple nodes, this can lead to corruption of application data or instability where the connection to the disks is not reliable.

Note

Vendors must submit their solutions for testing; Microsoft does not dictate which components they can use. With Windows Server 2003, there is a new qualification process called the Enterprise Qualification Process. The difference you will notice is that the lists displayed in the HCL or the Windows Catalog point back to the vendor’s own Web sites, so it is up to the vendor to make sure the information is accurate.

Should you choose to ignore the HCL or Windows Catalog and put a non-compliant cluster into your production environment, your solution will technically be considered unsupported. Microsoft Product Support Services (PSS) will assist you as best they can, but they might not be able to resolve your issues because your solution is nonstandard. If you do buy a supported solution, what does "supported" mean? Because Microsoft cannot debug or support software from third-party vendors, or support and troubleshoot complex hardware issues alone, who supports what?

Microsoft supports qualified configurations, along with the vendors where appropriate, to provide root cause analysis. There are some deviations allowed, which are detailed in Table 5-3. When necessary, Microsoft escalates any issues found in the Windows operating system through the Windows escalation process. This process is used to ensure that any required hotfixes for the Windows operating system are provided to the customer. Microsoft supports and troubleshoots the Windows operating system and the server cluster components to determine the root cause. Part of that process might involve disabling non-Microsoft products in an effort to isolate issues to specific components or reduce the number of variables, but only if it does not impact the environment. For example, it would be possible to disable quota management software, but disabling storage multipath software would require infrastructure changes in the SAN; even worse, disabling volume management software might mean that the data is no longer available to applications.

Table 5-3. Supported Deviations from Windows Catalog

Component

Server Model

Deviations Allowed

Server model

Model number

Number of processors

  

Memory size (unless it is lower than what is required for Windows)

  

Number of network cards (but there must be at least two physically separate network cards)

  

Processor speed (can be different up to 500 MHz only)

Host bus adapter (Fibre Channel only)

Driver version (miniport or full port)

See note

 

Firmware version

 

Multipath software (MPIO)

Driver version

See note

Storage controller

Firmware version

See note

Multisite interconnect for geographically dispersed clusters

Network technology (Dense Wavelength Division Multiplexing [DWDM], Asynchronous Transfer Mode [ATM], and so on)

No deviations allowed

Latency between sites must be less than 500 ms round trip

 

Switch type and firmware version

 

Geographically dispersed cluster software components

Driver version

Software version

No deviations allowed

If the analysis points toward non-Microsoft components in the configuration, the customer must work with Microsoft to engage the appropriate vendor within the context of agreements between the customer and the vendor. Microsoft does not provide a support path to vendors, so it is important that all components in the system are covered by appropriate service agreements between the customer and the third-party vendors.

Note

Third-party vendors might update versions of the drivers and firmware that deviate from the Windows Catalog and HCL listings. These can be qualified for your server clusters, but a major version change for any component requires the vendor to submit the solution for requalification, which will subsequently be listed separately. A minor version change does not require a resubmission of the clustered solution, but your vendor must provide a statement listing what combinations are supported. A vendor typically tests a specific combination of driver version, HBA firmware, multipath software version, and controller version to make up the solution.

Because complete vendor-qualified combinations are the only supported solutions, you cannot mix and match versions. The same rules apply for hotfixes and service packs released by Microsoft. A qualification for Windows 2000 Server implies all service packs and hotfixes, but vendors might not have tested their components against the latest service packs or with the hotfixes. Vendors should provide a statement indicating whether different combinations are supported and tested.

If you plan on connecting multiple servers (either other cluster nodes or stand-alone servers) on a single SAN, you also need to consult the Cluster/Multi-Cluster Device list of the HCL. Although there are many SANs out there, not all are certified for use with a server cluster, and not all can support multiple devices without interrupting others in the chain.

Under no circumstances can you mix and match components from various clusters or lists to make up your cluster solution. Consider the following examples:

  • The following would not be a supported solution: Node 1 from vendor A listed under one of the System/Server categories on the HCL (or even listed as a node under the Cluster category), node 2 from vendor B listed under one of the System/Server categories on the HCL (or even listed as a node under the Cluster category), a Fibre Channel card from the Storage/FibreChannel Adapter (Large Memory Category), and a SAN from the Cluster/Multi-Cluster Device.

  • If you use two servers that are on the Cluster list, but you take a different fibre controller from the Cluster/FibreChannel Adapter list (that is, change the base configuration of the cluster solution listed on the Cluster list), and use a SAN that is not on the HCL at all, this would be an unsupported cluster.

  • You implement a Unisys ES-7000 as a one-node cluster because you want to use 16 processors for one Windows installation and plan on adding in the other node later. You then could not, say, go get another type of server and throw it into the cluster as a second node. Assuming your ES-7000 and disk array are part of what would be on the HCL as one of the Cluster categories, you could either add more processors to your ES-7000 and then carve out another server that way because it is a partitionable server, or buy another ES-7000.

More Info

The following are helpful Knowledge Base articles, located at http://support.microsoft.com/, about the Microsoft support policy for server clusters:

  • 309395: "The Microsoft Support Policy for Server Clusters and the Hardware Compatibility List"

  • 304415: "Support for Multiple Clusters Attached to the Same SAN Device"

  • 327831: "Support for a Single Server Cluster That Is Attached to Multiple Storage Area Networks (SANs)"

  • 280743: "Windows Clustering and Geographically Separate Sites"

  • 327518: "The Microsoft Support Policy for a SQL Server Failover Cluster"

Certified Cluster Applications

If you use Windows 2000 Server Advanced Server and Windows Server 2003 Enterprise Edition, you are not required to check that the applications you are running on your cluster are certified to work in a cluster. With all Datacenter Editions, the opposite is true: the application must be certified for use with Windows Datacenter. Applications that earn the Certified for Windows logo are listed in the Windows Catalog, and you can also consult Veritest’s Web site (http://cert.veritest.com/CfWreports/server/).

Ports, Firewalls, Remote Procedure Calls, and Server Clusters

Server clusters can work with firewalls, but you need to understand what ports you need to open. A server cluster uses User Datagram Protocol (UDP) port 3343 for the intracluster, or heartbeat, communication. Because this is a known port that is registered with the Internet Assigned Numbers Authority (IANA), you need to ensure that you cannot encounter a denial of service type of attack that can interfere with and potentially stop your server cluster.

A server cluster is dependent on remote procedure calls (RPCs) and the services that support them. You must ensure that the RPC service is always up and running on your cluster nodes.

More Info

For more information on ports, firewalls, and RPCs, reference the following resources:

  • Knowledge Base article 154596: "Configure RPC Dynamic Port Allocation to Work with Firewall"

  • Knowledge Base article 258469: "Cluster Service May Not Start After Restricting IP Ports for RPC"

  • Knowledge Base article 300083: "Restrict TCP/IP Ports on Windows 2000"

  • Knowledge Base article 318432: "BUG: Cannot Connect to a Clustered Named Instance Through a Firewall"

  • "TCP and User Datagram Protocol Port Assignments" (from the Windows 2000 Server Resource Kit)

  • On Microsoft TechNet, search for "TCP and UDP Port Assignments."

Geographically Dispersed Clusters

If you want to physically separate your cluster nodes to provide protection from site failure, you need a certified geographic cluster solution on the HCL or in the Windows Catalog. You cannot take two or more nodes, separate them, and implement a server cluster. That configuration is completely unsupported.

Antivirus Programs, Server Clusters, and SQL Server

In the past few years, use of antivirus software on computers has increased dramatically. In many environments, this software is installed as a de facto standard on all production servers. In a clustered environment and with SQL Server, antivirus requirements should be evaluated because antivirus programs are typically not cluster-aware, meaning they are not aware of how disks work in a clustered environment. The software might interfere with cluster operations. The server running SQL Server itself, unless it is, say, hosting a Web server or files for a solution, should technically have no need for virus scanning because the database-related .mdf, .ndf, and .ldf files are always managed by SQL Server and are not like a Microsoft Word or other text file that could be handled by an arbitrary application, or contain executable code or macros.

If you do need to place antivirus software on your server clusters, use the program’s filters to exclude the Mscs directory on the quorum for a standard server cluster. For SQL Server, use the filter to exclude all data and log directories on the shared disk array. If you do not filter the SQL Server files out, you might have problems on failover. You might also want to filter the Msdtc directory used by MS DTC.

For example, consider a situation in which you encounter a failover and there is no filter. When the shared disks are recognized by another node, the virus scanner scans them, preventing the failover from completing and SQL Server from recovering. If your databases are very large, the problem will be worse.

More Info

See Knowledge Base article 250355, "Antivirus Software May Cause Problems with Cluster Services," at http://support.microsoft.com.

Server Clusters, Domains, and Networking

Network requirements are a source of contention among some people considering a server cluster. These requirements are often misunderstood. No clustered solution can work without domain connectivity. If you cannot guarantee that the nodes will have domain access, do not attempt to implement a server cluster.

Network Configuration

The following are the requirements for configuring your network cards for use with a server cluster:

  • To configure a server cluster you will need the following dedicated IP addresses: one IP address for each node on the public network, one IP address for each node on the private network, one IP address for the server cluster itself, and at least one IP address for each SQL Server 2000 instance that will be installed.

  • All cluster nodes must have domain connectivity. There is no way to implement a server cluster without it. If your cluster is going to be in a demilitarized zone (DMZ), you must have a domain controller in the DMZ or open a hole to your corporate network for the clusters.

  • All cluster nodes must be members of the same domain.

  • The domain the nodes belong to must meet the following standards:

    • There must be redundant domain controllers.

    • There must be at least two domain controllers configured as global catalog servers.

    • If you are using Domain Name System (DNS), there must be redundant DNS servers.

    • DNS servers must support dynamic updates.

    • If the domain controllers are the DNS servers, they should each point to themselves for primary DNS resolution and to others for secondary resolution.

  • You should not configure your cluster nodes as domain controllers; instead, you should have dedicated domain controllers. If you do configure your nodes in any combination of a primary and backup domain controller, it could have direct implications for any SQL Server environment, as noted in Knowledge Base articles 298570, "BUG: Virtual SQL Server 2000 Installations May Fail if Installed to Windows 2000 Domain Controllers," and 281662, "Windows 2000 and Windows Server 2003 Cluster Nodes As Domain Controllers." In general, you should not install SQL Server 2000 on a domain controller.

  • Windows Server 2003 does not require NetBIOS, so you can disable NetBIOS support. However, you need to know the implications of doing this and ensure that nothing else you have running on your cluster needs NetBIOS name resolution. This includes Cluster Administrator, which uses NetBIOS to enumerate the clusters in the domain, meaning you cannot use the browse functionality. By default, NetBIOS is enabled, but you can disable it on each IP resource’s properties.

  • Use two (or more) completely independent networks that connect the two servers and can fail independently of each other to ensure that you have no single points of failure. This means that the public and private networks must have separate paths (including switches, routers, and hubs) and physically independent hardware. If you are using a multiport network card to serve both the private and public networks, it does not meet the stated requirement.

  • Each individual network used for a server cluster must be configured as a subnet that is distinct and different from the other cluster networks. For example, you could use 172.10.x.x and 172.20.x.x, both of which have subnet masks of 255.255.0.0. You should use separate subnets for the networks due to the implications of multiple adapters on the same network, as noted in Knowledge Base article 175767, "Expected Behavior of Multiple Adapters on Same Network."

  • If desired, you can use a crossover cable to connect your cluster nodes for intracluster communications. Using a regular network is recommended.

  • If you are implementing a geographically dispersed cluster, the nodes can be on different physical networks. The private and public networks, on the other hand, must appear to the cluster as a single, nonrouted LAN using something like a VLAN.

  • Round-trip time between cluster nodes for the heartbeat must be less than 500 ms for all types of server clusters, local or geographic.

Network Card Configuration

To configure your network cards for use in a server cluster, address the following public network and private network configuration points.

Public Network Configuration

Consider the following points when configuring your public network dedicated for cluster communications:

  • The speed of the network card should match the maximum speed of the network card and its underlying network. This must be set the same on each public adapter. Do not set auto detect.

  • You cannot enable Network Load Balancing on the same server or network card used for a server cluster.

  • Both a primary and secondary DNS must be configured.

  • Static IP addresses are strongly recommended. Do not use Dynamic Host Configuration Protocol (DHCP).

  • The public network should be configured for all cluster communications to have redundancy for your private network. Because that is a requirement of a server cluster, it is the recommended implementation method.

Private Network (Heartbeat) Configuration

Consider the following when configuring your private network dedicated for cluster communications:

  • The speed of the network card should match the maximum speed of the network card and its underlying network. This must be set the same on each private adapter. Do not set auto detect.

  • Do not set a default gateway.

  • Disable NetBIOS. This can be found on the WINS tab of the Advanced Properties dialog box for the network card.

  • Only TCP/IP should be enabled. No other protocol or service (such as Network Load Balancing or sharing) should be checked in the properties of the network card.

  • Although network teaming cards are supported in a server cluster, using them for the private network cards used for internal communication only in the cluster is not supported.

  • For private network redundancy, set the publicly faced network to handle both private and public traffic.

  • For a private network, the valid blocks of IP addresses are as follows:

    • 10.0.0.0

    • 172.16.0.0

    • 192.168.0.0

  • You can use a crossover cable, but a regular network is recommended. If you use a crossover cable between the cluster nodes, you must still use static IP addresses.

  • If you are using a crossover cable with Windows 2000, add the following registry key to each node:

    HKEY_LOCAL_MACHINESystemCurrentControlSetServicesTcpipParameters
    Value Name: DisableDHCPMediaSense
    Data Type: REG_DWORD
    Data: 1

    This disables the TCP/IP stack destruction feature of Media Sense.

    Warning

    Do not modify the registry entry for Media Sense on Windows Server 2003 clusters.

  • Disable autosensing or automatic detection of the speed of the network. This could cause problems on the private network.

  • The speed of the network card should match the maximum speed of the network card and its underlying network. This must be set the same on each private adapter. Setting this to 10 MB/sec and half duplex should provide sufficient bandwidth.

Implementing a Server Cluster

Whether you are implementing a standard server cluster or an MNS cluster, you will have to handle some preconfiguration and postconfiguration tasks in addition to installing the cluster itself.

Preconfiguration Tasks

Before you install your server cluster, you must perform some tasks to make sure you are prepared.

On the CD

Use the document Server_Cluster_Pre-Installation_ Checklist.doc to ensure that you are ready to install your server cluster. Also take this time to fill out the worksheet Node_Configuration_Worksheet.doc for each node’s configuration.

Configuring Network Cards

Configure your network cards per the recommendations given earlier.

Network Cards Used on the Public Network

To configure a network card for use on a public network, follow these steps:

  1. From your desktop, right-click My Network Places, and select Properties. Under Windows Server 2003, you might need to enable the Classic Start menu view from the Properties menu on the taskbar to see this.

  2. In the Network And Dial-up Connections (Windows 2000 Server) or Network Connections (Windows Server 2003) window, select the network card. Rename this to something recognizable and usable, such as Public Network, by selecting it, right-clicking, and selecting Rename. This is the same value on all nodes.

  3. Select the Public Network network card, right-click, and select Properties.

  4. Select Internet Protocol (TCP/IP) and click Properties. Set the static IP address of the card to a valid IP address on the externally facing network. This address is different for each node of the server cluster. These addresses will all be on the same subnet, but a different subnet from the private network. Click OK.

  5. Make sure the Subnet mask is correct.

  6. Enter your default gateway.

  7. Enter your primary and secondary DNS servers.

  8. Click OK to return to the Public Network Properties dialog box. Click Configure.

  9. In the Properties dialog box for the network card, select the Advanced tab, shown in Figure 5-12.

    The Advanced tab of the Properties dialog box for a network card.

    Figure 5-12. The Advanced tab of the Properties dialog box for a network card.

  10. For the External PHY property, set the value for the correct network speed. You set this to be the same on each node. Click OK.

  11. Click OK to close the Public Network properties dialog box.

Network Cards Used on the Private Network

To configure a network card for use on a private network, follow these steps:

  1. From your desktop, right-click My Network Places, and select Properties. Under Windows Server 2003, you might need to enable the Classic Start menu view from the Properties menu of the taskbar to see this.

  2. In the Network And Dial-up Connections (Windows 2000 Server) or Network Connections (Windows Server 2003) window, select the network card. This network card is located only on the private network on the approved subnets. Rename this to something recognizable and usable, such as Private Network, by selecting it, right-clicking, and selecting Rename. This is the same value on all nodes.

  3. Select the Private Network network card, right-click, and select Properties.

  4. Make sure that Client For Microsoft Networks, Network Load Balancing, File And Printer Sharing For Microsoft Networks, and any other options are not selected, as shown in Figure 5-13.

    The General tab of the Properties dialog box for a network card.

    Figure 5-13. The General tab of the Properties dialog box for a network card.

  5. Select Internet Protocol (TCP/IP) and click Properties. Set the static IP address of the card to a valid IP address on the externally facing network. This address is different for each node of the server cluster, and must be in the proper class. These addresses will all be on the same subnet, but a different subnet from the public network. Click OK.

  6. Make sure the subnet mask is correct.

  7. Do not enter a default gateway.

  8. Do not enter any DNS servers.

  9. Click Advanced.

  10. Select the WINS tab of the Advanced TCP/IP Settings dialog box, shown in Figure 5-14, and select Disable NetBIOS Over TCP/IP if you are not on an MNS cluster. Click OK.

    The WINS tab of the Advanced TCP/IP Settings dialog box.

    Figure 5-14. The WINS tab of the Advanced TCP/IP Settings dialog box.

  11. Click OK to return to the Private Network Properties dialog box. Click Configure.

  12. Click Advanced. In the Properties dialog box for the network card, select the Advanced tab.

  13. For the External PHY property, as shown in Figure 5-12, set the value for the correct network speed. Set this to be the same on each node. Click OK.

  14. Click OK to close the Private Network Properties dialog box.

  15. If you are on a Windows 2000 server, add the following registry key and its associated values only if you are using a crossover cable:

    HKEY_LOCAL_MACHINESystemCurrentControlSetServicesTcpip
    Parameters
    Value Name: DisableDHCPMediaSense
    Data Type: REG_DWORD
    Data: 1

    Warning

    Do not perform this step if you are using Windows Server 2003.

  16. Repeat this procedure for each node in the server cluster and for each private network.

Changing Network Priority

You need to configure your networks so that they have the right priority and one will not impede the other. At the server level, the public networks should have the priority. You can configure the network by following these steps:

  1. On your desktop, right-click My Network Places and select Properties. If you are using Windows Server 2003, you might need to enable the Classic Start menu view from the Properties menu on the taskbar to see this.

  2. From the Advanced menu, select Advanced to open the Advanced Settings dialog box, shown in Figure 5-15.

    Advanced Settings dialog box.

    Figure 5-15. Advanced Settings dialog box.

  3. All public, or externally faced, networks should have priority over the private ones. If this is not the case, set the proper order and click OK. If you have multiple networks, set them in the proper order. This order is the same on all nodes.

  4. Close the Network Connections (Windows Server 2003) or Network And Dial-Up Connections (Windows Server 2000) window.

  5. Repeat this procedure for each node of the cluster.

Verifying Your Network Connectivity

To verify that the private and public networks are communicating properly prior to installing your server cluster, perform the following steps. It is imperative to know the IP address for each network adapter in the cluster, as well as for the IP cluster itself.

Verifying Connectivity and Name Resolution from a Server Node This method shows how to check both IP connectivity and name resolution at the server level.

  1. On a node, from the Start menu, click Run, and then type cmd in the text box. Click OK.

  2. Type ping ipaddress where ipaddress is the IP address of another node in your server cluster configuration. This must be done for both the public and private networks. Repeat for every other node, including the node you are on.

  3. Type ping servername where servername is the name of another node in your server cluster configuration. Repeat for every other node, including the node you are on.

  4. Repeat Steps 1–3 on each node.

Verifying Connectivity and Name Resolution from a Client or Other Server This method shows how to check both IP connectivity and name resolution at the client or other servers that will access the server cluster.

  1. On a client computer, from the Start menu, click Run, and then type cmd in the text box. Click OK.

  2. Type ping ipaddress where ipaddress is the public IP address of one of the nodes in your server cluster configuration. If there is more than one public IP address, you must repeat this step.

  3. Type ping servername where servername is the name of one of the nodes in your server cluster configuration.

  4. Repeat Steps 2 and 3 for each node.

Creating the Shared Disks

Prior to installing your server cluster, you should also configure all disks that will be used in the cluster up front to minimize downtime later should you need to add a disk. Mountpoints do not get you around the main drive letter limitation, but they give you the ability to add a disk without a drive letter to an existing disk without interrupting the availability of a resource such as SQL Server.

Important

When configuring your server clusters, always configure the first node and the shared disks before you power on the other nodes and allow the operating system to start. You do not want more than one node to access the shared disk array prior to the first node being configured.

Also, for 64-bit editions of Windows Server 2003, the shared cluster disks must not only be basic disks, but they must be partitioned as master boot record (MBR) and not GUID partition table (GPT) disks.

More Info

For more information on using mountpoints with SQL Server 2000, see Chapter 6.

Creating Basic Disks To create a basic disk, follow these steps:

  1. Start Computer Management from the Administration Tools menu.

  2. Select Disk Management.

  3. Select the disk you want to use, right-click, and select New Partition.

  4. Click Next in the Welcome To The New Partition Wizard page.

  5. In the Select Partition Type wizard page, classify this as either a primary partition or an extended partition. Click Next.

  6. In the Specify Partition Size wizard page, enter the size (in megabytes) of the partition. You do not have to use the entire disk, but if you do not, all partitions created are presented as one disk to a server cluster. Click Next.

  7. In the Assign Drive Letter Or Path wizard page, select a drive letter from the Assign The Following Drive Letter drop-down list. Click Next.

  8. In the Format Partition wizard page, select NTFS from the File System drop-down list, define the Allocation Unit Size (which should be 64K for SQL Server disks that will contain data), and specify a name for the Volume. Click Next.

  9. If the disk is not a basic disk (which is the default under Windows Server 2003), convert it to a basic disk.

Creating a Volume Mountpoint To create a volume mountpoint, follow these steps:

  1. Start Cluster Administrator and pause all nodes of the cluster.

    Important

    You will experience a brief bit of downtime when configuring the mountpoint, but it will not be as bad as adding a new disk.

  2. Start Computer Management from the Administration Tools menu.

  3. Select Disk Management.

  4. Select the disk you want to use, right-click, and select New Partition.

  5. Click Next in the Welcome To The New Partition Wizard page.

  6. In the Select Partition Type wizard page, classify this as either a primary partition or an extended partition. Click Next.

  7. In the Specify Partition Size wizard page, enter the size (in megabytes) of the partition. You do not have to use the entire disk, but if you do not, all partitions created are presented as one disk to a server cluster. Click Next.

  8. In the Assign Drive Letter Or Path wizard page, do not assign a drive letter. Click Next.

  9. In the Format Partition wizard page, select NTFS from the File System drop-down list, define the Allocation Unit Size (which should be 64K for SQL Server disks that will contain data), and specify a name for the Volume. Click Next.

  10. If the disk is not a basic disk (which is the default under Windows Server 2003), convert it to a basic disk.

  11. On the disk you want to use as the root of your mountpoint, create a blank folder.

  12. In the Disk Management window, select the new volume that you created, right-click, and select Change Drive Letter And Paths. Click Add.

  13. In the Add Drive Letter Or Path dialog box, select the Mount In The Following Empty NTFS Folder option, and select the folder created in Step 11 (or you can also do it here). Click OK. The change will be reflected in the Change Drive Letter And Path dialog box.

  14. Click OK. Close the Computer Management console.

    Note

    Steps 15 and 16 refer to specific cluster steps to be done after your server cluster is configured, and they are listed here for completeness.

  15. Using Cluster Administrator, in the resource group with the disk that has the root folder, create a new disk resource (see the section "Adding a New Disk" later in this chapter). During the process, you must add the dependency of the root disk to the disk that will serve as a mountpoint.

  16. Unpause all nodes in Cluster Administrator and test failover for the new disk resource.

Important

Do not create a mountpoint on a drive that is not part of the shared disk array. Also, do not use the quorum disk or the disk for MS DTC. You can use mountpoints to expand these drives, however.

Installing the Server Cluster

Once you have completed the preconfiguration tasks, you are ready to install the server cluster. The process under Windows 2000 Server differs from the process under Windows Server 2003.

Under Windows 2000 Server, you have two options to configure the server cluster: using the Cluster Service Configuration Wizard or the command line. Under Windows Server 2003, you have three options to configure your server cluster: through the GUI, using the command line, and an unattended installation.

On the CD

For instructions on configuring a server cluster, see the document Server_Cluster_Installation_Instructions.doc.

Important

When configuring your server clusters, always configure the first node before powering on the other nodes and allowing the operating system to start. A Windows 2000 server cluster is dependent on the IIS Common Files being installed on each node during the operating system installation, or later adding the components through Add/Remove Windows Components. Do not attempt to copy these DLLs from another server into the right locations, as having the DLLs alone is not enough. For informational purposes only, here is a list of the DLLs installed as part of the IIS Common Files:

%systemroot%Admwprox.dll%systemroot%Adsiis.dll%systemroot%Exstrace.dll%systemroot%Iisclex4.dll%systemroot%Iisext.dll%systemroot%Iismap.dll%systemroot%IisRtl.dll%systemroot%Inetsloc.dll%systemroot%Infoadmn.dll%systemroot%Wamregps.dll%systemroot%System32InetsrvCoadmin.dll%systemroot%System32InetsrvIsatq.dll%systemroot%System32InetsrvIisui.dll%systemroot%System32InetsrvLogui.ocx

Postconfiguration Tasks

Once you have installed your server cluster, there are a few things that you must do prior to installing any applications like SQL Server.

On the CD

For a useful checklist to use when performing these tasks and for auditing capabilities, use the document Server_Cluster_Post-Installation_Checklist.doc.

Important

There are some post-Windows 2000 Service Pack 3 hotfixes that you should apply to your nodes that are important for server clusters. As of the writing of this book, the following list is accurate. Please check to see if there are any additional hotfixes for Windows 2000 or any for Windows Server 2003 before you configure your server clusters.

Information about the hotfixes can be found in the following Knowledge Base articles:

  • 325040: "Windows 2000: Drive Letter Changes After You Restart Your Computer"

  • 323233: "Clusres.dll Does Not Make File System Dismount When IsAlive Fails"

  • 307939: "Disks Discovered Without the Cluster Service Running Are Not Protected"

  • 815616: "Clustered Disk Drive Letter Unexpectedly Changes"

  • 326891: "The Clusdisk.sys Driver Does Not Permit Disks to Be Removed by Plug and Play"

There is one other fix that should not be applied unless you have spoken with the storage hardware vendor for your SAN or shared disk array and he or she indicates that the fix is valid and required. You can find more information about this in the following Knowledge Base article:

  • 332023: "Slow Disk Performance When Write Caching Is Enabled"

Configuring Network Priorities

Besides setting the network priorities at the server level, you need to do it in Cluster Administrator for the cluster. In the cluster, the private network is the ruler. Follow these steps to configure network priorities:

  1. Start Cluster Administrator. Select the name of the cluster, right-click, and select Properties, or you can select Properties from the File menu.

  2. Select the Network Priority tab, shown in Figure 5-16. Make sure that the private heartbeat network has priority over any public network. If you have multiple private and public networks, set the appropriate order.

    Network Priority tab.

    Figure 5-16. Network Priority tab.

  3. Click OK. Close Cluster Administrator.

Enabling Kerberos

If you are going to be using Kerberos with your cluster, ensure that the Cluster Service account has the appropriate permissions. Then perform the following steps:

  1. Start Cluster Administrator.

  2. Select the Groups tab, and select the Cluster Group resource group. In the right pane, select Cluster Name, right-click, and select Take Offline.

  3. Once Cluster Name is offline, right-click it and select Properties.

  4. On the Parameters tab, select the Enable Kerberos Authentication check box, and click OK.

  5. Bring online.

Changing the Size of the Cluster Log

The cluster log size defaults to 8 MB, which is not very big. If the cluster log becomes full, the Cluster Service overwrites the first half of the log with the second half. In that case, you can only guarantee that half of your cluster log is valid. To prevent this situation, you must create a system environment variable named ClusterlogSize, following these steps. You should make this value great enough to ensure validity of the cluster log in your environment.

More Info

For more information, consult Knowledge Base article 168801, "How to Turn On Cluster Logging in Microsoft Cluster Server."

  1. From Control Panel, select System (or right-click My Computer).

  2. Select the Advanced tab, and click Environment Variables.

  3. In the Environment Variables dialog box under System Variables, click New.

  4. In the New System Variables dialog box, type ClusterlogSize for the Variable Name, and for Variable Value, enter ClusterlogSize=x, where x is the size in MB.

  5. Click OK. The new environment variable will be displayed.

  6. Click OK two more times to exit.

  7. Reboot the cluster nodes one by one so that the change takes effect. Verify that the Cluster.log file is the appropriate size after the final reboot.

Configuring MS DTC

Microsoft Distributed Transaction Coordinator (MS DTC) is used by SQL Server and other applications. For a SQL virtual server to use MS DTC, it must also be clustered. Its configuration will vary depending on your needs. You cannot use a remote MS DTC; you must configure an MS DTC resource for your server cluster.

Note

MS DTC is shared for all resources in the cluster. If you are implementing multiple SQL Server instances, they will all use the same MS DTC resource.

MS DTC does require disk space; in general, the size used should be roughly 500 MB. However, some applications might have specific requirements. For example, Microsoft Operations Manager recommends a minimum MS DTC size of 512 MB. You obviously need to consider this in your overall disk subsystem planning.

Creating MS DTC on Windows 2000 Server

Under Windows 2000 Server, there are two schools of thought when it comes to designing a server cluster when MS DTC is used:

  • Use the default configuration, which configures MS DTC to use the quorum drive. This is the most popular and most often recommended solution for use with SQL Server. If MS DTC is going to utilize the quorum, be sure there is enough room for the cluster files. You should also set the MS DTC resource so that it does not affect the group in the event of a failure.

  • Plan in advance and create a separate cluster disk dedicated to MS DTC. This might reduce contention on the quorum drive if MS DTC is being used in the cluster, but it might also mean that not enough drives will be available for the SQL Server instances. It also involves a few more steps in configuring the cluster. For example, a clustered Microsoft BizTalk Server configuration requires that MS DTC be placed on a separate drive and in a separate cluster group.

For a default installation of MS DTC using the quorum, perform the following steps:

  1. From the Start menu, choose Run, and type comclust.exe.

  2. Repeat this on each cluster node.

If you want to create MS DTC in its own group, or you need to move it after creating it in the default location, follow the steps given next. Do not place it in the group with SQL Server or make it dependent on any of its resources. You should create a new group (or use an existing one with an unused disk) with its own dedicated IP address, network name, and disk for MS DTC, move the DTC resource into the new group, and add the network name and the disk resource as dependencies of the MS DTC resource.

Important

Make sure no users or applications are connected to your cluster while you are performing this procedure.

  1. Start Cluster Administrator. Select the group that has the dedicated disk you will use for MS DTC and rename the group appropriately.

  2. If you already have a clustered MS DTC in a group (such as the one containing the quorum), delete the existing MS DTC resource. If you have not yet configured MS DTC, you can skip this step.

  3. From the File menu, select New, and then Resource (or right-click the group and select the same options). In the New Resource dialog box, in the Name box, type an appropriate name, such as MS DTC IP Address; in the Resource Type drop-down list, select IP Address. In the Group drop-down list, make sure the right group is selected. Click Next.

  4. In the Possible Owner dialog box, all nodes of the cluster should appear as possible owners. If they do not, add the nodes and then click Next.

  5. In the Dependencies dialog box, select the disk resource in the group you selected from the Available Resources, and then click Add. The disk resource appears in the Resource Dependencies list. Click Next.

  6. In the TCP/IP Address Parameters dialog box, enter the TCP/IP information. In the Address text box, enter the static IP address that will be used with MS DTC. In the Subnet Mask text box, enter the IP subnet if it is not automatically chosen for you. In the Network To Use text box, select the public cluster network you want to use. Click Finish.

  7. You will see a message confirming that the IP address is successfully configured.

  8. In the Cluster Administrator window, the newly created resource appears in the right pane.

  9. From the File menu, select New, and then Resource (or right-click the group and select the same options). In the New Resource dialog box, in the Name text box, type an appropriate name such as MS DTC Network Name. In the Resource Type drop-down list, select Network Name. In the Group drop-down list, make sure the proper group is selected. Click Next.

  10. In the Possible Owner dialog box, all nodes of the cluster should appear as possible owners. If they do not, add the nodes, and click Next.

  11. In the Dependencies dialog box, the MS DTC IP address resource you configured previously appears in the Available Resources list. Select the resource, and then click Add. The resource appears in the Resource Dependencies list. Click Next.

  12. In the Network Name Parameters dialog box, type MSDTC, and then click Finish.

  13. You will see a message confirming that the Network Name resource is successfully configured.

  14. In the Cluster Administrator window, the newly created resource appears in the right pane.

  15. From the File menu, select New, and then Resource (or right-click the group and select the same options). In the New Resource dialog box, in the Name text box, type an appropriate name such as MS DTC. In the Resource Type drop-down list, select Distributed Transaction Coordinator. In the Group drop-down list, make sure the proper group is selected. Click Next.

  16. In the Possible Owner dialog box, all nodes of the cluster should appear as possible owners. If they do not, add the nodes, and click Next.

  17. In the Dependencies dialog box, the MS DTC IP address and network name resources you configured previously appear in the Available Resources list. Select both, and then click Add. The resource appears in the Resource Dependencies list. Click Next.

  18. In the Network Name Parameters dialog box, type MSDTC, and then click Finish.

  19. You will see a message confirming that the Distributed Transaction Coordinator resource is successfully configured.

  20. In the Cluster Administrator window, the newly created resource appears in the right pane.

  21. On each node, rerun Comclust.exe.

  22. On the node that currently owns the MS DTC disk resource, you must reset the log. At a command prompt, type msdtc -resetlog.

  23. To start the new resources, which are all offline, right-click each one, and then click Bring Online.

Creating MS DTC on Windows Server 2003

With Windows Server 2003, the process is completely different, and it is not unlike the second process detailed under Windows 2000 Server. You can no longer run Comclust.exe. You must now manually create an IP address, network name, and Distributed Transaction Coordinator resource, following these steps:

Important

When configuring MS DTC, do not use the group containing a disk with the quorum or any of the ones planned for use with SQL Server.

  1. Start Cluster Administrator. Select the group that has the dedicated disk you will use for MS DTC and rename the group appropriately.

  2. From the File menu, select New, and then Resource (or right-click the group and select the same options). In the New Resource dialog box, in the Name text box, type an appropriate name, such as MS DTC IP Address. In the Resource Type drop-down list, select IP Address. In the Group drop-down list, make sure the right group is selected. Click Next.

  3. In the Possible Owner dialog box, all nodes of the cluster should appear as possible owners. If they do not, add the nodes, and click Next.

  4. In the Dependencies dialog box, select the disk resource in the group you selected from the Available Resources, and then click Add. The disk resource appears in the Resource Dependencies list. Click Next.

  5. In the TCP/IP Address Parameters dialog box, enter the TCP/IP information. In the Address text box, enter the static IP address that will be used with MS DTC. In the Subnet Mask text box, enter the IP subnet if it is not automatically chosen for you. In the Network To Use list box, select the public cluster network you want to use. Click Finish.

  6. You will see a message confirming that the IP address is successfully configured.

  7. In the Cluster Administrator window, the newly created resource appears in the right pane.

  8. From the File menu, select New, and then Resource (or right-click the group and select the same options). In the New Resource dialog box, in the Name text box, type an appropriate name such as MS DTC Network Name. In the Resource Type drop-down list, select Network Name. In the Group drop-down list, make sure the proper group is selected. Click Next.

  9. In the Possible Owner dialog box, all nodes of the cluster should appear as possible owners. If they do not, add the nodes, and click Next.

  10. In the Dependencies dialog box, the MS DTC IP address resource you configured previously appears in the Available Resources list. Select the resource, and then click Add. The resource appears in the Resource Dependencies list. Click Next.

  11. In the Network Name Parameters dialog box, type MSDTC, and then click Finish.

  12. You will see a message confirming that the Network Name resource is successfully configured.

  13. In the Cluster Administrator window, the newly created resource appears in the right pane.

  14. From the File menu, select New, and then Resource (or right-click the group and select the same options). In the New Resource dialog box, in the Name text box, type an appropriate name such as MS DTC. In the Resource Type drop-down list, select Distributed Transaction Coordinator. In the Group drop-down list, make sure the proper group is selected. Click Next.

  15. In the Possible Owner dialog box, all nodes of the cluster should appear as possible owners. If they do not, add the nodes, and click Next.

  16. In the Dependencies dialog box, the MS DTC IP address and network name resources you configured previously appear in the Available Resources list. Select both, and then click Add. The resource appears in the Resource Dependencies list. Click Next.

  17. In the Network Name Parameters dialog box, type an appropriate name such as MS DTC, and then click Finish.

  18. You will see a message confirming that the Distributed Transaction Coordinator resource is successfully configured.

  19. In the Cluster Administrator window, the newly created resource appears in the right pane.

  20. To start the new resources, which are all offline, right-click each one, and then click Bring Online.

Verifying Your Server Cluster Installation

Once you have configured your base server cluster, you need to test it to ensure that it is configured and working properly.

Verifying Connectivity and Name Resolution

To verify that the private and public networks are communicating properly, perform the following steps. It is imperative to know the IP address for each network adapter in the cluster, as well as for the IP cluster itself.

  1. On a node, from the Start menu, click Run, and then type cmd in the text box. Click OK.

  2. Type ping serverclusteripaddress where serverclusteripaddress is the IP address for the server cluster you just configured.

  3. Type ping serverclustername where serverclustername is the network name for the server cluster you just configured.

  4. Repeat Steps 2 and 3 for each node.

  5. Repeat Steps 2 and 3 for representative servers or client machines that will need access to the server cluster.

Failover Validation

You need to ensure that all nodes can own the cluster resource groups that were created. To do this, follow these steps:

  1. Start Cluster Administrator.

  2. Verify that all nodes configured for the failover cluster appear in the bottom of the left pane of Cluster Administrator.

  3. For each cluster group, make sure it can be failed over and back from all nodes in the server. To do this, right-click the group, and select Move. If the server cluster has more than two nodes, you must also select the destination node. This change will be reflected in Cluster Administrator.

Server Cluster Administration

Whether you are implementing a standard server cluster or an MNS cluster, you still need to take care of some administrative tasks. Most administration is done either in Cluster Administrator or using the command-line cluster tool.

Changing Domains

Should you ever need to change the domain of your server cluster, perform the following steps. Changing domains will affect your availability.

  1. Create the Cluster Service account in the new domain, as well as any other necessary logins such as the SQL Server service account.

  2. On the current cluster, change the startup type for the Cluster Service to Manual on each node.

  3. Make sure no users are connected to your clustered applications such as SQL Server. Shut down your applications properly.

  4. Stop the Cluster Service on each node.

  5. Shut down all nodes except one.

  6. Change the domain of the node.

  7. Add the Cluster Service account and any other necessary users with their appropriate rights.

  8. Start the Cluster Service.

  9. Verify that the cluster is functioning properly.

  10. Repeat Steps 6 to 9 for each node.

  11. Reset the startup for the Cluster Service to Automatic on each node.

  12. Follow the instructions in Chapter 6 for changing domains for SQL Server.

Changing a Node’s IP Address or Name

You cannot change the name of a node without uninstalling and reconfiguring the server cluster. However, you can change the underlying IP addresses of the network cards by following these steps:

Warning

Do not do this unless your servers need to change subnets.

  1. Change the IP addresses of the network adapters on one node. This might require the computer to be rebooted.

  2. Start Cluster Administrator on the node whose IP addresses changed and connect to ".". Cluster Administrator might display the following error message because it attempts to connect to the last cluster it administered:

    A connection to the cluster at cluster name could not be opened. This
    may be caused by the Cluster Service on node cluster name not being
    started. Would you like Cluster Administrator to attempt to start the
    Cluster Service on node cluster name?
  3. Select the proper IP Address resource to open its properties, and double-click it.

  4. On the Parameters tab in the IP Address Resource Properties dialog box, make sure that the Network To Use box contains the new network as the network to use.

  5. Fail all groups over to the functional (which is the node whose IP address you changed).

  6. Change the IP addresses for the network adapters on all other nodes, and reboot them. On reboot, when all nodes agree on the subnets, the old networks disappear and the new networks are created.

  7. Rename the networks appropriately.

Changing Service Accounts and Passwords

Should you ever need to change the domain of your server cluster, follow the guidelines in this section.

On the CD

Included is a small utility named Update_Pwd.exe, which is found in Update_Pwd.zip along with the corresponding instruction document Update_Pwd_Readme.doc. This can be used on both clustered and nonclustered servers to change the password of a service while minimizing downtime. Do not use this for the SQL Server services, because they have their own documented procedure elsewhere in this book. Also, if this is Windows Server 2003 and a server cluster, use the command-line tool Cluster.exe to change the password, as described later.

Windows 2000 Server

Under Windows 2000 Server, changing the service account or password for a service account causes an availability outage for your cluster. Follow these steps:

  1. Make sure that no users are accessing the applications in the cluster.

  2. Gracefully stop the SQL Server virtual servers with the SQL Server tools.

  3. Under Administrative Tools, launch the Services tool.

  4. Stop the Cluster Service.

  5. Once Cluster Service is stopped, double-click it to open its properties.

  6. Select the Log On tab.

  7. Change the account and/or password for the Cluster Service.

  8. Click OK.

  9. Start Cluster Service.

  10. Although you might not encounter issues if everything seems to be working, while things are down, you should rerun SQL Server Setup from the node owning the SQL Server resource. Follow the instructions found in Chapter 6 in the section "Adding or Removing a Cluster Node from the Virtual Server Definition and Adding, Changing, or Updating a TCP/IP Address." Click Next for all options, but when you reach this step, enter the new account information. This ensures that the instance is associated with the right account and password.

  11. Repeat Step 10 for each clustered instance of SQL Server.

  12. Bring the SQL virtual servers online.

Windows Server 2003

With Windows Server 2003, you can change your service account and its password as an online operation. You can change the base cluster password on the fly, but you still need to take into account Steps 10 and 11 from the Windows 2000 Server instructions for SQL Server virtual servers. Change passwords through the command-line cluster tool with the /changepassword switch, as shown in Figure 5-17. You must then use one or both of these switches:

  • /skipdc to change the password only on the cluster nodes themselves. If you do not specify /skipdc, it changes the password at the domain level. This is useful if you already changed the password at the network level if you need to do a password update that was not completed on all nodes.

  • /force will force the password to be changed on all accessible nodes, even if they are down, joining, or in an unknown cluster state.

Using Cluster.exe to change the service account password under Windows Server 2003.

Figure 5-17. Using Cluster.exe to change the service account password under Windows Server 2003.

You can also use /verbose to see all output or /quiet to see no output other than errors.

So a full example might look like this:

Cluster /cluster:clustername /changepassword:newpassword,oldpassword /force
/verbose

Disk Management

At some point, you might need to expand your disk capacity. There are two ways to go about that: expanding the disks you have if there is space that has not been allocated, or physically adding a new LUN to present to Windows.

Adding a New Disk

If you need to physically add a new disk to the server cluster, follow the steps below.

Important

If you are using a Windows 2000–based server cluster that has Service Pack 2 or earlier, you need to shut down all nodes but one while performing the next steps. The disk is not automatically recognized even if you can create it dynamically using a SAN. For Windows 2000 with Service Pack 3 or later and the hotfix from Knowledge Base article 326891 (listed earlier), you can add the disks to Windows without restarting the nodes as long as the SAN supports it. There are two other ways to add a disk without rebooting the node with the active resources (such as rebooting a node with no resources to recognize the LUN or using Cluster.exe to force it in even though the GUI for Cluster Administrator does not recognize it, but it requires getting some hexadecimal values for the disk signature), but applying the hotfix is the recommended way.

  1. Create the new LUN on the SAN or DAS. Check with your storage vendor to see if this is an online operation, or one for which you need to shut down your cluster nodes, and then act appropriately.

  2. Once the LUN is added at the disk level, if you need to, restart your nodes. If this is a Windows 2000–based cluster with Service Pack 2 or earlier, only power up one node.

  3. Start the Computer Management console.

  4. Select Disk Management. The new disk should appear. Prepare it as instructed in the section "Creating the Shared Disks."

  5. Close the Computer Management console.

  6. Start Cluster Administrator.

  7. From the File menu, select New, and then Resource (or right-click the group and select the same options). In the New Resource dialog box, in the Name text box, type an appropriate name such as Disk M:. In the Resource Type drop-down list, select Physical Disk. In the Group drop-down list, make sure the proper group is selected. Click Next.

  8. In the Possible Owner dialog box, all nodes of the cluster should appear as possible owners. If they do not, add the nodes, and click Next.

  9. In the Dependencies dialog box, the MS DTC IP address resource you configured previously appears in the Available Resources list. Select the resource, and then click Add. The resource appears in the Resource Dependencies list. Click Next.

  10. In the Disk Parameters dialog box, select the new partition in the Disk drop-down list, and then click Finish.

  11. You will see a message confirming that the new disk resource is successfully configured.

  12. In the Cluster Administrator window, the newly created resource appears in the right pane. Bring the resource online by selecting it, right-clicking it, and selecting Bring Online.

Using DISKPART to Expand Disk Capacity

If you have the capacity on your cluster disk, but have not used all of it (that is, you did not format all of your disk), you can use Diskpart.exe to easily expand without affecting availability. This tool is built into Windows Server 2003 and is available for download for Windows 2000 Server at http://www.microsoft.com/windows2000/techinfo/reskit/tools/new/diskpart-o.asp. It is also available in the Windows 2000 Resource Kit. DISKPART is also a very useful utility to display information about your disks without having to use the Disk Management console.

Note

Make sure your disk subsystem manufacturer supports the use of DISKPART to expand your disk capacity before you use it.

Follow these steps to use DISKPART to expand the size of your disks:

  1. Open a command window and type diskpart. You are now running the utility, as shown in Figure 5-18.

    Starting DISKPART.

    Figure 5-18. Starting DISKPART.

  2. Use the list disk command to show the available disks and their respective numbers. When you see the disk that contains the volume you want to modify, type select disk n, where n is the number of the disk. An example is shown in Figure 5-19.

    Selecting a disk to modify.

    Figure 5-19. Selecting a disk to modify.

  3. Use the LIST VOLUME command to show the available volumes and their respective numbers. When you see the volume you want to modify, type select volume n, where n is the number of the disk. An example is shown in Figure 5-20.

    Selecting a volume.

    Figure 5-20. Selecting a volume.

    Tip

    You can use Computer Management’s Disk Management functionality to assist you in determining which disk and volume you want to use.

  4. Expand the disk capacity by typing the command extend disk=n size=n, as shown in Figure 5-21.

    Extending the capacity.

    Figure 5-21. Extending the capacity.

  5. To verify that the disk has been expanded, you can view it using DISKPART itself with the LIST DISK command (see Figure 5-22), Computer Management’s Disk Management functionality, Windows Explorer, or your application (such as SQL Server’s Enterprise Manager).

    Expanded disk verification.

    Figure 5-22. Expanded disk verification.

Tip

You can also create a script for DISKPART to use instead of entering the commands one by one. Create a text file with your commands, and from a command line, execute the following statement:

diskpart /s nameofscript.txt > output.txt

where nameofscript.txt is your command file, and output.txt is the filename (and location) of the file you want to store the results of the action.

Forcing Quorum for an MNS Cluster

If you need to allow the MNS cluster to continue to function even if a particular node does not own the quorum, you need to manually force the quorum. However, you should not do this unless it is necessary. A good example of an instance in which to force the quorum is if you are using a geographically dispersed cluster and you lose communications with the primary site or experience a failure. Obviously in an MNS cluster, the secondary nodes fail because you no longer have a majority of nodes available. However, to get back up and running, you force the quorum by following these steps:

More Info

You can also use the /forcequorum switch of Cluster.exe as described in the Windows Server 2003 documentation topic "Forcequorum command," but it is recommended that you follow the steps listed here.

Caution

Do not change your cluster configuration while the cluster is operating in this interim state. For example, do not add or remove nodes or install or modify applications running in the cluster. Changing the cluster configuration in this state might result in application data inconsistencies if the other nodes come back online.

  1. Start Cluster Administrator.

  2. Stop the Cluster Service on all remaining nodes.

  3. Open the Registry Editor with Regedt32.exe.

  4. Under HKEY_LOCAL_MACHINESYSTEMCurrentControlSet ServicesClusSvcParameters, create a string value called ForceQuorum.

  5. Set the value of ForceQuorum to a comma-separated list of the names of the nodes that are to be part of the quorum set. If you have a four-node MNS cluster, and Node1 and Node2 are the nodes still available, the values should be Node3 and Node4.

    Important

    If you do not list all the nodes stopped in Step 2 in the ForceQuorum key, then you must either power off the nodes not listed or physically disconnect them from the network before starting the Cluster Service on the remaining nodes.

  6. Start the Cluster Service on the remaining nodes.

    Note

    When you force quorum in an MNS cluster, you might see an error message indicating that the Cluster Service failed to start. Ignore the message, because the Cluster Service will continue its startup processes after the end of the startup timeout period for Service Control Manager.

Network Load Balancing

Network Load Balancing clusters are a feature of all versions of Windows 2000 Server and Windows Server 2003. This section details the specific configuration used as a role change or switch process for highly available SQL Servers or Analysis Servers, which might be different from how other programs utilize Network Load Balancing.

More Info

For a basic look at Network Load Balancing, read Chapter 3. You can also find much more information on Network Load Balancing at the Cluster Technologies Community Center at http://www.microsoft.com/windowsserver2003/community/centers/clustering/default.asp. This site covers Windows NT 4.0, Windows 2000 Server, and Windows Server 2003. If you want direct access to white papers in the Community Center, they can be found at http://www.microsoft.com/windowsserver2003/community/centers/clustering/more_resources.asp#NLB.

General Network Load Balancing Best Practices

This section describes some best practices and necessary information you need to know about Network Load Balancing:

  • You can have up to 32 nodes in a Network Load Balancing cluster.

  • There is the potential that Network Load Balancing can flood the ports with network traffic if you have a heavily used implementation. You must monitor this carefully. Windows Server 2003 introduces Internet Group Membership Protocol (IGMP) support, which limits flooding to only the ports used that have Network Load Balancing nodes. This allows non–Network Load Balancing nodes to not be affected by Network Load Balancing traffic. All other computers and servers can see the Network Load Balancing clusters, but no longer need to worry about the extra traffic.

  • Setting multicast increases the traffic generated by Network Load Balancing. Use unicast mode instead.

  • Network Load Balancing can be implemented over VLANs. This might also help segment the network traffic generated by Network Load Balancing to reduce flooding.

  • Network Load Balancing can work with one or two network cards. Two are not required, but you might want to consider two network cards if you are using unicast, because each node will have the same IP address, making them look identical from a network perspective.

  • All nodes need to be set to the same mode: unicast or multicast.

  • You can mix Windows 2000 Server and Windows Server 2003 servers in a Network Load Balancing cluster. However, if you are mixing versions, you might not be able to use things like IGMP support, which is a new feature of Windows Server 2003.

  • You can only bind Network Load Balancing to multiple network cards in Windows Server 2003. This is not possible in Windows 2000 Server.

  • Network Load Balancing does not support Windows Internet Name Service (WINS) resolution.

  • Network Load Balancing does not support dynamic DNS. You will have to add the name of the Network Load Balancing cluster to any DNS server if you are using name resolution.

  • If you are using remote administration of Network Load Balancing, UDP ports 1717 and 2504 are used. You should use Network Load Balancing Manager, and not a remote WLBS command, in Windows Server 2003.

  • You will need dedicated IP addresses for each node as well as for the Network Load Balancing cluster itself.

  • All nodes need to be in the same subnet.

  • All clients and servers accessing the Network Load Balancing cluster need to be in the same subnet.

  • You can use fault-tolerant network adapters (sometimes known as NIC teaming) with Network Load Balancing. For support information, consult Knowledge Base article 278431, "INFO: Using NIC Teaming Adapters with Network Load Balancing May Cause Network Problems."

Implementing Network Load Balancing for SQL Server–Based Architectures

There are two scenarios in which you can employ Network Load Balancing with your SQL Servers. One is for load balancing read-only (no writes from applications or clients at all) databases; the other is using it as the front mechanism hiding a server name change in a disaster recovery scenario for log shipping, Analysis Services, or replication.

On the CD

For a worksheet to fill in for your Network Load Balancing cluster configuration, see NLB_Cluster_Configuration_Worksheet.doc.

More Info

If you want to do an unattended install of Network Load Balancing, it is possible under Windows Server 2003. See the white paper at http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/windowsserver2003/deploy/confeat/nlbclust.asp for details.

Configuring the Network Cards

To configure your network cards, follow these steps:

  1. Log on to the server as someone with administrator privileges who can configure Network Load Balancing.

  2. Right-click My Network Places, and select Properties.

  3. In the Network Connections window, select the network card that you wish to assign Network Load Balancing to. This should be on an externally facing network. You should also rename this network card to something such as Public Network. This can be done by selecting it, right-clicking, and selecting Rename. For the other network card, you should rename it something like Private Network to designate that it is the dedicated network between the servers participating in Network Load Balancing.

  4. Select the Public Network network card, right-click it, and select Properties.

  5. Select Internet Protocol (TCP/IP) and click Properties. Set the static IP address of the card to a valid IP address on the externally facing network. Click OK.

  6. Click OK to close the Properties dialog box for the public network.

  7. In the Network And Dial-Up Connections window, select the private network, right-click it, and select Properties.

  8. Select Internet Protocol (TCP/IP) and click Properties. Set the static IP address of the card to a valid IP address (such as 10.0.0.1) on the internal-only network. Click OK.

  9. Click OK to close the Properties dialog box for the private network.

  10. Repeat this process for all nodes in the Network Load Balancing cluster.

Configuring Network Load Balancing for Use with SQL Server

To use Network Load Balancing with SQL Server, you must adhere to all of the following guidelines:

  • Two network cards must be installed in each computer. One of these cards must be dedicated to a private network that is not routable from the public network, similar to that of a server cluster.

  • Network Load Balancing is installed on only the externally facing adapter.

  • TCP/IP is the only network protocol present on the cluster adapter. Do not add any other protocols (such as IPX) to this adapter.

  • Only unicast is supported. If only one network adapter is used, no intercluster communications can occur.

  • For a switch situation (not read-only load balanced SQL Servers), all hosts are set to not start automatically. This ensures that, in the event of a switch, the nodes are not available. You must manually start Network Load Balancing.

  • In dual-homed configurations, network adapters are nonroutable to each other.

On the CD

For instructions on configuring Network Load Balancing for use when facilitating a server change for log shipping, replication, and Analysis Services, see the file Installing_NLB_For_A_Server_ Switch.doc. For instructions on configuring Network Load Balancing for load balancing read-only SQL Servers, see the file Installing_NLB_For_ Read_Only_SQL_Servers.doc.

Starting Network Load Balancing on the Primary

With Windows 2000 Server, to start Network Load Balancing on the first node, open a command window and type wlbs start, as shown in Figure 5-23. If this is a read-only load balanced use of SQL Server, execute this on each node.

Starting NLB under Windows 2000 Server.

Figure 5-23. Starting NLB under Windows 2000 Server.

Under Windows Server 2003, select the node in Network Load Balancing Manager, right-click the node, select Control Host, and then select Start. You can also accomplish this by selecting the Host menu, Control Host, and then Start. Once started, Network Load Balancing Manager should look like Figure 5-24.

Starting NLB under Windows Server 2003.

Figure 5-24. Starting NLB under Windows Server 2003.

Verifying a Network Load Balancing Cluster

Once you install your configuration, you need to verify it is working properly.

Switch Method

To verify a Network Load Balancing cluster using the switch method, follow these steps:

  1. In the command window on each node, type wlbs display for Windows 2000 Server. You should see a message about your current Network Load Balancing parameters. If this is Windows Server 2003, you should also see a message reflecting the fact that Network Load Balancing has been started in Network Load Balancing Manager.

  2. Type the command ping w.x.y.z where w.x.y.z is the IP address of your Network Load Balancing cluster. This should be successful. Repeat this step from client machines, other servers, and other nodes to ensure connectivity.

  3. If you are resolving by name, type the command ping nlbname where nlbname is the name of your Network Load Balancing server. This should be successful. Repeat this step from client machines, other servers, and other nodes to ensure connectivity. Only perform this step if you have manually added your NLB cluster to DNS.

  4. On your first node’s SQL Server, create a database with one table and one column with a unique value.

  5. On the other nodes, repeat Step 4 but add another unique value to the instance on that server and use the same name to create the database.

  6. Open Query Analyzer from anywhere. When connecting to the SQL Server, use the Network Load Balancing cluster name or the Network Load Balancing cluster IP address.

  7. Query for the unique record. Currently, it should get that value from the first node.

  8. In the command window for the first node, type the command wlbs drainstop using Windows 2000 Server, or using Windows Server 2003, select the node, right-click it, and select Control Host, and then Drainstop. This clears all connections to that node.

  9. To verify that the process is done, type wlbs query in Windows 2000 Server or look at the status window in Network Load Balancing Manager to ensure that everything is okay with Network Load Balancing.

  10. In the command window of the next node, type wlbs start under Windows 2000 or use Network Load Balancing Manager under Windows Server 2003 to start Network Load Balancing on the new node.

  11. Reissue your query to see that the unique value of the second node is displayed.

  12. Repeat Steps 6–11 for each subsequent node.

Read-Only SQL Servers

To verify a Network Load Balancing cluster for a readonly SQL Server, follow these steps:

  1. In the command window on each node, type wlbs display. You should see a message about your current Network Load Balancing parameters. If this is Windows Server 2003, use Network Load Balancing Manager to check the status and parameters for Network Load Balancing.

  2. Type the command ping w.x.y.z where w.x.y.z is the IP address of your Network Load Balancing cluster. This should be successful. Repeat this step from client machines, other servers, and other nodes to ensure connectivity.

  3. If you are resolving by name, type the command ping nlbname where nlbname is the name of your Network Load Balancing server. This should be successful. Repeat this step from client machines, other servers, and other nodes to ensure connectivity.

  4. On your first node’s SQL Server, create a database with one table and one column with a unique value.

  5. On the other nodes, repeat Step 4 but add another unique value to the instance on that server and use the same name to create the database.

  6. Open Query Analyzer from anywhere. When connecting to the SQL Server, use the Network Load Balancing name or the Network Load Balancing IP address.

  7. Query for one of the unique records.

  8. Reissue your query to see that the unique value of another node is displayed. With enough refreshes, you should eventually see all unique values.

  9. Repeat Steps 6–8 for each subsequent node.

Adding a Network Load Balancing Cluster to DNS

Once your Network Load Balancing cluster is installed, there are a few remaining tasks to perform. You need to add your Network Load Balancing cluster name manually to DNS. Depending on your DNS vendor, this procedure will vary. After you configure your Network Load Balancing cluster, you must manually update DNS to resolve the cluster name. Network Load Balancing does not utilize dynamic DNS updates when it is configured. You might also need to add it to the reverse lookup zone of your DNS as well.

Configuring Logging for Network Load Balancing Manager

When you are using Network Load Balancing Manager, you can log all commands issued to a log file. This creates a permanent record of all Network Load Balancing commands issued. To configure this option, perform the following steps:

  1. Start Network Load Balancing Manager.

  2. From the Options menu, select Log Settings.

  3. In the Log Settings dialog box, shown in Figure 5-25, select the Enable Logging check box. In the Log Filename text box, enter the name and path of the log file. If you do not specify a path name, the file is created in the C:Documents and Settings username directory, where the username is the name of the person currently logged into the server. Click OK.

    Log Settings dialog box.

    Figure 5-25. Log Settings dialog box.

Uninstalling Network Load Balancing

Under Windows Server 2003, in Network Load Balancing Manager, select the cluster name in the upper left pane. From the Cluster menu, select Delete, or right-click the cluster name and select Delete. You are then asked if you want to remove Network Load Balancing from all nodes, as shown in Figure 5-26. Click Yes.

Removing NLB confirmation.

Figure 5-26. Removing NLB confirmation.

When you are finished, the log entries in Network Load Balancing Manager should appear similar to the ones shown in Figure 5-27.

Network Load Balancing Manager after removing Network Load Balancing.

Figure 5-27. Network Load Balancing Manager after removing Network Load Balancing.

Summary

Configuring your Windows servers to be reliable, available, and scalable to form a strong base for SQL Server is not impossible. Although some of the recommendations in this chapter are specific to SQL Server, many are not, and the best practices can be applied to any Windows server. It is crucial to get this step of your entire high availability solution correct and verified; otherwise, you will be placing your SQL Server instances on a questionable foundation.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.136.17.252