Chapter 3

The Management Layer

In this chapter, we'll discuss the points you should take into account when you're designing your management layer. We'll examine the components of this layer and how you incorporate them into your design.

The management layer comprises several components. In this chapter, we'll address what you should consider for your design regarding these items, among others:

  • Operating system and resources to use for your vCenter Server
  • Deciding whether your vCenter Server should be physical or virtual
  • Providing redundancy for the vCenter Server
  • Planning for the security of the management layer

Reviewing the Components of the Management Layer

What is the management layer? It's definitely not the executives or the board of directors of your company. We're talking about the components you use to manage your entire virtual infrastructure on a day-to-day basis. In this section, we'll provide a quick overview of vSphere's management components. We'll start with the main component, vCenter Server.

VMware vCenter Server

vCenter Server (which was once known as VirtualCenter Server) is one of the most centrally critical elements of your virtual infrastructure. It's the management application you'll likely use to manage your virtual datacenter. You'll create datacenters, clusters, resource pools, networks, and datastores; assign permissions; configure alerts; and monitor performance. All of this functionality is centrally configured in the vCenter Server. You should therefore dedicate part of your design to building a robust and scalable vCenter Server.


Compatibility Matrix
Always check the current VMware compatibility matrix before deciding which platform you'll install vCenter Server on. VMware will provide support only if your infrastructure is installed on supported software. This includes the hardware for your ESXi hosts, vCenter Server, underlying database, vSphere Client, vSphere Update Manager, web browser, and so on.

As we'll show you shortly, vCenter Server comes in a couple of different forms (either as a Linux-based virtual appliance or as a Windows application you install on Windows Server). In either case, you can download the most up-to-date version of vCenter from the VMware site, but be warned: depending on which version of vCenter Server you're downloading, it can be a pretty large download (almost 4 GB for the Linux-based virtual appliance, for example).

vCenter Server Components

Prior to vSphere 5.1, vCenter Server was largely a monolithic application. There were really only three major components involved with vCenter Server in vSphere 4.x and vSphere 5.0:

  • An operating system instance. This could be either a Windows OS (a domain member server—it couldn't be a domain controller) or a preinstalled instance of Linux bundled with the vCenter Server Virtual Appliance (vCSA).
  • Access to a back-end database. The local computer where vCenter Server is running could host this database, or a remote computer could host the remote database instance.
  • vCenter Server itself, either installed onto an instance of Windows or preinstalled onto Linux as part of the vCSA.

With the introduction of vSphere 5.1, VMware has broken the monolithic vCenter Server into a number of different components. In addition to the three components listed previously, vCenter Server 5.1 also includes

  • vCenter Single Sign On, a centralized authentication service that enables authorized vCenter Server users to access multiple vCenter Server instances with a single login.
  • vCenter Inventory Service, a service that stores vCenter Server application and inventory data and that works across linked vCenter Server instances. Strictly speaking, this component isn't new to vSphere 5.1; what VMware has done with the 5.1 release is separate the Inventory Service so it can be installed on a separate computer for greater scalability.

The introduction of these additional components in vSphere 5.1 means you also have to decide if you'll run all these components on a single computer or break the roles apart on separate computers.

Let's take a closer look at each of the components in vCenter Server.

Operating System Instance and vCenter Server

We've grouped vCenter Server and the OS instance together because these two components are directly linked to each other. vCenter Server comes in two basic flavors: as an installable application running on an instance of Windows, or as a preinstalled application running on Linux as part of the vCSA. As you can see, choosing Windows as the OS on which to run vCenter Server means you're choosing the installable application version. Choosing Linux as the OS, on the other hand, means you're choosing the preinstalled version of vCenter Server in the virtual appliance. At the time of this writing, there was no option for a Windows-based virtual appliance or an installable version of vCenter Server that ran on Linux.

Later in this chapter, we'll discuss the considerations that go into selecting either the installable version of vCenter Server or the vCSA; we'll also discuss the considerations for choosing a version of Windows on which to run vCenter Server. (Note that you can't choose a Linux distribution or version; this is preinstalled as part of the virtual appliance.)

Back-End Database

Regardless of which form of vCenter Server you choose, you'll need a back-end database. One question that always comes up during the design phase is, “Do I use a central database server or install the database locally on the vCenter Server?” This question ranks right up there with similar questions like, “Should I use full-blown Microsoft SQL Server or Oracle as my database engine?”

We'll examine these questions and others in greater detail later in this chapter. In the section “Examining Key Management Layer Design Decisions,” we'll discuss the various reasons you might choose one database over another (vCenter Server supports several different database servers) and whether you should host the database locally or remotely.

At this stage, though, you just need to know that vCenter Server requires a back-end database. As a result, you'll need to account for the additional resources that a back-end database requires, and you'll need to plan for the maintenance and operation of the back-end database server. In the section “Creating the Management Layer Design,” we'll review how you can ensure the proper availability, manageability, performance, recoverability, and security for the back-end database. (Recall that these principles—introduced in Chapter 1, “An Introduction to Designing VMware Environments,” and referred to as AMPRS—help guide your design decisions.)

vCenter Single Sign On

With the release of vSphere 5.1, VMware introduced a new component to vCenter Server known as vCenter Single Sign On. This component introduces a centralized authentication service that vCenter Server uses to allow for authentication against multiple back-end services, such as Active Directory and LDAP. For smaller environments, vCenter Single Sign On can be installed on the same system as the rest of the vCenter Server components; for larger environments, it can be installed on a separate system. vCenter Single Sign On also supports a variety of topologies, including a single server, a cluster of servers, and a multisite topology.

vCenter Inventory Service

To enable greater scalability for vCenter Server, in vSphere 5.1 VMware also split off the inventory portion of vCenter Server into a separate component. The vCenter Inventory Service now supports the discovery and management of inventory objects across multiple linked vCenter Server instances. As with vCenter Single Sign On, you can install vCenter Inventory Service on the same system as the other components (in what is called a Simple Install), or you can split the Inventory Service onto a separate system for greater scalability (and perhaps high availability/redundancy).

vSphere Web Client Server

To enable the next-generation vSphere Web Client—something we discuss in the next section—vCenter requires a server-side component referred to as vSphere Web Client. In reality, it should be called the vSphere Web Client server, because this is the server-side component that enables the use of a web browser to manage vSphere environments. This component was introduced in vSphere 5.0, but in vSphere 5.1 it takes on much greater importance because some features and tasks in vSphere 5.1 are accessible only from the next-generation web client.

Now, let's shift our focus to the client side.

vSphere Client and vSphere Web Client

Traditionally, VMware administrators have used a Windows-based application known as the vSphere Client to perform the vast majority of the day-to-day management tasks. This is true for VMware vSphere 4.x as well as VMware vSphere 5.0. In vSphere 5.0, VMware introduced a rudimentary Web Client; in vSphere 5.1, the vSphere Web Client (sometimes referred to as the Next-Generation Client [NGC]) takes its first steps toward becoming the primary way of managing VMware vSphere environments.

The Windows-based vSphere Client can be installed on almost any Windows OS:

  • Windows XP Pro, SP3
  • Windows XP Pro 64-bit, SP2
  • Windows Server 2003, SP1
  • Windows Server 2003, SP2
  • Windows Server 2003 Standard, SP2
  • Windows Server 2003 Enterprise, SP2
  • Windows Server 2003 R2, SP2
  • Windows Vista Business, SP2
  • Windows Vista Enterprise, SP2
  • Windows Vista Business 64-bit, SP2
  • Windows Vista Enterprise 64-bit, SP2
  • Windows 7 Client (32-bit and 64-bit)
  • Windows Server 2008 Enterprise, SP2
  • Windows Server 2008 Standard, SP2
  • Windows Server 2008 Datacenter, SP2
  • Windows Server 2008 Enterprise 64-bit, SP2
  • Windows Server 2008 Standard 64-bit, SP2
  • Windows 2008 R2 64-bit

The minimum resources required for the Windows-based vSphere Client are as follows:

  • 266 MHz or faster Intel or AMD processor (500 MHz recommended)
  • 1 GB RAM
  • 1 GB free disk space for a complete installation, which includes the following components:
    • Microsoft .NET 3.5 SP1
    • Microsoft Visual J# 2.0 SE
  • You'll also need 400 MB free on the drive that has the %temp% directory during installation.
  • If all the prerequisites are already installed, 300 MB of free space is required on the drive that has the %temp directory, and 450 MB is required for vSphere.
  • A gigabit network connection is recommended.

For the vSphere Web Client, the following browsers are supported:

  • Microsoft Internet Explorer 7, 8, and 9
  • Mozilla Firefox 3.6 and higher
  • Google Chrome 14 and higher

Tip
As of this writing, Apple's Safari web browser was not supported for use with the vSphere Web Client.

As we stated earlier, the vSphere Web Client takes on significant new importance in vSphere 5.1 environments. To take advantage of vCenter Single Sign On, for example, you must use the vSphere Web Client. If you use the traditional Windows-based vSphere Client, authentication won't use the new vCenter Single Sign On component. You'll note that throughout this book we make an effort to show all screenshots from the new vSphere Web Client as opposed to the older, Windows-based vSphere Client (which remains largely unchanged from previous versions).

These components make up the core of the management layer for a vSphere implementation. However, VMware also offers a number of optional components that you can also choose to include in your design. The first of these that we'll examine is vSphere Update Manager.

vSphere Update Manager

vSphere Update Manager (VUM) is an add-on that VMware provides in order to update your ESXi hosts and VMs. VUM (www.vmware.com/products/update-manager) is VMware's patch-management solution for your virtual environment. It's included with most tiers of vSphere.

In versions of vSphere prior to vSphere 5.0, VUM provided a patch-management solution not only for your ESXi hosts but also for the OSes and certain applications in supported VMs. With the release of vSphere 5.0, VMware discontinued the patch-management component for the OSes and applications in the VMs. This allows organizations to use existing patch-management solutions in place to patch their VMs the same way they patch their physical machines.

A VUM installation requires a few components to get started:

  • A 64-bit instance of a Windows OS. This must be a domain member server (not a domain controller).
  • Access to a database. This can be a remote database (Oracle or Microsoft SQL), or it can be installed locally on the VUM server.

You can install VUM on the same instance of Windows as vCenter Server, if you like; or, for greater scalability and security, you can install VUM on a separate instance of Windows. Note that although VUM requires a 64-bit instance of Windows, VUM itself is a 32-bit application.

With regard to the VUM database, VUM can reside side-by-side with the vCenter database on the same database server, but it can't be the same database as the vCenter Server. In addition to setting up a separate database server (or using an existing separate database server), you also have the option of using an embedded SQL Server 2008 R2 Express database on the VUM server. As soon as you exceed 5 hosts and 50 VMs, though, you'll need to step up to a separate SQL Server or Oracle server for the VUM database.

When you install vCenter Server, you create a system data source name (DSN) entry for the database. You need to create an additional DSN entry for the VUM database as well, as you can see in Figure 3.1. Note that because VUM is a 32-bit application, this needs to be a 32-bit DSN.

Figure 3.1 vSphere Update Manager requires an additional ODBC connection.

3.1

Once you've installed VUM, you configure it for the updates you want to download, schedule the automatic download of patches, set up notifications, scan your ESXi hosts, and update them. Optionally, if the environment necessitates, you can set up a dedicated download server to download patches and distribute them to your VUM server(s). This is particularly applicable in high-security environments, where only certain server are permitted to access external resources on the Internet.


Note
The plug-in that VUM uses is only compatible with the vSphere Client. Thus, in vSphere 5.1 environments, you must use the vSphere Client—not the vSphere Web Client—to perform VUM configuration and VUM-related tasks.

Now let's look at some other applications that are also part of the management layer.

Management Applications

Logging in to each host to perform a management task—such as configuring a network vSwitch or setting the maximum number of NFS mounts you can connect on an ESXi host—can be a tiresome and repetitive task. It may become a nightmare if (and when) your infrastructure grows.

Let's start with the basics. ESXi does have a management console, although its use is generally discouraged in favor of other solutions (which we'll discuss later in this section). Prior to vSphere 5.0, vSphere included both ESX (which offered a Service Console based on a customized Linux kernel) and ESXi (which offers a management console based on a BusyBox environment). With the release of vSphere 5.0, vSphere now only includes ESXi.

To connect these consoles remotely, you need an SSH client. Most administrators prefer using PuTTY (www.chiark.greenend.org.uk/∼sgtatham/putty). Several other tools provide the same functionality, and every administrator has a preference. Administrators using Mac OS X or Linux can also use the preinstalled SSH client that is available via their native terminal application. In vSphere 5.0 and 5.1, you'll also need to enable SSH access to the ESXi management console (it's disabled by default). This is done from the Direct Console User Interface (DCUI).

Management consoles aren't the end of the story, though. ESXi and vCenter have an extensive API you can use to perform remote management. Building on this API, VMware provides several tools to manage vCenter and your ESXi hosts remotely:

  • vSphere command-line interface (vCLI)
  • PowerCLI
  • vSphere Management Assistant (vMA)

Note that we haven't included orchestration tools such as vCenter Orchestrator in this list, nor is vCloud Director included. Although they could be considered management tools, our focus here is on day-to-day management tools at the vSphere level. Note that we do discuss some vCloud Director design considerations in Chapter 12, “vCloud Design.”

The good thing about the different tools is that you don't have to choose between them: you can use them all. For example, both the Perl and PowerShell platforms have dedicated followers who constantly expand the ways you can use these tools in your environment.

Let's start with a look at the vCLI.

vSphere Command-Line Interface (vCLI)

The vCLI command set allows you to run common system-administration commands against ESXi systems from any machine with network access to those systems. You can run most vCLI commands against a vCenter Server instance and target any ESXi system that the vCenter Server instance manages. Because vSphere 5.0 doesn't include ESX, vCLI commands are especially useful for passing commands to ESXi hosts where direct use of the management console is generally discouraged.

vCLI commands run on top of the vSphere SDK for Perl. vCLI and the vSphere SDK for Perl are included in the same installation package.

The supported platforms for vCLI are as follows:

  • Windows Vista Enterprise SP1 (32-bit and 64-bit)
  • Windows 2008 (64-bit)
  • Windows 7 (32-bit and 64-bit)
  • Red Hat Enterprise Linux (RHEL) 5.5 (32-bit and 64-bit)
  • SUSE Linux Enterprise Server (SLES) 10 SP1 (32-bit and 64-bit)
  • SLES 11 and SLES 11 SP1 (32-bit and 64-bit)
  • Ubuntu 10.04 (32-bit and 64-bit)

The commands you run on vCLI for management aren't exactly the same as the commands you can run on the ESXi console (or the commands you might have used in previous versions with the ESX Service Console), so you may have to adjust to the difference. Table 3.1 shows a couple of examples.

Table 3.1 vCLI syntax as compared to console syntax

vCLI ESXi Shell
vicfg-vmknic esxcfg-vmknic
vicfg-nics esxcfg-nics

VMware KB Article 1008194 (http://kb.vmware.com/kb/1008194) gives a list of some of the differences/similarities between vCLI commands and ESX/ESXi console commands.

When you're connecting to a remote host, you must—at minimum—specify the following parameters: server, username, password, and command to perform. Authentication precedence is in the order described in Table 3.2.

Table 3.2 vCLI authentication precedence

Authentication Method Description
Command line. Password (--password), session file (--sessionfile), or configuration file (--config) specified on the command line.
Configuration file. Password specified in a .visdkrc configuration file.
Environment variable. Password specified in an environment variable.
Credential store. Password retrieved from the credential store.
Current account (Active Directory). Current account information used to establish an SSPI connection. Available only on Windows.
Prompt the user for a password. Password isn't echoed to the screen.

Let's consider an example to see how these work. The environment is built as detailed in Table 3.3.

Table 3.3 Example for vSphere environment

vCenter Server vcenter.design.local
Username viadmin
Password a:123456

You need to create a configuration file to define these settings. The file must be accessible while running your vCLI session. A session file for the configuration in Table 3.3 looks like Figure 3.2.

Figure 3.2 This configuration file supplies the necessary parameters for vCLI commands.

3.2

Warning
This file isn't encrypted in any way, so all passwords in the file are stored in plain text. Access to this file should be limited!

You can run your commands against the hosts defined in the configuration file without having to input the credentials each time you connect to a server. Here's an example of using the vCLI on a Windows-based system:

vicfg-nas.pl --config c:usersadministratorvcli.txt --vihost esxi51-01.design.local -a -o storage1.design.local -s /shared NFS_datastore1
vicfg-nas—The command to configure NFS storage (the equivalent in the ESXi shell is esxcfg-nas)
--config—The path to the configuration file containing your stored credentials
--vihost—The name of the specific ESXi host in vCenter to which this command should be directed
-a—Adds a new datastore
-o storage1.design.local—The fully qualified domain name (FQDN; you can also enter an IP) of the storage device to which you're connecting
/shared—The mount point to which you're connecting
NFS_datastore1—The name given to the datastore

PowerCLI

PowerShell is becoming the default scripting language for all Windows applications. Many articles explain why it's easier, better, and simpler to use PowerShell to manage your environment. The good thing is that VMware has made a strategic decision to follow suit and provide a PowerShell management environment for vCenter and ESX: PowerCLI.

PowerCLI has hundreds of cmdlets (pronounced “command-lets”) that deal with practically all the elements of your infrastructure. You can configure and manage ESXi hosts, VMs, the OSes of the VMs—more or less anything. A vibrant community is constantly developing ways you can use PowerCLI to allow administrators to perform their jobs more easily.

For a list of all the cmdlets, see the online cmdlet reference:

http://pubs.vmware.com/vsphere-50/topic/com.vmware.powercli.cmdletref.doc_50/Overview.html


Note
The URL provided for a list of all PowerCLI cmdlets is valid for vSphere 5.0 Update 1, which was the latest version of the documentation as of the writing of this book.

What can you do with PowerCLI? In addition to making your toast in the morning, pretty much anything. Seriously, though, anything that is exposed in the vSphere SDK, you can access (and therefore manipulate).

Just as you create in vCLI a configuration file to store your credentials so you don't have to enter them each time you connect to your vCenter/ESXi host, you can do the same in PowerCLI. This isn't a unique feature of PowerCLI—it's more a PowerShell feature—but because you're using PowerShell, you can use it. Let's see how. First, let's look at the code for saving the credentials:

$vicredential = get-credential
$vicredential.Password | ConvertFrom-SecureString | Out-File c:	empviadmin.txt
$VIcred = New-Object System.Management.Automation.PsCredential “[email protected]”, (Get-Content c:	empviadmin.txt | ConvertTo-SecureString)

Connect-VIServer -Server vcenter.design.local -Credential $VIcred

Get-Credential is a PowerShell cmdlet that lets you store credentials in an object for use without having to expose the contents of the password. Here, you store it in a variable named $vicredential. Next, you export the password from the variable into a text file. Don't worry, unlike in vCLI, the password isn't in plain text; rather, it looks something like this:

01000000d08c9ddf0115d1118c7a00c04fc297eb01000000c39f99b56e206a40a56c6e8e4ebc6ec00000000002000000000003660000c000000010000000395d0ba992b59f39e42e30a30e9c972b0000000004800000a0000000100000008deee87fd1ebd9d74990d6ed44d984d018000000494b8c989a3f55018cd9b7a743450f1d09a214843fda25b4140000005226217e4587317d235557ad8e5177541859094b

That is pretty hard to decipher. You export the password because you'll use this credential more than once—instead of having to insert the password each time, you can create a variable to store it. This is done in line 3: a variable named $VIcred is created with an already known username, but the password is imported from the file you saved. This is what the variable holds:

UserName                      Password
--------                      --------
[email protected]             System.Security.SecureString

Last but not least, you connect to your vCenter Server with the credentials supplied in the variable.

After you've connected, let's use the same example as before with vCLI:

New-Datastore -Nfs -VMHost (get-vmhost) -Name NFS_datastore1 -Path “/shared” -NfsHost $storage1.design.local
New-Datastore--The command to configure storage (the equivalent in the ESXi shell is esxcfg-nas; the equivalent in vMA or vCLI is vicfg-nas)
-NFS--Parameter for the kind of datastore (VMFS/NFS)
-VMHost--The host on which this datastore will be created (in this case, all hosts registered under vCenter)
-Name NFS_datastore1--The name you give to the datastore
-Path “/shared--The mount point to which you're connecting
-NfsHost storage1.design.local--The FQDN (you can also enter an IP) of the storage device to which you're connecting

A Good PowerCLI Reference
For more detailed information on using PowerCLI, we recommend VMware vSphere PowerCLI Reference: Automating vSphere Administration, also published by Sybex.

vMA

vMA is a Linux-based virtual appliance provided by VMware that includes prepackaged software; specifically, it includes a preinstalled version of vCLI, which in turn includes the vSphere SDK for Perl; an authentication component (vi-fastpass) that allows a centralized direct connection to established target servers; and a logging component (vi-logger) that lets you collect logs from ESX/ESXi and vCenter Server systems and store the logs on vMA for analysis. Administrators and developers can use vMA to run scripts and agents to manage both ESX/ESXi and vCenter Server systems.

With the full transition to ESXi only in vSphere 5.0, vMA is much more relevant than it might have been in past versions of vSphere. Although ESXi includes a console shell, users are strongly encouraged to use vMA instead of the ESXi shell. In some cases, customers may prefer to use vCLI installed on their own Linux distribution; in other cases, deploying vMA might be easier. Both options allow administrators and other users access to run management and configuration commands without needing the ESXi shell.

Now, let's shift our focus to how you assemble these components to satisfy the requirements of the design. To help you better understand what's involved in creating the management layer design, we'll first look at key design decisions involved in creating a management layer design.

Examining Key Management Layer Design Decisions

In this section, we'll examine some—but not all—of the key decisions involved in crafting the management layer design for your VMware vSphere implementation. Specifically, we'll examine four key decisions:

  • Virtual or physical vCenter Server?
  • vCenter Server on Windows, or vCenter Server appliance?
  • Local or remote database server?
  • Which OS for vCenter Server?

Let's kick things off by tackling the decision whether to run vCenter Server on a physical system or in a VM.

Virtual or Physical vCenter Server?

There is a long-standing discussion in the VMware world about whether you should install vCenter Server in a VM or on a physical server. In the following section, we'll cover the support for both sides of the argument.

Physical Server

This book is about designing a virtual infrastructure environment, so why are we talking about physical hardware? Well, a certain amount of hardware has to be involved: your ESXi hosts, at a minimum. But why would you consider installing vCenter on a physical server? Here are some possible reasons.

The Chicken and the Egg We're sure you're familiar with the chicken-and-egg dilemma. Some people use this analogy for vCenter as a VM.
Let's say you have a large environment—100 ESXi hosts. For some reason, you have a serious outage. For example, you lose the LUN on which the vCenter VM is stored. For all sorts of reasons, you won't have the VM backup for another 4–6 hours. You may say, “OK, no problem”—until something goes seriously wrong with your environment. A VM starts to not behave due to high CPU RAM usage, so you try to find it in the mass of VMs among your 100 ESXi hosts and 2,000 VMs. Are you thinking “nightmare”? You locate the VM. Lucky you—it was on the third ESXi host you checked. And you want to vMotion the VM, but hey, you have no vCenter to perform the migration.
This is one example of what can happen if your vCenter is a VM. Another could be in a VMware View or vCloud Director environment—you can't deploy any new VMs because your vCenter is down. You can't perform certain actions to restore your environment because you have no vCenter, and you have no vCenter because your environment isn't functioning correctly. As you can see, serious issues may arise if your virtual vCenter Server isn't available.
Separation of Duties Certain organizations insist that because of issues like those just discussed, thou shall not have the management application of your environment running as part the environment itself. This doesn't mean you can't run vCenter as a VM—it can be run on a stand-alone host—but you'll lose a certain number of features, as you'll see shortly.
Amount of Resources vCenter Server can be relatively resource-intensive. If you run vCenter Server 5.0 and a back-end database server in the same VM, you could easily need 4 vCPUs and 8 GB of RAM. In vSphere 5.1, the same applies if you run vCenter Single Sign On, vCenter Inventory Service, vCenter Server, vCenter Web Client, and the back-end database server in the same VM. In fact, the requirements might be even higher. Do you want to use that amount of resources in your infrastructure on a single VM?

Virtual

We've discussed why it could be necessary, or better, for your environment to have vCenter on physical hardware. Now, let's look at the other side of the coin: why you could choose to go for vCenter as a VM. Note that VMware now lists the use of vCenter as a VM as a best practice; however, as we stated in Chapter 1, it's imperative you understand why something is recommended as a best practice:

The Chicken and the Egg With proper planning and some preparation, you can mitigate all the problems that may arise from the earlier scenario. It's all about how to find the issues that may occur and provide a solution to address those issues if or when they happen. Let's take the issue of the LUN going down.
vCenter is an application. The guts of the environment is the database on which the vCenter relies. So in the earlier case, there is no way you should be down for 6 hours. If you separate the vCenter database from the vCenter Server, they're on separate servers. If they're VMs, they should be kept on separate ESXi hosts.
Table 3.4 shows how you can address some of these issues.

Table 3.4 Risk mitigation for a virtual vCenter Server

Potential Risk Mitigation Action
vCenter is lost when the database fails. Separate the SQL instance on a different server to enable the use of high-availability features such as clustering.
Both vCenter and SQL are VMs on the virtual infrastructure. Place them on separate hosts with anti-affinity rules.
Both vCenter and SQL are on the same storage. Place the VMs on different LUNS or different storage devices.
SQL data corruption occurs on the LUN. Plan for database snapshots at regular intervals.
vCenter/SQL suffers a performance hit due to other VMs. Ensure that your vCenter/SQL has guaranteed resources with reservations.
vCenter completely crashes. Install a new VM to replace the failed VM and attach to the database, and you're back in business.
Server Consolidation The whole idea of using a virtual infrastructure is to consolidate your physical servers into VMs running on ESXi hosts. And here you're doing exactly the opposite. With the proper planning, there is no reason not to virtualize any workload. It's also officially a VMware best practice.
Snapshots Suppose you're about to patch your vCenter Server, add a plug-in, or make a configuration change to a .cfg file. One of the greatest advantages you have with vCenter as a VM is the built-in ability to snapshot the VM before making any changes. If something goes drastically wrong, you can always revert to the snapshot you took before the change.
Portability A VM can be replicated to your disaster recovery (DR) site if need be. You can duplicate the vCenter Server easily if you want to create a test environment. You can even keep a cold backup of the vCenter Server in the event of a vCenter crash—you can bring the machine back up running on anything from VMware Player to Server to Workstation to a stand-alone ESXi server. (Of course, this could result in some level of data loss, depending on how old the cold backup is.)
Redundancy As soon as you install vCenter on a vSphere HA-enabled cluster, you're automatically making the vCenter resilient to hardware failure. In order to get the same level of redundancy with physical hardware, you'd need a Microsoft cluster—and that would only be for the SQL database. vCenter isn't a cluster-aware application. Several third-party tools can provide this level of redundancy, but they aren't cheap.
And today, with vSphere Fault Tolerance (FT), you can provide a higher level of redundancy for your vCenter. At the present time, vSphere FT doesn't support more than one vCPU; therefore you can't protect the vCenter this way. Support for multiple vCPUs will be available in the future.

Support for Fault Tolerance with Multiple vCPUs
At both VMworld 2011 and VMworld 2012, VMware demonstrated and discussed the development of a version of vSphere FT that supports multiple vCPUs. However, VMware has not (as of the time of this writing) provided a date for when to expect this feature in a released version of vSphere.

Eating Your Own Dog Food How can you as the virtualization administrator say to all your clients and customers that you have faith in the platform, its capabilities, and its features, when you aren't willing to put one of your critical servers on the infrastructure? We all know that, with the correct planning, the product will perform just as well as a physical server, and you'll get the huge additional benefit of running the server as a VM. If you believe in the product, then you should use it.

vCenter Server on Windows or vCenter Server Appliance?

This question is a relatively new addition to the list of commonly asked management design questions. With vSphere 5.0, VMware introduced the vCenter Server virtual appliance (vCSA), a Linux-based virtual appliance that comes with vCenter Server preinstalled. Prior to the introduction of the vCSA, you had only a single choice: a Windows-based installation of vCenter Server. Now you have to decide: Windows-based vCenter Server or vCSA? Let's look at a few reasons in favor of each option.

Windows-Based vCenter Server

Here are some reasons you might deploy the Windows-based vCenter Server into your design:

The Devil You Know Many customers choose to deploy vCenter Server on Windows simply because it's “the devil you know.” vCenter Server on Windows is the option that has been around for the longest, and it's therefore the option more people are familiar with and comfortable deploying. Although this reasoning might seem a bit arbitrary, we'll remind you that it's important to keep operational considerations such as this in mind when crafting your design. Deploying an option that your staff already knows and understands might be the best option for your environment.
Support for Linked Mode If you need support for Linked Mode instances of vCenter Server, then the Windows-based installation of vCenter Server is the only way to go. No, really—the virtual appliance version doesn't support Linked Mode.
Strong Operational Support for Windows Servers If your environment has strong operational support for Windows servers—meaning you have a solid patching solution in place, good monitoring and management options running, and a proactive team of Windows admins—then deploying the Windows Server-based version of vCenter Server makes more sense than introducing a Linux-based virtual appliance that may or may not integrate as well with your existing operational support systems.

vCenter Server Virtual Appliance

Now that we've discussed some reasons for deploying the Windows Server–based version of vCenter Server, what are some reasons for deploying the vCSA? Here are a few that you might consider:

It's Not Windows The very fact that the vCSA isn't Windows might be enough for some environments. Perhaps you work in a Linux/Unix-heavy environment and therefore don't have strong operational support systems in place for Windows. Perhaps your administrators are more comfortable managing and maintaining Linux-based systems than Windows-based systems. Perhaps your organization just has an ideological opposition to Windows. The vCSA also doesn't require monthly patches (although this doesn't imply that it will never need to be patched), and the vCSA doesn't require a Windows license. For any of these reasons, a Linux-based virtual appliance might make more sense.
Simplified Deployment In smaller environments and organizations, the ease with which an instance of the vCSA can be deployed is a very attractive benefit. Simply deploy the Open Virtualization Format (OVF) package, configure the vApp as it's being deployed, and then hit the web-based management tool afterward.

Local or Remote Database Server?

Here's a topic for endless discussion: “Where do I install the database: on the vCenter Server or on a remote server?” Let's go into the rationale behind both options.

Local

Having your database local means you've installed it on the same OS as your vCenter Server. Here are some of the benefits of having all the components on one server:

Bundled Database For the Windows-based version of vCenter Server, Microsoft SQL Express is bundled with the vCenter Server installation. The software doesn't cost anything, but it isn't suitable for large-scale enterprise environments. On the vCSA side, you can use a bundled version of DB2 (in vSphere 5.0) or PostgreSQL (in vSphere 5.1). This can be suitable for small environments or test environments.
Full Database Installation You can install a full database server on your vCenter Server—be it Oracle or Microsoft SQL—assuming that you're using the Windows-based vCenter Server installation. This provides enterprise-level functions that are suitable for a production environment. Note that you can't install a full-blown instance of a database engine on the vCSA.
Faster Access to the Database Having the data local on the same box is in most cases quicker than accessing the data over the network. You don't have to go over the wire, and you're not dependent on factors such as network congestion.
An All-in-One Box You know where your weaknesses lie. You won't be affected by other applications abusing resources on the network on the shared database server (be it Oracle or SQL or DB2).
Backup and Restore A sound and solid backup infrastructure doesn't come cheap. Having all the components on one server saves you from having to back up multiple servers and track which part of the infrastructure is located in which part of your enterprise.

Remote

As opposed to local, here we're talking about installing vCenter software on one server and connecting to a database that doesn't reside on the same machine. The reasons to choose this option are as follows:

Central Database Server You already have a central location for the databases in your organization. If this is the case, you shouldn't start spreading around additional database servers for each and every application. Your DBA won't like you.
Deploying the vCSA You've chosen to deploy the vCSA, but you need a bit more horsepower than the bundled database can provide. In this case, you'll need to have a separate computer running the database server software.
Separation of Duties between Databases and Applications This is perceived as best practice. Servers have dedicated roles, and these roles should be on separate boxes. vCenter software and VUM software should be installed on one server, and the database on which these applications rely should be on a different machine. This ensures that loss of data on one of the servers doesn't cause lengthy downtime. This separation of duties can also, under the right circumstances, provide greater scalability for the management layer.
Corporate Policies (Separate DBA Team) Your organization has an established database administrator. In most cases, they will probably be more knowledgeable about performance, optimization, and troubleshooting of any issues that arise in the future. So, a database installed on the central server is usually maintained in a better fashion than you could administer it on your own. The Virtual(ization) Infrastructure Admin (VIAdmin) usually gets busy very quickly and doesn't have the time or the resources to acquire the knowledge needed to manage and maintain your database server in addition to all the new duties you inherited with the system.
Fewer Resources Needed for vCenter Server If you're going to join the roles of application and database on the same box, you need larger resources for the vCenter Server. (See the next section, “Local or Remote from a Resource Perspective.”)
Providing Clustered Services (Redundancy) for the Database SQL and Oracle can be clustered. Doing so provides resilience in case of a database server failure. You can't do this if the database resides on the same server as vCenter.

Local or Remote from a Resource Perspective

The major database vendors—Microsoft, Oracle, and IBM—provide recommended resource configurations for optimal performance of their database servers. For example, Microsoft recommends a minimum of 1 GB of RAM and recommends 4 GB for optimal performance for SQL Server. In addition, take into account the resources needed for the vCenter service, web services, and plug-ins—you're looking at another 2–4 GB RAM for the vCenter host. It's pretty likely that in an enterprise environment, you have a properly sized database server that can accommodate the vCenter database without any major issues.

The following article provides more information about the recommended resources for SQL Server 2012:

http://msdn.microsoft.com/en-us/library/ms143506.aspx

This article explains that you need at least 1 GB RAM with at least one 1.4 GHz CPU. But more realistically, the recommended resources are as follows:

  • 1 CPU, 2.0 GHz and up
  • 4 GB RAM

Taking this into account, if you install both the vCenter and the database server on the same box, you need approximately 8 GB RAM and at least 4 CPUs (either a quad core physical machine or a VM with four vCPUs configured). Later in this chapter, we'll discuss some sizing recommendations, and you'll see more information about the resources that would be required for vCenter based on the anticipated size of the environment it will manage.

Local or Remote with Redundancy in Mind

We'll get into this topic a bit later in the chapter. But in short, there's no point in providing a clustered solution for only one database. If you're setting up a cluster to protect the database, you may as well protect more databases at the same time. Thus it's useless to install the database locally.

Which Operating System for vCenter Server?

You don't see this question as much, but it's a valid design consideration. Naturally, vCenter is a server and should be installed on a server OS. But which server OS?

As of vSphere 4.1, the requirement for the Windows-based version of vCenter Server is a 64-bit build of Windows Server. This aligns well with Microsoft's stated OS strategy, which declares that Windows 2008 R2 and all subsequent server OSes will be 64-bit only. Given that you have to go with a 64-bit version of Windows Server, then the consideration of which edition of Windows Server is essentially irrelevant—the 64-bit Standard Edition of Windows Server 2008 and Windows Server 2008 R2 are both capable of addressing up to 32 GB of RAM. (32-bit versions of Windows Server 2008 Standard were limited to 4 GB of RAM, which didn't make it an ideal platform for vCenter Server.)

More information on the memory limits for various versions of Windows can be found here:

http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx

With these considerations in mind, it seems reasonable that selecting Windows Server 2008 R2 Standard Edition should be fine for deploying the Windows-based version of vCenter Server.

If you've chosen to go down the vCSA route, then this question doesn't apply to you because the vCSA comes with Linux preinstalled. At the time of this writing, there was no supported way to roll your own Linux-based vCenter Server installation.

There are many, many more potential management layer design questions, but in the interest of time (and pages!), let's move on to a discussion of actually creating the management layer design.

Creating the Management Layer Design

In this section, we'll focus on creating the design for the management layer. To structure the discussion, we'll use the basic principles of design as described in Chapter 1—availability, manageability, performance, recoverability, and security—as a framework around which we'll organize the information. We'll start with a discussion of availability.

Availability

The first basic principle of design to consider when creating the management layer design is availability. Availability, as we discussed in Chapter 1, encompasses metrics like uptime and, in some cases, performance (if the application is so slow as to be unusable, is it still available?).

There are a couple of ways to provide a level of availability for your vCenter Server:

  • vSphere HA
  • vCenter Server Heartbeat

Before we talk about how to provide availability, you need to understand the implications of not having your vCenter available. Table 3.5 lists the functions that will or won't be affected.

Table 3.5 Functions not available when vCenter fails

images
images

VMware

As you can see from the table, most of the management tasks related to vCenter won't work or will only partially function. Although a number of management functions are impacted, the good news is that not all VMs are affected.

Availability should be divided into two parts: the vCenter application and the database. Let's examine how you can provide availability for vCenter—first, the easiest way.

Providing Availability for vCenter

The first component you'll be protecting is vCenter. You should consider several options; it all depends on what level of redundancy is necessary for your environment and how critical it is to have your vCenter available the entire time.

vSphere HA

By running vCenter as a VM on a vSphere HA-enabled cluster, you automatically improve availability by protecting against hardware failure. This is true for both the Windows version of vCenter Server as well as the vCSA. If your ESXi host fails, then vCenter will automatically restart on another host within a short period of time and reconnect to the database. This is a very acceptable solution in the case of host hardware failure; you shouldn't have any major downtime.

As we mentioned earlier in this chapter, current limitations on vSphere FT (limited to a single vCPU only) prevent its use with vCenter Server, given that vCenter would typically require more than a single vCPU.

If you're running vCenter Server on a physical system, then vSphere HA isn't an option, and you'll have to look at vCenter Server Heartbeat instead.

vCenter Server Heartbeat

vCenter Server Heartbeat is a relatively new product from VMware that's specifically targeted at organizations that can't afford even the slightest downtime or failure of their vCenter Server. The product provides automatic failover and failback of VMware vCenter Server using failure detection and the failover process, making manual recovery and failover process definitions unnecessary. It enables administrators to schedule maintenance windows and maintain availability by initiating a manual switchover to the standby server. You can find vCenter Server Heartbeat at

www.vmware.com/products/vcenter-server-heartbeat/

The product also provides protection for the vCenter database, even if the database is installed on a separate server. The technology is based on a product from Neverfail (www.neverfailgroup.com). Figure 3.3 shows the product design.

Figure 3.3 This diagram provides a conceptual overview of vCenter Server Heartbeat.

3.3

The great thing about vCenter Server Heartbeat is that it can be placed across the WAN. If you have two datacenters (US and Europe), you can place each node in a different location to provide potentially greater levels of availability.

You'll need to address certain prerequisites and limitations:

  • vCenter Server Heartbeat can't be installed on a domain controller, global catalog, or DNS. Of course, vCenter Server can't be on a domain controller, either.
  • It only protects Microsoft SQL Server applications.
  • No other critical business applications should be installed on the SQL Server besides the vCenter database.
  • Both primary and secondary servers must have identical system date, time, and time zone settings.
  • Latency on the link between the two locations must not fall below the standard defined for a T1 connection.

You can install the configuration in three ways:

  • Virtual to virtual (V2V)
  • Physical to virtual (P2V). If you're setting up a P2V configuration, then
    • The systems should have similar CPUs.
    • The systems should have identical amounts of memory.
    • The secondary VM must have sufficient priority in resource-management settings to assure the performance of the VM.
    • Each virtual NIC must use a separate virtual switch.
  • Physical to physical (P2P). In a P2P setting, then
    • The primary server must meet certain hardware and software requirements.
    • The secondary should be equivalent to the primary server to ensure adequate performance.
    • Advanced Configuration and Power Interface (ACPI) compliance must match the primary server.

We won't go into the details of how to install, because this isn't a step-by-step book (if you haven't noticed by now).

At this point we do need to point out a few things. First, the product is relatively new in comparison to the rest of the vSphere product suite. We haven't met anyone who has implemented it in their environment, because it's targeted at the highest percentile of customers who can't—at any cost—have their vCenter down. It also isn't a cheap solution: it starts at approximately $12,000 per license. Finally, vCenter Server Heartbeat is only supported for the Windows-based version of vCenter Server, not the vCSA.

Also, judging from the community activity (or lack thereof), vCenter Server Heartbeat hasn't been implemented widely. Or perhaps it works like a charm, so no one has any issues with it. See

http://communities.vmware.com/community/vmtn/mgmt/heartbeat

Now, on to the second component: the database.

Providing Availability for the SQL/Oracle Database

There are several ways to protect the database for your vCenter Server. Database protection is important because the database is the heart and soul of your virtual environment. All the settings (vSphere DRS, vSphere HA, permissions—basically, everything) are stored in the database.

vSphere HA

The vCenter Server can be protected with vSphere HA, and the same goes for your database server. But the chance of data corruption is much higher with a database server. If the server crashes during a write operation, the data can become corrupt, and your database may be rendered unusable. In this case, a single database on a vSphere HA-enabled cluster might cause you problems.

Microsoft Cluster/Oracle Cluster

Both Microsoft and Oracle provide their own solution for active-active and active-passive clusters for databases. The great thing is that you can combine these two options: vSphere HA and Microsoft/Oracle Cluster.

You can create a highly available database on vSphere. VMware has best practices for both platforms:

Here are a few considerations and best practices you should take into account when virtualizing these applications, distilled from the documents just referenced:

Upgrade to the Latest Version of vSphere By upgrading to new versions of vSphere, you can—in some instances—gain a 10–20% performance boost on the same hardware.
Create a Computing Environment Optimized for vSphere You should set the BIOS settings for ESXi hosts accordingly. These recommendations include the following:
  • Enable virtualization technology to support running 64-bit guest OSes.
  • Enable Turbo Mode to balance the workload over unused cores.
  • Enable node interleaving if your system supports non-uniform memory architecture (NUMA).
  • Enable VT-x, AMD-V, EPT, and RVI to use hardware-based virtualization support.
  • Disable C1E Halt State to prefer performance over power saving.
  • Disable power-saving to prevent power-saving features from slowing down processor speeds when idle.
  • Enable HyperThreading (HT) with most new processors that support this feature.
  • Enable Wake On LAN to support the required features for the Distributed Power Management feature.
  • Set Execute Disable to Yes for the vMotion and Distributed Resource Scheduler features.
Optimize Your OS Remove nonvital software, and activate only required services.
Use as Few vCPUs as Possible One of the biggest mistakes administrators make is overallocating resources to a VM. When you allocate four vCPUs to a VM that barely uses one vCPU, not only will the performance of the VM be degraded, but you can also seriously impact the rest of the VMs running in the same host.
Enable HT for Intel Core i7 Processors This is a new recommendation for the Xeon 5500 family of processors. Until now, VMware had no recommendation about enabling HT.
Allow vSphere to Choose the Best Virtual Machine Monitor Based on the CPU and Guest OS Combination Workloads like those from a database that has a large amount of page-table activity are likely to benefit from hardware assistance. But it's still best to leave the BIOS settings as recommended above.
Use the VMXNET Family of Paravirtualized Network Adapters The paravirtualized network adapters in the VMXNET family implement an idealized network interface that passes network traffic between the VM and the physical network interface cards with minimal overhead.
Enable Jumbo Frames for IP-Based Storage iSCSI and NFS This will reduce load on the network switches and give a certain percentage of performance boost. Note that this feature must be enabled end-to-end (VMkernel, vSwitch, physical switch, and storage) for it to work properly.
Create Dedicated Datastores to Service Database Workloads It's a generally accepted best practice to create a dedicated datastore if the application has a demanding I/O profile, and databases fall into this category. The creation of dedicated datastores lets you define individual service-level guarantees for different applications and is analogous to provisioning dedicated LUNs in the physical world.
It's important to understand that a datastore is an abstraction of the storage tier and, therefore, is a logical representation of the storage tier, not a physical representation of the storage tier. So, if you create a dedicated datastore to isolate a particular I/O workload (whether log or database files) without isolating the physical storage layer as well, you won't get the desired effect on performance.
Make Sure VMFS Is Properly Aligned We'll get into disk alignment in more detail in Chapter 6, “Storage,” and Chapter 7, “Virtual Machines.” But note that the performance hit of not aligning your VMFS/NFS datastores can be significant.

vCenter Server Heartbeat

As noted earlier, this product provides redundancy for an SQL database only. That database must be the only one on the server in order for this to work.

Availability is obviously a key principle to keep in mind when creating the management layer design. The second principle we're going to discuss is manageability, and that's the topic of the next section.

Manageability

In Chapter 1, we stated that the principle of manageability includes such concepts as compatibility, usability, interoperability, and scalability. Some of these concepts are outside of your control. For example, you can't control the usability of most portions of the management layer, because VMware drives that. If your users find the vSphere Web Client to be unintuitive or the syntax of the vCLI to be unintelligible, there's very little you can do to alter or address those complaints. Other aspects of manageability, however, are under your control:

Compatibility and Interoperability If the environment requires that vCenter Server interoperates with other management components, then you might need to add third-party components to bridge any compatibility or interoperability concerns. Integration with a product like Microsoft Systems Center Operations Manager (SCOM) is one such example; third-party products like Veeam Management Pack for VMware provide the required interoperability and compatibility between vCenter Server and SCOM. Similar solutions exist to provide greater compatibility and interoperability between vCenter Server and storage, networking, and server hardware. Where dictated by the design factors, these additional components need to be factored into the management layer design.
Scalability Another aspect of manageability is scalability. If the management layer can't scale to handle the anticipated growth of the environment, the environment will become unmanageable. This aspect of manageability is closely related to performance, and so we'll address it later in this section under “Performance.”

Scalability is an aspect that can be addressed in a couple of different ways. One of these ways, Linked Mode, is a solution that will have a number of different design impacts and therefore deserves a bit more consideration.

Linked Mode

In this section, we'll go into detail about vCenter Linked Mode as one potential method of providing scalability to the management layer design. VMware added this feature in the release of vSphere 4. It solves the issue of very large environments that need multiple vCenters to manage the infrastructure due to the sheer size and number of hosts and VMs.

The limit on the number of ESXi hosts that one vCenter can manage is 1,000; the limit for powered-on VMs is 10,000 per vCenter (15,000 total). For many organizations, this is more than sufficient—they can only dream of reaching that number of VMs. But for others, this limit is too small.

In comes Linked Mode. By joining your vCenter instances together, you can expand your infrastructure to 10 vCenter Servers, 3,000 hosts, and 30,000 powered-on VMs (50,000 total).


Note
You need to be aware of a licensing caveat here. Linked Mode can only be used with the Standard Edition of vCenter Server. If you've purchased an Essentials or Foundation Bundle (which doesn't have a Standard vCenter license), you can't join the vCenter instances together in Linked Mode.

Prerequisites

To use Linked Mode for additional scalability to the management layer design, keep in mind the following requirements. These requirements apply to each vCenter Server system that will be a member of a Linked Mode group:

  • DNS infrastructure must be operational for Linked Mode replication to work. Each member of the Linked Mode group must be able to resolve the names of the other members in the group.
  • The vCenter Server instances in a Linked Mode group can be in different domains as long as the domains have a two-way trust relationship.
  • When adding a vCenter Server instance to a Linked Mode group, the installer must be run by a domain user who has administrator credentials on both the machine where vCenter Server is installed and the target machine of the Linked Mode group.
  • All vCenter Server instances must have network time synchronization. The vCenter Server installer validates that the machine clocks aren't more than 5 minutes apart. Within a single Active Directory domain, this is normally handled automatically. (Keep in mind that Linked Mode is only available for the Windows versions of vCenter Server, so naturally Active Directory is almost always involved.)

Design Considerations

You should take into account several considerations before you configure a Linked Mode group:

  • Each vCenter Server user sees the vCenter Server instances on which they have valid permissions. If you wish to block certain parts of your environment from other users, you need to specifically deny permissions on that section.
  • When you're first setting up a vCenter Server Linked Mode group, the first vCenter Server must be installed as a stand-alone instance because you don't yet have a remote vCenter Server machine to join. Any vCenter Server instances thereafter can join the first vCenter Server or other vCenter Server instances that have joined the Linked Mode group.
  • If you're joining a vCenter Server to a stand-alone instance that isn't part of a domain, you must add the stand-alone instance to a domain and add a domain user as an administrator. Linked Mode isn't supported in workgroup environments—but then again, why would you use Linked Mode in a workgroup?
  • The vCenter Server instances in a Linked Mode group don't need to have the same domain user login.
  • The vCenter service can run under different accounts. By default, it runs as the machine's LocalSystem account.
  • You can't join a Linked Mode group during the upgrade procedure when you're upgrading from VirtualCenter 2.5 to vCenter Server 4.1. Only after you've completed the upgrade can you join Linked Mode.
  • Linked Mode groups can only contain vCenter instances running the same version of vCenter Server. If you have vCenter 5.0 and want to add vCenter 5.1, you'll need to upgrade to vCenter 5.1 on all systems before you can create a Linked Mode group.

With these considerations in mind, let's take a peek under the covers of vCenter Server's Linked Mode groups.

Under the Covers

How does Linked Mode work? VMware uses Active Directory Lightweight Directory Services (AD LDS, previously known as Active Directory Application Mode [ADAM]).

Instead of using your organization's Active Directory database to store the directory-enabled application data, you can use AD LDS to store the data. AD LDS can be used in conjunction with AD DS so you can have a central location for security accounts (AD DS) and another location to support the application configuration and directory data (AD LDS). By using AD LDS, you can reduce the overhead associated with Active Directory replication, you don't have to extend the Active Directory schema to support the application, and you can partition the directory structure so the AD LDS service is deployed only to the servers that need to support the directory-enabled application.

Linked Mode behaves like an Active Directory application. Let's connect to it and see what information is inside. Figure 3.4 shows an example of using ldp.exe (http://support.microsoft.com/kb/224543) to connect to a vCenter Server on port 389 (unless you've changed the default port).

Figure 3.4 LDP shows the data found in a vCenter LDS instance.

3.4

It's easier to use Active Directory Services Interfaces (ADSI) Edit to see all the properties. Connect to your vCenter Server with the correct credentials, using the naming context DC=virtualcenter,DC=vmware,DC=int as shown in Figure 3.5 and Figure 3.6.

You can connect to three naming contexts:

  • DC=virtualcenter,DC=vmware,DC=int
  • CN=Schema,CN=Configuration,CN={382444B2-7267-4593-9735-42AE0E2C4530} (the GUID is unique to each vCenter installation)
  • CN=Configuration,CN={382444B2-7267-4593-9735-42AE0E2C4530}

Figure 3.5 You can also use ADSI Edit to connect to a vCenter LDS instance.

3.5

Figure 3.6 ADSI Edit shows the different vCenter LDS naming contexts.

3.6

It's a good idea to get acquainted with the structure of the schema and the configuration of the LDS instance. They aren't documented well, and the information may come in handy one day.

Considerations for vCenter Roles in Linked Mode

You need to know several things about roles when joining two or more servers in Linked Mode:

  • The roles defined on each vCenter Server system are replicated to all the other vCenter Servers connected in the Linked Mode group.
  • If the roles defined on each vCenter Server system are different, the role lists of the systems are combined into a single common list. For example, if vCenter Server 1 has a role named ESX_Admins and vCenter Server 2 has a role named Admins_ESX, then both servers will have both ESX_Admins and Admins_ESX after they're joined in a Linked Mode group.
  • If two vCenter Server systems have roles with the same name, the roles are combined into a single role if they contain the same privileges on each vCenter Server system. If the role exists on two vCenter Servers but they're assigned different privileges on the two servers, this conflict must be resolved by renaming at least one of the roles. You can choose to resolve the conflicting roles either automatically or manually.
  • If you choose to reconcile the roles automatically, the role on the joining system is renamed to <vcenter_name> <role_name>, where <vcenter_name> is the name of the vCenter Server system that is joining the Linked Mode group and <role_name> is the name of the original role.
  • If you choose to reconcile the roles manually, you have to connect to one of the vCenter Server systems with the vSphere Client and rename one instance of the role before proceeding to join the vCenter Server system to the Linked Mode group.
  • If you remove a vCenter Server system from a Linked Mode group, the vCenter Server system retains all the roles it had as part of the group.

Linked Mode offers a great way to improve the manageability of a large vSphere environment by both enabling greater scalability as well as aggregating information from multiple vCenter Server instances. In fact, prior to vSphere 5.1, the only way to see information from multiple vCenter Server instances at the same time from the vSphere Client was using Linked Mode.

With the release of vSphere 5.1, though, architectural changes in vCenter Server—specifically, splitting the vCenter Inventory Service into a separate component that can service multiple vCenter Server instances—and the introduction of the new vSphere Web Client now allow environments running vSphere 5.1 to aggregate information from multiple vCenter Server instances in the next-generation Web Client without having to use Linked Mode groups.

Let's move on to the third principle of design, which is performance.

Performance

Invariably, most aspects of performance come down to sizing the components properly. So, as you consider the principle of performance when creating the management layer design, sizing the components properly will be a primary concern. With that in mind, let's discuss how you size the various management components.

Sizing vCenter Server

You need to consider the following elements while sizing your vCenter Server:

  • OS
  • Database placement
  • Number of objects managed
  • VUM

Let's look at each of these elements.

Operating System

As we described earlier in the section “Which Operating System for vCenter Server?” vCenter Server (if you're using the Windows version) must be installed on a 64-bit version of Windows. To avoid potential memory constraints, our recommendation is to use Windows Server 2008 R2.

To recap the reasons behind this recommendation, recall that one limitation of a Windows 32-bit OS is that it can only natively support 4 GB of RAM. Many vCenter Server instances were installed in the past with Enterprise Edition OSes to overcome that boundary. With a Standard license of Windows Server 2008 R2, you're limited to 32 GB of RAM, which should be more than sufficient for the vast majority of vCenter deployments. In addition, licensing for the higher versions of both the Windows OS and SQL Server is substantially more expensive than a standard license.

Database Placement

As we explained earlier in the section “Local or Remote Database Server?” the placement of the database for vCenter Server (and related components like vSphere Update Manager) has a significant impact on the resources that must be allocated to vCenter Server. In that section, we stated that the recommendations for Microsoft SQL Server, for example, are 4 GB of RAM and a 2.0 GHz or higher CPU. When you add these requirements to the requirements described next based on the number of managed objects, you can see that accounting for database placement is critical. If you're going to run the database local to vCenter Server, be sure to allocate appropriate resources. For maximum performance, we recommend splitting the database server onto a separate system.

Number of Objects Managed

VMware's documentation for installing and setting up vCenter provides the numbers in Table 3.6 as recommendations for optimal performance.

Table 3.6 Recommendations for optimal performance

images

http://pubs.vmware.com/vsp40/install/wwhelp/wwhimpl/common/html/wwhelp.htm#href=c_vc_hw.html&single=true

You can clearly see that at most, you need four CPUs and 8 GB RAM. The sizing of your vCenter will depend on how big a deployment you're planning.

Update Manager

Will you be using your vCenter Server as a VUM server as well? Why does this make a difference?

In terms of CPU or RAM resources, you won't need additional resources if you don't have the SQL Server running on the vCenter Server. What you do need is disk space. VMware provides a tool that assists in sizing your installation: the VMware vCenter Update Manager Sizing Estimator, found at this URL:

www.vmware.com/support/vsphere4/doc/vsp_vum_41_sizing_estimator.xls

At the time this book was written, the VUM 4.1 document listed here was the latest version available, and it should be applicable to newer versions of VUM as well. Note that there are also VUM patch-management guidelines included in VMware's “Configuration Maximums” documentation for each vSphere release.

The vCenter Update Manager Sizing Estimator is an Excel spreadsheet that requests information about your environment:

  • Version of hosts you'll be patching (3.0x/3.5x/4.x)
  • Number of concurrent upgrades
  • Number of hosts
  • Number of VMs
  • OS locale
  • Service pack levels
  • Frequency of scans for hosts, VMs, and VMware tools

The results you get from the spreadsheet will look like Table 3.7.

Table 3.7 Results from Update Manager Sizing Estimator

images

The biggest resource you need for VUM is disk space. Because of all the patches you'll download, the required disk space will increase depending on how many versions of the software you're downloading for (either version of ESX, and the different OS service packs).

It's recommended that you not install the patch repository in the default location provided during the installation (C:Documents and SettingsAll UsersApplication DataVMwareVMware Update ManagerData). Most administrators don't notice this question during the installation; then, somewhere along the way, they begin to run out of space on the C: drive of their vCenter Server, and they wonder why. It's better to allocate a separate partition and folder for the downloaded updates, for these reasons:

Backup You don't always want to back up patches that are downloaded. The content hardly changes, and it can easily be downloaded again if needed.
Not Enough Space on the System Drive If you aren't careful, your system drive will fill up.

In addition to sizing vCenter Server itself, it's also critical to properly size the database, as we describe in the next section.

Database Sizing for vCenter and Update Manager

When you follow the previous recommendations and decide to use a database server that isn't on the same system as vCenter Server, you next have to plan the size of the database. Before we get into sizing, let's review the purpose of the database in a vCenter installation.

The database is the central repository of the logic and structure of your virtual infrastructure: resource pools, permissions on each item in vCenter, alarms, thresholds, the cluster structure, distributed resource scheduling (DRS), and, of course, the biggest consumer—statistics. All the statistics for every object and counter in your environment are stored in the database, including CPU, RAM, disk, network, and uptime. And each category has multiple counters associated with it.

VMware also provides the vCenter Server 4.x Database Sizing Calculator for Microsoft SQL Server (this was the latest version available at the time of writing):

www.vmware.com/support/vsphere4/doc/vsp_4x_db_calculator.xls

It's very similar to the calculator mentioned earlier for VUM. But in this case, a larger number of parameters are required in addition to those we've already covered:

  • Number of NICs per host
  • Number of NICs per VM
  • Number of datastores per host
  • Number of VMDKs per VM
  • Number of physical CPUs
  • Number of virtual CPUs

And most important, how long will you keep each level of statistics? You configure this setting in vCenter as shown in Figure 3.7.

Figure 3.7 The vCenter statistics configuration has a profound impact on database size.

3.7

The higher the level of statistics collected for each interval, the larger your database will become. VMware's report “VMware vCenter 4.0 Database Performance for Microsoft SQL Server 2008” includes the recommendations in Table 3.8.

Table 3.8 VMware recommendations for database performance with Microsoft SQL Server 2008

Statistics Interval Statistics Level
Past day Level 2
Past week Level 2
Past month Level 1
Past year Level 1

The document also provides a few additional recommendations, most of which we've already discussed or are general rules of thumb. You need to allocate enough RAM for your database server. One guideline for all relational databases is to allocate the server as much memory as necessary to allow for the caching of all the needed data in memory. You should refer to the documentation for the amount of memory to give to the OS and any other applications that run on the server you use for your database. You should go with a 64-bit server for the same reason we mentioned earlier regarding the 4 GB RAM memory limitation on Windows 32-bit OSes.

Another area to consider when performing sizing calculations for vCenter Server is the database recovery model. For example, with Microsoft SQL Server the default recovery model is Full. This means the database logs will grow endlessly if no backup is performed. If your organization doesn't have a DBA on staff to manage backups of the SQL Server database, then we recommend changing this to the Simple recovery model. If you still prefer the Full recovery model, be sure to account for the additional disk space that is required to store the logs between database backups.

Plug-ins

Plug-ins are great. They allow you to perform all the tasks you need in one management console without having to use a multitude of tools to manage your environment. As an example, most of the major storage vendors (NetApp, EMC, Dell, HP, and IBM) already have plug-ins that let you configure the virtualization portions of the storage array. Using these plug-ins, you can create new LUNs/datastores, view statistics of the VMs in correlation to the storage back-end, optimize the ESXi hosts according to vendor best practices, and more.

Unfortunately, with the advantages come some disadvantages that you didn't plan for—especially the additional resources required to run the plug-ins. You need to take into account the following resources:

  • Disk space
  • Memory

You must also consider whether these plug-ins are client-side plug-ins (designed to work with the Windows-based vSphere Client) or server-side plug-ins (designed to work with the next-generation vSphere Web Client). As you can probably surmise, client-side plug-ins have a resource impact on the clients where the plug-ins are installed, and server-side plug-ins have a resource impact on the vCenter Server or the vCenter Web Client server. Because the resource requirements vary greatly among plug-ins, we can't provide any specific recommendations here other than to consult with the plug-in provider to get complete details on the resource impact, configuration impact, and operational impact of the plug-ins.

Hardware Resources

All this talk of sizing and estimators is great, but ultimately it comes down to hardware resources (physical or virtual). According to the VMware documentation, the minimum resources required for vCenter are as follows:

  • 2 CPUs (physical/virtual cores). A processor is defined as a 2.0 GHz or faster Intel or AMD processor. The requirements may be higher if the database runs on the same machine.
  • 3 GB RAM. This requirement may be higher if the database runs on the same machine. VMware VirtualCenter Management Webservices requires from 128 MB to 1.5 GB of additional memory. The VirtualCenter Management Webservices process allocates the required memory at startup. The engine that drives these web services is Tomcat JVM.
  • 2 GB hard disk space. The requirements may be higher if the database runs on the same machine and/or if you host the Update Manager database and store the updates on that machine. While installing vCenter, you'll need up to 2 GB in which to decompress the Microsoft SQL Server 2005 Express installation archive. Approximately 1.5 GB of these files are deleted after the installation is complete.
  • A gigabit network connection is recommended.

These minimum requirements are fine for a test environment and for kicking the tires. But if you already know that you'll be deploying a large number of hosts and VMs, you'll need more than the minimums. In fact, we've already shared with you some recommendations based on a few key factors: OS, database placement, number of objects managed, and the presence (or absence) of VUM. In selecting or allocating hardware resources for vCenter Server, you shouldn't plan for the immediate need, but rather for the expected or required expansion.

We've now considered three of the five principles of design: availability, manageability, and performance. Our next principle is recoverability.

Recoverability

When you design the management layer with recoverability in mind, you're designing with failure in mind. How quickly will you be able to recover a failed vCenter Server instance? How quickly will you be able to recover from a failed vCenter database? As we described in Chapter 1, recoverability describes several different ideas, including well-known metrics like Recovery Time Objective (RTO) and Recovery Point Objective (RPO) in addition to other measurements like Mean Time To Repair (MTTR). Concepts of disaster recovery and business continuity (DR/BC) are also included in this principle of design.

How does all this apply to the design of the management layer? As with all areas of vSphere design, the ideas of this principle are closely related to and heavily intertwined with the ideas in the other principles of design:

  • Using vSphere HA or vCenter Server Heartbeat, as described when discussing availability, also has a direct impact on recoverability.
  • Splitting the database onto a separate server, as described and recommended in the section on the principle of performance, can also help with MTTR by isolating different components and limiting the fault domain(s).

Backups are a key part of ensuring the recoverability of the management layer, and you should be sure that you're properly incorporating backup of the management layer into the design. You'll need to account for backups of all the various management layer components—vCenter Server, vSphere Update Manager (if present), other management applications, and the appropriate database servers—when crafting the backup strategy for the management layer.

Finally, complete and comprehensive documentation is key to recoverability. A measure of recoverability is not only the time required to recover from a failure or other event, but also the amount of effort required to recover. A well-crafted set of documentation that is both comprehensive and up to date can greatly reduce the amount of time required to recover the environment.

The fifth and final principle of design that you need to apply to the creation of the management layer design is security, and it's the focus of the next section.

Security

Perhaps this part of the chapter should have come first, but its current position has no reflection on its importance in the design process. Nine out of ten administrators would rank security as one of the most important considerations in virtual infrastructure design.

How can you design your vCenter Server for security? Here are three considerations you should keep in mind:

  • Isolation
  • Permissions
  • SSL certificates

Let's look at each of these considerations in a bit more detail.

Isolation

Your vCenter Server, from a security perspective, is probably the most important component of your infrastructure. If someone compromises the vCenter Server, they can cause immense damage, from powering off VMs, to deleting data from a VM, to deleting a VM or—worse—a complete datastore.

Your vCenter Server shouldn't be accessible from all workstations in your corporate network. You can achieve this by using any of the following measures:

  • Put the vCenter on a separate management VLAN.
  • Put the vCenter behind a firewall.
  • Define an access list on the network switches.
  • Set firewall rules on the vCenter Server.

Permissions

By default, the Administrators Security group on a vCenter Server has full permissions to the entire environment. Do you always want the administrators on the vCenter to control every VM? This isn't always the best idea. Limiting the default Administrators group's access to the full vCenter can be achieved by doing the following:

1. Create a local security group on the vCenter Server (vi-admins).
2. Create a local user on the vCenter Server (viadmin-local). Then, if for some reason your domain account isn't available or the vCenter can't contact the domain, at least you'll have a way to control your environment.
3. Create a domain user (viadmin).
4. Add both the viadmin and viadmin-local accounts to the vi-admins local group on the vCenter Server.
5. Change the Administrator permissions from the default to the vi-admins group.

In addition, you should always follow the model of least-privilege permissions. Only give a user account the minimum required permissions needed to perform a task. There is no need to give a user the Administrator role if all they need is to access the VM console. It's a better idea to create a custom role that contains only the relevant permissions needed for the task and then assign that permission to that user.

One excellent example is VM access. Consider this: would you give any user who asked you the code for the server room so they could power on a server if they wanted to? That wouldn't be wise.

You should view your vCenter Server as a secure area of your datacenter. Not everyone should have access to the vSphere infrastructure. If you want to give users access to the server, they should access it through either Remote Desktop or an SSH session (depending on the OS).

It isn't recommended that you enable a VNC client on a VM; this requires additional configuration on each and every ESXi to allow for such remote management. In such a case, you're better off with a custom role.

SSL Certificates

Client sessions with vCenter Server may be initiated from any vSphere API client, such as vSphere Client and PowerCLI. By default, all traffic is protected by SSL encryption, but the default certificates aren't signed by a trusted certificate authority and therefore don't provide the authentication security you may need in a production environment. These self-signed certificates are vulnerable to man-in-the-middle attacks, and clients receive a warning when they connect to the vCenter Server.

If you intend to use encrypted remote connections externally, consider purchasing a certificate from a trusted certificate authority, or use your own PKI infrastructure in your domain to provide a valid certificate for your SSL connections.

You can locate the official “Replacing vCenter Server Certificates” guidelines here (this was the latest version available at the time of writing):

www.vmware.com/files/pdf/vsp_4_vcserver_certificates.pdf

Summary

The factors involved in the design of your vCenter Server shouldn't be taken lightly. You'll need to design the management layer and account for all five of the design principles: availability, manageability, performance, recoverability, and, last but not least, security.

Take into account how many hosts and VMs you have and what kind of statistics you'll need to maintain for your environment.

Size your server correctly from the ground up, so you won't need to redeploy when you outgrow your environment.

Remember that your vCenter Server should be separated from your database server, providing a separation of duties for the different components of the infrastructure. You should plan for redundancy for both components and take into account what kind of outage you can afford to sustain. When you have that information, you can plan the level of redundancy you need.

Don't be afraid to run your vCenter as a VM. In certain cases, you can provide a greater level of resilience, with more ease, than you can with a physical server.

Your vCenter is the key to your kingdom. Leaving it exposed places your kingdom out in the open. A great number of attacks today are carried out from within the network, for a number of reasons: disgruntled employees, malicious software, and so on. Protect that key with the methods we've discussed.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.224.59.145