The VMkernel represents the heart of ESXi, and it is the component of ESXi that provides a virtual interface to the virtual machines that run on your hosts. The VMkernel manages interaction with the host’s hardware devices, allocates memory to different processes, and schedules central processing unit (CPU) access. The VMkernel was designed for no other purpose than to run virtual machines, and the API that it exposes is limited to that purpose.

The VMkernel includes the following features to provide a secure environment for virtual machines: kernel module integrity, memory hardening, and the Trusted Platform Module (TPM).

Kernel module integrity consists of digital and module signing. Digital signing is used to ensure the authenticity of integrity of applications, drivers, and modules as they are loaded by the VMkernel. Module signing enables ESXi to identify the developers of applications, drivers, and modules and to verify whether VMware has approved those components.

The purpose of memory hardening is to use memory techniques and modern processor capabilities to protect the system from common buffer overflow attacks that can be used to exploit weaknesses in running code. ESXi uses Address Space Layout Randomization (ASLR) to load the VMkernel, user-mode applications, and executable components such as drivers and libraries at random and unpredictable memory locations. ESXi also takes advantage of Intel XD (eXecute Disable) and AMD NX (No eXecute) support to mark writable areas of memory as nonexecutable.

TPM provides platform verification during the boot process as well as cryptographic key storage and protection. TPM requires that the server have a special hardware chip installed that contains a unique Remote Supervisor Adapter (RSA) key. Ensure that TPM is properly enabled in the Basic Input/Output System (BIOS) of the host. During booting, ESXi measures the VMkernel using the TPM and stores the value in one of its platform configuration registers. TPM measurements are propagated to vCenter Server when the ESXi host has been added to vCenter, and the measurements can be obtained by third-party agents using the vCenter Server application programming interface (API). In addition to enabling TPM in the BIOS, enable the Misc.enableTboot option shown in Figure 7.1.

Figure 7.1. Enabling TPM on your ESXi host with the Misc.enableTboot setting.

Virtual machines are software containers that encapsulate the necessary components for a guest operating system (OS) to execute. When a guest OS is being installed, it sees a CPU, memory, the network card, and the disk in the same way that it would on a physical host. But these resources are virtualized and thus the guest operating system does not have direct control of the hardware resources in the ESXi host. This design effectively isolates virtual machines from one another. In the case of an OS running on a physical server, an OS failure is isolated to a server’s hardware and does not impact other hosts on the network directly. With a virtual machine, it is critical that a failure in the OS be limited to just the virtual machine and that the failure have no impact on the ESXi host or other virtual machines running on that host. The security design of ESXi virtual machines provides this protection.

This protection also ensures that one virtual machine cannot arbitrarily breach the security of another virtual machine or the host. A user operating in one virtual machine will not be able to take advantage of the fact that other operating systems are running virtual on the same host to circumvent their security unless the user has been granted specific privileges on the ESXi host. All access to hardware on the host is made through the VMkernel, which ensures that the environments are kept isolated and that one virtual machine will not be able to control or circumvent processes within another virtual machine.

Virtual machines require the ability to communicate with other virtual machines and physical hosts, and this is done through a virtual network interface card (NIC). A virtual network card (vmnic) is connected to a virtual switch (vSwitch), which can then provide access to your network when the vSwitch is linked to physical adapters in your ESXi host. A virtual machine with a vmnic is not able to bypass the security of your vSwitch design. If the vSwitch to which a virtual machine is connected has not been linked to your network, the virtual machine will effectively be isolated and only able to communicate with other virtual machines on the same vSwitch. Likewise, two virtual machines on different vSwitches on the same host will not be able to communicate with each other unless both vSwitches have been configured to be linked to your physical network through the network adapters in your ESXi host.

To protect virtual machines further, you should consider the use of limits, shares, and resource reservations. This topic was discussed briefly in Chapter 3, “Management Tools.” By default, ESXi imposes a form of resource reservation to ensure that the resources of the host are divided equally among the virtual machines running on the host. If a virtual machine on the host were to be compromised by a denial-of-service (DoS) attack, the other virtual machines would be partially isolated from the resource drain caused by the DoS. You could further limit the impact of a problem virtual machine by setting specific reservation and limits. With a reservation, you ensure that a virtual machine has a guaranteed quantity of a specific resource and you could also use a limit to ensure that a virtual machine would not consume excessive hardware resources. For low-resource virtual machines that are accessible to the Internet, you may consider lowering the CPU shares or setting a CPU limit for the virtual machines. In the case of a compromise, the virtual machine would have limited CPU resources to attack other hosts and consume fewer CPU resources on the host.

The virtual networking layer includes both the vmnics used in virtual machines and vSwitches. The networking layer provides network connectivity between virtual machines and other hosts on your network. The networking layer is not just limited to providing network service to virtual machines; ESXi uses it for management, communication with other ESXi hosts and vCenter Server, Internet Small Computer System Interface (iSCSI), and network attached storage (NAS).

With your implementation of ESXi, you need to spend some time carefully planning your network design to ensure that virtual machines are properly placed in the correct vSwitches and to ensure that virtual machines and other ESXi network components are properly isolated and protected from each other. ESXi supports 802.1q virtual local area networks (VLANs), which you can use to protect your virtual machines further and to isolate virtual machines and other ESXi network services even within the same vSwitch.

With the isolation that the virtual networking layer can provide, you can securely deploy ESXi to provide access to virtual machines that need to be placed into a demilitarized zone (DMZ) without compromising the security for your other virtual machines and, as important, without exposing management access to ESXi within the DMZ. DMZ virtual machines are typically exposed to the Internet and represent the virtual machines in your environment that are most open to attack by intruders. If you consider the default installation of ESXi, a single vSwitch is created with a virtual machine port group and a management port. If you want to place virtual machines within your DMZ, you don’t want to unplug the network port linked to the management vSwitch and connect that to the DMZ. Rather, create a new vSwitch with a second network port on the host, which would be connected to the physical switch of your DMZ. On this second vSwitch, add a new virtual machine port group for the DMZ. The result would be two isolated vSwitches, and virtual machines on both would not be able to communicate to virtual machines on the other without routing their traffic through an established firewall on your network. Further, no management interface for ESXi would be exposed in the DMZ, and given what you’ve seen with virtual machine isolation, the ESXi host would be safe from attack from a compromised DMZ virtual machine.

As you add physical NIC ports to your host, you could also create additional vSwitches to isolate network traffic for other virtual machines on different network segments or to isolate ESXi storage network traffic. For additional reading on vSwitch design, an excellent online resource is http://kensvirtualreality.wordpress.com/2009/03/29/the-great-vswitch-debate-part-1/, and the ESXi Configuration Guide contains a best practice guide for various network scenarios.

In some scenarios, you may have a limited number of NIC ports on your ESXi host and with the emergence of 10GB networking, you may be consolidating the virtual networking layer onto fewer NICs. In such a case, you can configure your vSwitches with multiple port groups for various network services and then isolate each within the vSwitch with VLANs. The sample shown in Figure 7.2 has a single vSwitch connected to two physical NIC ports in the host. A number of different network services are provided in this scenario, including the management interface for ESX, the network storage interface for ESXi, and virtual machines hosted in a DMZ. Those are obviously not types of traffic that you would want to mix on the same network segment, as compromising a DMZ virtual machine would open the rest of your network infrastructure to attack. In this case, the traffic types are isolated through the use of VLANs.

Figure 7.2. Isolating network traffic on a vSwitch with VLANs.

You may be wondering about network communication between virtual machines on the same vSwitch. If the virtual machines are separated with VLANs, network traffic between the two virtual machines must leave the vSwitch and will be routed between the two subnets through an existing router or firewall. But for two virtual machines on the same VLAN and vSwitch, the network traffic that passes between the virtual machines receives no protection. You can consider deploying firewalls within your virtual machines or may consider employing virtual firewalls. VMware vShield App is an example of a hypervisor-level firewall and provides connection control based on network, application port, protocol type, and application type. This product is discussed later in this chapter.

Each VMware ESXi host that you deploy will maintain its own list of users, groups, roles, and permissions. To view the default accounts on your host, select the Home > Inventory > Inventory view. Select the Local Users & Groups tab and click Users. You will see the list shown in Table 7.2, and as you will note, it is a much shorter list than with ESX due to the removal of the Linux-based Service Console in ESXi. ESXi likewise has far fewer locally defined groups than ESX. These groups include root, daemon, nfsnobody, nobody, tty, users, and vimuser.

To create a new user, you can use the following process:

pam_passwdqc.so retry=3 min=8,8,8,7,6

Merely adding an account will be insufficient to enable a user to log in to your ESXi host. The account must be granted a role on your ESXi host. Roles are a collection of privileges; when you assign user or group permissions to an object on your ESXi host, you’ll grant the user or group a specific role to an object, and that role will define which privileges are granted to the object.

On an ESXi host, there are three predefined roles that cannot be modified or deleted. These roles can be used to assign privileges to your user or as models for more refined roles. Table 7.3 lists the default roles that you will find on your ESXi host.

Use the following steps to create a new role:

When you create a new role, you will observe that there are a significant number of individual privileges that can be assigned to a role. When you select an individual privilege, a short description is shown for it. You can also refer to The vSphere Datacenter Administration Guide for a list of required privileges for common tasks.

The final step in granting permissions to a user or group consists of granting that user or group a role for an inventory object. For a standalone ESXi host, you can assign permissions to the host, resource pools, or virtual machines. By default, the root and dcui accounts will have the administrator role on all objects on the host. The permission for these accounts is granted at the host level with the Propagate to Child Objects option enabled so that the accounts will have full control of all objects created on the host. When the host is added to vCenter Server, the vpxuser is granted the Administrator role at the host level as well. This is also the case for the AD group ESX Admins when you configure AD Integration.

To assign permissions to an object on your ESXi host, follow these steps:

For a user who has limited permissions on your host, the vSphere client will display only the objects for which the user has permissions assigned. In Figure 7.10, the vSphere client displays only the ESXi host and the specific virtual machine for which the user has been assigned a virtual machine administrator type role. If the user selects the host object, no information will be displayed, and he or she will see the error message You do not have the permissions to access this object. If the user right-clicks on the host, all options will be grayed out. The user will be able to control the virtual machine, but not perform any actions that require host permissions. For example, Figure 7.10 shows no datastore listed, as the required permissions have not been granted. Likewise, the user is not able to modify the permissions for the virtual machine. The Events tab and the Recent Tasks pane will filter information based on the user’s permissions. If a user just has permission to a virtual machine, he or she will not see tasks and events related to the host or other virtual machines.

Figure 7.10. Logging in to ESXi with a limited account.

If you need to assign permissions to a number of virtual machines on your standalone host, you may find it cumbersome to manage permissions on each virtual machine. You can use a resource pool to create a folder for your virtual machines and then just assign permissions on the resource pool, which will propagate to each virtual machine. Use the following steps to create a resource pool and to move your virtual machines to it:

Once the resource pool has been set up with the appropriate virtual machines and permissions have been set for a user to manage the virtual machines in that resource pool, the user will log in with the vSphere client and be able to view only the resource pool and virtual machines in that resource pool. The user’s capabilities will be the same as described previously when a user was set up with permissions for a single virtual machine and not the host, as shown in Figure 7.10.

When you move virtual machines, any permissions assigned to them will remain in place after the object is moved into a resource pool. If a virtual machine is assigned the No Access role to a specific user, the user will not have access to or be able to see the virtual machine, regardless of the permissions that were set on the resource pool.

After you have established your permissions, you may find the need to review, update, or remove permissions. In the vSphere client, select the Roles view to view the currently configured roles. If you select a role, you will see whether the role is unused and the objects to which the role is assigned. In Figure 7.11, you can see that the Virtual Machine Administrator role is assigned to the XP01 virtual machine and the Production Servers resource pool. A green arrow on the user or group indicates that the permission has been set with the Propagate to Child Objects option enabled. To change the privileges for the role, simply right-click, and then select Edit Role. You can also remove the role by right-clicking and selecting Remove. If the role is currently in use, you will be given the choice to remove role assignments or to reassign the affected users to another role.

Figure 7.11. Viewing permission assignments for a role.

To view permissions for an object in the vSphere client, you select the object, which in the case of a standalone host will be the host, a resource pool, or virtual machine, and then select the Permissions tab. You will be able to view the permissions set on that object and any permissions set on parent objects that have been propagated to the object you are viewing. To remove a permission, right-click on the entry and select Delete. To change the role for a user or group, right-click, and then select Properties. You can then select the new role to use and also change the Propagate to Child Objects option.

Managing permissions with vCenter Server follows the same process as with a standalone ESXi host. Users or groups are assigned a specific role to an object in vCenter Server. vCenter Server does not rely on the local account on your ESXi host, but rather the domain or workgroup accounts of which the vCenter Server is a member. The number of objects with a standalone host on which you can assign permissions is limited to the host, resource pools, and virtual machines. With vCenter Server, that list is significantly expanded to the following items:

Given the increase in the number of objects, it is more important to understand the hierarchical inheritance of permissions that is used with vCenter Server. With a standalone host, choosing the Propagate to Child Objects option on permissions causes permissions on the host to be propagated to resource pools and virtual machines. Those objects would have the same permissions as the parent object unless permissions were specifically set on an object. In that case, with both a standalone host and vCenter Server, the permissions on a child object always override the permissions that are propagated from parent objects. With vCenter Server, the permissions hierarchy is more complex and is shown in Figure 7.12. Permissions can be propagated from a number of parent objects, so it is important to review existing permissions carefully when making changes and to plan your hierarchy design carefully to minimize permissions complexity.

Figure 7.12. The vCenter Server permissions hierarchy.

When you first install vCenter Server and begin to configure it, one of the first things you have to do is to add a datacenter object. This is required before you can add your ESXi hosts. Your datacenter objects can be used to organize hosts by geographical location, department, or other methods of grouping. If your management needs vary by datacenter, you can then organize your permissions at the datacenter level to provide each with a unique set of permissions.

Folders are another addition in vCenter Server that allow you to organize your hierarchy. Folders can be created under the vCenter Server object as well as under any datacenter objects that you create. You can create multiple levels of folders. This expands your options for organizing your vCenter objects as you can create a folder structure to segment your datacenters and then use additional folders under the datacenters to organize clusters and hosts.

It is also worthwhile to be aware that folders can be created within a number of views. Generally, you may deal with the Hosts and Clusters view and create your folder structure there. But you can do the same in the following views: VMs and Templates, Datastores, and Networking. In each view, the folder structure below a datacenter object is unique to the view. In the VMs and Templates view, you can create a folder structure to organize your virtual machines as shown in Figure 7.13. You can also set permissions as appropriate on any of the folders in the hierarchy. The folder structure for the Hosts and Clusters view can reflect different organizational needs and you can set permissions on those folders as needed, as shown in Figure 7.14.

Figure 7.13. Organizing virtual machines with folders in the VMs and Templates view.

Figure 7.14. Organizing vCenter objects with folders in the Hosts and Clusters view.

With the flexibility that vCenter Server offers in setting permissions on various objects and in different views, it is important to understand how multiple permissions granted will interact. Permissions applied to a child object will always override permissions applied on any parent object. If no permissions are defined for the user or group, permissions propagated from parent objects will be applied. You should note that virtual machine folders and resource pools are equivalent levels in the hierarchy, as shown in Figure 7.12. Permissions for virtual machines will combine the propagating permissions from both objects. In the case of a user belonging to two or more groups that have permissions to an object, the following situations will apply:

The following examples will demonstrate some various scenarios for permissions and how they interact in different situations. For each of these examples, the following will apply:

In the first example shown in Figure 7.15, Group A has been granted Role 1 at the virtual machine folder level, and Group B has been granted Role 2 at the same level. Both have the Propagate to Child Objects option enabled. User A does not have any specific roles granted on any of the objects shown. Since User A does not have any specific permission granted, the user inherits the combined permissions granted to Group A and Group B and is able to power on and create snapshots for both virtual machines.

Figure 7.15. Inheritance of multiple permissions.

In the second example shown in Figure 7.16, Group A has been granted Role 1 at the virtual machine folder level. Group B has been granted Role 2 on Virtual Machine B. User 1 does not have any specific roles granted on any of the objects shown. In this case, the user will have the ability to power on Virtual Machine A through permission inheritance. However, Role 2 is assigned at a lower point in the vCenter object hierarchy, as shown in Figure 7.12. Thus, for Virtual Machine B, the privileges of Role 2 override the privileges granted in Role 1 at the virtual machine folder level. User 1 is only able to create snapshots on Virtual Machine B.

Figure 7.16. Overriding permission inheritance.

The last example, shown in Figure 7.17, demonstrates how user permissions will override group permissions. Group A has been granted Role 1 at the virtual machine folder level. User 1 has been granted the No Access system role at the same level. In this case, the permissions granted to the user override any group permissions and User 1 does not have any access to the folder or the virtual machines. In the vSphere client, the folder and virtual machines would not be visible. If User 1 had been granted the No Access role on Virtual Machine A, User 1 would have been able to power on Virtual Machine B, but would have no access to Virtual Machine A. In the vSphere client, Virtual Machine A would not be visible to User 1.

Figure 7.17. User permissions overriding group permissions.

When using vCenter Server and assigning permissions to AD users and groups, it is important to understand how vCenter Server interacts with AD to validate users and groups. vCenter Server validates with AD when the VMware VirtualCenter Server Windows service is started. This occurs when the vCenter host is restarted or if you manually start the service. vCenter Server also periodically validates with AD. The default setting for this is every 24 hours, as shown in Figure 7.18. If a user were to be deleted in the domain, the corresponding permissions in vCenter Server granted to the individual’s user account would be removed at the next validation period. If a new account with the same name were to be added before the validation period had occurred, that new account would receive all permissions granted to the old account. Thus it is important to ensure that the Enable Validation setting is enabled and that the Validation Period is appropriate for your domain.

Figure 7.18. vCenter Server Active Directory settings.

To change the validation period for your vCenter Server, use the following steps:

The validation period will also have an impact on the removal of permissions granted to users or groups. If you change a group or user name in Active Directory, when vCenter Server validates with AD, it will assume the group or user has been deleted and remove any permissions granted to the AD object. Also when you remove an AD user or group, the permissions will remain listed in vCenter Server until the next validation period.

When assigning permissions with a domain that contains thousands of users or groups, you may find that searches of AD take a long time. You can adjust other settings on the Active Directory vCenter Server Settings screen, shown in Figure 7.18. The settings are listed in Table 7.4.

Get-VMHostAccount
Id             Description               Groups
--             -----------               -------------
root           Administrator             {root}
nobody         Nobody                    {}
nfsnobody      Anonymous NFS User        {}
dcui           DCUI User                 {}
daemon         daemon                    {}
vimuser        vimuser                   {}
vpxuser        VMware VirtualCen…        {users}

New-VMHostAccount -Id dave -Password SecUre34 -Description "Admin account"
Id        Description        Groups
--        -----------        --------
dave      Admin account      {users}

Get-VIRole
Name              IsSystem
----              ---------
NoAccess          True
Anonymous         True
View              True
ReadOnly          True
Admin             True
VM Administrator  False
Get-VIPermission
EntityId            Role                Principal     IsGroup    Propagate
--------            ----                -------       -----      -------
VirtualMachine-48   Virtual Machine     MISHCHENKO    False      True
                    Admini…             d…
Folder-ha-folder-…  Admin               vpxuser       False      True
Folder-ha-folder-…  Admin               dcui          False      True
Folder-ha-folder-…  Admin               root          False      True
Folder-ha-folder-…  Admin               MISHCHENKOe… True       True

New-VIPermission -Role ReadOnly -Principal dave -Entity esx01
EntityId              Role          Principal     IsGroup       Propagate
------                ----          -------       ------        -------
HostSystem-ha-host    ReadOnly      dave          False         True

New-VIPermission -Role "VM Administrator" -Principal dave -Entity LINUX02
EntityId              Role                    Principal IsGroup Propagate
------                ----                    -------   ------  -------
VirtualMachine-48     Virtual Machine Admini… dave      False   True

Get-VIPermission -Principal dave
EntityId              Role              Principal     IsGroup   Propagate
------                ---               -------       ------    -------
VirtualMachine-48     VM Administator   dave          False     True
ComputeResource-h…    ReadOnly          dave          False     True
HostSystem-ha-host    ReadOnly          dave          False     True
ResourcePool-ha-r…    ReadOnly          dave          False     True

new-vipermission -role admin -principal "MishchenkoDave.Mishchenko" -Entity
   ESX05.mishchenko.net
New-VIPermission : 4/17/2010 5:26:19 PM   New-VIPermission     Could not
   find VIAccount with name 'MishchenkoDave.Mishchenko'.

function New-VIAccount($principal) {
   $flags = '
        [System.Reflection.BindingFlags]::NonPublic    -bor
        [System.Reflection.BindingFlags]::Public       -bor
        [System.Reflection.BindingFlags]::DeclaredOnly -bor
        [System.Reflection.BindingFlags]::Instance
   $method = $defaultviserver.GetType().GetMethods($flags) |
      where { $_.Name -eq "VMware.VimAutomation.Types.VIObjectCore.get_
      Client" }
   $client = $method.Invoke($global:DefaultVIServer, $null)
   Write-Output '
       (New-Object VMware.VimAutomation.Client20.PermissionManagement.
       VCUserAccountImpl '
         -ArgumentList $principal, "", $client)
}

$account = New-VIAccount "MishchenkoDave.Mishchenko"
Get-VMHost esx01.mishchenko.net | New-VIPermission -Role Admin -Principal
   $account
EntityId             Role             Principal      IsGroup      Propagate
------               ----             -------        ------       ------
HostSystem-host-18   Admin            MISHCHENKOD…  False        True

There are two kinds of SSL certificates that you may deal with in your vCenter Server infrastructure. Self-signed certificates have been generated by the server to which you are trying to connect. Use of self-signed certificates does not indicate that the session data will be less securely encrypted. The client host must implicitly trust the server host to be the server that it claims to be. Some applications will detect a self-signed certificate and warn the user of the situation. The following certificate error is generated when connecting with PowerCLI to a vCenter Server host with a self-signed certificate:

Connect-VIServer vcenter41.mishchenko.net
WARNING: There were one or more problems with the server certificate:
* The X509 chain could not be built up to the root certificate.
Name                               Port               User
----                               ----               ----
vcenter41.mishchenko.net           443                Administrator

The other kind of SSL certificate that you will encounter is one signed by a certificate authority (CA). A CA can be used by the client and server to confirm the validity of the SSL certificate being sent by the destination host. The CA used can be a well-known and publically trusted CA such as RSA or VeriSign, or it may be an internal CA set up as part of your company’s information technology (IT) infrastructure. In either case, the client or server trust the CA to verify the certificate and thus confirm the identity of the other host involved in the SSL session.

As noted earlier in this chapter, SSL encrypted traffic is used between the vSphere client and hosts, between vCenter Server and ESXi hosts, and, in certain functions, between ESXi hosts. Both ESXi and vCenter Server support SSL v3 and Transport Layer Security (TLS) v1, which will both be referred to as SSL. SSL v3 was released in 1996 and replaced SSL v2, which contained a number of security flaws. TLS was defined in 1999 as an upgrade to SSL v3. The differences between the protocols are not significant, but they are sufficient to make it necessary to prevent interoperation between the two.

Both products also install with a self-signed certificate. In the case of a vCenter Server host, the certificate is issued to VMware Default Certificate and signed by the VMware Installer, as shown in Figure 7.19. With VMware ESXi, the certificate for a default installation is issued to localhost.localdomain, as that is the default hostname after an installation. With a scripted installation, if you set the hostname, the certificate will be issued to the hostname that you have set in the installation script.

Figure 7.19. vCenter Server creates a self-signed SSL certificate during the installation process.

The Security Warning dialog box as shown in Figure 7.19 provides the following options when connecting to a host that has generated an SSL certificate warning:

When you receive a security warning, the details should be carefully reviewed. In some cases, the warning is generated due to detection of a self-signed certificate. In other cases, it may be that the hostname to which you are connecting does not match the hostname on the SSL certificate. Or it could be that the SSL certificate has expired and thus is no longer valid.

Regardless of the case of using self-signed certificates or a security warning being generated due to an SSL certificate problem, choosing to ignore the warning opens the possibility of a Man in the Middle Attack (MiTM). As noted earlier, even with self-signed certificates, network communication is encrypted and thus protected from casual network sniffing. But if it is not possible to verify the identity of the client or server involved in the SSL handshake, there is the possibility that a third host may launch a MiTM attack.

The following sections describe the process to replace the SSL certificates used by vCenter Server and ESXi with those generated by a trusted certificate authority. The processes use certificates created by a Windows 2003 Server certificate authority that has been set up as the enterprise CA for a Active Directory domain. The certificate requests are generated by the OpenSSL Toolkit, which you can download from this link: http://www.openssl.org/related/binaries.html. OpenSSL is run on a Windows server to generate the requests for both vCenter Server and the ESXi hosts, but you could follow the same process on a Linux server, and many distributions already include OpenSSL. You could likewise use certificates generated by a trusted, well-known CA. For additional information on the certificates used by vCenter Server and ESXi and the options to replace the certificates, you can refer to the ESXi Configuration Guide and http://www.vmware.com/pdf/vsp_4_vcserver_certificates.pdf.

To install OpenSSL on a Windows server, use the following process:

With vCenter Server, you can enable verification of host SSL certificates. This option is enabled by default and it is recommended to leave this setting enabled. The verification process affects operations such as adding a host, connecting to a virtual machine, and making virtual machine devices remotely available. This option is required for Fault Tolerance to operate.

To verify the SHA1 thumbprint of certificates on hosts, follow this procedure:

With a traditional FC SAN deployment, each ESXi host contains an FC Host Bus Adapter (FC-HBA) that connects to an isolated storage network. Each FC-HBA has what is essentially a storage Media Access Control (MAC) address called a worldwide port name (WWPN). For security, the SAN presents a storage logical unit number (LUN) to the WWPN. The ESXi host with that FC-HBA is then able to access that storage LUN and use it as needed. The SAN can also be configured to provide access to a single LUN to a number of WWPNs. In this way, the LUN is configured to be shared among several ESXi hosts to allow for clustering. If a host fails, the FC-HBA can be moved to another host and the new host will have the same LUN access as the old host. While this is a benefit to recovery, it also poses a security concern, as the new host has no requirement to authenticate or validate its identity with the SAN. When configuring your SAN, you must carefully ensure that the LUNs are zoned to the appropriate WWPNs.

As Fibre Channel storage evolves as a technology, new security concerns come to light. With the emergence of Fibre Channel over Ethernet (FCOE), it is possible to send FC data over your Ethernet network. While this technology provides the opportunity to converge your network design, you must consider the risk of comingling your storage data with other network data types.

Another new Fibre Channel technology to be aware of is N_Port ID Virtualization (NPIV). With NPIV, a virtual machine can be granted an identity in the FC storage fabric. Instead of just granting access to a LUN to a host and all the virtual machines running on that host, the SAN administrator can grant access to a specific virtual machine. This provides an additional layer of LUN masking and allows the SAN administrator to identity traffic for that specific virtual machine.

VMware ESXi supports NFS version 3 over TCP. NFS storage is easy to set up, but it represents the least secure level of storage that you can use with ESXi. Authentication is often based upon an IP address, which can be spoofed. NFS is a clear text protocol, and ESXi authenticates with the NFS storage with the root login. Additionally, accessing NFS from ESXi requires that the no_root_squash option be enabled. With root squashing, the root account is mapped to another unprivileged account such as nobody on the NFS server to prevent root accounts on client machines from gaining privileged access to the NFS mounts. ESXi, however, requires read and write access to the mounts, so it is necessary to disable root squashing.

If you employ NFS storage, the primary concern is traffic isolation. If your environment requires physical separation, your NFS storage should be on a physically isolated network, and the vSwitch that connects to that network should have only a VMkernel port configured on it. No virtual machine port groups should be in that vSwitch, to prevent a virtual machine from attempting to attack or capture network traffic from the NFS server. You can also configure the NFS server to restrict access to the specific IP addresses of your ESXi host or use a firewall to restrict traffic to a specific host. If your NFS server supports IPSec, you might also consider configured IPSec for the traffic between your ESXi host and NFS server. If you are employing blade servers with a minimal number of NICs or 10GB networking, you should ensure that NFS traffic is isolated with a VLAN.

Provisioning iSCSI storage is similar to Fibre Channel storage in that storage LUNs are presented to hosts that then choose how to partition and format the LUN. Access to your iSCSI storage can be via iSCSI Host Bus Adapters, which offload the overhead associated with iSCSI, or with standard NICs configured within your ESXi host’s vSwitches.

Given that iSCSI traffic is over an IP-based network, it is vulnerable to the attacks discussed throughout this chapter. As with the other storage options, isolation is the first step to securing iSCSI. The iSCSI protocol supports the use of Challenge-Handshake Authentication Protocol (CHAP), but post authentication traffic is sent in clear text. As is the case with NFS traffic, consider the use of IPSec to secure traffic between your ESXi hosts and iSCSI target. You might also consider the use of firewalls to protect the hosts involved. When you use iSCSI, ESXi does not open any ports to listen for network connections. This reduces the attack surface of your ESXi host, but you should still isolate the VMkernel port that is used for iSCSI access from general network traffic.

When designing your networking for iSCSI, it is ideal to be able to separate your iSCSI traffic either physically or with the use of a VLAN. If you are using iSCSI-HBAs, these devices will connect directly to your isolated network ports. If you’re using the software iSCSI initiator within ESXi, you will configure a new vSwitch for storage traffic, and then configure this vSwitch to connect to the isolated storage network.

When configuring iSCSI storage for your ESXi hosts, you should enable CHAP authentication. When the ESXi initiator contacts the iSCSI target, the target responds with a predefined ID value and a random key. The initiator then creates a one-way hash value that it sends to the target. The hash contains the ID value, random key, and the CHAP secret that has been configured on both the ESXi host and iSCSI server. When the target receives the hash value, it computes its own hash value for the same elements. If both hashes match, the iSCSI target has verified the identity of the initiator.

ESXi supports both unidirectional and bidirectional CHAP. With unidirectional CHAP, only the iSCSI target authenticates the ESXi host. With bidirectional CHAP, the ESXi host also authenticates the iSCSI target providing an additional layer of security. It is possible to disable CHAP authentication, and the ESXi host will still authenticate in a very simple manner in that the iSCSI target will map a LUN to the unique iSCSI name for the host. This is not a recommended method to set up your iSCSI access.

To enable CHAP authentication, you can follow this process:

The preceding process configured CHAP authentication at the adapter level, and these settings and credentials would apply to all iSCSI targets that the host would attempt to access. You can also configure CHAP authentication on a per-target basis, which allows for greater security. When you add or modify a dynamic or static iSCSI target, you can click the CHAP button to access CHAP settings that will apply only to that specific target.

In the cases where you have a limited number of network ports to configure, you have to rely on VLANs to segment your network traffic. There is a fair bit of debate of the security capabilities of VLANs, as they have been subject to attacks and circumventing in the past. However, the technology has matured and the vSwitch provides safeguards against certain threats to VLAN security, including the following:

The preceding list is not exhaustive and both technologies and security threats develop over time. However, use of VLANs in your vSwitches is a viable option for providing network isolation. Perhaps a more significant risk in using VLANs is the possibility of misconfiguration.

vSphere includes a number of improvements with permissions. An example includes the additional choices for network configuration for a virtual machine, as shown in Figure 7.30. In prior versions of ESXi and ESX, it was only possible to assign a Remove permission. With vSphere, you can also control the assignment of a virtual NIC to a vSwitch. You can also utilize Distributed Virtual Switches and Host Profiles to ensure consistent configurations across your ESXi host. There are also additional permissions you can control for Distributed Virtual Switches. Properly configured, these elements can help reduce the risk of a virtual machine or vSwitch being misconfigured either intentionally or accidentally, which could expose a virtual machine to an unauthorized VLAN.

Figure 7.30. Configuring network permissions for a role.

You should also be aware that VMware vSwitches do not support the concept of a native VLAN. Native VLANs are used by switches for switch control and management. Native VLANs are not tagged with a VLAN ID in some switch implementations, and trunk ports treat all untagged packets as belonging to the native VLAN. If you have created a number of virtual machine port groups in your switch, assigned them VLANS, but left one port group as unset, traffic from that port group could end up on the native VLAN of your physical switches. If your physical switches use VLAN ID 1 for the native VLAN, you should configure your vSwitches with VLAN IDs in the range of 2 to 4094. If another VLAN ID is used for the native VLAN, you should avoid using that VLAN ID on your vSwitches.

Use of VLAN ID 4095 enables Virtual Machine Guest Tagging (VGT) mode. In this mode, the vSwitch port group passes packets to the virtual machines without modifying any VLAN tags. It is the responsibility of the guest operating system to assign a VLAN ID to packets that it sends. If enabled inappropriately, this could allow a virtual machine to interact with a VLAN for which it is not authorized.

The properties for vSwitches and port groups include a number of security policies designed to safeguard your network. From the perspective of the guest operating system, the virtual NIC functions just as a physical NIC would, so a malicious program executing within the virtual machine is capable of forging MAC addresses or flooding the network to create a DoS attack. There are three security options that you can set to prevent malicious activity, and you can set limits on network traffic to prevent excessive network load.

To make changes to the security policy or traffic shaping policy of a vSwitch, you can follow these steps:

The first setting on the Security tab is Promiscuous Mode. The option is by default set to Reject. This eliminates the possibility that the virtual machine could be used to capture all traffic on the vSwitch, as the vSwitch will filter network traffic to ensure that the virtual machine receives only packets destined for that specific virtual machine. In some cases, if you are running a network sniffer or intrusion detection software, you may need to allow Promiscuous Mode on a vSwitch, but for most vSwitches, this setting can be left with the default setting.

The second option to configure is MAC Address Changes. This option is set to Accept. When a virtual NIC is added to a virtual machine, it is automatically assigned a MAC address. The guest operating system can change the MAC address and will begin to receive traffic destined for the new MAC address. This can be required in some situations such as using Microsoft Network Load Balancing in unicast mode. Additionally, the iSCSI initiator relies on being able to get MAC address changes from some types of storage. If the setting is change to Reject, ESXi will not honor a request by the virtual machine to change the MAC address from the initial value.

The last setting on the Security tab is Forged Transmits. When the option is set to Accept, ESXi does not compare source and effective MAC addresses. To protect against MAC impersonation, you can set this option to Reject. With that setting, ESXi compares the source MAC address as generated by the guest operating system with the effective MAC address for the adapters and drops the packet if there is not a match. This setting can impact applications that require a specific MAC address for licensing.

On the Traffic Shaping tab, you can set the traffic shaping policy for the vSwitch. This can be useful to ensure that a virtual machine is not used to saturate a vSwitch, which could create a DoS for other virtual machines or ESXi system services on that vSwitch. The traffic shaping policy is defined by the following three attributes: Average Bandwidth, Peak Bandwidth, and Burst Size. Average Bandwidth sets the bits per second to allow through the port averaged over time. Peak Bandwidth is the maximum number of bits per second allowed through the port when it is receiving or sending a burst of traffic. This setting, along with Burst Size, allows a vSwitch to transmit beyond the Average Bandwidth restriction. The Burst Size setting specifies the maximum number of bytes to allow in a burst of network traffic.

If you select the properties for a virtual machine port group or VMkernel port, the Security and Traffic Shaping tabs can also be found on these objects. You can check an option as shown in Figure 7.33 to override the setting enabled on the vSwitch. If you make no changes, the settings for the virtual machine port group or VMkernel port inherit the settings from the vSwitch.

Figure 7.33. Overriding the security policy on a virtual machine port group.