Chapter 17. Migrating Virtual Machines
This chapter covers the following subjects:
ESXi Host and Virtual Machine Requirements for vMotion and Storage vMotion
Enhanced vMotion Compatibility CPU Requirements
Snapshot Requirements for vMotion and Storage vMotion Migration
Migrating Virtual Machines Using vMotion/Storage vMotion
Migrating a Powered-Off or Suspended Virtual Machine
When you think about it, migration is not a technical term; in fact, even birds migrate. To migrate means to move from one place to another. So, what is actually moving with regard to our studies? What is moving is either the state of the VMs, the files that the VMs use, or both. These moves can be done with the VMs powered off or powered on.
When the VMs are powered off, we refer to the moving of their state and their files as cold migration. When they are powered on, we refer to the moving of their state as vMotion and the moving of their files as Storage vMotion. There are advantages and disadvantages to each option. To facilitate any option, you must have your vSphere configured properly.
This chapter first identifies the ESXi host and VM requirements for vMotion, Storage vMotion, and the newest type of vMotion called Enhanced vMotion or Cross-Host vMotion. Next are the requirements for Enhanced Storage vMotion and Long Distance vMotion. The new requirements (or actually lack of requirements) with regard to snapshots and vMotion/Storage vMotion are also discussed. In addition, you learn how you can configure the VM swap file location. Finally, the chapter covers utilizing Storage vMotion techniques for other creative uses, such as changing the disk type, changing the provisioning, and renaming the VM folders.
“Do I Know This Already?” Quiz
The “Do I Know This Already?” quiz allows you to assess whether you should read this entire chapter or simply jump to the “Exam Preparation Tasks” section for review. If you are in doubt, read the entire chapter. Table 17-1 outlines the major headings in this chapter and the corresponding “Do I Know This Already?” quiz questions. You can find the answers in Appendix A, “Answers to the ‘Do I Know This Already?’ Quizzes and Chapter Review Questions.”
1. An administrator uses vMotion to migrate a VM. Which of the following will occur? (Choose two.)
a. The state of the VM will move from one datastore to another datastore.
b. The VMs files will remain on the same datastore.
c. The state of the VM will move from one host to another host.
d. The VM files will move from one datastore to another one.
2. An administrator of a vSphere network with a Standard license attempts to migrate a VM using vMotion but receives an error message indicating that vMotion is not enabled. Which of the following could be the cause of this message?
a. There is no shared storage configured.
b. The VMkernel port intended for vMotion is not enabled for vMotion.
c. There is not enough memory to provide for vMotion.
d. The CPUs are the same on both source and destination hosts.
3. Which of the following would prevent an administrator from using Storage vMotion?
a. The VMkernel port is not enabled for vMotion.
b. The vSphere uses a Standard license.
c. The host does not have visibility to the destination datastore.
d. The CPU of the host is not compatible with Storage vMotion.
4. An administrator attempts to configure EVC on a cluster but does not meet the requirements. Which of the following would prevent EVC from succeeding?
a. Hosts are a mixture of vSphere 4, 5, and 6.
b. Host CPUs are a mixture of Intel and AMD.
c. All hosts are in the same host cluster.
d. All hosts are configured for vMotion.
5. Which of the following would prevent long-distance vMotion from operating properly? (Choose two.)
a. RTT of 50 ms
b. A vSphere Standard license
c. RTT of 200 ms
d. A common Layer 2 environment
6. Which of the following is not a part of a snapshot?
a. Settings state
b. Disk state
c. Memory state
d. Power state
7. Which of the following are true in regard to vMotion and Storage vMotion with vSphere 6? (Choose two.)
a. You can migrate a VM using vMotion when it contains snapshots.
b. You cannot migrate a VM’s files using Storage vMotion if the VM contains snapshots.
c. You cannot migrate a VM using vMotion if the VM contains snapshots.
d. You can migrate the files of a VM using Storage vMotion when the VM contains snapshots.
8. Which of the following would prevent a VM from using vMotion?
a. An internal switch on its host, to which the VM is not connected
b. A snapshot that has not been deleted
c. A swap file that is local to a host
d. An ISO mounted on the local host, to which the VM is connected
9. Which of the following are true regarding vMotion? (Choose two.)
a. By definition, you cannot vMotion a VM that is powered off.
b. You can vMotion a VM, whether it is powered on or off.
c. vMotion will always move the files of a VM to another location.
d. Just vMotion will never move the files of a VM to another location.
10. Which of the following are true? (Choose two.)
a. To cold migrate a VM’s files, you must first power off the VM.
b. You can cold migrate a VM’s files with a VM powered on or off.
c. The cold migration of the state of a VM is referred to as vMotion.
d. Cold migration of the state of a VM happens with the VM is powered off.
Foundation Topics
ESXi Host and Virtual Machine Requirements for vMotion and Storage vMotion
Identifying ESXi host and virtual machine requirements is too big a “bite to chew” all at once. We’ll break this into two discussions: one for vMotion and the other for Storage vMotion. Although some of the requirements are the same for both, not all the requirements are the same.
ESXi and VM Requirements for vMotion
As you might remember, vMotion means moving the state of the VM from one host to another without disrupting the user. You may be wondering, “What exactly is the state of the VM?” The state of the VM consists of three components:
Settings: These are the settings that you will see on all the tabs if you right-click the VM and select Edit Settings. These are also the configuration settings as defined in the VMX file. These settings describe the VM and all its attributes and virtual hardware.
Disk: These are all the VMDK files and virtual compatibility RDMs that make up the source volumes for the VM.
Memory: This is the active memory on the VM at the time that you are performing the vMotion.
Each component of the state of the VM is handled in a different manner. The Settings state is simple to move, or actually re-create, on another host. The Disk state cannot be re-created in the short amount of time allowed; therefore, we have to use shared storage. The Memory state must be gradually copied over to the destination host so that when the guest OS on the VM is quiesced, the OS “wakes up” on the destination host with the same memory it had on the source host.
We are in essence fooling the guest OS on the VM into thinking that it hasn’t migrated. This means that all the components of the host and the VM must be configured properly to pull off the trick. The requirements for the host with regard to vMotion are as follows:
Source and destination hosts must both have visibility to shared storage used by the VM on Fibre Channel, iSCSI, or NAS (for original vMotion).
All hosts must have access to at least a 1 Gbps Ethernet network.
Source and destination hosts must have access to the same physical network or the same VLAN or VXLAN.
Source and destination hosts must have VMkernel ports configured on a vSS or a vDS that are enabled for vMotion and have a proper IP configuration. It is possible to vMotion from one vDS to another with vSphere 6. This is called cross-switch vMotion.
Host CPUs must be the same vendor and family (for example, Intel Xeon) and must share the same CPUID unless configured otherwise in Advanced settings or by using EVC. (These options are discussed in the next section.)
Just as the hosts must be configured properly for vMotion, the VMs must also be configured properly. If you remember what you are trying to pull off here, you will see that you can’t have anything that the VM can “see” on the source host that it won’t be able to “see” when it gets to the destination host. With that in mind, the following are the VM requirements with regard to vMotion:
VMs must not have a connection to an internal switch (zero uplink adapters).
VMs must not have a connection to a virtual device with a local image mounted (locally mounted CD, floppy drive, or ISO).
VMs must not have CPU affinity configured.
If the VM’s swap file is not accessible to the destination host, vMotion must be able to create a swap file accessible to the destination host before migration can begin. (This option can be advantageous when using SSD hard drives locally on a host.)
If the VM uses an RDM, the RDM must be accessible to the destination host as well.
ESXi and VM Requirements for Storage vMotion
The main thing about Storage vMotion that you should understand is that the VM’s state doesn’t move at all! What moves (or rather, is copied) are the files that the VM is using. In other words, when you use Storage vMotion, you have a running VM (in our case, a server) that continues to operate while the files that it requires to operate are moved to another physical location. When the migration happens, the OS on the running server is unaware that it has taken place at all. The advantages of this technology are as follows:
You can perform storage maintenance and reconfiguration without VM downtime.
You can redistribute storage loads without VM downtime.
You can evacuate physical storage about to be retired and perform storage tiering, without VM downtime.
Another cool fact is that Storage vMotion is completely storage type independent. That means that you can move files that are on Fibre Channel SAN LUNs to NAS (for an extreme example) while the server is running without disruption to the OS or the user. This is just the beginning, and you learn some other creative things that you can do with Storage vMotion later in this section. The bottom line is that having to tell users that they won’t be able to get onto the system because you have to do some hardware maintenance to the drives should now be a thing of the past.
Enhanced vMotion (Cross-Host vMotion)
VMware originally named it Enhanced vMotion, but too many people were confusing that term with Enhanced vMotion Compatibility (EVC), which (as you see in the next section) is not at all the same thing. So, for lack of a better term, VMware decided to call it Cross-Host vMotion instead. This term doesn’t really describe what is different about it, because vMotion is always going “cross host”! However, what is so different about Cross-Host vMotion that it should deserve its own name?
Cross-Host vMotion can do two things that the original vMotion cannot. The first, in my opinion, is cool, but the second has the potential to be a game changer. The two things that Cross-Host vMotion can do that the original vMotion cannot are as follows:
Migrate the state of a VM to a new host at the same time that the files are migrating to another datastore: This, in my opinion, is cool because it ranks high on the “cool factor” and makes me change my story when I’m teaching class, but I don’t see myself having to use it much in real life because I am generally performing only one or the other of those actions. Figure 17-1 shows the capability to migrate a powered-on VM’s state to another host while simultaneously migrating its files to another datastore.
Perform a migration of a VM from one host to another without having a shared datastore between the hosts: This also causes me to change my story in class, but it has the potential to be a game changer. At present, DRS and SDRS do not take advantage of this option, but if they do some day, will that change some data center design options? Make no mistake, shared storage is always best, and we are coming up with more and new ways to have shared storage, but this might offer a “one-off” solution of the future.
Enhanced vMotion Compatibility CPU Requirements
When you initiate the use of vMotion or a migration of a suspended machine (still running a CPU instruction set), the wizard checks the destination host for compatibility. The CPU feature set that was detected when the VM was powered on to the source host is checked against what will be available on the destination host. The CPU feature set includes all of the following:
Host CPU vendor, family, and model
Settings in the BIOS that might disable CPU features
ESX/ESXi version running on the host
VM’s virtual hardware version (for example 7, 8, 9, 10, or 11)
VM’s guest OS
If the feature set is determined to be different on the destination host than on the source host, the system lets you know with a detailed error message if there are compatibility problems that will prevent the successful migration of the VM. You can hide some of the instruction set from the guest OS and satisfy the wizard by masking the CPUID in the CPU settings of each VM on the host, as shown in Figure 17-2.
An easier way to improve CPU compatibility between hosts of varying feature sets is to configure Enhanced vMotion Compatibility (EVC) on your cluster. EVC ensures that all hosts in your cluster present the same feature set to VMs, even if the actual CPUs on the host differ from each other. When you configure EVC, you are telling all host processors to present a feature set of the baseline processor. This feature set is referred to as the EVC mode. As you might remember, you can configure EVC in your cluster, but only if certain compatibility requirements are met first. The following are the compatibility requirements for EVC:
All VMs that are running on the cluster with a feature set greater than that of the EVC mode you intend to enable must be powered off or migrated out of the cluster before EVC is enabled.
All hosts in the cluster must have CPUs from a single vendor, either AMD or Intel (not both).
All hosts in the cluster must be running ESX/ESXi 3.5 Update 2 or later.
All hosts in the cluster must be connected to the vCenter System.
All hosts in the cluster must have advanced CPU features, such as hardware virtualization support (AMD-V or Intel VT) and AMD No eXecute (NX) or Intel eXecute Disable (XD), enabled in the BIOS if they are available.
All hosts in the cluster should be configured for vMotion.
All hosts in the cluster must have supported CPUs for the EVC mode you want to enable.
Any new host that is added to the cluster after EVC is enabled must also meet these requirements.
Long-Distance vMotion
When I first started teaching about vMotion with ESX 3.5 hosts and Virtual Center 2.5, the “magic” pretty much stopped in the local-area network (LAN). In other words, the technology worked within the “walls” of the LAN, but how could it possibly be extended to the WAN? After all, it required shared storage and a shared Layer 2 network, right?
Technologies have progressed, and we now have shared storage on the “cloud” as a common resource. In addition, VMware has made advancements that allow for vMotion in environments that do not have shared storage and that have higher latencies than what was possible in the past. This, combined with the newer technologies such as VXLAN (which is beyond the scope of this text), allow us to create a Layer 2 network that stretches across multiple Layer 3 networks.
For these reasons, it’s now possible to vMotion VMs from New York to California! That’s right, intracontinental long-distance vMotion is supported with vSphere 6! Now, to take advantage of this option, your network must meet the following criteria:
You must have an Enterprise Plus vSphere license.
Latency must be a round-trip time (RTT) of 150 milliseconds or less.
Both hosts must exist in the same Layer 2 environment, which means that a VXLAN-based technology such as NSX must be used.
Snapshot Requirements for vMotion/Storage vMotion Migration
Basically, a snapshot is a capture of the state of a VM at a specific point in time. As you should remember, the state can include settings, disk, and memory. Until vSphere 5.0, it was important to know that you could use vMotion to migrate VMs that had VM snapshots but that you could not use Storage vMotion to move VM files. The reason was that the technologies that worked with VM snapshots and those that worked with Storage vMotion were “oil and water”; in other words, they didn’t “blend” well together. Starting with vSphere 5.0, the technologies and methods used for Storage vMotion VM files have changed completely, so on vSphere 6.0 you can use Storage vMotion on VMs that have snapshots. If you think about it, this had to be done if Storage DRS was going to be used. Otherwise, how would the system automatically move VM files from one datastore to another on a datastore cluster if the administrator was using snapshots at the same time?
Migrating Virtual Machines Using vMotion/Storage vMotion
As mentioned earlier, this objective about migrating virtual machines is broken up into two objectives to stress that these are two different features in vSphere. First you learn about the process of migrating VMs using vMotion, and then the process of migrating VM files using Storage vMotion is covered.
Migrating a VM Using vMotion
When I say that we are migrating the VM, I am saying that the state of the VM is moving from one host to another host. When you do this with the VM powered on, by definition, it’s vMotion. I’ve heard some people use this term improperly by saying, “We vMotioned them, but we powered them down first.” You should understand clearly that what they did was a cold migration, not an actual vMotion.
However, you know that DRS uses vMotion to balance loads automatically, but what if you want to initiate the process yourself? If your hosts, your VMs, and your network meet all the requirements for vMotion, the process is easy. For example, to vMotion a VM from one host to another on the same cluster, follow the steps outlined in Activity 17-1.
Activity 17-1 Migrating a VM Using vMotion
1. Log on to your vSphere client.
2. Select Home and then Hosts and Clusters.
3. Right-click the powered-on VM that you want to vMotion and select Migrate, as shown in Figure 17-3.
4. On the Select Migration Type page, select the option button for Change VM Compute Resource Only, as shown in Figure 17-4, and then click Next.
5. Choose the host, cluster, resource pool, or vApp to which you want to migrate the VM, in this case esx60a.vclass.local; verify compatibility, as shown in Figure 17-5; and click Next.
6. Select the destination network to which you want to connect the VM, as shown in Figure 17-6, and click Next. (In vSphere 6, you can select Advanced to migrate to another vDS.)
7. Choose between reserving CPU resources first or performing with available CPU resources, as shown in Figure 17-7, and click Next. The first (default) option is almost always the right choice because you are performing this manually as a step in a process; therefore, you want it to work (or not) right now!
8. Review your settings, and click Finish.
9. Monitor your Recent Tasks pane and your inventory for the results of the vMotion.
Migrating a VM’s Files Using Storage vMotion
As I mentioned before, Storage vMotion does not move the state of the VM from one host to another. Storage vMotion moves the files that the VM is using, while it’s using them, from one physical storage area to another, without disrupting your VM’s OS or your end users. As discussed earlier, migrating this way greatly enhances your flexibility when making changes to storage due to growth, ages of disk, and so on.
It’s such a cool technology that you might think that it would be complicated to use, but you would not be correct. In fact, Storage vMotion is as easy to use as vMotion. To Storage vMotion a VM’s files from one datastore connected to your vSphere to another, follow the steps outlined in Activity 17-2.
Activity 17-2 Migrating a VM’s Files Using Storage vMotion
1. Log on to your vSphere Web Client.
2. Select Home and then Hosts and Clusters.
3. Right-click the powered-on VM that you want to Storage vMotion and select Migrate.
4. On the Select Migration Type page, select the option button for Change Storage Only, as shown in Figure 17-8, and then click Next. Note that the option Change Both Host and Datastore at the same time is available on the vSphere Web Client, even though it’s grayed out on the Windows-based vSphere Client. Isn’t that interesting?
5. Select the destination datastore or datastore cluster on your inventory and the disk format for the datastore, wait for validation of compatibility, as shown in Figure 17-9, and click Next. Note that you can also specify a specific VM Storage Policy.
6. Review your settings, as shown in Figure 17-10, and click Finish.
7. Monitor your Recent Tasks pane and your inventory for the results of the Storage vMotion.
Configuring Virtual Machine Swap File Location
This might seem like a funny place to discuss the VM swap file, but there is a method to my madness, and it’s not just because of the exam blueprint this time. The VM swap file is created when a VM is powered on and deleted when a VM is powered off. Its size is equal to the available memory of the VM (what the guest OS is told that it has) minus the reservation assigned to the VM.
So, what does the VM swap file have to do with vMotion? For vMotion to succeed, the VM swap file that was created by the source host must be visible to the destination host (shared) or must be created on the destination host before the migration can begin. This usually means that the VM swap file is stored in the same location with the other VM files. Now, you might think that having to create a new swap file would slow down migration significantly, and you would be right, unless you used the right kinds of drives for the VM swap files. What is slower than RAM but still much faster than conventional disk? That’s right, solid state drives (SSDs).
If you are using local SSDs, you might want to allow the host to provide the swap file locally. This could provide better performance in the event that the swap file is used when memory is in serious contention. If you are using a conventional disk, make sure that the datastore containing the swap file is shared by the source and destination hosts; otherwise, your vMotion performance might be degraded. You can configure this option on the cluster as a default and then you can make changes further down in your hierarchy by configuring it on the host and even on the individual VM. Figures 17-11, 17-12, and 17-13 show the three places where you can configure the location of your VM swap files.
Migrating a Powered-Off or Suspended Virtual Machine
Again, migrating a powered-off or suspended virtual machine is really two objectives, but they are similar enough this time that I’ll handle them as one. The main thing you should realize is that a suspended VM is still running a CPU instruction set and therefore must meet the CPU compatibility requirements just like a VM that is being vMotioned. Other than that, migrating a VM that is powered off or suspended is incredibly simple, and there are very few limitations. You can even migrate a VM’s state to another host at the same time you migrate its files to another datastore (on either client), as shown in Figure 17-14.
Utilizing Storage vMotion Techniques
I already discussed the main reason that Storage vMotion was created: to make it easier to move the VM’s files from one datastore to another while the VM was powered on. This is likely the main reason that you will use Storage vMotion in your vCenter. However, you might find that some of the options that are given to you when you Storage vMotion can come in handy if you need to make a change but really don’t need to move the VM’s files at all. In addition, one of the side effects might interest you as well.
As you might have noticed when we did the Storage vMotion exercise earlier in this chapter, you were given the choice of virtual disk format on the Storage page of the wizard, as shown in Figure 17-15. What if you just want to change the virtual disk on a VM from thick-provisioned eager-zeroed to thin-provisioned (for example)? Could you move the VM files to another location just to make that change? Yes, you could; in fact, any change in disk provisioning is possible. Then, if you want them back where they were, you can Storage vMotion again. Remember that you are not disrupting the user, at least not directly.
Here’s another example of using Storage vMotion creatively by taking advantage of a side effect of moving the VM files. Suppose that you have decided to change the display name of a VM so that it shows up differently in your inventory. You should know that just changing the display name does not change the VM filenames that are in the datastore. However (as of vSphere 5.5), if you Storage vMotion the VM files to another datastore, the VM’s folder name and the files within it are changed to match that of the new display name that you’ve given the VM. Now, the NetBIOS name and hostname are not changed, and you are on your own there. In fact, that’s a good reason to seriously consider your options before you change a display name on a VM.
Summary
This chapter covered the following main topics:
The requirements for vMotion and Storage vMotion. In addition, EVC, snapshots, and swap files as they relate to vMotion were discussed. You examined the capabilities of Storage vMotion with regard to changing virtual disk type and renaming VMs. I addition, you learned about the newest capabilities of vMotion, such as Long Distance and Cross-Switch vMotion.
The capabilities of Storage vMotion with regard to changing virtual disk type and renaming VMs. Finally, you learned about Cross-Host vMotion and its additional capabilities.
Exam Preparation Tasks
Review All the Key Topics
Review the most important topics from the chapter, noted with the Key Topic icon in the outer margin of the page. Table 17-2 lists these key topics and the page numbers where each is found.
Review Questions
The answers to these review questions are in Appendix A.
1. An administrator uses Storage vMotion to migrate a VM. Which of the following will occur? (Choose two.)
a. The state of the VM will remain on the same host.
b. The VMs files will remain on the same datastore.
c. The state of the VM will move from one host to another host.
d. The VM files will move from one datastore to another one.
2. An administrator of a vSphere network with a Standard license attempts to vMotion a VM but receives an error message indicating the vMotion is not enabled. Which of the following could be the cause of this message?
a. There is no shared storage configured.
b. The VMkernel port intended for vMotion is not enabled for vMotion.
c. There is not enough memory to provide for vMotion.
d. vMotion requires an Enterprise Plus license.
3. Which of the following would prevent an administrator from using vMotion? (Choose two.)
a. The VMkernel port is not enabled for vMotion.
b. The vSphere uses a Standard license.
c. The host does not have visibility to the destination datastore.
d. The CPU of one host is not compatible with the other.
4. Which of the following are part of a CPU feature set? (Choose two.)
a. Host CPU vendor, family, and model
b. VM CPU vendor, family, and model
c. ESXi version running on the host
d. Power state of VM
5. Which of the following would prevent long-distance vMotion from operating properly? (Choose two.)
a. RTT of 50ms
b. A vSphere Enterprise Plus license
c. RTT of 160 ms
d. No common Layer 2 environment
6. Which of the following is not a part of a snapshot?
a. Settings state
b. Disk state
c. Memory state
d. VM backup
7. Which of the following are not true in regard to vMotion and Storage vMotion with vSphere 6?
a. You can vMotion a VM that contains snapshots.
b. You cannot Storage vMotion a VM that contains snapshots.
c. You cannot vMotion a VM that contains snapshots.
d. You can Storage vMotion a VM that contains snapshots.
8. Which of the following would prevent a VM from using vMotion?
a. An internal switch on its host, to which the VM is not connected
b. A snapshot that has not been deleted
c. A swap file that is shared between the hosts
d. A VMkernel port that is not enabled for vMotion
9. Which of the following are true regarding Storage vMotion? (Choose two.)
a. By definition, you cannot Storage vMotion a VM that is powered off.
b. You can Storage vMotion a VM, whether it is powered on or off.
c. Storage vMotion will always move the files of a VM to another location.
d. Storage vMotion will never move the files of a VM to another location.
10. Which of the following are not true? (Choose two.)
a. To cold migrate a VM’s files, you must first power off the VM.
b. You can cold migrate a VM’s files with a VM powered on or off.
c. The cold migration of the state of a VM is referred to as vMotion.
d. Cold migration of the state of a VM happens with the VM is powered off.