If you are working in a Windows environment, the first step to setting up PXE booting is to configure your DHCP server. You may already have that server running, so the following steps will just cover what is necessary to configure your DHCP server to point clients to the TFTP server. The following process uses the DHCP server included with Windows Server 2003 R2:

Rather than configuring these options at the scope level, you may want to create a DHCP reservation for your ESXi host. In that case, you right-click on the Reservation node and select New Reservation. You will enter the name, the IP address for the reservation, the MAC address of the adapter in the ESXi host that you will use for the installation, and optionally a description. Once you have created the reservation, you right-click on it and select Configure Options. You can then follow the preceding steps to add the necessary DHCP options. When you have completed your changes, you should see the two-option setup as shown in Figure 4.6.

Figure 4.6. Windows DHCP server configuration for PXE booting.

Once you have configured your DHCP server, you are ready to set up a TFTP server. The following example uses tftpd32, but as you’ll see, it is fairly easy to set up another package or modify an existing TFTP server to allow for the booting of the ESXi Installer.

You can optionally test your setup to ensure that both the DHCP and TFTP server setups are correct. When you boot your ESXi host and select to boot from the network adapter, the NIC should be able to obtain an Internet Protocol (IP) address and then make a connection to the TFTP server. The process should then generate an error as the file pxelinux.0, as you had configured for the Bootfile Name DHCP option, does not exist on the TFTP server yet. The Log Viewer tab for tftpd32 should also show a connection attempt and “file not found” error.

The next step in your setup is to add the necessary PXELINUX files to the TFTP server. You can find a link to download the latest release of PXELINUX from http://syslinux.zytor.com/ or http://www.kernel.org/pub/linux/utils/boot/syslinux/. The download includes the entire SYSLINUX package, but you’ll only need the file pxelinux.o from the download.

At this point, you are ready to run an interactive PXE installation. Power on your ESXi host, and in the BIOS, select to boot from the system’s network card. How you do so varies from system to system, but you can probably press F12 to start a network boot or press Esc to access a boot menu. If the boot menu does not show an option to boot from a network card, you may have to edit the BIOS boot options to enable this option.

Once your host has passed through the BIOS startup and initialized the host’s network card ROM, the NIC attempts to obtain an IP address by broadcasting for a DHCP server. The DHCP server that you had configured earlier should respond with a DHCP lease containing the required IP information as well as the DHCP options for the TFTP server and boot file to use for a PXE boot. Your ESXi host contacts the TFTP server to download and start pxelinux.0, which will in turn load the configuration file that you created. The host’s display should appear similar to Figure 4.7.

Figure 4.7. PXE boot configuration file enumeration and configured boot menu.

Notice that the PXE boot process attempts to load a number of configuration files prior to finding and loading the file default. PXELINUX first searches for a configuration file based on the MAC address of the NIC being used. For a MAC address of 00:0c:29:32:71:00, PXELINUX searches for a configuration file called 00-0c-29-32-71-00. If the program finds such a file, your host uses that configuration instead of searching for other files. Note that the file should have all letters in lowercase. If a configuration file based on the host’s MAC address is not found, a search is made for a configuration file based on the hexadecimal conversion of the host’s IP address. In the case of Figure 4.7, the host has an IP address of 192.168.1.52. This converts to a value in hexadecimal of C0A80134. PXELINUX starts a search for that file, and if it finds no match, one hex digit is removed to see whether a further match can be found.

The PXELINUX search pattern opens the possibility for you to tailor configuration files on a per-host or subnet level. As you’ll see later in this chapter, a kick start script (highlighted with boldface in this listing) can be added to the configuration file to automate the installation process. In such a case, you might create a sample configuration file such as the following and name the file to match the MAC address for the host to which it will apply:

default Local
prompt 1
timeout 5
label ESX05
kernel esxi41/mboot.c32
append esxi41/vmkboot.gz ks=http://server02.mishchenko.net/ks/esx05.cfg
   --- esxi41/vmkernel.gz --- esxi41/sys.vgz --- esxi41/cim.vgz
   --- esxi41/ienviron.vgz --- esxi41/install.vgz
label Locallocalboot 0x80

The file would be saved as 00-0c-29-32-71-00. When the ESXi host boots, no menu would be displayed, but entering ESX05 would start a scripted install process. If you press Enter or wait five seconds for the timeout to expire, the host would boot from the server’s primary boot device.

In the following section, you’ll see how to set up a Linux host to run the DHCP and TFTP roles to allow for the running of the ESXi Installer via a PXE boot. If you’re adding another DHCP server to your network, ensure that it will not conflict with an existing server. If you are planning to isolate your management network for your VMware ESXi host, you may find it a better solution to use an existing DHCP server on your main network. It is likely the case that the router providing access to the management network can be configured to provide DHCP relay, removing the need to have a dedicated DHCP server on the management network. If you plan to place your TFTP server on an open network, consider the fact that TFTP provides no security, and installation scripts, as you’ll see later in this chapter, contain sensitive configuration information.

The setup in the following pages also makes use of gPXE. gPXE was formerly known as Ether-boot and is an Open Source PXE bootloader. gPXE works with the built-in PXE support that your hosts will have, but extends PXE with the use of additional protocols such as HTTP, iSCSI, and ATA over Ethernet. Using gPXE to boot your ESXi Installer is significantly faster than using PXELINUX and a TFTP server for transfer of the installer files. While TFTP is simple and lightweight to implement, the protocol is not designed to be robust. When transferring large files, the TFTP server may suffer from reliability issues, resulting in the failure of the ESXi installation process. With gPXE, only the gpxelinux binary file and configuration file are transferred to your ESXi host from the TFTP server. gPXE can instead use a Web server for the transfer of the kernel and ramdisk required to boot the installer. This will ensure more reliable network transfers under heavy loads.

There are obviously several options that you can choose for a Linux host. The following example uses an Ubuntu Server VMware image from www.thoughtpolice.co.uk, which hosts a number of VMware images that you might find useful.

To set up the Linux image and configure DHCP and TFTP servers, use the following process:

Now that your DHCP and TFTP servers are running, you’re ready to configure the components to gPXE boot the ESXi Installer. As you’ll have seen in the Windows section, the first step to enable this is to configure the DHCP server with the necessary information for the ESXi host to find the TFTP server.

You start this process by editing the file /etc/dhcp3/dhcpd.conf. As you are using gPXE, you add the following text to the file. The next-server option specifies the IP address or hostname for the TFTP server and filename tells the client ESXi host which file to load from the TFTP server.

allow booting;
allow bootp;
# gPXE options
option space gpxe;
option gpxe-encap-opts code 175 = encapsulate gpxe;
option gpxe.bus-id code 177 = string;
class "pxeclients" {
match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
next-server 192.168.1.9;
if not exists gpxe.bus-id {
filename "/gpxelinux.0";
}
}

After you have changed your configuration, run the command sudo /etc/init.d/dhcp3-server restart to restart the DHCP server. If you plan to use PXELINUX with your setup, the configuration change to dhcpd.conf is as follows. The rest of the setup for PXELINUX is the same as described in the Windows section, except that you place the files in the default location of /var/lib/tftpboot/ on the Linux host.

allow booting;
allow bootp;
# PXELINUX options
class "pxeclients" {
match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
next-server 192.168.1.9;
filename = "pxelinux.0";
}

The next step in the process is to set up gPXE on the TFTP server. As mentioned earlier, only the gpxelinux.o binary and the configuration file need to be on the TFTP server. The files for the ESXi Installer can be hosted on a Web server or other server that gPXE will support.

You are now ready to test your configuration. Boot up your target ESXi host, and in the BIOS, select to boot from the system’s network card. How you do so varies from system to system, but you probably will be able to press F12 to start a network boot or press Esc to access a boot menu. As was shown in the prior section, the host’s NIC obtains an IP address and the necessary information to connect to the TFTP server. In this case, the ESXi host downloads gpxelinux.o and a configuration file to start the PXE installation process.

As shown in Figure 4.9, gPXE searches the pxelinux.cfg folder for configuration files based on the MAC address of the ESXi host and the IP address assigned to it. As discussed earlier, this provides the opportunity to configure a custom gPXE install menu for a specific host or subnet. Once you have selected the gPXE menu choice, gPXE begins to load the appropriate files. Figure 4.10 shows the files being loaded from a Web URL rather than the TFTP server, as was the case when setting up PXELINUX.

Figure 4.9. gPXE enumerating configuration files.

Figure 4.10. gPXE loading ESXi Installer files from a Web server.

If you’ve been running installation scripts for vSphere ESX, you’ll likely already have a set strategy for running installation scripts. With little difficulty, you’ll be able to modify those scripts to run on ESXi. If you’re new to scripted installations for vSphere, you’ll need to give some thought to whether to run a single install script for all your ESXi hosts or to manage individual scripts for each host.

If you create individual scripts, you will be able to set a static IP address for each ESXi host you’ll install this way. In the section “PXE Booting the ESXi Installer” in this chapter, you learned how you can configure PXE booting to use a specific PXELINUX or gPXE configuration file for each host you have.

If you choose to use a custom script for each host, you can then configure PXELINUX or gPXE to use a specific ESXi installation script for each host. Should you choose to maintain a single script, you’ll need to use DHCP for the management network and then change that to a static IP address after the installation. VMware recommends the use of static IP addresses for your ESXi hosts. You could use a static IP address in a single script for all your host installations, but if you were to fail to change the IP address of a host after an installation, this could lead to a duplicate IP address between two of your hosts and you would not be able to manage the one host over the network.

You will also have to decide where you will store your installation scripts. You’ll be able to use the following storage options:

If you plan to use NFS, the ESXi Installer attempts to connect to the share with the login of root and a blank password.

The storage option you choose for your installation script determines the syntax of the ks command that you enter on the VMware VMvisor Boot Menu screen. Table 4.3 lists the available options.

To enter the ks command, you’ll follow the process used for the default installation script shown earlier in this section, updating the URL and storage type to match where you have stored your script.

Along with configuring the ks bootstrap command, you’ll also be able to configure the networking setup used for your ESXi installation process. These additional options are summarized in Table 4.4.

With the additional bootstrap commands, you can either edit the PXELINUX or gPXE configuration files you use for your installations or add them manually at the VMware VMvisor Boot Menu, as shown in Figure 4.18.

Figure 4.18. The ESXi Installer applying an installation script.

If you’ve written kickstart scripts for vSphere ESX, the command options in the following section will be familiar. But you should be aware that the partitioning commands autopart, clearpart, and part are different from what you would use with kickstart scripts. If your current ESX scripts set specific sizes for the system partition, you will need to change that. As there is no Service Console to configure with ESXi, the ESXi Installer will size all the system partitions.

If you’re creating a new script, you will have to include the following four commands, as they are considered required. If one of these options is missing or incorrectly formatted, your automated installation will generate an error and will not be able to proceed.

autopart --firstdisk=local,remote --overwritevmfs

install url http://server02.mishchenko.net/Media_depot/ESXi_4.1/

sudo grub-md5-crypt
Password: ********
Retype password: ********
$1$E4XdT/$xYUqVgsVE1gEeyTqLazX41

paranoid.

The default behavior of the installation script process is to continue if warnings are encountered, but to halt on errors. If your script were missing the install command, this would generate an error and the installation process would terminate. But if the script were to set a static IP address but did not include the nameserver flag, it would generate a warning and you would be able to proceed with the installation, as shown in Figure 4.19. If you include the paranoid command, the script terminates when it encounters any error or warning.

Figure 4.19. An ESXi installation script warning.

%include partition.cfg

%pre --intepreter=busybox --unsupported
cat > /tmp/part.cfg <<EOF
part datastore2 --fstype=vmfs3 --size=200000 --onfirstdisk="remote"
part datastore3 --fstype=vmfs3 --size=100000 --grow
   --ondisk="mpx.vmhba1:C0:T0:L0"
EOF

# Disk Partitioning
%include partition.cfg

The following sample script makes use of the commands found in the previous section to automate the installation of VMware ESXi. The required commands of vmaccepteula, rootpw, install, and autopart are included. In this case, the root password has been encrypted. The install command is set to use an HTTP media depot for the installation disk image. The autopart command tells the installer to use the first disk found and to overwrite any existing VMFS datastore.

A number of optional commands are also included. The keyboard command is set to be Russian and a serial number is set for the host. The serial number is validated by the installer, so if you use an invalid serial number, such as the one in the sample, or have not properly formatted the serial number, an error will be generated when the script is initially parsed at the start of the automated installation process. The script also uses the paranoid command, which ensures that the script is parsed without any warning; otherwise, the installation will be aborted.

The network command is used to set a static IP address for the host, the hostname, and DNS servers to use. The vmnic to use for the management port is not specified, so vmnic0 defaults to this purpose. The commands clearpart and part are used to clear the partitions on two other disks in the ESXi host and then create new VMFS datastores on them. The size for the datastores is set to 1200MB, which is the minimum value that you can use, but the --grow flag instructs the installer to use all available space on those disks.

The last part of the script uses the %firstboot command to execute a script during the initial boot of the new ESXi installation. As mentioned earlier, this portion of the script is not checked during the installation process, so you won’t know of any errors until after the first time the host has booted up. The %firstboot command is used to include console commands that will create a new vSwitch with vmnic2 and vmnic3 and then add a virtual machine port group called DMZ on VLAN 200. The script then mounts an NFS datastore and finally takes a configuration backup with the installer stores on the NFS share.

# Accept the VMware End User License Agreement
vmaccepteula

# Set the root password for the DCUI and Tech Support Mode
rootpw --iscrypted $1$vFXeT/$oqg2GUIgz9ucneF4YsKq9.

# Install from media depot
install url http://server02.mishchenko.net/Media_Depot/ESXi_4.1/

# partitioning command
autopart --firstdisk --overwritevmfs

# set keyboard type
keyboard Russian

# set serial number
serialnum --esx=10ADP-14OE5-NZO08-0ODH2-821KJ

# abort script if any warnings are generated
paranoid

# Network settings for the management port
network --bootproto=static --ip=192.168.1.34 --gateway=192.168.1.1
   --netmask=255.255.255.0 --hostname=esx05.mishchenko.net
   --nameserver="192.168.1.5,192.168.1.6"

# formatting of other datasources to maximum of LUN size
part datastore2 --fstype=vmfs3 --size=1200 --grow --ondisk=mpx.vmhba1:C0:T1:L0
part datastore3 --fstype=vmfs3 --size=1200 --grow --ondisk=mpx.vmhba1:C0:T2:L0

# initial configuration after install process
%firstboot --unsupported --interpreter=busybox
esxcfg-vswitch --add vSwitch1
esxcfg-vswitch --link=vmnic2 vSwitch1
esxcfg-vswitch --link=vmnic3 vSwitch1
esxcfg-vswitch --add-pg=DMZ vSwitch1
esxcfg-vswitch --pg=DMZ vSwitch1 --vlan=200
esxcfg-nas --add --host 192.168.1.100 --share /NFS2 NFS2
sleep 60
cd /vmfs/volumes/NFS2/
mkdir ESX05
backup.sh 0 /vmfs/volumes/NFS2/ESX05

If there is a problem in your script, a warning or error message will be displayed after the ESXi Installer has parsed the configuration script. A sample of a warning is shown in Figure 4.20. In this case, the network command was not found in the script, but the installation process would be able to continue after a timeout had expired or you had pressed the Enter key to continue. If you had used the paranoid command in the script, the installation process would have terminated and prompted you to reboot the host. If your script has an error, such as missing the install command, you would be informed of the error and only be able to reboot the host.

Figure 4.20. A warning message during an automated ESXi installation.

In most cases, such as the error in Figure 4.20, the error or warning message will be self-explanatory and you’ll be able to correct your script based on the information provided. That may not always be the case; at the conclusion of the script, you may be informed that a warning was generated and that you should review the log file for more information.

To begin gathering information about the script problem, you can access the VMkernel log file. At the console of the host or in your remote control session, press Alt+F12. The display will change from the installation DCUI screen to display the VMkernel log, as shown in Figure 4.21. The file displayed on this screen corresponds with the file /var/log/messages on the host. You can scroll back and forth through the log file to see the commands run by the installation script and any warnings or errors that were generated. You can press the H key to access a help menu and then Alt+F2 to return to the main installation screen.

Figure 4.21. Accessing the VMkernel log file during an installation.

You may also want to access the console to open the installation log file. In some cases, the installation may complete, but the program will inform you that there were warning messages and that you should check the log file. To get to the log file, press Alt+F1 to access the console. You will be prompted for a login and you can enter root. The password will be blank. Once you have logged in, you will have access to the console, as shown in Figure 4.22.

Figure 4.22. Accessing the ESXi Installer console during an installation.

You’ll find the log files in the folder /var/log. The messages file will be the VMkernel log file that you can access by pressing Alt+F12. You will also see the file esxi_install.log, which is a symlink to the install log file weasel.log. You can view the contents of those files with the command cat or the vi editor. As the ESXi Installer will have initialized a network connection, you could optionally copy the log files to another host with the scp command. Alternatively, you could enable Secure Shell (SSH) on the host by following these steps:

Once you’ve completed these steps, you’ll be able to connect with your preferred SSH client and download the log files that you need to troubleshoot your issue. If you’re having problems with disk formatting, you could run the command fdisk –l to identify the device names corresponding to the formats provided in Table 4.6.

If you’re troubleshooting a problem with the %firstboot portion of the script, you’ll have to wait until after the first reboot to access the log files. After the reboot, you can enable Tech Support Mode to access either the local console or SSH, as described in Chapter 3. Once you have access, you can check the install log file /var/log/esxi_install.log. As you’ll see later, /var/log is part of the ESXi ramdisk and files within that folder are not preserved across a server restart. In this case, however, the installer bundles the log file into the file onetime. tgz and stores that file on the system partition that will be mounted as /bootbank when the host next reboots. The bootstrap command to start the VMkernel on /bootbank includes a link to this file, so the log file will be accessible after the first reboot following installation. The esxi_install.log file contains elements such as the following.

The onetime.tgz file also contains the file /etc/vmware/init/init.d/999.firstboot_001. If you used the ––level flag with the %firstboot command, you will see that reflected in the filename. If you have used the scripting command keyboard, you will also see 999.firstboot. keyboard, 999.firstboot_license for the license command, and, as a required command, 999.firstboot_password for the password command. There is also a file called 999. firstboot_remove.

If you’ve used busybox as the interpreter for your %firstboot script, you’ll be able to open the file and review the syntax of your commands. The keyboard and license files contain scripts that manipulate configuration data within the /etc folder. The 999.firstboot_password file executes the python script /var/lib/firstboot/999.firstboot_password.py. This script in turn edits /etc/shadow to insert the MD5 hash for the password you used in the installation script.

The 999.firstboot_remove script cleans up your %firstboot script and other scripts for commands, such as the keyboard and password commands, which are run after the install process. The script starts the python script /var/lib/firstboot/999.firstboot_remove.py, which edits /bootbank/boot.cfg to remove the bootstrap entry to the onetime.tgz file and also deletes the onetime.tgz file.