CHAPTER

12

Configuring the Oracle
VM Server Network

The two most important components of the Oracle VM server that are most likely to be modified on a regular basis are network and storage. The network configuration tends to change as the requirements for network bandwidth and subnets change. Also, within most environments, the need for storage always grows and never shrinks. Network requirements typically change at the virtual machine level, but the Oracle VM Server network does not need to change very often, except as needed to support new virtual LANs (VLANs). The configuration and functionality of the network are important because the network is the foundation for accessing virtual machines. By making sure that network capacity has not been exceeded, you can optimize the performance of the virtual machines.

This chapter covers the Oracle VM Server network—how it is configured and how it is managed. Once it is configured and working properly, making significant changes is often unnecessary. The VM Server network configuration requirements are simple to manage once you understand the basics of networking technologies—mainly bonds and VLANs.

What Makes Up a Network?

Virtual machines can share and use the networks configured on the host Oracle VM server; therefore, for multiple networks to be used with virtual machines, each virtual network must be created for the Oracle VM server.

Although the physical configuration is simple (just a few Ethernet wires), the logical configuration can be far more complex, as shown in Figure 12-1. The basic building block of the network is the Ethernet port. It is not uncommon to have multiple physical ports bonded into a single logical port. The new logical port can then have multiple VLANs stacked on top of the “port,” with bonds, VLANs, and then finally virtual interfaces layered on top. Although the logical configuration may initially seem daunting, once you understand the building blocks involved, it rapidly becomes simple to understand and design.

FIGURE 12-1.    Sample network topology

All configuration is performed through the Oracle VM Manager (OVMM)—from the initial configuration of network on each OVM server, to the assignment of virtual network interfaces to the Oracle VM guests. To set up networking, perform the following steps:

1. Configure the initial networks on the Oracle VM server (OVS).

a.   Set the OVS for switched or bridged network management.

b.   Configure bonds for failover (recommended).

c.   Configure the maximum transmission unit (MTU) as needed.

d.   Add in optional IP addresses for troubleshooting.

2. Configure the networks in OVMM.

a.   Select what ports will be used on each OVS.

b.   Assign predefined network uses.

c.   Create and assign VLAN interfaces (optional).

Configuring the Oracle VM network is not extremely complicated. However, to understand how the Oracle VM network works, you need to understand networking, bonding, and VLANs.

Oracle VM Networking

The networks used by virtual machines are run through Xen bridges or switches. The Xen interface is the connection between the virtual world (virtual machines) and the physical world (your company network). You can configure Xen bridges in a number of different ways, including bridged, NAT, and routed. Although you can configure all these network types and they will work, only a bridged or switched network is supported by Oracle. Routed and NAT networks are usually found in smaller, home network environments, due to most enterprise environments providing these services as dedicated devices such as firewalls and routers. The bridged network is most commonly found in server environments. Because Oracle VM is designed for the data center, bridged networking is typically the network type of choice, with the switched configuration being popular in more security conscious environments. There are four key network concepts to understand:

   Bridged networking    Bridged networks allow the virtual machine network to reside on the same network as the Xen bridge. In order to be seen by other systems on the network, the virtual machine network adapter must be configured so that it’s on the same subnet as the dom0 xen bridge (xenbr) adapter. Bridged networks allow you to easily add new virtual machines to the virtual machine farm that are accessible from other systems on the network. The downside of using a bridge is that all VMs on the bridge can sniff the network traffic of other nodes.

   Switched networking    Switched networks perform the same functionality as the default bridged configuration. The main difference is that the ability of one VM to sniff the traffic of another VM is limited.

   NAT networking    A Network Address Translation (NAT) network takes the IP address of the Oracle VM server and allows multiple virtual machines to appear to both themselves and other virtual machines as if they have their own IP addresses. However, to the outside world, it appears as if there is only the Oracle VM server’s IP address. This is similar to how your home router works. With NAT networks, connecting outside systems to the NAT system is difficult.

   Routed networking    With routed networking, traffic is routed through the dom0 (Oracle VM Server) system to a private subnet that is accessible by one or more virtual machines. Routed networks must be set up manually and are more difficult to maintain, but they provide functionality that might be useful in some cases.

Network Bonding

A degree of redundancy at the network level is often desirable. With redundancy, the Oracle VM Server system continues to operate even in the event of a single network failure. Common practice is to use network teaming, or bonding, to provide this extra bit of protection. Network teaming and network bonding are just different terms for the same thing: network redundancy. For the sake of this chapter, the term network bonding is used to describe this function.

If you use active switching, you can use network interface controller (NIC) bonding to increase the throughput of a VM network. For instance, if you have many virtual machines, a single 1GbE network connection might not provide enough throughput. By increasing the number of NICs in the bond, you increase the throughput. Although having multiple 1GbE ports sounds like a good idea, each communication path between a VM and another system on the network is still limited to a 1GbE connection, even if the aggregate is more than a single 1GbE pipe. This is one of the reasons why 10GbE networks have become so popular. One advantage with Oracle servers is that almost all models have four built-in 10GbE ports.

In addition to the physical interfaces in the network stack, each physical interface can be combined in what is called a bond. When you install an OVS, bond0 is automatically created on it, with the management IP already allocated. As shown in Figure 12-2, you can see the same bond0 on a newly installed server. You can navigate to this page by clicking the Servers and VMs tab and then selecting the physical server. From there, select Bond Ports from the Perspective drop-down. In this example, the server ovs3.m57.local was assigned an IP address of 192.168.200.26.

FIGURE 12-2.    Default bond

Although a bond with a single port works, adding a second port to the bond will provide several advantages. To understand these advantages, we first have to look at the type of bonds available:

   Active/Backup    This is the default bond type, and it provides an automatic failover between the ports in the bond. Its major advantage is that it is simple to configure because nothing special has to be done of the switch side. This bond type is recommended for all types of traffic, including VLANs.

   Link Aggregation    This bond type not only provides the highest level of uptime for the connection but can also improve network performance because multiple physical ports are combined into a single logical interface. The downside is that the switch ports must be configured correctly, using the IEEE 802.3ad Dynamic Link Aggregation standard (also known as LACP). This complexity also imposes some physical limitations because most switches require that all Ethernet ports in an LACP bind be connected to the same physical switch. This solution also works well with VLANs but is not recommended for the management bond.

   Load Balanced    This technology is a hybrid of Active/Backup and Link Aggregation. It can provide some load balancing between the physical ports in the bond (VLAN technology is notoriously unreliable). Its main advantage is that no switch configuration is required to support the technology. It is often used for management bonds, especially when live migration traffic traverses the same bond. This bond mode does not work at all with Xen bridges that are associated with a VLAN device, so it cannot be used as an interface for NetFront devices on Oracle VM guests.

To modify bond0 to a Load Balanced configuration, highlight bond0 from the Bond Ports perspective view and then click the edit icon, as shown in Figure 12-3.

FIGURE 12-3.    Selecting a bond to edit

Next, you will see a dialog (shown in Figure 12-4) that details how the bond is configured. This dialog includes the following information:

FIGURE 12-4.    Bond configuration dialog

   Bonding    The bonding method used.

   Addressing    Static, Dynamic (DHCP), or None.

   IP Address    If configured for a static IP address.

   Netmask    If configured for a static IP address.

   MTU    The maximum transmission unit defines the largest block of data that can be sent in a single network packet. This is set to 1500 by default, but is often set to a larger value on 10GbE networks. All servers on the network must have the same MTU setting.

   Available Ports    What ports are available to be added to the bond.

   Selected Ports    What ports are currently in the bond.

To add a port, simply double-click it. In the example shown in Figure 12-5, we see eth1 being added to the ports and Bonding being set to Load Balanced.

FIGURE 12-5.    Reconfiguring the management bond

Additional bonds can be created. In this example, bond1 will be created on two Oracle VM servers, using both Link Aggregation and VLANs. Prior to the bond being configured, the switch was configured and wired to ovs3.m57.local and ovs4.m57.local, with ports net2 and net3 on the switch side being configured into a single bond per pair. In addition, VLANs 200–210 were configured to work on the aggregates.

The next bond device, bond1, will be created on both OVS3 and OVS4, with link aggregation being used for the bond mode, and the VLANs riding on top of the new bond1 device. On both servers, bond1 will be created. You do this by going back into the Bond Ports perspective on both nodes and using the “+” icon to add a bond. Because VLANs will be used, no IP address is added to the bond. When this is completed, the dialog should look like Figure 12-6.

FIGURE 12-6.    Creating a bond port

After bond1 is added, it should show in the list of bond ports, and if the switch is configured correctly, it should have a status of “up,” as you can see in Figure 12-7.

FIGURE 12-7.    Bond port status

Network VLANs

Often, when building large or complex private clouds, the number of networks that virtual machines need access to will exceed the number of physical ports of the Oracle VM Server. In order to support this, the network industry created a standard (IEEE 802.1Q) that allows for multiple virtual local area networks (VLAN) to share the same physical cable. These VLANs also can share a bonded connection, just as created in the previous section.

The next step will be to create the initial VLANs. VLANs allow multiple networks to use the same physical connection. This enables efficient use of the network ports. To enable VLANs on the OVS systems, go to the Network tab in OVMM and select the VLAN Interfaces option. Initially, Oracle VM will not create any VLANs, so the system should show empty, as you can see in Figure 12-8.

FIGURE 12-8.    Initial VLAN configuration

To create the VLANs, select the green “+” symbol, which will start a guided process for creating them. First, we need to identify the ports the VLANs will use. In this example, we select bond1, as shown in Figure 12-9.

FIGURE 12-9.    VLAN port selection

Next, the VLANs will need to be created. A VLAN is identified with a number between 1 and 4094. The network administrator will know the VLAN numbers and should inform the OVM administrator what numbers to use. In our example, we will create VLANs 200–210 in bulk by entering in the numbers and clicking the Add button. Figure 12-10 shows the dialog after these VLANs have been added.

FIGURE 12-10.    VLAN numbers

The final part of the process allows each VLAN to have an IP address set for each server, as well as a unique MTU set. Having an IP address on the OVS makes troubleshooting VLAN configurations much simpler, so this is highly recommended. If you are working with a large number of VLANs, it is recommended that you only add a few at a time, because the complexity can quickly become difficult to manage, as seen in Figure 12-11.

FIGURE 12-11.    VLAN IPs

Once you click Finish, an OVM job will run, adding the VLANs to all the Oracle VM servers that were selected. Depending on the number of VLANs added, this can take several minutes. Once the job completes, the VLAN Interfaces screen will list all the available VLANs, as shown in Figure 12-12.

FIGURE 12-12.    VLAN summary

Configuring the Network Adapters

The first step in setting up networking is to set up the adapters and bonds on each OVS. This is all performed from the Oracle VM Manager. Once you have set up the OVS network, you’ll define the networks in OVM, allocating how each network is used.

The primary network adapter is configured during the installation process and is created as a bond with an Active/Standby configuration. The IP address and type of address assignment are determined during the initial installation setup. Using static networking is recommended instead of using DHCP-assigned IP addresses.

Configuring Networks

Once the bonds and VLANs are created, the next task is to assign a role to each virtual network. As shown in Figure 12-13, each network can be used for the following five predefined roles (under “Network Channels”):

FIGURE 12-13.    Network roles

   Server Management    This network role is used for management access from the OVMM to the OVS systems. The traffic should be routed (NAT access to this network is not supported). Only a single network can have this functionality on a single OVM server; however, you can have different management networks across different OVM servers.

   Cluster Heartbeat     This network role is used by the OVS systems to communicate via TCP packets. The cluster is sensitive to network stability and should not be shared with roles that can cause spikes in network traffic, such as Live Migrate and Storage. The network with the heartbeat should have some redundancy. Usually this is the same network used for Oracle VM Server management. This cannot be changed once the server pool has been created.

   Live Migrate    The Live Migrate network role is used to migrate virtual machines from one OVS to another in a server pool without a VM outage. In most environments, live migration should not be occurring frequently because it could cause interruption to other services, particularly the cluster heartbeat. As a result, configuring a separate network for this purpose can improve the performance and availability of the VMs and also prevent cluster heartbeat timeouts due to heavy network traffic.

   Storage    Although the storage network role has no specific functionality, it allows the administrator to quickly identify what networks are used to connect to storage.

   Virtual Machine    Network with this functionality allows VMs to have access to it. This is the most commonly used role for networks. Enabling this role on a network will automatically create a Xen bridge for the network.

Adding a new network is a straightforward task. Click the green “+” icon from the Networks screen to start the guided process, as shown in Figure 12-14.

FIGURE 12-14.    Adding a network, step 1

You have two paths you can take here. The first path is to add a network using physical ports on multiple OVM servers (either physical ports or bonds). The second path is just for single servers. You can enable a local network that allows VMs to communicate between themselves only on that server and no other systems. For our example, we will add a new VLAN, so the default (first) option will be chosen. VLAN 200 will be the name, and it will be configured for VMs and flagged for storage as well. This is shown in Figure 12-15.

FIGURE 12-15.    Adding a network, step 2

If physical ports are being used, they can be added in the next screen. In most cases, though, the network team will use VLANs, so this step will remain empty, as shown in Figure 12-16.

FIGURE 12-16.    Adding a network, step 3

Next, click the green “+” button and select the physical server that will have access to the VLANs. You must highlight the server to see the VLANs available to it. In the example shown in Figure 12-17, the server ovs3.m57.local is selected and the VLAN200 bond is checked.

FIGURE 12-17.    Adding a network, step 4

This single VLAN/server combination will now be added. This same process needs to be repeated for all Oracle VM servers that will be hosting VMs using this network. In the example shown in Figure 12-18, both ovs3.m57.local and ovs4.m57.local have been added.

FIGURE 12-18.    Adding a network, step 5

Click Finish. The Network tab will now list network VLAN200, as shown in Figure 12-19, with both storage and virtual machine access.

FIGURE 12-19.    Network added

Configuring Virtual NICs

This last task is optional but highly recommended in any large organization because it will prevent duplicate addresses. Each virtual machine interface will receive a Media Access Control (MAC) address, which is a unique identifier assigned to a network interface. Outside of a few special cases, all network interfaces on a network should have a unique MAC. The IEEE manages ranges of MAC addresses and can assign ranges to vendors and companies that request them. Oracle has many ranges, and Oracle VM by default will start with 00:21:F6:0:0:0 for new interfaces. In many large organizations, this is broken down into a smaller subset, with some organizations even owning their own range of addresses. When an interface is created, a new MAC is automatically reserved from the Oracle VM Manager. If you need to change the range, you can go to the Virtual NICs option in the Networking tab (see Figure 12-20).

FIGURE 12-20.    Virtual NICs

Here, the range can be edited. Be careful making changes outside of the 00:21:f6 range, because it is possible to accidently create a duplicate MAC address. The most common use case is to reduce the available range within the existing range—usually one subset per each Oracle VM instance in the enterprise (see Figure 12-21).

FIGURE 12-21.    MAC range

In this example, the range has been limited to 00:21:f6:01:00:00 to 00:21:f6:01:ff:ff. Once MAC addresses have been assigned, the same page will show each MAC address as well as the network and VM using it, as shown in Figure 12-22.

FIGURE 12-22.    Allocated MAC addresses

Summary

This chapter covered the configuration of the Oracle VM Server network, starting with Xen networking and ending with the configuration of additional virtual network adapters. You do not need to do a lot of work to configure the network, from a general administration standpoint. As mentioned in this chapter, you configure only the first network adapter as part of the initial installation and then configure other adapters after the Oracle VM server is running. During the normal course of operations, changes do not typically need to be made—only if new adapters are added or networks are changed due to business requirements.