Chapter 6. Cisco Cloud Infrastructure

As you have seen in previous chapters, a cloud infrastructure is composed of a variety of components, from physical switches, storage arrays, and compute node to many virtual appliances. Each of these has a key function in the operation and administration of a cloud infrastructure, and this chapter looks at both the virtual and physical components necessary to build a cloud infrastructure suited for automation and orchestration.

This chapter starts with an overview of the main Cisco and third-party products that can be connected into a Cisco Powered cloud for automation and orchestration. The main virtual and physical components that make up an on-premises cloud infrastructure will be reviewed. The major sections provide an overview of the physical and virtual switches, the virtual appliances, the compute components, storage, and Cisco Intercloud Fabric. We’ll then summarize how these components can be joined together to work in a Cisco Powered cloud.

1. You can connect using SSH into which of the following switches to manage them? (Choose all that apply.)

a. Nexus 1000V Series

b. Nexus 2000 Series

c. Nexus 5000 Series

d. Nexus 7000 Series

e. Nexus 9000 Series

f. All of the above

2. FEX stands for:

a. Fabric Enhanced Extension

b. Fabric Ethernet Extension

c. Fabric Extender

d. Fiber Extender

3. The Nexus 9000 line of switches supports which of the following GE port speeds?

a. 1/10

b. 1/10/100

c. 10/100

d. 1/10/25/40/50/100

e. 10/50/100

4. What are the main components of the Nexus 1000V? (Choose all that apply.)

a. Virtual Ethernet Module

b. Logical Ethernet Module

c. Logical Supervisor Module

d. Virtual Supervisor Module

e. Application Policy Connector

5. AVS stands for:

a. Application Velocity System

b. Application Virtual System

c. Application Virtualized System

d. Application Virtual Switch

6. For an ACI-enabled network, which would be the best product to use to deliver virtual switching services?

a. VMware NSX

b. Nexus 1000V

c. VMware DVS

d. Application Virtual Switch

7. Which of the following are not benefits of VACS? (Choose all that apply.)

a. Easy installation within UCS Director

b. Port-based firewall

c. Zone-based firewall

d. Hardware-based load balancing

e. All the above are benefits of VACs

8. Choose all the valid VACS container types.

a. 2 Tier

b. n Tier

c. 3 Tier Internal

d. 3 Tier External

9. Which of the following can be a VACS container gateway of choice? (Choose all that apply.)

a. The built-in virtual gateway, the CSR 1000V

b. A physical gateway such as one from Palo Alto Networks

c. A logical gateway within the application tier of an n-tier software application

d. Another virtual gateway such as the ASAv, vGW, or vPAN

e. None of the above

10. What level(s) of encryption is supported by Intercloud secure network extension? (Choose all that apply.)

a. AES 128 bit

b. AES 256 bit

c. AES 512 bit

d. AES 1024 bit

e. None of the above

Today, many IT tasks are done manually on an ad hoc basis. For example, new business requirements are often brought to the IT team. As a member of either the storage, network, compute, or virtualization team, you then have to make a variety of decisions. See Figure 6-1. Do we have the capacity on hand today or do we need to order additional gear? What team needs to do their part first? How long will a project to meet these requirements take? What other projects am I currently working on that may impact my ability to accomplish this new project?

In addition to adding new projects and capabilities, you are often faced with having to make daily small adds and changes to your existing infrastructure. Some of these tasks are trivial. Others, such as adding a VLAN across a data center campus, can be fairly daunting in nature due to the high likelihood of human error or misconfiguration. Who hasn’t missed a switch configuration on a device or two during a change control at 2:00 a.m.?

The main goal of automation and orchestration is to allow you, the administrator or architect, to accomplish common tasks quickly and easily, the same way, time after time. Think of automation and orchestration as a very similar concept to a hopefully not too distant in the future concept. How great would it be if you could travel to a remote destination simply by hopping into your autonomous vehicle, inputting a destination address, and letting your vehicle drive you safely and reliably to your destination, giving you the chance to read, surf the web, talk on the phone, or simply attend to other tasks during the driving portion of your journey? All the while, you would have complete control to step in and take over if necessary to adjust your route, respond to an unforeseen situation, etc.

IT automation and orchestration are much like the previously described scenario. You have a mountain of tasks that you face daily in your job. Some are simple, one-step tasks. Others are compound tasks that are fairly order specific and complex. With automation and orchestration, much like the autonomous vehicle, you program in the tasks you want to accomplish and the vehicle takes care of the rest. In a cloud infrastructure, UCS Director is your vehicle. You program it to automate simple or complex IT tasks. You do this in conjunction with members of your organization from the other areas of infrastructure management discipline. The end results are great. You have taken a task or set of tasks that is manual in nature, time consuming, and prone to error and you’ve automated it with a few mouse clicks. More importantly, if there is a problem, you have the ability to roll back these changes immediately.

So to get your mind around the power of automation and orchestration, look at the list of daily/weekly tasks or projects that you face in IT and ask, “Is this something that I could automate? If I automated it, would it save time and allow me to work on more exciting projects?” If the answer to those questions is yes, congratulations, you are well on your way to bringing the wonderful benefits of a Cisco Powered cloud architecture to your organization!

Let’s take a look at the Cisco current switching portfolio.

Prior to Fabric Extenders, management functions were typically all located within the confines of a single chassis. For example, switches were either large (modular) or small (fixed), but configuration happened on just that switch. In many designs, large switches were located in the middle of a data center row or in the end of a data center row and used to aggregate connections from multiple smaller switches. Configurations had to be applied to both the large switch and the smaller switches. See Figure 6-3.

This led to multiple challenges:

Increase in cabling between switch rows

Increased requirements for firmware management

Increased port and switch management

Less scalability

With the advent of the Nexus 2000 Fabric Extender, the Nexus 2000 device acts like a remote linecard to a central Nexus switch. All configuration for all ports gets applied to the single central switch. Also, multiple FEXs are supported per central Nexus switch. This leads to multiple benefits:

Decrease in cabling between switches in rows

Decreased number of firmware management points (because FEX firmware is managed from the parent switch it is connected to)

Decreased port and switch management (one device as opposed to many devices)

More scalability and density

A Fabric Extender gets managed from its parent switch. This makes management much simpler, as shown in Figure 6-4.

As of the writing of this book, Cisco has the following FEX models available (see Figure 6-5):

100/1000 BASE-T: Nexus 2224TP, Nexus 2248TP (48 ports), and Nexus 2248TP-E (48 ports with extended buffers)

10GBASE-T Fabric Extender: Nexus 2232TQ, 2348TQ, 2348TQ-E, 2232TM-E, and 2232TM

10G SFP+ Fabric Extender: Nexus 2348UPQ, 2248PQ, and 2232PP

Fabric Extenders can be mixed and matched in different types to create the connections and port densities required. Be sure to check with the configuration limits of each parent switch to ensure you don’t try to connect to many Fabric Extenders in a topology.

In addition, the Nexus 5000 line supports both Fibre Channel over Ethernet (FCoE) and Fibre Channel (FC) in 1/2/4/8/16-Gbps speeds (support and speed vary per switch model). As of the time of this writing, Cisco had six Nexus 5000 switches available: Nexus 5672UP, 5672UP-16G, 56128P, 5624Q, 5648Q, and 5696Q.

The concept behind ACI is simple yet powerful. Using a set of redundant Application Policy Infrastructure Controllers (APICs), you can define and apply network policy in a single location to literally hundreds or thousands of devices all across your network. Once defined at the APIC level, that policy can easily be deployed across a variety of physical (bare-metal) network ports as well as ports within a variety of hypervisors such as VMware ESX, Microsoft Hyper-V, and Red Hat KVM (see Figure 6-13).

These policies are defined as high-level application connectivity/security needs, but instead of tightly coupling them with a large network the way you previously did on a per physical or virtual port basis, these policies can now be abstracted from the complicated details of a large network topology and then easily applied where needed.

Underneath the concept of the tenant in ACI there exists a context. A context can be thought of as a set of rules for separate IP spaces or virtual routing and forwarding instances. A context is simply a further way to segment and separate forwarding requirements within a tenant. A tenant can also, hierarchically, have multiple contexts.

Within the context you will find a series of objects that define actual applications on the network called endpoints (EPs) and endpoint groups (EPGs) as well as the policies that define the relationships between the EPs and EPGs. These policies are not only access control lists (ACL), but also inbound and outbound filters, quality of service (QoS) settings, marking rules, redirection rules, and other settings that define specific L4–L7 interactions between the EPGs (see Figure 6-14).

Now that we have a frame of reference for how objects are referenced within an ACI environment, we can look at how application network profiles (ANP) work. An ANP can be used to define permissible (or denied) interactions between EPGs. For example, if we imagine a three-tier application of Web, App, and Database, each tier would exist as an EPG and potentially contain endpoints. We will discuss a theoretical application that has five web servers, two app servers, and one database server (see Figure 6-15):

Web EPG, consisting of five web EPs

Application EPG, consisting of two app EPs

Database EPG, consisting of one database EPs

Once you have an idea of how you need the application tiers to interact, you would create the EPGs per tier, then create the policies to define connectivity between EPGs (such as log, mark, redirect, permit, deny, copy, etc.), and then create the connection between the EPGs. These connections between EPGs are known as contracts.

The ANP therefore is composed of a contract or contracts between multiple EPGs containing one or more EPs per EPG.

As your organization transitions from traditional NX-OS and CLI-based administration to an ACI-based network, UCS Director once again can provide a tremendous amount of value to you as many of the ACI- and APIC-related functions that you would perform on a daily basis are available within UCS Director as tasks that can be chained together into workflows to help create self-service capabilities for an ACI-enabled network.

Physical Ethernet switches such as those in the Cisco Nexus family and Catalyst family as well as various third-party switches

Virtual Ethernet switches such as the Cisco Nexus 1000V or Application Virtual Switch (AVS), as well as various third-party virtual switches, including the VMware Distributed Virtual Switch (DVS)

Nexus 9000 APIC, which enables the UCS Director administrator to issue automation and orchestration commands against a Nexus 9000 Application Programmable Interrupt Controller

Other Ethernet network physical and virtual appliances such as Cisco VACS, Cisco CSR 1000V, ASA 1000V, and third-party physical and virtual networking appliances such as an F5 BIG-IP load balancer

Fibre Channel switches including Cisco MDS and Brocade Fibre Channel switches

Fibre Channel and IP-based storage arrays, including many popular arrays from Pure Storage, EMC, NetApp, IBM, and others

Compute platforms including support for Cisco UCS Central, Cisco UCS Manager, Standalone UCS C-Series rackmount servers (utilizing Cisco Integrated Management Controller, or CIMC), as well as third-party servers such as those from HP, Dell, and IBM through standards like IP Management Interface (IPMI)

Before servers were virtualized, life was much simpler for network and server administrators. One (physical) server usually ran one application and consumed one (physical) network port. With the advent of the hypervisor, IT administrators also entered a new chapter in their careers, where multiple servers and applications, each with potentially different connectivity requirements, started to intermingle on a single physical host with a hypervisor installed, such as VMware ESX. With these changing requirements, the days of having a single host with a single application connected to a single switch port were gone. IT simply needed more flexibility. This brings us to the advent of the virtual switch.

As virtualization continued to grow in popularity, the industry started moving beyond the capabilities of a single virtual switch on a single physical host. As multiple physical hosts were joined together into more powerful clusters, the concept of having a single virtual switch tied to a single virtual host was no longer practical because every time a new hypervisor-based host was added to the network, another virtual switch had to be configured.

This led to the advent of the distributed virtual switch (DVS). A DVS sits within the hypervisor of multiple physical hosts, providing distributed networking services between multiple hosts in a cluster. As clusters have grown in size from 2 hosts to now up to 32 or more hosts, having a DVS is no longer a luxury; it is a necessity.

Cisco has two offerings in the DVS category: Nexus 1000V and the Application Virtual Switch. Let’s take a quick look at each.

In the physical switching world, the main Nexus 5000 device acts as the management and configuration plane for building the switch configuration and services. Ports on a Nexus 2000–connected FEX receive their port configuration from the Nexus 5000.

A Nexus 1000V architecture (see Figure 6-17) typically has three main components:

A Virtual Supervisor Module (VSM)

A Virtual Ethernet Module (VEM)

Integration with a virtual management console such as VMware Virtual Center or Microsoft SCVMM

In this model, you log in (usually via SSH/CLI) to the VSM and create a port configuration(s) for a virtual machine (called a virtual Ethernet, or vEthernet, interface). Those configurations then get pushed out from the VSM to VEMs that should receive that configuration.

The Nexus 1000V is fully integrated with UCS Director and comes with many prebuilt tasks required for administering and managing the Nexus 1000V within automation tasks.

Similar in concept, the use case for AVS differs from Nexus 1000V as highlighted in Table 6-2.

So while materially similar, the AVS is targeted for ACI-enabled networks, whereas the Nexus 1000V is the right tool of choice for non-ACI-enabled networks.

Architecturally, AVS has two major components (see Figure 6-18):

The AVS-DVS (Distributed Virtual Switch) on vCenter

An AVS kernel module on each ESXi host that is ACI enabled

In situations where ACI may not yet be deployed but the need still exists for rich networking services to be deployed alongside virtual machines, Cisco Virtual Application Cloud Segmentation (VACS) can be a great fit (see Figure 6-19). VACS offers the following:

Easy installation within UCS Director for automated provisioning and orchestration

Load balancing

Routing (using the Cisco CSR 1000V)

Edge firewall capability (using Cisco CSR 1000V)

Zone-based firewall (using Cisco Virtual Security Gateway)

Virtual fabric (using Cisco Nexus 1000V)

With deep integration into UCS Director, this allows secure segmentation and rapid deployment of applications in virtual data centers by helping to consolidate physical assets onto a single infrastructure that can then be virtualized to provide deep security and network services. In addition, VACS brings all the products previously mentioned into a single unified license, making it easy to deploy, license, and manage all the components included within VACS.

When you are allocating L4–L7 network services physically for virtual machines, service times typically increase as complexity increases and many points of management have to be configured in a solution. For example, if a particular virtualized server and application are using network load-balancing services as well as firewall services that are physical, you would be required to not only procure, rack, stack, and configure those physical devices, but also create configurations for your virtual services on each independent device (see Figure 6-20).

Alternatively, leveraging virtual segmentation with VACS allows single-click installation within the Cisco Powered cloud framework (within UCS Director) and allows you to focus on one place to go to set up best-in-class virtual networking and security services. This is accomplished easily with wizards (see Figure 6-21).

Virtual network and security services templates for application workloads

Topology configurations designed for logical secure isolation and compliance

Exposed through the UCS Director GUI to allow rapid and consistent provisioning of secure applications and rich L4–L7 services

Containers can be thought of as situation-specific templates that govern connectivity, security, and services in a multitiered virtualized application environment. Again, using our previous example of a three-tier application consisting of a web tier, an application tier, and a database tier, let’s look at two different types of VACS containers, the VACS 3-tier internal container template and the 3-tier external container template (see Figure 6-22). A third example would be a VACS custom container template, which won’t be covered here.

Deploying virtual web servers and configuring the application layers of the VMs

Deploying virtual application servers and configuring the application layers of those VMs

Deploying virtual database servers and configuring the application layers of those VMs

Deploying a physical firewall and configuration application connectivity policies between the web tier, app tier, and database tier of the preceding three virtual systems

Determining the best way to route traffic securely through each application tier into the firewall and applying the appropriate firewall rules to each step of the application

Inserting load-balancing services in front of the web tier to provide proper load balancing

This would be quite a bit of configuration, just to get that one instance of the three-tier application up. Whenever another application owner came along with the same request, you would find yourself completing the same steps yet again (and again as more application requirements came to you from the business side of your organization).

Using one of the defined VACS 3-tier internal container templates, you could simply click the container template and create a new container for the application. All the proper rules for firewalling, load balancing, and routing would be predefined within the 3-tier internal container template via policy (see Figure 6-23).

As you can see, given that security requirements are not quite as tight for this internal application, connectivity from the Virtual Security Gateway (VSG) zone-based firewall flows toward the web zone’s load balancer and, from there, to the app zone and DB zone. Now you have a template that you can predictably and repeatedly invoke whenever a new three-tier application requirement comes your way.

As previously mentioned, it is also very straightforward to create custom VACS Container Templates for one-tier, two-tier, three-tier, or n-tier applications and define communication, security, load balancing, and other requirements between the different zones in the VACS custom container template. Saving this template will then give you a quick way to consistently and predictably deploy similar application services should the need arise in the future.

For example, as shown in Figure 6-25, VACS can use

The built-in virtual gateway, the CSR 1000V

A physical gateway, such as one from Palo Alto Networks or Checkpoint, or a Cisco ASA

Another virtual gateway, such as the Cisco Adaptive Security Virtual Appliance (ASAv), Juniper Virtual Gateway (vGW), or Virtual Palo Alto Networks (vPAN)

When you’re deploying VACS and choosing the gateway of choice, it’s important to know how to operationalize this choice. You will need the gateway IP and you will get it installed as part of the VACS report after you spin up a VACS container from a template. You can use this information to program in any device (physical or virtual) of your choice as mentioned in the previous list.

Therefore, any good cloud solution should not only address multiple clouds (on premises and public as well as hybrid), but also different capabilities for configuration of an on-premises or public cloud infrastructure across a wide variety of public cloud services. Such is the role of Intercloud Fabric.

As with any strategy, the decision on where to host IT infrastructure (on premises or in the public cloud) boils down to a variety of factors. Some of them may be regulatory (such as needs around data sovereignty), or economic (workloads that are highly elastic or only needed during a short period of time each month/quarter/year). Figure 6-27 shows some of the criteria that go into choosing where to host a particular workload in a hybrid cloud deployment.

This gives you the ability to address public cloud resources in much the same way operationally as you do your internal private cloud resources. Hybrid really provides the best of both worlds. Host-critical or latency-sensitive applications and virtual machines on premises “burst or flex” less important (or seasonal) workloads out into the public cloud. This provides many benefits:

Layer 2 IP addressing used within your organization can be securely extended into various public cloud spaces, allowing VMs to use similar IP addresses as they would use internally.

It enables integration with Prime Service Catalog to allow IT to become the broker for public cloud services. Today your IT team may be largely unaware of the extent and use of public cloud services. With Intercloud Fabric, you become aware of public cloud projects and can quickly and securely offer these services to your internal consumers of IT.

Security and policy can be set within the organization and applied/enforced into public cloud services.

Virtual machine portability and the ability to move virtual machines from on premises to public cloud providers and back.

Automation, including complete VM lifecycle management, automated operations, and an open API for easy programmatic access.

Let’s look at how this is accomplished.

Cisco Intercloud Fabric Director

Cisco Intercloud Fabric Secure Extender

Cisco Intercloud Fabric Director is used to provide workload management in your hybrid cloud environment by allowing you to not only create, manage, and extend network services between private and public clouds, but also offer policy and a self-service IT portal that makes it easy for you to define and provide services (such as virtual machines and the secure network policies associated with them) to your end users. This allows you to publish catalogs to your internal end users that will rapidly allow them to request, for instance, different virtual machine types in different public cloud providers.

Shadow IT: In many cases you don’t have visibility into public cloud resources being used by your organization.

Compliance: Your job of securing access to different resources gets obfuscated once workloads move from your purview/control on premises to public cloud infrastructures.

Different IP/VLAN/Layer 2 address ranges: If, for example, an application is rapidly prototyped externally in a public cloud space, how would you easily bring it back on premises to move it from dev/test into production?

Intercloud Fabric solves these and many other challenges you face in interacting with public cloud services. For example, Intercloud Fabric provides these solutions (see Figure 6-30):

Secure Layer 2 network extension from on premises to dozens of public cloud providers

Automated workload mobility to and from public cloud providers

The ability for you to manage both on-premises and public cloud resources through a single management console

In the end, this helps you meet enterprise compliance more easily because secure networking and policy get extended into a myriad of public cloud offerings. Additionally, this gives you and your organization freedom of public cloud provider choice and prevents cloud provider vendor lock-in.

Application Policy Infrastructure Controller (APIC)

Application Centric Infrastructure (ACI)

Catalyst

Nexus

Fibre Channel over Ethernet (FCoE)

Multilayer Director Switch (MDS)

Fabric Extender (FEX)

virtual device context (VDC)

in-service software upgrade (ISSU)

ACI tenant

ACI endpoint group (EPG)

ACI contract

Virtual Supervisor Module (VSM)

Virtual Ethernet Module (VEM)

Application Virtual Switch (AVS)

Virtual Application Cloud Segmentation (VACS)

software-defined networking (SDN)

Nexus 1000V (N1KV)

Virtual Security Gateway (VSG)

Cisco Adaptive Security Virtual Appliance (ASAv)

Juniper Virtual Gateway (vGW)

Virtual Palo Alto Networks appliance (vPAN)