Chapter 1. Identifying Cisco Collaboration Deployments

While the capabilities of Cisco Collaboration System Solutions have expanded immensely over the past couple of decades, the underlying system has remained the same. Its purpose is to provide a firm foundation for a collaboration architecture that enables businesses to transform the way they function and communicate by eliminating the limitations imposed by traditional communication systems and methodologies.

The telephone created by Alexander Graham Bell, aside from a few key evolutionary alterations, has remained somewhat static. For over a century, the evolution of communications has undergone relatively little advancement, from a “big picture” point of view. The telephone of the nineteenth and twentieth centuries kept its users tied to a desk or specific location. The first really big leap in terms of technological advancement was the cellular phone. Of course, there was architectural progress with analog versus digital. There were also feature advances, such as dual-tone multi-frequency (DTMF), automatic number identification (ANI), wireless phones, and so on. Looking back, we might argue that this evolution focused on extending the reach of the desk phone itself rather than truly transformational, metamorphic advances that resulted in the creation of an altogether new creature. In terms of metamorphosis, the comparison really does resemble the caterpillar and the butterfly. The caterpillar is stuck on the ground, relegated to a very small space, whereas the butterfly is free to fly where it will, regardless of geography. When compared to the legacy time-division multiplexed (TDM) private branch exchange (PBX) in terms of capabilities, the Cisco Collaboration System is a transformational leap of a similar magnitude.

The Cisco Collaboration System is an architecture framework upon which applications and services can be deployed, customized, or created to meet the individual needs required by businesses in terms of their business model(s) on down to the needs of individual users. It is extremely flexible and customizable. Yet, it is often seen as complex. We simply need to view the solution as just that, a solution. It is no longer merely a platform for delivering dial tone. It is a foundation for a single collaboration system meant to provide communications in whatever manner, on whatever devices/platforms, required by a business with hundreds of thousands of users, a few users, or any size business in between.

This chapter examines some of the essential concepts relevant to the function and troubleshooting of the collaboration architecture.

Chapter Objectives

Upon completing this chapter, you will be able to

• Identify typical problem areas in the network that normally need troubleshooting

• Describe the element of the Cisco Collaboration Systems solution that is considered to be “network infrastructure,” explain its purpose in the Cisco Collaboration Systems solution, and describe the types of issues that require troubleshooting

• Explain the purpose of Cisco Unified Communications Manager (CUCM) in a Cisco Collaboration Systems solution and list the common issues that occur

• List the range of endpoints that can exist in a Cisco Collaboration Systems solution and describe the types of endpoint issues that require troubleshooting

• List the different types of media resources that can be used in a Cisco Collaboration Systems solution

• Explain the purpose of Cisco Collaboration Systems applications in a Cisco Collaboration Systems solution and list common issues that occur with them

Overview of Cisco Collaboration Systems Solution Components

Cisco Collaboration Systems range anywhere from small implementations that provide a dial tone to a few phones all the way up to tens of thousands of endpoints. Each implementation has its own unique factors in terms of dial plan, class of restriction, and desired functionality.

A number of core components should be considered to create the base system. They include a call control element, an instant messaging element and presence, and a voice messaging element. Of course, they may vary depending on the core function of the system. In terms of a true Cisco Collaboration System, the core applications are considered the CUCM, for the call control; CUCM Instant Messaging & Presence (IM&P), for presence and instant messaging; and Cisco Unity Connection (CUC), for voice messaging. As the solution scope grows in size, additional pieces may be added to provide for interoperability with a third-party private branch exchange (PBX), external access for clients and remote workers, and any other number of imaginable scenarios or features. Knowing the environment in which you are functioning is key to troubleshooting. Common issues such as choppy voice, one-way audio, video quality issues, and jitter may present themselves. Understanding where and how to troubleshoot these issues depends largely on design, implementation, and intent. Figure 1-1 shows a number of the Collaboration System elements commonly encountered in an enterprise collaboration architecture.

Figure 1-1. Cisco Collaboration System Solutions Components

At first glance, it may seem confusing from a troubleshooting standpoint. You might ask, “Just how am I supposed to be able to troubleshoot with so many pieces and potentially moving parts?” The answer to that question varies based on the type of issue experienced. Over the last few years, Cisco has added a significant number of new applications, features, and capabilities to the overall Collaboration Systems solution. The authoritative source of information regarding best practices for the collaboration systems architecture is in the form of the solution reference network design (SRND) document. The current version is always accessible via this URL:

http://www.cisco.com/go/srnd

The hyperlink mentioned redirects to the main landing page for Cisco Collaboration Systems documentation. As new features and functions were added to the solution, the SRND document grew at a surprising pace. At present, the page count is well over 1200. This makes the document somewhat unwieldy. To break down the SRND into usable, digestible, and technology-specific chunks, Cisco created a new set of documents known as the Preferred Architectures (PA). The PA documents are generally 30–40-page documents targeted to specific aspects of the Cisco Collaboration Systems solution. They are now considered the go-to documents for design best practices. The PA represents prescriptive design guidance. They are intended to be modular and scalable and provide a base design for any customer. This drives design consistency for Cisco Collaboration Systems deployments. The PA documents are also found at the preceding link. Note that the PA documents present the architectural overview in simpler terms than the SRND. Figure 1-2 shows the PA document architectural overview.

Figure 1-2. The Cisco Preferred Architecture for Enterprise Collaboration

As shown in Figure 1-2, the PA document shifts the focus a bit differently and adds additional granularity. This is convenient when you are troubleshooting because it has divided the architecture into more easily understood pieces.

The information presented by the course and that presented by the PA document cover the same information. The existence of the two entities doesn’t change the way in which troubleshooting is performed. The PA is simply a different organization and a more clearly presented view. The PA provides for high availability of critical services and applications. The architecture extends to mobile workers, partners, and customers through voice communications, IM&P, high-definition (HD) video/content sharing, rich media conferencing, business-to-business (B2B) communications, and unified voice messaging.

The PA actually has eight focal points in terms of architecture:

• Call Control

• Conferencing

• Collaboration Edge

• Applications

• Endpoints

• Mobile/Teleworkers

• Remote Sites

• Cloud-based Services (such as WebEx)

PA illustration and component delineation are examined in the sections that follow.

Network Infrastructure

The network infrastructure within an enterprise determines the success of a collaboration systems deployment. The network is the platform that delivers the applications and services demanded by the enterprise user community. A well-designed, robust network infrastructure is the goal from a collaboration solution perspective. Redundancy, dynamic routing, and high availability used to be features of the network; now they are requirements.

A significant part of troubleshooting depends on how the network is dividing finite bandwidth resources while trying to handle infinite traffic types and loads. Mission-critical traffic is a necessity and has a direct correlation with the needs of the traffic flow(s) that makes business function, be it Oracle, SAP, Salesforce.com, e-mail, or voice and video. Video is not simply video, a single category of traffic. For example, is it streaming, on-demand, real-time, immersive, or a combination thereof?

Proper quality of service (QoS) configuration is no longer optional. QoS is not simply a transmission methodology that can be ignored if there are huge amounts of available bandwidth. In the late 1990s, simple queuing and compression were sufficient to fix network issues regarding bandwidth. QoS is more than a mere queuing mechanism. It is a network protection mechanism. It not only prioritizes traffic flows but also allows the network to react to excessive traffic such as a virus outbreak, even before it has been detected. It is not necessarily about bandwidth utilization. It’s about priority/enforcement on the ingress interface, the processor, egress interface, and everything in between. QoS is not an optional component in the collaboration systems architecture. In fact, the lack of QoS is a huge contributor to voice quality and one-way audio issues and more.

Call Control Systems in Cisco Collaboration Systems Solutions

The Cisco Collaboration Systems solution is an architecture composed of multiple elements—no longer just a means of providing dial tone and voice messaging.

Understanding that readers of this book will come from voice and/or network backgrounds, we need to set a baseline that allows everyone to comprehend the topics covered in this book. This section begins with a review of basic private branch exchange (PBX) architecture and how it maps to the Cisco Collaboration Systems solution. Figure 1-3 shows a basic layout of a PBX.

Figure 1-3. Basic PBX Architecture

In Figure 1-3, the basic architectural components of the PBX are labeled. They are as follows:

• Processor: Acts as the decision-making engine for dial pattern lookup, call routing, and so on

• Backplane: Provides physical transport of signaling and media

• Line Card: Provides connectivity for end stations such as phones and fax machines

• Trunk Card: Provides connectivity to other PBXs, applications, and public switched telephone network (PSTN) central offices (CO)

Cisco has a similar architecture in terms of functionality. Key components provide the same services as those contained within the PBX chassis. Figure 1-4 shows the same functions in terms of how they are represented in the logical architecture of the Cisco Collaboration Systems.

Figure 1-4. Cisco Collaboration Systems Logical Architecture

All the same pieces are shown in both Figure 1-3 and Figure 1-4. Table 1-1 illustrates the differences between the two.

Table 1-1. Legacy PBX and Cisco Collaboration Systems Comparison

The means by which the functions are provided is vastly different. Cisco Unified Communications Manager (CUCM) provides the call control element. It was created with diversity and redundancy in mind, leading to the concept of a CUCM cluster. There is one Publisher node and multiple Subscriber nodes. The Publisher node is the owner of the database. All administration is performed at the Publisher node. Changes are then replicated to the Subscriber nodes. The nodes (Publisher and all associated Subscribers), represented as a single unit, comprise a cluster. The cluster is the PBX.

The ability to use an IP network for backplane functionality provides an immense degree of fault tolerance because of IP routing protocol functionality and convergence. Dynamic routing capabilities ensure that there is reachability among the components, so long as there is a redundant path through the network. This same aspect tends to make it more difficult for PBX administrators to achieve the transition from the known architecture to any IP-based solution. Administration, support, and troubleshooting of the Cisco Collaboration Systems solution usually requires a high degree of routing and switching knowledge. Often, issues with voice, voice messaging, or other applications involve network troubleshooting, which may also lead to high complexity to the overall solution. Keeping all of this in mind, we can narrow the scope of troubleshooting to four essential elements:

• Cisco Unified Communications and Unified Contact Center Applications: Includes Cisco Unified Communications Manager (CUCM), CUCM Instant Messaging & Presence (CUCM IM&P), Cisco Unity Connection, Cisco Unified Contact Center Express (UCCX), Cisco Unified Contact Center Enterprise (UCCE), Cisco Expressway, Survivable Remote Site Telephony (SRST), Cisco Unified Communications Manager Express (CME), Cisco Unity Express (CUE), Cisco Integrated Services Router (ISR) voice gateways, and Cisco Unified Border Element (CUBE), and the Prime Collaboration Suite.

• Collaborative Conferencing: Includes Cisco Conductor, Cisco TelePresence Server (TS), Cisco Virtual TelePresence Server (vTS), Media Resources, Conference bridging (audio and video), Multipoint Conference Unit (MCU), and TelePresence Management System (TMS).

• Network Infrastructure: Includes the LAN/WAN infrastructure, including the routing and routed protocols transporting voice/video/data traffic.

• Endpoints: Includes IP desktop collaboration endpoints, desktop software clients, mobile clients, TelePresence endpoints, and room-based immersive endpoints.

At the foundational level, call control binds the entire solution together. In general terms, it is a decision-making engine that allows the routing of calls, audio, or video. The primary call control element is CUCM. The call control element provides endpoint registration, call processing, media resource management, and IM&P for end users. It also includes remote survivability for remote and teleworker sites.

A significant degree of the learning curve associated with the Cisco Collaboration Systems is understanding how the various components interact. It is no longer just a dial-tone-only service. It ceased being solely a PBX when it began providing a development platform for application developers. The traditional voice/PBX technicians understand call routing, class of restriction, hunt groups, and so on. The traditional data technicians understand the distributed architecture and nature of route/switch infrastructure. It is important that technicians truly understand that, collectively, call routing and infrastructure are simply pieces of the foundation. For technicians to succeed in collaboration systems support, it is crucial to understand the nature of voice/video communications and IP networks as well as software development and life cycle.

Call control includes the following:

• Cisco Unified Communications Manager (CUCM)

• Cisco Unified Communications Manager Instant Messaging & Presence (CUCM IM&P, integrated into CUCM)

• Cisco Unified Communications Manager Express (CUCME)

• Cisco Survivable Remote Site Telephony (SRST)

• Cisco Video Communications Server (VCS)

From a call control perspective, the bulk of the focus is placed on CUCM. CUCM now includes the IM&P functionality (formerly known as Cisco Unified Presence Server). The IM&P features are still on a separate virtual machine but administered via the CUCM administration interface. Hence, they are now technically part of the call control structure.

CUCME (also known as CME) is a full-featured call control software package that runs on a Cisco Integrated Services Router (ISR). It supports both Cisco Skinny Call Control Protocol (SCCP) and Session Initiation Protocol (SIP) endpoints with full feature parity.

SRST is a call control element of last resort. It provides endpoint registration when CUCM is unreachable. SRST can provide a robust feature set. However, it is an exceedingly reduced feature set meant to provide the most fundamental of services during a WAN outage. Endpoints detecting loss of connection to CUCM make contact with the local SRST router to maintain calling functionality.

VCS is a call control element for video endpoints. It may be deployed as a physical appliance or a virtual server. However, there is a very limited time left for it in physical appliance form. VCS comes in two flavors: Control (VCS-C) and Expressway (VCS-E). VCS-C functions in the internal network and handles video endpoint registrations, whereas VCS-E functions in the demilitarized zone (DMZ), or in some cases outside the firewall, to handle video calls to and from video endpoints outside the network.

Cisco has introduced Expressway Core (Expressway-C) and Expressway Edge (Expressway-E) as the collaboration edge element for business-to-business (B2B) calling, mobile and remote access (MRA), and other edge-related technologies. While these virtual-only servers are running virtually identical software to VCS, they are not call control elements. VCS-C has endpoints registered to it, whereas Expressway-C does not. As such, they are not mentioned here in the call control section.

These call control elements are key to troubleshooting voice and video issues. The logging of calls is performed here, which includes error messages, management, configuration, monitoring, and more. The call control element within the architecture represents the central authority for dial plan (that is, call routing and handling). It also provides the platform on which application integrations are constructed and third-party interoperability is configured. When media resources are required for transcoding (XCode), music on hold (MoH), media termination points (MTP), conference bridges (CFB), and so on, the call control element is what invokes those resources and brings to the call sessions in play. Call control is the heart of the collaboration systems. Without it, the rest of the architecture simply cannot function.

Endpoints in Cisco Collaboration System Solutions

The endpoints within a collaboration systems solution are crucial because the line between voice and video continues to blur. The endpoint defines the user experience. These endpoints can range from a single-line phone with no functionality to placing or receiving a single call to immersive room systems that eliminate the distance between the parties on the call and provide a face-to-face experience. Figure 1-5 shows a Cisco IP Phone 8865. It illustrates how technological advances and expectations have changed from the basic concept of a desk phone.

Figure 1-5. Cisco IP Phone 8865

The 8865 is as an example of the endpoint evolution. It has integrated 1080p video capability, Bluetooth (for headset and smartphone pairing), Intelligent Proximity, a USB port for smartphone charging, a USB port for tablet charging, 802.11a/b/g/n/ac wireless network capability, and the capability to have up to three attached key expansion modules (KEM). It represents the higher end of Cisco’s current desk phone portfolio. Also in the desktop realm are the DX Series collaboration endpoints. Figure 1-6 shows these three endpoints: the DX650, DX70, and DX80.

Figure 1-6. Cisco DX Series Desktop Collaboration Endpoints

These endpoints are desktop video endpoints that run Collaboration Experience (CE) software. Like other endpoints, they register to CUCM natively. At initial release, they ran an Android OS. They are still capable of doing so. But development efforts have slowed to a trickle, at best, in terms of Android functionality. The optimal user experience is found on the CE code.

The endpoint is the user experience. The company, the network, the architecture, and more are judged every time a call is placed to or from any endpoint. In most cases, this process is successful. However, it is important to know where to begin troubleshooting. Endpoints are often the first place to look because most issues are reported by end users.

The endpoints and applications deployed allow end users to communicate at the time, place, and manner of their choosing. Soft clients on desktops or laptops, smartphones, tablets, conference phones, desktop collaboration video endpoints, immersive room systems, and more are all treated the same when it comes to expectations of communication. If the experience is less than perfect, end users will begin to find better ways to communicate. That ends up reflecting poorly on the network infrastructure, collaboration systems, and the teams supporting all of those technologies.

Media Resources in Cisco Collaboration Systems Solutions

One of the more overlooked and/or misunderstood aspects of troubleshooting lies within the realm of media resources. Media resources include

• Annunciator Services

• Interactive Voice Response (IVR) Services

• Media Termination Point (MTP) Services

• Music on Hold (MoH) Services

• Transcoding Services

• Conference Bridging Services

Although there are additional media resources, this chapter focuses on the major ones with which nearly every collaboration system will be configured.

The Annunciator is the system that plays the reorder tones and recordings on the local CUCM system when a caller can’t be reached or a number cannot be completed as dialed.

The IVR piece, which is new to CUCM 11.x, provides flexibility in hunt groups and queues, which are now native to CUCM.

MTP resources are used with trunking connections, such as SIP trunks that need to provide additional mid-call functionality.

Music on hold (MoH) services are some of the most misunderstood topics in collaboration. The use of unicast versus multicast and the selection of user hold source versus network hold source and when each is used are sources of significant troubleshooting needs.

Transcoding is the conversion of a media stream from one codec to another. For example, when an endpoint configured for G.711 needs to talk to an endpoint configured for G.729, a transcoder must to be invoked to convert the media back and forth.

Conference bridging services are required when a call, either audio or video, needs to include more than just the two original parties on the call. When a third party joins, there has to be some kind of resource to mix the audio/video and send the streams to all parties.

Media resources can be provided by software or hardware. CUCM provides software audio conference resources. A virtual TelePresence (vTS) server or TelePresence server appliance also offers software-based resources for conferencing. The vTS is a virtual machine that provides virtualized multipoint control unit (MCU) resources for audio/video conferences. From a hardware perspective, media resources can be provided by Cisco Integrated Services Routers (ISR) or Aggregation Services Routers (ASR) with digital signal processor (DSP) modules installed on-board in the form of PVDM modules. Additionally, hardware conference resources can be provided by a hardware-based MCU utilizing either a fixed configuration or a larger chassis/blade configuration. Table 1-2 shows media resources and the types of devices that host them.

Table 1-2. Media Resources

Applications in Cisco Collaboration Systems Solutions

In reality, it is the applications that make the collaboration system more than just another dial-tone-providing solution. Applications such as IM/Presence, voice messaging (unified messaging, integrated messaging, web-based access), Unified Contact Center Enterprise and Express, Paging applications, attendant console applications, text messaging applications, Cisco Intelligent Proximity, Cisco WebEx, and more all bring together a pleasant user experience.

There are also integrations, open application programing interfaces (APIs). The possibilities seem endless. Numerous companies (Cisco partners, Certified Development Partners) are building very attractive bottom lines simply by building applications for Cisco Collaboration Systems. As the complexity and volume of applications, application integrations, and vendors supporting solutions increase, so does the need to fully understand the architectural needs of the traffic types involved. Where are the application servers placed in relation to the collaboration server applications? How is the traffic flow architecture constructed and protected to ensure that the traffic, both signaling and media, is protected? Collaboration systems architecture has very limited tolerance for latency across the network. It has to be fast and built to last. It becomes important to understand that every application added to the network increases the size of the potential troubleshooting domain.

Getting Help

The first rule of dealing with a big problem in the network is, well, to avoid it. Barring that, the rule is, “Never be afraid to ask for help.” It has been said in numerous ways, but the underlying message is the same: leverage every available resource.

From a troubleshooting perspective, taking this advice means engaging the Cisco Technical Assistance Center (TAC). TAC is Cisco’s world-class support team. Team members utilize what is known as a “follow the sun” model and are available 24 hours a day, 7 days a week, 365 days a year. They are an amazing organization. One of the single biggest selling points for Cisco architectures, solutions, and products is the TAC organization standing behind it.

For all customers, partners, resellers, and distributors who hold valid Cisco service contracts, engaging TAC is relatively simple. It can be done via phone by dialing +1-800-553-2447 in the United States. However, there are phone numbers for other countries as well. The full list is available here:

http://www.cisco.com/c/en/us/support/web/tsd-cisco-worldwide-contacts.html

TAC can also be reached online at

http://www.cisco.com/go/tac

or

http://www.cisco.com/techsupport

Cases can be opened, updated, monitored, and so on through that portal. Cases can also be opened through e-mail by sending a detailed message to [email protected]. Be sure to include your Cisco Connection Online ID (CCO ID) in the body of the message.

The means by which TAC should be engaged varies by the severity of the situation. Cases are ranked by severity or priority. The four levels of severity are S1, S2, S3, and S4 (also designated as P1, P2, P3, and P4). S1 and S2 cases represent urgent situations in which the network is entirely down or severely impacted, respectively. When you open these cases, it is best to call the local TAC phone number and declare a network-down emergency. This results in an immediate transfer to an available engineer. S3 and S4 cases are those in which there is not significant degradation of the network or there is a requirement for product information, licensing, and so on. S3 and S4 cases can be opened by phone or e-mail, online, or even via the Cisco Tech Support application on mobile devices (available in the App Store or Google Play). In fact, only S3 and S4 cases can be opened via e-mail or online. Once opened, the case is queued, where it can be serviced by the next available engineer. The method by which an engineer contacts you back is determined by the information used when the case was opened, usually e-mail or phone.

The more information included when opening the case, the more quickly it can be resolved. Information such as contract number, network diagrams, products in place, detailed description of the issue at hand, and more is exceedingly helpful in getting the case properly routed and getting down to troubleshooting quickly.

TAC should never be considered a last resort. In fact, they should typically be engaged fairly early in the troubleshooting process. The engineers within the organization are well versed at digging down to the root cause of an issue and correcting it in rapid fashion. Make use of the TAC organization. The support the engineers provide will assist in preserving your sanity in times of trouble.

Chapter Summary

The common theme of this chapter is to offer an exceptional user experience. This means protecting the collaboration signaling, media, and application traffic flows across the network from all sorts of issues or disruptions. When you are delving into the troubleshooting realm, a deep understanding of the network infrastructure, call control layout, and how the applications involved interact with both the network and the call control elements is critical.

The starting point in troubleshooting is usually with the endpoint(s) and the call control element. Replicating the issue is an exceedingly valuable task but cannot always be relied upon. At that point, it becomes an exercise of tracing the call path, digging through log files, and so on, which is the focus of the remainder of this book.

References

For additional information, refer to the following:

• SRND and Preferred Architecture Documents: http://www.cisco.com/go/srnd

• To get in touch with Cisco TAC: Full phone list: http://www.cisco.com/c/en/us/support/web/tsd-cisco-worldwide-contacts.html

• TAC can also be reached online at the following:

http://www.cisco.com/go/tac

or

http://www.cisco.com/techsupport

Review Questions

Use these questions to review what you’ve learned in this chapter. The answers appear in Appendix A, “Answers to Chapter Review Questions.”

1. Which is an architectural focal point in the Preferred Architecture?

a. IM & Presence

b. Collaboration Edge

c. Unified Contact Center Express

d. Cisco WebEx Meetings Server

2. Which component provides an integration interface to external entities and/or networks?

a. Call processor

b. Network infrastructure

c. Voice gateways

d. Line cards

3. Which feature provides call control redundancy for branch sites in the case of a WAN outage?

a. CUBE

b. CUCM

c. SRST

d. VCS

4. Which element provides the primary user interface to the collaboration system?

a. Endpoints

b. Call Control

c. Conferencing

d. Collaboration Edge

5. Which media resource is concerned with converting media from one codec to another?

a. MTP

b. MoH

c. Transcoder

d. Annunciator

6. By which method(s) may a TAC case be opened? (Select all that apply.)

a. Online

b. Telephone

c. Email

d. Mobile Application