HOUR 21
Managing a Network

What You’ll Learn in This Hour:

• Ideas for maintaining and upgrading a network

• Server and client licensing

• More details on backing up files

• Disaster recovery planning

In this hour, we examine key issues a network administrator faces while managing a network. We discuss upgrading network hardware and delve into more detail on redundant array of inexpensive disks (RAID, introduced in Hour 5, “Network Concepts”), with an analysis of how to configure RAID arrays and establish a backup schedule. We also look at some of the issues related to client and Network Operating System (NOS) licensing. We complete the hour with an examination of one of the most important jobs of a network manager: planning and executing a disaster recovery plan for the network.

Upgrading the Network

After your network is built, a natural response is to sit back, relax, and enjoy the fruits of your labor. If the network is operating properly and is stable, it’s tempting to put your feet up on the desk and kick back. However, given that hardware and software versions change often, it’s pretty much a given that you’re going to get caught in upgrade cycles. Obviously, it’s in your best interests and the best interests of your users to keep changes transparent. It’s also in your best interests to temper management enthusiasm for a new technology they happen to have seen advertised on TV. By advising moderation and testing, you help maintain network functionality and cut down on unnecessary upgrades.

Although it’s difficult to serve in the role of naysayer, an important aspect of your job is to maintain and upgrade the network so that your company is getting the most bang for its buck and using the appropriate technology for its business. For various reasons, return on investment (ROI) is difficult to calculate for computer equipment and software. The use of computer hardware or software can’t be calculated in an absolute fashion; it’s pointless to try to calculate the value of a memo written on a user’s PC. But it’s your job to keep the network in the best shape possible and substantiate any budgetary needs in as logical a fashion as possible.

Network upgrades should be based on the needs of the company in terms of the business tools that employees need to get their jobs done effectively (both in terms of cost and the users’ time). So, it’s obvious that as computer hardware and software evolves, you’ll need to replace servers, client machines, NOSs, and client applications.

Let’s look at some basic strategies for managing hardware growth and upgrades. We can then examine issues related to software upgrades and growth. Before delving into the details, keep this idea in mind: The three most important factors for making the management of a network a successful affair are (1) a satisfied user community; (2) a satisfied user community; and (3) a satisfied user community.

Managing Hardware Upgrades and Growth

Not only will network hardware (including client PCs) need to be upgraded over time, but you’ll add systems and other supporting hardware because of network growth. Any successful company will grow. This means you won’t only have to keep existing employees up and running effectively on the network, but you’ll need to plan for and act on network growth. Some strategies for managing upgrades and growth follow:

• Set flexible standards for hardware—Every year or so, create a standard computer configuration based on the current most powerful computers. Try to stick to it for as long as possible, but not for too long. The benefits of this approach are twofold. The first benefit is that the computers are a known quantity. (If you spec a computer and discover a hardware bug, quirk, or incompatibility, you know that the remainder of the computers of that type will likely share that trait.) The second benefit is cost; over one year, the cost of most computers will decline significantly, making your bottom-line staff very happy.

• Determine whether a complete computer upgrade is required or whether an incremental upgrade is acceptable—Given the impressive pace at which computers are becoming faster and more powerful, many OEM computer manufacturers and a host of third-party companies are building upgrades that range from faster processors that plug in to the original processor socket to memory upgrades. It’s possible that a processor upgrade and an extra shot of memory can extend the life of many an old PC by providing adequate performance at a bargain price (at least in comparison to the cost of a new computer).

• Maintain a complete inventory of what’s in your computers, not just the CPU serial number—If you know what’s inside the computer, you can more readily make a call about whether it’s wiser to upgrade or replace it. This inventory list also helps when a part fails; rather than taking the case apart, you can simply look up the failed part in your database and order another.

• Perform regular hardware audits with network management software—If you have to do your inventory by going from computer to computer, it will never get done. However, most network management software has a hardware audit feature; used properly, this feature can help you diagnose a system on-the-fly.

The bottom line is that no matter how big your network and client base, you have to know the different hardware and software configurations that are running on the network. New employees will likely receive newer equipment when they come on board (if they’re an addition to the staff rather than replacing a staff member). But you must be sure that veteran employees aren’t left in the lurch with older systems that make them less effective (than a newer employee).

Managing Software Upgrades and Growth

In many ways, software is easier to manage than hardware. First, companies tend to standardize on certain software; not using that software puts a user outside the loop. Second, use of software has legal ramifications that force a company to treat software usage more rigorously than hardware. After all, hardware is a capital asset and can be depreciated out of existence; software—even expensive software—is often expensed. It’s simply written off as a cost of doing business.

Unlike with hardware, with software, you can do a tremendous amount of management, ranging from setting corporate standards to auditing the versions used. The strategies you can follow for managing software are as follows:

• Use site licensing or volume licensing on your network—Most software vendors sell volume licenses. Even if only a few workers are using the application, you might still qualify for some kind of volume or site licensing. If you can’t use volume or site licensing, it’s important that you have enough individual licenses to cover all your products. We examine another aspect of licensing—server and client licensing—in the next section.

• Work with senior management to come up with company standards for software—This thought should be self-evident, but it’s worth discussing. Suppose your company has standardized on Microsoft Word for word processing, but you have a user who insists on using another product. It’s much easier to support and license a single word processing product. So having a mandate from senior management that all users will use a particular product will provide you with better control of client behavior and software use.

• Unauthorized software is unsupported software—On any network, no matter how few users, you must lay down this rule related to software installation: No user-installed software is allowed on the network. If a user installs her own software in defiance of such an edict, and the installation creates a problem, the only support you’ll provide is to reset the user’s PC back to the original, approved configuration. Allowing unauthorized software is problematic, not just because it’s wrong (users do have a proclivity for installing unlicensed copies of software in a work environment), but because it raises the bar on the management hurdle to an unacceptable level.

• Create a standard installation, and stick to it if possible—If you can install a fixed set of applications on each hard disk so that all disks match, that’s good. Most NOSs let you install standard installations on network clients. (For example, Microsoft Server 2003 provides the Remote Installation Service.) Also, products, such as Norton Ghost and Symantec Drive Image, allow you to create a standard client configuration (including applications) that can be quickly installed on any client system using the same standard hardware.

• Use a license management utility—License management utilities ensure you’re never in violation of the terms of your software agreements. License management utilities can be a pain for users who can’t get an application because it’s at its license limits, but they ensure that you’re in compliance. Thanks to the efforts of the Business Software Alliance (BSA) and the Software Publishers Association (PSA), noncompliance is becoming increasingly expensive. It’s not uncommon to see extremely large fines for gross and willful license violators.

In terms of managing software upgrades, it’s important you make new software tools and more up-to-date versions of software products available to as many users as possible. In most cases, your software upgrade cycle is linked to your hardware upgrade cycle. This approach stems from most new software packages requiring a more robust hardware configuration to operate efficiently.

Dealing with NOS and Client Licensing

It’s not only essential that you ensure that all applications running on the network and individual client computers are properly licensed, but you must also ensure that you have the appropriate number of client licenses to access the various servers and their NOS. Application licensing is fairly straightforward; you need a license for every occurrence of that application running on the network clients, including remote users. This can take the form of individual licenses or some sort of volume or site licensing (or a combination of these licensing strategies). Client licensing in relation to NOSs, however, can be a little more complicated because more than one licensing scenario can be available for a particular NOS.

A NOS requires that you have a server license for your server (a separate license for each server) and client licenses for your network clients. This doesn’t just mean that you have a license for the client operating system (OS) but a license that makes it legal for you to connect to the server as a client.

For example, you can buy a Novell NetWare base package that licenses the server and five client connections. To license more clients, you buy what’s called a connection additive license. These additive client licenses range from the addition of 5 to 500 users.

Each NOS platform has its own licensing scheme. When you work with open source Linux products, you might not have to deal with licensing, but no matter what platform you’re using, you should take the time to research the type of licensing required for each client on the network. Be sure to examine the discounts offered for multiple users of a package. They vary widely among the vendors.

Microsoft Windows Server 2003 Licensing

Microsoft Windows servers put an interesting spin on client licensing. Windows Server 2003 provides you with three possibilities for licensing network clients: Per User, Per Device, or Per Server.

Per User means you’ll purchase a license for each network user on the network. Each of these users can connect to any and all the servers on the network. Per Device means you can license each computer or device, such as a Windows-based PDA. Because of the device license, the device can then legally connect to any and all servers on the network. Per Server means you’re licensed for a certain number of concurrent connections to the server. If you have 50 licenses, 50 clients can connect to the server.

Per Server licenses can save you money if you have a network situation in which your employees actually work in shifts. Because only a subset of the employees is connected to the network servers, you can go with the Per Server connection model. When employees put in the same hours, you’re probably better off going with the Per User or Per Device models.

All NOSs supply you with some type of utility that you use to add server or client licenses to the network. Microsoft Windows Server 2003, for example, provides the Licensing snap-in, which allows you to add licenses to the network. Figure 21.1 shows the Windows Server 2003 Licensing snap-in.

FIGURE 21.1 NOSs, such as Windows Server 2003, include a utility for recording server and client licenses.

Microsoft Windows Server 2008 Licensing

Windows Server 2008 offers Per Server licensing. It also offers Per Seat mode, in which you can purchase a license of each network user. Each of these users can connect to any and all the servers on the network. It’s akin to 2003’s Per User license, but with a different name.

2008 also offers the User Access license, which allows a user to connect to network services using any device, such as a computer or a PDA. In addition, customers can use the External Connector license to connect to licensed network services. For more information on Microsoft licensing, go to www.microsoft.com/licensing.

It’s quite important that you keep track of all your server and client licenses. The same goes for application licenses. You should have a well-organized filing system that allows you to access any hard copy licenses you have; also make sure you use software utilities that allow you to keep track of your licenses. Being caught without the appropriate number of licenses is a good way to lose your job because it could result in fines and bad publicity for your company.

Backing Up Network Data

Hour 5 introduced RAID implementations. RAID provides a method of creating redundancy on network servers, which can help protect valuable network data. The best way to protect network data, however, is backing up that data. Creating a backup plan and implementing that plan is an important aspect of managing an existing network.

When your network is operating smoothly and you aren’t detecting problems in your server logs and performance monitoring tables, it might be difficult to accept that you could have a sudden meltdown, resulting in the loss of data. But it does happen. It’s much wiser to assume that you’ll have a crash at some point and to prepare adequately for it.

The Basics of Backup

On first inspection, it might seem a little confusing when you’re trying to put together a backup plan for your network. You want to back up all the important data, but you want to do it as effectively (in terms of time and effort) as possible. Although you might throw up your hands and determine that you’ll just do a time-consuming and arduous complete backup periodically, there are ways to plan a backup strategy that will protect all the network data (with minimal loss) yet not require you to spend every evening backing up your servers.

Creating an effective backup strategy begins with a consideration of the following:

• How much data you have to back up

• How often you want to back up

• Whether or not backing up your data requires dealing with open files and ensuring consistent data sets

• On what kind of media you want to store your backed up data

• What kind of backup scheme you want to use

The first three items determine the choice of the fourth. We discuss backup schemes in more detail later in this hour.

You can use various types of backup media. Some of the popular tape backup types are as follows:

• Digital audio tape (DAT)—Developed for sound recording, this small high-density tape format can store up to 80GB of data, depending on the specific tape and compression methods. In 2005, Sony announced its intention to move away from this technology, but DAT is still widely used.

• DLTtape (or just DLT)—This is a half-inch tape format that’s quite popular in the industry. Many Fortune 500 companies use DLT for their backup operations. Some DLT units can store 1,600GB of data.

• 8mm—Similar to the 8mm video format, these 8mm cartridges (also known as 8mm Backup Format) can hold up to 40GB of data and can transfer data at speeds up to 3MB per second.

You can also back up data to removable media drives, such as the Zip and Jaz drives made by Iomega. In addition, you can copy files to CD or DVD if you have access to an appropriate burner. Again, the media type you choose is dictated by the amount of data you need to back up. The media type, obviously, dictates the tape backup drives you use.

Many hardware manufacturers produce tape backup drives, including Seagate, Hewlett-Packard, and Iomega. Some of these products come with decent backup software, and some don’t. (You might have to buy the backup software separately.) Most NOSs also supply some type of backup utility. Some are better than others, and you’ll have to assess whether you can get by with the NOS backup software or you need something more sophisticated.

After you’ve chosen and deployed the backup hardware you’ll use, you need to establish a backup scheme (as mentioned in our list), which needs to include a backup schedule. Let’s look at the different types of backups you can make, and then we’ll look at a simple scheme called the “Grandfather-Father-Son scheme.”

Types of Backups

Three backup methods are available: full, differential, and incremental. These different types of backups are possible because of file markers, which are attributes placed on a file. (In other words, the file is tagged.) Typically, any OS you work with marks or tags a file after that file has been backed up. A file that has changed since its last backup is also tagged. The use of these tags or markers to denote which files have been backed up and which files have not enables backup software to perform various types of backups. Here’s a breakdown of how these backup methods work:

• Full backup—This type of backup is also called a normal backup or a daily backup (depending on the backup software you’re using). A full backup takes all the files you select for backup and backs them up (no matter how the files are currently marked). The files’ attributes are then changed to mark the fact that they have been backed up. (If you change the file after the backup, the marker changes and indicates that the file hasn’t been backed up since the last changes were made.)

• Differential backup—This type of backup only backs up the files that have changed since their last backup. The differential backup doesn’t, however, change the marker attribute indicating that the file has been backed up. It leaves the marker alone, meaning the file still reads that it hasn’t been backed up since it was last changed.

• Incremental backup—This type of backup backs up only the files that have been changed since the last backup (just as a differential backup does). However, an incremental backup changes the archive marker on the files that are backed up to identify those files as having been backed up (which differs from the differential backup method).

The type of backup you should use depends on the backup scheme that you devise. You should determine a particular time of week when you do a full backup (perhaps on the weekend). You can then use differential and incremental backups (which don’t take as long as a full backup) to make sure you have the most recent copies of files that have changed since the full backup. A simple use of a single full backup and then sequential differential backups is discussed in the next section.

The Grandfather-Father-Son Scheme

A simple backup scheme is Grandfather-Father-Son. It sets up a sequence of tapes ensuring proper tape rotation so that you don’t lose data.

How do you do it? First, label four tapes (or tape sets, if you’re employing a tape changer that uses more than one tape per day) Monday, Tuesday, Wednesday, and Thursday. Then, take four more tapes or tape sets and label them Friday 1, Friday 2, Friday 3, and Friday 4. After you’ve done this, you’ve created all your repeating tapes. The remainder of the tapes is labeled Friday 5. The Friday 5 tapes are your archive tapes; you use each one only once (at the end of the cycle) and then archive it. This approach ensures that every five weeks, you have an archive tape.

Next, you have to configure your backup software. Typically, the easiest way to back up is via a differential backup. In differential backups, the Friday tape is a full backup, and each successive tape (Monday through Thursday) captures all the changes since the last Friday full backup tape. With differential backups, you only need two tapes to restore a crashed server: the last Friday full backup and the most recent weekday tape. Most commercial backup software can be configured to do differential backups and often have a Grandfather-Father-Son backup scheme or wizard to set it up.

After your software is configured for differential backup (it’s seldom hard to configure), you have to start rotating the tapes. Table 21.1 shows how that works.

TABLE 21.1 The Grandfather-Father-Son Backup Scheme

Each fifth Friday tape is the Grandfather; Friday tapes 1 through 4 are the Father tapes; and the Monday through Thursday tapes are the Sons. Every Friday tape except the Friday 5 tape is reused in each five-week period; Monday through Thursday tapes are reused each week.

The operation is simple. It requires some time to set it up and understand it, but then it usually runs smoothly. One caveat: Make sure you change tapes according to the schedule. If you don’t, you’ll have incomplete backups and you’ll be asking for trouble. Also, store all your Friday 5 (archive) tapes offsite as part of disaster recovery. We examine disaster recovery issues in the next section.

One more thing: Make sure that you periodically inspect your tapes. Magnetic media have a limited life span and become more likely to fail over time. Just think about how scratched up a home videotape gets when you constantly reuse it to tape shows on your VCR. It makes sense to periodically work new tapes into the process so that you’re not working with old and possibly unreliable media.

Network and Disaster Recovery Planning

Having a backup strategy for your network is only one part of what should be a complete disaster recovery plan. No one can predict when disaster will strike. Unfortunately, it often takes a disaster to make people consider disaster recovery planning. In the wake of the September 11, 2001 World Trade Center disaster, disaster recovery has become a hot corporate topic. Companies and institutions have spent more time and resources related to planning what they would do in the event of a disaster.

As the network administrator, your responsibilities related to disaster recovery are making plans that allow you to get important data back into the hands of people who need it, and getting it back quickly. This means your disaster recovery plan needs to be multifaceted and anticipate different levels of disaster. Your plan should not center around one type of disaster, such as an earthquake or fire.

For example, in the case of a major snowstorm, employees might not be able to get to the physical locations of the company offices. Yet your business needs them to be online and working. In this case, your disaster recovery plan might dictate that you activate VPN or dial-in access that allows employees to work from home. On the other hand, in the case of a disaster such as a fire that destroys the corporate offices, you will need to use your backup information to rebuild the network data servers at a new location and provide network access to that data.

You can see from the previous paragraph that different disruptions of business continuity (a fancy way of saying recovering from a disaster) require different solutions. Any recovery plan you implement needs to address different disaster scenarios. Let’s look at some of the basics of fashioning a disaster recovery plan.

Creating a disaster recovery plan requires several stages. Those creating the plan need to know several things, such as the current computing infrastructure, the business impact when the infrastructure is damaged, and suspected vulnerabilities in the infrastructure. Let’s examine some of the stages required to assemble the information needed to create the disaster recovery plan.

Defining the Computing Infrastructure

Before planning can take place, you must inventory and define the company’s computing environment. This means you need to know how many workstations, servers, and other devices are present on the network.

You can create an inventory of network devices in any spreadsheet program, such as Excel. This information is vital to the recovery plan and should be kept in a safe but easily accessible place (such as offsite) in case you need it. (Inventories are also important when dealing with insurance companies.)

Create a detailed network map (you should have created one when you planned the network), as well as other documentation that offers an understanding of what’s on the network. Before you can create a recovery plan for the IT infrastructure, the project team creating the plan must understand the IT infrastructure.

You can create network maps using a variety of software programs. Microsoft Visio and SmartDraw are easy-to-use tools for creating network diagrams (and were used to create many of the network diagrams in this book). Another aspect of defining the network infrastructure is a listing of network support positions. It’s important that management be aware of the personnel required to keep the network functioning on a daily basis. This means that a listing of positions and functions should be created to use in the disaster recovery planning process.

Not only do you need to assess the current state of the network infrastructure, including personnel, but you should assess future needs of the computing environment. This provides information on how the disaster recovery plan needs to be amended over time to remain effective.

Assessing Business Impact

The next phase of the disaster recovery planning process requires you to assess business impact. This involves identifying critical functions and systems in the computing environment and how their disruption would affect the core business of the company.

For example, let’s assume you work at a university. If there were a lightning strike that destroyed switches providing Internet connectivity to dormitories, you would end up with disgruntled students who wanted to surf the Web. However, the impact on the overall business of the institution isn’t affected as dramatically as the meltdown of a database system that holds student records and accounts receivables.

You can see that Internet access in the dorms isn’t as critical as access to an important database. Part of the process of identifying critical functions and systems on the network is assessing how long the institution can function with the key system unavailable.

Assessing Computing Environment Vulnerabilities

Another aspect of pooling the information for the disaster recovery plan is assessing the various vulnerabilities in your computing environment. For example, to use the university example again, one vulnerability is the large amount of data traffic that can be caused by students downloading programs, MP3s, and other treasures we don’t even want to know about. Let’s say that there’s a snowstorm and students don’t have classes. To bide their time, they sit in their dorm rooms downloading videos and other images. This clogs the network, and employees working from home are affected.

You’ve identified a potential infrastructure problem. Obviously, this scenario wouldn’t constitute a major disaster, but a good thing about doing the network vulnerability assessment is that you can fix some of the things that could create problems before disaster actually strikes.

Another vulnerability is your company’s connection to an important resource (such as a financial institution) through a single wide area network (WAN) connection. You can minimize this vulnerability by having a redundant connection made to the resource. Identifying vulnerabilities is really troubleshooting the network before trouble happens (which we discuss in Hour 22).

Other vulnerabilities that you’ll find during the assessment might lend themselves to being fixed immediately, removing the vulnerability, or being detailed in the disaster recovery plan. Going through the assessment can be an eye-opening experience in relation to how well your network infrastructure was initially planned and laid out.

Developing the Plan

After the corporate computing environment, disaster business impact, and vulnerabilities have been assessed, your project team can begin to develop the disaster recovery plan. This means that vulnerabilities, contingency plans, and how the plan will be implemented in light of a certain type of disaster must be documented, along with other information such as the inventory and other items discussed previously in this hour.

Although a disaster recovery plan is specific to each type of business or organization, there’s information that’s common to most disaster plans. These items are as follows:

• Employee contact information—The plan should include information on contacting key employees in case of a disaster.

• Vendor and customer contact information—Depending on your business, the plan should include contact information for key vendors and contact information for key customers.

• Location of backup information—The location of offsite storage facilities for data backup tapes and other network information should be identified.

• A listing of security information—The plan should provide a way for obtaining user IDs and passwords—particularly those for administrative tasks. If the disaster is the loss of network administrators, this type of information must be available. However, because it’s so important to the security of the system, you must consider the disaster recovery plan a highly sensitive document.

• Disaster regrouping location—The plan should provide information on where employees should go if a disaster strikes and the corporate facilities are no longer usable. This location can be a branch office or another designated space (such as conference space provided by a local hotel). It’s a good idea to have a place for all employees to meet; think about the World Trade Center disaster and the fact that it was very difficult to determine whether employees had made it out of the buildings.

• Declaring a disaster—Although this might seem to go without saying, it’s important that the plan detail who is in charge of declaring a disaster. Detailed staff information should also be included in the plan so that employees offsite (or on vacation) can be notified that a disaster has occurred.

• Succession planning—This is another one of those grim subjects; there needs to be information in the plan designating who is in charge if circumstances change. In other words, if the CEO or president of the company is a victim of the disaster, who should lead the company in the aftermath of the disaster? When John Hinckley shot President Ronald Reagan, Alexander Haig, then secretary of state, declared in a press conference that he was in charge. However, the order of presidential succession places the secretary of state below the vice president, the speaker of the House, and the president pro tempore of the Senate. It’s a good idea to specify who will be in charge. Why add more confusion to a bad situation?

This list contains just some of the information that should be part of the recovery plan. Creating a disaster recovery plan is a major undertaking requiring a lot of research and input from people within and outside your organization. Because IT disaster recovery plans mandate the assessment of tangibles (computers, software, and data) and intangibles (user behavior), you should do some research before you begin the planning process. Check out www.sans.org. It’s the website of the SANS (SysAdmin, Audit, Network, Security) Institute. You’ll find numerous papers and articles on different aspects of disaster planning in the IT environment.

Remember that the purpose of a disaster recovery plan is to allow your company to survive a disaster and then continue with its normal day-to-day business. Although your customers will certainly be sympathetic to your plight, they’ll quickly begin to look elsewhere for the services that you provide if it begins to affect their bottom line.

Archives

We mentioned archives several times in this hour. I recommend that you and your team establish a plan for long-range archival of data. The plan must include the user community’s directions, because users are the proxy owners of the company’s data. Archiving can become expensive, and users might not be aware of the costs of storing data and periodically moving it to fresh physical media. On more than one occasion, I’ve heard users say, “Save everything, forever! Never know when we’ll need it.” Compromises are likely in order, and you can make valuable contributions to this important operation by making sure users are aware of the complexity and costs of archiving.

Summary

In this hour, we discussed issues related to managing a network, including upgrading and growing a network. We examined data backup and data backup strategies and issues related to server and client licensing. We also discussed some of the basics of creating a disaster recovery plan for your network.

Q&A

Q. Do small companies need to worry about making sure they have the appropriate software licensing?

A. Even a company with one employee needs to adhere to the licensing agreements for software products that are used on its computers. It’s illegal to run software without the appropriate licenses. Just because a company is small doesn’t mean that a software vendor wouldn’t consider action if a license violation was brought to its attention.

Q. What’s the best way to sort through the many choices relating to different types of software licensing programs?

A. Talk to a knowledgeable software reseller or with the software vendor. Don’t be afraid to ask questions and get the facts. Selecting the best licensing scenario for your company can often result in substantial savings.

Q. Is it necessary to have a formal disaster recovery plan for a small company?

A. Yes! Even if you’re a one-person company, you should have a plan (even if it’s stored in your gray matter) to back up your files. It won’t be a happy situation to learn your photo disk file of your pet dog is lost forever. Not to mention photos of your mate. Jokes aside, all businesses, regardless of size, should have some sort of disaster recovery plan for their network. There must be a set of steps that have been recorded (and tested) that allow you to get users back on to the network with minimal downtime.