CHAPTER 9

Testing Your LMS and Going Live

As discussed in the previous two chapters, LMS implementation typically involves six major steps. This chapter covers the final two steps, testing and going live.

Figure 9-1. The 6 Steps of LMS Implementation: Testing and Going Live

After all the work you’ve done planning, configuring, integrating, and migrating your content to your new LMS, you are probably very eager to get the new system up and running for your users. Don’t rush; you’re at a critical point. Many organizations do not provide adequate time in the project schedule for testing and debugging the system. If there are bugs in the system configuration, data, content, or integrations and you do not catch them before going live, your help desk may be overwhelmed with calls. Your users and sponsors may have a negative reaction to the system.

There are many administrator and user features to be tested, so you’ll need to be methodical and thorough. You can start planning your testing effort early in the implementation project in parallel with other tasks, but the testing itself happens at the end of the project, just before going live.

Planning your go live process is also critically important. Going live involves a sequence of steps that must be carefully mapped out and followed. To ensure success and a smooth transition to your new LMS, you will need to anticipate potential problems and determine how to mitigate them in advance so that you can respond quickly to any problems that emerge.

Step 5: User Acceptance Testing

The last major step before going live with your LMS is to conduct user acceptance testing, or UAT, as shortened in the product development world. In this case, your organization is the “user,” and you are testing the LMS to make sure the vendor has delivered a fully working, bug-free system. You are also testing to make sure your configuration, courses, and data are available and working properly in the system. Your testing continues until you are satisfied, at which point you are essentially “accepting” the system from the vendor.

The key to user acceptance testing is to be thorough in testing every part of the system. Your goal is to find and correct bugs before you go live. Keep in mind that your users will be affected by any bugs that you miss during this phase. When people have a bad experience in the first few weeks or months after the LMS is introduced, their trust in the system and in your organization can be damaged. It can be very difficult to recover from that. All the time and effort you put into discovering and correcting bugs in the system before you go live will be well worth it.

Start by convening the core team to brainstorm a list of procedures to test. Include all the procedures your team can think of: those performed by administrators, learners, report users, instructors, and any other security roles you have defined. Table 9-1 provides some examples of learner, instructor, and administrator procedures to be tested.

Once you have your list of procedures to test, you will need to develop a test script for each procedure. Divide the list among team members. There is likely to be a large amount of work to be done here, so you may want to engage some of the extended team. Ask each team member to run through the procedure and make a note of every action performed: menu items selected, fields entered, checkboxes checked, buttons pushed, and so on.

You can provide a spreadsheet format for the test planners to use and create a worksheet for each test. Each worksheet may include a header with the test name, the role of the person executing the test (administrator, instructor, student, and so on), and any requisite data required to be configured in the system to run the test. For example, if your test specifies that a student find a course and register for it, the student’s account and the course must both be present in the system. The rows in the worksheet may represent the steps in the procedure. You may want to include three columns to represent a description of the action to be taken, a description of the expected result, and a place for the tester to enter the actual result. Table 9-2 illustrates an example of a test script format, where the tester would fill in the column labeled Actual Result.

Table 9-2. Example Test Script

 

After all test procedure scripts have been defined and documented, identify resources to perform the testing and create a test schedule. Here, you will want to involve extended team members. Assign relevant test procedures to team members based on their role in relation to the LMS. The tests they perform will begin to acclimate them to the new system, increasing their proficiency and confidence in using the system before you go live. You may want to schedule no more than one or two hours of testing per day for each tester so that they can continue to perform other work duties.

Your test cases will require specific data to be set up in the system. Do your best to execute the tests in a logical order to make test setup easier:

• Test 1: Administrator creates a course.
• Test 2: Student registers for the course.
• Test 3: Instructor marks student complete.
• Test 4: Student views transcript.
• Test 5: Administrator runs a completion report for the course.

Establish a spreadsheet or database for compiling and tracking all the bugs your testers find. Alternatively, consider using a commercial bug-tracking system. There are several good ones available in the cloud, which you can license for use during your user acceptance testing phase. Ask your IT department whether they already have one in place that you can use. These systems allow multiple users to check and update the status of bugs, run reports, receive notifications, and so on. Depending on the number of people involved, a single person tracking the bugs in a spreadsheet can work equally well.

To manage the bugs uncovered in your testing:

Once all tests have been completed and all critical and moderate bugs have been fixed, you are ready to go live with your new LMS. I have seen many situations where organizations had to postpone the target go live date until user acceptance testing was satisfactorily completed. While not ideal, this delay ultimately benefits the organization by going live with a fully tested and stable LMS that users can rely on.

Step 6: Go Live

The final step is to go live with your new LMS. Many organizations use IT jargon by referring to the date they go live as GoLive or Day One. To prepare for going live, it is a good idea to meet with the core team to brainstorm a list of risks and a contingency plan to mitigate each risk. Notify your course owners and administrators well in advance of the go live date. Advise them to avoid scheduling critical training during the week before and the week after the go live date, if possible. Release the appropriate communications to your sponsors, extended team members, and end users to ensure that everyone is aware and expectations are set.

Preparing the Users and Administrators

LMS products must strike a balance between offering robust functionality and being easy to use. Due to the many ways they are used by various organizations, most systems are complex enough that they require some training, particularly for administrators. During your user acceptance testing effort, you may have identified some administrative procedures that could benefit from a job aid, an administrator guide, or hands-on training. You should prepare and distribute training materials and schedule training activities prior to going live. Some of the organizations I worked with have scheduled weekly or biweekly LMS administrator web conferences, where the LMS screen is shared and advanced features are demonstrated, time-saving shortcuts are introduced, and emerging questions and concerns are discussed.

You may also consider creating short videos or systems simulations to orient first-time users to the basic system features, capabilities, and navigation. You can make these self-paced tutorials available on the LMS homepage.

While many LMS users may prefer to explore and learn to use the system on their own, others may prefer to learn through a tutorial or hands-on training. While this type of training is not always necessary for a new LMS implementation, it can be very helpful in getting the organization up to speed quickly, minimizing disruption, and helping to increase user satisfaction.

Preparing the Help Desk

When planning how to resolve any issues end users of your LMS may encounter, it may be helpful for you to think in terms of a tiered support structure. Tier one is your help desk. This is the front line; your help desk responds directly to user inquiries. The help desk may operate during business hours, extended hours, or around the clock. It may be accessible through the phone, email, or online chat. Check with your IT organization on how they provide help desk support for other enterprise systems. They may have staff, procedures, and help desk software that can be used to support your LMS.

Tier two typically consists of LMS administrators who troubleshoot problems that cannot be resolved by tier one. If tier two cannot resolve the problem, they will probably know how to direct it to someone who can.

Tier three may consist of several different groups, each responsible for its own set of issues. For example, the LMS vendor is responsible for system errors and other malfunctions; your IT organization may be responsible for problems with the network, single sign-on, or data feeds; and your e-learning course developers may be responsible for bugs in a specific online course. The idea behind a tiered structure is that most of the user issues get resolved at tier one, while more complex problems are escalated to the appropriate tier.

Many LMS vendors provide support to your administrators, but not to end users. You may need to provide your own help desk or contract with a third party to respond to questions and problems reported by people who take courses in your LMS. If so, you will need to prepare the help desk to resolve as many issues as possible, so that fewer problems are passed up the chain to your administrators. In the IT world, the service expected from your help desk beginning on the go live date is called day one support.

Consider providing a help desk script to your support staff. Create a list of anticipated call topics and describe how to respond to each one, including questions to ask, information to record, and solutions to suggest. Some examples include how to get to the LMS, how to recover a lost password, what to do if something does not appear in your transcript, how to register for a course, and so on. Provide an escalation process for issues your help desk staff cannot resolve, including whom to contact for any given issue and what information to collect from the user (such as username, email address, phone number, date and time the problem occurred, steps to reproduce the problem, screenshot, and the error message) before escalating a problem.

Before you go live, meet with your help desk staff to walk through the help desk script and answer any questions they may have. Provide them with access to the LMS and demonstrate the resolution procedures for any issues that may require them to use the system. Some LMS products provide a way for help desk staff to “impersonate” a user in the system to see exactly what the user sees. Introduce this and other relevant features to the help desk staff. The more familiar they are with the LMS prior to the go live date, the more helpful they can be to your end users.

The Blackout Period

As mentioned in the “Migration Stages” section of chapter 8, you will need to migrate data from your old LMS to your new LMS in three stages. The first stage is a test of the data migration process with just a sample of data. The second stage is a full data migration, after which user acceptance testing is performed. This testing can take several weeks, and during that time, people will continue taking courses and generating new data on your old LMS. So the final data migration stage occurs during the end user blackout period, when the old LMS is deactivated so that no one can use it and any new or changed data are migrated one last time. This is called a delta data migration because it only involves the difference between the state of data in your old LMS between the second and third data migrations.

While you’re performing the final, delta data migration, make sure that neither your old system nor the new LMS is available to end users so that no more data are generated or changed. During this “blackout” period, you will want to shut down your old LMS, perform the delta data migration, spot-check the data and functionality in the new LMS, and redirect the legacy system’s web address to the new LMS. If anything goes wrong, you should be prepared to reactivate the old system and postpone your go live date until the problem is diagnosed and resolved.

Some organizations plan for two blackout periods: the one described previously for end users, and another for administrators. The administrator blackout occurs after the second data migration and lasts until going live. Its purpose is to limit the delta data migration to transactional data, which speeds the go live process and reduces risk of errors and data loss by eliminating the need for course data migration and simplifying go live testing. During an administrator blackout, LMS administrators are instructed not to add new courses, change or deactivate existing courses, or schedule new classes in the old LMS. This can be accomplished in some organizations with adequate up-front planning and expectation setting. For other organizations where the pace of training is rapid, an administrator blackout is not practical.

To minimize disruption to your end users, you may want to schedule the end user blackout and final go live tasks to occur over a weekend, assuming that the resources involved in your go live process are available.

Key Takeaways

This chapter focused on the final two steps of the implementation process: user acceptance testing and going live. The key takeaways are:

• User acceptance testing involves thorough, methodical end-to-end testing of all learner and administrator features of the LMS. This is where you, the customer, make sure the LMS is configured, loaded, and operating exactly the way you expect—before you go live!
• There are a number of things you must do to plan your user acceptance testing efforts, including identifying and documenting test cases, preparing testing worksheets, scheduling the tests, and assigning people to perform them.
• As your organization executes the user acceptance testing process, you will need to meet at the end of each day to inventory, document, prioritize, and route bugs that are found. You can create your own bug-tracking spreadsheets or use a commercial bug-management system.
• You must update the status of bugs as they are reported and fixed. Even after a bug is declared “fixed,” you must be sure it is retested and the fix is confirmed. You may also need to rerun some of your test cases to make sure that fixing one bug didn’t introduce other bugs.
• Going live involves some critical steps that must happen in a prescribed sequence. You will need to work with your team to anticipate things that might go wrong and how to mitigate them.
• You must prepare your users, administrators, and help desk in advance of going live.

Once your LMS has gone live, you need to properly organize your LMS operations. The next chapter will focus on getting the most from your LMS with standards, policies, processes, and governance.