CHAPTER 8

Integrating Your LMS With Other Systems and Migrating Legacy Data

As discussed in the last chapter, LMS implementation typically involves six major steps. This chapter covers steps three and four, integration and migration.

Integration is required when two applications must continually share the same data. Migration happens when data must be moved out of an old system that will be retired and into the new system that will replace it. You will need to work with your IT department on these steps. If you don’t have an IT department, ask your vendors to help with integration and migration. Typically, your LMS vendor will provide professional services to help you manage these two steps. You can also engage the vendor of any systems to be integrated with the LMS and, for migration, the vendor of the old LMS you are replacing.

While some organizations deploy an LMS without integrating with any other systems, most need to fit the LMS into their broader IT technology infrastructure. This may involve integration with HR systems, single sign-on solutions, portals, enterprise search tools, financial systems, data warehouses, and other applications.

Migrating your existing courseware and data to an LMS can be one of the most complex aspects of implementation. If this is your organization’s first LMS, you may need to migrate data from a homegrown system, spreadsheets, or a lightweight database. If you are moving from one system to another, you’ll need to get the data out of the old system and into the new one.

No two LMS products are exactly alike; each product has its own data structure. Your new system may require fields that are optional or completely missing from your old one. Your new LMS may allow up to 1,024 characters in the course description field, while your old one was restricted to 512 characters. Your new LMS may keep track of the user’s course registration date, while the old one only kept track of the completion date. These differences complicate the data migration process.

Step 3: Systems Integration

Systems integrations streamline the user experience and improve data integrity by synchronizing data between systems. Let’s say you have two separate systems that serve the same set of users. Both systems have user accounts and user profiles. Without integration, every user must create a separate user account, populate a separate user profile, and log in to each system separately. If users are not careful about maintaining the exact same information in their accounts and profiles in both systems, the data in the two systems will not match. When the data in two systems are duplicated and do not match, it is difficult or impossible to determine which system has the correct data.

If these two systems were integrated, a user would simply create an account and profile in one system. The data from that system would automatically be synchronized with the second system. A user would only need to log in to one to be granted access to both systems.

There are a number of systems with which an LMS may be integrated. This chapter describes some of the most common systems integrations, many of which you are likely to encounter.

Your core team member from IT will probably take the lead on any integration projects. The amount of time needed to complete an integration project varies based on the complexity of the integration and the availability of resources to specify, design, develop, and test the integration solution.

Systems That Manage User Accounts and Profiles

One of the most common LMS systems integrations is with another system that contains user accounts and profile information. For an academic organization, this is a student information system, which contains information about each student’s degree program, major area of study, coursework, grades and grade point average, academic year, contact, financial status, and demographics. For a professional association, this is a member management system or associate management system, which contains information about each member’s professional role, professional certifications, contact, membership status, and demographics. For a company, this is a human resource management system or human resource information system, which contains information about each employee’s job role, organization, region, hire date, and level. Some businesses use their LMS for training the “extended enterprise,” which may include some combination of customers, suppliers, dealers, agents, and distributers. These organizations have some sort of identity management solution in place to keep track of their extended enterprise users. If your organization has one of these systems, you will probably want to develop a system interface to synchronize user accounts with the LMS.

It does not make sense to manage the same data separately in two different systems. Because these user accounts and profiles originate in another system, you can avoid countless hours of duplicated effort and costly mistakes by simply importing the data from the originating system, rather than re-creating it manually in the LMS.

You can import the user accounts before going live with your LMS so that you can test the various features and functionality of the system. But what happens when new people are added to the organization, others leave, and user profiles change? Your LMS needs to remain synchronized with the originating system. This happens using recurring data feeds, which can be automated so that no human intervention is required. They can be scheduled to occur during off-peak hours, typically nights and weekends. If your organization operates in multiple time zones, check with your IT organization. They will have knowledge of your organization’s network traffic and will be able to recommend the best time for data feeds to be scheduled.

LMS products provide mechanisms to import recurring data feeds from these systems. Your IT group will need to develop a program to extract the data from the user account system and format it according to your LMS vendor’s specifications so that it can be imported into the LMS. The entire process can be scheduled to run automatically on a nightly basis. Your IT group may refer to this as a nightly feed or extract-transform-load (ETL) process.

If your organization does not have a system where user accounts are originated, you will need to configure the LMS to create and manage user accounts. This is often the case in many government agency LMS implementations.

If you have a limited number of users, you may want to implement an account creation or change request process outside the LMS and restrict LMS account creation and management to administrators. This will ensure that only authorized users can access the LMS and decrease the risk of user-generated errors. For example, a community college instructor might create accounts for the continuing adult education students participating in a semester-long class.

If you have too many users to manage accounts administratively, you can configure the system to allow users to create their own accounts. Be aware that this approach is more likely to result in duplicate accounts, incomplete user profiles, and other inaccurate user data. One approach to remedy this is to find an LMS that requires users to enter a unique email address when creating a new account. The system then emails the user and waits for a response before validating the account. This approach is used by many public websites where users can create their own accounts.

Whatever approach you take to managing user accounts, be sure to check your license agreement with the LMS vendor to ensure that you are licensed for the number of users who have accounts in your system.

Single Sign-On

Another frequent LMS integration is with a single sign-on solution. To avoid requiring end users to log in to different systems with different logins, some organizations have implemented single sign-on, which enables a user to log in to the network once and gain access to multiple applications through a silent authentication process that accepts credentials from the SSO solution and circumvents the login page of those systems.

Each LMS product may support its own selection of commonly used SSO methods. You may hear your vendor spew head-spinning technical terms like SAML (security assertion markup language), oAuth (open authorization), Microsoft ADFS (Active Directory Federation Services), or Kerberos. Put your IT department in touch with your LMS vendor directly to determine how they can best integrate the LMS with your organization’s SSO solution.

An important factor your IT organization must be aware of while planning LMS SSO implementation is the handling of deep links, which may be embedded in system-generated email notifications. A deep link is a web address to a specific page in the LMS, such as a course details page or a user profile page. Usually, people must already be logged in to the LMS to view these pages. So, clicking on the deep link in an email message when you are not already logged in to the LMS may result in an error message like “page not found” or “you are not authorized to view this page,” or even more confusingly, it may drop you off at the LMS homepage. The deep link problem is not limited to email messages. You may also need to place a deep link on your company’s portal or another website, to function as a shortcut to a specific page within the LMS.

The best way to handle a deep link is to ensure that the SSO solution checks whether the user is already authenticated and, if not, redirects the user to the login page. It should then redirect the user to the original deep link. If, after a user logs in, the LMS finds that the user’s profile has some required fields that have not been populated, it may redirect them to the profile page(s) to enter values into those fields. During the entire transaction, the SSO must hold onto the deep link URL so that the user can be sent there after authentication is completed.

Portal

In some cases, it makes sense to pull data from the LMS and display it on your portal. The data may be general, such as a list of courses, or personalized, such as a transcript. I have worked with companies that have advertised courses by placing them in noticeable places on the portal, and with professional associations that have advertised their certification programs with a general description and link to each course in the program.

Many LMS products offer two methods for integration with a portal. One of these methods is to capture the web address of a specific course in your LMS and paste it into a portal page as a deep link. When users click on the link on the portal page, it takes them directly to the course details page in the LMS. This approach must be maintained manually. For example, if you deactivate a course in the LMS, you must be sure to remove any corresponding deep link from your portal pages.

Another method, called an application programming interface (API), allows your IT department to access the data and functionality of your LMS programmatically. Your IT group can write a program that uses the LMS API to pull data from your LMS and post it in the portal. This approach can be fully automated so that no human intervention is required, and, if designed and developed properly, it ensures that the LMS data shown in the portal are always accurate.

Ask your LMS vendor to provide documentation on how to use its API and provide it to your IT department. Explain to IT what you are trying to accomplish through your portal integration and ask them to evaluate the LMS API to determine whether it can be done. You may want to provide some mock-ups illustrating your vision for what the portal pages will look like once the integration is complete. Mock-ups will help your IT department plan and implement the solution as you intend.

Enterprise Search

A 2013 study I conducted with the eLearning Guild showed that one of the three most important LMS features to respondents was search (Foreman 2013). The same survey showed that search also happened to be one of the three features respondents were unhappy with in their current LMS.

Many large organizations have deployed an enterprise search platform to find content stored in multiple systems. In this case, an employee can search on a topic and view results from the company’s portal, knowledge bases, document management systems, and social networking sites. For example, let’s say an employee who wants information on how the company handles quality assurance (QA) for the products it manufactures searches “quality assurance.” A list of items is returned, including a hyperlink to the employee portal page about the organization’s QA policies, a collection of knowledge base articles explaining quality processes and giving examples of how they are applied, several documents containing specific quality process steps and procedures, and hyperlinks to contact information in an employee directory for QA specialists who can answer questions on the topic.

In the past, LMS content has rarely been included in enterprise search results. In recent years, some organizations have integrated their enterprise search platform with the LMS so that courses are included in the search results along with documents, posts, and articles from other repositories. In the QA example, search results would also contain courses on quality assurance.

There are two main methods for integration: federated search and aggregated search.

A federated search solution allows a user to enter a search string into the enterprise search box, which in turn triggers multiple concurrent searches using the native search features in each content repository. Figure 8-2 illustrates how a federated search works.

The key benefit of this approach is that a user can enter a search string in one place and get results that include LMS courses along with information and documents from a document management, knowledge management, or content management system.

An aggregated search solution “crawls” each repository periodically, creating a centralized, master index of the content. The user’s search string is compared with the master index, and results are displayed with links to their locations in each of the repositories. Figure 8-3 illustrates how an aggregated search works.

This approach has the same benefit as the federated system, as well as the added benefit of a superior search tool. Most enterprise search platforms operate similarly to the web searches to which people have become accustomed. These tools offer advanced search features such as proximity ranking (most to least relevant), controlled vocabulary (synonyms and acronyms), all forms of a word, and spelling correction (“did you mean…”). The challenge lies in getting the search platform to crawl the LMS database and index its course titles, descriptions, and metadata in a way that makes it searchable. This can involve some programming time and effort from your IT department, and the results depend on the quality of the LMS data. When done well, this type of search is far superior to the search feature offered by most LMS products, where an incorrect spelling, alternative word form, or mismatched capitalization can exclude relevant content from the search results.

Integrating your LMS with a federated search platform is often accomplished through some programming that uses the search platform’s API to trigger an LMS search using the LMS API.

Integrating your LMS with an aggregated search platform is often done through development of a custom software program that runs on a scheduled, recurring basis, such as nightly. The program uses the LMS API to retrieve the course title, description, and metadata and a hyperlink to the course. It then creates a temporary webpage with this information for each course. These webpages are not available to users. They are used only by the search platform, which crawls the pages and adds the information to its index. When a user searches for something that matches a course in the LMS, the search platform shows the relevant information and provides a hyperlink to the course page in the LMS. The program runs nightly to keep the search platform synchronized with any changes to the LMS courses.

General Ledger

Your accounting department likely uses a financial system to manage the books. At the heart of this financial system is the general ledger, which contains a complete record of the organization’s financial transactions. The general ledger records external transactions with customers and suppliers as well as internal transactions, where money is moved between departments. Let’s say that department A purchases a course and makes it available in the LMS. No financial transactions occur when people from department A take the course. But when people from departments B and C take the course, their departments must reimburse department A. This interdepartmental billing is managed in the general ledger.

If your organization manages interdepartmental billing for courses that are licensed by one organization and taken by people in other organizations, then you may want to consider automating the process by integrating your LMS with your general ledger.

The first step is to ensure that the courses and users in your LMS are configured with the appropriate data to support interdepartmental billing. Every relevant course must have a price and a sponsoring organization. This is the organization that licensed or developed the course and will receive payment whenever someone in another organization takes the course. Every employee in the LMS must be associated with the organization in which they work. This is the organization that will pay for the employee to take the course. Decide whether you want to transfer payment from the user’s organization to the sponsoring organization when the user enrolls in the course, or when the user completes the course.

The next step is to make sure that the LMS provides a method to extract the information about course enrollment or completion status, the course price, the user’s organization, and the course’s sponsoring organization. These data may be accessible through the API, a scheduled report in Excel format, or some other method.

Finally, your IT group, your LMS vendor, or a third party will need to develop a program to extract the LMS data and, in the financial system’s general ledger, debit course price from the user’s organization and credit the sponsoring organization.

Data Warehouse

If your organization needs to provide reports on data from multiple systems, such as an LMS or knowledge management, social networking and collaboration, enterprise resource planning, or financial systems, your IT organization may have deployed a data warehouse. In this case, data are extracted from multiple systems and stored in a common database where the data can be combined, reported, and analyzed.

Typically, data can be exported from an LMS in either of two ways: the API provided with the LMS or a data export, which may be a scheduled report generated in a CSV or XML format.

* * *

LMS integration offers many benefits to LMS users, administrators, and the organization. Integration enables data sharing between systems, which streamlines the user experience and enhances data integrity. It can also be used to automate complex transactions, which reduces the risk of data entry errors and omissions. Your integration efforts will require programming support from your IT department or LMS vendor.

A close cousin to integration is data migration. While integration projects enable two or more systems to share and synchronize data on an ongoing basis, migration involves a one-time transfer of data from one system to another, in this case from your old LMS to your new one.

Step 4: Course and Data Migration

If you are changing LMS products or moving from a custom LMS to an off-the-shelf product, you will probably need to move the training data and courses from your legacy system to the new LMS. This is what course and data migration is all about, and it is one of the more technical tasks in an LMS implementation.

First, you will need to decide how much data to migrate and how to map the data from your old system to your new one. Then, you will need to work with your IT department, who will develop a migration software program. There is some “choreography” involved. The data must be migrated in three stages, in a specific sequence at each stage.

If your IT department is not equipped to support you, you will need to get support from your vendors. Your legacy LMS vendor may be best positioned to pull the data out of your old LMS. Your new LMS vendor can help with importing the data into the new LMS.

Deciding How Much Data to Migrate

The first step is to decide how much data to migrate. A good practice is to move as little data as possible. The more data you migrate, the greater potential for errors and problems that may delay your implementation project. Check with your IT and legal departments; they may already have an information retention policy for your organization, which you can use to guide your decision.

Suppose your organization has been using its former LMS for the last 10 years. All the employee training records for those 10 years are stored in the old system. After checking with IT and legal, you find out that your organization’s policy is to retain employment records for three years. So rather than migrating all the data from your old LMS, you decide to migrate just the data from the last three years. You ask IT or your LMS vendor to create a database archive of all older data so that if the need arises, you can ask them to restore the backup long enough for you to run a report. Archiving the data is not difficult; IT probably archives databases and puts them into “deep storage” all the time. Because the older data are not immediately accessible in your LMS, archived data may take several days to retrieve from deep storage, but the removal of older data keeps your LMS clean and simplifies your implementation project.

There may be some exceptions to the policy. Let’s say you have a course, Advanced Inventory Management, that has a prerequisite course, Basic Inventory Management. Before people are allowed to take the advanced course, they must complete the basic one. In this case, you may want to migrate the last five years of course completion data for the prerequisite basic course. This will allow anyone who has completed the basic course in the last five years to take the advanced course.

Another exception to the policy may be compliance and certification courses. Suppose you offer a certification course. Upon completing the course, a person becomes certified. The certification expires after four years, at which time they must take another course to be recertified. In this case, you should migrate the last four years of course completion data for the certification course so that you can maintain proof of certification for anyone who earned their certification three or four years ago.

The amount of data you migrate affects how long the data migration process takes, from programming to execution, testing, and validation. To streamline your LMS implementation process and lower the risk of errors, migrate only the data that are absolutely needed. Use this rationale, along with your organization’s broader data retention policies, to counter objections from team members who think all legacy data should be migrated to the new LMS.

Mapping the Data

As you begin to plan the migration of your legacy data to the new LMS, you will need to do some data mapping. Create a spreadsheet or table listing the data fields in the old LMS and how they correspond, or map, to fields in the new LMS. Do this for user data, course data, and user-course status data.

Your data map will ensure that the data extracted from your old system are placed in the appropriate location in the new LMS. As you develop your data map, you will need to address any incompatibilities between the way the data and courses were stored in the legacy system versus the new LMS.

Your data map should include the field type, such as text, number, or date, and the field length for each data field in the old LMS so that you can map it to an appropriate data field of the same type in your new system. Table 8-1 shows an example of a data map for migrating data from an old LMS to a new one.

Pay attention to the length of text fields, like the course description, to ensure that the text coming from the old system is not truncated when entered into the new LMS. If the text is too long for the new LMS, decide whether to edit the text down to the appropriate length before or after you move it. Table 8-2 shows an example of old LMS data that are too long for the new LMS.

Sometimes, data mapping requires some creative problem solving. For example, your old LMS may have stored each user’s manager as an email address, while the new LMS may store the manager’s user ID. In this case, you will need to find a way to identify manager user IDs from their email addresses in the legacy data to populate that field in the new system.

Your new LMS may enforce stricter data integrity rules than your legacy system. For example, let’s say that some of the courses in your legacy system were entered in such a way that their end date was earlier than their start date. The old LMS did not have a problem with this anomaly, but the new LMS has built-in rules to ensure the integrity of the data and will not accept these courses. In a situation like this, you would need to adjust the dates in the old system before moving them to the new LMS.

You may find opportunities to normalize the data from your old LMS. Normalization is a best practice for good data design, because it reduces data redundancy and improves data integrity. For example, some fields may have been entered as text in the old system, but they can be implemented as a drop-down list of predefined values in the new LMS. Using predefined values, wherever appropriate, improves data consistency, making it easier to use data in reports. In this case, you will need to clean up the old data so that each text value can be mapped to the appropriate predefined drop-down list value in the new system. You may also find that you were keeping track of the same information in multiple fields in the old system, which can lead to errors and inconsistencies.

Your IT organization will likely have the skills to help you find opportunities to normalize and improve your data as you map the data from your old LMS to the new one. Use the IT architect on your implementation team, or someone from IT, as a resource.

Migrating User Data

Once you have decided how much data to move and mapped your data to the new LMS, you are ready to get started with your data migration.

LMS data must be migrated in a specific sequence. First, the user accounts and profile data must be loaded into the system. Next, the course content and descriptive data are loaded. The last step is migrating the transcript data. Transcript data describe the relationship between users and courses, so the users and courses must already be in place before the transcript data are loaded.

If you have decided to integrate the LMS with a system that manages user accounts, as described earlier in this chapter under step 3, the feed must be developed, implemented, and tested before you migrate anything else.

To develop your user data feed, you must:

In the event that you have no other system to supply your user data, then instead of a feed, you will need to migrate the user data from your old LMS.

To migrate user data from an old LMS, you must:

Migrating Standards-Based Courseware

The courseware migration process depends on the type of LMS and the courseware standards.

If you are moving from one academic LMS to another, you will need to export your course to the appropriate format for your new LMS and then import it. The IMS Global Learning Consortium^® has developed a number of standards for doing this, including the Common Cartridge^®, LTI^®, and QTI^® standards, all covered in chapter 5. Check to see whether your legacy and new LMS support these standards. It will make course migration between academic LMS products a lot easier.

If you are moving from one corporate LMS to another, the process can be a bit more complex.

If you have implemented SCORM-based courses, you will need to reinstall the SCORM package for each of the courses on your new LMS. When courses are developed and exported to SCORM, a zipped file (.zip) is generated containing all the course files. This zipped file is called a SCORM package. Some LMS products provide a batch method that enables you to queue up the installation of multiple SCORM course packages. Other LMS products will require you install one SCORM course package at a time.

Even though the SCORM courses may have worked well on your legacy system, they may need some adjustment to work optimally on the new LMS.

For SCORM courses, you will need to:

For courseware developed around a specification other than SCORM, the process is likely to be easier; however, there are still some risks.

For AICC courses, you will need to:

Experience API and cmi5 are newer specifications. Increasingly, organizations are deploying courses using these standards instead of SCORM. A best practice is to host your content on a file server that is separate from the LMS. Doing so will require fewer URL adjustments and make migrating to a new LMS much easier. However, the reality is that many organizations upload their cmi5 courses to the LMS in a way similar to SCORM content. In this case, migrating cmi5 courses to a new LMS will require a process similar to migrating SCORM.

For cmi5 courses, you will need to:

Migrating Course Data

There are course-related data in the LMS for all courses, regardless of whether they are webinars, classroom, or self-paced. These course-related data include a unique course ID, course title, description, schedule, locations, instructors, metadata, credits, and more.

If you’re implementing an academic LMS that supports the IMS Global Learning Consortium standards, your course data can be transferred using the Common Cartridge, LTI, and QTI specifications. Work with your old LMS vendor to extract the data and your new vendor to import them.

For a corporate LMS, you must:

Migrating Transcript Data

Once you have added users and courses to your new LMS, the last step is to move your legacy transcript data to the new system. Transcripts represent the relationship between users and courses, which is why the users and courses must be present in the system before you move the transcript data. Remember to migrate the least amount of historical data necessary based on your organization’s data retention policy.

To migrate transcript data, you must:

Migration Stages

The user, course, and transcript migrations should be performed in three stages:

Clearly, you’ll want to avoid any data loss, so the first stage is to migrate a relatively small sample of the data and test it to verify that the migration programming is working properly.

The second stage is to migrate all the current data. Once this is done, you will be able to enhance the course setups by configuring any features and functionality that you may not have had in your old LMS and perform user acceptance testing.

The third stage happens just prior to going live, when you will transfer the data that have been added or changed since the full migration you performed in the second stage. You may want to manage separate “blackout” periods, one where LMS administrators do not add new courses or classes, and another when end users do not register. These blackouts will help ensure that your data migration is thorough and will avoid administrator rework, but you will want to minimize the length of each blackout period to avoid disrupting your training operations too much. More information about how to manage these and other steps related to going live are described in chapter 9.

Key Takeaways

This chapter focused on steps 3 and 4 of the implementation process: systems integration and data migration. The key takeaways are:

• LMS products are often integrated with systems that manage user accounts and profiles such as the human resource management system used by a business, the member management system used by a professional association, or the student information system used by an academic institution.
• Implementing a single sign-on solution streamlines the user experience for people who have already logged in to the organization’s network, allowing their login credentials to be passed to the LMS automatically so that they do not need to log in again.
• LMS products may also be integrated with portals, enterprise search tools, a company’s general ledger, or a data warehouse.
• There are many ways to integrate LMS systems, including data feeds, application programming interfaces, and deep links to specific LMS webpages.
• When replacing an old LMS with a new one, the training data from the old system must be migrated to the new LMS. This process must be done sequentially and repeated in three different stages.
• A guiding principle is to move as little data as possible to reduce risk and speed the implementation process.
• To migrate data from your old LMS to your new one, you will need to create a map of the data fields across the two systems and compensate for differences in field types, character lengths, and data integrity rules.
• Moving standards-based e-learning courseware can be tricky and must be coordinated with the training data migration process. Different procedures are required to move SCORM, AICC, and cmi5 courseware.

This chapter introduced the types of systems integrations you may need to position your LMS in the broader IT technology infrastructure. We have also covered all the processes involved in moving your data and content from a legacy system to your new LMS.

The next chapter covers the final steps of the implementation process: testing and going live.