The journey post go-live starts with the goal of keeping the system up, but you ultimately need to lead it through to business transformation. While the initial focus is to keep the lights on, eventually, the focus needs to shift on taking the business to the next level, with the new ERP platform as its foundation. The organization needs to start taking ownership of the new platform in this phase, and start scaling back on consulting resources.
It is not uncommon to see firefighting situations as you fly into the storm at go-live. The amount of preparation and testing that was done ahead of the release will minimize the impact of the storm. Moreover, the way you react and keep control of the situation can magnify or reduce the impact.
In this chapter, we will talk about the different phases post go-live, what to expect, and the preparation required for these phases, along with a few real life examples.
In this chapter, we will learn about handling the key components of post go-live, which are as follows:
- Initial stabilization
- Proactive preparation – what's coming
- Post-implementation review
Now that you are live, the first few weeks—usually until the first month-end close—are going to be long weeks. During the initial stabilization period, it is important to ensure that the business is not hurting due to system issues. Communication is most critical, and investments in your support plan will pay off now by handling the communication to all stakeholders effectively.
As part of the initial stabilization process, you should be prepared to handle the prioritization of issues, management of bug fixes (along with their deployment into production), as well as transitioning support to the business by building a repository of FAQ's that they can begin to manage. The following sections walk through these key components of stabilization.
Regularly triage the open issues with the business and IT. Prioritize issues and provide due dates. Communication is a very important part of this process:
- Limiting the noise: Very often, there is more noise than real issues. You need to set up a process to control any miscommunication and handle the noise. Direct reviews with the business teams and triage can help reduce the noise and get first-hand information.
- Don't forget about your support plan: As noted in Chapter 12, Go-live Planning, you should develop a support plan that includes a process for the business users to document, report, and, ultimately, track issues. It is very important that the entire organization understands this process and abides by it to aid in issue prioritization. This also provides another tool to help limit the noise.
- Confidence and body language: Things may be messy initially. However, it is important for the leaders to demonstrate that things are in control. ERP projects are complex by nature, and you are not the first one going through such issues. Higher stress and reduced confidence can only add to the issues.
Now that you are in production, there are multiple aspects of bugs that need to be fixed. The business impact of each bug needs to be evaluated carefully:
- Stop the bleeding: How do you stop more damage because of an issue? Say, for example, you know that the invoices for XYZ customers are not showing freight. Put a hold on the invoicing process for that customer, and do not send any more invoices until the issue is fixed. That would avoid any further damage.
- Recovery of data: While the team is fixing the issue for future occurrences, an analysis needs to be done to understand the scope/impact on the data and the way in which the data that has been damaged by the issue can be recovered/fixed.
- Fix for the issue: The issue needs to be analyzed, and the root cause needs to be fixed, tested thoroughly, and promoted to the production environment.
- What we learned and how to avoid it in the future: An analysis needs to be done to find out the cause for this issue to reach the production environment and the reason it was not caught in the previous rounds of testing. The analysis should help in finding a solution so that a similar instance can be avoided in the future.
- RCA process: Implementing the RCA process as part of bug fixes would help reduce the repeating bugs and improve the process.
One of my customers had a very thorough process for RCA (Root Cause Analysis). Every time that a bug was introduced in production due to any release, you had to put together the complete RCA for the issue. It would include documenting the symptoms, impact to business, how and when the problem was reported, steps taken, short term, and long term resolution, and the changes made in the process due to the learnings. They made sure that they would not have any issue because of the same/similar mistake in the future. Moreover, it was so much work (and the team had to answer so many tough questions in a room full of people) to put together an RCA that the team made sure they did everything they could to avoid any issues in the future.
It is important to have a formal release process and schedule. You need to reduce the number of deployments every week. Otherwise, it will be hard to provide a stable system to the business. The following are some components of a good deployment process:
- The deployment process should be well-documented and automated.
- The developers should not deploy into production. Make sure that your support team is up to speed on the deployment process prior to going live. The customer should own this process, and to enable ownership of the system and to save consulting dollars.
- Thorough testing and reducing deployments is important. While there may be some low-hanging fixes, they may backfire and create issues in production. Troubleshooting of issues becomes difficult with too many changes going into the production environment.
- Avoid deployments close to the month-end or on peak business days.
- Have a set of scripts created to verify the common business processes after the release. This is very important when you have multiple systems integrated with Dynamics AX. Changes in one system (or timing issues in deploying the related changes) may break integrations. Continue to improve the release validation process based on the issues that come up and which have a high business impact.
Note
I was involved in a post-implementation review, and what triggered the review is as follows. The customer/partner had the development team working on the deployment. On the second day of going live, the developers made a mistake costing fifty-thousand dollars while doing a deployment. They forgot to take a backup of the database (they were tired due to long hours, and are human) and deployed the new code. Unfortunately, there was some object conflict as part of the database sync process, and Dynamics AX ended up dropping and recreating the table for credit cards and authorizations. Fifty-thousand dollars worth of credit cards and authorizations were lost; B2C customers could not be charged.
Now that you have seen the trend of the support issues, start adding to the FAQs and troubleshooting techniques that can be used by the users prior to reaching out to support.
- Clear the usage data: There will be a high number of deployments required to stabilize the environment, and these deployments may cause conflicts with personalization on the forms. As a result, specific user(s) may see unexpected system behavior. Such issues may go away when the user resets their usage data. (Tools | Options | Usage data | Reset).
- Shortcut keys: Prepare documentation for the shortcut keys that can be used.
- Execute in CIL: If you have issues with CIL, some processes that run in IL may have issues as well. Try disabling the CIL operations as a temporary fix (Tools | Options | Development | Execute Business logic in CIL).
- Limit the number of windows.
- Limit the inactive session time.
- Uploading or downloading files from the local computer while using the remote desktop or a Citrix client for Dynamics AX.
- Printing from Dynamics AX to network printers.