Chapter 19
Testing, Rolling Out, and Sustaining the Data Warehouse

The bitterness of poor quality remains long after the sweetness of meeting the schedule has been forgotten.

Urban Wisdom

In this chapter you will learn how to:

•   Put together and execute a testing approach that ensures a trusted system

•   Create and execute a plan for rolling out your data warehouse

•   Organize a sustainable program to keep your BI and data warehousing efforts moving in the right direction.

In Chapters 1 through 18 of this book you gained knowledge about the data warehousing and BI software development lifecycle. You started with the business case and project management. Next you learned about business and technical architecture. You explored data topics and then BI tools and applications. Now is the time for a strong finish.

The chapter is organized in three sections that show how to successfully complete data warehousing / business intelligence projects.

•   Section 19A – Testing the Data Warehouse

•   Section 19B – Rolling Out the Data Warehouse

•   Section 19C – Sustaining the Data Warehouse.

When you have completed this section you will be able to:

•   Discuss elements of the data warehousing test plan

•   Understand the roles needed for data warehouse testing

•   Discuss data warehousing test responsibilities

•   Link requirements with tests

•   Specify the types of tests required

•   Select test environments.

Testing the data warehouse and business intelligence system is critical to its success. Without testing, the data warehouse could produce incorrect answers and quickly lose the faith of the business intelligence users. Effective testing requires putting together the right processes, people, and technology, and deploying them in productive ways.

Who should be involved with testing? The right team is essential to success:

•   Business Analysts elicit and document requirements

•   QA (Quality Assurance) Testers develop and execute test plans and test scripts

•   Infrastructure People set up test environments

•   Developers perform unit tests of their deliverables

•   DBAs test for performance and stress

•   Business Users perform functional tests, including User Acceptance Tests (UAT).

When should your project begin to think about testing? The answer is simple – at the beginning of the project. Successful testing begins with the elicitation and documentation of requirements. Without requirements, it is difficult to measure system correctness.

Expect to produce a Requirements Traceability Matrix (RTM) that cross references data warehouse and business intelligence features to business requirements. The RTM is a primary input to the Test Plan.

The Test Plan, typically prepared by the QA Testers, describes the tests that must be performed to validate the data warehousing and business intelligence system. It describes the types of tests to be performed and the required system features that will be covered.

Test Cases are the detailed components that enable implementation of the Test Plan. Each Test Case itemizes steps that must be taken to test the system, along with their expected results. A Test Execution Log tracks each test along with the results (pass or fail) of each iteration.

Typically, multiple environments and database versions are set up and maintained to support the system during its lifecycle:

•   Development

•   Quality Assurance (QA)

•   Staging / Performance Testing

•   Production

•   Disaster Recovery (DR).

These database environments and versions improve productivity and system quality. Developers can be producing new system functionality at the same time as testers that are validating the system without interfering with the business who are using the production versions of the system. A Disaster Recovery (DR) version of the system is kept up to date so that service in not interrupted if the system stops functioning.

The following kinds of tools can facilitate testing and problem correction:

•   Automated test tool – enables tests to be created, managed and run in a repeatable fashion through a user interface.

•   Test data generator – produces data to test the data warehouse based on input parameters.

•   Test data masker – hides or obscures confidential data such a social security number.

•   Defect manager – tracks defects including description, correction and validation.

•   Automated test scripts – tests to be run and validated in a repeatable fashion through files of text based commands.

Developers should perform tests on their deliverables during and after their development process. The unit test is performed on individual components and is based on the developer's knowledge of the requirements and what their deliverable should produce. It should be performed before deliverables are turned over to QA. Tested components are likely to have fewer bugs.

QA Testers design and execute a number of tests including:

Integration Test

Tests the system operation from beginning to end, focusing on how data flows through the system. This is sometimes called “system testing” or “end-to-end testing”.

Regression Test

Validates that the system continues to function correctly after being changed.

Tests that determine how well the system performs with heavy loads of data should be designed and executed.

Extract Performance Test

Test the performance of the system when extracting a large amount of data.

Transform and Load Performance Test

Test the performance of the system when transforming and loading a large amount of data. High volume testing is sometimes called stress testing.

Analytics Performance Test

Test the performance of the system when manipulating the data through calculations.

Does the system produce the results desired by the business users? The main concern is functionality, so business users perform functional tests to make sure that the system meets their requirements. The testing is performed through the user interface (UI), which includes data exploration and reporting.

Correctness Test

The system must produce correct results. The measures and supporting context need to match numbers in other systems and must be calculated correctly.

Usability Test

The system should be as easy to use as possible. Usability testing involves business users exercising the business intelligence system to ensure it does what they wanted and expected.

Performance Test

The system must be able to return results quickly without bogging down other resources.

Quality must be baked into the data warehouse or users will quickly lose faith in the business intelligence produced. It then becomes very difficult to get people back on board.

Putting the quality in requires both the testing described in this chapter and data quality at the source described in Chapter 10, Database Technology, to launch a successful data warehousing / business intelligence effort.

Testing Key Points

•   Testing the data warehousing and business intelligence solution is critical to success. Without testing, the system may produce incorrect results and lose credibility with its users.

•   Testing is a team effort that involves QA testers, business users, business analysts, and support personnel.

•   Documented business requirements determine the criteria for system correctness and are essential for testing.

•   Computer environments such as development, QA, and model office support testing.

•   Multiple types of tests are needed, including unit tests, performance tests, business correctness tests, and usability tests.

 

Projects happen in two ways: a) Planned and then executed or b) Executed, stopped, planned and then executed.

Urban Wisdom

After studying this section you will be able to:

•   Prepare for rolling out your data warehouse

•   Understand rollout critical success factors

•   Avoid rollout traps and pitfalls.

In the prior section, you learned about the importance of testing for data warehouse and business intelligence success. Test plans were discussed, as well as testing by developers, QA people, and business people.

In this section, you will learn that rolling out a successful data warehousing project requires following a Data Warehousing and Business Intelligence Methodology to increase your probability of success. The following topics are explained:

•   Pre-deployment and planning

•   Deployment

•   Training

•   Follow up

•   Assessment of results.

Prepare an announcement that shows business users and others the benefits they can expect from the business intelligence and data warehousing system. When stakeholders understand how the new system can help them, they are more likely to be supporters instead of detractors. You need all of the supporters that you can get. Building and continuing to build support is a continuation of the effort to obtain support starting at the beginning of the project.

Documentation is also critical to successful data warehousing roll-out. Some important documentation items include:

•   Preview of what is coming

•   Schedule of activities and deliverables for the deployment

•   Schedule of training

•   Training manuals and other training materials

•   Procedures and how to instructions

•   Frequently Asked Questions (FAQ) or online help.

Capacity review and performance testing are also important parts of the pre-deployment step. Capacity review helps to confirm the computer resources that will be needed. Performance testing tests the load on the computer hardware and software to make sure that they can perform required activities in a timely manner as needed by the business. Finally, I recommend running a test in the production environment with a full database.

Sometimes a deployment does not work as expected. A rollback and fallback plan should be prepared to respond to this contingency. The fallback plan should describe what needs to be done to ensure that business can continue in case of problems.

A training plan is important to the success of the new system. It should emphasize benefits that trainees will derive from the system. For example, the trainees may spend more time doing the interesting work of analysis and less time doing the tedious work of data gathering In addition, the new system should enable the trainees to be more valuable to the business. Understanding of these benefits should raise their interest level and their focus, as well as help get them onboard with the new system.

The training plan should:

•   Assess the need for training

•   Develop the trainers and power users

•   Plan a pilot deployment

•   Plan training for a wider audience.

Training will also be needed for the follow-up phase. This will include training of users on a wider scale.

I recommend a pilot deployment to a select group, probably power users or those in a particular department or division because you will be learning from the pilot deployment. Start small, using a pilot, then obtain feedback that can be added to the FAQ. This approach will head off future questions, enabling quick answers and smoother operation. Also, changes to the system may be required as the system is used in the real world. Be ready to make adjustments in response to results of the pilot deployment.

A phased deployment works well in many organizations. Start in one department and then extend to additional departments; or, as recommended above, begin with power users who will, in turn, train other users and expand use of the system.

There are a number of ways training can be delivered. One way is through brownbag lunches. This leads to informal, friendly discussions with business people and other stakeholders. Brownbag lunches will enable them to ask questions and get immediate answers. These sessions are also a great source of feedback.

Webinars are another effective method of training for business intelligence and data warehousing. They are available through the Internet or you may create these training sessions yourself. Webinars are very helpful when people who are to be trained are spread out geographically. For example, you may have users in both Kansas and California. In that case, a webinar is a great way to obtain or provide training.

The data warehousing system's support people also need training and support documentation. Acquiring or developing training material should be part of the project plan. Support people should be included in system testing so they gain experience prior to deployment. They have distinct tasks that are different from business users. They need to be ready to carry out operations such as monitoring the system, backing out data and correcting problems. Monitoring tools and recovery jobs should be ready to assist the support team.

Administrative support people will require training, as well. The administrators will be configuring the system to make it more responsive. They will also be performing functions such as adding new users, executing reports and making reports available.

The business intelligence system does not run by itself. The business and other stakeholders will require ongoing support to have a successful system. A help desk that can be accessed by telephone or e-mail is a good way to provide follow-up support. The help desk can be equipped with trained people ready to answer questions in the FAQ and beyond.

A successful business intelligence system will gain in demand and users will want to do more advanced work. Follow-up training in advanced techniques will help with this process. In addition, new people may be joining the group of users, so earlier training sessions may need to be repeated for this audience.

It is important to measure the success or failure of a program like the business intelligence / data warehouse program through follow-up with users of the system. Check with users to make sure business requirements have been met. Conduct surveys, face-to-face meetings, and focus groups to determine what went right and what went wrong. Ask what should be done differently.

A project audit report should be developed and shared with project stakeholders and management. Use the lessons learned from the project to add to the overall knowledge base for future projects. The project audit report answers questions:

•   How well did the project meet goals and objectives?

•   What went well?

•   What did not go well?

•   Was the schedule achieved?

•   Was the budget met?

 

Deployment Key Points

•   Communication is critical to the success of data warehousing and business intelligence rollout. Let business users and others know what to expect.

•   Establish and define roles and responsibilities for deployment, training and system administration.

•   Develop a training plan that is addressed to the appropriate audience, from power users to support personnel.

•   Start with a pilot deployment and then gradually make the system available to a wider audience.

•   Provide follow-up support including a help desk armed with needed information.

•   A follow up assessment measures the degree of success or failure of programs. This leads to improvements in the system and better management of future projects.

 

In our view, successful reform is not an event. It is a sustainable process that will build on its own successes – a virtuous cycle of change.

Abdallah II, King of Jordan

After studying this section you will be able to:

•   Understand how DWBI can be sustained over longer periods of time

•   Move data warehousing forward by following best practices

•   Avoid data warehousing traps.

In the previous section, you gained an understanding of how to rollout a successful data warehouse. Topics covered included pre-deployment and planning, deployment, training, and follow-up and assessment of the results.

Are you concerned that your data warehousing and business intelligence effort could lose steam and grind to a halt? Remember that success is based on the 80/20 rule – 80% of success is based on people and process factors, while only 20% of success is based on technology.

The most critical factors in the success of business intelligence and data warehousing are people and process oriented. To maintain the “political will” to keep the DWBI initiative moving forward, be sure to:

•   Communicate – stay “top of mind”

•   Maintain executive support

•   Support enterprise strategies and mission

•   Focus on business needs and pain points

•   Relate to business capabilities and process

•   Add value beyond standard reporting

•   Build in manageable-sized pieces

•   Expand the user audience through training

•   Obtain and keep funding

•   Identify quantifiable measures

•   Under promise and over deliver.

Successful DWBI initiatives are led through an ongoing governance process with buy-in from top executives. The Return On Investment (ROI) from a winning program can be high (100 to over 1000%) and thus earn executive support.

Disregarding or minimizing the importance of people and process is often at the heart of failed DWBI efforts. Put people and process first to avoid these traps:

•   Working in the background, keeping it quiet

•   Focusing on technology rather than the business

•   Talking in generalities and principles

•   “Build it and they will come”

•   Using “Big Bang” implementation

•   Assuming that benefits are intuitively obvious

•   Using informal organization instead of formal governance.

Failing DWBI initiatives often lose touch with the organization and people that they should be helping.

Technology is a necessary part of winning DWBI projects. While technology by itself cannot make a project successful, poorly chosen or utilized technology can be a killer. Use technology appropriately by:

•   Monitoring data warehouse performance

•   Enabling self-service

•   Securing the data warehouse

•   Adding processing capacity to support the user community

•   Adding storage capacity to satisfy growing data needs

•   Monitoring errors and correcting root causes

•   Using proven architectural patterns

•   Building a federated architecture to support varied environments and requirements.

The need for technical resources often skyrockets for the successful DWBI effort. More and more people access the system, increasing the need for processing power. Also, the amount of data storage needed increases as further detail is captured.

Improper technology or an over-emphasis on technology can drag down the DWBI system. Avoid these traps:

•   Ignoring increases in system use

•   Ignoring system performance monitoring

•   Neglecting upgrades

•   Engaging in “Religious Technology Wars”

•   Acquiring incompatible software or hardware.

The winning Business Intelligence / Data Warehousing effort focuses on the highest priority areas (people and process) while doing a good job with technology. Making the right moves while avoiding the bad plays should lead to victory.

Sustaining Key Points

•   Keep up the momentum. Communicate direction and wins to a wide audience, including executive management.

•   Quantify results – show that the business case is being realized.

•   Avoid people and process traps such as keeping it quiet and using an informal organization rather than explicit governance.

•   Provide good data warehouse and business intelligence performance by monitoring system performance and correcting any slowdown before it causes user problems.

•   Avoid data warehousing technology traps such as neglecting to monitor the system or make upgrades.

 

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.142.43.239