Unlike other IT systems, which tend to have very daily procedural or coded steps, business intelligence systems have an inherent complexity due to the query engines that form a large part of the system. OBIEE is no different, and with an upgrade like this, it is vital that we should test the system before releasing it live to our business users. In addition to the extra validation rules in the consistency check, there are changes to the query engine that can subtly alter queries that have been modeled identically in both 11g and 12c. Also, due to the upgrade and changed functionality in the Web Catalog, upgraded reports may show changed behavior. This can range from missing labels to reporting aggregates, producing results different from those previously expected.
The basic stages of a full regression test are described in the following sections.
Once the upgrade has been completed as mentioned, a global consistency check should be run on the RPD and, if any errors occur, they should be noted and solved. Individual reports should be sanity checked by the technical team to ensure that results are being presented and that there are no other errors, such as, database connectivity or server errors, discrepancies in results, and visualization errors.
The behavior of customizations, such as the performance of any custom JavaScript, should also be checked. This will enable you to decide if any report actually needs to be completely rebuilt.
If you have updated the scheduler, then agents should be run to check for errors.
A test team should be used to see whether the datasets being generated by reports are the same as before the upgrade.
As mentioned, the security model has changed in 12c. Tests should also check security rules and visibility. This will involve logging into dashboards/reports with different users and checking if the appropriate object and/or data is being displayed for that user and user group. This is even more important if you have decided to make radical changes to your security model in order to better utilize the changes to security functionality and capability that arrive with 12c.
The final stage of testing will be User Acceptance testing by the end business users, to ensure that they are happy that the system is fit for purpose and in line with the requirements for the previous implementation. This will also include the testing of agents and ad hoc answers' requests.
If the project is well run, the scripts for all of these tests should be available from the last iterative implementation in the project's life cycle. If not, we cannot stress enough how important it is to create a robust set of scripts that have been created in conjunction with the business and end users.
At each stage of testing, errors should be tracked and dealt with by a developer.
To aid a robust test cycle, you may wish to do further research into third-party test-automation tools such as HP LoadRunner or the open-source Selenium.