Planning a performance test is the most important part of performance testing. Let's take a deeper look at it. The very first thing in planning a performance test is analysis:
- Analysis phase: This phase predominantly focuses and summarizes in detail the analysis and findings which will underpin the performance test of the systems to validate their ability to handle the forecasted load. During this phase, vendor representatives will be heavily engaged with business and technical representatives to understand and document the historic, current, and anticipated user profiles and projects. The data gathered will be used to model the load profiles and the test data model, at the end of this phase a report will be presented to the stakeholders. The following diagram represents the sequence of events carried out during this phase:
- Delivery phase: This phase states the methodology to successfully implement performance testing. The core emphasis behind this methodology and its application is to provide stakeholders with a comprehensive view of how the performance objectives will be achieved and the level of assurance that will be delivered, for instance through explanation of any constraints or decisions that could impact the framework and its implementation. This process eliminates overlap and ambiguity in performance testing and provides a detailed plan for implementation, it also aids in de-risking the implementation phase for the performance testing engagement, and the following diagram details the process and its associated deliverables:
- WorkLoad model: In order to successfully execute performance testing, a Workload model will be designed to model the logical set of activities that need to be performed by users (business or customers using the system) towards achieving a certain (business) goal. The main driver behind the workload model has been the Business Workload, where the performance testing team will consider scripting based on the logical set of activities that need to be performed in order to meet the business objectives and goals, different business initiatives that will impact performance of the tool, and the underlying infrastructure.
The following phases of tests will be executed as part of this performance testing effort:
Let's have a look at the various types of performance test types:
The test execution phase will follow a streamlined process; the main aim of the process is to review, assess and execute tests in a formal and structured way during the entire duration of the execution phase.
Test execution will only commence when all the entry criteria for a cycle have been met, and will only end when all the exit criteria have been fulfilled and reviewed and mitigating actions for key risks/issues agreed.
To ensure quality and effectiveness throughout the performance testing program and the integrity of testing it is vital that quality gates are established between phases and that these are formally controlled by the test manager and stakeholders. At the end of each phase of testing, the exit criteria of the phase just completed will be assessed. If these have been fulfilled, then the entry criteria of the succeeding test phase will themselves be assessed in turn before that test phase can be allowed to commence.
The following diagram gives an overview of the quality gates proposed for this testing program:
Test environments: The test environment is very important for knowing whether another user is accessing the application at that point in time. In cases where other users are accessing the application at the same time, the results of the test would not be real.
Based on the peak load number, we might need optimal configuration of the load generation machine or we need to use distributed mode for load generation.
Following are sample of the hardware configuration required (the numbers used are sample numbers).
- Hardware configuration of current environment:
- AMD Quad Core processor @ 2.6 GHz
- Ram: 'x' GB
- Performance stats collected
- Peak CPU utilization: 5.1%
- Peak memory footprint: 145 MB
- Resources required for 100 users
- RAM requirements for 100 users test (peak memory footprint: 145MB/5/CPU utilization)*100 = 2900MB ~ 2.84 GB
- Considering 80% CPU utilization as optimal for a load generator, we would need 1.28 times the CPU processing of the current machine
Considering the preceding calculations, we can generate a user load of 78 concurrent users using a generator with quad core CPU at 2.6 GHz and 4 GB RAM.
Following are the infrastructure requirements:
- The test environment will need to be established to provide for login and general system access
- Access to the testing systems and applications will need to be established and connections tested
- Servers need to be 100% dedicated to the test during the scheduled times of the test to prevent undue interference from other activities
- Each workstation will need to have login and application connections tested prior to performance tests being conducted
Test data: Test data is very important in performance testing. We need to gather the test data or generate it using scripts.
The test data should be unique and should be similar to the data that would be set up in production.
Following is the list of deliverables which are usually required by the client:
Dependency: Following is an example of a dependency list:
|
ID |
Dependency |
Description |
Impact |
|---|---|---|---|
|
1 |
Pre-Production Build Completed |
The pre-production environment should contain a stable and tested build, the emphasis behind this is to minimize any impact on re-work of performance test scripts. |
H |
|
2 |
Pre-Production Access Provided |
Access to relevant databases, application is provided to Vendor. Resources would be predominantly used to record the necessary test scripts identified for performance testing. |
H |
|
3 |
Performance Test Tools Deployed/Installed |
All the relevant tools are deployed, installed, and smoke tested. |
H |
|
4 |
Test Data requirements agreed |
A set of appropriate and valid test data is loaded. |
H |
|
5 |
Early Visibility of build to Pre-Prod |
Early indication of the build that is expected to be deployed to pre-production. |
M |
|
6 |
Cycle 1 Test Data Loaded |
Test data loaded and sanity tested prior to commencement of cycle 1 testing. |
H |
|
7 |
Drop X Deployed to Pre-Prod |
Deployed build successfully completed in pre-production environment, sanity tested and meets the Entry criteria for Cycle 1 test execution. |
M |
Now that we have taken a detailed look at the Performance testing planning, we should now see how we implement the script creation and implementation using SoapUI.