Operational validation refers to the process of validating and verifying that the environments are not only provisioned and configured in the desired state but also are operational and running as intended. Although unit tests are performed on environments such as development, testing, and preproduction, operational validation can be executed against the production environment. However, this does not mean that operational validation cannot be executed against development, test, or any other environment. It can be executed against any environment.
Another important point to remember about operational validation is that it should not modify the environment while executing the tests. If it needs to perform any action that would eventually modify the environment, the tests should proactively create additional resources and use them. After completion of the tests, these temporary resources should be teared down.
Operational validation is a must have tool that should be employed for effective continuous delivery and deployment. Operational validation provides immediate feedback about a release to decide if it is suitable for going live on production. It can also execute tests such as A/B tests that can provide additional feedback about the deployment.
Operational validation can be executed through two different ways, as follows:
- Pester
- Operational validation module
Operational validation tests are Pester test cases authored and executed exactly in the same as way unit tests were shown earlier. The difference is in the scope of the tests. Although unit tests focus on an individual component and its working, operational validation checks for end-to-end working and operational effectiveness of the environment and application. Microsoft provides the operational validation module in order to execute operational tests for an environment. The operational validation module internally executes Pester test cases.
The tests should be converted into a PowerShell module for the operational validation module to be able to search and execute these test cases. The advantage operational validation module brings in is that it mandates that the tests are available as PowerShell modules. They can be easily searched and discovered easily on a computer. There is no need to use a file path in order to execute the tests. Moreover, these modules are easily shareable with communities, and other developers within and outside the team. These test modules are searchable, discoverable, identifiable, and they can be deployed using package management as well. Chapter 10, Continuous Delivery and Deployment will show the way to execute operational validation tests using Pester.
There are specific steps that are undertaken to use the operational validation module for operation validation test cases:
- Create a folder structure expected by the operational validation module.
- Write operational validation test cases.
- Convert the tests into the PowerShell module.
The operational validation module expects test cases within specific folders and has a predefined folder structure. This helps the operational module to deterministically load and execute the operational validation tests.
An OnlineMedicine folder is created within the OperationalValidation folder. OnlineMedicine contains the OnlineMedicine.psd1 PowerShell data file and Diagnostics folder. The Diagnostics folder further contains two subfolders: Simple and Comprehensive. The Simple folder should contain simple test cases focusing on single resources, whereas the Comprehensive folder should contain integration and operational validation tests that involve more than one resource and are usually more time consuming than executing a simple test case.
To generate the psd1 file in the OnlineMedicine folder, the new-ModuleManifest cmdlet can be used as shown here:
New-ModuleManifest -Path ".. OnlinePharmacy.Configuration estsOperational ValidationOnlineMedicineOnlineMedicine.psd1"
Although the authoring and saving of script files happens within the overall Project folder, eventually, the OnlineMedicine module should be copied over to Program Files | WindowsPowershell | Modules directory of the machine on which the validation operations tests are executed.
As part of the operation validation, it is important to test whether the web application is reachable, whether it can send response to requests, and that the entire request-response mechanism is successful. The web application comprises multiple pages, and tests should be conducted in order to retrieve multiple pages successfully. The requests will only get a successful response if provisioning and configuration of every resource is in the desired state, and there are no deviations in them. The public IP address to the virtual machine is assigned at deployment time, and the web application port number is provided as a parameter by the administrator. The values for the public IP address and web application port number are available as part of the output from the template deployment. The code for operation validation tests is shown here. The operational validation tests for the first instance of the web application are defined in WebAppVirtualMachine-01.Tests.ps1:
<#
Purpose:
Verify the web application is operational on first virtual machine.
Action:
Run Invoke-WebRequest on multiple pages of web application.
Expected Result:
invoking the request for index page of web application and comparing the returned status, text and
description
invoking the request for Drug's create page of web application and comparing the returned status, text
and description
invoking the request for Drugs page of web application and comparing the returned status, text and
description
invoking the request for Drug Inventory page of web application and comparing the returned status,
text and description
invoking the request for sales page of web application and comparing the returned status, text and
description
#>
param(
[string] $deploymentName,
[string] $resourceGroupName
)
Describe "Web application requests to first virtual machine and container" {
BeforeAll {
$deployment = (Get-AzureRmResourceGroupDeployment -ResourceGroupName $resourceGroupName -Name
$deploymentName)
$vm01IP = $deployment.Outputs.vM01PublicIPAddress.value
$webappPort= $deployment.Outputs.webappPort.value
}
It "invoking the request for index page of web application and comparing the returned status, text and description.." {
$parturl = $($($vm01IP) + ":" + $($webappPort))
$indexPage = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/index"
$indexPage.Content.Contains("Welcome to Medical Point of sale application") | should be $true
$indexPage.StatusCode | should be 200
$indexPage.StatusDescription | should be "OK"
}
It "invoking the request for Drug's create page of web application and comparing the returned status, text and description.." {
$parturl = $($($vm01IP) + ":" + $($webappPort))
$createDrug = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/Drugs/Create" -Method
Get
$createDrug.Content.Contains("Create new Drug Master - DevOps with Windows Server 2016 sample
application") | should be $true
$createDrug.StatusCode | should be 200
$createDrug.StatusDescription | should be "OK"
}
It "invoking the request for Drugs page of web application and comparing the returned status, text and description.." {
$parturl = $($($vm01IP) + ":" + $($webappPort))
$drugs = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/Drugs"
$drugs.Content.Contains("List of Drugs - DevOps with Windows Server 2016 sample application") | should
be $true
$drugs.StatusCode | should be 200
$drugs.StatusDescription | should be "OK"
}
It "invoking the request for Drug Inventory page of web application and comparing the returned status, text and description.." {
$parturl = $($($vm01IP) + ":" + $($webappPort))
$drugInventory = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/DrugInventories"
$drugInventory.Content.Contains("List of inventory - DevOps with Windows Server 2016 sample
application") | should be $true
$drugInventory.StatusCode | should be 200
$drugInventory.StatusDescription | should be "OK"
}
It "invoking the request for sales page of web application and comparing the returned status, text and description.." {
$parturl = $($($vm01IP) + ":" + $($webappPort))
$sales = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/Sales"
$sales.Content.Contains("All sales.. - DevOps with Windows Server 2016 sample application") | should
be $true
$sales.StatusCode | should be 200
$sales.StatusDescription | should be "OK"
}
}The operational validation tests for the second virtual machine are similar to the first one. The code for operation validation tests is shown here. The operational validation tests for the second instance of the web application are defined in WebAppVirtualMachine-02.Tests.ps1:
<#
Purpose:
Verify the web application is operational on second virtual machine.
Action:
Run Invoke-WebRequest on multiple pages of web application.
Expected Result:
invoking the request for index page of web application and comparing the returned status, text and
description
invoking the request for Drug's create page of web application and comparing the returned status, text
and description
invoking the request for Drugs page of web application and comparing the returned status, text and
description
invoking the request for Drug Inventory page of web application and comparing the returned status,
text and description
invoking the request for sales page of web application and comparing the returned status, text and
description
#>
param(
[string] $deploymentName,
[string] $resourceGroupName
)
Describe "Web application requests to first virtual machine and container" {
BeforeAll {
$deployment = (Get-AzureRmResourceGroupDeployment -ResourceGroupName $resourceGroupName -Name
$deploymentName)
$vm02IP = $deployment.Outputs.vM02PublicIPAddress.value
$webappPort= $deployment.Outputs.webappPort.value
}
It "invoking the request for index page of web application and comparing the returned status, text and
description.." {
$parturl = $($($vm02IP) + ":" + $($webappPort))
$indexPage = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/index"
$indexPage.Content.Contains("Welcome to Medical Point of sale application") | should be $true
$indexPage.StatusCode | should be 200
$indexPage.StatusDescription | should be "OK"
}
It "invoking the request for Drug's create page of web application and comparing the returned status, text
and description.." {
$parturl = $($($vm02IP) + ":" + $($webappPort))
$createDrug = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/Drugs/Create" -Method
Get
$createDrug.Content.Contains("Create new Drug Master - DevOps with Windows Server 2016 sample
application") | should be $true
$createDrug.StatusCode | should be 200
$createDrug.StatusDescription | should be "OK"
}
It "invoking the request for Drugs page of web application and comparing the returned status, text and
description.." {
$parturl = $($($vm02IP) + ":" + $($webappPort))
$drugs = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/Drugs"
$drugs.Content.Contains("List of Drugs - DevOps with Windows Server 2016 sample application") | should
be $true
$drugs.StatusCode | should be 200
$drugs.StatusDescription | should be "OK"
}
It "invoking the request for Drug Inventory page of web application and comparing the returned status, text and description.." {
$parturl = $($($vm02IP) + ":" + $($webappPort))
$drugInventory = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/DrugInventories"
$drugInventory.Content.Contains("List of inventory - DevOps with Windows Server 2016 sample
application") | should be $true
$drugInventory.StatusCode | should be 200
$drugInventory.StatusDescription | should be "OK"
}
It "invoking the request for sales page of web application and comparing the returned status, text and description..." {
$parturl = $($($vm02IP) + ":" + $($webappPort))
$sales = Invoke-WebRequest -UseBasicParsing -Uri "http://$parturl/newapp/Sales"
$sales.Content.Contains("All sales.. - DevOps with Windows Server 2016 sample application") | should
be $true
$sales.StatusCode | should be 200
$sales.StatusDescription | should be "OK"
}
}The operational validation tests using a load balancer is the same as that of virtual machines; however, instead of using the public IP address of the virtual machines, the public IP address of load balancer is used. The code for operational tests using load balancer is shown next. The operational validation tests for the web application load balancer are defined in WebAppLoadBalancer.Tests.ps1:
<#
Purpose:
Verify the web application is operational using Azure Load balancer.
Action:
Run Invoke-WebRequest on multiple pages of web application.
Expected Result:
invoking the request for index page of web application and comparing the returned status, text and
description
invoking the request for Drug's create page of web application and comparing the returned status, text
and description
invoking the request for Drugs page of web application and comparing the returned status, text and
description
invoking the request for Drug Inventory page of web application and comparing the returned status,
text and description
invoking the request for sales page of web application and comparing the returned status, text and
description
#>
param(
[string] $deploymentName,
[string] $resourceGroupName
)
Describe "Web application requests to virtual machine and container using load balancer" {
BeforeAll {
$deployment = (Get-AzureRmResourceGroupDeployment -ResourceGroupName $resourceGroupName -Name
$deploymentName)
$lbip = $deployment.Outputs.loadBalancerPublicIPAddress.value
$webappPort= $deployment.Outputs.webappPort.value
}
it "invoking the request for index page of web application and comparing the returned status, text and
description.." {
$parturl = $($($lbip) + ":" + $($webappPort))
$fullurl = "http://$parturl/newapp/index"
$indexPage = Invoke-WebRequest -UseBasicParsing -Uri (New-Object System.Uri -ArgumentList "$fullurl")
$indexPage.Content.Contains("Welcome to Medical Point of sale application") | should be $true
$indexPage.StatusCode | should be 200
$indexPage.StatusDescription | should be "OK"
}
it "invoking the request for index page of web application and comparing the returned status, text and description.." {
$parturl = $($($lbip) + ":" + $($webappPort))
$fullurl = "http://$parturl/newapp/index"
$indexPage = Invoke-WebRequest -UseBasicParsing -Uri "$fullurl"
$indexPage.Content.Contains("Welcome to Medical Point of sale application") | should be $true
$indexPage.StatusCode | should be 200
$indexPage.StatusDescription | should be "OK"
}
it "invoking the request for Drug's create page of web application and comparing the returned status, text and description.." {
$parturl = $($($lbip) + ":" + $($webappPort))
$fullurl = "http://$parturl/newapp/Drugs/Create"
$createDrug = Invoke-WebRequest -UseBasicParsing -Uri "$fullurl" -Method Get
$createDrug.Content.Contains("Create new Drug Master - DevOps with Windows Server 2016 sample
application") | should be $true
$createDrug.StatusCode | should be 200
$createDrug.StatusDescription | should be "OK"
}
it "invoking the request for Drugs page of web application and comparing the returned status, text and description.." {
$parturl = $($($lbip) + ":" + $($webappPort))
$fullurl = "http://$parturl/newapp/Drugs"
$drugs = Invoke-WebRequest -UseBasicParsing -Uri "$fullurl"
$drugs.Content.Contains("List of Drugs - DevOps with Windows Server 2016 sample application") | should
be $true
$drugs.StatusCode | should be 200
$drugs.StatusDescription | should be "OK"
}
it "invoking the request for Drug Inventory page of web application and comparing the returned status, text and description.." {
$parturl = $($($lbip) + ":" + $($webappPort))
$fullurl = "http://$parturl/newapp/DrugInventories"
$drugInventory = Invoke-WebRequest -UseBasicParsing -Uri "$fullurl"
$drugInventory.Content.Contains("List of inventory - DevOps with Windows Server 2016 sample
application") | should be $true
$drugInventory.StatusCode | should be 200
$drugInventory.StatusDescription | should be "OK"
}
it "invoking the request for sales page of web application and comparing the returned status, text and description.." {
$parturl = $($($lbip) + ":" + $($webappPort))
$fullurl = "http://$parturl/newapp/Sales"
$sales = Invoke-WebRequest -UseBasicParsing -Uri "$fullurl"
$sales.Content.Contains("All sales.. - DevOps with Windows Server 2016 sample application") | should
be $true
$sales.StatusCode | should be 200
$sales.StatusDescription | should be "OK"
}
}Pester provides the Invoke-Pester PowerShell function in order to execute Pester test cases. It accepts a single script file as well as a path to the folder containing multiple test script files through the Script parameter:
Invoke-Pester -Script "$env:ProgramFilesWindowsPowershellModulesUnit Tests"
The preceding command executes all the test scripts while the following command executes test cases only in a single script file:
Invoke-Pester -Script "$env:ProgramFilesWindowsPowershellModulesUnit Tests Availabilityset.Tests.ps1"
Operational validation tests can be executed similar to the way unit tests are executed using Invoke-Pester. The operational validation module also provides a function Invoke-OperationValidation to execute operational validation test cases. Invoke-OperationValidation internally invokes the Invoke-Pester function in order to execute the test cases. If no parameters are specified for this function, it searches all the modules on the machine that adheres to the folder structure prescribed by the operational validation module and executes the tests within them. It has an option to choose if only simple or comprehensive tests are to be executed. By default, it executes both types of test cases. Even the ModuleName parameter can be specified containing the test cases. When the ModuleName parameter is provided, this function will search only for this module to execute the test cases.
The execution report of the operational validation module is different in format as compared with the Pester report although it internally uses the Pester module. However the Pester output can be included with the operational validation report.
The operational validation tests are executed directly using Invoke-Pester instead of Invoke-OperationValidation in Chapter 10, Continuous Delivery and Deployment.