Disaster recovery refers to having business continuity in case of a natural disaster or hacking that terminates an entire Azure region.
There are two ways to implement disaster recovery: Standard Geo-replication and Active Geo-replication.
Standard Geo-replication asynchronously replicates committed transactions from an online primary to an offline secondary in a predefined Azure region:
In standard geo-replication:
The standard geo-replication is designed for environments where the only purpose of having geo-replication is to implement disaster recovery and not to offload reads to the readable secondaries.
If an Azure region fails, you will get an alert on the Azure Portal about the region failure. The state of the Azure SQL Server on that region is set to degraded. You have two options from here:
Let's say that the East US region fails, and after the failover in the environment, it will be like what is shown in the preceding. The West US will act as the primary, with the East US acting as the new secondary once the region comes back online. However, you can use a different region other than East US to create a secondary database instead of waiting for the East US to come online.
Active Geo-replication uses Always On technology to asynchronously replicate data to a maximum of four readable secondaries in the same or any other Azure region.
Active Geo-replication is available across all performance tiers.
A typical Active Geo-replication environment is shown in the following diagram:
The database DB1 is primarily stored in the South Central US region, with two readable secondaries at US West and US East.
The manual failover is similar to what was explained under the Standard Geo-Replication section. However, when you fail over to the secondary database, the endpoint or the connection string is changed and you will have to make changes in the application so that you can connect to the new primary.
Once the failover is complete, all secondary databases automatically point to the new primary.
In addition to manual failover, Active Geo-replication also supports automatic failover using auto failover groups.
The
default replication in Active Geo-replication
is asynchronous. However, if the application's need is to have synchronous replication, then you can do so by calling
sp_wait_for_database_copy_sync
immediately after committing the transaction. This will block the calling thread until all of the committed transactions have been replicated to the secondary.
The procedure can add significant delay to the calling thread if the size of the transaction log is being replicated.
It's advised to use this procedure to prevent the loss of critical data and not all data.
Auto-failover groups allow you to automatically recover one or more groups of SQL Databases in case of a region failure.
All databases in an auto-failover group should belong to a single server, and they will fail over to a single server as well.
Failover Group
A group of databases between the primary server and the secondary server which are to be recovered as a unit if there is an outage in the primary region.
The primary server is the one that hosts the primary databases. The application can read and write on the primary database.
The secondary server is the one which has the secondary databases. An application can only read from the secondary databases. The data is asynchronously replicated from the primary databases to the secondary databases.
Primary and secondary servers can't be on the same region.
Adding Databases to the Failover Group
When a database within a server or an elastic pool is added to the failover group, a secondary database with a performance level similar to that of the primary database is automatically created on the secondary server. If the primary database is in an Elastic Pool, then an elastic pool with the same name is automatically created on the secondary server.
When adding a database which already exists in the secondary database server however, it's not part of the failover group, and so a new secondary database is created in the secondary server.
Read-Write Listener
This is a DNS CNAME record that points to the primary server URL. It allows the application to transparently connect to the available primary server in case of a failover. This is similar to AG listener in an on-premises Always On configuration.
The application didn't connect to the primary or the secondary server URL. Instead, it connected to the Read-Write listener. In case of a failover, the read-write listener automatically points to the new primary (secondary) server. Therefore, unlike manual failover, the user doesn't have to change the application connection string in case of a failover.
Read-Only Listener
This is a DNS CNAME record that points to the secondary server. It allows the application to transparently connect to the secondary for read-only queries. If you want to offload the read workload to the secondary server automatically, you can do so by adding ApplicationIntent=ReadOnly in the application connection string. However, the read workload should be tolerant to a certain staleness of data. This is because of the fact that the replication is asynchronous and the secondary database will be some data behind the primary database.
Failover Policy
The default failover policy is set to automatic, however, this can be turned off if the failover process is controlled by the application.
The manual failover is required if automatic failover is turned off and the failover process isn't controlled by the application.
The manual failover can also be initiated any time if required, independent of the automatic failover policy. An example of manual failover is switching back to the primary region once the region recovers from the outage and is available to host resources.
Grace Period with Data Loss Hours
This setting controls the duration the system fails for before initiating the automatic failover. For example, if the grace period with data loss hours is set to two hours, then in case of an outage in the primary region, the failover will take place after two hours. However, if the outage is resolved before the grace period expires, the failover isn't performed.
Friendly Failover
If there is an outage in the primary region which has not yet impacted the databases (but may impact the databases in the near future) and the databases are still online, a friendly failover with full synchronization is immediately initiated, bypassing the grace period with the data loss hours value. There is no data loss in a friendly failover. The grace period setting is only effective when friendly failover isn't possible.
Upgrading the Primary Database Service Tier
The service tier and performance level of the primary database can be modified as and when required. The performance level within the same service tier can be modified without disconnecting the secondary database, in other words, you can upgrade the primary database from Standard S0 to Standard S1 without disconnecting the corresponding secondary database connection.
However, if you are switching between service tiers, then it's recommended (and enforced) to first upgrade the secondary database and then the primary database so as to avoid the termination of the secondary database connection.
If the secondary database is part of an auto-failover group, then it's advised not to downgrade the secondary database service tier. This is to avoid performance degradation in case of a failover.
Let us consider a scenario, wherein Mike needs to ensure that the data of Toystore Ltd. is shielded from disaster or failure of an entire region. For this, Mike can configure Active Geo-replication using the Azure portal to recover data and maintain business continuity. He can also take precautions by performing a manual failover from the primary server to the secondary server. This activity has the following aims:
The following section explains how to configure Active Geo-replication for a standalone Azure SQL Database:
toyfactory
database overview blade.The blue hexagon with a tick mark inside it is the current region that has the primary database. The highlighted (in red) purple-outlined hexagon is the recommended Azure region for the secondary database. The green-outlined hexagons represent other Azure regions across the globe.
When connecting to the secondary server, you'll have to add the firewall rule. It is therefore advised to use the database-level firewall at the primary server. This makes sure that the firewall rules are also copied to the secondary database during the Active Geo-replication setup so that you can log in easily.
It's also advised to use contained users so that you don't have to move server logins to the secondary server.
toystore
database. Expand the toystore
database. You should see all the objects in the toystore
database:SELECT COUNT(*) FROM Sales.Customers GO INSERT INTO Warehouse.Colors VALUES(100,'Light Green',1,Getdate(),Getdate()+10);
toystore
database in the primary server. Do not close the secondary server query window:INSERT INTO Warehouse.Colors VALUES(100,'Magenta',1,Getdate(),Getdate()+10);
SELECT @@ServerName As SecondaryServerName,* FROM Warehouse.Colors WHERE ColorName='Magenta'
Performing Manual Failover
The failover request will be submitted and the failover will be initiated.
Observe that the region color is also reversed. The blue hexagon, which denotes that the primary region is now the Central US region, and the green hexagon, which denotes that the secondary region is now East US 2.
Additionally, you can perform steps 5–8 to verify the failover.
This section covers configuration of auto-failover groups for a standalone Azure SQL Database. Consider the following scenario from ToyStore Ltd. Mike wants to ensure that whenever there is a disaster or failure of an entire region, there is no effect to the business of Toystore Ltd., so he configures auto-failover groups that allow him to automatically recover one or more groups of SQL Databases. To configure the auto-failover group for a standalone Azure SQL Database, the below steps need to be followed.
Adding databases to an existing failover group hasn't been supported in Azure Portal until now.
toystore_Shard_1_50
database overview blade.toystore_Shard_1_50 Geo-Replication
blade, click on the information icon above the world map:In the
Summary section, observe that we only have one database in the failover group, which is
toystore_Shard_1_50
.
Click Create to provision the secondary server and the failover group.
Each failover group has a monthly cost associated with it. This cost is separate from the cost of the secondary database.
Once created, you can follow the steps from the previous section to fail over and verify the replication.
In this activity, we will configure Active Geo-Replication for the
toystore
database using PowerShell.
PowerShell_ISE.exe
in the run command window and hit Enter. This will open a new PowerShell ISE editor window. This is where you'll write the PowerShell commands:C:CodeLesson08
directory. In the file name textbox, type Manage-ActiveGeoReplication
and press Save to save the file:Manage-ElasticPool.ps1
file, one after the other. The code explanation, wherever required, is given below the code snippet and in the comments within the code snippet.param
( [parameter(Mandatory=$true)] [String] $ResourceGroup, [parameter(Mandatory=$true)] [String] $PrimarySqlServer, [parameter(Mandatory=$true)] [String] $UserName, [parameter(Mandatory=$true)] [String] $Password, [parameter(Mandatory=$true)] [String] $SecondarySqlServer, [parameter(Mandatory=$true)] [String] $SecondaryServerLocation, [parameter(Mandatory=$false)] [bool] $Failover = $false, [parameter(Mandatory=$false)] [String] $DatabasesToReplicate, [parameter(Mandatory=$false)] # Add/Remove database to/from secondary server [String] $Operation = "none", [parameter(Mandatory=$false)] [String] $AzureProfileFilePath )
The parameter description is as follows:
# log the execution of the script Start-Transcript -Path ".LogManage-ElasticPool.txt" -Append # Set AzureProfileFilePath relative to the script directory if it's not provided as parameter if([string]::IsNullOrEmpty($AzureProfileFilePath)) { $AzureProfileFilePath="....MyAzureProfile.json" } #Login to Azure Account … # Provide your Azure Credentials in the login dialog box $profile = Login-AzureRmAccount $SubscriptionID = $profile.Context.Subscription.SubscriptionId } #Set the Azure Context Set-AzureRmContext -SubscriptionId $SubscriptionID | Out-Null
Manage-ElasticPool.txt
file created under the Log
directory within the parent directory of the Manage-ElasticPool.ps1
script.AzureProfileFilePath
. If found, it then sets the PowerShell context to the subscription ID, as specified in the profile file; otherwise, it asks to manually log in to the Azure account to set the context.if($Operation -eq "Add") { # Check if Azure SQL Server Exists # An error is returned and stored in notexists variable if resource group exists Get-AzureRmSqlServer -ServerName $SecondarySqlServer -ResourceGroupName $ResourceGroup -ErrorVariable notexists -ErrorAction SilentlyContinue # provision the secondary server if it doesn't exists if($notexists) { Write-Host "Provisioning Azure SQL Server $SecondarySqlServer" -ForegroundColor Green $credentials = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $UserName, $(ConvertTo-SecureString -String $Password -AsPlainText -Force) $_SecondarySqlServer = @{ ResourceGroupName = $ResourceGroup; ServerName = $SecondarySqlServer; Location = $SecondaryServerLocation; SqlAdministratorCredentials = $credentials; ServerVersion = '12.0'; … { Write-Host $notexits -ForegroundColor Yellow }
The preceding code
will provision a new secondary Azure SQL Server if the
$Operation
parameter is set to
Add. The SQL Server creation code is similar to what was used in
Chapter 1,
Microsoft Azure SQL Database Primer.
# Configure Active Geo-Replication for individual databases if(![string]::IsNullOrEmpty($DatabasesToReplicate.Replace(',','')) -and $Operation -eq "Add") { $dbname = $DatabasesToReplicate.Split(','); foreach($db in $dbname) { Write-Host "Replicating database $db to $SecondarySqlServer " -ForegroundColor Green #Get the database object for the given database name $database = Get-AzureRmSqlDatabase -DatabaseName $db -ResourceGroupName $ResourceGroup -ServerName $PrimarySqlServer #pipe the database object to New-AzureRmSqlDatabaseSecondary cmdlet $database | New-AzureRmSqlDatabaseSecondary -PartnerResourceGroupName $ResourceGroup -PartnerServerName $SecondarySqlServer -AllowConnections "All" } }
$DatabaseToReplicate
parameter isn't empty. If it's not and the $operation
parameter is set to Add
, it splits the comma -delimited list of the databases and configures Active Geo-replication for each one of them using the New-AzureRmSqlDatabaseSecondary
cmdlet.New-AzureRmSqlDatabaseSecondary
takes three parameters:PartnerResourceGroupName
: The resource group name that contains the secondary SQL Server. The primary and secondary resource groups are assumed to be the same in this script.PartnerServerName
: The name of the secondary Azure SQL Server.Allowconnections
: This specifies the read intent of the secondary database. It's set to All.if($Operation -eq "Remove") { $dbname = $DatabasesToReplicate.Split(','); foreach($db in $dbname) { Write-Host "Removing replication for database $db " -ForegroundColor Green $database = Get-AzureRmSqlDatabase -DatabaseName $db -ResourceGroupName $ResourceGroup -ServerName $PrimarySqlServer $database | Remove-AzureRmSqlDatabaseSecondary -PartnerResourceGroupName $ResourceGroup -ServerName $PrimarySqlServer -PartnerServerName $SecondarySqlServer } }
$Operation
is set to Remove. It first splits the comma-separated database list in the $DatabaseToReplicate
parameter. It then removes the replication link for each database using the Remove-AzureRmSqlDatabaseSecondary
cmdlet.Remove-AzureRmSqlDatabaseSecondary
accepts three parameters:PartnerResourceGroupName
: The resource group of the secondary SQL Server. The script assumes that it's the same as the primary SQL Server.ServerName
: The primary SQL Server name.PartnerServerName
: The secondary SQL Server name.# failover individual databases from primary to secondary if($Failover -eq $true) { $dbname = $DatabasesToReplicate.Split(','); foreach($db in $dbname) { Write-Host "Failover $db to $SecondarySqlServer..." -ForegroundColor Green $database = Get-AzureRmSqlDatabase -DatabaseName $db -ResourceGroupName $ResourceGroup -ServerName $SecondarySqlServer $database | Set-AzureRmSqlDatabaseSecondary -PartnerResourceGroupName $ResourceGroup -Failover } }
$Failover
parameter is set to true. It first splits the comma delimited list of the databases in $DatabaseToReplicate
and then performs a manual failover from the primary server to the secondary server using Set-AzureRmSqlDatabaseSecondary
.Set-AzureRmSqlDatabaseSecondary
accepts two parameters:PartnerResourceGroupName
: The resource group of the secondary SQL Server. The script assumes that it's the same as the primary SQL Server.Failover
: Initiates the failover.Set-AzureRmSqlDatabaseSecondary
cmdlet.Manage-ActiveGeoReplication.ps1
script inside of it. For example, if the script is at the C:CodeLesson08
directory, then run the following command to switch to this directory:cd C:CodeLesson08ActiveGeoReplication
toystore_Shard_1_50
and toystore_Shard_50_100
databases:.Manage-ActiveGeoReplication.ps1 -ResourceGroup toystore -PrimarySqlServer toyfactory -UserName sqladmin -Password Packt@pub2 -SecondarySqlServer toyfactory-centralus -SecondaryServerLocation "Central US" -DatabasesToReplicate "toystore_Shard_1_50,toystore_Shard_50_100" -Operation "Add" -AzureProfileFilePath C:CodeMyAzureProfile.json
Manage-ActiveGeoReplication.ps1
to start Active Geo-replication for the toystore_Shard_1_50
and toystore_Shard_50_100
databases on toyfactory
databases to the toyfactory-centralus
secondary Azure SQL Server..Manage-ActiveGeoReplication.ps1 -ResourceGroup toystore -PrimarySqlServer toyfactory -UserName sqladmin -Password Packt@pub2 -SecondarySqlServer toyfactory-centralus -SecondaryServerLocation "Central US" -DatabasesToReplicate "toystore_Shard_1_50,toystore_Shard_50_100" -failover $true -AzureProfileFilePath C:CodeMyAzureProfile.json
.Manage-ActiveGeoReplication.ps1 -ResourceGroup toystore -PrimarySqlServer toyfactory -UserName sqladmin -Password Packt@pub2 -SecondarySqlServer toyfactory-centralus -SecondaryServerLocation "Central US" -DatabasesToReplicate "toystore_Shard_1_50,toystore_Shard_50_100" -Operation "Remove" -AzureProfileFilePath C:CodeMyAzureProfile.json
This completes the activity.
In this activity, we will configure auto failover groups using PowerShell for ToyStore Ltd.
PowerShell_ISE.exe
in the run command window and hit enter. This will open a new PowerShell ISE editor window. This is where you'll write the PowerShell commands:Manage-ElasticPool.ps1
file, one after the other. The code explanation, wherever required, is given below the code snippet and in the comments within the code snippet.param ( [parameter(Mandatory=$true)] [String] $ResourceGroup, [parameter(Mandatory=$true)] [String] $PrimarySqlServer, [parameter(Mandatory=$false)] [String] $UserName, [parameter(Mandatory=$false)] [String] $Password, [parameter(Mandatory=$true)] [String] $SecondarySqlServer, [parameter(Mandatory=$false)] [String] $SecondaryServerLocation, [parameter(Mandatory=$false)] [bool] $Failover = $false, [parameter(Mandatory=$false)] [String] $DatabasesToReplicate, [parameter(Mandatory=$true)] [String] $FailoverGroupName, [parameter(Mandatory=$false)] [String] $Operation = "none", [parameter(Mandatory=$false)] [String] $AzureProfileFilePath )
# log the execution of the script Start-Transcript -Path ".LogManage-ElasticPool.txt" -Append # Set AzureProfileFilePath relative to the script directory if it's not provided as parameter if([string]::IsNullOrEmpty($AzureProfileFilePath)) { $AzureProfileFilePath="....MyAzureProfile.json" } #Login to Azure Account #Login to Azure Account if((Test-Path -Path $AzureProfileFilePath)) { $profile = Select-AzureRmProfile -Path $AzureProfileFilePath $SubscriptionID = $profile.Context.Subscription.SubscriptionId … } #Set the Azure Context Set-AzureRmContext -SubscriptionId $SubscriptionID | Out-Null
Manage-ElasticPool.txt
file, created under the Log directory within the parent directory of the Manage-ElasticPool.ps1
script.AzureProfileFilePath
variable. If found, then it sets the PowerShell context to the subscription ID, as specified in the profile file; otherwise, it asks to manually log in to the Azure account to set the context.IF($Operation -eq "Create") { # An error is returned and stored in notexists variable if resource group exists Get-AzureRmSqlServer -ServerName $SecondarySqlServer -ResourceGroupName $ResourceGroup -ErrorVariable notexists -ErrorAction SilentlyContinue # provision the secondary server if it doesn't exists if($notexists) { Write-Host "Provisioning Azure SQL Server $SecondarySqlServer" -ForegroundColor Green … } New-AzureRmSqlServer @_SecondarySqlServer; } else { Write-Host $notexits -ForegroundColor Yellow }
The preceding code is the same as what explained in
Chapter 1, Microsoft Azure SQL Database Primer to provision a new SQL Server. The new server is provisioned only when
$Operation
is set to Create.
# Create the failover group Write-Host "Creating the failover group $FailoverGroupName " -ForegroundColor Green $failovergroup = New-AzureRMSqlDatabaseFailoverGroup ' –ResourceGroupName $ResourceGroup ' -ServerName $PrimarySqlServer ' -PartnerServerName $SecondarySqlServer ' –FailoverGroupName $FailoverGroupName ' –FailoverPolicy Automatic ' -GracePeriodWithDataLossHours 1 }
$Operation
parameter is set to Create. The New-AzureRMSqlDatabaseFailoverGroup cmdlet accepts the following parameters:ResourceGroupName
: The name of the resource group that contains the primary SQL ServerServerName
: The primary SQL Server namePartnerServerName
: The secondary SQL Server nameFailoverGroupName
: The name of the failover group to be createdFailoverPolicy
: The failover policy, Automatic or ManualGracePeriodWithDataLossHours
: The value in hours for the duration the automatic failover should wait after a region outage# Add databases to the failover group if(![string]::IsNullOrEmpty($DatabasesToReplicate.Replace(',','')) -and $Failover -eq $false -and $Operation -eq "Create") { $dbname = $DatabasesToReplicate.Split(','); foreach($db in $dbname) { Write-Host "Adding database $db to failover group $FailoverGroupName " -ForegroundColor Green $database = Get-AzureRmSqlDatabase -DatabaseName $db -ResourceGroupName $ResourceGroup -ServerName $PrimarySqlServer Add-AzureRmSqlDatabaseToFailoverGroup -ResourceGroupName $ResourceGroup -ServerName $PrimarySqlServer -FailoverGroupName $FailoverGroupName -Database $database } }
$DatabasesToReplicate
parameter and adds them to the group.ResourceGroupName
: The name of the primary SQL Server resource group.ServerName
: The primary SQL Server name.FailoverGroupName
: The name of the failover group the databases are to be added to.Database
: The database object of the database to be added. This is set by calling the Get-AzureRMSqlDatabase
cmdlet.# failover to secondary if($Failover) { Write-Host "Failover to secondary server $SecondarySqlServer " -ForegroundColor Green Switch-AzureRMSqlDatabaseFailoverGroup -ResourceGroupName $ResourceGroup -ServerName $SecondarySqlServer -FailoverGroupName $FailoverGroupName }
if($Operation -eq "Remove") { Write-Host "Deleting the failover group $FailoverGroupName " -ForegroundColor Green Remove-AzureRmSqlDatabaseFailoverGroup -ResourceGroupName $ResourceGroup -ServerName $PrimarySqlServer -FailoverGroupName $FailoverGroupName # remove the replication link $dbname = $DatabasesToReplicate.Split(','); foreach($db in $dbname) { Write-Host "Removing replication for database $db " -ForegroundColor Green $database = Get-AzureRmSqlDatabase -DatabaseName $db -ResourceGroupName $ResourceGroup -ServerName $PrimarySqlServer $database | Remove-AzureRmSqlDatabaseSecondary -PartnerResourceGroupName $ResourceGroup -ServerName $PrimarySqlServer -PartnerServerName $SecondarySqlServer } }
Remove-AzureRmSqlDatabaseFailoverGroup
deletes the failover group. It accepts the following parameters:ResourceGroupName
: The failover group resource group nameServerName
: The primary SQL Server nameFailoverGroupName
: The name of the failover group which is to be deletedRemove-AzureRmSqlDatabaseSecondary
removes or stops the replication, as explained in the previous activity.Manage-ActiveGeoReplication.ps1
script in it. For example, if the script is at the C:CodeLesson08
directory, then run the following command to switch to this directory:cd C:CodeLesson08
.Manage-FailoverGroup.ps1 -ResourceGroup toystore -PrimarySqlServer toyfactory -UserName sqladmin -Password Packt@pub2 -SecondarySqlServer toyfactory-centralus -SecondaryServerLocation "Central US" -DatabasesToReplicate "toystore_Shard_100_150,toystore_Shard_150_200" -Operation "Create" -FailoverGroupName toyfactoryfailovergroup -AzureProfileFilePath c:CodeMyAzureProfile.json
toyfactoryfailovergroup
, and add the toystore_Shard_100_150
and toystore_150_200
databases to the newly created failover group.toyfactoryfailovergroup.database.windows.net
and not individual primary or secondary database endpoints..Manage-FailoverGroup.ps1 -ResourceGroup toystore -SecondarySqlServer toyfactory-centralus -FailoverGroupName toyfactoryfailovergroup -Failover $true -AzureProfileFilePath c:CodeMyAzureProfile.json
The preceding command will fail over all the databases in the failover group
toyfactoryfailovergroup
to the secondary server and make it the new primary server.
You can verify this from the Azure Portal, as explained in this chapter's section.
# delete failover group and stop the replication .Manage-FailoverGroup.ps1 -ResourceGroup toystore -PrimarySqlServer toyfactory-centralus -UserName sqladmin -Password Packt@pub2 -SecondarySqlServer toyfactory -SecondaryServerLocation "Central US" -DatabasesToReplicate "toystore_Shard_100_150,toystore_Shard_150_200" -Operation "Remove" -FailoverGroupName toyfactoryfailovergroup -AzureProfileFilePath c:CodeMyAzureProfile.json
toyfactoryfailovergroup
and break the replication link between the primary and secondary databases. However, the secondary server and the databases aren't deleted.PrimarySqlServer
parameter value is toyfactory-centralus
and that the SecondarySqlServer
parameter value is toyfactory
, which is the reverse of what we provided in step 1 when creating the failover group.3.14.251.128