Intel Software Guard Extensions

Azure VMs in the DC-series support Intel’s Software Guard Extensions (SGX). SGX is a security technology that creates one or more trusted execution environments (TEEs), which Intel calls enclaves. Enclaves are isolated and encrypted protected memory whose isolation control, cryptographic keys, and security boundary are managed by the CPU. This is important, because you want security defenses like these as low as possible in the stack, and for the root of trust to be immutable such that these defenses cannot be compromised by malware or even a bad actor who works at a cloud service provider (CSP).

From the outset, SGX was designed for purpose-built applications that can be refactored or built from the ground up to run in this restricted environment. However, there are now library OS (LibOS) solutions that run the most sensitive parts of their workloads in an SGX enclave and require little or no code changes to do so.

Simply deploying a DC-series VM (series 2 and 3) offers no security benefit until you load some trusted code into the TEE. This requires that you either write some code or use some code that comes as part of a product. For example, under the hood, Azure SQL Database uses these VM types when the Always Encrypted with Secure Enclaves setting is applied. (This is covered in Chapter 13, “Database security.”) Azure SQL Database loads part of the query engine into the TEE, where the query engine decrypts and analyzes data. The only time data is in plaintext is in the TEE; outside the TEE—both on disk and when the query results are sent to a client for client-side decryption—only ciphertext is used.

There are two confidential computing models that use a TEE. The first is a per-process TEE, which is how Azure SQL Database works when using Always Encrypted with Secure Enclaves. The second is an app-level TEE, which is used by tools like Enarx and Occlum.

The Signal Messenger app is a great example of effective use of SGX using the first model. To quote Signal’s website (https://signal.org/):

“State-of-the-art end-to-end encryption (powered by the open source Signal Protocol) keeps your conversations secure. We can't read your messages or listen to your calls, and no one else can either. Privacy isn’t an optional mode—it’s just the way that Signal works. Every message, every call, every time.”

The Signal server runs on Azure and parts of its internal system run on VMs that support SGX. So, when Signal says, “We can’t read your messages or listen to your calls,” this is how they do it. It is a great example of plausible deniability. You can read a case study on Microsoft’s website about Signal’s use of SGX here: https://azsec.tech/sig.

TEE code is frequently written in C and C++. This is because the runtime dependencies tend to be smaller for C and C++, and you always want to run the smallest amount of code possible in secured environments. These small, highly secure environments are often referred to as trusted computing bases (TCBs). Use of the Rust programming language (https://www.rust-lang.org/) is growing in popularity, however, because of its correctness and safety guarantees.

If you want to write your own code that executes in a TEE, you can use an SDK. The recommended approach is to use the Open Enclave SDK because it is hardware agnostic. You can learn more at https://openenclave.io/sdk/.

AMD Secure Encrypted Virtualization-Secure Nested Paging

AMD Secure Encrypted Virtualization-Secure Nested Paging (SEV-SNP) feature bears no resemblance to Intel’s SGX. Although they both exist under the banner of Azure confidential computing, they solve hardware-isolation problems differently.

When you use AMD SEV-SNP, the entire VM image is protected. So, everything in the VM is encrypted and isolated down to the CPU level. This is different from SGX because you need not create custom code or modify existing code to take advantage of the hardware-enforced security boundary. This makes it easier to lift and shift workloads that run on-prem or other VMs and move them to a considerably more secure and isolated environment.

At the time of this writing, AMD SEV-SNP VMs are available in preview using the following VM types:

• DCasv5 A confidential VM with remote storage and no local temporary disk

• DCadsv5 A confidential VM with a local temporary disk

• ECasv5 A memory-optimized confidential VM with remote storage and no local temporary disk

• ECadsv5 A memory-optimized confidential VM with a local temporary disk

Be aware that if an Azure VM using AMD SEV-SNP requires support, Microsoft employees cannot access it, even if authorized by the customer. This renders certain recovery and support scenarios unavailable for confidential VMs. This is by design.

Arm TrustZone

Arm TrustZone takes a different approach to isolation. With the previously discussed architectures, the normal OS can address enclave memory, but encryption prevents it from accessing the enclave memory. In contrast, with TrustZone, the normal OS cannot even address the enclave memory. For example, consider a 32-bit processor. With this type of processor, addresses use a 33-bit address space (re-read that if you missed it: 33-bit, not 32-bit), where the extra bit is secure versus normal. So, a 32-bit register can express an address only within its own 32-bit space. Although Azure IoT Edge uses Arm TrustZones, you cannot currently create a VM image that supports Arm processors. You can learn more at https://azsec.tech/arm/.

DCsv3-series VMs, SGX, Intel Total Memory Encryption, and Intel Total Memory Encryption Multi-Key

SGX requires the use of DCv2 VMs. However, the newer DCsv3 VMs (https://azsec.tech/6qc) support Intel SGX and Intel Total Memory Encryption Multi-Key (TME-MK). On the surface, TME-MK is similar in principle to the VM encryption provided by AMD, but it is not. If you define confidential computing the way the CCC does, Intel TME/TME-MK is not confidential computing. To be more precise, TME/TME-MK is closer to another AMD technology, called Secure Memory Encryption (SME), than to SEV-SNP.

Figure 11-1 compares TME/TME-MK, SEV-SNP, and SGX. Note the trust boundaries and that the Intel TME/TME-MK attack surface is larger than that of SEV-SNP and SGX. Also notice that SEV-SNP trusts the guest VM, but on the Azure host, SGX does not trust the guest VM.

Attestation

Attestation is a process that verifies that software binaries are correct and properly instantiated on a trusted platform, such as SGX or an SEV-SNP VM. The Confidential Computing Consortium defines attestation as follows:

Attestation is the process by which one party, called a “Verifier,” assesses the trustworthiness of a potentially untrusted peer, i.e., the “Attester.” (These terms are consistent with the Internet Engineering Task Force’s “Remote Attestation Procedures Architecture.”) The goal of attestation is to allow the Verifier to gain confidence in the trustworthiness of the Attester by obtaining an authentic, accurate, and timely report about the software and data state of the Attester.

Attestation uses policies to validate that some code is valid and meets your security requirements. A sample policy follows:

Click here to view code image

version= 1.0;
authorizationrules
{
     [type=="x-ms-sgx-is-debuggable", value==false]
      && [type=="x-ms-sgx-product-id", value==4639]
      && [type=="x-ms-sgx-svn", value >= 0]
      && [type=="x-ms-sgx-mrsigner", value=="e31c9e505f3 <snip> c450b6e33e5"]
  => permit();
};

Here is what some of these claims mean:

• x-ms-sgx-is-debuggable This can be set to true or false. If it is set to false, the enclave cannot be debugged.

• x-ms-sgx-product-id This is an integer that identifies the code in the enclave. It allows for the segmentation of enclave code signed by the same entity. In the preceding example, 4639 is the ID for the Azure SQL Database Always Encrypted enclave.

• x-ms-sgx-svn This is the security version number of the code that runs in the enclave. This is important when fixing vulnerabilities in the code. If v1 has a vulnerability and you fix it, you can make the fixed version v2 and can set the attestation to load only v2. If someone tries to load the insecure v1, the operation will fail.

• x-ms-sgx-mrsigner This is a hex-encoded hash of the signer’s public key associated with the private key used to sign the binary that runs in the enclave.

Notice the => permit() line at the end. This essentially means, if all these claims are correct, then permit the code to execute in the enclave.

You can also deny the attestation based on specific claims. For example, the following code is part of a policy that is applied when loading Azure VMs configured for Trusted Boot (covered later):

Click here to view code image

[type=="secureboot", value==false] => deny();

This code indicates that if the VM is not configured for Secure Boot, the load operation should be denied.

When using a feature that uses attestation, you must reference the attestation endpoint using HTTPS. For example, when using Azure SQL Database with secure enclaves, you can set the URL in tools like SSMS and Azure Data Explorer, as shown in Figure 11-2.

At some point, if you are dealing with anything related to confidential computing, you will probably need to use an attestation service. The Microsoft Azure Attestation Service is a general-purpose service that will probably meet your needs.

Trusted launch for Azure VMs

Some years ago, Microsoft added Secure Boot and Trusted Boot technology to Windows in response to increasingly sophisticated malware targeting the boot sequence using kernel malware, bootkits, and rootkits. (You can read more about Secure Boot and Trusted Boot here: https://azsec.tech/9is.) Trusted launch is a similar technology, but for Generation 2 Azure VMs.

Trusted launch is a simple option you can set during deployment or in an ARM template:

Click here to view code image

            "securityProfile": {
                "uefiSettings": {
                    "secureBootEnabled": true,
                    "vTpmEnabled": true
                },
                "securityType": "TrustedLaunch"
            },

And it is no cost!

The goal of Azure trusted launch is to provide confidence in the entire boot chain’s integrity and to serve as a major defense against sophisticated attackers. You should use this feature for your VMs if possible. (Note, however, that at the time of this writing, this feature does not support Ultra disks.)

Certain layered technologies are used to enable Azure trusted launch. One of these is Virtual Trusted Platform Module (vTPM), a virtualized TPM, like those found in Windows 11 laptops; it is also TPM 2.0 compliant. Trusted launch provides your VM with its own dedicated TPM that is not directly accessible by code running in any VM and that can provide Microsoft Defender for Cloud telemetry and alerts about the security health of your VMs.

Table 11-1 compares mitigations provided by trusted launch VMs, hardware-confidential VMs with AMD SEV-SNP, and hardware enclaves with Intel SGX.

Here is another way to look at the protections provided by the three capabilities:

• Trusted launch VMs Only known, trusted code runs on the VM.

• AMD SEV-SNP VMs Microsoft cannot touch the code or data in a VM.

• Intel SGX VMs Users can trust only their application and the processor.

SQL Server Always Encrypted

Various aspects of Always Encrypted are covered elsewhere in the book. For example, Chapter 10 explains which cryptographic primitives are used, and Chapter 13 explains the database-specific aspects of using Always Encrypted, such as how to configure Always Encrypted for SQL Server, Azure SQL Database, and Cosmos DB. The SQL Database products can use enclaves (with a small caveat we discuss in a moment), but Cosmos DB does not.

There is an important difference between the current implementation of Always Encrypted using secure enclaves in Azure SQL Database and SQL Server running in a Windows VM or on-premises. Azure SQL Database loads the query engine into an SGX enclave. SQL Server uses a different approach: it loads the query engine into another kind of isolation technology called virtualization-based security (VBS). VBS is similar in principle to SGX, but the implementation is different. The isolated region of memory used by VBS is controlled entirely by the Windows hypervisor. Essentially, the hypervisor creates a logical separation between the “untrusted world” and the “secure world,” designated by virtual trust levels (VTLs)—VTL0 and VT1, respectively. You can learn more about VBS technical details at https://azsec.tech/vbs.

Another difference is the way attestation works in SQL Server. Specifically, rather than using Microsoft Azure Attestation, SQL Server uses Host Guardian Service (HGS). (For more information, see https://azsec.tech/8gh.)

Azure Confidential ledger

Chapter 13 discusses Azure SQL Database ledger. In its coverage of this topic, the chapter mentions that you can store hashes created by Azure SQL Database ledger in Azure Confidential ledger (ACL—not to be confused with access control lists!). These services might share the same word (ledger), but they are different products that solve different problems.

Azure Confidential ledger is a managed PaaS offering of the Confidential Consortium Framework (CCF). CCF is a Microsoft Research–developed open source framework for building secure, highly available, and high-performing applications that focus on consortium-based (that is, multiparty) solutions. CCF has a similar goal to blockchain, but Azure Confidential ledger users and developers are not expected to have blockchain knowledge. For more information about the CCF, see https://microsoft.github.io/CCF/.

The Azure Confidential ledger managed service hides the internal complexity of the CCF. It provides Azure customers and developers with a REST API endpoint that can be used to store and retrieve key-value pairs to and from an immutable, tamper-proof ledger, and to verify the integrity of that ledger.

Azure Confidential ledgers can be private (encryption and integrity) or public (integrity only) and require Azure Active Directory (AD) or certificates to authenticate for both control-plane and data-plane operations. There is no anonymous access.

The code behind Azure Confidential ledgers runs in SGX enclaves. When client code sends data to an Azure Confidential ledger over a protected TLS channel, the data enters an SGX enclave. There, cryptographic operations are performed over the ledger plaintext. The cryptographic keys never leave the enclave in an unprotected form.

Confidential ledgers are created as blocks in blob storage containers in an Azure Storage account. Transaction data can be stored either encrypted or in plaintext. But remember, the core rationale for Azure Confidential ledgers and Azure SQL Database ledgers is to mitigate tampering threats. It is a strong nonrepudiation defense.

You will need to know some settings for your ACL instance, such as the identity service URI (for example, https://identity.confidential-ledger.core.azure.com). You can get the core metadata by using the following Azure CLI command:

Click here to view code image

az confidentialledger show --name "<ledgername>" --resource-group "<rg>"

The name is just the name, not the complete URI. For example, the name for https://ledger-clinic-tst.confidential-ledger.azure.com is ledger-clinic-tst.

Confidential containers

Confidential containers, used in AKS and ACI, enable you to wrap a container inside an SGX enclave with little work and little SGX knowledge. You can learn about the process here: https://azsec.tech/gub.

When considering deploying any container-based workload, you must think about it from an application-development standpoint as well as a DevOps standpoint, and the same holds true for confidential containers.

Confidential container–related tasks for developers are as follows:

• Developing applications following security practices

• Running security and validation tests

• Containerizing the application

• Wrapping the container in an SGX runtime

• Rerunning all tests to make sure there are no regressions

• Storing the image in a container registry

For the DevOps teams, these are the tasks:

• Adding SGX-enabled nodes to the node pool

• Adding the confidential computing AKS add-on

• Deploying SGX-enabled containers

Microsoft also provides two references regarding confidential container architectures: a Confidential Healthcare Platform demo, at https://azsec.tech/00n, and an article on big data analytics on confidential computing with Apache Spark on Kubernetes, at https://azsec.tech/gmr. Finally, Chapter 12, “Container security,” includes additional container-related security considerations.

Summary

Azure confidential computing opens a new world of possibilities for highly secure workloads in Azure. If you are deploying highly sensitive workloads that handle confidential data, then you should consider using some aspects of confidential computing, such as isolated and encrypted VMs that use AMD CPUs to enforce strong security boundaries that deny access even to malicious admins and cloud operators. Alternatively, you could use specific features, such as Always Encrypted or Confidential ledger, to secure specific data-oriented solutions. Finally, you can orchestrate confidential containers using AKS.

The confidential computing technology and services currently available in Azure today are only the start. There is more to come. To stay up to date, read the Azure confidential computing team’s blog at https://aka.ms/ACCblog.