Chapter 15. PKI and Encryption Protocols

This chapter covers the following subjects:

Image Public Key Infrastructure: In this section, we discuss PKI and its components, including private and public keys, certificates, certificate authorities, and the web of trust model.

Image Security Protocols: Here, we define more security protocols such as S/MIME, SSL, TLS, SSH, and VPN-related protocols such as PPTP, L2TP, and IPsec. And three cheers if you want—these are the last of the TCP/IP security protocols in the book!

This short chapter wraps up the rest of the encryption concepts you need to know for the Security+ exam. You are required to understand public key infrastructures and should have the ability to explain what is entailed when a secure connection is made, for example, to a secure e-commerce web server. There is an entire system involved with public key infrastructures, from the users to servers, encryption methods, and much more. It’s a big topic that can be confusing due to how many and what variety of keys are used. Take it slow, and reread the section if necessary. Several protocols use public key infrastructures as well, many of which you have probably heard of, such as S/MIME, SSL, TLS, SSH, and so on. Keep in mind that the security protocols discussed in this section are intertwined with the concepts of a public key infrastructure.

Foundation Topics

Public Key Infrastructure

A public key infrastructure (PKI) is an entire system of hardware and software, policies and procedures, and people. It is used to create, distribute, manage, store, and revoke digital certificates. If you have connected to a secure website in the past, you have been a part of a PKI! But a PKI can be used for other things as well, such as secure e-mail transmissions and secure connections to remote computers and remote networks. The PKI is all encompassing: It includes users, client computers, servers, services, and most of all, encryption. Don’t confuse PKI with public key encryption. Though they are related, PKI is a way of accomplishing public key encryption, but not all public key encryption schemes are PKI. PKI creates asymmetric key pairs, a public key and a private key: The private key is kept secret, whereas the public key can be distributed. If the key pair is generated at a server, it is considered to be centralized, and the public key is distributed as needed. If the key pair is generated at a local computer, it is considered to be decentralized, and the keys are not distributed; instead, they are used by that local system. An example of public key usage would be a certificate obtained by a web browser during an encrypted session with an e-commerce website. An example of private key usage would be when a user needs to encrypt the digital signature of a private e-mail. The difference is the level of confidentiality. The public key certificate obtained by the web browser is public and might be obtained by thousands of individuals. The private key used to encrypt the e-mail is not to be shared with anyone.

In a nutshell, public key infrastructures are set up in such a way so as to bind public keys with user identities. This is usually done with certificates distributed by a certificate authority. Less commonly it is done by means of a web of trust.

Let’s go ahead and describe these concepts in a little more detail.


Certificates are digitally signed electronic documents that bind a public key with a user identity. The identity information might include a person’s name and organization, or other details relevant to the user to whom the certificate is to be issued. Most certificates are based on the X.509 standard, which is a common PKI standard developed by the ITU-T that often incorporates the single sign-on (SSO) authentication method. This way, a recipient of a single X.509 certificate has access to multiple resources, possibly in multiple locations. Although difficult, X.509 certificates that use MD5 and SHA1 hashes can be compromised. A more powerful hashing algorithm such as SHA2 should be implemented with the certificate. X.509 is the core of the PKIX, which is the IETF’s Public Key Infrastructure (X.509) working group. Components of an X.509 certificate include the following:

Image Owner (user) information, including their public key

Image Certificate authority information, including their name, digital signature, serial number, issue and expiration dates, and version

Certificates can be used for connections to websites, for e-mail, and for many other things in the Internet world, as well as encryption done locally. For example, a user working in a Windows environment might want to use the Encrypting File System (EFS) to encrypt data locally. The Windows domain can be configured to allow for user certificates governing and enhancing this encryption process. So, certificates can be used internally or externally, but most people are more familiar with the certificate used to make secure HTTP connections, usually with SSL/TLS-based certificates. We’ll focus mostly on that type of certificate as we move forward.

SSL Certificate Types

It’s a good idea to classify certificates the way the companies that sell them do. This includes domain, organizational, and extended validation certificates. Domain validation (DV) certificates are where the certificate authority checks the rights of the applicant to use a specific domain name. We’ll discuss the term certificate authority (CA) later in the chapter, but for now, simply put, it is the entity that issues the certificate. Organizational validation (OV) certificates go beyond this by also conducting some vetting of the organization involved, the result of which is displayed to customers. Extended validation (EV) certificates go further by conducting a thorough vetting of the organization. Issuance of these certificates is strictly defined.

Many companies have subdomains for their websites. For example, I own I might also opt to create subdomains such as and Generally, if you connect to a secure website that uses subdomains, a single certificate will allow for connections to the main website and the subdomains. This is known as a wildcard certificate; for example, *, meaning all subdomains of Your organization might allow this, or for additional security might use a different certificate for each subdomain (possibly from different providers), but this can prove to be expensive. For small businesses and organizations, the single certificate is usually enough. In fact, if the provider allows it, a small organization can use a multidomain certificate. By modifying the subject alternative name (SAN) field, an organization can specify additional hostnames, domain names, IP addresses, and so on.

Single-Sided and Dual-Sided Certificates

Most communication sessions, such as secure web sessions, use single-sided certificates. This is when the server validates itself to recipients of the certificate, such as users who are accessing the website. In these types of scenarios, users do not need to validate their own identity. This would be resource-intensive, especially for a secure web server that might have thousands of concurrent connections.

Sometimes, an organization might choose to have the server and the user validate their identities. This would be using a dual-sided certificate; it works well when a limited number of computers and sessions are involved. When more computers are added to the mix, the amount of resources necessary might be a strain on the issuing CA.

Certificate Chain of Trust

A certificate chain is a list of certificates that utilizes the chain of trust concept. This is where each component of the system is validated from the bottom up. In a certificate chain, also known as a certification path, the anchor for this trust is the root certificate authority. At the bottom of the chain we have the end-entity certificate for a machine/computer. That’s the certificate that you would see if you looked at the details of a secure HTTPS session in your web browser. That then handshakes with an intermediate certificate belonging to a subordinate certificate authority. This certificate signs the end-entity certificate. That then handshakes with the root certificate, which represents the root certificate authority. It signs the intermediate certificate and in and of itself is self-signing, which means that it not only creates the certificate, but signs it as well. It signs it with its own private key. The root CA will employ code signing: digitally signing and time stamping the certificate to provide integrity and authenticity. However, even this can be defeated, so the security admin has to always be on the lookout for CVEs detailing revoked certificates, and even entire issuing certificate companies that may have been compromised.

Certificate Formats

There are several certificate formats you should know for the exam. They can be identified in part by their file extension or encoding type used. First, let’s briefly discuss the ITU-T X.690 encoding formats:

Image Basic Encoding Rules (BER): This is the original ruleset governing the encoding of ASN.1 data structures. Any data created is encoded with a type identifier, a length description, and the content’s value. BER can use one of several encoding methods.

Image Canonical Encoding Rules (CER): This is a restricted version of BER in that it only allows the use of one encoding type; all others are restricted.

Image Distinguished Encoding Rules (DER): Another restricted variant of BER, this only allows for one type of encoding, and has restrictive rules for length, character strings, and how elements are sorted. It is widely used for X.509 certificates. For example, certificate enrollment in Windows Servers uses DER exclusively.

Now, let’s briefly define the certificate formats and extensions you might encounter. PEM is a very common format that uses base64-encoded ASCII files. It stands for Privacy-enhanced Electronic Mail and can be identified with the .pem file extension, though the format might also use .crt (for example, Microsoft), .cer, or .key extensions. It uses the DER encoding method. If the certificate uses a file extension different from .pem, you can tell if it is a PEM by opening the file with a text editor and looking for the “Begin Certificate” and “End Certificate” statements. However, if the certificate uses the .der extension, then the certificate file is in binary form instead of ASCII. Because it is in binary, you will not see the Begin and End Certificate statements that are displayed in a .pem.

P12/PFX is a binary format based on PKCS#12 used to store a server certificate, intermediate certificates, and the private key in one encryptable file. It is typically used to import and export certificates and private keys. You may see the .pfx and .p12 extensions associated with PKCS#12-based files. .pfx stands for Personal Information Exchange and is used by Microsoft for release signing. The certificate and its private and public keys are stored in the .pfx file. A .pfx file can also be developed by combining a private key with a PKCS #7 .p7b file, as might be done in Windows Internet Information Services (IIS). Or, .p7b format certificates can be used by themselves in IIS as the basis for S/MIME and single sign-on.


Some of these extensions will also be used for different types of data such as private keys, and not only for certificates. It is also possible to convert from one format to another using tools such as OpenSSL.

Certificate Authorities

A certificate authority (CA) is the entity (usually a server) that issues certificates to users. In a PKI system that uses a CA, the CA is known as a trusted third party. Most PKI systems use a CA. The CA is also responsible for verifying the identity of the recipient of the certificate. An example of a technology that uses certificates would be secure websites. If you opened your browser and connected to a secure site, the browser would first check the certificate that comes from VeriSign or another similar company; it would validate the certificate. You (the user) and the website are the two parties attempting to communicate. The CA is a third party that negotiates the security of the connection between you and the website. For a user to obtain a digital identity certificate from a CA, the user’s computer must initiate a certificate signing request (CSR) and present two items of information: The first is proof of the user’s identity; the second is a public key. This public key is then matched to the CA’s private key, and if successful the certificate is granted to the user.

A basic example of this would be if you connect to When connecting to this website, it automatically redirects you to, which is secured by way of a VeriSign-issued certificate. You know you have been redirected to a secure site because the browser has various indicators. For instance, the web browser will probably show a padlock in the locked position and it and the name of the company will be displayed in green, as shown in Figure 15-1.


Figure 15-1 Example of a Secure Connection, Shown in Firefox

If you were to click on the green area of the address field in Firefox, you could ultimately get to the certificate details. Figure 15-2 shows the default General tab for the certificate associated with this domain name.


Figure 15-2 Details of a Typical VeriSign Certificate

The General tab shows that the certificate gets a super-long hexadecimal serial number, and shows when the certificate was originally issued and when it expires, among other information. You can also note that the certificate has been fingerprinted with SHA-256 (a variant of SHA-2) and SHA1, enabling you or the website (or issuer) to verify the integrity of the certificate. If for some reason the certificate cannot be verified by any of the parties, and the issuer confirms this, then the issuer would need to revoke it and place it in the certificate revocation list (CRL). The Details tab gives us advanced and more complete information about the certificate used. I suggest you take a look at a few more websites that use SSL/TLS certificates and peruse the General and Details tabs. Compare the certificates with each other to learn more about the different levels of encryption, different levels of fingerprinting, and the different issuing companies.

One way to add security to the certificate validation process is to use certificate pinning, also known as SSL pinning or public key pinning. This can help to detect and block many types of MITM attacks by adding an extra step beyond normal X.509 certificate validation. Essentially, a client obtains a certificate from a CA in the normal way, but also checks the public key in the server’s certificate against a hashed public key used for the server name. This functionality must be incorporated into the client side, so it is important to use a secure and up-to-date web browser on each client in order to take advantage of certificate pinning.

Recipients can use one or more certificates. Certificate mapping defines how many certificates are associated with a particular recipient. If an individual certificate is mapped to a recipient, it is known as a one-to-one mapping. If multiple certificates are mapped to a recipient, it is known as many-to-one mapping. Multiple certificates might be used if the recipient requires multiple secure (and separate) communications channels.

In some cases, a registration authority (RA) is used to verify requests for certificates. If the request is deemed valid, the RA informs the CA to issue the certificate. An RA might also be used if the organization deals with several CAs. In this case, the RA is at the top of a hierarchical structure and verifies the identity of the user. An RA isn’t necessary in a PKI, but if you are centrally storing certificates, a CA is necessary.

Certificate authorities aren’t just for the rich and famous (for example, PayPal using VeriSign as the issuer). You can have a CA, too! If you are running a Windows Server, you can install your own CA—for example, one that utilizes L2TP or possibly SSL/TLS; more on those protocols later in this chapter. Of course, a server’s built-in certificates are not necessarily secure. If you were to implement this technology in a secure environment in your organization, you would probably want to obtain proper certificates from a trusted source to use with the Windows Server. When implementing certificates in Windows Server, you would use the Active Directory Certificate Services (AD CS) utility. From there you can define object identifiers (OIDs), which are built into AD CS for either low, medium, or high assurance. Or, you can have Windows randomly assign them. For security purposes, obtain the OID before completing the configuration of the CA.

Certificate authorities can be subverted through the use of social engineering. If a person posing as a legitimate company managed to obtain certificates from a trusted source, those certificates would appear to be valid certificates and could cause widespread damage due to connections made by unsuspecting users. That is, until the certificates were revoked. This happens sometimes, but the CA issuer usually finds out quickly and takes steps to mitigate the problem, including revoking the certificate(s) and notifying any involved parties of the incident.

The certificate revocation list (CRL) is a list of certificates that are no longer valid or that have been revoked by the issuer. There are two possible states of revocation: revoked, which is when a certificate has been irreversibly revoked and cannot be used again, and hold, which is used to temporarily invalidate a certificate. Reasons for revoking a certificate include the compromising or theft of a certificate or entire CA, unspecified certificates, superseded certificates, held certificates, and key or encryption compromise. The CRL is published periodically, usually every 24 hours. This enables users of an issuer’s certificates to find out whether a certificate is valid. CRLs, like the certificates themselves, carry digital signatures to prevent DoS and spoofing attacks; the CRL is digitally signed by the CA.

An alternative to the CRL is the Online Certificate Status Protocol (OCSP). It contains less information than a CRL does, and the client side of the communication is less complex. However, OCSP does not require encryption, making it less secure than CRL. An alternative to OCSP is OCSP stapling (previously known as TLS Certificate Status Request), which allows the presenter of the certificate to bear the cost involved when providing OCSP responses.

Certificate keys can also be held in escrow. Key escrow is when a secure copy of a user’s private key is held in case the key is lost. This may be necessary so that third parties such as government or other organizations can ultimately gain access to communications and data encrypted with that key. If data loss is unacceptable, you should implement key escrow in your PKI.

When installing a certificate authority to a Windows Server, you can set up a recovery agent for lost or corrupted keys. To do this, you need Windows Server and need to set up an Enterprise-level CA. In this configuration, the certificates (or private keys) are archived at the CA. If a key recovery agent has been configured, lost, or corrupted, keys can be restored. It’s important to use some type of software that can archive and restore keys in case of an incident or disaster.

Another way to avoid single points of failure, such as a single CA, is to organize certificate authorities in a hierarchical manner. At the top of the tree is a root CA; underneath are subordinate, or intermediate, CAs that offer redundancy and can sign certificates on behalf of the root CA. Though CA exclusivity is common, it is not the only type of architecture used to bind public keys to users. In some cases, a centralized model for certificates is not required or desired.

Remember that if a root CA is compromised, all of its certificates are then also compromised, which could affect an entire organization and beyond. The entire certificate chain of trust can be affected. One way to add a layer of security to avoid root CA compromise is to set up an offline root CA. Because it is offline, it will not be able to communicate over the network with the subordinate CAs, or any other computers for that matter. Certificates are transported to the subordinate CAs physically using USB flash drives or other removable media. We discussed the mindset of offline computing in Chapter 9, “Securing Network Media and Devices”; this is yet another example. Of course, you would need to have secure policies regarding the use and transport of media, and would need to incorporate data loss prevention (DLP), among other things. But the offline root CA has some obvious security advantages compared to an online root CA. Consider this offline mindset when dealing with critical data and encryption methods.


One thing you can take away from this discussion of certificates is that there have been many certificate exploits in the past and lots of vulnerabilities still exist. Be very careful during the planning stage of certificates.

Web of Trust

A web of trust is a decentralized trust model that addresses issues associated with the public authentication of public keys common to CA-based PKIs. It is considered peer-to-peer in that there is no root CA; instead, self-signed certificates are created and used that have been attested to by the creator. Users can decide what certificates they want to trust and can share those trusted certificates with others, making the web of trust grow larger. Of course, one of the most common reasons that a certificate issuer is not recognized by a web browser is due to unknown self-signed certificates. This model can also interoperate with standard CA architectures inherent to PKI. The more people that show trust of a certificate, the higher the chance that it is legitimate. This model is used by PGP, which enables users to start their own web of trust, self-publishing their own public key information.

Security Protocols

You can use a variety of security protocols to allow for more secure connections to other systems and networks. But the question is: What should be secured when connecting to other computers? I like to break it down into four categories:

Image E-mail and other communications: This can be accomplished with the use of S/MIME or PGP.

Image E-commerce and web logins: This can be brought about with the aid of protocols such as SSL and TLS.

Image Direct connections to other computers: This can be done with a protocol such as SSH.

Image Virtual connections to remote networks: This can be achieved with virtual private networks and protocols such as PPTP and L2TP.

Each of these scenarios builds on the concepts you learned in the previous PKI section. Let’s define each of these scenarios and the security protocols used in more depth.


Originally developed by RSA Security, Secure/Multipurpose Internet Mail Extensions (S/MIME) is an IETF standard that provides cryptographic security for electronic messaging such as e-mail. It is used for authentication, message integrity, and non-repudiation of origin. Most e-mail clients have S/MIME functionality built-in. S/MIME uses a separate session key for each e-mail message.

S/MIME relies on PKI and the obtaining and validating of certificates from a CA, namely X.509v3 certificates. It also relies on digital signatures when attempting to establish non-repudiation. S/MIME enables users to send both encrypted and digitally signed e-mail messages.

S/MIME can be implemented in Outlook by first obtaining a certificate known as a Digital ID, publishing the certificate within Outlook, and then modifying the settings for Outlook, as shown in Figure 15-3.


Figure 15-3 S/MIME Settings in Outlook

One of the issues with S/MIME is that it encrypts not only messages but also any malware that found its way into the message. This could compromise systems between the sender and receiver. To defeat this, scan messages at a network gateway that has a copy of the private keys used with S/MIME. Do this after decryption. If an e-mail program stores an S/MIME-encrypted message and the private key used for encryption/decryption is lost, deleted, or corrupted, the message cannot be decrypted.


Secure Sockets Layer (SSL) and its successor Transport Layer Security (TLS) are cryptographic protocols that provide secure Internet communications such as web browsing, instant messaging, e-mail, and VoIP. These protocols rely on a PKI for the obtaining and validating of certificates.

Many people refer to the secure connections they make to websites as SSL, but actually some of these will be TLS. The last version of SSL, version 3, was released in 1996. TLS is a more secure solution; version 1 of TLS supersedes SSLv3. As of the writing of this book, the latest version of TLS is 1.2 (defined in 2008), with 1.3 as a working draft and an eventuality. However, what you should be most interested in is the strength of the cipher—something to keep in mind when inquiring as to SSL or TLS certificates. TLS and SSL work in much the same manner. Two types of keys are required when any two computers attempt to communicate with the SSL or TLS protocols: a public key and a session key. Asymmetric encryption is used to encrypt and share session keys, and symmetric encryption is used to encrypt the session data. Session keys used by protocol such as TLS are used only once—a separate session key is utilized for every connection. A recovery key will be necessary if any data is lost in an SSL/TLS session. SSL and TLS encrypt segments of network connections that start at the transport layer of the OSI model. The actual encryption occurs at the session layer. In general, SSL and TLS are known as application layer protocols.

If a server running SSL/TLS requires additional processing, consider a SSL/TLS accelerator. This can be another computer, or more commonly, an add-on card that solely works on the CPU-intensive public-key encryption, namely the SSL/TLS handshake process. Your organization might have a policy that states that SSL/TLS-encrypted data needs to be decrypted when it reaches the internal network, and then analyzed for malware and potential attacks. It is often then re-encrypted and sent to its final destination. This is also very CPU intensive, and an SSL/TLS accelerator can provide the additional power required. Know that SSL/TLS decryption and re-encryption can be a security risk and a privacy issue (especially for BYOD users). Careful consideration is required regarding where the decryption/re-encryption will take place, how it is implemented, and how people are notified about this policy.

HTTPS, which stands for Hypertext Transfer Protocol Secure, is a combination of HTTP and either SSL or TLS. Web servers that enable HTTPS inbound connections must have inbound port 443 open. This is common for e-commerce. If you connect to an online shopping portal such as Amazon, your credit card transactions should be protected by HTTPS, and you should see the protocol within the address bar of your browser when you enter a secure area of the website. HTTPS should not be confused with Secure HTTP (SHTTP). SHTTP is an alternative to HTTPS that works in much the same way. Because SHTTP was neglected by Microsoft, Netscape, and others in the 1990s, and because SHTTP encrypts only application layer messages, HTTPS became the widely used standard. HTTPS can encrypt all data passed between the client and the server, including data passing through layer 3.

E-mail protocols can use SSL/TLS as well. For example, there is SSL/TLS-encrypted POP (which uses port 995), SSL/TLS SMTP (uses port 465), and SSL/TLS IMAP (uses port 993).

One attack to watch for is the downgrade attack—when a protocol is downgraded from a high-quality mode or higher version to a low-quality mode or lower version. Many types of encryption protocols can be downgraded, but perhaps the most commonly targeted protocols are SSL and TLS. This is accomplished when backward compatibility is enabled on a system, and is often implemented as part of an MITM attack. Obviously, the removal of backward compatibility can help prevent the attack on the server side and on the client side, but also preventive measures against MITM and similar enveloping attacks can be beneficial. For example, using an IDS/IPS solution within the company network, and utilizing encrypted VPN tunnels for data sessions, are preventive measures that can be used against downgrade attacks.

SSL can be used by attackers as well. SSL-encrypted malware such as the Zeus or Gameover banking Trojans utilize the secure nature of SSL to exist undetected. Victims are often spammed a false update program that (if opened) downloads the Trojan payload through an SSL-encrypted connection from an infected website. The only way an individual user can protect from this is to have updated anti-malware running, and not open any unknown attachments. However, organizations can use next-generation firewalls (NGFWs) to filter out SSL-encrypted traffic. They might use these in addition to their regular firewalls, used for unencrypted traffic.


Secure Shell (SSH) is a protocol that can create a secure channel between two computers or network devices, enabling one computer or device to remotely control the other. Designed as a replacement for Telnet, it is commonly used on Linux and Unix systems, and nowadays also has widespread use on Windows clients. It depends on public key cryptography to authenticate remote computers. One computer (the one to be controlled) runs the SSH daemon, while the other computer runs the SSH client and makes secure connections to the first computer (which is known as a server), as long as a certificate can be obtained and validated.

Computers that run the SSH daemon have inbound port 22 open. If a proper SSH connection is made, files can also be transferred securely using SFTP (Secure File Transfer Protocol) or SCP (Secure Copy Protocol). Tunneling is also supported.

Vulnerabilities to SSH 1 and 1.5, such as the unauthorized insertion of content, the forwarding of client authentications to other servers (daemons), and integer overflow, precipitated the development of SSH 2.0, which is incompatible with SSH version 1. Improvements to SSH 2.0 include usage of the Diffie-Hellman key exchange and integrity checking with message authentication codes (MACs).

PPTP, L2TP, and IPsec

Virtual private networks (VPNs) were developed to enable quick, secure, remote connections using the inherent capacity of the Internet. They were also developed to take advantage of faster Internet connections such as cable, DSL, and so on but still work with dial-up connections. The issue with VPNs is how to secure those connections. Basically, there are two common protocols used to do so: PPTP and L2TP (with the aid of IPsec).


The Point-to-Point Tunneling Protocol (PPTP) is a protocol used in VPNs. It encapsulates PPP packets, ultimately sending encrypted traffic. PPP by itself is useful for dial-up connections but is not suitable for a VPN by itself without a protocol such as PPTP. Servers and other devices running the PPTP protocol and accepting incoming VPN connections need to have inbound port 1723 open.

PPTP can be used with CHAP-based authentication protocols. Because the protocol MSCHAPv1 is considered inherently insecure, and MSCHAPv2 is vulnerable to dictionary attacks, PPTP is often deemed to be vulnerable. These authentication vulnerabilities can be dismissed if PPTP is used with an authentication method such as EAP-TLS. This relies on the existence of a PKI for the client and server computers. If this infrastructure is not readily available, PEAP can be used instead, as long as the computers are running the Windows Vista operating system or newer. Otherwise, L2TP with IPsec or other tunneling protocols is recommended for environments in which session and data security is of paramount importance.


The Layer 2 Tunneling Protocol (L2TP) is a tunneling protocol used to connect VPNs. In essence, it creates an unencrypted tunnel if used by itself (which would be unwise). It does not include confidentiality or encryption on its own, but when paired with a security protocol such as IPsec it is considered a formidable tunneling protocol.

Its starting point is based on the Layer 2 Forwarding Protocol (L2F) and PPTP. The latest version is L2TPv3, which has improved encapsulation and increased security features. Servers and other devices accepting incoming VPN connections need to have inbound port 1701 open.

When installed on a Windows Server, it uses a PKI. Valid certificates need to be downloaded to clients before they can make a VPN connection to the server. Security must be configured on the server side and the client side. Generally, the IPsec protocol is used to accomplish the secure connection within the L2TP tunnel.


Internet Protocol Security (IPsec) authenticates and encrypts IP packets, effectively securing communications between the computers and devices that use this protocol. IPsec operates at the network layer of the OSI model. It differs from SSH, SSL, and TLS in that it is the only protocol that does not operate within the upper layers of the OSI model. It can negotiate cryptographic keys and establish mutual authentication. IPsec is made up of three other protocols that perform its functions, including:


Image Security association (SA): This is the establishment of secure connections and shared security information, using either certificates or cryptographic keys. It is set up most often through the Internet Key Exchange (IKE) or via Kerberized Internet Negotiation of Keys. The IKE can select varying levels of security protocols for the computers in a connection, which can differ in a VPN due to the dissimilar computers (with disparate protocols) that might attempt to connect to it.

Image Authentication header (AH): This offers integrity and authentication. The authentication information is a keyed hash based on all the bytes in the packet. It can be used with the Encapsulating Security Payload (ESP) protocol. It can protect against replay attacks by employing sliding window protocols, which put limits on the total amount of packets that can be transceived in a given timeframe but ultimately enables an unlimited number of packets to be communicated using fixed-size sequence numbers.

Image Encapsulating Security Payload (ESP): This provides integrity, confidentiality, and authenticity of packets. Protected data is encapsulated and encrypted.

IPsec can be implemented in two ways:

Image Transport mode: In host-to-host transport mode, the payload of the IP packet is encrypted, but the header information is not. The AH is still hashed though. This mode allows for the secure transfer of data among computers in a LAN or other private, internal type of network. It can be used for transmission over networks, using NAT traversal (NAT-T), but tunnel mode is more often used for that purpose.

Image Tunnel mode: Network tunneling mode is when the entire IP packet is encrypted. It takes the regular IP packet and encapsulates that inside of a new IP packet with a separate header. This mode is the more commonly used mode for transmission between networks, and is the one that facilitates VPNs. Keep in mind that an organization might favor on-demand VPNs (which make use of SSL/TLS certificates) over IPsec-based VPNs.

IPsec uses algorithms such as SHA2 (and possibly HMAC-SHA1) for integrity and authenticity, which hashes the packets of data; afterward the hash is encrypted. It can also use Triple DES and AES for confidentiality.

Chapter Review Activities

Use the features in this section to study and review the topics in this chapter.

Chapter Summary

Because public keys are common knowledge, they must be protected from compromise. One way to do this is to implement a public key infrastructure (PKI). This is a complete environment for the public key, including hardware, software, and procedures.

Users who want to access a website securely are required to request a certificate; this will bind the user identity with the public key. The certificate is issued from a certificate authority (CA) such as VeriSign. To be validated, a user’s computer initiates a certificate signing request (CSR) with proof of the user’s identity. If a certificate used by a website is no longer valid, or is suspected to be compromised, it needs to be revoked right away and placed on a public certificate revocation list (CRL).

PKI is used during secure web sessions (HTTPS, for example) that use SSL or TLS protocols, e-mail sessions that use S/MIME, and secure sessions between computers with SSH. Virtual private networks can also use a PKI; for example, a VPN that relies on L2TP and therefore certificates as well. To further secure an L2TP tunneled connection, IPsec is implemented. Keep in mind that always-on VPN technology is also available, which makes use of SSL/TLS instead of PPTP or L2TP.

However, PKI is most commonly used for secure web transactions. Within that system, a certificate will employ asymmetric encryption (such as RSA or ECC) and a cryptographic hash (such as SHA) for the key exchange. Afterward, the rest of the session’s data will be encrypted in a symmetric format (such as AES or RC4), which uses less computational power than asymmetric encryption.

As you travel the Internet, check on the validity (and level of security) of the certificates used by websites. Any site that you can log into, or conduct any type of transaction regarding anything of value, should have an HTTPS connection, as well as SSL or TLS, RSA or ECC, SHA-2 or MD5, and AES or RC4 cryptographic protocols.

Review Key Topics

Review the most important topics in the chapter, noted with the Key Topic icon in the outer margin of the page. Table 15-1 lists a reference of these key topics and the page number on which each is found.


Table 15-1 Key Topics for Chapter 15

Key Topic Element Description Page Number
Figure 15-1 Example of a secure connection, shown in Firefox 354
Figure 15-2 Details of a typical VeriSign certificate 354
Bulleted list IPsec protocols 360

Define Key Terms

Define the following key terms from this chapter, and check your answers in the glossary:

public key infrastructure (PKI)



wildcard certificate

subject alternative name (SAN)

certificate authority (CA)

one-to-one mapping

many-to-one mapping

registration authority (RA)

certificate revocation list (CRL)

Online Certificate Status Protocol (OCSP)

key escrow

key recovery agent

web of trust

Secure/Multipurpose Internet Mail Extensions (S/MIME)

Secure Sockets Layer (SSL)

Transport Layer Security (TLS)

downgrade attack

Secure Shell (SSH)

Point-to-Point Tunneling Protocol (PPTP)

Layer 2 Tunneling Protocol (L2TP)

Internet Protocol Security (IPsec)

Complete the Real-World Scenarios

Complete the Real-World Scenarios found on the companion website ( You will find a PDF containing the scenario and questions, and also supporting videos and simulations.

Review Questions

Answer the following review questions. Check your answers in Appendix A, “Answers to the Review Questions.”

1. Which of the following does not apply to an X.509 certificate?

A. Certificate version

B. The issuer of the certificate

C. Public key information

D. Owner’s symmetric key

2. What two items are included in a digital certificate? (Select the two best answers.)

A. User’s private key

B. Certificate authority’s digital signature

C. The user’s public key

D. Certificate authority’s IP address

3. Rick has a local computer that uses software to generate and store key pairs. What type of PKI implementation is this?

A. Distributed key

B. Centralized

C. Hub and spoke

D. Decentralized

4. Which of the following is usually used with L2TP?

A. IPsec




5. What ensures that a CRL is authentic and has not been modified?

A. The CRL can be accessed by anyone.

B. The CRL is digitally signed by the CA.

C. The CRL is always authentic.

D. The CRL is encrypted by the CA.

6. Which of the following encryption concepts is PKI based on?

A. Asymmetric

B. Symmetric

C. Elliptical curve

D. Quantum

7. You are in charge of PKI certificates. What should you implement so that stolen certificates cannot be used?





8. Which of the following are certificate-based authentication mapping schemes? (Select the two best answers.)

A. One to-many mapping

B. One-to-one mapping

C. Many-to-many mapping

D. Many-to-one mapping

9. Which of the following network protocols sends data between two computers while using a secure channel?




D. P2P

10. Which of the following protocols uses port 443?





11. Which of the following protocols creates an unencrypted tunnel?



C. IPsec


12. In a public key infrastructure setup, which of the following should be used to encrypt the signature of an e-mail?

A. Private key

B. Public key

C. Shared key

D. Hash

13. Two computers are attempting to communicate with the SSL protocol. Which two types of keys will be used? (Select the two best answers.)

A. Recovery key

B. Session key

C. Public key

D. Key card

14. Which layer of the OSI model does IPsec operate at?

A. Data link

B. Network

C. Transport

D. Application

15. Which layer of the OSI model is where SSL provides encryption?

A. Network

B. Transport

C. Session

D. Application

16. Which of the following details one of the primary benefits of using S/MIME?

A. S/MIME expedites the delivery of e-mail messages.

B. S/MIME enables users to send e-mail messages with a return receipt.

C. S/MIME enables users to send both encrypted and digitally signed e-mail messages.

D. S/MIME enables users to send anonymous e-mail messages.

17. What should you do to make sure that a compromised PKI key cannot be used again?

A. Renew the key.

B. Reconfigure the key.

C. Revoke the key.

D. Create a new key.

18. Which of the following statements is correct about IPsec authentication headers?

A. The authentication information is a keyed hash based on half of the bytes in the packet.

B. The authentication information is a keyed hash based on all the bytes in the packet.

C. The authentication information hash will remain the same even if the bytes change on transfer.

D. The authentication header cannot be used in combination with the IP Encapsulating Security Payload.

19. Which of the following protocols is not used to create a VPN tunnel and not used to encrypt VPN tunnels?




D. IPsec

20. Which of the following answers are not part of IPsec? (Select the two best answers.)


B. Key exchange


D. Authentication header

21. What should you publish a compromised certificate to?





22. You have been asked to set up authentication through PKI, and encryption of a database using a different cryptographic process to decrease latency. What encryption types should you use?

A. Public key encryption to authenticate users and public keys to encrypt the database

B. Public key encryption to authenticate users and private keys to encrypt the database

C. Private key encryption to authenticate users and private keys to encrypt the database

D. Private key encryption to authenticate users and public keys to encrypt the database

23. Which of the following uses an asymmetric key to open a session, and then establishes a symmetric key for the remainder of the session?






24. Which of the following describes key escrow?

A. Maintains a secured copy of the user’s private key for the purpose of recovering the CRL

B. Maintains a secured copy of the user’s private key for the purpose of recovering the key if it is lost

C. Maintains a secured copy of the user’s public key for the purpose of recovering messages if the key is lost

D. Maintains a secured copy of the user’s public key for the purpose of increasing network performance

25. When a user’s web browser communicates with a CA, what PKI element does the CA require from the browser?

A. Public key

B. Private key

C. Symmetric key

D. Secret key

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.