Cryptographic Algorithms: The Basics

A cryptographic algorithm defines or specifies a series of steps—some mathematical, some logical, some grouping or ungrouping of symbols, or other kinds of operations—that must be applied, in the specified sequence, to achieve the required operation of the system. Think of the algorithm as the total set of “swap rules” that you need to use, and the correct order to apply those rules in, to make the cryptographic system work properly. (Note, too, that I sometimes use cryptographic algorithm and encryption algorithm as interchangeable terms, even though a decryption algorithm is part of the same system too.) We mentioned before that the basic processes of substitution and permutation (also called transposition) can be repetitively or iteratively applied in a given cryptographic process. The number of rounds that an algorithm iterates over is a measure of this repetition. A combination of hardware and software features can implement this repetition. In many cases, these algorithms require a set of control parameters, such as seeds, salts, keys, block size, and cycle or chain (iteration) values. Both sender and (intended) receiver must agree to use a mutually consistent set of algorithms and control parameters if they are to successfully use cryptographic processes to send and receive information.

Encryption and decryption processes can suffer from what we call a collision, which can render them unusable. This can occur if one of the following happens:

• Two different plaintext phrases should not map (encrypt) to the same ciphertext phrase; otherwise, you lose the difference in meaning between the two plaintext inputs.
• Two different ciphertext phrases should not map (decrypt) to the same plaintext phrase; otherwise, you have no idea which plaintext meaning was intended.

Substitution and permutation are done in a series of steps to help make the encryption harder to break. Different cryptographic algorithms define these operations in unique ways and then go on to specify the order in which these combinations of substitutions and permutations is to be performed. Two broad classes of encryption algorithms, symmetric and asymmetric, make up the backbone of our cryptographic infrastructures.

Symmetric vs. Asymmetric Encryption

Both symmetric and asymmetric algorithms depend heavily on advanced mathematical concepts drawn from set theory, group theory, and number theory. These algorithms treat their input plaintext as a set of numbers—not as characters, phrases in a human language, images, or even executable machine language. As a result, the same algorithm, when used with the same keys and cryptovariable settings, will demonstrate the same cryptographic strength regardless of the type of data used as input plaintext.

Symmetric algorithms were the first to find extensive use in electronic and electromechanical form, notably in the cipher machines used throughout the first half of the 20th century. These algorithms perform relatively simple operations on each element of the plaintext input (bit, byte, or block), but they do a substantial number of them in combination and iteration to achieve some surprisingly powerful encryption results. By arranging the plaintext into blocks, and those blocks into two-dimensional matrices, permutation can swap rows, columns, or both in various blocks of an input plaintext. Substitution can also use these rows or columns (or whole blocks) as input values to functions that further obscure the meaning of the plaintext. Symmetric algorithms use the same key for encryption and decryption, or a simple transformation of one into the other. Symmetric algorithms usually must work on fixed-length blocks, and thus the last block will be padded to the block length specified by the controlling cryptovariables.

In addition to overall cryptographic strength (measured by resistance to attacks or the work factor or time required for a successful brute-force attack), the speed of processing is a major requirement for an effective symmetric algorithm. As a result, many hardware and software implementations will take the explicit iteration expressed as loops in the algorithm’s design and implement them in sequentially repeated blocks of code, hardware, or both. Symmetric encryption provides straightforward ways to protect large sets of data, whether in motion, at rest, or in use. However, this protection fundamentally rests upon the strategy used to choose and manage the encryption keys used with the chosen algorithm.

Asymmetric encryption does not use the same key for encryption and decryption; in fact, its cryptographic strength and utility derives from the fact that it is almost impossible to determine the decryption key from the algorithm alone, the encryption key itself, and the ciphertext it produces. It relies on so-called trapdoor functions that are relatively easy and straightforward to compute in one direction (for encryption), but their logical inverse is difficult if not impossible to compute in the other direction.

Asymmetric encryption came into practical use as a way of addressing three inherent problems with the widespread use of symmetric encryption: key distribution and management, improved security, and the management of trust relationships. These three sets of concepts are so interrelated that in many respects, it’s hard to explain one of these big sets of ideas without using the other two to do so!

Figure 5.1 summarizes the modern families of cryptographic algorithms by types, mathematical algorithms, and use. It also gives a quick round-up of the various protocols used in the public key infrastructure and in key management in general. As an SSCP, you’re going to need to be on a first-name basis with most if not all of the items shown on this “family tree.” Throughout this chapter we’ll use members of each branch of this tree to demonstrate important concepts and provide relevant details.

Cryptographic Keys

Cryptographic keys are a set of data that the encryption algorithm combines with the input plaintext to produce the encrypted, protected output—the ciphertext. Many different processes have been used over the centuries to produce encryption and decryption keys for use with a particular encryption algorithm or process.

• Published books, such as a specific edition of Shakespeare’s Romeo and Juliet, Caesar’s Commentaries, or even holy scriptures can provide a lookup table for either substitution or permutation operations. Bob, for example, could use such a book to encrypt a message by starting on a pre-agreed page, substituting the first letter in his plaintext for the first letter of the first line on the page. Carol would decrypt his message by using the same print edition of the book and go to the same pre-agreed page.
• One-time pads are a variation of using published books (and predate the invention of movable type). The key generator writes out a series of key words or phrases, one per sheet of paper, and makes only one copy of this set of sheets. Carol encrypts her message using the first sheet in the one-time pad and then destroys that sheet. Alice decrypts the ciphertext she receives from Carol using that same sheet and then destroys that sheet.
• Pseudorandom numbers of various length are also commonly used as keys. Senders and recipients each have a copy of the same pseudorandom number generator algorithm, which uses a seed value to start with. A sequence of pseudorandom numbers from such an algorithm provides either a one-time pad of encryption keys or a keystream for stream cipher use.
• Hardware random and pseudorandom number generators, combined with software functions, can also generate keys or keystreams. The latest of these use quantum computing technologies to generate unique keystreams.

In theory and in practice, the one-time pad encryption process is the only truly unbreakable system—when it is used correctly. Well-designed and properly used one-time systems have still resisted attempts to crack them, decades later. In fact, there are products available today that use quantum effects to generate gigabit-length key streams as one-time pads, burned into read-once memory devices, which are distributed to the two parties via bonded, trusted couriers. Once all the bits of that key have been used, the parties must buy new keys. Attempts to read the key values consumes them. Session keys are a form of one-time key system, but they aren’t a one-time pad key distribution approach.

If the algorithm is known to our adversaries (and probably published in an information security, mathematics, or cryptography journal anyway), how is it that the key provides us the security we need? Cryptologic key strength is a way to measure or estimate how much effort would be required to break (that is, illicitly decrypt) a cleartext message encrypted by a given algorithm using such a key. In most cases, this is directly related to the key size, defined as how many bits make up a key. Another way to think of this is that the key strength determines the size of the key space—the total number of values that such a key can take on. Thus, an 8-bit key can represent the decimal numbers 0 through 255, which means that an 8-bit key space has 256 unique values in it. Secure Socket Layer (SSL) protocol, for example, uses a 256-bit key as its session key (to encrypt and decrypt all exchanges of information during a session), which would mean that someone trying to brute-force crack your session would need to try 2²⁵⁶ possible values (that’s a 78-digit base-10 number) of a key to decrypt packets they’ve sniffed from your session. With one million zombie botnet computers each trying a million key values per second, that still needs 10⁵⁹ years to go through all values. (If you’re thinking of precomputing all such values, how many petabytes might such a “rainbow table” take up?)

Key distribution and management become the biggest challenges in running almost any cryptographic system. Keying material is a term that collectively refers to all materials and information that govern how keys are generated and distributed to users in a cryptographic system and how those users validate that the keys are legitimate. Key management processes govern how long a key can be used and what users and systems managers must do if a key has been compromised (by falling into the wrong hands or by a published cryptanalysis demonstrating how easily such keys can be guessed). Key distribution describes how newly generated keys are issued to each legitimate user, along with any updates to the rules for their period of use and their safe disposal.

Key distribution follows the same topological considerations as networks do; point-to-point, star, and full-mesh models express how many users of a cryptographic system need to share keys if they are to communicate with each other and under what rules for sharing of protected information. Consider these three typical topologies from a key distribution and management perspective. The simple one-time pad system connects only two users; only one pair of pads is needed. Most real-world needs for secure communication require much larger sets of users, however. For a given set of n users, the star topology requires n pairs of keys to keep traffic between each user and the central site secure and private—from all other users as well as from outsiders. A full-mesh system requires [n × (n – 1)] sets of keys to provide unique and secure communication for each pair of users on this mesh.

Exploitable vulnerabilities will exist in whatever choices you make regarding key generation, distribution, management, and use. Examining these vulnerabilities can also reveal ways that you as the security architect can mitigate the risks that they raise; this is covered in some detail later in this section by addressing cryptanalysis attacks and other attacks on the encryption elements of your information security posture. You’ll also look closely at a number of countermeasures you can use to mitigate these risks.

Protocols and Modules

The term cryptographic protocols can refer to two different sets of processes and techniques. The first is the use of cryptography itself in the operation of a cryptographic system, which typically can refer to key management and key distribution techniques. The second usage refers to the use of cryptographic systems and techniques to solve a particular problem. Secure email, for example, can be achieved in a variety of ways using different protocols, each of which uses different cryptographic techniques. We’ll look at these more closely later in this chapter.

A cryptographic module, according to Federal Information Processing Standards (FIPS) publication 140, is any combination of hardware, firmware, or software that implements cryptographic functions. What’s interesting about FIPS 140 is that it directly addresses the security of an information systems supply chain with respect to the underlying supply chain of its cryptographic elements. To earn certification as a cryptographic module, vendors must submit their works to the Cryptographic Module Validation Program (CMVP) for testing.

Notice that a vital element of encryption and decryption is that the original meaning of the plaintext message is returned to us—encrypting, transmitting, and then decrypting it did not change its meaning or content. The ciphertext version of information can be used as a signature of sorts—a separate verification of the authenticity or validity of the plaintext version of the message. Digital signatures use encryption techniques to provide this separate validation of the content of the message, file, or information they are associated with.

This brings us to a cryptographic system, which is the sum total of all the elements we need to make a specific application of cryptography be part of our information systems. It includes the algorithm for encrypting and decrypting our information; the control parameters, keys, and procedural information necessary to use the algorithm correctly; and any other specialized support hardware, software, or procedures necessary to make a complete solution.

  Sets and Functions

The simple concepts of sets and functions make cryptography the powerful concept that it is. As an SSCP, you should have a solid, intuitive grasp of both. The good news? As a human being, your brain is already 90 percent of the way to where you need to go!

Sets provide for grouping of objects or items based on characteristics that they have in common. It’s quite common to represent sets as Venn diagrams, using nested or overlapping shapes (they don’t always have to be circles). In the following figure, part (a) shows an example of proper subsets—one set is entirely contained within the one “outside” it—and of subsets, where not all members of one set are part of another (they simply overlap). Part (b) of the figure shows a group of people who’ve earned one or more computer security-related certifications; many only hold one, some hold two, and a few hold all three, as shown in the overlapping regions. If a subset contains all elements of another subset, it is called an improper subset.

Functions are mathematical constructs that apply a given set of operations to a set of input values, producing an output value as the result. We write this as:

f(x) = y or f(x) → y

The second form, written as a production function, shows that by applying the function f to the value x, we produce the value y.

Note that for any given value of x there can be only one y as a result.

For example, a simple cryptographic substation might be written as f(x) = mod(xor(x,keypart),b), where b is the number base for the modulo function, x is the input plaintext, and key is the portion of the key to exclusive-or with x. Cryptographic permutation might be written as f(x₁x₂x₃….x₈) = x₈…x₃x₂x₁, which transposes an 8-symbol input plaintext into its reverse order as ciphertext.

One powerful application of functions is to consider them as mapping one set to another. The previous function says that the set of all values of x is mapped to the set y. This is shown in part (c) of the figure, which shows how a list of out-of-limit conditions is mapped to a list of alarms. (This looks like a table lookup function.) If you wanted any of a set of conditions to trigger the same alarm, you wouldn’t use a function; you’d end up with something like the “check engine” light in automobiles, which loses more meaning than it conveys!

Not all mappings have to map every element of the source set into the destination set, nor do they use every element in the destination; some of these pairs (x,y) are just undefined. For example, the division function f(x) = y/x is undefined when x = 0 but not when y = 0.

Cryptography, Cryptology, or ?

There are many different names for very different aspects of how we study, think about, use, and try to crack “secret writing” systems. Some caution is advised, and as an SSCP you need to understand the context you’re in to make sure you’re using the right terms for the right sets of ideas.

For example, as Wikipedia and many others point out, many people, agencies, and academics use the terms cryptography and cryptology interchangeably, as if they mean the same things. Within the U.S. military and intelligence communities and those of many NATO nations, however, these terms have definite meanings.

• Cryptography refers specifically to the use and practice of cryptographic techniques.
• Cryptanalysis refers to the study of vulnerabilities (theoretical or practical) in cryptographic algorithms and systems and the use of exploits against those vulnerabilities to break such systems.
• Cryptology refers to the combined study of cryptography (the secret writing) and cryptanalysis (trying to break other people’s secret writing systems or find weaknesses in your own).
• Cryptolinguistics, however, refers to translating between human languages to produce useful information, insight, or actionable intelligence (and has little to do with cryptography).

You may also find that other ways of hiding messages in plain sight, such as steganography, are sometimes included in discussions of cryptography or cryptology.

Note, though, that cryptograms are not part of this field of study or practice—they are games, like logic puzzles, which present ciphers as challenges to those who want something more than a crossword puzzle to play with.

Black hats, gray hats, and white hats alike conduct all of these various kinds of cryptography-related activities.

Block Cipher Basics

Block ciphers take the input plaintext and break it into a fixed-length series of symbols (a block) and then encrypt and decrypt the block as if it was a single symbol. A block of 64 bits (8 eight-bit bytes) can be thought of as a 64-digit binary number, which is what the encryption algorithm would then work on. Block ciphers typically need to pad the last block of a fixed-length plaintext message (such as a file or an email) so that each block has the required length. Block ciphers are used both with symmetric and asymmetric encryption algorithms and keys; in the asymmetric case, the entire padded block is treated as a number, which then has some algebraic function applied to it, such as raising it to a power, to produce the resultant ciphertext.

As cryptographic engineers began building more and more complex systems, they modularized different functions into easily replicated hardware and software elements. Substitution is implemented in S-boxes, for example, while P-boxes perform permutation. Figure 5.3 shows a notional substitution operation (an S-box that does a table lookup), while Figure 5.4 shows a P-box that implements one particular transposition of its eight different input lines.

Many algorithms will split the input to each round of processing into pieces, such as a left half and a right half, and process them separately and recombine at the end of each round. Note that different algorithms will have their own unique definitions of what its S-boxes or P-boxes must perform (XOR, at least, is still an XOR). The Data Encryption Standard (DES) algorithm, which uses a 16-round Feistel network, is an example of rounds of S-Boxes and P-Boxes layered together. Figure 5.5 shows a notional Feistel network being used in encryption and decryption, and as you follow the flow through both processes, you’ll get a clearer sense of what cryptographers mean when they talk about iterating the permutations and substitutions.

Feistel networks or Feistel ciphers are named after Horst Feistel, a German-born cryptographer who designed block ciphers for the U.S. Air Force and IBM. Feistel’s passion was developing a cipher with repeating iterations or rounds. The use of S-boxes and P-boxes to create a repeating, often reversible structure, makes for a more easily wired or coded implementation that can be used to process a stream of text. DES is one example of a Feistel cipher construction. (AES, by contrast, is not a Feistel network.) Learn more about how the Feistel cipher process works during encryption and decryption at: https://www.tutorialspoint.com/cryptography/feistel_block_cipher.htm.

Block ciphers present some interesting problems and opportunities worth examining for a moment—padding, chaining and feedback, and optimization strategies such as the Electronic Code Book approach.

Padding and Block Ciphers

All block ciphers work on fixed-length blocks of plaintext (on one block at a time, of course), and these blocks must be padded out to the fixed block size that the algorithm is designed to use. This can be done by adding bytes (or bits) to the end of the short blocks, along with a counter value that indicates how much padding was used. Note how during decryption the last byte of the message block is examined to determine how many padding bytes have been added. If the plaintext is a multiple of the block size, then a final block that just contains padding must have been added. The padding bytes that have been added will thus need to be removed. Padding is not without its own risks, such as the Padding Oracle Attack described later in the “Side-Channel Attacks” section.

Cipher Block and Feedback Chaining

With Cipher Block Chaining (CBC), the first block of data is XORed with a block of random data called the initialization vector (IV). Every subsequent block of plaintext is XORed with the previous block of ciphertext before being encrypted. (See Figure 5.6.)

With Cipher Feedback (CFB) mode, the IV is encrypted and then XORed with the first block of the plaintext, producing the first block of ciphertext. Then that block of ciphertext is encrypted, and the result is XORed with the next block of plaintext, producing the next block of ciphertext. (See Figure 5.7.)

Because with both CBC and CFB the encryption of block P_n+1 depends on the encryption of block P_n, neither mode is amenable to the parallel encryption of data. Both modes can, however, be decrypted in parallel.

The main differences between CBC and CFB are as follows:

• With CBC, a one-bit change in the IV will result in the same change in the same bit in the first block of decrypted ciphertext. Depending on the application, this could permit an attacker who can tamper with the IV to introduce changes to the first block of the message. This means with CBC it is necessary to ensure the integrity of the IV.
• With CFB, a one-bit change in the IV will result in random errors in the decrypted message and thus is not a method of effectively tampering with the message.
• With CBC, the decryption of messages requires the use of the block cipher in decryption mode. With CFB, the block cipher is used in the encryption mode for both encryption and decryption, which can result in a simpler implementation. Their benefit comes from being structured in a way that is reversible or nearly reversible. For a mathematical view on how the reversibility trait is achieved, read this technical walk-through: http://cryptowiki.net/index.php?title=Generalized_Feistel_networks. For a presented discussion of an eight-round Feistel cipher, this video is recommended: https://www.youtube.com/watch?v=3kr6DbulIVc.

The problem with both modes is that encryption cannot be parallelized, which affects speed and throughput, and random access is complicated by the need to decrypt block C_n-1 before one can decrypt the desired block Cn.

Another mode called counter or CTR mode addresses this problem by not using previous blocks of the plaintext (CBC) or ciphertext (CFB) in producing the ciphertext. (See Figure 5.8.) By using an IV combined with a counter value, one can both parallelize the encryption process as well as decrypt a single block of the ciphertext. You’ll note that Figure 5.8 includes a nonce value. That unique, randomly generated value is inserted into each block cipher encryption round. Similar to how a random “salt” value is used to ensure different hash values (to prevent comparing to rainbow tables), a nonce is unique and is intended to prevent replay attacks.

With all of the modes other than ECB, you need an initialization vector (IV), which either must be communicated to the receiver, or the message must be prefixed by a throw-away block of data (since decryption of an CBC or CFB stream of data without knowing the IV will only cause problems for the first block).

The IV need not be secret (it can be transmitted in plaintext along with the ciphertext), but it must be unpredictable. If an attacker can predict the next IV that will be used and is able to launch a chosen plaintext attack, then that may enable launching a dictionary attack on the ciphertext.

Electronic Code Book

Once the message has been padded to be an exact multiple of the cipher’s block size, it can be encrypted. The easiest, obvious, and least secure method (for longer messages) is the Electronic Code Book (ECB) mode of operation. In this mode, each block of plaintext is processed independently by the cipher. And each block is processed or encrypted using the same key.

Using the same key to encrypt each block brings both a significant advantage and disadvantage. Using the same key greatly simplifies the process. The disadvantage is, while it may be adequate for messages that are no greater than the block size, a serious weakness develops for longer messages as identical blocks of plaintext will produce identical blocks of ciphertext. An example of this weakness can be seen in Figure 5.9, where the graphic is encrypted using ECB, using a comparatively small block size. While each block is adequately encrypted, the process is repeated so often to encrypt the entire graphic that the overall picture is actually recognizable.

Even in situations in which the data to be encrypted is the same or smaller than the block size (e.g., a numeric field in a database), use of ECB may be ill-advised if revealing that different rows of the table have the same value might compromise confidentiality. As a trivial example, if one were to use ECB to encrypt the birthdate field, then one could easily determine all the people in the database born on the same day, and if one could determine the birthdate of one of those individuals, you would know the birthdate of all (with the same encrypted birthdate).

The advantage of ECB, apart from its simplicity, is that encryption can be done in parallel (i.e., divided up across multiple processors), and so can decryption. Consequently, an error in one block does not affect subsequent blocks.

Table 5.1 provides a quick overview of some of the block ciphers you’ll frequently encounter or hear about. It indicates the type, block sizes, key sizes, and number of rounds (or iterations) that the algorithm defines.

Prior to the 1970s, there were no publicly available encryption systems of any market significance. In fact, U.S. law at the time still reflected post World War I sentiment that only the government should be able to transmit messages in codes. Building on a seminal paper published by Claude Shannon in 1949, commercial, government, and academic researchers around the globe began developing the next generation of complex, powerful block encryption algorithms. The explosive growth of computing in the 1960s and 1970s, the shift of public telephony to digital technologies, and even the birth of the hacking and hobbyist communities came together to make the demand for commercially available encryption systems seem like too good a business opportunity to pass up. Thus was born the first public competition for a new encryption standard.

Data Encryption Standard and Triple

The Data Encryption Standard (DES) was, and still is, quite controversial. It was the first published and open competition by the U.S. government for a new symmetric-key block encryption algorithm. Some reviewers alleged that NSA had inserted elements into the design (its “S-box” circuits) to allow DES-encrypted traffic to be decrypted by NSA without needing the original encryption key; others, in turn, insisted these S-boxes were there to defeat still other backdoors built into DES. To date, no one has been able to convincingly confirm or deny these fears, and the disclosure of many NSA secrets by Edward Snowden only reheated this simmering controversy. There were many arguments about the key length as well; IBM originally proposed using 64-bit keys, which were downsized at NSA’s insistence to 56 bits. (The key actually remains 64 bits in length, but since 8 bits are used for parity checking, the effective key length is still 56 bits.) DES was made a U.S. Federal Information Processing Standard in 1977, despite much outcry within the community that it was insecure right from the start.

DES used 16 rounds of processing, and its design reflects the capabilities of 1970s-era hardware. (This was the era of the first 8-bit microprocessors, and most minicomputer architectures had only a 16-bit address space.)

Although many people argued whether DES was in fact secure or insecure, the Electronic Frontier Foundation (EFF) spent $250,000 to build a custom “DES Cracking Machine” to prove their point. Its 29 circuit boards hosted 64 custom application-specific integrated circuit chips (ASICs), with the whole assemblage controlled by a single personal computer. At more than 90 billion key tests per second, it would take about nine days to brute-force test the entire 56-bit DES key space, with typical cracks happening in one to two days. At about the same time, another machine, the Cost Optimized Parallel COdeBreaker or COPACABANA, hit comparable cracking speeds but at substantially lower cost.

Significant work was done to try to tighten up DES, including the Triple DES standard published in 1999, which in effect applied three super-iterations of the basic DES algorithm. But it remained unsecure, and DES in all forms was finally withdrawn as a U.S. government standard in 2002 when it was superseded by AES.

DES remains important, not because it is no longer secure but because in the opinion of academics, industry, and government experts it stimulated the explosive growth in the study of cryptography by those who had no connections at all to the military and intelligence communities and their cryptographers. Even today it is still worth studying as you begin to understand cryptography, cryptanalysis, and common attack strategies.

Advanced Encryption Standard

Throughout the 1980s and 1990s work continued to find a replacement for DES, and in November 2001 the U.S. government published FIPS 197 describing the Advanced Encryption Standard (AES). AES is also a symmetric block encryption algorithm but one that is much more secure in design and implementation than DES or 3DES proved to be. The open competition that NSA had sponsored reviewed a number of promising designs, and the Rijndael (pronounced “rhine-dahl”) algorithm by Vincent Rijmen and Joan Daeman was the clear winner. It is the first and still the only publicly available cipher that is approved by NSA for use on government classified information up through Top Secret, when it is used as part of an NSA-approved cryptographic module.

AES uses a fixed block size of 128 bits and provided a number of possible key sizes, which were directly related to the number of rounds of processing used, as shown in Table 5.1. AES follows the design principles of a substitution-permutation network, which in some respects looks like a pretty straightforward application of substitutions, permutations, XORs, and matrix manipulations. AES is not, however, a Feistel network (which is also a permutation-substitution design concept). AES uses the principles of finite field arithmetic to manipulate its arrays of bytes, known as the state of the algorithm. (These states can have a varying number of columns but are always four rows of bytes.) Yet despite this apparent simplicity, it has withstood a number of very ambitious attacks and is still a primary standard of choice by security architects and savvy users.

Internally, the AES algorithm goes through four major phases of processing:

• Key expansion, in which keys for a given round of processing are derived from the cipher key using the Rijndael key schedule, which uses a set of round constants to generate a unique 128-bit round key block for each round (along with one extra round key block).
• Initial round key addition, which does a bitwise XOR of the state and the round key.
• An additional set of 9, 11, or 13 rounds, depending upon key length, which combine byte substitution, row shifting, and column mixing, culminating in another XOR of the final state and the round key.
• A final round consisting of byte substitution, row shifting, and round key addition.

A number of optimization techniques are included in the basic AES design, which allow implementation decisions to trade memory space for processing time, for example.

As usual, the cryptography community has produced a quite lucid set of pages on AES, which you can find starting at https://en.wikipedia.org/wiki/Advanced_Encryption_Standard.

Until May 2009, the only known successful attacks on AES were side-channel attacks, which rely on observing possible data leaks that reveal internal characteristics of the implemented cryptosystem (such as heat, vibration, power fluctuations, or noise on signal or power connections), rather than attacks on the mathematics of the cipher itself. As of the Snowden disclosures in 2013, there were still no known practical attacks against a properly implemented AES system using 256-bit keys.

Blowfish and Twofish

These algorithms were both developed by Bruce Schneier as part of his work to find a stronger replacement for DES. Twofish, derived from Blowfish’s design, was one of the five finalist designs in the AES competition; when that competition chose the Rijndael algorithm, Schneier placed the reference implementation of the designs of Blowfish and Twofish into the public domain. They’ve been widely incorporated into a wide variety of commercial and open source cryptographic systems and products, with Twofish being one of the few ciphers included in the OpenPGP standard (RFC 4880) by the Internet Engineering Task Force. They both are Feistel substitution-permutation designs, and thus far, the only successful attack has been the SWEET32 or “birthday attack,” in an HTTPS context, which was able to recover plaintext from ciphertext for ciphers using a 64-bit block size. As a result, the GnuPG project recommended that so long as Blowfish is not used on files larger than 4 GB in size, it should still be secure.

OpenPGP and GnuPG are both part of what can only be called the public service aspect of the white hat cryptographic community of researchers, designers, and users. This community firmly believes that all of us need alternatives to the systems provided to us by governments or large multinational corporations. The intersection of very big business interests and governments’ natural drives for control can, this community warns us, have us becoming too dependent upon systems that only are secure against us. (As an international banker acquaintance of mine puts it, the Bank Secrecy Acts of various nations are about one-way secrets: banks keeping secrets from their customers.) The commercial and open source successes of GnuPG, OpenPGP, PGP, and GPG are in part a testament to the credibility of these warnings.

International Data Encryption Algorithm

The International Data Encryption Algorithm (IDEA) was first proposed by James Massey and Xuijia Lai in 1991 and was at that time intended as a proposed replacement for DES. It is a symmetric block cipher using a 64-bit block size and 128-bit keys. It consists of eight rounds of bitwise XORs, addition module 2¹⁶, and multiplication modulo 2¹⁶ + 1; a “half-round” provides an output transformation and swap. Thus far there have been no published linear or algebraic weaknesses demonstrated in the IDEA design, and the only successful attacks thus far (in 2012) demonstrated an effective reduction in cryptographic strength of about two bits (roughly equivalent to reducing the key from 128 to 126 bits, or roughly from 3.4 × 10³⁸ to 8.5 × 10³⁷ as a measure of the time required to crack it). Practically speaking, this does not reduce the security of the IDEA algorithm in use, if properly implemented.

Part of that concern about proper implementation regards what are called weak keys, that is, keys that have long repeating stretches of 0 or 1 bits. While there’s still debate in cryptologist circles as to the real risks of weak keys, there are a number of practical mitigations—including using IDEA as part of a hybrid system that co-generates a one-time session key.

Although originally protected by a number of patents, IDEA became freely available as a reference design in 2011. MediaCrypt AG, in Zurich, Switzerland, has been working to make commercial versions of it, notably IDEA NXT, available and has been positioning it as the successor to IDEA. IDEA NXT supports block sizes of 64 or 128 bits and key sizes of 128 or 256 bits.

CAST

Carlisle Adams and Stafford Tavares created the CAST family of ciphers, with CAST-128 first made available in 1996. (Bruce Schneier has reported that although the name represents its creators’ initials, they meant it to “conjure up images of randomness,”as in the casting of the dice.) These are Feistel networks with large S-boxes that make use of what are called bent functions (the name suggests functions that are different from all others in Boolean logic), along with a mix of key-dependent rotations, modular arithmetic, and XOR operations. Entrust, Inc., of Dallas, Texas, holds patents on the CAST design procedure, but CAST-128 itself is available worldwide on a royalty-free basis for both commercial and noncommercial use. CAST-256 (also called CAST6) was published in 1998. As of 2012, the last published results show a theoretical cryptanalysis attack against a 28-round CAST-256 design, which would take 2^246.9 in time to complete; to date, no known successful attacks against CAST-256 have taken place.

PGP, OpenPGP, and GnuPG

These three cryptographic systems—for they are more than just an algorithm or a protocol—have their origins in the social and legal debate over whether private citizens should have both legal and ethical rights to use strong encryption in their private lives. The security services of national governments have long fought against this concept, as you can see in the history of many encryption algorithms and protocols to date. This social agenda is still felt by many to be vitally important today, when many national governments are using cloud-hosted data mining systems combined with mass surveillance to pursue their own agendas. Whether you agree with those agendas in whole or in part is not the issue; the underlying operational security needs and issues are.

The “PGP family” (if I can call these three related but separate systems that) implement an alternative to the hierarchy of trust that the Public Key Infrastructure (PKI) has provided us all. PKI and its certificate authentication system is a monoculture, a single ecosystem, the PGP family advocates rightly point out. As the mass market backbone of trust, it is all that we have. If it fails—if it can be corrupted or subverted by anyone—we are all at risk. An alternative is necessary for survival, these advocates claim. The PGP family implements nonhierarchical ways of asserting trust, managing public key exchanges, and providing for user storage and protection of their private keys. (See the “Web of Trust” section later in this chapter for more on these concepts.)

The good news about PGP et al. is that when used correctly it provides comparable levels of security to that provided by traditional PKI and its algorithms. The encryption algorithms used in PGP and its follow-on systems have proven exceptionally difficult to break; notorious examples include attempts by various law enforcement agencies to break PGP-encrypted emails and files. While some successful cracks by law enforcement have been reported, there seems to be some question as to whether they broke the encryption algorithms per se, brute-forced a weak password, or used side-channel attacks on the device in question (such as a BlackBerry or other smartphone) itself.

This gets to what may be the heart of the controversy. Opponents of PGP, OpenPGP, and GnuPGP state that these systems are technically challenging to use; they simply do not scale well into consumer-friendly products and service offerings. While PGP plugins are readily available for most email systems, this sidesteps the issues of certificate generation and management, certificate revocation, and user protection and use of private keys. Bruce Schneier referred to this as “Giving Up on PGP” in 2016¹ ; Thaddeus T. Grugq expresses this by proclaiming “I am here to liberate you from PGP” with its emphasis on knowing many “arcane obscure weird commands” in order to use it safely and effectively.² A more reasoned approach is posted at AT&T Security’s Alienvault site, in which the author (CryptoCypher, a student and intern no less) points out that if you do not understand your own operational security needs first, you may actually complicate your own risk posture by diving into PGP use.³ This advice no doubt applies equally to using mainstream encryption products and services.

While many in industry and research are advising users to “dump PGP,” there are also a growing number of voices to the contrary. It is clear that some elements in the marketplace recognize the risk of having all of one’s security eggs placed in one basket. Some are also looking at blockchain technologies as a smarter way to make webs of trust more scalable and useful. Some of these voices view the mainstream product offerings as focused too intensely on scaling into the corporate and enterprise user base, with little concern for how individual netizens need to ensure their own information security needs are met. Many of these netizens do see the CIANA+PS model in very different terms than the monoculture does after all.

Watch this space. There are good reasons why PGP, OpenPGP, and GPG haven’t gone away yet, in which case they may represent opportunities to be understood and seized.

A5/1, A5/2

A5/1 and its deliberately weakened stepchild A5/2 were developed in the late 1980s when some Western nations believed that their dominance of cell phone communications markets would remain unchanged for some time. The Berlin Wall had not fallen, yet, and worries about the Soviet Union still dominated government and international business thinking alike. The GSM cell phone system was becoming more widely used in Western Europe but was not considered for use outside of that marketplace at that point in time.

In that context, the arguments reported by security researcher Ross Anderson in 1994 make sense. NATO signals intelligence officials, according to Anderson, argued over whether GSM calls ought to be deliberately made easy to decrypt to plaintext voice or not. One outcome of this was the deliberate weakening of the A5/1 algorithm to produce A5/2 and then later A5/3 variants.

Without getting too much into the cryptanalytic details, it’s important to note that this family of ciphers has suffered many different types of attacks. Some of these, such as the Universities of Bochum and Kiel’s COPACABANA field programmable gate array (FPGA) cryptographic accelerator, or Karsten Nohl’s and Sascha Krißler’s use of 12 Nvidia GeForce graphics processors, are reminiscent of EFF’s hardware-based attacks on DES some years earlier.

Finally, you do have to consider that documents released to the public by Edward Snowden in 2013 show that the NSA can easily decrypt cell phone traffic encrypted with A5/1.

RC4

Rivest Cipher 4, developed by Ron Rivest of RSA Security in 1987, was initially protected as a trade secret by RSA Security; but in 1994 it was leaked anonymously to the Cipherpunks mailing list and soon became known widely across the Internet cryptanalysis communities (black hat and white). It was originally designed to support wireless encryption, applications in which its efficient, small implementation (hardware or software) and speed made it an ideal solution. It was an important element of the Wired Equivalent Privacy (WEP) algorithm for IEEE 802.11 wireless networking and then later in Wi-Fi™ Protected Access (WPA). Its use in SSL and then later in TLS ended in 2015, when its inherent weaknesses no longer allowed these protocols to be secure in any real sense. RC4 proved vulnerable to a number of attacks, many of which were enabled by lack of attention to detail by various implementations (such as those that did not use a strong message authentication code or tag).

RC4 is no longer recommended for use.

Salsa20/ChaCha20

These two closely related ciphers, designed by Daniel J. Bernstein, are proving remarkably secure and popular in hardware and software implementations. Not protected by patent, Bernstein has also released several different implementations optimized for common architectures to the public domain. Salsa20 was selected as a finalist in the EU’s ECRYPT competition, which is discussed later in the “EU ECRYPT” section. ChaCha20 has been adopted by Google, along with another of Bernstein’s ciphers (the Poly1305 message authentication code) as a replacement for RC4 in TLS; this strengthens Google’s port of Chrome, for example, into its Android operating systems for mobile devices. ChaCha20 has also been adopted as part of the arc4random random number generator, used in the NetBSD, OpenBSD, and FreeBSD implementations of Unix. Dragonfly BSD also uses ChaCha20, and from version 8.4 onward, the Linux kernel uses it as part of the /dev/urandom device.

In May 2015, the IETF published an implementation reference for ChaCha20 in RFC 7539, which did include some modifications to Bernstein’s original design. The RFC does point out these modifications may limit effectively secure use to a maximum message length of 256 GB (which we used to think was incredibly large!) and suggests that the original Bernstein design be used if this is insufficient, such as for full disk volume encryption. The IEFT has also proposed, in RFC 7634, the use of ChaCha20 as a standard for Internet Key Exchange (IKE) use and IPsec.

Forward Secrecy

By itself, any form of symmetric encryption by itself has always been vulnerable to revealing the content of encrypted messages or files once an attacker had broken the key. This would often force users to hope that when (not if) their keys were compromised or broken, they had already fully enjoyed the value of the secrets protected in their encrypted files or messages; those secrets could then be made publicly available at little or no harm or loss to their originators and users. More often than not, such hope was folly. On the other hand, if those messages had been encrypted with a true one-time pad and that pad was destroyed after use, then only the holder of the other copy of the pad could decrypt those messages, even if that was not until years later. Session keys, generated and used in hybrid encryption systems, provide such forward secrecy, so long as the session keys are destroyed immediately upon the end of the session. (Recall that if I send you a file via such an encrypted process and you store it—but don’t decrypt it—you can decrypt it later only if you still have that session key handy. The pseudorandom nature of the way in which good session keys get generated should prevent you or anyone else from decrypting the file without it.) Since a well-designed hybrid system distributes the session key using asymmetric encryption, then by definition there is nothing an eavesdropper can glean from intercepted traffic that can lead to being able to decrypt intercepted ciphertext, even if the originally used keys are later compromised.

A variation on forward secrecy known as weak forward secrecy describes the situation where the sender and recipient private keys are compromised. This does not affect the secrecy of the previously generated session keys, but it might allow an attacker to masquerade as the party whose keys have been compromised.

As with any cryptosystem, it takes a set of protocols to make the most of this asymmetric advantage, and we’ll look at these in the upcoming sections. First, however, let’s look at the basic math (and keep it somewhat basic as we do so). Then let’s look at some of the asymmetric algorithms in widespread use, in chronological order rather than any other.

Note the difficulty of using asymmetric encryption systems to protect archival copies of data at rest: at some point in this process, you have to separately store the session keys used in that encryption, or you’ll never be able to retrieve the data from the archive. To date, the best approach to this is investing in a hardware security module (HSM) and wisely using its multifactor, shared-secrets architecture to enforce your backup and restore teams to segregate their duties.

Discrete Logarithm Problems

The first broad category of trapdoor functions and the ones first described in the paper by Whitfield Diffie and Martin Hellman (who based their work on concepts developed by Ralph Merkle) capitalized on the problem of computing discrete logarithms. For any given number base b (such as 2, e, or 10), the logarithm is the value of the exponent that b must be raised to in order to produce the required result a. Functionally, that’s written as:

log_b(a) = x such that b^x = a

Except for a few examples, it is computationally difficult to compute discrete logarithms for any particular value a, when a is a member of a particular type of group or set. The details of how you choose such a set of values for a, b, and x, and how you put them to work in a cryptographic system, are all part of each protocol’s unique approach.

Diffie-Hellman-Merkle, Elliptical Curve, ElGamal, and the Digital Signature Algorithm are all examples of discrete logarithm-based trapdoor functions at work.

Factoring Problems

The second broad category of asymmetric algorithms involves factoring extremely large semiprime numbers. A semiprime number is the product of two prime numbers: 5 and 7 are prime, so 35 is a semi-prime; 3 and 3 are primes, so 9 is a semi-prime; and so on. Calculating a semiprime from two times is a simple, single multiplication. But given a large number such as 33,782, it takes a lot more CPU time and a more complex algorithm to determine whether it is a semiprime or not and, if it is, what its two prime factors are. (It’s not, by the way. But its nearest neighbor, 33,783, is.) When such candidate semiprime numbers have hundreds of digits, the amount of computer time necessary to test it (regardless of how efficient your numerical approach) can become astronomical; even with rooms full of today’s GPU-based bitcoin cryptocurrency mining systems, it can be no better than brute force or dumb luck to determine which two prime factors produce a given result. (In 2009, it was reported that factoring a 232-digit number required 1,500 processor years to complete; the large semiprimes in use today typically have in excess of 600 digits.) Thus, the two prime factors and the semiprime that results from them become elements in our encryption and decryption system.

Rivest-Shamir-Adleman (RSA) and LUC are examples of trapdoor algorithms based on factoring very large numbers involving semiprimes.

Diffie-Hellman-Merkle

It’s difficult to focus on Diffie-Hellman (or Diffie-Hellman-Merkle, to be precise) as an algorithm without putting it in the context of shared key generation, particularly between two parties who have no shared prior knowledge of each other nor any basis upon which to trust in a shared communications process with each other. Each party must somehow prove their identity—that is, demonstrate that they are who they assert that they are—before they can jointly authorize each other to participate in the session that’s about to take place. One important distinction must be recognized at the start: key exchange is not about exchanging secret information between the parties; rather, it is about creating a shared key to use for subsequent encrypted sharing of secrets. Furthermore, it’s important to realize that the “public” part of public key exchange is that you can quite literally publish parts of that key exchange without compromising the security of the encryption it supports. Whitfield Diffie and Martin Hellman, in a 1976 article published in IEEE Transactions on Information Theory, first showed that public key exchange requires the use of what they called trapdoor functions—a class of mathematical problems that are easy to do in one direction (like falling through the trapdoor in the floor) but extremely difficult if not impossible to do in the other direction. Their work leaned heavily on concepts developed by their friend Ralph Merkle, who was not a co-author on their paper; later, in 2002, Hellman suggested that their work be referred to using all three of their names. Twenty-one years later, the Government Communications Headquarters (GCHQ), the British counterpart to the National Security Agency in the United States, revealed that in 1969, three of their employees had developed similar concepts demonstrating the practicality of a public key cryptographic system. U.S. Patent 4,200,770 was issued in 1977, citing Diffie, Hellman, and Merkle as inventors, and describing the reference implementation of their algorithm and approach; it is now expired.

Diffie, Hellman, and Merkle described their system using what’s known as a multiplicative group of integers, modulo p, where the modulus p is a large prime number, and g, the generator value, is also another large prime. Choosing these values appropriately allows for a resulting shared secret (later known as the session key) to be in the range between 1 and p - 1. P and g are both publicly available. What happens next depends upon the two (or more) parties choosing other values, which will be used as exponents in computing the shared secret value.

Let’s start with a simple illustration. Suppose Bob and Carol want to establish their own encrypted Internet connection with each other. Here’s what happens:

1. Bob and Carol choose a suitable trapdoor function; they choose the key parameters that they will use. What they agree on can be shared in open, unsecured email with each other.
2. Carol chooses her private key and keeps it secret; she uses the trapdoor function to calculate her public key, which she sends to Bob. (Anyone can see her public key. More on this in a moment.) Bob, too, chooses a private key and uses the same trapdoor function to calculate his public key and sends that to Carol.
3. Carol applies the trapdoor function to Bob’s public key, using her own private key; call the result the session key. Carol keeps this secret; she doesn’t have to send it to Bob, and she shouldn’t!
4. Bob applies the same trapdoor function to Carol’s public key, using his own private key. This produces the same session key by the magic of the mathematics of the chosen trapdoor function. (The proof is left to the mathematically inclined reader.)
5. Carol and Bob now share a new secret, the session key. This key can be used with an appropriate (and agreed to) symmetric encryption algorithm so that Bob and Carol can exchange information with each other and keep others from being able to read it.

This is shown with small values for all numbers in Figure 5.11.

What about Eve, sitting along the sidelines of this conversation? Suppose Eve is, well, eavesdropping on Bob and Carol’s key exchange; she somehow is trapping packets going back and forth and recognizes that they’ve agreed to an algorithm and its control parameters; she recognizes the exchange of Bob’s and Carol’s public keys for what they are. As long as Eve does not have a secret key that participated in the computation of the session key, she does not have anything that lets her read the traffic that Bob and Carol encrypt with the session key. Eve is left to using brute-force, side channel, or other attacks to attempt to break the session encryption.

Ted, on the other hand, is someone Bob and Carol want to include in a three-way secure conversation (still keeping Eve out in the cold, of course). The process previously shown in steps 1 through 5 can easily be expanded to include three or more parties who share the choices about algorithms and parameters, who then compute their own public keys and share them; they then use everybody else’s public keys to compute their own copy of the session key.

Obviously, this simplified description of the Diffie-Hellman key exchange process has some vulnerabilities. It doesn’t actually authenticate that Bob is Bob, or Carol is Carol, thus tempting Ted to be the “man in the middle” who masquerades to be the other party from the initial handshake and key generation through to the end of the session. The choice of trapdoor function, and the control values for it, can also present exploitable vulnerabilities. But in its simplest form, this is where the public key infrastructure (PKI) got its start.

Building a public key infrastructure starts with the algorithms used to generate the shared secret keys used to establish trustworthy communications. Those algorithms have to be implemented in some combination of software and hardware, which are then made available to users to incorporate into their systems or use as standalone messaging apps. These apps themselves and the software and hardware distribution channels (wholesale, retail, original equipment manufacturer (OEM), or other) all have to be part of a network of trust relationships if two end users are going to trust in such apps to protect their communication with each other. Thus, the problem of building a public key infrastructure must also embrace the problem of trusted software (and hardware) distribution and update. This will be explored further in the “Understand Public Key Infrastructure Systems” section of this chapter.

RSA

Immediately after Diffie and Hellman published their article in 1976, two MIT computer scientists, Ron Rivest and Adi Shamir, teamed with MIT mathematician Leonard Adleman and set out to create a suitable trapdoor or one-way function for use in a public key exchange process. These three focused on both an algorithm (based on modular exponentiation) as well as on a process by which users could authenticate themselves, hence eliminating the risk of the man-in-the-middle attack. As is typical in the scientific and technical literature, they named the algorithm after themselves (thus the acronym RSA). The three authors founded RSA Security, Inc., in 1982, and MIT was granted a U.S. Patent in 1983 that used the RSA algorithm. Prior publication in 1973 by Clifford Cocks in the United Kingdom of very similar concepts precluded patenting RSA in other countries, and had that publication by Cocks been known, it would have invalidated even the U.S. patent (it was not disclosed by GCHQ until 1997). RSA later released the algorithm into the public domain in September 2000.

Like Diffie-Hellman, RSA uses the properties of modulo arithmetic applied to exponentiation of very large integers, where the modulus is also a very large prime number. Prior to the 1990s, the compute power needed to perform such operations (just to create the keys) was substantial, and the compute power necessary to break such algorithms was thought to be unaffordable by even the security services of major nation-states.

The founders of RSA did spend most of the 1980s and 1990s in what can only be called a pitched battle with the NSA and the White House. As this was during the heart of the Cold War and the Reagan-Bush defense buildup, it’s not surprising that the government saw any widespread use of powerful encryption by anybody as a threat to national security. (They still see that threat, particularly since “anybody” can be a terrorist, while in the same breath they know how our modern digital economy cannot function without widespread public use of highly secure encryption.) This history in and of itself is worth your time and study, as an SSCP and as a citizen, but it is beyond the scope of this book.

ElGamal

First described by Taher ElGamal in 1985, this asymmetric encryption algorithm is based on the mathematical theory of cyclic groups and the inherent difficulties in computing discrete logarithms in such groups. Borrowing from Diffie-Hellman-Merkle key exchange concepts, ElGamal provides for asymmetric encryption of keys previously used in symmetric encryption schemes. ElGamal also proposed a digital signature mechanism that allows third parties to confirm the authenticity of a message signed with it; this signature mechanism is not widely used today, but it did lead the NSA to develop its Digital Signature Algorithm (DSA) as part of the Digital Signature Standard (DSS). DSS was adopted as FIPS 186 in 1996, and it has undergone four revisions since then. (Don’t confuse DSA with ElGamal signature schemes.)

Some hybrid encryption systems use ElGamal to encrypt the symmetric keys used to encrypt message content. It is vulnerable to the chosen-ciphertext attack, in which the attacker somehow tricks or spoofs a legitimate user (an oracle) into decrypting an arbitrary message block and then sharing those results with the attacker. (Variations on this kind of attack were first known as lunchtime attacks since the user’s machine was assumed to be available while they were at lunch.) ElGamal does provide padding and other means to limit this vulnerability.

ElGamal encryption is used in the GNU Privacy Guard system, which we’ll look at in concert with PGP in the “Pretty Good Privacy” section.

Quantum Cryptography

Quantum mechanics has been a field of study and argument since 1905, when Albert Einstein published his now-famous paper on the photoelectric effect. One of the outcomes of that paper and the next few decades of incredible theoretical and practical research that flowed on from it was what some physicists called the “spooky side effects” of a universe made up of indivisible, small quanta or packets of force—packets that also behaved as if they were waves. Two of those side effects are at the heart of what may be the next revolution in computing and cryptography, based on quantum computing.

The first is what’s known as the observer effect, which means that many of the properties of a quantum (such as its position, velocity, spin, etc.) cannot be known unless you observe it, and since observation is a kind of interaction, your observation changes the state of the quantum. (This is similar to our definition in Chapter 4 of an event in your information system: if something has changed its value or state, that’s an indication that an event has occurred. The change of state is the relevant observation, whether you notice it or not.) Prior to observing such a quantum, or a system of quanta, its state is defined as being a set of probable outcomes all superimposed on each other. Observing the system collapses all of those probabilities into the right-now result. Quantum computing looks to have each of those probable outcomes be the outcome of a branch in a calculation; reading out that qubit (quantum bit) forces it to collapse into one result. Quantum computing may provide significant reductions in run time for massively parallel algorithms, which could give advantage both in computing new public and private keys, performing encryption and decryption, and in attacks against encryption systems.

The other spooky side effect is known as entanglement. Two particles created together are entangled, which means if you move one of them someplace else and then observe it, both it and its entangled twin will instantly take on the same state (the same spin, for example). Note that “instantly”—this change of state happens without any known message, energy, or force traveling the distance between the entangled twins. While that suggests science-fictional ideas such as faster-than-light communication, what it practically demonstrates is that an entangled quantum communications link is a tamper-proof way to ensure that no one has observed the content of your message or file once it was encrypted. In 2017, the Peoples’ Republic of China launched its Micius satellite, which demonstrated this and other “action at a distance” effects across its 1,200-kilometer space-to-ground links.

Quantum key distribution is, at this writing, the only use of these spooky side effects in practical cryptologic systems. These systems generate large quantities of entangled particle pairs that are stored in some fashion and used as a keystream or one-time pad. Any attempts to interrogate the stored copy of the keystream would invalidate it. A variety of commercial products provide such one-time pads, and several different protocols based on these quantum effects are in use. Some of these are positioned at the executive communications market, for example, as they arguably provide greater than normal communications confidentiality and integrity protection. https://en.wikipedia.org/wiki/Quantum_key_distribution provides a useful summary of work to date in this field.

In the meantime, researchers are working to develop algorithms that should be more resistant to quantum computing attacks. NIST plans to have proposals for post-quantum cryptography standards ready by 2025, and current estimates are that quantum computers capable of compromising today’s algorithms will not become available until 2030 at the earliest. The European Union is also doing work in this direction with its PQCRYPTO project, which interestingly enough asserts (on its home page) that while RSA and similar uses of discrete logarithms, finite fields, or elliptic curves offer rich alternatives now, “these systems are all broken as soon as large quantum computers are built…[and] society needs to be prepared for the consequences, including cryptographic attacks accelerated by these computers.” You might find “Post-quantum cryptography and dealing with the fallout of physics success,” by Daniel Bernstein and Tanja Lange, 2017, an illuminating read; it’s available at https://eprint.iacr.org/2017/314.pdf.

As costs come down through early adoption, and as hardware and systems capabilities continue to improve, you may find yourself dealing with spooky side effects on the job sooner than you might think. Watch this space…well, that space, please.

Hybrid Cryptosystems

Hybrid cryptosystems use multiple approaches to encrypt and decrypt the plaintext they are protecting. The most common hybrid systems are ones that combine asymmetric and symmetric algorithms. In most if not all of these systems, the cryptologic strength of the asymmetric algorithms are used to overcome potentially exploitable weaknesses in symmetric algorithms by means of computing a one-time session key, nonce, initialization vector, or other set of data that allows the far faster but weaker symmetric encryption process to handle the throughput needs of the systems’ users. Marrying the two provides two new approaches you’ll need to be familiar with.

• Key encapsulation processes, which are typically built with public key infrastructures (PKIs) to handle key exchange
• Data (or payload) encapsulation processes, which use more runtime-efficient symmetric-key algorithms

Most of the protocols we’ll look at use some variation of this approach. As we examine these, keep the OSI protocol stack in mind. Somewhere in that stack, the user, an application, or a lower-level service must be able to initiate a secure exchange with a host, negotiate with that host, control the secure session’s exchange of information, and then cleanly terminate the session. The protocols we’ll examine in some detail support these tasks.

Registered Email

Let’s consider one of the most common examples of the failure to provide reliable nonrepudiation—the use of typical email systems. Although email protocols provide ways for senders and recipients to exchange delivery and read receipts, these fail in nearly all circumstances to provide any proof that what one party claims was sent in an email was received and opened by the intended recipients. Within an organization (that is, when on a single, unified email server), delivery and read receipts are somewhat reliable, but no one relies on them as legally acceptable evidence or proof. It’s also trivially easy for senders or recipients to edit the email after it’s been sent or received, falsifying address, delivery, or content information in the process. Recipients can easily claim that they never received the email in question, and this lack of verified receipt and viewing of an email can give rise to deception or fraud.

Postal mail systems have long used registered and certified mail delivery processes to provide legally acceptable proof that a letter or package sent by one party to another was in fact delivered to the recipient and received by them. These processes require proof of identification of sender and recipient, and in the case of certified mail they record every step along the delivery path. Courts of law have long recognized that these processes, and similar ones offered by private document or package courier companies, provide acceptable evidence of delivery and receipt. Of course, the U.S. Postal Service cannot prove that the envelope containing the letter was opened or that the letter was read or understood by the addressee—but by denying the opportunity to claim “I never received that letter,” many contract disputes or simple misunderstandings can be quickly resolved.

There are several examples of commercial service providers who offer something conceptually similar to registered mail for email and e-documents. Many national postal authorities around the world have started to offer these “registered email” services to their individual, business, and government customers. The European Union set standards in place via the European Electronic Commerce Directive 2000/31/EC, for example, which specifies the technical standards such proof of receipt systems must meet so as to provide legally acceptable evidence of delivery and receipt. One of these systems, provided by RPost, uses multiple cryptographic techniques (many of which are patented) to provide these capabilities. The U.S. Department of Defense and other NATO nations have long used proprietary or closed government systems to ensure that when electronic messages are sent by one command to another, or to a subordinate unit, the recipient cannot ignore that message simply by claiming that “we never got that order.” These systems, too, make extensive use of cryptographic techniques. Key to these systems is that strong identity verification, authentication, and information integrity protection measures must work together.

Digital Signatures and Nonrepudiation

Digital signatures use cryptographic techniques to authenticate the identity of the file, its contents, its originator or sender, or any combination of these assertions. Several important elements of a digital signature infrastructure are needed in order to be able to make these assertions hold true over time.

• Authentication: Some process must be able to validate that the person or entity signing the file, document, or message is who the signature is associated with.
• Integrity: Some process must be able to confirm that the file, document, or message has not been altered—in any way—from the state it was in at the moment it was signed.
• Nonrepudiation: Once signed, some process must exist that can defeat any claim by the signer that they did not actually sign the file, message, or document; the system must deny the possibility of a forged signature being affixed to a document, file, or message. This is a one-way declaration of nonrepudiation by the originator; it does not specifically bind an instance of the message, document, or file, to the identity of a specific recipient.

Note that digital signatures, by themselves, are not responsible for assuring the confidentiality of the content of the message, document, or file, nor of the identity of the person or entity that signed it. Privacy, too, is not protected (for either the sender/originator or the recipient).

From a practical perspective, such a digital signature system should be scalable; it should not require significant effort to add new signatories to it or for recipients of files or messages to validate the signatures attached to them. As with key management, there should also be mechanisms to expire a signature in ways that do not invalidate documents signed prior to that expiration date. In many respects this is building an infrastructure of trust, by providing the mechanisms to assert trust between two parties, extend it from them to a third, and so on.

With the publication of Diffie-Hellman and the RSA algorithms, efforts accelerated to develop a workable infrastructure concept to support digital signatures.

The basic digital signature process involves the use of a highly secure hash function and asymmetric encryption in the following way. Suppose our friend Carol wants to send a message to Bob, but in doing so, she needs to prove to Bob that the message is inarguably from her and not from some imposter. Carol and Bob agree to use the following digital signature protocol:

1. Carol produces a strong hash of the message content. This is known as the secure message digest.
2. Carol “decrypts” that hash value, using the trapdoor function and her private key. This new value is her digital signature.
3. Carol sends the message and her digital signature to Bob.
4. Bob “encrypts” Carol’s digital signature, using the same trapdoor algorithm and Carol’s public signature, to produce the signed hash value.
5. Bob uses the same hash function to produce a comparison hash of the message he received (not including the signature). If this matches the value he computed in step 4, he has proven that Carol (who is the only one who knows her private key) is the only one who could have sent that message.

Again, please note that signing the message does not require Carol to encrypt its content, nor Bob to decrypt it. They can, of course, agree to use an encryption process (ideally a hybrid one relying upon their public and private keys) if so required. Note, too, that if a hostile third party such as Ted or Alice attempts to spoof Carol’s digital signature, they can succeed only if they already have Carol’s private key—or have hacked it via a brute force, burglary, or other attack mechanism. Note, too, that digitally signing a file for storage provides the same mix of integrity, authentication, and nonrepudiation protection. (Digitally signing an entire storage backup medium or file set, for example, provides a powerful assertion that the backup content is what the signature block asserts it to be.)

In many ways this process is independent of the algorithms used for producing the message hash, public and private keys, or the encryption and decryption used in this process. The root of this all, of course, is in how and where those public and private keys come from, and that bit of the infrastructure became associated with digital certificates as the vehicles by which trust is bestowed, recognized, distributed, and managed.

Digital signatures have widespread application in almost every area of systems integrity, availability, safety, and overall trustworthiness. Digital signatures are used in validating that a software update package or distribution kit is authentic or that a database backup set is from the system, date, and time it claims to be. Systems health checks, configuration management and control, and other change control processes can also digitally sign a system (at the level of a file directory or a whole disk volume) at the time the approved change is completed and routinely check by computing the digest value and comparing it to the signature to detect any changes.

All of this, of course, relies upon the sender or originator keeping their secret key secret and that there is some process as described in this section that can authenticate the validity of that originator’s public (and therefore their private) key.

Hashed Message Authentication Codes

Message authentication codes have long been used as a way of asserting that a particular message did in fact come from a properly authorized originator. Some messaging systems would sequentially number each message as part of formatting it for transmission, which when combined with sending date, time, originator ID, and other information, gave a reasonably unique and verifiable way to prevent attempts by third parties to spoof traffic into the system. Hashed message authentication codes (HMACs) perform this same function and were originally generated by noncryptographic hash functions. In use, HMACs are similar to generating a digital signature for the message contents. Senders and recipients then need to agree to a protocol for the exchange of an HMAC and its associated message content in ways that preserve message integrity and sender authenticity.

Depending upon the hash used to produce a digital signature, there may be exploitable weaknesses that leave the system open to attack. Replacing a standard hash with a cryptographic hash, using a strong cryptographic key, and using the HMAC algorithm allows systems to verify both the integrity of the message data and authenticate the sender of the message (or signer of the digital signature). HMAC first breaks the key into two halves (the inner and outer keys), then hashes the message first with the inner key, and then hashes that result with the outer key. This double-hash, using two different keys, leads to HMAC having greater immunity against length-extension attacks, which systems with only one pass through the hash function can be susceptible to. Similar to SHA-256 and other hash functions, HMAC breaks the message (or file) into fixed-length blocks and iteratively applies a compression function to them. It does not encrypt the message; as with all digital signature processes, the original version of the file or message used to generate the signature must accompany the signature so that the recipient (or later reader) can repeat the signature generation process and compare results to demonstrate authenticity.

Bellare, Canetti, and Krawczyk first published HMAC in a 1996 paper, and then in 1997 wrote RFC 2104 to bring it into the Internet community. It can make use of most any iterative hash function and is part of IPsec, TLS, and JSON Web Tokens. It was later included in FIPS 198 and in its later version FIPS 198-1, which specify the use of cryptographic hash functions to generate reliable message authentication codes for authenticating the information in the message.

Digital Signature Algorithm

The Digital Signature Algorithm (DSA) was first proposed by NIST in August 1991 as part of its Digital Signatures Standard (DSS) proposal. Although many software companies complained (having made significant investments in their own digital signature systems, many based on RSA’s cryptosystem), NIST moved ahead with its efforts and adopted DSA as Federal Information Processing Standard 186 in 1994. As of this writing there have been four revisions to DSA, the latest in 2013. DSA was patented by the U.S. government, which has made it available royalty-free worldwide.

DSA uses a combination of modular exponentiation and discrete logarithms to generate a digital signature given a public and private key pair. It supports full verification of authentic signatures and thus provides authentication, integrity, and nonrepudiation. DSA uses a four-step process.

• Key generation, which consists of parameter generation and then computation of a per-user pair of public and private keys based on those parameters
• Key distribution
• Signing
• Signature verification

The algorithm has demonstrated extreme sensitivity to the entropy, secrecy, and uniqueness of the random signature value k that is used during the signing process. Violating any one of these three cryptologic requirements, even by revealing just a few bits of k, can end up revealing the entire private key to an attacker. Experience with and analysis of DSA shows that typical cryptologic hygiene functions—not reusing a key value or a portion of one, for example—must be followed to keep the system secure. This vulnerability is also present in ECDSA, the elliptic curve variant of DSA, and was famously exploited in 2010 by the overflow group’s attack on Sony, which had been using ECDSA to sign software for its PlayStation 3 console. (Sony had failed to generate a new k for each pair of keys.)

DSA has been incorporated into multiple cryptographic libraries, such as Botan, Bouncy Castle, OpenSSL, and wolfCrypt.

Sometimes 256 May Not Be Greater Than 128

The search space argument suggests that it’s reasonable to expect a longer key will be more secure than a shorter one. All else being equal, the longer the key, the larger the key space and therefore the longer it will take to brute-force a key. But with AES, it turns out that all else is not equal. Owing to problems with the design of AES-256, Alex Biryukov, Orr Dunkelman, Nathan Keller, Dmitry Khovratovich, and Adi Shamir reported in 2009 that there is an attack on AES-256 that requires only 2¹¹⁹ time (compared with 2¹²⁸ time for AES-128, or the 2²⁵⁶ time one would expect from AES-256). Practically, this does not matter as a 2¹¹⁹ time attack is still completely infeasible using any technology available or likely to be available within the next decade or two. (Work that out in MIP-years to see why it’s infeasible for the moment.) The attacks analyzed by Biryukov et al. are of a type known as related-key attacks that are impractical in properly implemented systems. Biryukov et al. also found a weakness in AES-192, but that attack takes 2¹⁷⁶, not 2¹⁹² as it ought to, if AES-192 had no flaws, but that’s still much better than AES-128.

These and similar cryptanalyst findings demonstrate that as cryptographic algorithms become more complex, a measure of their strength (or weakness) needs to consider other factors than processing needs alone. The size of a data space associated with part of an algorithm, the number of rounds used, or other parameters may have a marked effect on the strength of an algorithm.

Cryptographic Safety Factor

Other cryptanalysts have also struggled with the difficulty in making a meaningful estimate of the strength of a cryptographic algorithm. One approach was taken in the selection process for the algorithm that became the Advanced Encryption Standard; for this, cryptographer Eli Biham introduced the concept of a safety factor.

Biham’s safety factor notes that all modern ciphers are built as a series of rounds, each using a subkey derived from the main key. Cryptographers typically attempt to break ciphers by first attacking a simplified version of the cipher with a reduced number of rounds. For example, early cryptographic attacks on DES (before it fell to simple brute-force) revealed an attack on eight rounds (the full DES has 16 rounds). With AES-256, there is an attack that works on a simplified version of 10 rounds (the full AES has 14 rounds). This was developed after attacks on six-, seven-, and nine-round versions.

Biham’s safety factor is the ratio of the number of rounds in the cipher, divided by the largest number of rounds that have been successfully attacked so far. While obviously dependent on the level of effort expended by cryptographers trying to undermine a cipher, it is still a useful metric, at least when comparing ciphers that have received sufficient attention from cryptographers.

Using this measure, AES-256 currently has a safety factor of 1.4. Other ciphers, such as Twofish, have greater safety factors, and it was for this reason that the team that developed the Twofish algorithm argued that it ought to have been selected as the Advanced Encryption Standard instead of the algorithm (Rijndael) that was selected.

In a somewhat tongue-in-cheek fashion, Lenstra, Kleinjung and Thomé have suggested that another, and possibly more useful, approach, would be to estimate the amount of energy (as heat) that each such attack process needs. In their paper titled “Universal Security, from bits and mips to pools, lakes, and beyond,” they focus on how difficult it really is to usefully predict (or model) the strength of an encryption process in terms that are comparable to non-encryption-based ideas of work and time. For more food for thought you can head to https://eprint.iacr.org/2013/635.pdf.

The sad news is, however, that no matter what algorithm you choose and no matter how carefully you choose its cryptovariables, you are still in a race against time. Processing power in the hands of the attackers will continue to increase, especially as they become more adept at sharing the load around in their dark web bastion areas. You can stay a few steps ahead of them, however, if you are diligent in keeping your systems and their cryptologic components properly maintained and updated and operate them in ways that do not inadvertently lead to compromising them.

A Starter Set of Crypto-Hygiene Practices

Let’s take a look at some starting points for putting a cryptologic hygiene program into action. There are any number of standards documents that address aspects of this hygiene process, many of which address information security processes across the board. Others are very intensely focused on what might be called the high-capacity, high-demand cryptologic user base, such as what you’d expect in a major financial institution, a credit card payment processor, or even at a certificate authority. The following rules of thumb should give you a simpler place to start, which may be enough to address the needs of a small office/home office (SOHO) or small to medium-sized organization infrastructure.

DIY Need Not Apply An important rule in cryptographic hygiene is to never invent your own cryptographic algorithm, process, or system, even if you think it’s absolutely necessary to do so. Although technology professionals are generally smart people, it is entirely too common that we become overconfident in our abilities. In 1864, famed cryptographer Charles Babbage wrote, “One of the most singular characteristics of the art of deciphering is the strong conviction possessed by every person, even moderately acquainted with it, that he is able to construct a cipher which nobody else can decipher.” This leads to a false sense of security. Simply because you have invented a system that you cannot defeat does not mean it cannot (or will not) be defeated by someone else.

Know Your Baselines and How to Maintain Them You need to know what you’ve already got and how to use it. Your systems will no doubt already have significant cryptologic components in them. Anything with a modern web browser installed on it, for example, already brings you the capabilities to manage the use of encryption suites, digital certificates, digital signatures, and other encryption-based security measures. As with any aspect of your baseline, start with a very detailed inventory; discover and catalog every element. Then make sure you have current vendor-supplied operational and maintenance information on hand, or at least know where to get it. Systems vendors, platform providers, and even your browser provider offer substantial libraries of how-to information—from simple procedures through in-depth knowledge base articles—which you should be familiar with. Use these support pages to identify active community support boards; join in the learning by joining the conversation.

Manage Your Baselines You also need to put your cryptographic infrastructure under solid configuration management and control. Plan and schedule changes, whether those are certificate renewals, minor patches and updates, or major upgrades. Identify the human elements associated with each cryptographic configuration item (CCI, to coin an acronym); these are opportunities to keep end users engaged with security by building their awareness and their skills.

Stay Sharp It is crucial to stay current. Knowing and managing your baselines should point you to the communities of practice that directly apply to the systems and technologies your organization depends upon. Find the places you need to frequent to stay informed about new exploits or attacks or cryptanalysis results that suggest a vulnerability might exist in your systems.

Protect Across the Life Cycle It is also essential to practice good housekeeping. Your systems’ baseline and configuration management processes should identify components of your systems that need special end-of-life treatment, so as not to compromise your cryptographic defenses. You may have little or no direct involvement in key management or the destruction of expired, revoked, or compromised keying materials, if you’re like the majority of business and nonprofit organizations and are totally dependent upon commodity product systems, such as servers, workstations, and endpoints, and their built-in hybrid encryption systems. Nonetheless, it’s good to have your hardware, software, firmware, and data baselines clearly identify cryptographically sensitive items that may need special protection while in use, when undergoing repairs, or when being disposed of.

If You Can’t Update, Increase Your Watchfulness Think about the CVE process and overlay its timeline onto the kill chain conceptual model. Chances are that many exploiters in the wild will learn about that vulnerability the same way you will, by means of notifications from the CVE database operators. Three time windows start clocking out starting on “zero day”—the time to develop a fix or patch, the time for you to validate that you can safely install the fix without causing more disruption to business processes, and the time for attackers to take advantage of that vulnerability and get into your systems. It’s not hard to guess which one of those time windows runs out the fastest.

Prudent software and systems management dictates that you test and evaluate any change to your systems before putting it into production or operational use, no matter how urgent and compelling the need to update may seem to be. Security hygiene—and therefore cryptologic hygiene—dictates that between the time you have the fix in hand and the time you’ve finished testing it and declaring it the “new normal” that you increase your watchfulness. The larger your deployed architecture and the greater the mix of systems, versions, and endpoint technologies employed with it, the harder it becomes to baseline it and manage to that baseline. You should be doing this alone: the IT departments and the formal configuration management and control teams in your organization own the lion’s share of this burden. Your role should focus on the cryptologic aspects of that architecture, particularly the procedural ones, to make sure that someone else’s major change package doesn’t demolish the crypto-walls that are relied upon today.

Your first response to seeing a new CVE announcement should be to ask yourself, “How would I spot that new exploit in action against my systems?” Think like the adversary; red team your current surveillance and monitoring processes and see whether you can detect this new exploit in action. Any new exploits you cannot detect, which might remotely allow an intruder into your systems, are crying out for changing something in your monitoring, analysis, detection, and reporting operations.

As you continue to apply a cryptographic hygiene mindset, you’ll probably discover more rules to put into your cryptographic process maturity handbook. One way to do that is to look at common attack patterns; along the way, let’s also look at some “famous fails” when it comes to design, implementation, and use cases gone wrong.

Cryptography Is Not a Standalone Answer

This may be a blinding flash of the obvious, but it’s part of counteracting the “sprinkle a little crypto dust” fallacy, so you may have to keep this rule handy and use it often. For example, consider the private keys on your client endpoint systems. Where are they kept? How do you keep them safe and secure?

If you’re working in a SOHO system or any systems infrastructure where you do not buy and manage your own certificates (and thereby generate and manage your users’ private and public keys on your own), then you are vitally dependent upon the access control systems and policies on your in-house file servers, database servers, web servers, network control systems, and each and every one of your client endpoint devices. That means that anybody with superuser or systems administrative privileges, who has physical or logical access to any of these machines, has access to where its host operating system stores its certificates, public keys, private keys, and the tables that OS uses to associate each user identity with its keys (and possibly the password salts, hashed passwords, and other encryption-related parameters). Because we need systems that are easy to administer, it’s easy for such a user to call up the right built-in utility functions and export those files.

So how do you stop a lunchtime attack from purloining a copy of those key caches?

Administratively, you teach your users what the risks might be and how each of them is a vital component in the overall security system. You motivate them to become true believers in the idle-lock settings; you work with them to eliminate the yellow stickies with the password cribs. You get them to be suspicious of the pizza delivery man wandering the hallways (one of my favorite force protection drills, after I received an unordered pizza box with a “boom!” note inside it).

Physically, you identify the right mix of protection mechanisms that keep both your on-site and your mobile devices protected; this may include mobile device managers and their capabilities to lock or brick such a device if it wanders off site.

Logically, you thoroughly understand what you need in the way of access control capabilities; use what’s built into your OS and servers for all that it’s worth. Understand what to log, and what to look for in those logs, as it might relate to attempted attacks on your key stores and caches.

Without every other trick in your information security playbook working like clockwork for you, you cannot protect the number-one set of secret ingredients in your competitive advantage: your users’ and your company’s private keys, and the cryptovariables, seeds, and salts that go with them.

The “Understand Public Key Infrastructure Systems” section in this chapter will go into more depth and detail on key and certificate management and the cryptologic hygiene measures to consider. This is enough, for now, to get you seeing your systems from the black hats’ perspective, cryptologically speaking.

Attacks Against the Human Element

The people in your organization are perhaps its greatest strength and its greatest opportunity for exploitation by a determined adversary. Too many businesses have suffered when an insider becomes susceptible to coercion, blackmail, or other “undue influences,” as they say. Some have had employees kidnapped and pressured or otherwise influenced into revealing information that allows an attacker to defeat otherwise impregnable information defenses. Technical exploitation of lost or stolen endpoint devices is becoming commonplace. Burglary and other clandestine means of entering a facility to gain information are risks that should not be ignored.

You’ve addressed those risks to your organization with various physical, logical, and administrative mitigations and security controls. Be sure to extend that to the cryptologic elements of your systems. You may be safe in relying on system and data archives protected by encryption, if you’ve used strong hybrid systems to provide you assurances of the forward secrecy of those archives; but to read and use those archives when you need them, you’ve got to have those session keys somewhere. How do you protect them against the exploitable vulnerabilities in the human elements of your systems?

Algorithm Attacks

These are attack patterns that depend upon intercepting and collecting quantities of ciphertext and plaintext and then analyzing them to look for patterns. Traffic between multiple senders and recipients may reveal coincidental changes in message length or frequency when correlated to other observable activities, for example. Four main types of attack fall into this category.

• Ciphertext-only attacks occur when the attacker has access only to the encrypted traffic (ciphertext). In many cases, some information about the plaintext can be guessed (such as the language of the message, which can lead to knowledge of the character probability distribution or the format of the message, which can give clues to parts of the plaintext). Wired Equivalent Privacy (WEP), the original security algorithm for Wi-Fi™, is vulnerable to a number of ciphertext-only attacks. By capturing a sufficient number of packets (which typically can be gathered within minutes on a busy network), it is possible to derive the key used in the RC4 stream cipher. It is thought that the 45 million credit cards purloined from the American retail giant T.J. Maxx were obtained by exploiting WEP.
• Known-plaintext attacks can happen when the attacker knows some or all of the plaintext of one or more messages (as well as the ciphertext). This frequently happens when parts of the message tend to be fixed (such as protocol headers or other relatively invariant parts of the messages being communicated). An example of a known-plaintext attack is the famous German Enigma cipher machine, which was cracked in large part by relying upon known plaintexts. Many messages contained the same word in the same place or contained the same text (e.g., “Nothing to report.”), making deciphering the messages possible.
• Chosen-plaintext attacks see the attacker is able to inject any plaintext the attacker chooses into the target’s communications systems (or a copy of them) and thereby obtain the corresponding ciphertext. The classic example of a chosen-plaintext attack occurred during WWII when the United States intercepted messages indicating the Japanese were planning an attack on a location known as “AF” in code. The United States suspected this might be Midway Island, and to confirm their hypothesis, the United States arranged for a plaintext message to be sent from Midway Island indicating that the island’s water purification plant had broken down. When the Japanese intercepted the message and then transmitted a coded message referring to AF, the United States had the confirmation they needed.
• Chosen-ciphertext attacks occur when the attacker is able to inject any ciphertext into the target’s communications systems (or a copy of them) and thereby obtain the corresponding plaintext. An example of this was the attack on SSL 3.0 developed by Bleichenbacher of Bell Labs that could obtain the RSA private key of a website after trying between 300,000 and 2 million chosen ciphertexts.

As you might expect, different cryptographic systems and their algorithms have differing levels of vulnerability to these types of attacks.

Brute Force

A brute-force attack is one that simply steps through all possible values until one of them works (requiring little thought, planning, reconnaissance, or analysis). Typically, this involves building (or mathematically generating) a search space of all possible values, such as 2⁶⁴ binary numbers for a 64-bit key space. If luck is with you (the defender), your key or hash value won’t match in the first few (days or weeks) of an attacker’s search of that space. Then again, luck may favor the attacker. Brute-force attacks against password hashes, ciphertext, digital signatures, or almost any element of your systems are possible. Brute-force attacks are commonly targeted against intercepted individual passwords or against exfiltrated password hash tables. Cryptanalytic attacks may also resort to brute-force attacks against ciphertext.

WEP, as a case study, reveals the value of a well-placed brute-force attack. Wired Equivalent Privacy (WEP) was the first approach to encryption used for Wi-Fi™ networks. One of its major design flaws was its reliance on the RC4 encryption algorithm, which was quickly shown to be susceptible to brute-force attacks. RC4 did not have an initialization vector built into its algorithm, so the WEP designers chose to use a 24-bit IV combined with the symmetric encryption key to feed to the keystream generator. Such a short IV repeats far too quickly; in typical usage, less than three minutes of Wi-Fi™ packet sniffing would provide more than enough samples to use to attack and break WEP’s encryption as a result. (As a result, WEP should not be used, especially when the stronger WPA and WPA-2 are available.)

Modified or optimized brute-force attacks are ones in which a little knowledge on the attacker’s part is used to drastically reduce their search time. This can involve any number of techniques, such as precomputing a dictionary or rainbow table.

Countermeasures to brute-force attacks include strong password and passphrase hygiene, using properly updated cryptographic hash, encryption and decryption suites, and effective, operational security procedures.

Man-in-the-Middle Attack

Man-in-the-middle (MiTM) attacks show up at almost every layer of the OSI protocol stack—including layer 8, the human-to-human layer. When they involve the attacker (imposter) attempting to sidestep, subvert, or otherwise attack the cryptographic processes used by the parties, it’s worth considering them as a cryptologic attack. For example, a MiTM attacker (Mallory, let’s say) attempts to insert herself into a conversation between Bob and Carol, who are using HTTPS as part of their processes. Bob and Carol are relying upon the digital certificates each provides to the other (via their browsers’ use of them and their built-in encryption suites) to confirm Bob’s assertion that he is in fact Bob, and Carol’s that she actually is Carol. However, if one or more of these certificates were issues by a compromised certificate authority (such as the Dutch CA DigiNotar, which was compromised in 2011), Bob and Carol might believe they are talking to each other, when in fact they are talking through Mallory. (The DigiNotar compromise affected more than 300,000 Gmail users.)

Side-Channel Attacks

Side-channel attacks involve measuring observable characteristics of the cryptographic process to deduce information to assist with compromising encrypted information. This usually requires the attacker to have a copy of the system being attacked and the ability to open it up, capture internal or intermediate data, make measurements, or even take infrared high-speed images to observe changes in heat signatures. These can include the following:

• Timing
• Cache access
• Power consumption
• Electromagnetic emanations
• Error information

The time taken to encrypt or decrypt a block of data can vary depending on the key or plaintext, and a careful analysis of the time taken to encrypt or decrypt the data can reveal information. The time to perform a cryptographic operation can vary for a number of reasons.

• Conditional branches within the code, which can change the time of execution depending on the branches taken, which in turn depend on the value of the key or plaintext
• CPU instructions that take variable time to complete depending on the operands (e.g., multiplication and division)
• Memory access, which can vary depending on where the data is located (type of memory) or the access history (thus affecting the cache and thus the speed of memory access)

Cache attacks typically involve processes running on different virtual machines on the same physical processor. As the VM performing the encryption is time-sliced with the VM running the attacker’s processes, the attacker can probe the processor’s cache to deduce information about the plaintext and the key and thus compromise the encryption process. A cache-timing attack was at the heart of the Spectre and Meltdown attacks revealed in 2018 as methods of extracting data from protected regions of a processor’s memory (e.g., keys or plaintext messages).

The power consumed by the device performing the cryptographic operation may vary depending on the instructions executed, which in turn depend on the key and data being encrypted. By carefully monitoring the power consumed by the device, it can sometimes be possible to extract information about the key or plaintext. This type of attack has been most successfully demonstrated against smartcards because of the relative ease with which the device’s power consumption can be monitored, but the attack mechanism has wide applicability.

All electronic systems emit electromagnetic radiation, and it is possible to capture this, sometimes at some distance from the device. These radio signals can sometimes be analyzed to reveal information about the data being processed by the device. Early examples of this type of attack involved analyzing the emanations of cryptographic devices that printed the decrypted message on teletypewriters to determine which characters were being printed.

Error information provided (or leaked) by decryption software can provide useful information for attackers. In the Padding Oracle Attack, a system that can be sent any number of test messages and that generates a distinctive error for encrypted messages that are not properly padded can be used to decrypt messages without knowing the key. The defense is to report generic errors and not to distinguish between padding errors and other errors.

Countermeasures exist, but in some cases, they can be difficult to implement or can exact a considerable performance penalty. In the case of timing attacks, it is necessary to modify the algorithm so that it is isochronous, which is to say runs in constant time regardless of the key and data being processed.

The difficulty of implementing a secure algorithm that is secure against side-channel attacks is another reason for the “DIY need not apply” edict. Do not attempt to d write your own cryptographic implementation—use a tested and widely used cryptographic library instead.

Differential Fault Analysis

Differential fault analysis is a cryptographic attack in which faults are induced in the circuitry performing the cryptographic operation in the expectation that the device can be forced to reveal information on its internal state that can be used to deduce the key.

For example, in 2004 Christophe Giraud published an attack on AES-128 implemented on a smart card. By using a Xenon strobe and removing the cover of the smart card processor, his team was able to induce faults in the execution of the algorithm that enabled them, after multiple induced faults, to derive enough information to determine the full key value.

Birthday Attack

A birthday attack is a method of compromising cryptographic hashes, such as those used for digital signatures. The name is derived from the observation that while the odds that anyone in a group of 23 people has a specific date as their birthday is 23 out of 365, or 6 percent, the odds that there are two people in the group of 23 with the same birthday are 50 percent. Thus, any seemingly rare event might not be so rare when considering multiple possible occurrences.

A birthday attack against a digitally signed document would attempt to create a bogus document that somehow generates the same hash value as the original does, which would be a hash collision. Cryptographic hash functions are required to be free from collisions, and algorithm designers go to great length to design to prevent this and then test to verify their success. A birthday attack might attempt to have any type of file (a document or an executable binary) be mistaken as genuine.

The choice of a provably collision-free cryptographic hash is your best countermeasure for such attacks.

Related-Key Attack

A related-key attack is a form of known-ciphertext attack in which the attacker is able to observe the encryption of the same block of data using two keys, neither of which is known to the attacker but that have a known mathematical relationship.

While it is rare that the attacker can arrange for two mathematically related keys to be used, poor implementations of cryptography can lead to keys being generated that have a known relationship that can be exploited by the attacker. Short, poorly chosen keys can make it easier for attackers to identify possible related-key pairs; this was another means of successfully attacking WEP, for example.

Meet-in-the-Middle Attack

A meet-in-the-middle is a known-plaintext attack against block ciphers that perform two or more rounds of encryption. One might think double (or triple) encryption would increase the security in proportion to the combined length of the keys. However, the math behind this doesn’t support the increased complexity.

By creating separate lookup tables that capture the intermediate result (that is, after each round of encryption) of the possible combinations of both plaintext and ciphertext, it is possible to find matches between values that will limit the possible keys that must be tested in order to find the correct key. This attack, which first came to prominence when it was used to defeat the 2DES algorithm, can be applied to a broad range of ciphers.

Unfortunately, chaining together cryptography algorithms by running them multiple times does not add as much additional security as one might think. With 2DES, the total impact of the second run was to increase complexity from 2⁵⁶ to 2⁵⁷. This has a small overall impact compared to the work done.

Effective countermeasures may include choosing a stronger algorithm that uses longer keys and of course choosing those keys wisely. If this is not an option, your current algorithms may produce somewhat greater security by increasing the number of rounds they execute. 3DES, for example, does not get you all the way to a 168-bit key strength—but its 112-bit equivalent strength is much greater than that provided by 56-bit “single” DES.

Replay Attack

A replay attack is one in which the attacker does not decrypt the message. Instead, the attacker merely sends the same encrypted message. The attacker hopes that the receiving system will assume that the message originated with the authorized party because the information was encrypted.

Replay attacks can involve user or subject access request credentials, transaction replay, or almost any step or series of steps in one of your business processes. (A rather stealthy replay is the invoice replay, in which a “vendor” repeatedly bills a larger firm for the same small purchase, with the purchase amount carefully chosen to not trigger human authorizations.) Most of these are not cryptographic attacks, per se, but might need to bypass or sidestep parts of your cryptographic defenses. Replay attacks are also possible against systems that require multiple-factor authentication, such as online banking systems that require a separate authentication step as part of some transaction requests.

Most countermeasures to replay attacks via encrypted traffic involve using session keys, cryptographic nonces, timestamps, block sequence identifiers, or other means to prevent any two blocks or messages from being bit-for-bit identical to each other within the same session. (Since a different session key should be used for each session, the replay of blocks from one session should be rejected by a subsequent session.) Each of these tactics, of course, will have its own false positive and false negative error rates to contend with.

Cryptanalytic Attacks

These types of attacks tend to be more theoretical in nature, as they often require more knowledge and skill with the underlying mathematics used by the algorithms and systems being analyzed. In most cases, there are few countermeasures available to end users or administrators, other than selection of a different algorithm or process.

Trusted Platform Module

A Trusted Platform Module (TPM) is a separate processor that provides secure storage and cryptographic services as specified by ISO/IEC 11889. A TPM can be used by the operating system, processor BIOS, or application (if the OS provides access to the TPM) to provide a number of cryptographic and security services.

• Generate private/public key pairs such that the private key never leaves the TPM in plaintext, substantially increasing the security related to the private key. (Public/private keys are discussed later in this chapter.)
• Digitally sign data using a private key that is stored on the TPM and that never leaves the confines of the TPM, significantly decreasing the possibility that the key can become known by an attacker and used to forge identities and launch man-in-the-middle attacks. (Digital signatures are discussed later in this chapter.)
• Encrypt data such that it can only be decrypted using the same TPM.
• Verify the state of the machine the TPM is installed on to detect certain forms of tampering (i.e., with the BIOS).

The Private Endorsement Key is a fundamental component of a TPM’s security. This key is generated by the TPM manufacturer and burned into the TPM hardware during the manufacturing process. Because of this, the user/system owner depends upon the security of the TPM manufacturer to ensure that the PEK remains confidential.

We also depend on the quality of the TPM manufacturer’s processes. In 2017 it was revealed that a defect in the software library used by Infineon for its line of smartcards and TPMs contained a flaw that made it possible to deduce the private key stored internally. As a result, there were millions of cryptographic keys made unreliable and vulnerable. Attackers were able to calculate the private portion of an account holder’s key from having access to only the public portion. What happened, unfortunately, is that hackers impersonated legitimate users with the assurance and nonrepudiation provided by having their private keys.

Cryptographic Module

A cryptographic module is typically a hardware device that implements key generation and other cryptographic functions and is embedded in a larger system.

The advantages of using a cryptographic module as opposed to obtaining the equivalent functionality from a cryptographic software library include the following:

• By performing critical cryptographic functions on a separate device that is dedicated to that purpose, it is much harder for malware or other software-based attacks to compromise the security of the cryptographic operation.
• By isolating security-sensitive functionality in an isolated device with limited interfaces and attack surfaces, it is easier to provide assurances about the secure operation of the device. It also makes it easier to provide secure functions to larger systems by embedding a cryptographic module within the larger system.
• Increased availability of noncryptographic dedicated resources.
• Most secure cryptographic modules contain physical security protections including tamper resistance and tamper detection, making it difficult to compromise the security of the device even if the device has been physically compromised.
• Some cryptographic modules can enforce separation of duties so that certain sensitive operations such as manipulating key storage can be done only with the cooperation of two different individuals who authenticate to the cryptographic module separately.

Some government organizations have issued standards related to the security of cryptographic modules and have established evaluation and certification processes so that manufacturers can have the security of their devices validated by an independent third party and users can have confidence in the security that using the module will provide their larger system.

For example, the U.S. government’s FIPS 140-2, “Security Requirements for Cryptographic Modules,” specifies the requirements for cryptographic hardware and software to meet four different levels of security. It also provides for certification of products to validate they meet the requirements.

Internationally, the “Common Criteria for Information Technology Security Evaluation,” documented in the ISO/IEC 15408 standard, provides an alternate set of requirements and certification processes to validate information security products.

Hardware Security Module

A hardware security module is a physically separate device used to safely store and protect various sets of information, which may be cryptographic keys or other highly sensitive mission-critical data. They are typically designed to be tamper-resistant and visibly show (or alarm) if tampering has been attempted. Data centers, especially those serving financial institutions, may have HSMs clustered or racked together with redundant, separately serviceable power, cooling, and network access. HSMs can perform a variety of services via APIs to other network systems, such as:

• Onboard secure cryptographic key generation.
• Onboard secure cryptographic key storage, especially for highest-level or master keys.
• Key management.
• Encryption and digital signature services.
• Transparent Data Encryption (TDE) key generation and management. TDE is a database encryption technology used by Oracle, Microsoft, IBM, and others for protecting data at rest, at the file level.

HSMs can also offload asymmetric encryption and decryption from application servers; when specialized for this role, they may be referred to as cryptographic accelerators. While they may not achieve throughput rates that can match dedicated hardware encryption and decryption systems, they have been able to perform as many as 10,000 1024-bit RSA signs per second. To help meet NIST’s recommendation, in 2010, to move to 2,048-bit keys for RSA use, there has been increasing emphasis in the HSM market to provide greater performance capability, as well as providing elliptic curve cryptographic support.

Some HSMs have the capability to host a full operating system, hypervisor, and suite of virtual machines within their physically and logically protected enclosure. This provides an enhanced degree of protection for an entire applications stack (the set of all components from the operating system on up through all layers of applications logic). Use cases that require this, particularly ones requiring a simple and reliable means to validate full stack integrity, can make use of HSM-provided remote attestation features.

Certificate authorities and registration authorities make extensive use of HSMs to have reliable, secure, high-performance capabilities to generate, store, manage, and control distribution of asymmetric key pairs and certificates. These applications call for HSMs to have very strong physical and logical security protection built in, quite often involving multipart user authentication such as the Blakley-Shamir secret sharing schema. Full audit and logging of all activities and secure key backup and recovery capabilities are must-have features in this application.

Banking applications often use specialized HSMs, using a nonstandard API, to support specific security requirements in the payment card industry. Protocols and processes involved with verifying a user’s PINs, controlling ATMs and point of sales terminals, verifying card transaction attempts, generating and validating card magnetic stripe data and other card-related security codes, and supporting smart card transactions are some of the unique needs that HSMs need to support.

Domain name registrars are making increasing use of HSMs to store and protect information, such as to store key material used to sign zonefiles. The OpenDNSSEC suite supports the use of an HSM in this.

On a smaller note, HSMs are also available as a hardware cryptocurrency wallet.

IPsec

Internet Protocol Security (IPsec) makes extensive use of asymmetric encryption algorithms (packaged as encryption suites), key exchange infrastructures, digital certificates, and bulk encryption protocols to deliver the security services that the original Internet Protocol could not. It was developed during the late 1980s and early 1990s to provide Internet layer (level 3) security functions, specifically the authentication and encryption of packets as they are transferred around the Internet. It needed to provide a variety of security benefits: peer authentication, sender (data origination) authentication, data integrity and confidentiality, and protection against replay attacks. IPsec can provide these services automatically, without needing application layer interaction or setup. IPsec introduced several new protocols to support these services.

• Authentication headers (AH) achieve connectionless data integrity for IP datagrams and support data origin authentication. These allow recipients to validate that messages have not been altered in transit and provides a degree of protection against replay attacks.
• Encapsulating security payloads (ESP) also provide a partial sequence integrity mechanism, which also adds to anti-replay protection. This also provides a limited degree of traffic flow confidentiality, while protecting the confidentiality and integrity of datagram content.
• Internet Security Association and Key Management Protocol (ISAKAMP) offers an overall process and framework for key exchange and authentication of parties. Keying material itself can be provided by manually configuring pre-shared keys or using the Internet Key Exchange protocols (IKE and IKEv2). Kerberized Internet Negotiation of Keys (KINK) or the IPSECKEY DNS record can also be used as ways for parties to exchange and manage keys.

IPsec also provides the mechanisms to form security associations (SA) that are created and used by two (or more) cooperating systems agreeing to sets of encryption suites, encryption and authentication keys, and all of the cryptovariables that go with that (such as key lifetimes). This allows those internetworked systems to then implement AH, ESP, or both, to meet their agreed-to needs. Implementing AH and ESP would require multiple SAs be created so that the sequence of steps be clearly established and well-regulated to provide the required security.

As you might expect, the list of current cryptographic algorithm suites that are supported by IPsec or recommended for use with it changes as new exploits or cryptanalytic attacks make such changes advisable.

IPsec provides two methods of operation, known as transport mode and tunnel mode.

• Transport mode encrypts only the payload (data content) of the IP packets being sent, which leaves all of the routing information intact. However, when transport mode uses the IPsec authentication header, services like NAT cannot operate because this will invalidate the hash value associated with the header and the routing information in it.
• Tunnel mode encrypts the entire IP packet, routing headers and all; it then encapsulates that encrypted payload into a new IP packet, with a new header. This can be used to build virtual private networks (VPNs) and can also be used for private host-to-host chat functions. Since the “as-built” packets from the sending system are encrypted and encapsulated for actual transmission through the network, any packet-centric services such as NAT can function correctly.

IPsec can be implemented in three different ways. It’s normally built right into the operating system by including its functions within the IP stack (the set of operating systems service routines that implement the Internet Protocol in that environment). When such modification of the operating system is not desired, IPsec can be implemented as a separate set of functions that sit (in effect) between the device drivers and the operating system’s IP stack, earning it the name bump-in-the-stack. If external cryptoprocessors are used (that is, not under the direct, integrated control of the operating system), it’s also possible to do what’s called a bump-in-the-wire implementation.

Originally developed for IPv4, work is in process to fully port IPsec over to IPv6.

TLS

Transport Layer Security (TLS) provides for secure connections, but it’s hard to say exactly where in the TCP/IP or OSI protocol stacks it actually sits. It runs on top of the transport layer, and yet it is treated by many applications as if it is the transport layer. But applications that use TLS must actively take steps to initiate and control its use. It’s also further confusing, since the presentation layer is normally thought to provide encryption services for higher layers (such as the application layer in the OSI model). Perhaps it’s best to think of it as providing services at the transport layer and above, as required, and leave it at that. It has largely replaced its predecessor, Secure Sockets Layer, which was found to be vulnerable to attacks on SSL’s block cipher algorithms. (SSL also had this “identity problem” in terms of which layer of the protocol stack it did or didn’t belong to.) Nonetheless, many in the industry still talk of “SSL encryption certificates” when the actual protocol using them is TLS.

The TLS handshake dictates the process by which a secure session is established:

1. The handshake starts when the client requests a TLS connection to a server, typically on port 443, or uses a specific protocol like STARTTLS when using mail or news protocols.
2. Client and server negotiate what cipher suite (cryptographic algorithms and hash functions) will be used for the session.
3. The server authenticates its identity, usually by using a digital certificate (which identifies the server, the CA that authenticates that certificate), and provides the client with the server’s public encryption key.
4. The client confirms the certificate’s validity.
5. Session keys are generated, either by the client encrypting a random number or by using the Diffie-Hellman key exchange to securely generate and exchange this random number.

• If any of these steps fail, the secure connection is not created.

6. The session key is used to symmetrically encrypt and decrypt all subsequent data exchanges during this session, until the client or server signals the end of the session.

The process is shown in Figure 5.12.

The TLS cipher suite is the set of cryptographic algorithms used within TLS across its four major operational phases of key exchange and agreement, authentication, block and stream encryption, and message authentication. This suite is updated as older algorithms are shown to be too vulnerable and as new algorithms become adopted by the Internet Engineering Task Force (IETF) and the web community. As with all algorithms and protocols involving security, the two versions of TLS cipher suite now in common use, V1 and V1.2, are coming to their end of life. On June 30, 2018, SSL, TLS 1.1, and TLS 1.2 were declared obsolete by the IETF. The major browsers, such as Firefox, Chrome, and Bing, have been phasing them out in favor of their replacements. Be sure to check to see if your organization is using them anywhere else. Note that the Payment Card Industry Data Security Standard (PCI DSS) requires use of the new versions, so any credit, debit, or payment processing systems you support may need to be double-checked as well.

TLS has gone through two revisions since its first introduction, and in creating TLS 1.3, RFC 8446 in August 2018 added significant improvements to TLS. One key set of changes involved strengthening forward secrecy of TLS sessions. Forward secrecy (also known as perfect forward secrecy) provides for the protection of past sessions in the event that the server’s private key has been compromised. This protection is ensured by requiring a unique session key for every session a client initiates; in doing so, it offers protection against the Heartbleed exploit that affected SSL and OpenSSL, first reported in 2014. TLS 1.3 also removes support for other cryptographic and hash functions that have proven weak.

TLS 1.3 implements the concept of ephemeral key exchange, supported by many algorithms and cryptographic suites. “Traditional” Diffie-Hellman-Merkle, for example, dictates mathematically that unless one of the parties changes their private key (or they agree to changing some other cryptovariable), then the same session key will result. Numerous workarounds have been attempted to avoid the risks that this introduces. How each key exchange generates this one-moment-in-time (that is, ephemeral) shared key is not germane to us right now; what is important to consider is that this can mean that Diffie-Hellman Ephemeral (DHE), Elliptic Curve D-H Ephemeral (ECDHE), and others weaken the authentication mechanism: if the key is different every time, so is the digital signature. Some other authentication process, such as RSA, PSK, or ECDHA, must then be used to complete the process.

However, TLS 1.3 also added some potential pitfalls that you should be aware of. TLS in its initial versions required at least two round-trip cycles to perform its initialization handshake, which involved using asymmetric encryption. This slowed down the loading of web pages, which impacts customer satisfaction and the business bottom line. TLS 1.3 cut this down to one round-trip time (RTT) and added a capability to allow servers to “remember” a previous session’s handshakes and pick up, in effect, where the session left off. If that sounds to you like reusing a one-time session key, you’d be right to think so. This “zero RTT” option is fast, of course, but it opens the door to replay attacks. Some organizations are already planning on turning this feature off as they migrate to TLS 1.3.

Pretty Good Privacy

In much the same timeframe in which Rivest, Shamir and Adleman were battling with the U.S. government over making powerful encryption available to private citizens, businesses, and others, another battle started to rage over a software package called Pretty Good Privacy. PGP had been created by Phil Zimmerman, a long-time anti-nuclear activist, in 1991; he released it into the wild via a friend who posted it in Usenet and on Peacenet, which was an ISP that focused on supporting various grass-roots political and social movements around the world. Almost immediately, the government realized that PGP’s use of 128-bit (and larger) encryption keys violated the 40-bit limit established for export of munitions as defined in the Militarily Critical Technologies List; the government began a criminal investigation of Zimmerman, his associates, and PGP. Zimmerman then published the source code of PGP and its underlying symmetric encryption algorithm (the Bassomatic) in book form (via MIT Press), which was protected as free speech under the First Amendment of the U.S. Constitution. By 1996, the government backed down and did not bring criminal charges against Zimmerman.

PGP uses a web of trust concept but does embody a concept of key servers that can act as a decentralized mesh of repositories and clearinghouses. Its design provides not only for encryption of data in motion but also for data at rest.

Initially, PGP as a software product allowed end users to encrypt any content, whether that was a file or the body of an email message. Various distributions used different encryption algorithms, such as ElGamal, DSA, and CAST-128. The designs and source code of PGP have moved through a variety of commercial products, including the z/OS encryption facility for the IBM z mainframe computer family.

PGP is described by some as being “as the closest you’re likely to get to military-grade encryption.” As of this writing, there do not seem to be known methods, computational or cryptographic, for breaking PGP encryption. Wikipedia and other sources cite a 2006 case in which U.S. Customs agents could not break PGP-encrypted content, suspected to be child pornography, on a laptop they had seized. A bug in certain implementations of PGP was discovered in May 2018, which under certain circumstances could lead to disclosing the plaintext associated with a given ciphertext of emails encrypted by these email variants.

Since inception, PGP has evolved in several directions. It still is available in various free software and open source distributions; it’s also available in a variety of commercial product forms.

Hypertext Transfer Protocol Secure

Hypertext Transfer Protocol Secure (or HTTPS) is an application layer protocol in TCP/IP and the OSI model; it is simply Hypertext Transfer Protocol (HTTP) using TLS (now that SSL is deprecated) to provide secure, encrypted interactions between clients and servers using hypertext. HTTPS is commonly used by web browser applications programs. HTTPS provides important benefits to clients and servers alike.

• Authentication of identity, especially of the server’s identity to the client
• Privacy and integrity of the data transferred during the session
• Protection against man-in-the-middle attacks that could attempt to hijack an HTTP session
• Simplicity

By building directly on TLS, HTTPS provides for strong encryption of the entire HTTPS session’s data content or payload, using the CAs that were preinstalled in the browser by the browser application developer (Mozilla, Microsoft, DuckDuckGo, Apple, etc.). This leads to a hierarchy of trust, in which the end user should trust the security of the session only if the following conditions hold true:

• The browser software correctly implements HTTPS.
• Certificates are correctly installed in the browser.
• The CA vouches only for legitimate websites.
• The certificate correctly identifies the website.
• The negotiated encryption sufficiently protects the user’s data.

Users should be aware that HTTPS use alone cannot protect everything about the user’s web browsing activities. HTTPS still needs resolvable IP addresses at both ends of the session; even if the content of the session is kept safe, traffic analysis of captured packets may still reveal more than some users want. Metadata about individual page viewings may also be available for others to sniff and inspect.

Secure Multipurpose Internet Mail Extensions

Secure Multipurpose Internet Mail Extensions (S/MIME) provides presentation-layer authentication, message integrity, nonrepudiation, privacy, and data security benefits to users. Using PKI, it requires the user to obtain and install their own certificate, which is then used in forming a digital signature. It provides end-to-end encryption of the email payload and thus makes it difficult for organizations to implement outgoing and incoming email inspection for malware or other contraband without performing this inspection on each end-user workstation after receipt and decryption.

As an end-to-end security solution, S/MIME defeats—or complicates—attempts to have enterprise-wide or server-hosted antimalware scanning (or scanning for other banned content), since S/MIME has encrypted such malware or banned content by encrypting the message content and its attachments. S/MIME may also be difficult, or at least not well suited, to use with a webmail client, if the user’s private key is not accessible from the webmail server. Workarounds, such as those used by PGP Desktop and some versions of GnuPG, will implement the signature process via the client system’s clipboard, which does offer better protection of the private key.

SMIME has other issues, which may mean it is limited in the security it can offer to users of organizational email systems. Its signatures are “detached”—that is, they are not tied to the content of the message itself, so all that they authenticate is the sender’s identity and not that the sender sent the message in question. In May 2018, the EFF announced that there were critical vulnerabilities in S/MIME, particularly when forms of OpenPGP are used. EFAIL, as this vulnerability is called, can allow attackers to hide unknown plaintext within the original message (using various HTML tags). EFAIL affects many email systems, and as such, it will require much coordination between vendors to fix.

DomainKeys Identified Mail

DomainKeys Identified Mail (DKIM) provides an infrastructure for authenticating that an email came from the domain its address information claims it did and was thus (presumably) authorized by that domain operator or owner. It can prevent or limit the vulnerability of an organization’s email system to phishing and email spam attacks. It works by attaching a digital signature to the email message, and the receiving email service validates that signature. This confirms that the email itself (and possibly some of the attachments to it) was not tampered with during transmission, providing a degree of data integrity protection. As an infrastructure service, DKIM is not normally visible to the end users (senders or recipients), which means it does not function as an end-to-end email authentication service.

DKIM can provide some support to spam filtering, acting as an anti-phishing defense as well. Using the domain-based message authentication, reporting, and conformance (DMARC) protocol, mailing services that use DKIM can protect their domain from being spoofed as part of a phishing operation. Based on DNS records and a header field added to them by RFC 5322, DMARC and DKIM together offer a degree of nonrepudiation. This has proven useful to journalists investigating claims regarding the legitimacy of emails leaked during recent political campaigns, such as during the U.S. Presidential Election in 2016. This use of DNS records also makes DKIM compatible with S/MIME and OpenPGP for example; it is compatible with DNSSEC and with the Sender Policy Framework (SPF) as well.

Note that the use of cryptographic checksums by DKIM does impose a computational overhead not usually associated with sending email. Time will tell whether this burden will make bulk spamming too expensive or not.

Both the original RFC that proposed DKIM and work since then have identified a number of possible attack vectors and weaknesses. Some of these are related to the use of short (weak) encryption keys that can easily fall prey to a brute-force attack; others relate to ways that clever spammers can spoof, misroute, forward, or otherwise misuse the email infrastructure in ways DKIM cannot help secure. Note, too, that while DKIM use may help in authentication of senders, this is not the same as offering protection against arbitrary forwarding of emails (bulk or not). A malicious sender within a reputable domain could conceivably compose a bad message, have it DKIM-signed, and send it to themselves at some mailbox from which they can retrieve it as a file. From there, they can resend it to targets who will have no effective way to determine it is fraudulent. Including an expiration time on signatures or even periodically revoking a public key or doing so upon notice of an incident involving such emails remain as options for the domain owners. Filtering outgoing email from the domain may be of limited utility as well, without having clear rules to differentiate emails potentially being misused by spammers from legitimate outgoing ones.

Ongoing work in the IETF Standards Track is adding elliptic curve cryptography to the RSA, which should facilitate making more secure use of shorter keys, which are easier to publish via DNS. There are also some concerns with DKIM and content modification, in that centralized (enterprise) antivirus systems may break the DKIM signature process; workarounds to this may add incompatibilities with MIME messages. Attempts were made to initiate author domain signing practices (ADSP), but by 2013 these were declared historic, as no significant deployment or use took place. (If you have signs of ADSP lingering in your email systems, they may be worth a closer look as candidates for upgrade or replacement.)

Blockchain

Think about the message digest process; it produces a hash value of a message (or file) that demonstrates that the content of that message has not been changed since the message digest was computed. A blockchain is nothing more than a series of messages, each with its own message digest, that taken together represent a transaction history about an item of interest; the message digest for the first block is computed normally, and then this is used as an input into the message digest for the next block, and so on. Thus, any attempt to make a change to the content of a block will invalidate all subsequent block-level message digests.

Blockchain technology is poised to turn much of our digital world sideways, if not completely onto its head. It does this by transforming the way we think about data. Traditionally, data is created (written), read, updated, tweaked, made into “old” and “updated” versions, updated again, deleted, restored; in many real-world practical situations this endless life cycle of change causes more trouble than we recognize. Blockchain treats all data as write-once, read-forever. And it never throws data away.

Blockchain also encourages (if not enforces) a fully distributed data model: every user of any data set has a full and complete copy of it; as new blocks (records) are added to that set, all registered users receive the update, and when the majority have agreed it was a legitimate update, it gets posted to this fully distributed ledger. In fact, referring to a blockchain as a ledger reveals that it views data sets as something that just keep growing, one record or transaction or event at a time. Mistakes in data are dealt with in blockchains by leaving the original data as it was created and then adding additional records as required to reverse the effect of the mistake. This provides a full audit trail of such corrections; and since blockchains are a distributed ledger, all parties to the blockchain must approve such changes for them to take effect.

In its simplest form, a blockchain starts with that first transaction (sometimes called the genesis block) and digitally signs it. Adding on the next block means that it is appended to the first block plus the digital signature attached to it, and then this whole aggregate block is digitally signed by its originator. Clearly, no changes can be made to the data in either block without it invalidating the second digest. As each processing node (or holder of this distributed data set) receives it, it generates a separate secure message digest and compares it with the arriving signature. Figure 5.13 provides a somewhat simplified sketch of this concept in action.

There are a lot more details to it than this simple sketch reveals. The basic concept, however, can be implemented using PKI, PGP/GPG, or a proprietary public/private key infrastructure. The contents of each block can be encrypted for storage, transmission, and use. Individual pairs or sets of users can and do use their own asymmetric encryption to keep their transactions private from others who are sharing the same ledger.

Bitcoin and many cryptocurrencies do something similar. Every Bitcoin user has the entire Bitcoin transaction history, for every Bitcoin ever mined, all in their individual copy of the database. But the only transactions they can read, and thereby extract data from and use in other ways, are the transactions (or blocks) that they can decrypt. The rest are as protected as anything else in this digital age is when it is using properly managed asymmetric encryption supported by a strong, trustworthy digital certificate infrastructure.

Note that blockchains are unforgiving of errors. If I authorize my Bitcoin wallet to pay you 378,456,223 Satoshis (or 3.78 Bitcoins), then as soon as I hit Send, those Satoshis are yours. If what I meant to do is send you only 2.0 Bitcoins, then I have to depend upon your goodwill (or other contractual arrangements that bind us together) to get you to send the difference back to me.

By providing strong nonrepudiation and data integrity for the transactions contained in the individual blocks, blockchains can implement digital provenance systems.

• Chain of custody control, auditing, and record-keeping for cyberforensics could use blockchains to irrefutably record who touched the evidence, when, how, and what they did to it.
• Parts or document provenance systems can prove the authenticity of the underlying data to help prove that safety-critical components (physical hardware, computer or network hardware, software, or firmware) are in fact what they claim to be.
• Representations of any kind of value can be made extremely difficult to counterfeit.

It is this last that explains the dramatic rise in the use of cryptocurrencies—the use of blockchains to represent money and to record and attest to the transactions done with that money.

• The cryptocurrency “miner” uses significant computing power to generate a new unique cryptocurrency identifier (similar to printing a new piece of paper currency with a unique combination of serial numbers, paper security markings, etc.). This “cryptodollar” is represented by a blockchain and is stored in the mining company’s wallet.
• Bob buys that cryptodollar from the miner, and the underlying blockchain transfers to Bob’s wallet; the new message digest reflects this transfer into Bob’s wallet. The blockchain in the miner’s wallet is updated to show this transaction.
• Later, Bob uses that cryptodollar to buy something from Ted’s online store; the blockchain that is Bob’s wallet is updated to reflect the “sell,” and the blockchain that is Ted’s wallet is updated to reflect the “buy.”

If all we do is use strong message digest functions in the blockchain, we provide some very powerful nonrepudiation and data integrity to our cryptocurrency users. We must combine this with a suitable exchange of public and private keys to be able to protect the confidentiality of the data and to ensure that only people or processes Bob authorizes (for example) can see into Bob’s wallet, read the transaction history that is there, or initiate a new transaction.

Finally, cryptocurrency systems need to address the issue of authority: who is it, exactly, that we trust as a “miner” of a cryptodollar? Bitcoin, for example, solves this problem by being a completely decentralized system with no “central bank” or authority involved. The “miners” are in fact the maintainers of copies of the total Bitcoin ledger, which records every Bitcoin owner’s wallet information and its balance; voting algorithms provide for each distributed copy of the ledger to synchronize with the most correct copy. This maintenance function is computationally intensive, typically requiring many high-performance workstations running in parallel, so the Bitcoin system rewards or incentivizes its miners by letting them earn a fraction of a Bitcoin as they maintain the system’s ledger.

One irony of the rise in popularity and widespread adoption of blockchains and cryptocurrencies is the false perception that since many money launderers, drug smugglers, and organized crime use these technologies, therefore anyone using them must also be a criminal. Of course, nearly all criminals use money, but that does not mean that all users of money are criminals!

Federated Systems

Extending access to your IT infrastructure outside of your organization—and inviting outsiders to enjoy (hopefully limited) access to your systems and information assets—is made simpler and easier to manage and keep safe via a variety of federated access control strategies and mechanisms; Chapter 1 looks at these in some detail, including the use of mechanisms like Security Assertions Markup Language (SAML) to help automate and regulate such connections. Underneath all of these mechanisms is the encryption infrastructure that provides strong hash functions, point-to-point data integrity, confidentiality, and nonrepudiation, to name but a few. Your chosen access control systems should be your first port of call to understand how they negotiate the use of encryption with other systems in establishing connections such as single sign-on (SSO).

It’s possible that in a fairly diverse federated environment, one of the partner organizations is far less adept and skillful at keeping its cryptographic systems up to date; it may also have inherent, systemic issues in the ways in which it manages its users, its keys, digital certificates, and everything else. Part of your threat-hunting skepticism might be to see if your systems can report on any connections that routinely negotiate to the weakest choices in your installed base of acceptable encryption suites and algorithms. (And you should certainly be hunting out and locking down any such connections that keep trying to use SSL, for example!)

Transaction and Workflow Processing

Many business processes—not just financial ones—are based on a transaction processing model. Inventory systems, for example, will receive a transaction from an assembly line system each time it allocates a part or subassembly to the flow at a workstation and thus reduce the number of such parts on hand. Students enrolling in a university course generate transactions that “sell” a seat in a particular class. Workflow processing systems build up sets of information related to a set of tasks and pass that information from workstation to workstation to control the actual work itself, model it, and provide a structure and framework to measure and assess it with. Patient care systems quite often model the flow of the patient from initial intake through post-discharge follow-up care by means of a workflow model.

Workflows depend upon data integrity; many also must protect the confidentiality and privacy of the data that they collect, process, route, and use. In many respects, this is a role-based access control problem that the workflow management system needs to help you solve. Depending upon the chosen platform, it may or may not integrate seamlessly with your existing access control system; lack of good integration means that each time a workflow is changed and worker roles are redefined, you may have to update your access control lists independently; similarly, as employees change jobs or depart the organization, you could have multiple systems to update. This is discussed in Chapter 1 as well.

Underneath that, your transaction and workflow processing systems may or may not be using their own encryption technologies to deliver on their promises of data protection and systems integrity. Some large database systems (which are underneath some of these applications platforms) do use encryption to implement record-level or even field-level access control capabilities. Whether these systems turn to the underlying server for encryption infrastructure support, or have their own baked-in encryption systems, may be an issue you need to investigate.

Many industries are beginning to adopt vertical applications systems that integrate transaction and workflow processing with email and communications management. Several vendors, such as Adobe and RPost, provide such capabilities. Adobe’s approach tends to be focused around documents that flow between people and organizational units; RPost’s does that too, but by replacing email and fax traffic as steps in transaction processing, it is showing interesting application in integrated logistics and process control environments. Blockchain technologies are being applied in this use case area quite successfully.

Regulatory and compliance requirements are driving organizations to have reliable nonrepudiation, safety, auditability, and transparency with respect to the fine-grained details of how products are assembled, how acceptance testing gets accomplished, and how customer orders are handled. These capabilities fundamentally depend upon the underlying cryptographic infrastructures and the cryptographic technologies your organization is using.

Integrated Logistics Support

Integrated logistics support (ILS) takes supply chain management several steps further by finding the most cost-effective ways to use information to improve and mature the organization’s value chain. ILS extends that value chain far beyond the walls of the company or organization itself, by reaching as deep into the systems and information flows of its vendors, shippers, service providers, customers, and even their customers’ customers, as there is value to be had in doing so.

From an information security perspective, that means that an ILS system crosses many organizational boundaries; it must meld together many different cultural perspectives on privacy, auditability, compliance, data and systems integrity, information quality, and protection. As an ILS may reach across many national borders, it must also face multiple legal, regulatory, and other compliance regimes. This is a federated systems problem writ large; this is transaction and workflow management taken across time zones, human languages and dialects, and organizational imperatives. This is also another problem domain that blockchain technologies are being put to use in, and they are already achieving some significant results.

One vexing question in ILS security is raised when the many different organizations in an extended ILS value chain do not have the same information security needs in common or if they have wildly different interpretations of what “confidentiality” means, for example. Your own organization might also find that some of its other customers or suppliers have their own unique perspectives on the risks that some of your value chains’ members potentially expose their information and their systems to.

Keeping an ILS system safe and secure relies on much the same technical and managerial approaches to providing and maintaining the underlying cryptographic systems that support it.

Secure Collaboration

Collaboration suites and platforms are becoming essential in many businesses large and small. They provide much of the human-to-human activity that knits together complex project management teams, while being layer 8 of the integrated logistics support systems as well. They rely on the underlying access control systems, including SSO connections, to function effectively. Some collaboration platforms may provide separate encryption capabilities which users can invoke on a per-session basis or as part of their default settings. They may also provide the option to produce encrypted or plaintext recordings of sessions. Your organization may need to understand these capabilities if their use presents a data classification and management concern or if other compliance issues are associated with recordings of such collaboration sessions.

IoT, UAS, and ICS: The Untended Endpoints

These three different systems domains have two characteristics in common: they all have endpoint devices that generally are not directly tended to by people, and their technologies do not usually come with encrypted, secure links for data, commands, device health, and status information.

• Internet of Things (IoT) devices can range from the simplest of control devices, such as a home heating thermostat, to complex, semi-autonomous and mobile devices. Mobile freight and package handling devices used in warehouse and order fulfillment processing in warehouse operations, such as at Amazon, are but one example.
• Industrial process control systems (ICSs) interact directly on one side with physical manufacturing systems—furnaces, pumps, heavy machinery, or industrial robots—and an ICS-specific set of network protocols on the other.
• Untended or uncrewed aerial systems (UASs), and their cousins that move on land, water, or underwater, also have unique interfaces to the systems that control them.

UAS tend to have more complex mission management, dispatch, navigation planning, and safety requirements than the other two categories sometimes do. The UAS device itself may also have a “fail-safe” mission mode that causes the device to navigate to a predetermined safe point and come to rest, in the event of a protracted communications link failure or other conditions.

Many of these types of devices, in all three of these systems domains, tend to have limited onboard capacity to support sophisticated encryption of their uplinks and downlinks. Even if they do, the protocols used on those links may not always be compatible with TCP/IP-based protocols and services that the rest of your organization’s networks and systems are using.

You may be lucky and find out that the IoT, UAS, ICS, or other robotics types of systems your organization is using or wants to begin using are directly supportable by your existing cryptographic systems and technologies. You may find that their systems, links, and operating procedures can hold to the same standards for key strength, key management, and use that you already have in place; they may also be compatible with your existing access control systems to the degree that this is an issue for your use of these untended endpoints. Then again, you may find that your organization has to either accept the potential for security compromise or plan to upgrade these systems and their interfaces to be more compatible with the rest of your cryptographic infrastructure.

Trusting SOHO

Taken altogether, small businesses (of less than 20 people) comprise the largest aggregate employer in most nations. This is the marketplace for small office/home office (SOHO) systems built with consumer-grade equipment and software and firmware installed and managed by the original equipment manufacturer (OEM) on their own schedule. It’s the marketplace of little in-house cryptographic expertise (if any), and it’s the never-ending consulting opportunity for the freelance or moonlighting security professional.

Such environments are anything but server-free, of course. Their router (typically provided by their ISP) provides DHCP services at a minimum; each endpoint has multiple servers built into it, and any network-attached storage devices provide even more. These environments are also prone to using many cloud-based storage solutions, particularly in the free end of the spectrum. Some of their uses may have set these up with encryption on some files, on some folders, or on a whole storage volume.

Many organizations in such circumstances do not get their own digital certificates; if they have a website, they may unknowingly rely on its hosting service (which might be WordPress or similar easy-entry web page development and hosting solutions) for HTTPS session support. User IDs, passwords, authorized devices (for MAC address filtering on routers), and even the use of built-in access controls can be unmanaged and disorganized. Document e-signing services may be used to provide digital signatures as part of some business processes, but the organization may not have any formal management process to control their use, maintain them, expire, or revoke them (as faces change at the company).

In such environments, some standard cryptographic hygiene measures are desperately needed. These organizations may have little or no procedural grasp of what to do when an employee quits, becomes disgruntled, or compromised. Dealing with lost or forgotten passwords or encryption keys can be a traumatic case study in learning on the fly every time it happens.

On-Premises Data Center

Once an organization has migrated to more of a structured data center as the power behind its IT infrastructure, it’s also making the step toward more formalized information security measures. Greater awareness of compliance and regulatory demands for tighter security, even the use of encryption to protect sensitive information and restrict access to sensitive processes, may also be more common in such a setting. Most server systems come with certificate generation and management capabilities built right in; even consumer-grade NAS solutions often have these features although few SOHO architectures ever see them put into effective use. At this point, systems administrators can now generate and issue certificates, and the organization can digitally sign files, documents, archive copies of data sets, and so on.

Key management becomes an issue at this point; including HSM solutions as part of their IT architecture may be prudent.

Investment in a data center can be the tipping point at which management starts to seriously worry about data exfiltration, ransom attacks, and other high-impact threats to the company’s survival and success. This brings two encryption-based worry beads to the table.

• Weaponized and encrypted inbound content: Their architecture’s threat surface exposes them to many possible ways that weaponized content can make it into their systems, and if the connection being used by that attack vector has its encryption managed independently of the organization’s control, there’s no useful way to detect it or prevent it. Other means, such as tighter control of endpoint activities, software and even process-based whitelisting, endpoint and host-based intrusion detection and prevention, and behavior modeling and analysis can help but may not be enough.
• Encrypted “east-west” internal traffic: Almost every business process supported by your systems and architectures will move data internally, as different servers, operational locations, departments, and human beings get involved in the end-to-end of the business of doing business. This has to happen efficiently and reliably, of course. Some of those processes may have information security needs that dictate encryption be used to protect the data while in transit internally. How can you tell whether a particular stream of encrypted packets on your internal network is legitimate, is an exfiltration in process, or is some other part of an attacker’s kill chain in action? Inspecting the unencrypted content of every packet might help, but doing this requires you to be your own MITM attacker. You have to ensure that every internal encrypted data flow path potentially carrying highly sensitive information must route itself through an encryption proxy service that provides you with opportunities for (automated) inspection, analysis, and logging. This has throughput performance penalties that can seriously affect your competitive advantage. It may also end up putting all of your most sensitive information into one high-value point of potential exploitation (the so-called “all eggs in one basket” problem).

High-Compliance Architectures

As an organization extends its business activities into areas that must meet stringent regulatory and other compliance requirements for data protection, safety, and reliability, the needs for encryption-based security solutions increase. By one count, more than 83 different types of business activities, from healthcare through gambling and from banking through medical care are subject to compliance regimes that dictate such architectures be put in place, managed effectively, used consistently, and be subject to verification testing, audit, inspection, and ongoing security assessment. All of the issues looked at throughout this chapter, and then some, require that the organization have some seriously capable in-house cryptologic talent—or trust its fate to its vendors and consultants.

These types of environments and architectures need powerful capabilities for monitoring the use of encryption by applications; this may require extensive analysis of behavioral norms to identify acceptable use versus usage worthy of further inspection and analysis. They also need to have a similar capability to monitor cloud-hosted service usage, in particular the ebb and flow of encrypted traffic and data via cloud-based application platforms, in order to be able to spot anomalous patterns of activity. These monitoring capabilities should be in-house, directly coupled to organizational management and leadership, to strengthen the sense that the “watchers” and the managers and leaders jointly own the challenges that maintaining such vigilance entails.

Key Strength and Key Generation

Key strength is the result of two principle factors: the size of the key itself (expressed as the number of bits to represent it) and the randomness associated with the key value and its generation process. Your choices about both of these factors should always be driven by your particular information security requirements, especially the time period over which you need the data you’re about to encrypt to be kept safe from prying eyes or corrupting fingers.

Key Size Generally, the longer the key, the more difficult it is to attack encrypted messages; it determines the search space, the set of all possible numeric values that fit within the number of bits in the key that a brute-force attack must attempt in order to find a match. What constitutes a secure key length changes with time as computer power increases and cryptanalysis techniques become more sophisticated. In 1977, the U.S. government selected a symmetric block cipher they dubbed the Data Encryption Standard (DES), which was considered secure for nonclassified use, with the intent that banking systems and other commercial business users could start to use it to protect their data. But by 1999, multiple cryptanalysts demonstrated that DES could be cracked in as little as 24 hours using large arrays of processors such as the EFF’s DES Cracking Machine, for a cost of only $250,000. Governments’ monopoly on powerful code-breaking hardware had crumbled with the birth and explosive growth of the personal computer market, especially its appetite for higher-performance single-chip CPUs, math co-processors, and graphics processors.

As DES was replaced with AES, key sizes grew from 56 bits (DES) to 128, then 192, and then 256 bits. A brute-force attack on a 128-bit key, using a single CPU, would take something on the order of 14 billion years of CPU time to try every possible value. Of course, the attacker could be lucky on their first handful of attempts. Any clues that the attacker can use to reduce the size of that search space, bit by bit, raises their chance of success. AES, for example, was described in a 1999 paper to be vulnerable to attack by precomputing intermediate values (the Bleichenbacher Oracles attack mentioned earlier). Such intelligent guesswork can cut the effective key size (and resultant search space) down to a much more manageable size, perhaps back to the DES 56-bit window of vulnerability. Various organizations such as NIST in the United States, ANSSI in France, or the BSI in the United Kingdom provide concrete recommendations such as disallowing the use of 80-bit symmetric keys after 2012, and 112-bit keys after 2030. A strong key should be random enough to make it unlikely that an attacker can predict any of the bits of the key. If the mechanism used to generate keys is even somewhat predictable, then the system becomes easier to crack.

The choice of algorithm has a profound effect on the strength of a given size key as well. By comparing the estimated run time or work factor necessary to backward solve the decryption algorithm without the decryption key, it’s been shown for example that ECC demonstrates the same level of key strength at about one-fourth the key size of the RSA algorithm.

Randomness The strength of a symmetric key also depends on its being unpredictable (both “unguessable” in the human sense and resistant to mathematical attempts to generate the next key in a sequence, given one such key). Even if only some of the bits can be guessed, that can significantly reduce the strength of the cipher. Using a mechanism that will generate high-quality (i.e., cryptographically secure) random numbers is essential for key generation. The best method is to use hardware-based true random number generators (HRNGs) that use very low-level physical phenomena, such as thermal noise, to generate sequences of numbers as outputs that are statistically highly random. Such devices are embedded in various cryptographic hardware devices such as Trusted Platform Modules and Hardware Security Modules, discussed earlier, as well as in some microprocessors themselves.

Software-based RNGs are very hard to get right. For example, from 2006 until 2008, a defect introduced into the OpenSSL package in the Debian and Ubuntu Linux distributions caused all keys generated to be weak because of a bug in the random number generator (CVE-2008-0166).

Key Generation Keys should be generated in a manner appropriate for the cryptographic algorithm being used. The proper method to generate a symmetric key is different from a public/private key pair. NIST SP800-133, “Recommendation for Cryptographic Key Generation,” provides specific guidance. The best approach for generating asymmetric keys, however, is to use a trusted digital certificate provider, which will generate a public/private key pair bound to a specific identity. At some point, however, your organization’s need for multiple public/private key pairs will dictate that you become your own certificate authority and take on the management burden and expense of generating and issuing key pairs to users within the organization. (At this point, investing in an HSM as a secure storage and processing part of your CA capabilities makes sense.) Generating your own certificates should not be confused with self-signing certificates. Self-signing certificates actually authenticate nothing but the certificate itself; they are quite useful in software and systems test environments, where a working, fully functional certificate, private key, and public key might be needed but are not required to provide security. Do not attempt to use self-signing certificates to protect anything!

Secure Key Storage and Use

Once you have generated a nice long and random key, how do you keep it secret? Obviously the first step is to encrypt your data encryption key (DEK) with another key, the key encryption key (KEK). That solves the problem but creates another: how do you secure your KEK? This depends on the sensitivity of the underlying data being encrypted and the mechanisms that are appropriate for the system you are designing.

A hardware security module (HSM) is specifically designed to securely generate and store KEKs and is among the more secure method of protecting keys. HSMs also provide for secure ways of replicating the KEKs between redundant HSMs and for enforcing controls so that no one individual can use the KEK to decrypt a data encryption key.

For systems that are deployed in the public cloud, the major cloud providers offer cloud implementations of HSM technology. Numerous hardware security module as a service (HSMaaS) vendors are available in the marketplace, often as a service offering in conjunction with other cryptographic functionality such as key escrow and cryptographic acceleration.

For less demanding applications, the approach must be tailored to the specific requirements and risks. This might permit storage of master keys in a: hardware-encrypted USB key, a password management app, or a secrets management package. Another factor to consider is how your keys are (securely) backed up. Losing access to your keys can cause you to lose access to significant amounts of other data (and backups of that data).

Key Distribution, Exchange, and Trust

Even the smallest of systems using encryption—two nodes or even one node that writes encrypted data to storage to retrieve, decrypt, and use it later—has the issue of getting the keys from the generation function to each point of use. In the more general case of a network of multiple users, all of whom are cleared to the same level of information security classification and all of whom need to communicate with each other, the key generation function has to distribute a copy of the key (and any other required cryptovariables) to each node that will participate in a multiway sharing of information. For n users in a system, that’s n different sets of messages going back and forth to send the key and verify its receipt. On the other hand, for systems that require each pair of those n users to have their own separate key so that pair-wise communication and data sharing remains secret from the others on the network that’s n(n - 1), or roughly n² key exchanges that need to be managed, confirmed, and kept track of. (Keeping track of which user nodes or systems have which key at what time is a major part of what keeps the encryption system synchronized across those users; it’s also necessary to investigate lost or possibly compromised keys, and it’s a major component in providing nonrepudiation of messages sent through the system as well.)

Three different trust relationships exist in a key distribution system:

• The key distributor has to trust that the recipients are who they claim they are (which requires that they are still using the same identities they had the last time keys were sent to them) and that their facility, local key storage, systems, and users have not been compromised without it being reported to the key distributor.
• The recipients have to trust that the key distributor has not been compromised without their knowledge.
• Individual user recipients have to trust that no other users on the network (whether they share keys with them or not) have been compromised without their knowledge.

The net effect of these trust relationships is that all parties can trust that the keys they’ve just been issued will provide the level of security protection that the system, with proper keys, is supposed to provide. The ultimate organizational owners of the network may be the anchor of all of these trust relationships (that is, the one whose assertions of trust the other nodes can depend upon), or it may be the key distributor that does this. (We’ll look at this in more detail in the “Hierarchies of Trust” and “Web of Trust” sections.)

Keys are sent from the distributor to user nodes in one of two ways.

• In-band distribution uses the same communications channels and protocols that the parties use for normal message or data traffic between them.
• Out-of-band distribution uses a physically, electrically, logically, and administratively separate and distinct channel for key distribution and management. This may be a physical channel, such as a courier, a different VPN, or a dedicated circuit used solely for key distribution.

Part of the trust that all parties must repose in each other includes the fact that they share the protection responsibility for that distribution system, whether it is in-band or out-of-band.

Key exchange, which is used with hybrid encryption systems as part of a public key encryption infrastructure, is different than key distribution. It may still use in-band or out-of-band communication channels to support the exchange process; but by its nature, it does not depend upon the parties in the key exchange having a previously established trust relationship with each other.

Key Rotation, Expiration, and Revocation

As with everything in information security, your cryptographic keys are part of your race against time with your attacker. The longer you use any given key—the more packets, messages, or files you use it to encrypt—the greater the opportunity you give your attackers to possibly succeed in attacking your encryption processes. One time-tested countermeasure is to change your keys, early and often. Ideally, you’d use a new key every byte (which sounds like an infinitely long one-time pad key being put to use). Practically, you’ve got to find the balance between the costs and efforts involved in changing keys versus the risk that adversaries can break your encryption and get at your data or systems as a result.

Many organizations establish a key rotation schedule, by which they plan for the routine replacement of keys currently in use with newly minted keys. NIST, ISO, and the Payment Card Industry Security Standards Council recommend that keys be changed at least annually, and this includes public/private key pairs as well as symmetric keys generated with them and used to encrypt archival data backups. Other industries may require keys to be changed more frequently.

In practice, the keys used to encrypt new data are changed each year, and all of the previously encrypted data is decrypted using the retired keys and then re-encrypted using the new key within the year following the key rotation. Thus, by the end of a year after the key rotation, there is no data encrypted using the retired key, at which time it can be destroyed. In cases in which backups must be maintained for longer than one year, either a process for securely archiving retired keys must be instituted or backups will have to also be re-encrypted with the new key.

Key revocation is the process of formally notifying key users, and users of data encrypted with a key, that the key has been compromised or its safety and security is in doubt. If there is evidence or even suspicion that a key has been compromised, it ought to be rotated (that is, replaced) as soon as feasible. Best practice also dictates that keys be replaced when essential personnel with access to cryptographic material leave their position.

Key rotation, expiration, and revocation all reduce the likelihood that an attacker can break your encryption systems, or otherwise circumvent them, and then have unrestricted access to your information and information systems. By smartly rotating and expiring keys, you can also manage the amount of archival or backup data that is protected by any particular set of keys. (Think of this as limiting your exposure to losing that data, in the event the key is compromised without your knowledge.)

Key Destruction

Once a key has been retired and it has been determined that there is no data that has been encrypted using that key that will need to be decrypted, then it must be securely destroyed. This involves locating every copy of the key and deleting it in a manner appropriate for the media on which it was stored to ensure that it cannot be recovered.

Depending on the media and the risk of it becoming accessible to unauthorized individuals, this may require overwriting the storage, degaussing of the media, or physical destruction of the media or device containing the media.

Records must be kept documenting the locations of the destroyed keys and the means used to ensure secure destruction. In many regulatory and compliance regimes, these records are subject to audit.

Key Management Vulnerabilities

There are a number of vulnerabilities that can be introduced through the incorrect use, storage, and management of cryptographic keys.

There are potential trust concerns when dealing with some agencies. While NIST is highly regarded as operating with virtues in line with most cryptography users, agencies such as the NSA also possess both the expertise and responsibility to monitor electronic communications, and therefore it cannot be assumed that they will always prioritize protecting privacy over intelligence. They’ve also repeatedly stated that their national security duties require them to attempt to find ways to build in backdoors or otherwise weaken encryption processes, whether they acknowledge that they’ve done so or not. “Don’t worry if your system crashes, because NSA has a backup copy of it” may be an apocryphal bit of black humor, but if the thought of NSA having a copy of your communications or data makes you or your company uncomfortable, consider using an alternative such as a PGP-based system that’s outside of the NSA/NIST nexus of the white-hat community.

Symmetric and private keys depend upon confidentiality to be effective. This means great care must be taken with how the keys are stored to reduce the possibility of their becoming known to unauthorized entities. There are a number of approaches to secure key storage, from key management software to key management services provided by cloud service providers to dedicated hardware devices that keep the keys stored internally in a tamper-resistant secure device.

Keys that have reached the end of their lifetime (and all properly managed keys should have a defined lifetime) must be securely destroyed to prevent their misuse in the future. Keys should not be reused and should be rotated (replaced) periodically to ensure that the amount of data encrypted using a single key is limited and that the lifetime of data encrypted using a given key is likewise limited.

Another vulnerability can arise from insider threats. A rogue employee with access to key material can use that access to defeat the security of encrypted material (or enable another to do the same). Dual control or segregation of duties can be employed to ensure that at least two people must be actively involved before any key management operation that might compromise the security of the system can be completed. Another popular method to mitigate the risk of a rogue employee exposing cryptographic material is to use a third party to safeguard keys. A key escrow agent is a trusted external party that ensures the cryptographic keys are safe from unintended abuse or leaks. As an analogy, this is similar to how an escrow agreement is used to safeguard a large sum of money when purchasing a home.

The final leg of the CIA triad must also be considered: availability. If the key management system cannot provide access to the key material to authorized processes when required, then access to the encrypted material will be denied, even if the encrypted data is readily accessible.

Key operations must be logged in a manner that ensures accountability and traceability, providing the forensic evidence essential to analysis of a suspected possible compromise, intrusion, or data breach.

Finally, where possible, key management functions ought to be automated. Manual processes are more prone to error (either by commission or omission), leading to weaknesses in the system that depends upon the keys for security.

Escrow and Key Recovery

During the 1980s and 1990s, the U.S. government (and some of its NATO allies) pushed hard to get legislative support for a concept they called key escrow. Key escrow would require any private person (corporate or individual) who created a cryptographic key to place a copy of it on deposit with the government, who would have to have legal authority, such as a search or surveillance (wiretap) warrant from a court before they could take the key out of escrow and use it to decrypt intercepted files or messages involving that party. Needless to say, the outcry across many communities was substantial. This was also still at a time when the U.S. government believed it could systematically restrict export of encryption technologies, and only allow downgraded, more easily breakable versions of American-made encryption products to be exported. Market economics quickly demonstrated that foreign systems vendors would dominate the world market, which would leave American technology and software firms with nothing but a domestic market. Eventually, key escrow as a government agenda item lost what luster it might have had. But it’s not dead yet; periodically, governments protest that they need the ability to break everybody’s encryption, just in case they find the nuggets of information they need to thwart a terrorist attack. Key escrow, alas, cannot force the criminal or terrorist—or a foreign intelligence agent—to submit their keys to a government escrow agency.

Lost Key Insurance As the use of password managers, hardware security modules, and other such systems has grown, business and personal users are finding legitimate reasons to want some form of “lost or forgotten key insurance,” so to speak. Key escrow and key recovery are often discussed as options to mitigate the risk of lost, corrupted, or forgotten keys and digital certificates.

Internally, key escrow processes can be useful, and solutions such as hardware security modules (or even HSMaaS approaches) may be worth considering. When an organization takes on the task of being its own CA and issues certificates (and therefore public-private key pairs) to users, it may want to look further at its needs for internal key escrow. There are also legitimate business continuity needs to identify ways to safely escrow (or archive) certificates and keys and then be able to quickly reinstall them as part of recovering from an incident or disaster. In most cases, recovering a private key associated with a digital certificate issued by a public-facing CA is not possible.

Hacking Back a Lost Key Key recovery has become the collective, polite term for any capabilities, efforts, or attempts to reverse engineer or hack back the decryption keys associated with a set of ciphertext, such as account passwords, digital signatures, encrypted files or disk drives, smartphones, network traffic, or messages of any kind, for the purpose of making the key-protected content available and useful to its owners again. Presumably, these attacks are carried out by ethical hackers, whose actions are constrained and authorized by a suitable contract with the owners of the systems or accounts in question.

There are two difficulties with such concepts, one technical and the other legal.

The legal difficulty is that for any activity involving a U.S. person, the U.S. government can serve anyone even remotely involved with that person with a National Security Letter (NSL). An NSL is a warrant issued by the U.S. Foreign Intelligence Surveillance Court, and it requires the person it is served on to surrender or make available any information the NSL asks for, without informing anyone, even their attorney, about the NSL. Presumably, LastPass, Mozilla, Google, Microsoft, or any firm making a password manager service available could be served with such an NSL; they would have no legal recourse but to comply. If your password manager can reveal or recover your master password to you, it can do so when compelled by an NSL (or even a much weaker search warrant or subpoena. (The penalties for not complying are quite painful, as spelled out in the Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism Act of 2001, known as the USA PATRIOT Act.)

The technical difficulty goes to the heart of our widely held assumption that our whole public cryptography infrastructure, when used with strong keys and with proper cryptographic hygiene measures in place, will really keep our information and systems safe, secure, and reliable. If the encryption keys and algorithms used are as strong as we all hope they are, any attempts at key recovery should be expensive, time-consuming, and impossible to guarantee to be completed in any specific or reasonable amount of time. Such brute-force key recovery, aided by guesses, fragmentary memory, or other information provided by the owner to the key recovery service, may help speed the process up; but at the end of the day, either all of our secrets are safe and key recovery is a pipe dream or key recovery is real, affordable, and practical, and nothing is as safe as we hope and need it to be.

Separation of Duties, Dual Control, and Split Knowledge

It’s fundamental to risk management and mitigation that any business process that involves a high-impact risk should be designed, implemented, monitored, and managed in such a way that no one person or system element has the opportunity to corrupt that process on their own. We add redundancy to such high-impact risk control processes in a variety of ways.

• Majority voting: Redundant, parallel processing elements (people or systems) each execute separately on the same inputs and previous system state; the results are compared, and differences beyond a tolerance value are flagged to supervisory functions for investigation and resolution. (Blockchain systems and other distributed, high-reliability systems use this technique.)
• Split (or shared) knowledge: Critical process knowledge is not granted to a sole individual person, node, or component; instead, multiple such people or nodes must bring their knowledge together to a trusted system component that combines them, and validates that result, before allowing the process to continue. This is similar to multifactor authentication; some hardware security modules use physically separate and distinct additional factors for each human interacting with the HSM, all of which (or a set majority of which) must be present in order to allow HSM critical maintenance or management functions to be performed. Using asymmetric encryption in a key exchange process to generate a session key is an example of a split knowledge process.
• Dual (or multiple) control: The process is designed with layers or sequences of approval and authorization steps, each of which is performed by a unique individual person or systems component; all process steps must be correctly completed in the proper sequence for the results to be valid. High-value financial transactions often require two or three different company officers to authorize before a check can be issued or an electronic payment can be processed. This can also include process steps where all parties must execute their part of the step simultaneously, if not also at the same location. Such locations are sometimes known as no lone zones, since policy and process design prohibit one person being in the area and attempting to perform the process task by themselves.

Each of these control techniques can play a part in applying a separation of duties control strategy. Separation of duties reduces the risk that one employee can take an action that can cause harm to the organization, its information, its systems, or its customers, without that action being detected and possibly prevented or remedied in a timely fashion. Separation of duties also protects the employees from potentially being falsely accused of negligence or willfully taking harmful action by helping to isolate where in the chain of business processes an error, misstep, or improper action actually took place.

Protecting your cryptographic systems, keying material, keys, and even the data remaining in the devices or system after an encryption session is completed may very well require some form of dual control or separation of duties. Compliance requirements in many business areas may also require that your systems have no “lone zones” or one-person tasks or functions in critical process flows.