1   Introduction to Blockchain

For the world of technology users, blockchain represents a dramatic improvement to the landscape of information collection, distribution, and governance. That point has been espoused these past few years in the books and presentations that hype and imagine this new world. This book is one of the first to address the development of blockchain applications. As such, we will present a development road map to the emerging options and trends. That said, this is the first edition of what will be a series following the blockchain development evolution. This book is aimed at all levels of developers, software engineers, and anyone interested in the basics of blockchain technology, as well as the languages and tools required to build decentralized applications. We will introduce everything needed to understand the technology, write “smart contracts,” build applications that interact with them, and deploy and maintain these applications on a host of emerging platforms.

So, let’s begin. Simply put, a blockchain is a database encompassing a physical chain of fixed-length blocks that include 1 to N transactions, where each transaction added to a new block is validated and then inserted into the block. When the block is completed, it is added to the end of the existing chain of blocks. Moreover, the only two operations—as opposed to the classic CRUD—are add transaction and view transaction. So the basic blockchain processing consists of the following steps, which are numbered 3, 4, and 5 in Figure 1-1:

FIGURE 1-1   Public blockchain transaction flow

1.   Add new and undeletable transactions and organize them into blocks.

2.   Cryptographically verify each transaction in the block.

3.   Append the new block to the end of the existing immutable blockchain.

More comprehensively, a blockchain is also a distributed database that maintains a doubly linked list of ordered blocks. Each block averages 1 megabyte (see https://blockchain.info/charts/avg-block-size) and contains control data of approximately 200 bytes, such as a timestamp, a link to a previous block, some other fields (as depicted in Figure 1-2, to be discussed later), and 1 to N transactions as can fit in the remaining space.

FIGURE 1-2   Blockchain data layout

The blocks once recorded are designed to be resistant to modification; the data in a block cannot be altered retroactively. Through the use of a peer-to-peer network and a distributed timestamping server, a public blockchain database is managed autonomously. Blockchains are an open, distributed ledger that can record transactions between two parties efficiently and in a verifiable and permanent way as depicted in Figure 1-1.

The ledger itself can also be programmed to trigger transactions automatically. Blockchains are secure by design and an example of a distributed computing system with high byzantine fault tolerance. Decentralized consensus can therefore be achieved with a public blockchain. As we shall discuss in detail later, these features make blockchains ideal for recording events, medical records and other records management activities, identity management, transaction processing, and a host of emerging applications. Moreover, blockchain technologies allow us to achieve large-scale and systematic cooperation in an entirely distributed and decentralized manner. This can be considered and implemented as a global governance tool, capable of managing social interactions on a large scale and dismissing traditional central authorities. For example, in 2015, libertarian political activist Vit Jedlička declared Gornja Siga—a seven-square-kilometer patch of uninhabited forest between Croatia and Serbia—to be the “Free Republic of Liberland.” He used the Bitcoin blockchain as a provisional government and released a constitutional document setting out how this new country would be governed: voluntary taxation, an almost nonexistent government, and zero restrictions whatsoever on speech and information.

Let’s look at some analogies that illustrate what is different about the public blockchain. It’s both a database and the software that envelops it. As software, it is like BitTorrent, a program that allows you to upload and download files directly with others also running the BitTorrent software. So instead of uploading a file to a file-sharing service, such as Dropbox, and then sending your friend a link to download the file, you just upload the file directly to your friend’s computer. This is what we mean by a peer-to-peer (p2p) program (see Figure 1-3).

FIGURE 1-3   Decentralized versus centralized data stores

The public blockchain is also a peer-to-peer program with one very important difference: not only does it move files (data) from peer to peer, it also ensures that all the peers have the same exact data. It enforces this. If the data changes on one machine, it changes on all the machines. There are rules specifying exactly how a change can be made, and if someone doesn’t follow them and modifies their copy illegally, they’re ignored. It’s no different from an email program trying to send an email without the proper SMTP headers—it won’t be recognized by other email programs. By the same token, if your version gets deleted or corrupted, it’s not a problem, just re-sync with your peers and you get a fresh valid copy.

As noted, the way current public blockchains like Bitcoin and Ethereum work is that instead of changing data within the dataset, new data is just appended onto the old. In other words, data is only written, never deleted. This is how it gets the name blockchain, because new data is added in batches, or blocks, and appended to the existing blocks, forming a chain of blocks. Not only does everyone have the same database (blockchain), but everyone gets a locker within the blockchain that only they can access. Normally, exclusive access to something is managed with usernames and passwords. The public blockchain has no central authority to manage usernames and passwords, so instead it uses cryptography. Each user is able to generate a locker address and a private key code that allows them to unlock the locker. The locker is only an analogy, of course. In reality it’s just an ID number (referred to as an address), which is tagged to a user’s data. The private key is a code that allows the user to prove they’re the creator (or owner) of that address. Only the person who generated the address would have the private key, and no one can ever determine what the private key is from the address alone.

So while everyone can see the data tagged with your address in the blockchain, no one is allowed to modify it. It can only be modified by the person who can prove they’re the owner via the private key. For example, if bitcoins are tagged with your address, they cannot be moved (i.e., tagged with another address) unless the private key is used.

What’s also amazing about this system is that anyone can generate an address by themselves, in isolation, without concern that it will clash with anyone else’s address. The reason for this is that there are so many possible addresses, it is essentially impossible to clash with another address, even if you tried.

It gets better. Not only is static data stored in the dataset, but you can store executable code in it too. Say you have a piece of code written in JavaScript-type language, such as Ethereum Solidity, sitting there on everyone’s machine waiting to be executed. Remember that data is only written to the blockchain, never erased, so you now have a piece of code no one can change. Everyone can be certain the way it’s written is the way it will always run.

This code is also tagged with someone’s address. The owner of that address gets to decide what operations are open to the public and what only they can execute. They only get to make this decision at the time the code is written. Once written, it cannot be changed. Everyone will still be able to see the code and what it’s doing, but can only interact with it in the ways specified by the owner.

Let’s start with the original motivation to create a blockchain: money. Our current monetary system is based on records of how much money is out there and who has how much of it. We rely on our governments and banks to maintain these records. But a blockchain allows us to keep these records ourselves, since it guarantees that the record is the same for everyone. We each keep this dataset—that is, the blockchain that contains a record of every single transaction that happens in the particular monetary system. Since everyone’s copy is synchronized with everyone else’s, no one has to worry about fraudulent or conflicting entries. There’s now no need for a bank to manage our records. The blockchain does it instead. As far as how money gets created and distributed in the first place, that is another story, but the Bitcoin network as well as other cryptocurrencies handle that as well.

That is just on the data, or ledger, side of things. It gets far more interesting when computer code is managed in that way too. Let’s imagine a legal contract: Certain actions are taken under certain conditions. Even after the parties sign, they must still rely on the good faith of the other or our justice system to carry out their side of the agreement. Let’s take an example. Donald hates flight delays. AIGore Insurance tells him if he pays them $5 and his flight is delayed by more than an hour, AIGore will return his $5 and pay him an additional $20. A simple insurance scheme, or perhaps it’s a bet. In any case, when Donald gives AIGore $5 he has to trust them that they will carry out their end of the bargain. However, by using a blockchain he can eliminate this risk. Collectively, they write the conditions of their agreement in computer code, and initialize the contract with enough funds to make good on either side: Donald sends $5 worth of cryptocurrency, and AIGore Insurance sends $20. Then an hour after Donald’s flight is scheduled to arrive, the computer code contract will do the following:

1.   Look up Donald’s flight on www.flightstats.com.

2.   If it was delayed more than an hour, send Donald $25.

3.   Otherwise, send AIGore Insurance $25.

This code, once it’s written to the blockchain, cannot be removed or altered. Neither party can unilaterally remove the money. Donald and AIGore are guaranteed that the terms of the contract will be executed. In the Ethereum blockchain, this is termed a smart contract; much more on that as we proceed. We will examine the code for an application that does just that. See http://fdd.etherisc.com/ for details on how this type of insurance is implemented using the blockchain.

Blockchain: An Information Technology

As mentioned, a blockchain is a distributed ledger of transactions implemented as data batched into blocks that use cryptographic validation to link the blocks together. Each block references and identifies the previous block using a hashing function which forms an unbroken chain (i.e., blockchain).

A public blockchain is not stored in one central computer. Nor is it managed by any central entity. Instead, it is distributed and maintained by multiple computers or nodes that compete to validate the newest block entries before the other nodes to gain a reward for doing so.

The block validation system is designed to be immutable. That is to say, all transactions old and new are preserved forever with no ability to delete. Anyone on the network can browse via a designated website and see the ledger. This provides a way for all participants to have an up-to-date ledger that reflects the most recent transactions or changes. In this way, blockchain establishes trust, which as we shall see facilitates transactions and brings many cost-saving efficiencies to all types of transactional interactions.

A Distributed Trusted Information Technology

From a technical point of view, the blockchain is a distributed, transparent, immutable, validated, secured, and pseudo-anonymous database existing as multiple nodes such that if 51 percent of the nodes agree then trust of the chain is guaranteed. The blockchain is distributed because a complete copy lives on as many nodes as there are in the system. The blockchain is immutable because none of the transactions can be changed. The blockchain is validated (e.g., in the Bitcoin space) by the miners who are compensated for building the next secure block. The blockchain is pseudo-anonymous because the identity of those involved in the transaction is represented by an address key in the form of a random string.

That said, this is an evolving space and, like the cloud computing paradigm, there are public, private, and even hybrid blockchains, which we will explore in detail later in this chapter. These blockchain variations on the basic theme are the result of enterprise architects looking to implement blockchain applications to save time and fees. Enterprise requirements around scaling, performance, the need to know the identity of those involved in the transaction, and other things provide its emerging variations. Blockchain evangelists reckon distributed ledger technology has the potential to upend centralized database practices in institutional finance and most other transaction-based technology. In 2017, the technology shifted from hype to commercial reality. For blockchain to succeed, the application development life cycle, which facilitated large web applications using tools like HTML, CSS, JavaScript, REST web services, Java, SQL, and NOSQL data stores, will have to be amended to integrate the blockchain onto that stack. We will need integrated development environments (IDEs) and continuous integration and testing procedures to move applications and their attendant code from development to QA and ultimately reliable production implementations. Additionally, because blockchains cannot access data outside their network on their own, third-party services (known as oracles, agents, or data feeds) will also need to be integrated. These oracles or agents typically access and verify real-world occurrences and submit this information to a blockchain to be used by smart contracts. They provide external data when needed and push it onto the blockchain. Such conditions could be anything, such as the flight delay information we saw in the insurance example. The blockchain will have to operate efficiently, scale well, handle the know-your-client (KYC) process, create the aforementioned oracles or APIs that produce and consume off-chain services to verify events and data and handle/convert real-world money to and from cryptocurrencies, and integrate well with different chains. This is all in progress, and we will explore some of these IDEs and development processes in detail as we proceed.

Implementation Trends

A lot has changed since blockchain first emerged as the technology underpinning the cryptocurrency Bitcoin as a distributed ledger of transactions and asset ownership that is maintained by a network of computers over the Internet. More proof of its ability to reduce costs and speed up post-trade processes has emerged in the past year. We will explore this in detail.

A key factor to its rapid growth is the backdrop against which it has launched. Financial institutions and infrastructures are under pressure both to comply with regulation and to reduce cost. That pressure coincided with this technology coming into existence. It’s the intersection of requirement and opportunity that’s causing the rapid growth. The technology will make moves from “proof-of-concept” technology into production, especially in cross-border payments and trade finance. In this book, we will examine the development life cycle that is emerging with the blockchain. Having been developers for over 30 years, we have witnessed lots of changes in technology starting from the IBM Assembler/COBOL days. We like to kid our colleagues with the fact that we may have worked on the first blockchain.

Back in 1974, no Wall Street firm had its own computer. All processing was done by the famous payroll firm Automatic Data Processing (ADP), located a few blocks away from the heart of Wall Street. The licensed securities exchange firms would drop their stack of punch cards containing the transactions into a designated dropbox. They also dropped off one of their master data magnetic tapes containing the sort of “blockchain”—that is, all transactions to date for that particular organization in sequential order stored in IBM’s QSAM format. The programs written in Fortran, COBOL, or IBM assembler could only read data sequentially front to back. The cards representing the transactions would be added to the end of the existing chain of QSAM records, creating/writing a new tape file and hence a new state of the database or simple blockchain. Next came the emergence of early database technology like IBM IMS, IDMS, and ADABAS, followed by the dawn of the ever-enduring SQL in the mid-1980s. Then the open source revolution led to the Linux/Python/Java/SQL/NOSQL/HTML/JavaScript technology stack where web development and database development have matured to create the big data/web-enabled world we live in. Blockchain will further disrupt this evolution to bring trust firmly back into the picture. That said, it will have to be integrated into this existing development paradigm. These changes are emerging. Early adopters of blockchain used first-generation IDEs to develop applications and, as we shall see, JavaScript-type languages for things like smart contracts, to be discussed in a later chapter. Moreover, integrating blockchain with existing applications will also present challenges that we will examine.

As further impetus to the rise of blockchain, we expect 2018 will be dominated by last-minute preparations for the incoming revised European Union Revised Payment Services Directive (PSD2) effective in May of that year. As PSD2 becomes implemented, banks’ monopoly on their customers’ account information and payment services will disappear. PSD2 enables bank customers—both consumers and businesses—to use third-party providers to manage their finances. That means you may be using Google to pay your bills, make transfers, and analyze your spending. Banks will provide these third parties access to their customers’ accounts through open APIs (application program interfaces). The EU directive opens the door to any interested company, with provisions that will make it easier for startups to access data from banks. This will allow the startups to use blockchain to better penetrate some functions currently performed by banks. With the creation of open banking platforms, there will be opportunities for financial technology (fintech) firms to partner with banks to create new wave customer experiences and provide increased transparency on performance and fee structures. We believe that 2018 is the year that fintech catches up with the hype: validating blockchain-based technologies with promise, scale, customers, and adoption. We are all seeing signs that are consistent with a technology going mainstream in the next five years and changing the transactional landscape for many more years to follow.

Trust: The Byzantine Generals Problem

Back in early days of business computing, circa 1980, computer scientists began to examine reliability and computers. It was determined that a reliable computer system must be able to cope with the failure of one or more of its components. A failed component may exhibit a type of behavior that is overlooked and problematic, namely, sending conflicting information to different parts of the system. The problem of coping with this type of failure is expressed abstractly as the Byzantine Generals Problem, from the 1982 scholarly paper by Leslie Lamport, Robert Shostak, and Marshall Pease (www-inst.eecs.berkeley.edu/~cs162/fa12/hand-outs/Original_Byzantine.pdf). This abstract problem and the solutions thereof are used in developing highly reliable and trusted blockchain implementations.

The Byzantine Generals Problem Explained: Why Trust Is So Important

The Byzantine Generals Problem (BGP) is one of many in the field of agreement protocols. Lamport framed his paper around a story problem. This was the style of the day as evidenced by the attention received by another computer scientist, Edsger Dijkstra and his dining philosophers problem, based on a classic operating system synchronization problem.

To set the table for this problem—and as the musical Hamilton will be on Broadway long after this book is out of print—we see that the BGP problem was relevant in the American Revolution. At the start of the conflict, the Continental Congress ordered an oath of allegiance to be administered to all army officers. George Washington, the commander-in-chief, administered it to the general officers. When Washington began to read the oath to Major General Charles Lee, Lee withdrew his hand from the Bible. When Washington demanded a reason for the strange conduct, according to The Writings of George Washington, Lee replied, “As to King George, I am ready enough to absolve myself from all allegiance to him; but I have some scruples about the Prince of Wales.” This odd reply elicited much laughter. Lee was then playing a desperate game of treason, and probably had some problems with his conscience about taking such an oath which he (and later Major General Benedict Arnold) would violate.

The BGP is built around a similar story line: the commanding general who makes a decision to attack or retreat, and must communicate the decision to his lieutenant generals. A given number of these actors are traitors (possibly including the general). Traitors cannot be relied upon to properly communicate orders; worse yet, they may actively alter messages in an attempt to subvert the process.

In the analogy, the generals are collectively known as processes, the general who initiates the order is the source process, and the orders sent to the other processes are messages. Traitorous generals and lieutenant generals are faulty processes, and loyal generals and lieutenant generals are correct processes. The order to retreat or attack is a message with a single bit of information: a one or a zero.

A solution to an agreement problem must pass three tests: termination, agreement, and validity. As applied to the Byzantine Generals Problem, these three tests are:

1.   A solution has to guarantee that all correct processes eventually reach a decision regarding the value of the order they have been given.

2.   All correct processes have to decide on the same value of the order they have been given.

3.   If the source process is a correct process, all processes must decide on the value that was original given by the source process.

The best way we know to implement a reliable trustworthy computer system (e.g., blockchain) is to use many different processors to compute the same result, and then to perform a majority vote on their outputs to obtain a single value. See Figure 1-4, which views the BGP and blockchain issues in a side-by-side analogy.

FIGURE 1-4   BGP and blockchain compared

This is true whether one is implementing a blockchain using distributed nodes to protect against the failure of reaching a consensus on the next block, or a missile defense system using redundant computing sites to protect against the destruction of individual sites by a nuclear attack. The only difference is in the size of the replicated processor. The use of majority voting to achieve reliability is based upon the assumption that all the nonfaulty processors will produce the same output. This is true so long as they all use the same input. However, any single input message comes from a single physical component, and a malfunctioning component can give different values to different processors. Moreover, different processors can get different values even from a nonfaulty input unit if they read the value while it is changing. The solution is a trust mechanism of value verification and acceptance for each input/transaction recorded by a majority of processors and synchronization of the input transaction across the distributed processors. The quality of trust is a foundational element of business. Trust, particularly in the global economy where every link in the transaction requires a separate ledger, is expensive, time-consuming, and inefficient. The application of blockchain as it matures will provide a viable alternative to the current procedural, organizational, and technological infrastructure required to create institutionalized trust.

Byzantine Fault Tolerance in Use Today: Why Airplanes Are Safe

Byzantine fault tolerance (BFT) refers to the aforementioned BGP. One example of BFT in use is Bitcoin. The Bitcoin network works in parallel to generate a chain of hashcash-style proof-of-work. The proof-of-work chain is the key to overcoming Byzantine failures and reaching a coherent global view of the system state.

Some aircraft systems, such as the Boeing 777 Aircraft Information Management System (via its ARINC 659 SAFEbus® network), the Boeing 777 flight control system, and the Boeing 787 flight control systems, use Byzantine fault tolerance (see Figure 1-5). Because these are real-time systems, their Byzantine fault tolerance solutions must have very low latency. For example, SAFEbus can achieve Byzantine fault tolerance on the order of a microsecond of added latency.

FIGURE 1-5   BFT and airplane safety ARINC 659 SAFEbus

Some spacecraft, such as the SpaceX Dragon flight system, consider Byzantine fault tolerance in their design. Byzantine fault tolerance mechanisms use components that repeat an incoming message (or just its signature) to other recipients of that incoming message. All these mechanisms make the assumption that the act of repeating a message blocks the propagation of Byzantine symptoms. For systems that have a high degree of safety or security criticality, these assumptions must be proven to be true to an acceptable level of fault coverage. When providing proof through testing, one difficulty is creating a sufficiently wide range of signals with Byzantine symptoms. Such testing likely will require specialized fault injectors.

Satoshi Nakamoto’s Blockchain Breakthrough

Satoshi Nakamoto is the name used by the unknown person or persons who designed Bitcoin and created its original reference implementation, Bitcoin Core. As a part of the implementation, they also devised the first blockchain database and solved the double-spending problem for digital currency. They were active in the development of Bitcoin up until December 2010.

Satoshi Nakamoto: The Man, the Myth, the Mystery

If the mystery is of interest, we suggest you view the film Banking on Bitcoin. The film reviews the life stories and public misconceptions surrounding the cryptocurrency’s rise. The movie is well paced and informative, using interviews with financial columnists from the Wall Street Journal and the New York Times, early adopting Bitcoin entrepreneurs like Charlie Shrem and Erik Voorhees, and establishment figures such as the Winklevoss brothers and former New York State Superintendent of Financial Services Benjamin Lawsky. It is a pretty good tour of the history of cryptographic technology. Like lots of new technology, blockchain and Bitcoin are the work of a small group of coders known as cypherpunks. They contributed to the ideas that became the building blocks of Bitcoin. In the mid-1990s, just a handful of people had the knowledge necessary to develop a blockchain currency. The movie explores theories that Satoshi Nakamoto, the unknown creator of Bitcoin, may well have been one the original cypherpunks as he is rumored to have lived within a few city blocks of other cypherpunks, such as cryptographer Hal Finney.

The narrative contrasts the differences between centralized banking systems and the public ledger at the heart of Bitcoin that removes the need for a central authority. It highlights Bitcoin’s power to reduce remittance fees, serve the two and a half billion people on the planet who remain unbanked, and put financial control back into the hands of the individual.

Erik Voorhees, the founder of ShapeShift, communicates this sentiment in one of the film’s first interviews: “I discovered Bitcoin’s power when I understood it was not controlled by a central company or central person. I knew that meant it couldn’t be shut down. And if it can’t get shut down, all it needs is to do something useful, and it will become more and more adopted.”

Nakamoto has claimed to be a man living in Japan, born in 1975. However, speculation about the true identity of Nakamoto has mostly focused on a number of cryptography and computer science experts of non-Japanese descent, living in the United States and Europe. It became a bit of “I’m Spartacus” as Australian programmer Craig Steven Wright has claimed to be Nakamoto, though he has not yet offered proof of this. As of February 2017, Nakamoto is believed to own up to roughly 1 million bitcoins (valued at about $4 billion USD) but has never spent even a single BTC.

Satoshi Nakamoto: Timing Is Everything

Curiously, Bitcoin’s emergence was around the same time as the financial crisis of 2008. According to https://bitcoin.org, a purely peer-to-peer version of electronic cash allows online payments to be sent directly from one party to another without going through a central financial institution. Digital signatures provide part of the solution, but the main benefits are lost if a trusted third party is still required to prevent double-spending. Bitcoin is a solution to the double-spending problem using a peer-to-peer network. The network timestamps transactions by hashing them into an ongoing chain of hash-based proof-of-work, forming a record that cannot be changed without redoing the proof-of-work. The longest chain not only serves as proof of the sequence of events witnessed, but proof that it came from the largest pool of CPU power. As long as the majority of CPU power is controlled by nodes that are not cooperating to attack the network, they’ll generate the longest chain and outpace attackers. The network itself requires minimal structure. Messages are broadcast on a best-effort basis, and nodes can leave and rejoin the network at will, accepting the longest proof-of-work chain as proof of what happened while they were gone. If people lose faith in a currency, the typical reaction is to start using another one.

Traditionally, money has moved to the most stable currency, which has typically been the US dollar. But Bitcoin has a couple of advantages. The first advantage is that it is not controlled by any central authority. In countries where people are distrustful of how central banks and governments manage the economy, Bitcoin may seem like a more sensible alternative.

The second is that bitcoins may be easier to obtain than other fiat currencies. It can be bought and sold via Bitcoin exchanges online but also in direct transactions via websites. Evidence suggests that during times of crisis, people are looking to Bitcoin as an alternative to their own problematic currencies. As the Greek debt crisis unfolded, Bitcoin exchanges reported an increase in volume as people traded the cryptocurrency around the world. The price of bitcoins also rose significantly as the Greece crisis deepened, lending further credence to the idea of Bitcoin as a “panic” currency.

Blockchain: Underpinning of Cryptocurrency

As we all know by now, blockchain provides the technology underpinnings of Bitcoin, which has been the subject of much interest and speculation within the technical, business, and law enforcement communities. It got a bad rap when Bitcoin became the exchange currency for dark-web sites like Silk Road. According to Coinmap (http://coinmap.org), a crowdsourced website that tracks businesses that accept Bitcoin as a method of payment, the number of such businesses is on a constant and growing rise. While the revenues from Bitcoin are still a fraction of overall revenue, wider adoption of Bitcoin and other cryptocurrencies is pretty much a given, especially as the financials see the savings of time and money gained in global transactions. For example, Rand Merchant Bank research (https://news.bitcoin.com/south-africa-bank-blockchain-40-revenue/) found that cryptocurrencies could make up to 40 percent of banks’ revenue if they become a global standard. Both IBM and Microsoft offer their own versions of Blockchain-as-a-Service (BaaS) as part of their cloud platforms, and Donald Tapscott in his book BlockChain Revolution says that “blockchains, the technology underpinning the cryptocurrency, have the potential to revolutionize the world economy.”

Types of Blockchain

As people began to understand how blockchain works, they started using it for other purposes: as data storage for things of value, identities, agreements, property rights, and a host of other things. Ethereum, which will be one of the main focuses of this book, is to date the most comprehensive blockchain innovation after Bitcoin. Like cloud computing implementations, different types or categories of blockchain have emerged. Analogous to the cloud, you have public blockchains that everyone can access and update, you have private blockchains for just a limited group within an organization to be able to access and update, and you have a third kind, a consortium of blockchains that are used in collaboration with others. While working on Wall Street, we saw this consortium type of arrangement as very common between five of the larger investment banking firms. The consortium facilitated trades at an institutional level among the members, so it makes sense that blockchain as a financial technology tool would emerge in this way. The following sections are a quick exploration of each blockchain type.

Public Blockchains

A public blockchain is one that initial creators envisioned as: a blockchain for all to be able to access and transact with; a blockchain where transactions are included if and only if they are valid; a blockchain where everyone can contribute to the consensus process. As discussed, the consensus process determines what blocks get added to the chain and what the current state is. On the public blockchain, instead of using a central server the blockchain is secured by cryptographic verification supported by incentives for the miners. Anyone can be a miner to aggregate and publish those transactions. In the public blockchain, because no user is implicitly trusted to verify transactions, all users follow an algorithm that verifies transactions by committing software and hardware resources to solving a problem by brute force (i.e., by solving the cryptographic puzzle). The miner who reaches the solution first is rewarded, and each new solution, along with the transactions that were used to verify it, forms the basis for the next problem to be solved. The verification concepts are proof-of-work or proof-of-stake.

Consortium Blockchains

A consortium blockchain such as R3 (www.r3cev.com) is a distributed ledger where the consensus process is controlled by a preselected set of nodes—for example, a consortium of nine financial institutions, each of which operates a node, and of which five (like the US Supreme Court) must sign every block in order for the block to be valid. The right to read the blockchain may be public or restricted to the participants, and there are also hybrid routes such as the root hashes of the blocks being public together with an API that allows members of the public to make a limited number of queries and get back cryptographic proofs of some parts of the blockchain state. These sort of blockchains are distributed ledgers that may be considered “partially decentralized.”

Private Blockchains

A fully private blockchain is a blockchain where write permissions are kept centralized to one organization. Read permissions may be public or restricted to an arbitrary extent. Likely applications include database management and auditing internal to a single company, so public readability may not be necessary in many cases at all, though in other cases public auditability is desired. Private blockchains could provide solutions to financial enterprise problems, including compliance agents for regulations such as the Health Insurance Portability and Accountability Act (HIPAA), anti–money laundering (AML), and know-your-customer (KYC) laws. The Hyperledger project from the Linux Foundation and the Gem Health network are private blockchain projects under development. See Chapter 8 for a detailed description of Hyperledger and other private and consortium blockchain technology.

Comparing Blockchains

The distinction between public, consortium, and private blockchains is important. Even for “old school” distributed ledger adopters who prefer a traditional centralized system, they still get the addition of cryptographic auditability attached. As compared to public blockchains, private blockchains have a number of advantages. The private blockchain operator can change the rules of a blockchain. If it is a blockchain among financial partners, then where errors are discovered they will be able to change transactions. Likewise, they will be able to modify balances and generally undo anything. That said, there is a trail. In some cases, this functionality is necessary, as with property registry if a mistaken transaction is issued or some nefarious type has gained access and made themselves the new owner. This is also true on a public blockchain if the government has backdoor access keys like they did in the Clinton era. On the private blockchain, transactions are less expensive, since they only need to be verified by a few nodes that can be trusted to have very high processing power. Public blockchains tend to have more expensive transaction fees, but this will change as scaling technologies emerge and bring public-blockchain costs down to create an efficient blockchain system.

Nodes can be trusted to be very well connected, and faults can quickly be fixed by manual intervention, allowing the use of consensus algorithms that offer finality after much shorter block times. Improvements in public blockchain technology, such as Ethereum’s proof-of-stake, can bring public blockchains much closer to the “instant confirmation” ideal, but private blockchains will always be faster, and the latency difference will never disappear as unfortunately the speed of light does not increase by 2x every two years like Moore’s law. If read permissions are restricted, private blockchains can provide a greater level of privacy.

Given all of this, it may seem like private blockchains are unquestionably a better choice for institutions. However, even in an institutional context, public blockchains still have a lot of value. In fact, this value lies to a substantial degree in the philosophical virtues that advocates of public blockchains have been promoting all along, among the chief of which are freedom, neutrality, and openness. The advantages of public blockchains generally fall into two major categories:

•   Public blockchains provide a way to protect the users of an application from the developers, establishing that there are certain things that even the developers of an application have no authority to do.

•   Public blockchains are open, and therefore used by many entities, This provides some networking effects. If we have asset-holding systems on a blockchain, and a currency on the same blockchain, then we can cut costs to near-zero with a smart contract: Party A can send the asset to a program which immediately sends it to Party B which sends the program money, and the program is trusted because it runs on a public blockchain. Note that in order for this to work efficiently, two completely heterogeneous asset classes from completely different industries must be on the same database. This can also be used by other asset holders such as land registries and title insurance.

Blockchain Implementations

The concept of decentralized digital currency, as well as alternative applications like property registries, has been around for decades, but none has produced viable production implementations until now. The anonymous e-cash protocols of the 1980s and 1990s were mostly reliant on a cryptographic primitive known as Chaumian blinding (after its developer, David Chaum). Chaumian blinding provided these new currencies with high degrees of privacy, but their underlying protocols largely failed to gain traction because of their reliance on a centralized intermediary. In 1998, Wei Dai’s b-money became the first proposal to introduce the idea of creating money through solving computational puzzles as well as decentralized consensus, but the proposal was scant on details as to how decentralized consensus could actually be implemented. In 2005, Hal Finney introduced a concept of “reusable proofs of work,” a system that uses ideas from b-money together with Adam Back’s computationally difficult Hashcash (http://hashcash.org) puzzles to create a concept for a cryptocurrency, but this once again fell short of the ideal by relying on trusted computing as a backend. As we all know, the blockchain concept was implemented as a core component of the digital currency Bitcoin. This critical and perhaps first production implementation of the blockchain made it the first digital currency to solve the double-spending problem, without the use of a trusted authority or central server. The Bitcoin design, which we examine briefly in the next section, has been the inspiration for other implementations we will explore in the chapters to come.

Bitcoin

As we mentioned, when the financial crisis of 2008 was in full throttle, Bitcoin (BTC), a decentralized currency, was implemented for the first time in practice by Satoshi Nakamoto. Bitcoin combines established primitives for managing ownership through public key cryptography with a consensus algorithm for keeping track of who owns coins, known as proof-of-work. The mechanism behind proof-of-work simultaneously solves two problems. First, it provides an effective consensus algorithm, allowing nodes in the network to collectively agree on a set of updates to the state of the Bitcoin ledger. Second, it provides a mechanism for allowing free entry into the consensus process, solving the political problem of deciding who gets to influence the consensus, while simultaneously preventing Sybil attacks—that is, attacks where a reputation system is subverted by forging identities in peer-to-peer networks. It is named after a case study of a woman diagnosed with dissociative identity disorder. It works by substituting a formal barrier to participation, such as the requirement to be registered as a unique entity on a particular list, with an economic barrier—the weight of a single node in the consensus voting process is directly proportional to the computing power that the node brings. More recently, an alternative approach has been proposed called proof-of-stake, calculating the weight of a node as being proportional to its currency holdings and not its computational resources. The discussion concerning the relative merits of the two approaches will be examined in the chapters that address the Ethereum-based blockchain and derivatives thereof. At this junction in 2018, all blockchain platforms are still evolving and will continue to do so for the foreseeable futures. As Bitcoin is the most widely used, we will explore it in some detail in the next sections. For in-depth details for developers, see https://bitcoin.org/en/developer-documentation.

Bitcoin State Transition

From a technical standpoint, the ledger of a cryptocurrency such as Bitcoin can be thought of as a state transition system, where there is a state S consisting of the ownership status of all existing bitcoins (or any asset for that matter) and a state transition function—that is, the API: EXECTX, which takes a state S and a transaction TX and outputs a new state S’ which is the result. In a standard banking system, for example, the state is a balance sheet, a transaction is a request to move $cash money from A to B, and the state transition function reduces the value of A’s account by X amount of $cash money and increases the value of B’s account by X amount of $cash money. If A’s account has less than X amount of $cash money in the first place, the state transition function returns an error. We define an API:

EXECTX(S,TX) results in S’ (new state) or ERROR and S (no change to state)

If A has enough $cash money:

EXECTX({ A:$1000, B:$500},“send $500 :A to B”) results in { A:$500, B:$1000 }

But if A does not have enough $cash money:

EXECTX({ A:$1000, B:$500 },“send $1001 from A to B”) results in ERROR

The state in a blockchain is the “consensus view” of all transactions at any given moment borne out by the existing authenticated ledger distributed among all nodes. In the world of Bitcoin, it is the collection of all unspent transaction outputs (UTXOs) that have been minted and not yet spent, with each UTXO having a denomination and an owner (defined by a 20-byte address which is essentially a cryptographic public key). With respect to UTXOs, because each output of a particular transaction can only be spent once, the outputs of all transactions included in the blockchain can be categorized as either unspent transaction outputs (see https://bitcoin.org/en/glossary/unspent-transaction-output) or spent transaction outputs. For a payment to be valid, it must only use UTXOs as inputs.

If the value of a transaction’s outputs exceed its inputs, the transaction will be rejected. But if the inputs exceed the value of the outputs, any difference in value may be claimed as a transaction fee by the Bitcoin miner who creates the block containing that transaction.

A transaction contains one or more inputs, with each input containing a reference to an existing UTXO and a cryptographic signature produced by the private key associated with the owner’s address, and one or more outputs, with each output containing a new UTXO for addition to the state.

The state transition function EXECTX(S,TX) -> S’ can be defined as follows:

For each input in TX:

1.   If the referenced UTXO is not in S, return an error; this prevents transaction senders from spending coins that do not exist.

2.   If the provided signature does not match the owner of the UTXO, return an error; this prevents transaction senders from spending other people’s coins.

3.   If the sum of the denominations of all input UTXO is less than the sum of the denominations of all output UTXO, return an error.

4.   Return S’ with all input UTXO removed and all output UTXO added.

That is a simple view of the transaction flow for Bitcoin.

Bitcoin Mining

Bitcoin combines the state transition system with a consensus system in order to ensure that everyone agrees on the order of transactions. Bitcoin’s decentralized consensus process requires nodes in the network to continuously attempt to produce blocks, i.e., 1 to N transactions. The Bitcoin network is intended to create one block approximately every 10 minutes, with each block containing a timestamp, a nonce, a reference to (i.e., hash of) the previous block, and a list of all transactions that have taken place since the previous block. Every block in the Bitcoin network has the exact same structure as shown in Figure 1-6.

FIGURE 1-6   Bitcoin blockchain schema

Each newly created block is “chained” to the last added block of the blockchain and stores its digital fingerprint. Let us examine the fields of a block, with byte sizes subject to change:

•   Block identifier (4 bytes): This is an identifier for the Blockchain network. It has a constant magic number value of 0xD9B4BEF9. The magic number is not something specific to Bitcoin. It identifies the type of the file or data structure you are consuming. The consumer can check the magic number and immediately know the supposed type of that file or data structure. In this case, it indicates the start of the block, and the data is from a production network.

•   Next block identifier (4 bytes).

•   Block size (4 bytes): Indicates how large the block is. Since the very beginning, each block has been fixed to 1 MB. This will be increased to 2 MB. The maximum capacity is 2 GB, so the scalability factor has already been taken care of.

•   Block version (4 bytes): Each node running the Bitcoin protocol has to implement the same version and it is mentioned in this field.

•   Previous block hash (32 bytes): This is a digital fingerprint (hash) of the block header of the previous (last added) block of the blockchain. It is calculated by taking all the fields of the header (version, nonce, etc.) together and applying a cryptographic function (SHA-256) twice by rearranging the bytes of the individual fields (little-endian format).

•   Block Merkle root (64 bytes).

•   Block timestamp (8 bytes).

•   Nonce (4 bytes).

The block header is composed of the fields from Version to Nonce.

•   Transaction counter (variable: 4 bytes): This is the count of transactions that are included with the block.

•   Transaction list (variable: total blocksize is 1 MB): Stores the digital fingerprint of all the transactions in that block. Each individual transaction has its own structure.

You can also see the height of the block (aka the count of blocks) since the first block was created, and genesis block, the first block that was mined.

Bitcoin Blocksize and Segregated Witness

So a block has a maximum file size of 1 MB. When this block’s space capacity is full, another block is created and added up in the blockchain. As the number of Bitcoin transactions is increasing, more blocks are created. This is stressing the Blockchain network and causing delays in transactions confirmation. Additionally, more mining means higher transaction. To address this the Segregated Witness (“SegWit “) was proposed. So let’s review—each Bitcoin transaction contains three elements:

1.   Input (sender details)

2.   Output (recipient details)

3.   Digital Signature: this signature is called the witness, one who verifies that the sender has the right amount of balance to make the transaction

All of these elements are needed to add the transaction to a block. Of the three elements, the file size of Digital Signature is the largest, making the transactions heavy in terms of size. As the maximum capacity of each block at present is 1 MB, therefore more heavy transactions equals less transactions getting added to the block for confirmation. SegWit proposes to remove the Digital Signature Element from the transaction and add it to another new block called Extended Block. This means that any transaction that gets added in the Block for confirmation will only contain Input and Output and not the Digital Signature. This will make transactions lighter. As a result more space is available in the block, which means more transactions can be added into the block. As more transactions will get verified in the same amount of time, the Bitcoin network will be faster. Thus, we are removing the Witness—the Digital Signature—segregating it to another block (hence the name “Segregated Witness”).

So, the advantages of SegWit are:

•   it will reduce the file size of transactions,

•   there will be faster confirmation of transactions, and

•   transaction fees will be lower.

Thus, SegWit will improve the Bitcoin network scaling ability. Moreover, consensus is not required to make SegWit work. SegWit even works where users do not upgrade their software versions to the newest version. That said, we will make numerous blockchain software improvements as it gains traction in the application development world.

Bitcoin and Merkle Root

Each block contains a list of all transactions. Once the block is part of the blockchain it is an immutable record, i.e., the transaction entry in it is permanent. It also means that if a transaction is present in one block it will not be present in any other block of the blockchain. The transactions are listed as Merkle tree or a binary hash tree. It is a very popular data structure used in programming languages.

The root of the tree is the topmost node. The nodes at the bottom are called leaf nodes. Each node is simply a cryptographic hash of a transaction. The Merkle tree does not contain a list of all the transactions, but rather a hash (digital fingerprint) of all transactions as a tree structure (see Figure 1-7).

FIGURE 1-7   Merkle tree

Hash of Transaction 0 = Hash[Tx(0)] = SHA256 (SHA 256 (Transaction A))
Each hash is calculated by applying the SHA-256 algorithm twice.

Similarly, to construct a parent node Hash(01), the 32-byte Hash[Tx(0)] and 32-byte Hash[Tx(1)] is concatenated as a 64-byte hash string and then SHA-256 is applied twice to give a 32-byte Hash(01).

This concept can be further expanded to any size. The biggest advantage is that it is very easy and highly efficient to determine whether a particular transaction has been included within a block (since the block contains the Merkle root—which is a digital fingerprint of all transactions contained in it).

Bitcoin and Secure Hashing

SHA stands for secure hash algorithm. It is used to prove data integrity. The same input(s) will always produce the exact same output. This output is always 256 bits or 32 bytes in length regardless of the length of the input (even if input is millions of bytes). Any change in the input(s) will result in a change of output. The same output can never be derived from different input(s). However, from the output we can never determine the inputs, which is why this is highly secure. You can test it yourself at a few online SHA-256 tools (such as www.xorbin.com/tools/sha256-hash-calculator). The input can be any string, even concatenating many strings. Regardless of the input the output remains 256 bits.

Over time, this creates a persistent, ever-growing blockchain that continually updates to represent the latest state of the Bitcoin ledger.

The algorithm for checking if a block is valid, expressed in this paradigm, is as follows:

1.   Check if the previous block referenced by the block exists and is valid.

2.   Check that the timestamp of the block is greater than that of the previous block.

3.   Check that the proof-of-work on the block is valid.

Let S[0] be the state at the end of the previous block.

Suppose TX is the block’s transaction list with n transactions.

For all i in 0...n-1, set S[i+1] = EXECTX({ (S[i],TX[i]) }.

If any application returns an error, exit and return false.

Return true, and register S[n] as the state at the end of this block.

Essentially, each transaction in the block must provide a valid state transition from what was the canonical state before the transaction was executed to some new state. Note that the state is not encoded in the block in any way; it is purely an abstraction to be remembered by the validating node and can only be computed for any block by starting from the genesis state and sequentially applying every transaction in every block. Additionally, note that the order in which the miner includes transactions into the block matters; if there are two transactions A and B in a block such that B spends a UTXO created by A, then the block will be valid if A comes before B but not otherwise.

The one validity condition present is the requirement for proof-of-work. The precise condition is that the double SHA-256 hash of every block, treated as a 256-bit number, must be less than a dynamically adjusted target. The purpose of this is to make block creation computationally hard, thereby preventing Sybil attackers from remaking the entire blockchain in their favor. Because SHA-256 is designed to be a completely unpredictable pseudorandom function, the only way to create a valid block is simply trial and error, repeatedly incrementing the nonce and seeing if the new hash matches.

In order to better understand the purpose of mining, let us examine what happens in the event of a malicious attack. Since Bitcoin’s underlying cryptography is known to be secure, the attacker will target the one part of the Bitcoin system that is not protected by cryptography directly: the order of transactions. The attacker’s strategy is a simple double-spend:

1.   Send 1,000 BTC to a merchant in exchange for some product (preferably a rapid-delivery digital good).

2.   Wait for delivery of the product.

3.   Produce another transaction sending the same 1,000 BTC to himself.

4.   Convince the network that his transaction to himself was the one that came first.

Once step 1 has taken place, after a few minutes some miner will include the transaction in a block. After about an hour, five more blocks will have been added to the chain after that block, with each of those blocks indirectly pointing to the transaction and thus “confirming” it. At this point, the merchant will accept the payment as finalized and deliver the product; since we are assuming this is a digital good, delivery is instant. Now the attacker creates another transaction sending the 1,000 BTC to himself. If the attacker simply releases it into the wild, the transaction will not be processed; miners will attempt to run EXECTX({ (S, TX) }) and notice that TX consumes a UTXO that is no longer in the state. So instead the attacker creates a “fork” of the blockchain, starting by mining another version of the block pointing to the same block as a parent but with the new transaction in place of the old one. Because the block data is different, this requires redoing the proof-of-work. Furthermore, the attacker’s new version of the block has a different hash, so the original blocks do not point to it; thus, the original chain and the attacker’s new chain are completely separate. The rule is that in a fork the longest blockchain is taken to be the truth. In order for the attacker to make his blockchain the longest, he would need to have more computational power than the rest of the network combined to catch up and effect the so-called “51% attack.”

Merkle Trees

An important scalability feature of Bitcoin is that the block is stored in a multilevel data structure. The “hash” of a block is actually only the hash of the block header, a roughly 200-byte piece of data that contains the timestamp, nonce, previous block hash, and the root hash of a data structure called the Merkle tree storing all transactions in the block. A Merkle tree is a type of binary tree, composed of a set of nodes with a large number of leaf nodes at the bottom of the tree containing the underlying data, a set of intermediate nodes where each node is the hash of its two children, and finally a single root node, also formed from the hash of its two children, representing the “top” of the tree. The purpose of the Merkle tree is to allow the data in a block to be delivered piecemeal: a node can download the header of a block from one source, the small part of the tree relevant to them from another source, and still be assured that all of the data is correct. The reason why this works is that hashes propagate upward. If a malicious user attempts to swap in a fake transaction to the bottom of a Merkle tree, this change will cause a change in the node above, and then a change in the node above that, finally changing the root of the tree and therefore the hash of the block, causing the protocol to register it as a completely different block and almost certainly with an invalid proof-of-work. The Merkle tree protocol is arguably essential to long-term sustainability.

Bitcoin Scripting

Even without any extensions, the Bitcoin protocol provides a less than robust version of the concept of smart contracts. UTXO in Bitcoin can be owned not just by a public key but also by a more complicated script expressed in a simple stack-based programming language. In this paradigm, a transaction spending that UTXO must provide data that satisfies the script. Indeed, even the basic public key ownership mechanism is implemented via a script: the script takes an elliptic curve signature as input, verifies it against the transaction and the address that owns the UTXO, and returns 1 if the verification is successful and 0 otherwise. Other, more complicated scripts exist for various additional use cases. For example, one can construct a script that requires signatures from two out of a given three private keys to validate (multisig), a setup useful for corporate accounts, secure savings accounts, and some merchant escrow situations. Scripts can also be used to pay bounties for solutions to computational problems, and one can even construct a script that says something like “this Bitcoin UTXO is yours if you can provide an SPV proof that you sent a Dogecoin transaction of this denomination to me,” essentially allowing decentralized cross-cryptocurrency exchange.

Namecoin

Created in 2010, Namecoin (http://namecoin.org) is a decentralized name registration database. In decentralized protocols like Tor, Bitcoin, and Bitmessage, there needs to be some way of identifying accounts so that other people can interact with them. Namecoin is the oldest and most successful implementation of a name registration system using such an idea. It is open-source technology which improves decentralization, security, censorship resistance, privacy, and speed of certain components of the Internet infrastructure such as DNS and identities. Namecoin is a key/value pair registration and transfer system based on the Bitcoin technology.

Ripple

Ripple (www.ripple.com) is seen as one of the most advanced distributed ledger technology (DLT) companies in the industry. It focuses on the using of blockchain-like technology for payments. The Ripple protocol has been adopted by an increasing number of financial institutions to offer alternative remittance options to consumers. Ripple even has obtained a virtual currency license from the New York State Department of Financial Services, making it one of the few companies with a BitLicense. As of 2017, Ripple is the third-largest cryptocurrency by market capitalization, after bitcoin and ether. Ripple is a financial real-time gross settlement solution, currency exchange, and remittance network using distributed ledger technology. Ripple is built upon a distributed open-source Internet protocol, consensus ledger, and native currency called XRP (ripples) enabling (cross-border) payments for retail customers, corporations, and other banks. The Ripple protocol, described as “basic (settlement) infrastructure technology for interbank transactions,” enables the interoperation of different ledgers and payment networks and brings together three aspects of modern payment solutions: messaging, settlement, and FX management. It allows banks and non-bank financial services companies to incorporate the Ripple protocol into their own systems and thereby allow their customers to use the service.

The protocol enables the instant and direct transfer of money between two parties. As such the protocol can circumvent the fees and wait times of the traditional correspondent banking system. Any type of currency can be exchanged, including USD, euros, RMB, yen, gold, airline miles, and rupees. Ripple has its own form of digital currency (dubbed XRP) in a manner similar to Bitcoin, using the currency to allow financial institutions to transfer money with negligible fees and wait time. One of the specific functions of XRP is as a bridge currency, which can be necessary if no direct exchange is available between two currencies at a specific time, for example, when transacting between two rarely traded currency pairs. Within the network’s currency exchange, XRP are traded freely against other currencies, and their market price fluctuates against dollars, euros, yen, bitcoins, etc.

Many financial companies have subsequently announced experimenting and integrations with Ripple. A host of major banks have adopted Ripple to improve their cross-border payments, and many have completed trial blockchain projects. These banking institutions—including Santander, UniCredit, UBS, Royal Bank of Canada, Westpac Banking Corporation, CIBC, and National Bank of Abu Dhabi, among others—view Ripple’s payment protocol and exchange network as a valid mechanism for offering real-time affordable money transfers.

Ethereum

Ethereum (https://ethereum.org) is a decentralized platform that runs smart contracts: applications that run exactly as programmed without any possibility of downtime, censorship, fraud, or third-party interference. If Bitcoin was blockchain 1.0, then Ethereum is blockchain 2.0 and beyond. See Figure 1-8.

FIGURE 1-8   Ethereum blockchain platform

Ethereum applications run on a custom-built blockchain, a shared global infrastructure that can move value around and represent the ownership of property. This enables developers to create markets, store registries of debts or promises, and move funds in accordance with instructions given long in the past (such as a will) or a futures contract without a middleman or counterparty risk. The project was bootstrapped via an ether presale in August 2014 by fans all around the world. It is developed by the Ethereum Foundation, a Swiss nonprofit, with contributions from great minds across the globe. We will explore Ethereum in great detail as it will be the focus and implementation of most of the development code we will present. From a developer’s point of view, Ethereum is the platform that’s most likely to succeed. In March 2017, the Ethereum Enterprise Alliance was formed. The list of participating organizations includes Microsoft, Intel, J.P. Morgan, BNY Mellon, and a host of others. Some startups have used derivatives of the Ethereum platform to produce collaborative and private blockchains that are not even concerned with cryptocurrency. The assets they transact with will be securities, insurance, title registrations, and so on.

Blockchain Collaborative Implementations

As large enterprises have started to take notice of how the blockchain technology could save them time and money, their interest in creating enterprise versions of the blockchain has led to the creation of some collaborative implementations. These collaborative implementations are growing in number. We will examine a few of the current ones. We believe these collaborations, although the subject of some widespread criticism, will lead to a mature blockchain technology equipped with interactive development environments (IDE), debugging techniques, deployment techniques, and all the trappings required for enterprise implementations.

Hyperledger

Hyperledger (https://www.hyperledger.org) is an open-source collaborative effort created to advance cross-industry blockchain technologies. It is a consortium of companies working together to develop standardized blockchain protocols. The project aims to develop open protocols and standards by providing a modular framework that supports different components for different uses. This would include a variety of blockchains with their own consensus and storage models, and services for identity, access control, and contracts. It is a global collaboration hosted by the Linux Foundation (see Figure 1-9).

FIGURE 1-9   Hyperledger project branches

Hyperledger Fabric is the IBM contribution, a distributed ledger technology (DLT) implementation for the enterprise, with capabilities including network security, scalability, confidentiality, and performance, in a modular blockchain architecture.

Hyperledger Burrow codebase is a contribution by Monax and is an Ethereum Virtual Machine–compatible blockchain. Hyperledger Sawtooth Lake codebase is a contribution by Intel. They have targeted some applications integrating Internet of Things (IoT) sensors to track data on a blockchain, recording data including ownership, position, location, temperature, humidity, motion, shock, and tilt. We will examine each of these branches in the “use case” chapters to follow.

Hyperledger’s reference architecture is depicted in Figure 1-10. These categories are a logical structure, not a physical depiction of partitioning of components into separate processes, address spaces, or (virtual) machines. Some of these components will be built from the ground up, some will use existing open-source code, and some will interface with existing services to fulfill the required functions.

FIGURE 1-10   Hyperledger’s reference architecture

Corda

Corda (www.corda.net) is a distributed ledger platform designed to record, manage, and automate legal agreements between business partners. It is a collaborative effort by R3, a group of more than 100 financial companies. The development team is made up of financial industry veterans, technologists, new tech entrepreneurs, and subject matter experts, bringing together expertise to reengineer the financial markets ecosystem. Corda is a distributed ledger made up of mutually distrusting nodes that would allow for a single global database that records the state of deals and obligations between institutions and people. This would eliminate much of the time-consuming manual effort currently required to keep disparate ledgers synchronized with each other. It would also allow for greater levels of code sharing than presently used in the financial industry, reducing the cost of financial services for everyone.

From the technical point of view, Corda is a decentralized database platform with the following novel features: New transaction types can be defined using JVM bytecode. Transactions may execute in parallel, on different nodes, without either node being aware of the other’s transactions. Nodes are arranged in an authenticated peer-to-peer network. All communication is direct. There is no blockchain. More likely, Corda could accurately be described as a messaging protocol. Transaction races are deconflicted using pluggable notaries. A single Corda network may contain multiple notaries that provide their guarantees using a variety of different algorithms. Thus, Corda is not tied to any particular consensus algorithm.

Data is shared on a need-to-know basis. Nodes provide the dependency graph of a transaction they are sending to another node on demand, but there is no global broadcast of all transactions. Bytecode-to-bytecode transpilation is used to allow complex, multistep transaction building protocols, called flows, to be modeled as blocking code. The code is transformed into an asynchronous state machine, with checkpoints written to the node’s backing database when messages are sent and received. A node may potentially have millions of flows active at once, and they may last days, across node restarts and even upgrades. Flows expose progress information to node administrators and users and may interact with people as well as other nodes. A flow library is provided to enable developers to reuse common flow types such as notarization, membership broadcast, and so on.

The data model allows for arbitrary object graphs to be stored in the ledger. These graphs are called states and are the atomic unit of data. Nodes are backed by a relational database. Data placed in the ledger can be queried using SQL as well as joined with private tables, thanks to slots in the state definitions that are reserved for join keys. The platform provides a rich type system for the representation of things like dates, currencies, legal entities, and financial entities such as cash, issuance, deals, and so on. States can declare a relational mapping and can be queried using SQL. Integration with existing systems is considered from the start. The network can support rapid bulk data imports from other database systems without placing load on the network. Events on the ledger are exposed via an embedded JMS compatible message broker. States can declare scheduled events. For example, a bond state may declare an automatic transition to an “in default” state if it is not repaid in time. Corda follows a general philosophy of reusing existing proven software systems and infrastructure where possible.

Blockchain in Practical Use Today

Blockchain technology has the potential to transform business operating models in the long term. Blockchain distributed ledger technology is foundational. The use of blockchains will garner significant efficiencies in global supply chains, financial transactions, asset ledgers, and decentralized social networking. Some blockchain applications, as we shall see in this section, are a disruptive innovation; they enable substantially lower-cost solutions to be created, thereby disrupting existing business models.

Blockchain in the Financial Technology Space

Having been in the financial technology (“fintech”) space before it was known as “fintech,” the growth of global transactions has required more and more advanced technology applications to speed up and reduce transaction costs for securities such as equities. Traditional trade processes within asset management can be slow, manual, cumbersome, and filled with risk when reconciling and matching—and they’re getting more complex with cross-border transactions and for nonstandard investment products, such as loans. Each party in the trade life cycle (e.g., broker dealers, intermediaries, custodians, clearing and settlement teams) currently keeps their own copy of the same record of a transaction, creating significant inefficiencies and room for error.

Unfortunately, a fair amount of trades have errors, requiring manual intervention and extending the time required to settle trades. Because it does not require an exchange to verify, clear, and settle security transactions (such as equities, repo, and leveraged loans), blockchain will save a large amount in fees and capital charges globally by moving to a shorter, and potentially customized, settlement window. Blockchain will eliminate significant fees across FX, commodities, and OTC derivatives. Blockchain technology could simplify and streamline this entire process, providing an automated trade life cycle where all parties in the transaction would have access to the exact same data about a trade. This would lead to substantial infrastructural cost savings, effective data management and transparency, faster processing cycles, minimal reconciliation, and the potential removal of brokers and intermediaries altogether.

Financial derivatives are the most common application of a smart contract, and one of the simplest to implement in code. The main challenge in implementing financial contracts is that the majority of them require reference to an external price ticker. For example, a very desirable application is a smart contract that hedges against the volatility of ether (or another cryptocurrency) with respect to the US dollar, but doing this requires the contract to know the value of ETH/USD or BTC/USD at any particular moment. They would have to leave the zone of trust. In practice, information issuers are not always trustworthy, and in some cases the banking infrastructure is too weak, or too hostile, for such services to exist. This approach is not fully decentralized, because a trusted source is still needed to provide the price ticker, although arguably this still is a massive improvement in terms of reducing infrastructure requirements (unlike being an issuer, issuing a price feed requires no licenses and can likely be categorized as free speech) and reducing the potential for fraud. The current industry trailblazers working to enable a smarter and more connected financial system by digitizing the world’s assets are Digital Asset Holdings (www.digitalasset.com), Chain.com, and the aforementioned Ripple.

Blockchain in the Sharing Economy

One thing we have noticed is that large organizations rely on information provided by users to generate value within their own platform. The problem with this platform is that the value produced by the crowd is not equally distributed among those who have contributed. Most of the profits are captured by the large intermediaries who operate the block. Uber and Airbnb could be obviated in a world where we store people’s online identities on the blockchain. Uber has disrupted the entire transportation industry. Some governments, however, have been quick to limit its reach in order to protect existing taxi companies. La`Zooz (http://lazooz.org) is an open-source and decentralized collaborative transportation system. More importantly, La`Zooz is a blockchain-based ride-sharing solution that rewards its users, developers, and drivers with tokens called zooz. Unlike Uber, La`Zooz has no central authority and cannot be blocked or shut down by governments. Your identity (as anonymous as you want it to be) could be linked to reviews in the “sharing economy” on the marketplace. People can check out your review as a trusted individual by checking your ID number. It would actually promote good behavior because if you get a bad reputation you cannot delete accounts and re-register. As we have mentioned, the blockchain can’t be tampered with or duplicated. Your identity and what you choose to expose can be valuable. Onename is a New York–based startup that has created an ID system using the Bitcoin blockchain. Its first service allows users to create blockchain IDs, which will soon be used to log in to websites without the need for a password.

P2P lodging sites have already begun to transform the lodging industry by making a public market in private housing. The blockchain would enable a secure, tamper-proof system for managing digital credentials and reputation; it could accelerate the adoption of P2P lodging.

Instead of central power providers, a distributed network built on blockchain technology lets people generate their own electricity to sell on the network, using their blockchain’s identity to perform the transaction. With solar and high-capacity battery technology, individuals can potentially act as distributed power providers. Blockchain could be used to facilitate secure transactions of power between individuals on a distributed network who do not have an existing relationship. The fact that all transactions are verified by a consensus network means you are protected from customers who claim the transaction did not happen. The blockchain records are a more reliable truth than the central database of the existing providers. The current applications are Sun Exchange, TransActive Grid, and Grid Singularity.

Blockchain and Real Estate

The real estate market will change for sure. Homeowners buying or refinancing property are subject to significant transaction costs, including title insurance, where the title search process can be labor intensive. An individual can put a property on the blockchain so that prospective buyers can review and verify the owner of a property. Currently, this process of title searching is done manually at a steep price. This specific-use case in the US could result in quickly obtained and less expensive title insurance. Such insurance is usually required by lenders issuing a mortgage to protect their interests. Transacting real estate is a cloudy and expensive process because of middlemen. Brokers, government property databases, title companies, insurance and property databases, escrow companies, inspectors and appraisers, and notary publics are all currently needed to complete a transaction. We wait and depend on them. They exist because they hold information that we cannot access or do not have the skills needed to operate in the existing property transaction ecosystem. Public blockchains will eliminate most if not all of these roadblocks and facilitate speedy and less expensive transaction fees. Current applications providers include Bitfury (www.bitfury.com), which registers land titles via a private blockchain, and Factom/Epigraph (www.factom.com).

Blockchain and Identity

Banking regulation requires that banks know their customers (KYC). Identity data stored on a blockchain facilitates the bank’s verification of the identity of new customers. Storing account and payment information in a blockchain could standardize the data required for an account, thereby improving data quality and reducing the number of falsely identified “suspicious” transactions. A tamper-proof record could also ease the process of getting to know a client and demonstrating compliance with AML regulations. Current users include the global financial messenger for payments and fund transfers, i.e., the Society for Worldwide Interbank Financial Telecommunication (SWIFT).

The blockchain can be used to keep track of digital assets. Today when we purchase a used car the seller must physically deliver the car, title, and key. In the future, car titles may be public on the blockchain and the key may be digital. All data could be verified through a blockchain. To sell a car, the seller would simply send the digital key to the new owner, who would use it to turn on and start the car. Exchanges like ShapeShift.io already demonstrate how simple the exchange of stocks, bonds, and other digital assets will be should blockchain technology take hold. How about the concept of “proof of existence” (POE) and transparency utilizing the blockchain? Provenance (https://www.provenance.org) is a real-time data platform that empowers brands to take steps toward greater transparency by tracing the origins and histories of products. With their technology, luxury brands could mitigate and potentially eliminate the sale of counterfeit goods. Outside of this area, one can only imagine what their technology could mean for the art world as well.

Blockchain and the Practice of Law

As an attorney, the concept of a smart contract immediately hooked me, Joseph, to my current obsession with blockchain. Some say blockchain will eliminate lawyers; I disagree. Lawyers will just have to be able to code or at least write a syllogistic specification describing the intent of the contract. The lawyer in the blockchain world must be a technologist as well. Creating smart contracts requires a new team, consisting of:

•   Attorneys who can design the code and who have legal knowledge of contract structure and enforcement

•   Technologists who provide the software engineering to implement legal constructs into smart self-executing transactional structures using blockchain

The law and smart contracts not only represent a new practice model but also a change in the global business model. A smart contract may start from a series of clauses drafted according to traditional contract doctrine, but as these clauses move onto a blockchain platform and become self-executing, doctrine becomes obscured. Don Tapscott, co-author of Blockchain Revolution, came up with the phrase “smart contract mediator” (SCM). Due to the self-enforcing nature of smart contracts, the nexus of disputes in transactions potentially will shift to the stage of dynamic execution. Lawyers with blockchain knowledge will act as mediators in this process, helping parties to navigate the smart contract process.

A smart contract looks and acts very different from a static contract. Understanding blockchain and DLT is critical to managing this process for clients. Many of the industries and practice areas discussed so far apply primarily to those working with large firms and representing enterprises. Hopefully for practitioners, blockchain is yet another technology that provides new opportunities for agile individual attorneys and small firms to compete for business. Smart contracts enable trustless financial services like loans, automatic execution of trade agreements, micropayments, and more. Smart contracts on the blockchain can also build on top of digital assets and stocks. The firm Steptoe and Johnson (www.steptoe.com) serves as counsel to the Blockchain Alliance (www.blockchainalliance.org), a coalition of more than 25 companies and 25 law enforcement and regulatory agencies around the world.

Blockchain Decentralized File Storage

Over the past few years, there have emerged a number of online file storage startups, the most prominent being Dropbox, seeking to allow users to upload a backup of their hard drive and have the service store the backup and allow the user to access it in exchange for a monthly fee. However, at this point the file storage market is at times relatively inefficient; a cursory look at various existing solutions shows that, particularly at the “uncanny valley” 20 to 200 GB level at which neither free quotas nor enterprise-level discounts kick in, monthly prices for mainstream file storage are such that you are paying for more than the cost of the entire hard drive in a single month.

IPFS is short for Interplanetary File System, and it does just what the name entails. Think of it as rewiring and rerouting the entire Internet so that trains run and stop on time and go where they need to go, all while greatly reducing redundancy. Designed by the technologist Juan Benet, IPFS is a peer-to-peer distributed file system that seeks to connect all computing devices with the same system of files. In some ways, IPFS is similar to the Web, but IPFS could be seen as a single BitTorrent swarm, exchanging objects within one Git repository. In other words, IPFS provides a high throughput content-addressed block storage model, with content-addressed hyperlinks. Projects like Storj intend to use blockchain technology to share files in a decentralized network. Users with extra disk space will automatically be able to rent out unused storage space. Users in need of cloud storage space can pay to store files on computers across the network. Storj estimates that this system will cut costs of data storage by about 80 percent.

Decentralized Autonomous Organizations

The general concept of a decentralized autonomous organization (DAO) is that of a virtual entity that has a certain set of members or shareholders which, perhaps with a 67 percent majority, have the right to spend the entity’s funds and modify its code. The members would collectively decide how the organization should allocate its funds. Methods for allocating a DAO’s funds could range from bounties and salaries to even more exotic mechanisms such as an internal currency to reward work. This essentially replicates the legal trappings of a traditional company or nonprofit but uses only cryptographic blockchain technology for enforcement. So far much of the talk around DAOs has been around the “capitalist” model of a decentralized autonomous corporation (DAC) with dividend-receiving shareholders and tradable shares. An alternative, perhaps described as a “decentralized autonomous community,” would have all members have an equal share in the decision making and require 67 percent of existing members to agree to add or remove a member. The requirement that one person can only have one membership would then need to be enforced collectively by the group. Backfeed (backfeed.cc) develops resilient technology and new economic models to support free, large-scale, systematic collaboration. Based on a distributed governance model, Backfeed protocols make it possible for people to easily deploy and maintain decentralized applications and DAOs that rely on the spontaneous and voluntary contribution of hundreds, thousands, or millions of people.

Blockchain and Cloud Computing

Enigma is a decentralized cloud platform that guarantees privacy (https://www.media.mit.edu/projects/enigma/overview/). Private data is stored, shared, and analyzed without ever being fully revealed to any party. Enigma provides secure multiparty computation, empowered by the blockchain. This is as futuristic and radical as blockchain gets and will be an important innovation as it solves some of the most difficult problems in technology today: privacy and security. These core functions, layered on top of distributed cloud technology, are a dynamic combination that transforms how data is stored and retrieved, providing industries like finance, health, and civil services the underlying trust and security to truly unlock the potential of next-generation mobile applications.

Blockchain Gambling and Betting

Gambling is a multi-billion-dollar industry and was revolutionized by the Internet. Online gambling accounts for a significant proportion of all betting. Mostly, the Internet just saw the same big gaming players move their infrastructure online. However, blockchain technology changes all that. It is the missing piece that protected the established gambling industry. Now there is a host of exciting new startups reimagining the gambling industry. They are decentralizing the entire structure and are building something truly new. VDice (vdice.io) is one example of such a startup. It is billed as the world’s first fully decentralized gambling platform and is wildly popular among blockchain geeks. Using the Ethereum blockchain, they have leveraged smart contract technology and have created game codes that exist without a server. These games live on the Ethereum P2P network. We will examine Ethereum and betting with a sample application developed by one of the authors, PeerBet, in detail in a later chapter that will provide a “start to finish” examination of the Ethereum blockchain development life cycle.

Summary

In summary, blockchain—the distributed database technology with the capability to execute smart contracts—is more than a platform for cryptography. The universe of “use cases,” which we briefly covered, is expanding exponentially. The efficiencies and cost savings provided by blockchain technologies, especially the private blockchains adopted by the financial community, will result in widespread adoption. This will cause the technology to mature at a rapid rate. In parallel, the use of blockchain to affect global transactions will also push it forward toward maturity. Blockchain and its timing are critical to maintaining global transactions and providing trust in the integrity of those transactions. For these reasons and more, blockchain is here to stay. We will provide you with a comprehensive journey through what you need as a developer to successfully make the trip.