9   Challenges

As blockchain evolves and new applications emerge, we the developers—and perhaps governance regulators and attorneys—will face a complex set of issues, as well as new dependencies. A key challenge associated with blockchain is a lack of awareness and technical skills needed to implement the technology, especially in sectors other than banking. A blockchain represents a large shift away from the traditional ways of developing digital technologies. It places trust and authority in a decentralized network rather than in a classic central institution. This loss of control makes business leaders hesitant. Blockchain is perhaps 80 percent business process change and 20 percent technology implementation. The speed and effectiveness with which blockchain networks currently execute peer-to-peer transactions comes at a high aggregate cost. As we have seen, this inefficiency arises because each node performs the same tasks as every other node on its own copy of the data. In this chapter, we will explore some possible alternatives. For the Bitcoin network, for example, which uses a proof-of-work approach in lieu of trusting participants in the network, total running costs associated with validating and sharing transactions is $600 million in 2017 and rising. Blockchains are a productivity paradox. Therefore, decisions about implementing blockchain applications need to be carefully considered and analyzed before implementation. The law and regulations they spawn have always struggled to keep up with advances in technology. Indeed, technologies like the Bitcoin blockchain bypass regulation completely to solve inefficiencies in conventional intermediated payment networks. That said, there are still strong arguments for blockchain applications to work within the laws and regulatory structures, not outside of them, but this means that lawyers and business leaders need to understand the technology and its impact on the businesses and consumers in their sector.

While no technology is completely secure, no one has managed as yet to compromise the encryption and decentralized architecture of a blockchain. For security and privacy, identities created within a blockchain are unique and offer a high level of assurance that the party is who they claim to be. We must consider how privacy and security can drive and inform the design. Public acceptance of blockchain applications will likely mean proactively framing the discussion of privacy around concepts of value, security, and trust. We will explore these challenges and other issues in this chapter.

All software systems as they evolve from proof of concept prototypes to virtual necessity naturally face technical challenges. Blockchains as distributed systems not only face technical challenges but, as we have seen, a host of governance challenges as well. We will explore challenges around governance, including the Bitcoin rift and the Ethereum fork. On the technical side, there are challenges such as “51 percent attacks”, and the all too familiar denial-of-service attacks we see in the cybersecurity news each day. Some new wave challenges include the growing use of smart contracts, and the reliability thereof, as well as the perennial scaling problems which all popular technology faces.

Blockchain Governance Challenges

Governance challenges encompass the philosophical, economic, and social issues that come from those already supporting a given blockchain or cryptocurrency, but disagree on how to move the technology forward. These are challenges that will consistently arise and be different every time. They all revolve around how to upgrade the code to deal with some issue but cannot agree on how to do so. In other words, it’s an issue of how to achieve consensus on the community level, of how protocol upgrades are managed. In a decentralized environment, you can’t force anyone to upgrade, but this has presented a number of problems, including:

•   Bitcoin blocksize debate

•   Ethereum DAO fork

•   Ethereum moving to proof-of-stake

Bitcoin Blocksize Debate

The Bitcoin blocksize debate has been a long-standing, growing rift in the Bitcoin space. It revolves around the 1MB limitation on blocksize that is hardcoded in the code (see https://github.com/bitcoin/bitcoin/blob/master/src/consensus/consensus.h):

static const unsigned int MAX_BLOCK_BASE_SIZE = 1000000;

The limitation was originally put there by Satoshi Nakamoto to prevent denial-of-service attacks. The fear was that without this limitation some miners could create blocks so large that smaller miners would not be able to process and keep up, in effect cutting them off from participating in the network. When Satoshi first put the limitation in place, it was with the understanding that it will be removed when the network is ready. But even after the capabilities of the network have improved due to hardware advances, the 1 MB limit has remained.

The problem with having such a limit is simply that there are only so many transactions that can fit into a 1MB block (roughly 2,000). Since there is a new block only every 10 minutes, this translates into three or four transactions per second. Obviously, this is untenable for a global system where, for comparison, Visa handles 20,000 transactions per second.

The blocksize issue first arose in 2013 with Jeff Garzik saying the blocksize needed to be increased. Gavin Andresen and Mike Hearn decided to weigh in in 2015 when they felt the issue needed to be taken seriously. Hearn wrote a blog post entitled “The Capacity Cliff,” urging the Bitcoin community to increase the blocksize immediately. He argued that once blocks reached capacity and people’s transactions were not processed they would lose faith in Bitcoin’s functionality, and there would be a downward spiral in Bitcoin’s market value. Hearn sold all his bitcoins at the time. Andresen argued that by March 2016 Bitcoin blocks would be full. He was accurate with his prediction. Hearn’s doomsday scenario did not come to pass, however.

Instead of the issue working its way closer to a resolution, it has instead drifted in the opposite direction, toward heated technical debates, arguments over the use and nature of Bitcoin, and name calling and infighting. The core developers felt that changing the blocksize would disadvantage small miners and lead to further miner centralization. They also felt a “fee market” is healthy for Bitcoin. They felt that there needs to be competition to get into a block in order to eliminate transaction spam and increase the transaction fees in general. They argued that a small (less than 10 cents) transaction fee is too small for a network that needs to store that transaction forever.

The community split itself into proponents of big blocks and proponents of small blocks. Each side accused the other of trying to destroy Bitcoin. On Reddit there was an r/btc spinoff from r/bitcoin, where r/btc accused r/bitcoin of censoring posts that were in favor of any size increase. There is indeed a lot of evidence backing up the claim of censorship. r/bitcoin in turn accused r/btc of just bashing anyone who wanted to keep the status quo. There were endless blog posts and articles from both sides, with everyone from Brian Armstrong, CEO of Coinbase, to Vitalik Buterin weighing in.

The effects of full blocks meant that transaction fees had to rise, and by late 2017 it was typical to pay $2 to $5 on a transaction when only a couple years back Bitcoin was heralded as being nearly free to use. Figure 9-1 shows the rise of transaction costs in 2017, in terms of satoshis (i.e., 0.00000001 bitcoin) per byte. In addition, instead of transactions confirming in the first block, it would often take several blocks to get a confirmation, turning what used to be a 10-minute wait into 30 minutes. Many transactions would not confirm at all and get stuck in the mempool. Eventually these transactions would drop out and a user could resend it with a higher fee, but nevertheless the ever-increasing number of transactions in the mempool does not bode well.

FIGURE 9-1   Average transaction costs January–September 2017, in terms of satoshis per byte

These issues led many bitcoiners to argue for “forking” Bitcoin—releasing a version of Bitcoin that can create 2MB blocks, known as Bitcoin Classic, and a version that removes the hardcoded limit altogether, known as Bitcoin Unlimited. In early summer 2017, a new version known as Bitcoin Cash was proposed, which would raise the blocksize to 8MB. On August 1, it had enough support to go into effect, and Bitcoin Cash was born. The Bitcoin network split into two.

The Bitcoin blocksize issue is disconcerting because even a relatively minor change could not get enough support to prevent a fork. If the Bitcoin community cannot agree on a small code change, how is there any hope for implementing significant improvements to the protocol needed for scaling and sidechains? We may be headed toward a world of many Bitcoin blockchains, each one born out of a modification that couldn’t get universal support.

The Ethereum DAO Fork

Before July 2016 there was only one Ethereum (ETH). Then the Ethereum network forked. Most of the community, around 90 percent, were behind the Ethereum Foundation, which had made just one change to the code: it undoes all transactions from one DAO transaction, the one that moved $60 million worth of ether from the DAO contract into a hacker’s personal contract. While most of the community was behind this change, there was some strong opposition too—not because people wanted the hacker to get away with it, only that it’s the lesser of two evils. The bigger evil was reneging on the promise of blockchains—indeed, the claim made on ethereum.org: “unstoppable code.” Blockchain promised code that cannot be changed and transactions that cannot be rolled back. If this transaction were rolled back, how could anyone have faith that their transaction wouldn’t suffer a similar fate? Of course, the fear is somewhat unjustified because you’re only at risk of having a transaction rolled back if the majority of the Ethereum community unites to undo a transaction, and this is only likely in the event of something monumental like the DAO hack. In any case, there were some strong holdouts who did not want to upgrade.

On July 20, 2016, at block number 1920000 the new version of Ethereum went into effect. It was a success—at least it was immediately heralded as one by most of the community. There was very little hash power on the non-upgraded chain, and within the first two days it seemed the old chain would die out completely—there would be no miners left creating blocks on that chain. But then something surprising happened. The chain started getting more hash power and more vocal proponents. Then the largest Ethereum exchange—Poloniex—listed these “old chain” tokens, dubbed Ethereum Classic (ETC). This was a watershed moment opening up ETC to traders and speculators, some crediting Poloniex for propping the coin up, if not criticizing them for adding it for their own self-interests. Ethereum Classic has continued to maintain a solid following and has been among the top coins by market cap ever since.

This all illustrates the challenge for public tokens: how to get the whole community in agreement on changing the code in some way. If at every change it spawns a sister chain, then the value proposition of cryptocurrencies is watered down.

Ethereum’s Move to PoS and Scaling Challenges

In continuation with the above, Ethereum faces more governance issues in its near future. Ethereum has always proposed to move to a proof-of-stake (PoS) mining algorithm. And while miners knew from day one that such a change will be implemented at some point, they now have a lot vested in proof-of-work mining equipment. It may not be in their best interests to go along with the transition to proof-of-stake. Indeed there is already such talk among a small percentage of Ethereum miners to maintain the consensus algorithm as is. Senior Ethereum developer Vlad Zamfir has also argued for reducing the mining rewards before the move to PoS, in light of the high price of ether. However good the arguments are, if they don’t satisfy the vast majority of miners, then any change will result in yet another version of Ethereum.

Perhaps more challenging will be Ethereum’s architecture changes to allow sharding—Ethereum’s scaling proposal. The changes will likely be quite significant—and risky. It’s hard to imagine there not being vocal opposition to implementing changes that can potentially introduce a bug to wipe out all of Ethereum’s market value.

In the early days of Bitcoin, it was also thought that it will undergo major changes to allow for scaling. Today, those dreams are squashed with the recognition that it would be too difficult to change Bitcoin now. However, unlike Ethereum, Bitcoin’s core function has always been as a “store of value.” Ethereum, on the other hand, has been championed as a world computer. A world computer must be able to handle more than a couple dozen transactions per second. It’s an existential threat that Ethereum uniquely faces.

Blockchain Technical Challenges

Blockchains face some unique challenges as a piece of software. There’s very little room for error in blockchain code, especially public blockchains. If there are billions of dollars of cryptocurrency riding on the soundness of the platform, a bug can cost literally billions and potentially be fatal altogether. The DAO hack has already demonstrated this: a subtle bug resulted in the loss of tens of millions of dollars, not to mention hurting confidence in the overall system. So blockchain code must be compared to software that flies a plane, or that monitors an oil drill.

Technical challenges come in two main varieties: security challenges and usability challenges. We will explore the following:

•   Security challenges: the core code, denial-of-service attacks, smart contracts

•   Usability challenges: scaling

Bugs in the Core Code

In Bitcoin’s history it’s had two issues with its core code that could have been potentially disastrous. On August 8, 2010, Bitcoin developer Jeff Garzik announced on bitcointalk.org, “The ‘value out’ in this block is quite strange,” referring to a block that had somehow contained 92 billion BTC, which is precisely 91,979,000,000 more bitcoins than are ever supposed to exist. Block number 74638 exploited an integer overflow to make a negative total transaction. Ultimately the blockchain needed to be rolled back to before the introduction of these bitcoins.

Then in 2012 there was an update to Bitcoin core that had an unintentional side effect. In the new version, 0.8, it was using an updated version of MongoDB (the backend DB of Bitcoin). This version was incompatible with 0.7 and caused blocks produced on the older version to not be recognized by 0.8. The blockchain forked. After a frantic few hours on the bitcointalk message board, it was decided that all those who did upgrade would roll back to the 0.7 version and abandon their chain. Thankfully, these miners—who were actually the more conscientious ones, following along with upgrades—were willing to take a hit and lose their mining rewards for the good of the network. Nevertheless, this was another demonstration of side effects arising in subtle ways.

Denial-of-Service Attacks

Ethereum also faced a problem that isn’t so much a bug but more of a miscalibration, which allowed for denial-of-service (DoS) attacks to be carried out against the network. We normally think of denial-of-service attacks as an adversary overwhelming a website or service with so many requests that the service cannot respond to legitimate users, effectively taking the service offline. Figure 9-2 shows this scenario.

FIGURE 9-2   Denial-of-service attack

In Ethereum this happened slightly differently. In Ethereum the user needs to pay for operations on the network. So you’d think this would block any denial-of-service attacks by the fact that it costs money to execute. But that assumes the cost to execute operations is commensurate with the processing overhead needed to run them; this was not the case. Each operation type in the EVM has a price associated with it. And as it happened some of the operations were mispriced, not reflecting the computational overhead needed to fulfill them. An attacker unleashed a smart contract and invoked it repeatedly to perform memory-intensive work, dramatically slowing down the Ethereum network. In fact it would have brought the network to a standstill if not for an alternative, less popular, client called Parity, which was able to process those transactions far more efficiently and keep up with the demand. The more popular Geth client became useless. The Geth developers quickly rolled out an improvement, but a full fix was only possible with a hard fork to put in appropriate pricing. Unfortunately, even with the fork there was still an operation—suicide—invoked when the contract was killed that took up too many resources, and another fork needed to be rolled out for that.

In short, there was a period of two months where there were repeated attacks on the Ethereum blockchain that resulted in some form of denial-of-service attacks and demonstrated that these networks must be vigilant and resilient against them.

Security in Smart Contracts

Perhaps the most important security concern in blockchain is the security of the smart contracts, because while the core blockchain is heavily vetted and tested, smart contracts are often rushed out the door. While the blockchains can be rolled back in a catastrophic attack, a smart contract generally cannot.

The DAO Hack

The DAO hack is the most dramatic example of what happens with poor smart contract design. The DAO contract had $150 million worth of ether when a hacker began to drain it. The hacker successfully moved $60 million worth to an intermediate holding contact where they were in sole control. The hack happened because the DAO allowed the user to withdraw their funds repeatedly, known as a reinjection vulnerability. The hole was in that when ether was withdrawn, it was sent to the withdrawer (the hacker in this case) and only then updated the withdrawer’s balance to 0. Figure 9-3 shows the general structure of the DAO, where users can invoke the “Split DAO” operation and request payout of their funds from the parent DAO to the child DAO. The hacker figured out a way to request a payout again and again before their balance was updated.

FIGURE 9-3   The normal movement of funds in the DAO, including the Split DAO function

Here are the problematic lines of code in the splitDAO() function where the user can withdraw their ether and move it into their own DAO:

The DAO contract wasn’t the only one to suffer from the reinjection vulnerability. MakerDAO, which stored millions of dollars in its decentralized exchange platform, also had this bug but luckily the developers were able to drain the funds themselves using the vulnerability before a hacker got to the funds.

Other Smart Contract Bugs

Many other projects suffered from bugs in their smart contracts. Ethereum Name Service (ENS) is a well-respected project led by Nick Johnson at the Ethereum Foundation. ENS is a system to associate user-friendly names with Ethereum addresses, whether user addresses or contracts, just as DNS associates domain names with IP addresses. Instead of referring to a long hexadecimal address you can refer to companyname.eth. ENS is a contract that acts as a registry, mapping names to addresses. It has a very fair system to reserve such names. In order to claim one, you must first open an auction for your name, which gives anyone three days to bid on the name. After the bidding period, there are two days where everyone can reveal their bids, with the highest bidder winning the name and being refunded whatever money they paid above the next highest bidder. In other words, the winning bidder only pays the amount of the second highest bid. Figure 9-4 illustrates the bidding flow. When bids are placed they are encrypted, masking the actual bid until the reveal period, when bids are uncovered. It’s an elegant system.

FIGURE 9-4   The stages of an ENS bid

A few hours after the ENS contract went live, a bug was found that allowed someone to keep bidding during the reveal period, overbidding any revealed bids. This flaw was especially damaging because it didn’t depend on special circumstances as the DAO hack did. It was an egregious error with a vital if condition completely missing. The flaw was that in the contract there was no check to determine whether the bidding period had closed.

The ENS team was quick to respond and deactivate the contract, promising a fix would be released that same evening. They blamed the bug on a last-minute refactor that introduced it—a bad oversight but quickly addressed. The fix was to change an above check to

It looked like all was well and back on track, but a second bug was reported a few hours later. The contract could be tricked into making someone the highest bidder even if they put in a very low bid. It came down to using a wrong but similarly named variable in one of the if conditions. The variable _value was the wrong reference and should have referred to actualValue:

In detail, a user enters a bid into the auction. They may pay ether here, but there is no enforced relationship between the amount of ether they pay and the value declared when defining the bid. The value of the deed will be defined by the amount paid, and the deed will own that amount, but this may not be the value the user declared when defining the bid.

When unsealBid is called, they must identify the bid with its declared value _value, which may differ from the value of the Deed that represents it. The function will compute a quantity actualValue, which will be the minimum of the value paid and the value declared. So, if the user underpaid, actualValue will be the value that they’ve actually paid.

However, when checking to see if they’ve won the auction (or if they have at least the minimum price), the declared value _value is used rather than actualValue to decide if the bidder won. If the declared value is sufficiently high, they eject the previous winner of the auction. The winning bid gets set to actualValue, which may be lower than the user’s declared value, and lower than the prior winning bid. So someone could win auctions by declaring very high values but underfunding them (and unsealing their bids late in the reveal period, since their low actually paid value becomes an easily displaceable highestBid).

Two embarrassing bugs on the very first day—not a good start for ENS. The team said they were going to take a time-out and postpone the release until a complete audit could be carried out. Why wasn’t it fully vetted before?

The ENS incident is a good example of a high-profile project by a high-profile team failing to meet the standards required for a blockchain program. If they couldn’t release solid code, what hope is there for amateur teams or engineers at large to create smart contracts that are safe and execute as intended?

The DAO and other smart contract bugs have convinced many in the community that there must be a better approach if this technology is to work. Hoping that everyone becomes a more vigilant coder is not enough. We will continue to see bugs. The alternative is to change the very nature of how smart contracts are written to make it difficult for bugs to appear in the first place—to use a formally verifiable language or formal verification tools, a feature of functional languages.

In functional code, the output value of a function depends only on the arguments that are passed to the function, so calling a function f twice with the same value for an argument x will produce the same result f(x) each time. This is in contrast to procedures depending on local or global state, which may produce different results at different times when called with the same arguments but different program state. Eliminating side effects—i.e., changes in state that do not depend on the function inputs—can make it much easier to understand and predict the behavior of a program, which is one of the key motivations for the development of functional programming.

Vitalik Buterin has talked at length about the need for such techniques, and it’s considered one of Ethereum’s existential needs. The problem is that not only does it require a lot of resources to develop such a language or tool, but it’s far more difficult to learn and program in such languages. Engineers will need to be retrained, and the pool of such engineers will likely be smaller.

Scaling

For public blockchains, the biggest challenge is scalability. The Bitcoin network can process 4 to 7 transactions per second, the Ethereum network around 15. This is minuscule for a global platform. What use is a public blockchain if it cannot actually serve a global audience? If Bitcoin or Ethereum were to have as many users as Visa or Instagram today, the network would grind to a halt.

When you think about it, if every transaction, every computation, is performed by every node in the network, then the processing capability of the entire network is limited by the processing capability of a single node. If the solution is to have super-powerful nodes that have massive memory, CPU, and bandwidth resources, the network is not very decentralized. Only the powerful will have powerful computers.

In reality, it’s overkill to have everyone do everything. And in order to allow blockchains to scale to the level of Visa, we must find a way around this design paradigm. The solution must take some form of not having everyone process everything. There are many solutions in the works, but none have been yet implemented. Here’s an overview of what we know.

Don’t Do Everything on the Blockchain

One solution is to build an intermediate layer or many intermediate layers. These are called second-layer solutions, and they go by the names Lightning Network (for Bitcoin) and Raiden Network (for Ethereum). Users can create state channels that allow them to make many transactions that effectively count as one transaction. After some period of time, perhaps a week, the channel is closed and the total balances for each party are updated on the blockchain.

However, even state channels only go so far. You still need to open a channel between every two parties. If you have too many users, the blockchain won’t be able to accommodate state channels for all of them.

Don’t Record Every Transaction on Every Node

Ultimately the only way to get around the scaling problem is to eliminate the needless redundancy of recording everything everywhere. In a sense, this is in practice already with transactions happening upon multiple blockchains. The existence of the Litecoin and Ethereum blockchains takes some of the load off the Bitcoin chain. It’s almost as if there were Bitcoin¹, Bitcoin², etc. Of course, the problem with such solutions is that there isn’t direct interoperability between all these chains. However, this has led to research into network-of-blockchain solutions, including Cosmos, Polkadot, and many others. The idea is that there is an external (decentralized) system that is listening for transactions or operations on one blockchain and relaying them to another blockchain.

However, multiple blockchains or even interconnectivity among multiple blockchains is not true scalability. What you want is that as your network grows and gives you more security, the transaction throughput grows accordingly. While having multiple blockchains does grow transaction throughput, it comes at a cost: the security of the entire system is only as strong as the weakest blockchain.

Sharding

We can view shards as mostly independent blockchains that are designed to communicate well with each other. Validators are assigned some shards to keep track of, which may vary hour by hour, and users maintain a light client to all the shards. If the number of shards is S = O(c), where c is the number of transactions a shard can support, then this approach can support up to O(c²) transactions. However, with more complex shard-of-shard schemes, the maximum number can be increased exponentially to O(exp(c)).

Since each shard now only has O(n/c) of nodes working for it, where n is the total number of nodes across all shards, the question is, can we still leverage the O(n) strength of the entire network for each shard? Indeed we can. If the mechanism for determining the state and history of each shard is done intelligently, we can leverage the strength of the entire network as if they were all working on one unified blockchain.

The easier scenario to deal with is when applications for the most part live entirely on one shard—either they don’t have many users, or there aren’t many interactions between them. When an interaction is needed with another shard, then cross-shard communication via receipts can be used to pass information securely. Note that we don’t restrict which shard a user or application has to use—everyone is free to use whatever shard they please.

When cross-shard communication is needed, shard X generates a receipt of some activity happening on it. Shard Y will consume that receipt, do some action because of it, and possibly pass back to X some acknowledgment that the task was completed. Generalizing this pattern is not difficult and can be built in to a high-level programming language. Figure 9-5 illustrates this concept.

FIGURE 9-5   Shard X generates a receipt to be consumed by Shard Y.

However, the process cross-shard communication will have weaker functionality than mechanisms for intra-shard communication, and some operations that we do on current non-scalable blockchains will only be doable within one shard.

Also, communicating between shards in an asynchronous way is not always easy. Consider the following example described by crypto-veteran Andrew Miller. Suppose a user wants to purchase a plane ticket and reserve a hotel room, but wants both reservations to go through or both fail. They don’t want to be stuck with one without the other. In the case where both systems are on the same shard, the procedure is easy—if both reservations don’t complete, revert everything. But in the case where the systems are on different shards, then in an asynchronous system there’s an extra requirement. Each system must allow for a temporary hold on a reservation. This will allow the user to confirm one booking and then the other. There is a faster, more efficient way with synchronous cross-shard transactions, but designing such a system is not trivial. If an application is larger than what can be supported on one shard, it will need to live on multiple shards and would in general require transactions to be processed serially across chains.

According to Amdahl’s law, in any process that has non-parallelizable components, those components quickly become the bottleneck in a system that can otherwise be parallelized. Since Ethereum is in general a computation system, we can easily come up with non-parallelizable programs. For instance, imagine a contract that keeps track of a value x and sets sha3(x, tx_data) upon receiving a transaction. No sharding solution can give performance better than O(c). So even though over time shared blockchains will get better and better at handling more diverse and involved applications, they will still fall short from single-blockchain architectures in some ways.

Security via Random Sampling

How do we coordinate all nodes to achieve the security of all of them working together? The answer is random sampling. Each validator is assigned randomly to a shard and can be reassigned every few hours or less. From the shard’s assigned set of validators (say, 150), one will be chosen from the pool to create a block. Even though far fewer nodes are verifying and creating blocks on each shard at any point in time, it’s almost as if all the nodes are doing so, statistically speaking. If we assume an honest two-thirds supermajority on all the nodes, then there is a 99.999 percent probability that two-thirds of a random sample of 150 will be honest. The probability can be brought very close to 100 percent if we assume an even higher honest supermajority.

Summary

While blockchains are heralded as being a technological breakthrough that will solve many problems, it’s clear that they face a large number of unique challenges. These challenges are not insurmountable, but they will still require a lot of work to develop infrastructure and safety mechanisms to overcome them.