Decentralized Data

This is the most important of the concepts to me. Currently, we trust “The Stacks” with our data. We willingly give them our data for free in return for a free service. Or, we pay them to store our data—but we only have enough data to make that worthwhile if our users are giving us their data for free! We trust that they won’t misuse and sell our data to entities to which we would rather not be exposed. Since Edward Snowden, we now know that trust can, has, and will be broken as long as we entrust our data to a central entity. Centralized stores of data are a surveillance state’s dream; all of your citizens’ data in one easily accessible place and the ability to monitor it without their knowledge. Amazon Web Services, Google Drive, Dropbox, and every other “cloud” provider, despite having a distributed computing backend, are centrally owned.

Additionally, as the global economy rapidly evolves from a labor-based economy to an information-based economy, with robotics and automation technology expanding at an accelerating rate, data will become the primary form of value. Although humans can’t compete with robots for labor, they can compete with them for data, the data they parse from their unique perception of the world, the processed output from their five senses. We not only possess our data, we need to own our data as the world evolves.

So how do we solve this? How do we store data in a decentralized way such that no one but yourself owns your data? This is a problem that has been under heavy research for at least a decade with several parties claiming a solution. The ideal solution should provide a method of storing data in a decentralized way that is robust and as trustless as possible.

Option 2: Storing Data in a Distributed Hash Table

Distributed Hash Tables (DHTs) have taken off in popularity in the past decade. They distribute not only copies of the data, but also the indexing functions that enable the data to be found, ensuring resiliency. Early peer-to-peer (P2P) filesharing programs like KaZaA, Napster, and Gnutella used their own versions of DHTs with varying levels of decentralization. Some had centralized trackers to monitor the movement of all data and some (like Napster) had central sources that all data had to go through, leaving them with a single point of failure (in this case, due to legal action).

The first implementation of a DHT to really take off was BitTorrent. BitTorrent is still used by more than 300 million users. Despite having a decentralized data store (the BitTorrent Mainline DHT), it still depends on centralized trackers (like Pirate Bay) to monitor the network. Sites like Pirate Bay are regularly shut down by legal action, so even with BitTorrent’s data resiliency, it still has some points of failure. If we use BitTorrent’s DHT to store our dapp’s data, that would be great, right? BitTorrent doesn’t just offer a decentralized data store; it offers a data distribution protocol that maximizes bandwidth via a tit-for-tat strategy between seeders and leechers.

BitTorrent’s data transfer protocol is even faster than the Web’s, and as such it’s become the de facto method of transferring large datasets like HD movies over the Web. The problem with using BitTorrent as a data store is that there is not enough incentive to store your data for the long term among nodes. The network is set up to prioritize files with high demand—people have to want your data for it to be replicated and continually stored in the network. In contrast, when using a reputable central server like Amazon Web Services, you know that your data is going to continue to exist even if you are the only user of the data, because their reputation is at stake, they are contractually obligated to do so, and they don’t depend on others needing the data to store it.

First, we don’t just want the decentralized storage capabilities of a DHT and the speed of BitTorrent’s file transfer, we also want data permanence. It’s necessary, therefore, to incentivize nodes to store data in some way. Second, we need to ensure that the links to the data don’t die. This is not a new idea. One of the original proposals for the Internet had link permanence baked in. That idea was called Project Xanadu, and it called for a Web in which every link worked two ways: one toward the destination, and one toward its source. That means that the content creator would always be able to get credit for their data because it would always link back to them. This Web never came to be, and so we have the HTTP-based Web we’ve grown to know and love today with one-way links.

But does there exist a system that implements these features? Yes, it’s called the Interplanetary File System (IPFS), and it’s an open source project that is currently in alpha phase. I’m a pretty big fan of IPFS and was one of the early contributors to the protocol. Juan Benet, the creator, had been thinking about data storage for five years and had finally put all those thoughts into action when he published the IPFS scientific paper. I spent many months understanding the system and his frame of thought, why IPFS was better than other solutions being worked on, and at this point I feel it has the best chance of bringing the most value.

IPFS aims to help us move toward a permanent, decentralized Web. That is, a Web whose links never die and no single entity controls your data. Upon downloading an IPFS client, a user is able to add any data to it and in return receives a hash. The user can then access that data via its hash. IPFS is a content-addressed system, in contrast to the Web, which is an IP-addressed system. In an IP-addressed system, if a nameserver fails, effectively so does all of its data. Content addressing is a much more efficient form of addressing data because it doesn’t rely on a single server’s uptime to access data. When you request data from a content address, you’ll receive it faster than you would IP-addressed data because it will route from whoever owns a copy of that content address closest to you.

So, what does it look like on the backend?

IPFS uses a DHT to store data. It’s based on the popular Kademlia DHT, and it borrows ideas from Chord and BitTorrent’s DHT. When users upload data to IPFS, that data is copied among a certain number of other nodes, so even if one node fails, the data remains. On top of that—and like BitTorrent—the more nodes that need the data, the more resilient it becomes as they each share the copy they download. Chord’s killer feature was its DHT circles, which created “chords” to maximize DHT lookups among nodes across the globe that were in close proximity to one another within larger chords. So, the globe would look like a series of increasingly larger chords (see Figure 2-1) and lookups would benefit from this efficiency, hopping between chords where necessary.

Currently, centralized services like Amazon and Google have datacenters across the world that a user can choose from—in many cases they will automatically choose it for you—through which to receive and route your data. Even with datacenters spread across the globe, data transfer efficiency can be increased with multiple nodes in a way only a truly decentralized system like Chord’s DHT can provide.

To give structure to the DHT and let users find the data they need when they need it, IPFS uses what it calls a merkleDAG. A merkleDAG is a simple flexible data structure that can be conceptualized as a series of nodes connected to each other. To be more specific, it’s a directed, acyclic graph (DAG). A merkleDAG can look like a linked list or a tree. When adding data to the DHT, the system generates an SHA-256 multihash public-private key pair, and the user gets both. Developers can link hashes programmatically to form their own mini-merkleDAGs, and it’s important to note that all data in IPFS forms the same generalized merkleDAG consisting of all nodes. All data on IPFS is public, so it’s the users’ responsibility to encrypt their data accordingly. The private keys, in addition to allowing access to the data, can prove ownership.

Figure 2-1. Chord

IPFS (shown in Figure 2-2) was inspired by BitTorrent’s speed of transfer and tit-for-tat mechanism of finding nearby peers to share data with, and the IPFS team believes the Web should work that way, as well. Take the example of an entire college class requesting a video from a centralized Facebook server. This takes up unnecessary bandwidth and is unnecessarily redundant. If the image is nearby, they don’t need to make a request from that far away. In a content-addressed system, if they know the content address of the data they want, they can just retrieve it from the nearest location. Nodes can share data among themselves without central coordination; the schema takes the server-client model that the HTTP web runs on and makes it distributed, just like BitTorrent.

Figure 2-2. IPFS

How Does IPFS Improve on BitTorrent?

IPFS has a sister protocol called Filecoin. Filecoin is used to pay miners (nodes that store data) using a novel value-for-data mechanism called BitSwap. Cryptocurrency makes sense here: its value transfer is fast and it allows for micropayments to pay for every correlated byte of storage. Filecoin is currently in development, but IPFS is already available to use. IPFS commands are currently free and the miners currently storing data are doing so out of their love of the network. Eventually, all uploads and downloads will require Filecoin. Filecoin will most likely be an asset built directly on Bitcoin’s blockchain, so users can just use their Bitcoin to pay for storage.

To top it all off, IPFS borrows from Git’s version-control model to version all data. Git uses a DAG to model versions of data and IPFS uses it to give structure to the entire system. Users can see the version history of their data (or any data to which they have decrypted access).

So, IPFS takes the best ideas from Git, DHTs, SFS, BitTorrent, and Bitcoin and combines them to create a decentralized data-storage network. IPFS hopes to one day replace the HTTP:// protocol of the Web with IPFS://, but they can work in unison as well in several ways that we’ll get into when we begin talking about implementation.

IPFS is the most robust, thought-through solution to decentralized storage out of all the cryptocurrency projects, although there are other notable contenders in the space, as well. Let’s survey the other options.

Ethereum Swarm

Ethereum is working to build a general-purpose (Turing-complete) blockchain computing language, including decentralized storage. As of this writing (2016), their efforts are focused on securing the DAO (in their usage, “Democratic Autonomous Organization”) and storage has been put on the back burner.

StorJ

StorJ has garnered a lot of hype lately; it has pre-mined a lot of StorJcoins and has made some pretty designs. The designs are neat—it won an Austin hackathon—and the group seems to know what it’s talking about. Despite all of this, more than a year post-hackathon it is still vaporware.

Maidsafe

Maidsafe, like Ethereum, is trying to do many things. They aren’t using proof-of-work and aim to create a decentralized platform for computing, storage, and currency. They’ve been working on their platform for six years and it seems like the project hasn’t gained that much traction.

Decentralized Wealth

Bitcoin was the first successful decentralized store of wealth. Before Bitcoin, trust was needed in a third-party provider (a bank) when transferring value over the Web. Bitcoin allowed for decentralized value transfer and fulfills the need for decentralized payments in a dapp.

But what about all of these altcoins? And what about Litecoin, dogecoin, peercoin, darkcoin, and kanyecoin? Altcoins are generally made by forking the Bitcoin source code and adding in an incremental feature that for a variety of reasons the core developers of Bitcoin have refused to adopt. For example, the Litecoin creator wanted faster payments than Bitcoin so he forked Bitcoin, added some speedy code, and Litecoin was born.

Litecoin has a pretty big market cap, having been consistently in the top five cryptocurrencies for at least a year. Litecoin is a rare example that serves a good purpose: faster payments. But most altcoins don’t: if they’re not jokes that become memes with money behind them (like dogecoin), they are just made for “pump-and-dump” schemes. The idea is that someone can just create a new coin, slap a label on it, and pump up its value by hyping it via media exposure (kanyecoin). They claim that a coin will be really valuable and that investors will make a lot of money by buying in early as the coins value goes up.

As soon as the coin appears to have enough value, the creators sell it all for a currency that is more stable and long-term like Bitcoin or fiat. This is a common scheme in the altcoin world, and it is terrible for the cryptocurrency ecosystem for obvious reasons. First of all, these coins tarnish the reputation of cryptocurrency in general because they make potential investors increasingly wary of the space. Second, they compete with the Bitcoin blockchain for market share unnecessarily while bringing no real value to the table. This in turn harms Bitcoin’s value and all of those that rely on it as the most-used cryptocurrency.

Bitcoin, of course, uses the proof-of-work scheme, which means that every miner in the network must generate a computational proof of their computing power and process transactions; in return miners get Bitcoins as payment for their maintenance of the network. Some people in the cryptocurrency space consider proof-of-work to be overly energy expensive and a short-term solution to Sybil resistance, so a whole host of cryptocurrency research is ongoing in the field of consensus mechanisms. Proof-of-work uses a lot of computing power and the total cost of the electricity spent by miners in maintaining the network is more than 15 million dollars. This is wasteful when there potentially could be better ways of maintaining the network. Two popular alternatives that have been proposed to replace proof-of-work are proof-of-stake and delegated proof-of-stake.

Despite the massive amount of ongoing research into novel consensus mechanisms post Bitcoin, like proof-of-stake, nothing has currently shown itself to be as Sybil-resistant as the proof-of-work. As computationally expensive as it is, it’s the best we have. The Bitcoin network has more than three billion dollars invested in it, and countless startups, investors, media, and retailers accept the currency. It has the first-mover advantage and has fought for five-plus years to gain recognition among the mainstream populace. We shouldn’t need to start over again. Even if a new consensus mechanism is found that is superior to proof-of-work, we should rely on the Bitcoin core developers to implement it rather than an altcoin so that we can progress faster as a community.

Some may call this idea “Bitcoin Maximalism.” The argument is that Bitcoin maximalists continually tout the first-mover advantage of Bitcoin and are staunchly against any competitor to protect their investment in the network. The negative implication of Bitcoin Maximalism is that any ideas outside the Bitcoin protocol, no matter how valuable, are quickly stamped out and not given their proper recognition by the community, and progress is put on hold.

There is a solution that gives us the best of both worlds: this is called the sidechain proposal. The sidechain proposal is based on a paper coauthored by Adam Back, the inventor of proof-of-work that Satoshi referenced in his Bitcoin Paper. This proposal starts from the idea that in order to experiment with consensus mechanisms and any novel cryptocurrency ideas currently, developers must fork the Bitcoin blockchain and create an entire new altchain to test out their hypothesis.

This is bad for Bitcoin, and it’s difficult for developers to bootstrap a blockchain. The solution proposed by Back’s team is code that effectively allows Bitcoins to move freely between the main chain (the Bitcoin blockchain) and sidechains. That means that you could create an entirely new blockchain and sidechain it to the Bitcoin blockchain easily. You would gain the security benefit of Bitcoin’s proof-of-work, so you wouldn’t need to bootstrap your own mining network. You would also get people already invested in the cryptocurrency experiment (people who own Bitcoin) as a base of potential users in your chain because they could just use the coins they already have. Lastly, you could send coins between the two chains without any conversion necessary. Two-way sidechains are currently in the works and will be released very soon.

The Bitcoin blockchain is the most secure blockchain; it has more computing power than all the world’s supercomputers combined, so it’s the most Sybil-resistant. Bootstrapping a proof-of-work blockchain from scratch is difficult; because computing resources are so small in the early stages, it’s easy for an attacker to build up 51 percent of the total computing power of the network and take it over. Besides that, developers shouldn’t need to worry about bootstrapping a blockchain in addition to the already challenging task of building a decentralized application that people want. Sidechains offer a solution here if you want to experiment with consensus mechanisms or implement some novel cryptocurrency technique.

But what about if you don’t really want to implement a new cryptocurrency technique and just want to issue your own internal currency for decentralized application? A currency that grows in value with the network, allows users to access scarce resources, and incentivizes them to grow the network? Then there is no need to create a new (side)chain. You can simply create an asset directly on Bitcoin itself. Colored coins is my project of choice on the matter, though as usual there are a few notable alternatives:

Counterparty

Counterparty is a Bitcoin 2.0 protocol that lets users create and manage assets, enact bids, and place bids, and even allows for users to create Turing-complete contracts on top of Bitcoin. This all sounds awesome, but the problem is that Counterparty baked all of these interesting features into the protocol instead of modularizing them and layering them on top of each other. Issuing assets on the Bitcoin blockchain and letting users transfer them with the ease of Bitcoin is an awesome idea. But they’ve combined that with the dividend function. Distributing dividends is a nice little feature to have, but they are their own internal operation within Counterparty, instead of using native Bitcoin to track assets. Everything has been forced into the same overwrought protocol.

Bets are an example of adding an experimental and challenging feature with a good one: assets. What would be better would be to build simple layers, each really good at doing one thing. Modularity is the hallmark of good software, and Counterparty, as ambitious as it is, is not modular at all. If we envision a market of libraries of different protocols, those libraries will compete with one another, and the ones that are best will win. Imagine if all those useful libraries were hopelessly intertwined as one package. You would need to install all of them or none of them at all. That would be a nightmare, and that’s exactly what Counterparty has you do.

Counterparty introduces an unwanted and confusing element to the developer who plans to use it: the XCP currency. If you want to build an appcoin using Counterparty’s API, you will need to deal with XCP and all the conversions that go along with it. If you want to create any kind of asset, according to the protocol you must destroy 0.5 XCP—greater than a dollar at current prices. The XCP monetary supply is fixed, and because the currency is continuously destroyed every time someone issues a new asset, the entire monetary base is constantly in decline.

The fact that XCP even exists and is required to use certain features of Counterparty is an annoyance to developers. It means that you have to continuously look at the ticker price (XCP/BTC) to use it. There are entire platforms that exist to track this price, with real-time bidding and asking for liquidity, just like in any market. But really, what’s the point? Why should you be forced to deal with all of that when all you want to do is create an internal currency for your app? It’s basically a huge barrier to entry that no one really needs and, as such, it’s a bad idea.

Counterparty updates its clients all the time, and people that depend on its API, like the Gems app, get mixed results because of this. Because there is no modularity, if there is a bug, everything will break and has broken all at once. All in all, Counterparty is too centrally administered; there are better alternatives (colored coins) that provide the necessary modularity and decentralization without the additional currency to use.

Hyperledger

Hyperledger believes itself to be “token agnostic,” in that it allows an issuer to issue coins that are based on no underlying currency: neither Bitcoin, nor fiat, nor any other altcoin. The principle behind this is sound, but in practice it isn’t because it relies on an unknown consensus mechanism that it touts is in the works. We’ve seen a lot of research in this field, and again nothing has demonstrated itself to be as versatile as proof-of-work.

One way to easily cut through the noise of any Blockchain 2.0 projects claims is to dig into their consensus mechanism. If it doesn’t use proof-of-work or isn’t based on Bitcoin, see how big their market cap is and dig into how many times there have been security breaches. Every time I’ve done this I’ve found security breaches. Table 2-1, from Meher Roy, summarizes the different beliefs in this field.

Token agnosticism is a strong set of views, but I believe Bitcoin can cooperate with existing financial systems. We’ve grown past the initial honeymoon phase of Bitcoin, during which many thought it would dominate the global currency landscape, and we’ve also realized that the banking system does have its place in the world despite how antiquated it is.

What about Turing-complete smart contracts? This is the second part of a financial instrument necessary for creating a decentralized payments system in a dapp. The Ethereum team has probably made the most progress on this, but it has big ambitions. Ethereum wants to create a Turing-complete blockchain, a decentralized storage network, a decentralized communications protocol, a new consensus mechanism and a new (bootstrapped) blockchain, a new browser in which to run Ethereum dapps, and a new scripting language with which to code Ethereum dapps.

Let’s step back for a second. One team cannot and should not aim to accomplish all of these company-sized ideas single-handedly. Ethereum has raised a lot of money and a lot of hype, but despite Vitalik Buterin’s brilliance, I don’t think we should expect them to create the next Bitcoin. As Gavin Andresen, head developer of the Bitcoin protocol, said, they’re either going to be playing security whack-a-mole or will scale down their blockchain massively.

The idea of a Turing-complete scripting language is useful; it allows you to do everything and anything that you want. Bitcoin’s scripting language is purposely limited to prevent malformed (whether through malice or incompetence) scripts like infinite loops. Gavin Andresen said most of Ethereum’s aims could be implemented in Bitcoin, and the core developers have already started implementing some of them.

For the sake of dapps and the scope of this book, we’re only going to concern ourselves with asset creation and smart contracts; bets, derivatives, and protocols that require a separate currency are not things I will be discussing. Pragmatically, the simplest way to get people to use your dapp is to ensure that it is compatible with a currency they already own, and the largest one is Bitcoin. So, you should issue a colored coin either on top of Bitcoin or on top of a sidechain, which is essentially just Bitcoin underneath the hood with some additional features like faster transactions.

Decentralized Identity

The notion of identity has been debated for centuries, and in the Internet age, the term takes on a whole new meaning. What is identity? Who owns identity? How should identity look on the Internet?

Due to the recent advancements in cryptography, a lot of the solutions have been “assume a public-key infrastructure.” Basically, assume that people would be willing to store a private key safely and identity will be decentralized. Only those with the keys would have access to it. BitAuth is a good current example of this. BitAuth uses Bitcoin’s existing technology to create a public-private key pair using secp256k1. It allows for passwordless authentication across web services. It gives you a system identification number (SIN) that is a hash of the public key. It uses signage to prevent man-in-the-middle (MITM) attacks, and a nonce to prevent replay attacks. Your private key is never revealed to the server and you can store it safely and securely. Identity is decentralized, so instead of having to trust a third party to store your identity, you can just store it yourself.

Other attempts to consolidate identity on the Internet have been made, with varying degrees of success. One of the most prominent attempts has been the OpenID protocol. OpenID is a decentralized identity protocol that takes advantage of existing web protocols like HTTP, SSL, and URI. The idea is that identity is fragmented across the web already, and by using the OpenID protocol, users can transform existing URIs into an account which can be used at any OpenID-supported sites.

OpenID abstracts the need to store your identity with the service provider so that you can only use a trusted source and that your identity will be carried around the multiple providers. This attempt seems to be the most successful so far in consolidating identities: companies like Google, Yahoo!, and Twitter have been OpenID providers. This is good in a way: now we can carry our identity across different sites without having to reregister again and again. It’s better not only because it’s more convenient to not need to reenter details, but also because you don’t have to trust novel services with storing your identity data. But OpenID still creates a potential security vulnerability because you are still trusting one of these service providers with your data.

The problem is also known as Zooko’s triangle (see Figure 2-3), and Namecoin was developed to help solve it.

Figure 2-3. Zooko’s triangle

Zooko’s triangle is the conjecture that states that in a system that is meant to give names in a protocol, only two of the three desirable attributes (human-meaningful, decentralized, secure) can be achieved. OpenID solved security and human-meaningfulness. Namecoin completed it by adding decentralization to the mix. Namecoin was essentially a third-party identity provider, the blockchain itself, that you could use as an intermediary between you and the service requesting your identity. The Namecoin blockchain was one of the first forks of the Bitcoin blockchain, and it has stood the test of time because of the value it provides.

Whereas most altcoins wither away, Namecoin remains because no other innovation has completed Zooko’s triangle. A user can “register” her name into the Namecoin blockchain by sending a transaction with her required name embedded in it under the /id namespace. When the user sends the transaction, Namecoin stores it if it’s unique—if no one has stored it before—otherwise it doesn’t. That means the namespace is limited to the names people can think up. Although this means that users can create and select their own (human-legible) identities, it can create a new problem because of the limits of human-legible phrases. Fragmentation of identity does benefit the user because in a new service, you get a whole new namespace from which to choose identities.

In other words, this is a tradeoff: having a universal identity provider that completes Zooko’s triangle is an important innovation, but the namespace is limited. Be that as it may, this is also the case with domain name registration. Currently ICANN controls name registration, a centralized organization backed by the United States Department of Commerce. (As of this writing, this is under tentative agreement to achieve formal independence in September 2016.) Namecoin has proved popular for registering .bit domains as a decentralized alternative to ICANN. These .bit domains can’t be accessed from regular browsers like Chrome or Firefox. To access them, the user currently must use either a .bit web proxy or download an extension. As .bit grows in popularity, it is possible that the protocol will be adapted by the browser natively.

Most people won’t need to create a .bit domain; it adds friction between you and your end users because they need to install extra software or go through a proxy to access your website. But if you have some message you need to get across, something your government or other authority figure doesn’t want you saying, .bit plays a perfect role in freeing you from Internet censorship.

So registering a username in the Namecoin blockchain is relatively easy. You just exchange some Bitcoin for Namecoin, download the wallet, and register the name. But what about logging in to the Namecoin blockchain? How does authentication and authorization work? Recently, NameID was created. It combined the best of both worlds; a user can use his Namecoin /id to log in to all of the thousands of openID-enabled websites. This reduces the barrier to entry for Namecoin to enter the mainstream app market.

So what’s the catch to decentralizing identity? Well, it’s the same catch as there is in decentralizing data and wealth with IPFS and Bitcoin, respectively: the user must store his private key. This is fine for hackers, who love decentralization and privacy. They are the most ideological population when it comes to using the right tools on the Internet. Hackers pride themselves on their innate drive for efficiency and perfection in the tools that they use. They encrypt their communication with GPG and use Tor clients to safeguard their browsing history from nosy ISPs and governments.

Storing some extra private keys for the sake of decentralization is undoubtedly the right thing to do for them. What about the mainstream populace? Do they really care about these things? I don’t think privacy and decentralization are on the top of their minds. Combine that with the average level of computer-security literacy, and I think it’s fair to say that most people are probably not able or willing to securely store encryption keys. The market demand for centralized storage is evident in the success of Coinbase, Bitcoin’s largest application so far. Coinbase is the exact opposite of decentralized: it’s a bank for Bitcoin. It provides private-key storage as a service.

A lot of the Bitcoin community is against any form of centralization; some eschew even the slight centralization in the forms of trackers in BitTorrent. The real question is this: How far are you willing to go to decentralize your software? Do you want to decentralize your domain name and make users store three separate sets of keys, as well? The answers to questions like this depend on who your audience is and if the benefits of decentralization are worth it. When it comes to something like “decentralized Dropbox,” a competitor to the current Dropbox, the answer may well be yes. If a competitor could come along and promise decentralization of its data with the same security benefits, I’d wager that there are enough people who think that’s a great reason to securely store a private key in order to make such a system work.

Even in cases for which those people don’t want to, some sort of business will come up that offers Storage-as-a-Service. I have to admit, even as a longtime Bitcoin user myself, I use Coinbase’s services to store my Bitcoins. I just don’t want to have to worry about them getting hacked on my computer! I’m willing to trust Coinbase more because it holds so many users’ assets, the CEO seems trustworthy to me (or at least much more so than Mark Karpelès of Mt Gox), and it is backed by a pair of trusted investors (Andreessen Horowitz and Union Square Ventures).

I don’t think we should necessarily be out to create completely trustless systems, but instead more trustful systems. I like the example of a train. Imagine a train that is going from San Francisco to Los Angeles that suddenly crashes. If the train has centralized its control to the conductor, the world knows whom to hold accountable (the conductor). If control of the train was completely decentralized to every passenger, no one could be personally held accountable so it would be difficult to find the bad actor.

Decentralization is not good for its own sake; it must have purpose and a real use case. Dapps can range in the level of decentralization they go for, and their levels will depend on their individual use cases. When it comes to creating a protest-forming app in China, the dapp should be decentralized, top-down, no questions asked. If it’s a social network for which your goal is wide acceptance, using .bit is probably not the best idea for a domain name.

So, if you’re using a dapp that is storing data on IPFS and issues a native currency using colored coins on the Bitcoin blockchain, you’ll probably also want to use NameID to store user identities. There will then be three sets of keys consolidated into some kind of either local or third-party key store that the user will employ to access and utilize your software.

Decentralized Computing

So, we’ve covered decentralizing identity, data, and wealth, but what about computing? Can you store your web app directly in IPFS and run it? Well, yes and no. IPFS is just a file system, and like any file system, you can easily run and display static websites on it perfectly fine from the browser. But when it comes to what we today call backend apps—dynamic apps, apps that require a shell and compiling environment to run, such as Node.js and Ruby on Rails—IPFS cannot do that. Thus, even if your app’s data is stored on IPFS, where do you store the source code?

Well, there are two options. The first is to store the data in IPFS and host your source code on a traditional virtual machine (VM) for web apps like Heroku. A VM is an emulation of a particular computer system. VM operation is based on the computer functions and architecture of a hypothetical or real computer. VM implementations might involve specialized software, hardware, or some combination of both. Heroku is a popular Platform-as-a-Service (PaaS) offering that provides the capabilities of a VM easily to a user. It can run dynamic backend code like Go and Node.js, and also store your data with an internal hosting service utilizing databases like MongoDB.

If you store your source code on Heroku and your data on IPFS, users can trust that the data belongs to them and you aren’t selling it to outside sources for profit. But what they can’t do is trust that the source code you open-source is what’s actually running on the server. Aside from this lack of verifiability, it also means that there is a central point of failure (Heroku). A second way to deploy is by storing your users’ data on IPFS and deploying your source code to a decentralized VM built on top of IPFS. Does this exist? The closest project to achieving this is astralboot. Basically, this is a golang server that pulls its files directly from IPFS and lets you run a Debian environment based on IPFS. This means that if you deploy a dynamic app on top of astralboot, it’s built on IPFS and you need only configure your particular environment on top of astralboot’s Linux environment.

There are other ideas like Ethereum’s own EVM (Ethereum virtual machine). Ethereum’s blockchain differs in many ways from the Bitcoin blockchain: different block times, Turing-complete contracts, and it acts as a decentralized-state machine because of it. I think it is a VM, but not a complete one and certainly not a VM that most developers would need. Requesting data from third-party sources is almost necessary in today’s software market; there are a multitude of competing services that specialize in niche areas of data that offer their API to other services to you. Instead of having to reinvent the wheel every time and create trusted data sources for your app, you want to be able to use third-party APIs. The problem with Ethereum’s EVM is that you can’t get data from outside the blockchain unless it’s been preconfigured to work with Ethereum by setting up a smart contract inside of its server.

This is great for new APIs that act as “oracles” (trusted federated sources of data), but it’s not good for existing services. Neither the Ethereum blockchain nor the Bitcoin blockchain can request data from outside of itself. This is both an inconvenience and a security constraint purposefully implemented. If it were possible to call APIs from the blockchain, a hacker could outrun the blockchain with varying data requests, and eventually it would result in bloating the network. So using the blockchain alone as a complete VM is not a good idea.

Another project is Go-circuit, which creates small server processes that run instances on a machine cluster. They form a churn-resilient and efficient network, which enables distributed process orchestration and synchronization from any single machine. It’s made for Go programs; it’s a distributed runtime, and it has Docker integration. It’s cool if your project is coded in Go, otherwise it doesn’t work for you.

All this talk of decentralized computation begs the question: What if computing were a marketplace? Imagine a sidechain in which the proof-of-work did actual useful computation for the network—some coins like gridcoin and primecoin already do this. But, they aren’t utilizable (in a verifiable way) for new computations that users decide; they’re based on existing computations that the coins’ creators need. What is needed is P2P decentralized computing that allows for an easy-to-access interface to which dapp developers can deploy their code.

The network we desire would have its own compute coin, and dapp developers would pay miners, engines of computation, to compute their source code. As more users come on board, the value of the network—and subsequently the coins—would grow. This is an area of research for crypto, so as of now I believe something like astralboot, which is under heavy development, is our best hope to fastest method to prototype.

Even if astralboot is too difficult to configure, Heroku works just fine; if Heroku fails and your code is open source, anyone can just reupload it to a new server, accessing the permanent, user-owned, and publicly verifiable data living on IPFS. If the computation marketplace becomes possible, you’ll be able to run dynamic apps directly from the browser as a domain on the web.

Decentralized Bandwidth

So far, we’ve talked about the four main points that can be decentralized in a dapp: identity, wealth, data, and computing. We’ve talked about domain registration, as well, which isn’t as necessary for most use cases. To add to that list, let’s discuss decentralized bandwidth.

For most users, ISPs act as gateways between you and the Internet (see Figure 2-4). There are several ISPs across nations that help connect people and act as the centralized hub; additionally they solve the “last mile” problem by connecting end users to the high-speed, high-capacity “trunk lines” of the Internet. The “last mile” is just a phrase used by telecommunication industries to refer to the last leg of the telecommunication network that delivers communication connectivity to retail customers. It’s the Internet cable that actually reaches the customer directly.

ISPs like AT&T and Comcast are beneficial in that they give us Internet access where there is currently no other alternative. The downside is that these centralized gateways are also central points of failure. Governments can shut them down at will if they so desire. Governements can ask the ISPs to censor certain IP addresses that they don’t want you accessing. ISPs will be forced to comply with the laws of the land and create a blacklist of web IPs that a user should not visit. There have been recent periods like the Hong Kong protests and the Arab Spring during which users feared government shutdown of ISPs. Some governments actually did shut down and censor Internet access by controlling these gateways maliciously to quell an uprising.

Figure 2-4. The Internet

It all works because there are no alternatives, but alternatives are beginning to appear. The latest example is the Firechat app for iOS, created by a company called Open Garden. Firechat lets phones speak to each other directly, peer to peer, using the iOS multipeer connectivity feature. No ISP is required. Firechat is an example of a mesh networking application. Mesh networks are the decentralized version of the standard centralized Internet. In a mesh network, users don’t need to go through a central gateway to access a site; they can connect directly to the nearest router, which would be a nearby computer.

There are many meshnets already in use: Spain has one of the largest with more than 50,000 users who needed Internet access that no ISP provided. That meshnet is still in use. During a hurricane in New York, a meshnet was used to relay valuable rescue information when the lines were down. There are tons of meshes in San Francisco that are “dark” nets, as well, only open to visitors inside of the secret societies they instantiate. Meshnets don’t generally have access to the regular Internet with all of its data. Without having tunneling in the loop, there is no access to the normal meshnet. Tunneling can only be done by switching from tunnel to mesh networking and back. Currently, doing both simultaneously isn’t supported by any major hardware manufacturer, but you can have a hybrid and that is being worked on by projects like Open Garden and CJDNS. This one is a little harder than the rest because in addition to software changes, it requires hardware changes. Routers should have the ability to access both nets so that they can pull data from the regular Internet and use it on the meshnet so that it can’t be shut down.

Although decentralized bandwidth is nice to have, it’s only necessary in certain countries that censor Internet websites and block access to the Web altogether. I think that if this happens on a larger scale, a real need for this will cause decentralized bandwidth to become mainstream. Using the blockchain, we can have other computing devices act as gateways instead of ISPs. They could be paid for routing data using a cryptocurrency and proof-of-bandwidth.

Just as cryptocurrency enables P2P marketplaces for computing and data storage, it can enable it for bandwidth sharing. Cryptocurrency enables marketplaces where centralized power structures once existed. These could be marketplaces for computing, storage, bandwidth, and any of a wide variety of scarce resources both “real” and “artificial” that people can think up. We’re going to see the economy shift more and more toward one based on information, and the market for data will more than likely be the biggest in the world as automation slowly eats away at everything labor based.

Decentralized Markets for Decentralized Assets

With marketplaces come financial instruments like derivatives, assets, currencies, and futures. A problem arises: Where can people exchange these assets? Traditionally we’ve used centralized stock exchanges to exchange assets, but how would this work in a decentralized information economy? Trading assets used to be decentralized before there was more infrastructure; for example, in barter economies. We can now combine the open exchange of information that the Internet provides with decentralized models of ownership verification and more. Goods can be thought of as assets in this case with no central oversight whatsoever. Governments have provided a trusted intermediary to ensure legitimate cross-border exchanges and value stability. A lot of trust goes into trading and securing assets; governments are the most trusted source we could create to pull this off so far.

Assets that are created by the government (like state currencies) have always been under the tight grip of government oversight. So, what happens when we add non-state-run actors into the mix? Currently, there are a lot of regulatory investigations into the Bitcoin network and how the United States government and other governments should regulate it. Bitcoin has been outright banned in some countries like Bangladesh. In some countries, Bitcoin is accepted by the government as legal tender and exchanges are able to occur with government oversight through existing stock exchanges.

For most countries, government opinion on Bitcoin is unsure at best—and suspicious at worst. Governments are still trying to understand this invention as it is only seven years old. They can be slow to change and the legal climate might prevent Bitcoin from ever entering their market. What is needed for an ungated, free, international market is a decentralized stock exchange for the new economy that doesn’t depend on any central government or corporate entity monitoring it. In the information economy where dapps are mainstream, each user of a dapp is also consequently a shareholder.

The shares that they own are the native currencies of the dapps, colored coins, or Bitcoin sidechains in the most likely scenario. These currencies will fluctuate in valuation with the valuation of the dapp: as more people find the app useful, its valuation will grow, and so will its shares. Further, these shares can pay out dividends from the dapp’s stream of revenue, making them even more valuable. Users will eventually want to trade these assets for more durable, stable forms of assets like currencies—maybe a USD-backed Bitcoin sidechain like bitUSD. BitUSD is a novel cryptocurrency pegged to USD that can be traded for BTC seamlessly. They could also trade for a price-stable cryptocurrency that has seignorage built in, or a currency with its own decentralized reserve. Users will needs stability in some of their assets, in short, and will eventually want to convert some coin to whatever will maintain their purchasing power when they are happy with their value and don’t want to risk losing it from investment in a dapp.

So far, exchanges for cryptocurrency have relied on a centralized source to hold your money, so for the duration of the exchange you had to trust them. This has resulted in numerous thefts from—and, it seems, by—central entities, mostly because they are not backed by any governing legal contracts,given that Bitcoin is still under examination by most governments. The most infamous example is Mt Gox, an exchange based out of Tokyo run by Mark Karpelès. Gox was huge in Bitcoin’s early days and a lot of people had used it. In February of 2014, it shut down, and there was no explanation as to the loss of customer funds. People were furious, but there was hardly a way to discipline Karpelès due to Bitcoin’s murky legal situation.

Since then, people have grown more and more distrustful of exchanges. Ideally, we could list our dapp asset/stock prices on government stock exchanges, but that would take a lot of regulatory approval and people don’t want to wait for that. On top of that, getting on a federal stock exchange requires a company to issue an IPO. An IPO requires a company to have a certain amount of capital in the millions, hire investment bankers and lawyers, and file mountains of paperwork. This is a very high barrier to entry and because the stock exchanges hold nation-based monopolies on trade, they are able to maintain that system.

Bitcoin has given us the opportunity—the choice—to opt out of the financial system and we should have that option. Not because the government is “evil” and “endlessly printing money,” but because people should be given the option, and the more options the better. Let’s decentralize the IPO and let early stage startups sell shares of their stock to people who aren’t accredited investors. The accredited investor requirement limits the people who can fund a corporation to those with at least a million in the bank. Crowdfunding sites are useful, but they don’t give funders equity in return because of this requirement. We need a decentralized stock exchange. So, how would it work?

There are some ideas that involve Ripple-style mechanism of exchange. (Ripple-style meaning you can choose who to reach consensus with, as opposed to proof-of-work where you trust the majority of work). Stellar is in some ways the new Ripple; its founder, Jed McCaleb, teamed up with some Stripe-backed teammates like David Mazieres (a brilliant Stanford researcher of distributed systems) and decided to create a new altchain. Stellar is quite literally the Ripple source code white-labeled with a new consensus mechanism that is still being ironed out.

The basic idea is that you can trade your currency with anyone else by trusting a third-party intermediary to administer the exchange. The trusted third party could be a bank or central exchange or even a friend. The system is interesting because unlike many altcoins, it’s not completely decentralized. It adds in an element of interpersonal or organizational—that is, social—trust to facilitate exchange.

This isn’t ideal, but truly decentralized order matching with zero trust hasn’t been solved and is most likely impossible prior to a major leap forward in AI. Centralization has its downsides, but it also offers a positive: accountability. In the Stellar model, the intermediary nodes are held accountable for any mishaps, and if any occur, their reputation goes down. This model in and of itself has value, but the problem with Stellar, as with most altcoins, is that it introduces a new cryptocurrency and its consensus mechanism has yet to be proven to work.

Proof-of-work is the only known Sybil-resistant solution that works at scale. What does that leave us with? The most decentralized exchange I know is called Mercury by a developer named Mappum. Mercury is a multicoin wallet that uses the cross-atomic chain (CAC) transfer protocol of Bitcoin to exchange value between cryptocurrencies. The CAC transfer protocol lets Alice and Bob, who own coins in separate cryptocurrencies, to exchange them without having to trust a third party. Both cryptocurrencies need to have the protocol implemented for it to work. The actual order matching is done on a server, but the wallet holds the values locally so there is no risk of theft. Mercury is an open source project that is available on GitHub and is actively under development but already has several users. As the wallet grows in popularity, users will want to store all of their assets in the wallet and exchange them with the click of a button for other forms of value. It’s the predecessor to the holy grail of wallets, one that holds any and all cryptocurrency.

There likely won’t be one that recognizes all the different competing protocols, just as web browsers don’t recognize the Xanadu proposal or any of HTTP’s other failed competitors as protocols. And yet browsers are used by everyone who wants to access the Web. Bitcoin, colored coins, and sidechains will win out in the end. BTC-blockchain-backed assets are the most secure due to the 500-plus supercomputer computation powered security of Bitcoin’s blockchain, its first-mover advantage for market share and mind share, and its superb community of developers working to increase its scale. Any company can add its coin to the open-source Mercury wallet, no IPO necessary.

Even if Mercury doesn’t win, the model is solid and it means the barrier to entry for IPO will be diminished for those who issue assets via cryptocurrency.

Practical Decentralization

Compliance with government regulations is a must for success in the modern world. One need only look at PayPal, and now Coinbase, to see how government compliance builds trust among users and lets a business grow internationally. Unfortunately, issuing assets for your registered corporation in the United States to people who aren’t accredited investors on a decentralized stock exchange doesn’t scream government compliance. So what can dapp developers do to create their dapps and move out of the shadows and into the mainstream alongside Facebook, Instagram, and Vine? Here are three suggestions:

• Start your corporation as a nonprofit

• In your legal documents, label your assets as app tokens to unlock features

• Add your asset listing to a decentralized exchange like Mercury

That’s a solid beginning, and it might be all that’s necessary with effective decentralization. You can offload the legal landmines of dealing with decentralized exchanges to the exchanges themselves. The exchange is decentralized, so it can’t be shut down, except for the server that order-matches. But because the app is open source, users can either continually make new server backups or insert their addresses.

Using sidechains and colored coins eliminates the stress of having to deal with different chains that might or might not support CAC swap, which would require a patchwork of code to accommodate compatibility. If everything is just Bitcoin under the hood, life is easier for a programmer to build apps that use several different currencies. For example, imagine a decentralized Dropbox built on IPFS. Users would pay the underlying networks fee (Filecoin), possibly a fee to register their usernames on the Namecoin blockchain, and possibly some internal currency fee for the service. How does one transaction split up into three different currencies, hitting different chains? If everything is just Bitcoin under the hood, value can transfer seamlessly between the coins.

This is why Bitcoin seems the strongest basis to be the blockchain on which all other financial assets are built: it works better than anything else so far, it’s simple to think about, and it’s simple to implement. But what about having to decide between coins and their use cases? Bitcoin’s transaction times could be faster. Litecoin as a sidechain can speed this up by a good amount. Then there’s Darkcoin, which keeps your transaction history private by scrambling and encrypting the transaction data on the blockchain. Primecoin and Gridcoin let you utilize the proof-of-work to solve scientific problems rather than just have it go to waste as computational power to back the network. How can we cherry-pick the features we want out of different cryptocurrencies without having to hold all of them and pick and choose between them?

The answer lies in a universal wrapper for all cryptocurrency. Picture a wallet that stores all Bitcoin-backed cryptocurrency, similar to what Mercury aspires to be. This is useful, but still presents the user with having to worry about which currencies to spend and when. Instead of this, the user is presented with one currency balance and features that she can turn on and off at will. The wrapper would turn currencies into features that end users could easily switch between.

Installation on the backend could be as simple as Node.js’s packet manager. Want faster transactions? bpm install lite. Private transactions? bpm install dark. Help compute scientific data? bpm install solar. Transactions would filter through the various blockchains necessary to ensure that they have all the features a user requests. Users could store assets separately, just as they do today in portfolio accounts/online wallets or in their currency wallet, as an all-in-one wealth management wallet.

Let’s consider the practicalities: Storing currency in your computer wallet is dangerous. It’s a decentralized alternative to banks and some people think it worth the risk because it’s an ideological decision. Some people don’t like the idea of banks—they can be corrupted, and given that they are central institutions they can fail; lots of people’s eggs in one basket. But most people aren’t ideological! It’s the sad truth; most people don’t care about centralization or data security or data ownership. They just want something that works well at solving their problems.

A bank solves a very important problem for people: secure storage of wealth. Storage as a service has been around for a very long time and continues to provide value to people. Coinbase, for example, is the most popular Bitcoin-based service and there’s nothing decentralized about it. It has essentially become a bank: it not only holds your Bitcoins in a secure online wallet, it also has begun holding your USD, as well! As Coinbase becomes more popular, so does Bitcoin. It’s very easy to use and takes the worry out of storage. If Bitcoin is ever going to become mainstream, banks will need to accept it as currency. It sounds radical, given that Satoshi created Bitcoin as a way to avoid the chargeback problem that all online transactions tend to have. Why would we store it in a bank? The distinction is akin to carrying dollars locally versus carrying them in your bankcard.

Bitcoin gives us the option to have chargeback-free transactions over the Web just like cash if we so choose. But most people will opt to use something like a bank. When a bank works well, it’s a great service. We can spend our currency anywhere without worrying about it being stolen due to bank’s insurance on our balance and with its security keeping us content. Traditional banks will either have to evolve to accept Bitcoin as a protocol for money transfer or be surpassed by new competitors like Coinbase. Imagine if your bank stored your Bitcoin balance, as well. It would show your balance in your native country’s currency and it would have an associated account number, routing number, and Bitcoin public keypair.

If anyone sent you Bitcoin, it would go directly to your bank account. If anyone sent you state currency, it would go directly to your bank account. You would be able to turn on price stability as a feature due to the universal wrapper so your balance would be as stable as it is currently. You could spend your Bitcoin anywhere you spend state currency because they would become indistinguishable.

Bitcoin will find its niche. Most people don’t care about the word Bitcoin and just want to use their own currency. The leaders in the Bitcoin space like Abra won’t even mention the word Bitcoin. They will just use it as a protocol for fast reduced-fee international money transfer and micropayments.

Abra is a startup that aims to tackle the huge remittances market in the developing world by creating a series of decentralized tellers that will exchange state currency for Bitcoin, and vice versa. They use the term “digital money” instead of Bitcoin, a smart move that won’t scare away mainstream consumers and will most likely boost their sales. As a very practical matter, outside of the Silicon Valley state of mind, people don’t really care for Bitcoin.

The rise of Bitcoin ATMs is fascinating, but visit any developing country without proper financial infrastructure and you’ll see the transition from an all-traditional ATM network to an exclusive all-Bitcoin ATM network isn’t going to happen in our lifetime. Instead, Bitcoin should complement the existing payment infrastructure by making it faster and cheaper when needed (wire transfers, especially internationally), and allow for all the features that cryptocurrency can offer a user, like micropayments—which can in turn enable dapps like microblogging marketplaces.

No one will be forced to use a bank, an identity, data, or centralized computation, but they can make life easier. We have the choice to decentralize when necessary, and hopefully as users get more aware of the value of their data, the world will slowly began to understand the importance of securely storing public keys themselves.