The complex user onboarding experience is one of the main issues for achieving mass adoption in Ethereum. Users new to the space need to install a dedicated browser or extension, create and back up an account, and then acquire ETH just to begin interacting with a DApp. While in previous chapters we have worked with web3-enabled users, in this chapter we will look into ways for simplifying the onboarding experience for new ones.
We will begin by exploring scenarios for interacting with the blockchain without having to create an account or creating one for our users behind the scenes. We will take this one step further by moving account management itself to the blockchain with smart accounts while exploring contract upgradeability along the way. Then, we will shift our focus to the problem of requiring ETH to interact with Ethereum, and introduce gasless transactions: a technique that removes the need for your users to pay for gas fees. We will wrap up with a review of the Ethereum Name System, which hides raw addresses behind user-friendly names, making them more accessible to your users.
The Problem
Try to remember what your first steps were when using an Ethereum-powered app. The first one was probably acquiring ETH to fuel your transactions. If you were not lucky enough to have someone gift you some crypto, this requires signing up in an exchange and potentially verifying your account. Depending on the exchange, this sometimes even requires uploading a selfie holding your passport or a utility bill to prove your address. Once whitelisted, you need to send funds to the exchange old style, such as via a wire, to get your ETH.
Next step was getting your funds out of the exchange into an account you control. After careful research on hardware, mobile, desktop, and online wallets, you can settle for one and actually create your account. Regardless of the wallet you have chosen, this involves writing down a set of 12 random words and keeping them somewhere safe – but not too safe, because losing them means losing your only backup and thus all your crypto. After choosing a passphrase for your wallet (but hadn’t you just written down 12 words for security?), you are finally presented with your public address and can move your funds out of the exchange and under your control.
Armed with your funded account, you now open your browser at a DApp to try it out. But alas, the site cannot find a web3 provider and is asking you to install a dedicated browser extension (such as Metamask) or use a web3-enabled browser (like Opera) to interact. After doing so, you decide whether to trust this new component with your 12 words – that were supposed to be kept super safe, is it a good idea to enter them on a browser extension you just downloaded? – or set up another new account. Should you pick the latter, you need to go once again through the same process of backing it up via the 12 words, choosing a passphrase, writing down the new account, and transferring your ETH to it.
After these steps, you can now finally send your first transaction and buy a digital-collectible crypto hat from your favorite decentralized store. Hooray.
Now, if you are reading this book, you are most likely a programmer and have a knack for technical challenges. Setting up your own accounts can be cumbersome, though it comes with the thrill of becoming part of the vanguard in a decentralized revolution.
But put yourself in the shoes of your average user, who will outright close a page if it takes more than a few seconds to load. Most users demand immediate satisfaction, and requesting them to go through such a complex process just to start using your application is a recipe for disaster.
While it is not always possible to remove all of the preceding required steps, it is certainly possible to hide some of them to the user or delay them until they are involved enough in your application that they are willing to invest a few minutes to go to the next level. We will review some techniques for achieving this throughout this chapter. Be warned though – there is no silver bullet for this problem.
Interacting Without Accounts
We will start by tackling the task of setting up of an account – which involves choosing a wallet, writing down the mnemonic, setting up a passphrase, and so on. The easiest way around these steps is for your app to not require an account at all for interacting with it. While this does not fit all use cases, it can be tweaked to cover many more than you would expect.
Sending Funds from an Exchange
The simplest scenario to get started with this method is just receiving plain ETH at an address. For example, a donations application only needs to list a payment address where users can send plain transactions directly from an exchange or from any wallet they control without forcing them to use a web3-enabled browser. Certain services even provide embeddable widgets that you can integrate in your application, so users can buy the ETH they need using fiat without even leaving your site.
Plain Transactions and Fallback Functions
Caution
Keep in mind that exchanges do not generate unique ETH addresses for each user or transaction. This means that you cannot rely on msg.sender in your contracts to try to identify the user to trigger the transaction, if you expect your users to send funds directly from an exchange.
Remember that every time a contract receives a vanilla transfer of funds, the fallback function is executed, meaning that you can actually react to a transfer via a certain action. In particular, you can rely on the precise amount of ETH sent. However, some exchanges will only send the transfer with a minimal amount of gas, so it may not be possible to perform complex actions as a response to an ETH transfer.
Note
An exchange that fails to set a reasonable gas limit to its transfers can actually be exploited. As described in the vulnerability “Failure to set gasLimit appropriately enables abuse”1 by Chris Whinfrey and others: “Many exchanges allow the withdrawal of Ethereum to arbitrary addresses with no gas usage limit. Since sending Ethereum to a contract address executes its fallback function, attackers can make these exchanges pay for arbitrary computation. This allows attackers to force exchanges to burn their own Ethereum on high transaction costs.” All exchanges should set a strict gas limit to their outbound transactions.
Forwarding Contracts
A neat trick to allow the user to send additional information along with their contribution is to set up several contracts that act as forwarding proxies and execute certain functions in a main contract as their fallback function. Though this does not allow for arbitrary data to be sent, it allows for a predefined set of functions to be run.
Sample implementation for a contract that accepts donations for one of two parties. The user must call either donateA or donateB when sending funds
Obtaining the data component for a transaction to the donateA method of the Donations contract, given its ABI
Sample DonateA contract that forwards all transfers to the main Donations contract by calling donateA. The code for DonateB is equivalent
Single-use Addresses
The msg.sender cannot be relied upon, since the exchange may use the same address for several withdrawals, or different ones for the same user.
The gas included with the transaction may not be enough to perform a computationally intensive operation or even writing to storage.
No data can be included in the transfer.
Users with their own wallets do not face these issues, since they hold an account of their own with its own funds, and from there they can send any arbitrary transaction. Though we could create an account for the user on the spot (as we will see later in this chapter), there is a simpler solution that works for short-lived interactions: single-use addresses.2 These are externally owned addresses that can only issue a single predefined transaction on their lifetime. After that transaction is executed, the address cannot be used again.
How to Use a Single-use Address
- 1.
The user chooses which action they want to execute in the application, including any arbitrary data, funds to be transferred, or gas to be used.
- 2.
A single-use address that can only execute that transaction is generated.
- 3.
The user sends funds from an exchange to that single-use address.
- 4.
Once the funds are received, the transaction is executed.
Note that if the transaction from the single-use address fails for any reason (whether it is a failed require in a contract or lack of gas), there is no way to issue a second transaction with the necessary fixes. This means that any funds that the user sent in step 3 will be effectively locked forever. As such, make sure to avoid using single-use addresses when a transaction could fail due to state changes.
As an example, the Donations contract from as in the previous section would be a poor choice, since the time while the donations are accepted is limited. A user may set up their single-use address while donations are open, but if they seed it after they close, then their funds will end up locked.
Another pitfall of single-use addresses is that they must be funded with the exact amount of ETH required for their execution. Any extra funds sent to the address are irretrievably lost. Make sure to clearly communicate this to your users!
Creating a Single-use Address
How are single-use addresses created? It is worth noting that they are not a special Ethereum construct, but the result of a clever hack. Also, creating them requires no initial gas at all.
Recall from previous chapters that all Ethereum transactions need to be signed with the sender’s private key in order to be valid. The sender’s address can then be derived from the transaction’s signature, so the from is actually never included in the transaction’s data – it’s calculated from the signature when needed.
- 1.
Pack the transaction parameters (recipient, gas, gas price, nonce, data, value, etc.) into a binary object.
- 2.
Hash the transaction binary and sign it with the sender’s private key.
- 3.
Broadcast the transaction’s binary and the signature.
- 4.
Calculate the hash of the transaction binary.
- 5.
Derive the sender’s address (i.e., from) from the hash and the signature.
- 6.
Unpack the transaction binary into its parameters (recipient, gas, gas price, nonce, data, value, etc.).
The trick for generating single-use addresses relies in sending a random signature with the transaction. Instead of actually signing the transaction in step 2, we just include a random set of bytes as a signature in step 3. This way, there is no private key associated with the process.
When processing the transaction, this results in a random sender address derived from the signature. And given that the private key associated with this address is unknown, it is not possible to generate any other transaction for the same address. This effectively yields a single-use address that can only broadcast one specific transaction. Then, as soon as the address is funded with ETH, the gas fees can be covered, and the transaction can be broadcasted to the network.
Sample Code
Simplified Donations contract that accepts a custom string that is emitted in an event in every donation
Caution
Keep in mind that actual gas usage may change depending on the state in which the transaction is executed. Due to the very nature of single-use addresses, if the transaction fails due to lack of gas, there is no way to execute it again with a higher allowance. This results in the user funds being locked in the address forever. If your function may end up consuming more gas in the future, make sure to account for it when building the transaction object in the upcoming steps.
Most parameters in the preceding snippet should be familiar by now: the recipient address, the amount of ETH being sent, the gas allowance and price, and the transaction data. Since this transaction will be sent from a new address that has sent no other transactions, the nonce must be zero.
The new parameters we see here are v, r, and s, which correspond to the transaction’s signature. The first of them must be derived from the chain id,3 while the two others are normally calculated from the user’s private key – we will replace them by an arbitrary bytes sequence here.
Note
It is important to use a recognizable fabricated sequence as r and s, such as a clear repetition of the same set of bytes, or a large amount of leading zeroes. Otherwise, a third party cannot know whether the resulting transaction object belongs indeed to a single-use address, or the signature was obtained from an actual private key. If a private key was indeed used to generate the transaction, then its holder can generate a different transaction when the user sends their funds to the allegedly single-use address.
However, since we have not yet funded the sender address, we will get a “sender doesn't have enough funds to send tx” error. Remember that the derived sender is a new account, so it will never have any previous funds.
Caution
Remember that any extra ETH sent to a single-use address will be lost.
Application Local Accounts
While in the previous sections we saw a few tricks for interacting with a smart contract application without needing an Ethereum account, we will now look into another alternative: creating an account for the user within the application. Instead of asking the user to download a web3-enabled browser or install an extension, we can manage the entire flow of creating a wallet directly in our app. This allows any user who navigates to our application being instantly supplied with an Ethereum address to start interacting. On the other hand, we also need to provide means for our users to safely back up their new wallets – without becoming custodians of their funds.
Creating and Using a Local Wallet
Creating an account using web3. Note that no connection to the network is needed to create a private key
Signing a transaction for sending ETH with the account’s private key and manually sending it to the network. Note that issuing this transaction requires seeding the sender address with some ETH
Registering an account as a wallet in web3. Any transaction sent from the wallet address will be automatically signed locally by the library and then sent to the network
Note
Recall from previous chapters that Metamask achieves a similar behavior using a web3 subprovider. Instead of relying on the node to sign a transaction, the transaction is intercepted, signed client-side, and then broacbasted.
Encrypted Keystores
However, this option is extremely insecure, as the browser’s local storage should never be used to keep sensitive information, since it is vulnerable to XSS attacks5 – not to mention the fact that the keys are lost if the user clears the site data or loses the device. Storing plain private keys like this is only recommended when managing very low amounts of ETH and loss of funds is acceptable, in other words, when usability is prioritized over security.6
Encrypting a user’s private key with a password to generate a v3 keystore object. The converse method, decrypt, takes the keystore and password and returns the decrypted account object
While this is a much more secure option, it introduces additional complexity by requesting a password from the user. Even though passwords are a common annoyance to web users, they are also used to being able to reset them whenever they forget them – an option that is not available in this case. If a user forgets the password to their keystore, its contents are lost for good.
In addition to this problem, the keystore itself must not be misplaced. If you are storing the encrypted private key just in the browser and the user happens to clear local data or simply loses the device, then all funds are gone with it. You must ensure that users perform proper backups (asking them to download the keystore from your page and saving it elsewhere) or save them on a safe location in one of your application’s servers (assuming you have one!).
Note
There is a big difference between acting as a custodian of your users’ funds and acting as a backup location for storing their encrypted private keys. While on the former any hack to your servers implies a loss of funds to your users, the latter is just a convenience for your users to keep their encrypted keys secure.
However, this difference may not be so big from a legal standpoint. Make sure to inform yourself on legal requirements in your jurisdiction if you do store users’ encrypted keystores server-side.
Mnemonics
A decentralized alternative for backing up your user keys is to generate a mnemonic. Mnemonics are sequences of words (usually between 12 and 24) that can be easily written down by a human for safekeeping, and you have probably already stumbled upon them when creating an account in a wallet. Keep in mind that they are meant just as a means of backup if the keystore is lost, as it is not reasonable to ask your users for a 24-word sequence every time they want to use your application.
The main benefit of mnemonics is that they provide a way to store digital information (a private key) in a traditional physical medium (a piece of paper). This way, even if a user gets all their electronic devices stolen or hacked, they can still rely on an old-fashioned method for restoring their accounts.
Unfortunately, it is not possible to generate a mnemonic from a private key. As we have seen in Chapter 5, the derivation process is one way: the mnemonic is used to calculate an extended key, from which one or more private keys are derived in turn. This means that if you want to allow your users to back up their accounts via this method, you need to build it from scratch.
Creating a mnemonic and deriving the corresponding private key. Note that more private keys can be generated from the same mnemonic by using different derivation paths
This snippet allows you to generate a mnemonic for your users’ accounts. Just note that your users need to save the mnemonic somewhere safe before refreshing the page, since you should never store the plain sequence of words anywhere, as it is as insecure as storing the unencrypted private key.
Up to this point, we have added several mechanisms for ensuring the security of your users’ funds, such as encrypting their private keys with a password, and providing an easy-to-write set of words for backing them up.
However, we have also reproduced the same complexity that we were trying to avoid in the first place – the complexity that prevents most users from actually getting onboard on Ethereum. Though we will see other approaches, the key takeaway from this section is for you to know the tools available, so you can settle for the proper balance between security, usability, and decentralization that best fits your application.
Smart Accounts
Local wallets have several shortcomings: not only they require additional steps for safekeeping, but they also offer very limited options for users accessing your application from multiple different devices. Even though you could rely on cloud services for synchronizing the encrypted private key across your user’s devices, revoking access from a device is out of the question. To make matters worse, additional security measures such as two-factor authentication for critical transactions cannot be implemented without trusting funds to a custodian.
Fortunately, we already have a decentralized and secure computing platform we can rely upon for building any additional security or recovery features for our users’ accounts. It is a matter of moving their very accounts to the blockchain as contracts.
Identities as Smart Contracts
The key concept behind smart accounts, also referred to as identity contracts , is that the user identity (along with its funds) is represented by a single smart contract instead of one or several externally owned accounts. The user controls a contract that forwards his or her calls to other applications and centrally holds the user’s assets. Managing this identity from a new device is simply a matter of registering a new externally owned account as a manager of the contract.
Note
Research on these contracts has gone under the name of identity contracts, smart accounts, bouncer proxies, multisig wallets, or wallet contracts. These contracts have usually gone hand in hand with meta transactions or gas relays, which we will review later in this chapter. While it is difficult to identify the first person who came up with this idea, it is worth mentioning Alex Van de Sande, Philippe Castonguay, Panashe Mahachi, Austin Griffith, Christian Lundkvist, and Fabian Vogelsteller for their work on this topic. Work on this area has also been inspired by the Account Abstraction Proposal8 by Vitalik Buterin, which aims at merging externally owned and contract accounts into a single type and moving the burden of signature verification onto the EVM.
The user can register multiple devices, where each device has its own externally owned account, to manage the same identity contract.
Different keys can have different access levels: from managing the identity contract itself to just being able to transfer a limited amount of funds.
Major transactions can require more than one key to confirm the operation, effectively imposing a two-factor authentication (2FA) for critical operations.
Social recovery of the account can be implemented by designating a group of trusted parties. This means you could pick a circle of close friends who could, all together, grant you access back to your identity contract in the event that you lose your keys. This could even be extended to testaments.
Keep in mind that using smart accounts does not remove the need for generating local externally owned accounts. Our application will still need to create a local key per device to manage the identity contract if the user does not already have a web3-enabled browser. The improvement on using smart contracts over regular accounts lies on the additional security and recovery features that can be implemented in a decentralized manner – for instance, social recovery can be used as a replacement for writing down a 24-word mnemonic.
Sample Implementation
A very basic identity contract with support for managing user accounts and forwarding calls
Creating an identity contract funded with 10 ETH and adding a second device. In this example, mainDevice and anotherDevice are accounts created on the user’s devices
Sending funds from the identity contract using anotherDevice
Calling the setGreeting method of a greeter contract instance from the identity, including 5000 Wei in the transaction
Most of the features discussed previously can be implemented on top of this base contract: accounts can be assigned different clearance levels, while restrictions based on the value transferred can be imposed on the forwarding function.
A good example of this is a multi-signature wallet .9 These contracts are designed to have a number N of accounts registered, and every transaction requires at least M (with M < N) confirmations. While they are often used for safekeeping of large amounts of funds by distributing the keys among different members of a team, they can also be used for managing personal funds by assigning different keys to different devices.
Deploying a Smart Account (Take One)
Given that the focus of this chapter is to make user onboarding easier, requiring the user to not only generate a local externally owned account but also to deploy a contract, fund it, and register their devices on it seems like a step (or many) on the opposite direction.
Nevertheless, many of these steps can be carried on by the application on behalf of the user – including deployments. After the user has performed a certain number of actions within our system, we can opt to deploy the identity contract on their behalf and transfer ownership of it to their account. Even though this implies costs in gas fees, in many cases this subsidy can be considered to be a necessary cost involved in user acquisition.
However, a more decentralized (and cheaper for us) option is to rely on single-use addresses for setting up the contract. Our application can generate a local wallet behind the scenes to act as the initial owner of the identity contract (if the user does not have a web3 browser already), as well as a single-use address that can trigger the deployment and configuration of the smart account once funded. When the single-use address is funded from an exchange, the identity is deployed and seeded with a predefined amount of ETH. From that point on, the user operates directly via their new identity contract.
Note
Remember from the single-use addresses section that any extra funds sent to the address are irretrievably lost. The user must choose how much ETH they want to initially send to their identity and then create the transaction and fund it with exactly the resulting amount.
Transaction parameters for deploying a contract. Note that the recipient is set as null, as we are creating a new contract instead of interacting with an existing one. Signature parameters are set as shown in the single-use addresses section of this chapter
Precalculating the deployment address using the ethereumjs-util library
However, an issue with this approach is that we are showing the user two different addresses: one address that he needs to fund one time for the deployment and another that will actually be their identity contract. This, compounded with the fact that any extra ETH sent to the address is lost, damages the user experience when setting up their smart account. We will analyze a more robust approach later in this chapter after we become familiarized with the concept of meta transactions.
Note
A much simpler option is to just have the user transfer their funds to their device account, whether it is local to our application or is managed by the web3 browser, and execute the deployment from it. However, this still has the issue of presenting two addresses to the user – one for initial funding and one for the identity.
Upgrading User Accounts
As we have seen, identity contracts can pack multiple features for your users, such as two-factor authentication, social recovery, daily transfer limits, and more. This begs the question of what to build into these contracts. As contracts are immutable, any change to add a new feature would require your users to ditch their current identity and deploy a new one. However, forcing a user to move all their assets to a new contract is the equivalent of forcing them to move to a new email account whenever they want access to a new feature – including changing their email in all online services they may be using.
Nevertheless, it is possible in Ethereum to actually upgrade a contract after it has been deployed, thus allowing iterative deployment and bugfixing as in traditional software development. To implement this, we first need to become familiarized with the concept of delegate calls in the Ethereum Virtual Machine, or EVM.
The DELEGATECALL Instruction
A regular CALL from one contract to another in the EVM works as a call from an actor or process to another: a new context is created, where the state (storage and balance) of the callee is loaded, the caller is set to msg.sender, and the code of the callee is executed. On the other hand, a DELEGATECALL works by executing the code of the callee but maintaining the original context of the caller. In other words, storage, balance, and even msg.sender are not changed by a DELEGATECALL – only the code being executed is.
This low-level instruction allows us to jump into an arbitrary contract and execute its code, where that code actually modifies the state of the current contract. The called contract then acts as a library,10 being used only as a repository of shared code, but not using its state.
Delegating Proxies
The DELEGATECALL instruction allows us to build a contract that simply delegates all calls to another contract which holds the actual logic to be executed. These contracts are usually called delegating proxies , since they delegate all calls to another contract, or transparent proxies, as any client interacting with them is oblivious to their existence. The contract being called by the proxy that holds the code being actually executed is usually called logic contract, implementation contract, or master copy.
Implementation of a delegating proxy contract based on the code from github.com/zeppelinos/zos. Note the usage of assembly in the fallback function, since Solidity does not allow a fallback function to return arbitrary data. And if we did not return any data, then any calls to our contract would yield an empty response
Caution
Upgradeability in the EVM can be very tricky to implement correctly, and it imposes some restrictions. For instance, a proxy contract cannot execute a constructor of a logic contract when it is created, so any contract to be upgradeable needs to rely on regular functions that act as initializers. Another issue is that updating a proxy to a logic contract with a different set of state variables may also inadvertently corrupt the contract’s state. If you are planning on relying on upgradeable contracts in your application, whether they are smart accounts or any other contract, it is strongly recommended that you use an existing upgradeability solution instead of rolling out your own.
The key concept here is that the address of the logic contract does not need to be hard-coded into the proxy – it can actually be kept in storage. When this address is changed, this effectively updates the code being executed by the proxy while keeping its state and address. We can now add this feature to our identity contracts.
Upgradeable Identities
Armed with this new building block, we can deploy a single identity contract to act as a logic contract and deploy one proxy for each of our users. Remember that since all state is kept in the proxy, there is no need for more than one copy of the logic contract, and a single one can be shared (like a library) among multiple proxies.
Variant of the identity contract with support for upgradeability. Note that the first contract variable is the implementation contract address, and the constructor has been replaced by a regular function that relies on a flag to keep track of whether the instance has been initialized or not. Upgrading to a different identity implementation just requires changing the implementation address in the first position of storage
Keep in mind that any state variables defined on this contract will not be actually stored on this contract’s storage, but in the proxy’s – thanks to the magic of delegate calls. For instance, the initialized flag is not actually set on the single UpgradeableIdentity contract deployed as a shared implementation, but is set on each of the proxies that are backed by it.
Because of this, the first variable declared on this contract must be the implementation address, as in the proxy. Also, the contract cannot extend from any base contract that defines additional contract variables. This ensures that the implementation address is stored in the same position in storage as where the proxy will look for it.11
Note
Contract upgradeability is a contentious issue for many in Ethereum. Having immutable contracts allows users to trust an application by knowing that the rules defined by it will not be subject to change. Adding upgradeability breaks this guarantee. However, this is not an issue with upgradeability itself, but with who can decide when a contract is upgraded. Having a decentralized token being controlled by a single developer who can unilaterally modify the contract by, let’s say, adding a transaction tax, is definitely not good. On the other hand, a contract that has a clear owner (or set of owners) is a good candidate to be upgradeable – and in this an identity contract is a perfect example.
Creating a proxy to an identity contract. Note that we need to use the logic contract ABI to build the initialization data and then use it when deploying the proxy
Interacting with the newly deployed proxy as if it were a regular identity contract. The proxy will delegate all calls to the logic contract and behave exactly like one
Upgrading to a V2 of the identity contract. The user’s identity address is unmodified, as is its balance and state, but it has access to new code with potentially new features and bugfixes
Implementing this pattern in smart accounts allows us to iteratively develop our application by gradually adding support for new features or fixing bugs and allows our users to adopt them when they want.
A good example of a feature we may want to add to our identity contracts is meta transaction support, which we will see in the next section.
Gasless Transactions
One of the root problems on user onboarding is that interacting with the Ethereum network requires ETH to begin with in order to be able to pay for gas fees. This involves an annoying set of steps just to make an initial ETH purchase with other currency. Gas fees are also problematic in multi-device solutions: all of a user’s devices must hold a bit of ETH just to run any transactions, even when relying on a smart account to centralize the user identity.
All of the above means that removing the requirement of gas for issuing Ethereum transactions yields major benefits in terms of usability. Gasless transactions, also referred to as meta transactions , tackle this problem by offering a mechanism to decouple the author of the transaction from the payer of the gas. In other words, an account can issue a transaction, while other pays for its execution. This allows users in your application to execute any transactions without needing to worry about having ETH to pay for their gas fees.
Note
Gasless transactions is one of the most actively developed techniques in the Ethereum ecosystem at the time of this writing, and already has many different flavors. This makes this section one of the most complex ones in the book. Feel free to gloss over the technical details, and consider looking into already established libraries, such as Universal Login, Marmo, or the Gas Station Network.
Signatures in Ethereum
Using web3 to sign a message from a user with their private key12
Note
Behind the scenes, web3 sign method prefixes the message with the string "x19Ethereum Signed Message: " and its length and then hashes it before signing. This is done to avoid tricking users into signing an actual transaction. Most APIs, even those part of the nodes or in hardware wallets, will handle this automatically.
Simple contract that relies on OpenZeppelin’s ECDSA to recover the signer from a message and its signature
Recovering the signer from the original message using the smart contract listed before. Here, signatures is a web3 contract instance that represents a deployed instance of the smart contract
Ethereum signatures are typically used off-chain to verify that a user controls a certain account, or that the owner of an account signals an intention (for instance, in off-chain voting) by signing a specific message. But signatures can also be used on-chain to verify that the owner of an account intends to execute a certain action. This is where meta transactions come in.
Introducing Meta Transactions
A meta transaction or gasless transaction is similar to a regular transaction in that it is a signed tuple of a recipient, an amount of ETH transferred, a set of data included in the call, a gas allowance and price, and a nonce. However, instead of being broadcasted to the network, the meta transaction is wrapped within another transaction and sent to a particular contract. This contract knows how to unwrap the nested transaction, verify its signature, and execute the requested action.
Note that the contract that first receives and processes the transaction is actually reproducing the same behavior as the Ethereum protocol: it verifies the nested transaction signature and nonce and performs a call to a recipient with the specified value and data.
The user, who signs a meta transaction that they want to be executed, but does not send it to the network
The relayer, who picks up that transaction, wraps it in another transaction of their own, and sends it to a contract, paying for its execution
We will go into relayers later in this chapter. Let’s dive into the implementation of meta transactions first, at the smart contract level.
Building on Our Smart Accounts
Meta transactions require a contract that can process them, validate them, check that the signer is authorized to perform the action requested, and carry it out on their behalf. Fortunately, we already have a contract that can hold funds and carry actions on behalf of different accounts held by a user: the identity contract.
We will modify the identity contract we have been building to handle meta transactions. Our forwarding function will now receive a meta transaction pre-signed by one of the user accounts instead of a direct call from one of them.
Modified forward function from the Identity contract (presented earlier in this chapter) to handle meta transactions
The address of the validator, which is the contract checking the signature
A nonce, increased on every transaction executed
The address of the validator is included to prevent a meta transaction sent to another validator to be reused in this one, while the nonce prevents replay attacks (i.e., relaying the same transaction multiple times). The resulting hash gets the magic prefix "x19Ethereum Signed Message: " prepended, so it is distinguishable from a regular Ethereum transaction, and is then hashed again.
Note
Certain implementations track a nonce per signer instead of one global to the contract. This depends strictly on your use case: if it is possible that multiple whitelisted accounts will be sending meta transactions at the same time, then you should have signer-specific nonces.
Next is the validation of the sender. Note that the contract no longer checks that msg.sender is a registered account – it now checks that the signer of the transaction is. The msg.sender becomes irrelevant, since who relayed the transaction does not matter at this stage.
Finally, and before executing the transaction, the nonce is increased. This prevents the same transaction to be sent to the contract multiple times by a malicious relayer.
Note
It is worth mentioning that almost all smart account, identity contract, or bouncer proxy implementations include some variant of meta transactions. Once the user identity is moved on-chain, it does not make sense to keep them bound to the restrictions of the protocol, such as having the sender account pay for the gas of its transactions.
Sending Meta Transactions
User crafts and signs transaction to be relayed. Here pk is the private key of the user, and greeter is an instance of a contract which has a setGreeting method
Relayer calls into the forwarding function of the identity contract with the parameters and the signature provided by the user. The identity contract will in turn call into the greeter, which changes its state
We are deliberately omitting from this example how the user communicates with the relayer or how the relayer decides whether to pay for the user’s transaction. This will vary from application to application. For instance, an application could provide a centralized relayer at a well-known URL, which pays for every transaction to a contract in their system. It may also force users to go through a CAPTCHA to prevent spamming.
There are also efforts toward building fully decentralized relayer networks, in which the logic on whether paying for a user transaction is actually part of the application’s smart contracts. These efforts also include approaches where the relayers are paid back for their execution, as we will see in the next section.14
Relayers and Rewards
The relayer is ultimately a process with a public interface (usually HTTP) that accepts signed messages from users, validates them, wraps them in a transaction, and pushes them to the network. For this last step, the relayer uses an account of its own – the gas fees are deducted from that account’s balance.
In all of our examples so far, we have assumed that the gas cost for executing the meta transactions was covered by the owner of the application. This owner would spin up a relayer and freely forward all transactions for users of their own application.
However, it is possible for the users to actually pay back to the relayer serving them. An addition to the forwarding function is to send some ETH back to the relayer (i.e., the msg.sender) to cover the costs of the execution. This way, the user does pay the gas fees associated with their transactions – only that they are paid from their smart account, entirely removing the need to keep ETH for gas in each of their devices.
This opens the door for new incentive systems: the relayer does not need to be centralized by the application owner and subsidize the execution costs, but it can be decentralized and just profit from relaying transactions. This leads to a new concept of desktop mining , where users can earn fees not from computational-intensive proof-of-work, but from relaying meta transactions for other users.
Relayer Reward
The easiest way to implement a payback to the relayer is to include a reward along with each request for execution. The relayer can then decide whether to relay the transaction or not, depending on the estimated execution cost for it.
The user can even request a specific gas price and gas allowance for its transaction and verify that these are satisfied within the identity contract. This prevents relayers from sending transactions with very low gas prices, paying lower fees at the expense of a user’s time.
Forwarding function of the identity contract with support for relayer rewards. The modified sections are highlighted in bold
An important change in the preceding snippet is that the function no longer requires the forwarded call to be successful (require(success)). The rationale for this is that if the forwarded call failed and reverted the entire transaction, then the relayer would not receive any reward, but would still lose the gas fees from the execution of the reverted transaction. To avoid punishing a relayer for a failed transaction that was correctly relayed, we drop that requirement. And to allow for determining whether forwarded call reverted, we add an event that reports the success in each transaction.
Also, note that the gas price validation is performed at the beginning of the method, since gas price is relative to the entire transaction, and cannot be changed in-between contract calls. Gas limit, on the other hand, can be enforced on each call within the transaction, so we can revert if there is not enough gas left. In this case, it is the relayer’s responsibility to include additional gas on the request to account for the meta transaction processing (which is about 60K in this implementation).
Estimating Profits
The code for generating the signed transaction in this scenario is analogous to the previous one, with the only difference that the user now needs to include values for the gas price, the gas limit, and the reward. The first two can be estimated as seen in Chapter 5 by using a price oracle like the ethgasstation API or the gasPrice JSON-RPC method for the gas price and running an estimateGas on the transaction to be sent for the gas limit. The value for the reward may depend on other factors, but must be greater than the total execution cost for the relayer, to have an incentive to relay the transaction.
In this scenario, the relayer needs not only to relay the requested transaction but also to evaluate whether it should – by calculating the profit. Transactions with an estimated profit below a certain threshold should be dropped, and if multiple transactions (from different identities) are enqueued, the profit can be used to prioritize which to execute first.
Note that this difference may actually cause the transaction to cost more than the relayer expected in case the estimation and the actual usage differ, as the relayer could be setting a higher gas allowance.
Payment in Kind
Gas fees from transaction execution are paid in ETH, since it is the native currency of the Ethereum network. However, nothing forces us to pay the rewards to relayers in the same currency, as there are plenty of other media of exchange on Ethereum: every fungible token (ERC20) is a potential currency.
This opens the door to allowing our users to transact exclusively using a token, since ETH is no longer needed to pay for gas fees. This is especially interesting if our application is built on top of a token-based protocol: it allows us to send tokens to our users as they engage in our network, which are in turn used to pay for relayer rewards. Our users never need to hold or purchase any ETH; they only work with our application’s token.
Keep in mind that while this mechanism is particularly useful when your application spins up relayers that trade tokens for executions, it imposes additional difficulties for decentralized relayers. A random relayer performing desktop mining now needs to check for the market value of the reward token against the execution cost to determine the profit. Not only that, but it also needs to ensure there is enough liquidity in the market to trade such token for ETH when they want to cash out their profits – and that is without considering the fees for that exchange.
In other words, paying relayer rewards in a protocol-specific ERC20 is useful if the relayer infrastructure you are using is specific to your application, but ETH or a widespread ERC2015 may be a better option for working with decentralized relayers.
Native Meta Transactions
Meta transactions, as we have just seen, require the use of an identity contract to act as a bouncer proxy to the actual contracts the user is interacting with. The identity contract holds the logic to process the signed transactions and then calls into a third-party contract, which does not require to be aware of meta transactions.
However, it is possible to remove the need for identity contracts if the application contracts already have support for processing meta transactions. This approach is named native meta transactions,16 since it is built-in natively on the application contracts, instead of requiring a proxy identity contract.
Adding native meta transactions to an ERC721 contract. Note that the nonces now need to be tracked per signer
By bundling this logic in the reward token contract itself directly, our users can directly sign token transfers that are then sent by a relayer and executed by the ERC721 contract. This allows a user with an existing account to benefit from meta transactions (which could be subsidized by our application) without having to deploy an identity contract.
Rewards in Native Meta Transactions
While moving the meta transaction logic to the application contract directly reduces complexity for the user, it introduces an additional difficulty: how to handle rewards to relayers. Identity contracts did not face this problem since they already held all of the user’s assets, so they could directly transfer them to relayers as payment.
Sample signedTransfer function in an ERC20 contract that supports meta transactions and relayer rewards using the same ERC20 token
However, this solution does not allow any rewards in any currency that is not that same token – for instance, ETH rewards are not possible under this scheme. It also does not translate well to other assets. What would be the reward for transferring an ERC721 non-fungible token? A digital collectible cannot have a piece removed and given to a relayer as a reward.
A way around this problem is to rely on ERC20 approvals. Recall from Chapter 3 that ERC20 tokens allow holders to appoint other addresses to manage their funds. This way, a user could grant an approval on their tokens to the application contract processing the relayer rewards, which then sends the tokens to the relayer as payment. This approach, however, still requires the user to have ETH to pay gas fees in the initial approve transaction – unless the reward token contract itself has native meta transaction support for approvals.
Another option is to simply subsidize the user’s transactions and have the application itself pay out to the relayers. This requires careful logic in the application contract to determine when to accept a meta transaction – otherwise, a malicious relayer could spam the application with fake transactions and drain the entire reward pool. Such logic will depend entirely on your use case, but keep in mind that you have much flexibility: your contracts may require the meta transaction to have an additional signature by an application key, so you can perform validations off-chain to approve a transaction for execution.
Revisiting Smart Accounts Deployment
We will go once more through the process of deploying a smart account contract. In the “Smart accounts” section, we discussed how to do this using single-use addresses, though it had certain limitations. We will now explore another approach, with support for relayer rewards, based on a different EVM operation: CREATE2.
The CREATE2 Instruction
Ethereum has, since its first versions, provided a CREATE instruction for creating a new contract from another. The address of the newly created contract, as we saw earlier, is a function of the sender address and its nonce. While this allows for deterministic deployments, it also means that reserving an address is tricky, since the sender must not send any other transactions besides the deployment one to prevent changing its nonce.
A user can now choose a contract, along with its constructor arguments, generate a random salt, and know the address where the resulting contract will be deployed. Not only that, but anyone who knows these parameters can now perform the deployment.
This means that the user can simply share the creation parameters and salt, plus the address of the factory contract to be used, and have any relayer execute the transaction, without even needing to sign the transaction – and certainly not paying any gas. Should a relayer attempt to modify the code or creation parameters, the contract would end up deployed at a different address. Let’s use this approach to deploy our identity contracts, providing rewards to the relayers.
Deployment Rewards
Before going into the code, we need to define how the payments to the relayer will be managed and which addresses will be initially funded by the user.
Recall from the previous approach that one of the downsides of single-use addresses is that the user needs to fund from an exchange one address, but then his or her identity is spawned at a different one. Even worse, any additional funds sent to the single-use address are lost. Then, it is desirable if we can have the user fund the address of the identity contract directly.
Modified Identity contract that pays out a reward to the relayer of the transaction
This approach allows the user to simply specify the owner account and the reward to be paid and broadcast those parameters along with the salt chosen, since these values alone are enough to determine the deployment address. Any relayer can then pick up this transaction, validate that there are enough funds on the deployment address to pay for the reward, and send the transaction to the factory contract. As a bonus, any surplus funds sent to the deployment address will still be there after the contract is created, ready to be used by their owner.
Identity Contract Factory
Deploy function of an IdentityFactory contract. The creationCode property of the contract type returns the bytecode used in the creation, and any constructor arguments just need to be appended at the end
Note that there is no need to validate the owner’s signature, since any change on the creation parameters would yield a different deployment address – one with no funds to pay back the reward to the relayer. Relayers should validate that the deployment address indeed has enough funds to pay back.
Using this strategy, you can precalculate the address where a user’s identity will be created given a salt and share the address with him or her. The user acknowledges that address as their own and seeds it with funds from an exchange. This in turn triggers a relayer to create an identity contract at that address once it is funded.
Note
This strategy can actually be carried out with single-use addresses instead of CREATE2 using a slightly more complex flow. The application can select a relayer and create a single-use address that will spawn a new identity contract and send a reward to the pre-selected relayer. After the user funds the address where the identity is to be deployed, the relayer in turn funds the single-use addresses, executes the deployment, and receives the reward.
Ethereum Names
The last onboarding challenge we will tackle in this chapter is that of Ethereum addresses themselves. While addresses are central to any Ethereum application, as they identify the actors of a decentralized system, they are far from user-friendly. Asking a user to understand their 40-character string of apparent gibberish as their global identifier is not good design. To solve this issue, we will look into ENS (Ethereum Name Service).
A DNS for Ethereum
Most web developers are familiar with the concept of DNS (Domain Name Service) , a protocol for mapping easily recognizable domain names (like “google.com”) to machine-friendly addresses (like 172.217.28.206) that identify a server in the Internet Protocol.18
The Ethereum Name Service19 (or ENS) is an analogous protocol that resolves user-friendly names (like “ethereumfoundation.eth”) to Ethereum addresses (0xfB6916095ca1df60bB79Ce92cE3Ea74c37c5d359 in this example, which is the Ethereum Foundation tip jar). Also like DNS, it supports registering non-address records (like content hashes or plain text), as well as reverse lookups.
The components involved in ENS loosely mimic those of DNS, with the difference that they are implemented as smart contracts within the Ethereum network. The ENS registry itself is a singleton contract that keeps track of all domain names and their owners and maps each domain to a resolver. A resolver contract provides methods for resolving a name to an address and optionally additional information such as text, content hashes, or public key records. Finally, the ENS registry is updated via registrars , contracts that manage the registration of subdomains at different levels of the tree, each with its own policy.
The most widely used top-level domain for ENS is .eth,20 operated by the so-called .eth permanent registrar . This registrar uses a commit-reveal scheme for purchasing second-level domains and allows anyone to chip in for extending a name registration. While it is possible to interact with this contract directly, it is suggested to use a tool (such as mycrypto21 or myetherwallet22) to purchase and manage your second-level domains.
Names Resolution
A key part of making your application ENS-aware is to allow your users to enter ENS names wherever an address is required. Allowing users to input ENS names in your application instead of addresses helps abstracting the complexity of Ethereum addresses, which means one less concept your users need to grasp to start using your app. This is the equivalent of allowing your users to navigate to “google.com” by entering the domain name in their browser instead of forcing them to type in its IP address.
Resolving a domain name to an address using the central ENS registry on mainnet
Abstracting your users of Ethereum addresses also means that your application must use Ethereum names instead of raw addresses when displaying information. It is no good if your users enter a user-friendly Ethereum name, and your application answers with a plethora of 40-character hexadecimal strings.
Caution
ENS does not enforce the correctness of reverse records; this means that anyone could register that their address maps to “ethereumfoundation.eth”. To protect against this, you should always run a forward name resolution on the result of a reverse resolution and verify that it matches the original address.
Giving Names to Our Users
ENS is a perfect match for smart accounts. Instead of requiring our users to remember the address of the smart contract that is their on-chain identity, we can allow them to assign it an Ethereum name.
As in DNS, instead of having them purchase a second-level domain directly from a network information center (NIC) , we can allocate names within our own domain. Email is a good analogy of this and something users are accustomed to – most people have an email account with a provider, like [email protected], instead of one managed by them, like [email protected]. Similarly, in ENS, we can allocate a subdomain for each user of our application, like john.myapp.eth.
To implement this, we need to set up our own registrar contract to manage our Ethereum domain and manage all subdomain registrations. We will use a simple FIFS (first-in first-served) registrar contract that will freely accept all subdomain registrations. Canonical implementations for this and other contracts we will be using can be found in the @ensdomains/[email protected] and @ensdomains/[email protected] packages.
Registering the Ethereum domain “myapp.test” on Rinkeby. The hash function in this code snippet is keccak256
- 1.
Registering a custom name (i.e., “john”) in our registrar and appointing the Identity contract as the owner
- 2.
Setting a resolver in the ENS registry for the new identity name (“john.myapp.test”)
- 3.
Setting the identity contract address in the resolver from the previous step
We will be using a public resolver for step 2 of the process. A public resolver is a public contract that accepts requests for managing the records of any address, but only from the owner of that address. This saves us the trouble of having to deploy a custom resolver for our app.
Note
An alternative is to add resolver methods to our identity contract and just let the identity return its own address upon a resolution request. However, this adds more complexity to our contract.
Identity contract function for registering a name and mapping it to the identity itself using our custom registrar and a public resolver. Code adapted from the UniversalLoginSDK repository24
Registering an identity contract as john.myapp.test using our custom registrar and a Rinkeby public resolver
Running this process for every new user allows them to refer to their identity using a friendly name provided by our application – a name that can be carried on to other applications and be used as a global Ethereum identity, handled by ENS.
Summary
We have gone through several tools and techniques for handling user onboarding and account management in general, making this one of the most content-heavy chapters in this book: fallback functions, forwarding contracts, single-use addresses, local accounts, mnemonics, smart accounts, upgradeability, meta transactions, native meta transactions, reserved deployment addresses, and Ethereum names, among others. All of these techniques help in different aspects of user onboarding, and their trade-offs make it difficult to settle for one solution that fits all use cases.
Work on user onboarding is still a very active field of research in Ethereum, and new mechanics to add to your toolbelt are bound to be developed in the near future. At the time of this writing, it is worth highlighting the work being done under Universal Logins.25 Universal Logins is a framework that creates smart accounts for users, managed by local accounts automatically generated on each device, with support for meta transactions, as well as ENS for allowing users to easily connect to their accounts. It also promotes subsidizing early user actions to ease onboarding and reward users in-app who go the extra mile to strengthen the security of their accounts.
Whatever solution you implement, remember that the more steps a user must go through to start using your application, the most likely it is for them to drop. On the other hand, sacrificing security for usability is a huge risk, given that “the worst user experience is when people lose their crypto.”26 Striking the perfect compromise between the two for the use case you are building is no easy feat.