© Karan Singh Garewal 2020
K. S. GarewalPractical Blockchains and Cryptocurrencieshttps://doi.org/10.1007/978-1-4842-5893-4_14

14. Helium Mining

Karan Singh Garewal1 
(1)
Toronto, ON, Canada
 
In the previous chapter, we have examined the cryptocurrency mining process in detail. Mining is a critical aspect of distributed cryptocurrencies that are modeled after Bitcoin. As you will recall, mining produces three results:
  1. 1.

    Attaches new blocks to the blockchain

     
  2. 2.

    Achieves distributed consensus on the content of the blockchain and hence distributed consensus on the transactions in the network

     
  3. 3.

    Creates fresh units of cryptocurrency that are distributed to miners

     

In this chapter, we will implement the theoretical concepts pertaining to distributed consensus in the form of mining nodes that implement consensus by solving a difficult mathematical problem. Solving this problem is called proof of work, and it lets a mining node attach a block to the blockchain and hence collect a substantial mining reward as well as transaction fees. Mining is a computationally intensive task and hence consumes a substantial quantum of electricity. Nodes compete with each other to obtain the financial benefits of mining. The only way to maximize profit is to mine as many blocks as possible and attach them to the longest blockchain. Longer blockchains have more computational effort extended upon them. Miners will not attempt to attach newly mined blocks to a shorter secondary blockchain as this blockchain will eventually be deleted by other miners.

In this chapter, we will write the Python program code for a Helium mining node. As usual, we will buttress this code with pytest unit tests. So without further ado, let us commence this task.

Distributed Consensus Algorithm in Detail

In Chapter 13, we described how blocks are processed in the Helium network and how the network arrives at a distributed consensus as to the blockchain. We are now going to provide the precise algorithm that a mining node uses to process transactions and blocks and achieve distributed consensus:

Handling Incoming Transactions

A transaction arrives at a miners interface. There are two cases:

Case A:

If the transaction is valid, add it to the mempool. In particular, since the transaction is valid, it does not spend transaction fragments in the Chainstate database which have already been spent.

 

Adding Transactions to a Candidate Block

 

As transactions are added to a candidate block, we maintain a temporary list of prior transaction fragments that are going to be spent by these transactions. If a transaction selected for the candidate block tries to spend a fragment that is in this list, it is rejected for inclusion in the block and also deleted from the mempool.

Case B:

The transaction is invalid. For example, the miner checks the Chainstate database and sees that this transaction references one or more previous transaction fragments that have already been spent. The transaction is rejected.

Handling Incoming Blocks

A block arrives at a mining node interface:

Get the block height of the incoming block. There are then five possible cases:

Case A:

The block height is less than the height of the blockchain.

Reject the block; it is stale. A block at this height has already been mined.

Case B:

The block height is greater than the height of the blockchain + 1.

Put the block in the orphan_blocks list. We will process this block later once our blockchain height increases.

Case C:

The block height is equal to the height of the blockchain + 1.

We can add the block to the primary blockchain or the secondary blockchain (if it exists) depending upon the value of the prevblockhash attribute (see the following note).

Case D:

The block height is equal to the current block height.

If the prevblock hash of the new block matches the block header hash of the parent of the block at the head of the primary blockchain, then fork the blockchain. Otherwise, reject the block (see the following note).

Case E:

A block arrives which is greater than the blockchain height + 1.

Trigger a call to update our blockchain if a number of such blocks have been received by the miner. The blockchain is possibly behind the blockchain of other miners.

Note:

Suppose that the head of our blockchain has a transaction that consumes a transaction fragment and the incoming block also has a transaction that consumes the same transaction fragment, then unless the miner that mined the incoming block is acting mala fides, the new block cannot have a prevblockhash value that points to the head of our primary blockchain.

The new block must either sit on top of a secondary blockchain (if it exists) or have a prevblockhash that matches the block that is the parent of the head block at the head of our blockchain. In the latter case, we will have a fork of the primary blockchain. But in either case, two double-spending transactions cannot exist on one blockchain. One will exist on the primary blockchain, and the second will exist on the secondary blockchain. This solves the double-spend problem.

Suppose further that we have a primary and secondary blockchain and a block arrives that has a prevblockhash value that matches the parent of the head on one of these blockchains. This block will be rejected, as it is in effect matching the block hash of the grandparent of the block at the head of the blockchain before it was forked (exercise: think this through with some diagrams).

Helium Mining Code Walkthrough

Create a file called hmining.py in the mining sub-directory and copy the following Python code into this file. Our walkthrough of the data structures and functions in the hmining module follows the program code.
"""
   hmining.py: The code in this module implements a Helium mining node.
   The node constructs candidate blocks with transactions in them and then mines these
   blocks
"""
import blk_index
import hconfig
import hblockchain as bchain
import hchaindb
import networknode
import rcrypt
import tx
import asyncio
import json
import math
import sys
import time
import threading
import pdb
import logging
"""
   log debugging messages to the file debug.log
"""
logging.basicConfig(filename="debug.log",filemode="w",  
    format='%(asctime)s:%(levelname)s:%(message)s',
    level=logging.DEBUG)
"""
address list of nodes on the Helium network
"""
address_list = []
"""
   specify a list container to hold transactions which are received by the miner.
"""
mempool       = []
"""
  blocks mined by other miners which are received by this mining node
  and are to intended to be appended to this miner's blockchain.
"""
received_blocks = []
"""
   blocks which are received and cannot be added to the head of the blockchain
   or the parent of the block at the head of the blockchain.
"""
orphan_blocks = []
"""
list of transactions in a block that have been mined and are to be removed from the
memcache
"""
remove_list = []
semaphore = threading.Semaphore()
def mining_reward(block_height:"integer") -> "non-negative integer":
    """
    The mining_reward function determines the mining reward.
    When a miner successfully mines a block he is entitled to a reward which
    is a number of Helium coins.
    The reward depends on the number of blocks that have been mined so far.
    The initial reward  is hconfig.conf["MINING_REWARD"] coins.
    This reward halves every hconfig.conf["REWARD_INTERVAL"] blocks.
    """
    try:
        # there is no reward for the genesis block
        sz = block_height
        sz = sz // hconfig.conf["REWARD_INTERVAL"]
        if sz == 0:  return hconfig.conf["MINING_REWARD"]
        # determine the mining reward
        reward = hconfig.conf["MINING_REWARD"]
        for __ctr in range(0,sz):
            reward = reward/2
        # truncate to an integer
        reward = int(round(reward))
        # the mining reward cannot be less than the lowest denominated
        # currency unit
        if reward < hconfig.conf["HELIUM_CENT"]: return 0
    except Exception as err:
        print(str(err))
        logging.debug('mining reward: exception: ' + str(err))
        return -1
    return reward
def receive_transaction(transaction: "dictionary") -> "bool":
    """
    receives a transaction propagating over the P2P cryptocurrency network
    this function executes in a thread
    """
    # do not add the transaction if:
    #     it already exists in the mempool
    #     it exists in the Chainstate
    #     it is invalid
    try:
        # if the transaction is in the mempool, return
        for trans in mempool:
            if trans == transaction: return False
        # do not add the transaction if it has been accounted for in
        # the chainstate database.
        tx_fragment_id = transaction["transactionid"] + "_" + "0"
        if hchaindb.get_transaction(tx_fragment_id) != False:
            return True
        # verify that the incoming transaction is valid
        if len(transaction["vin"]) >  0: zero_inputs = False
        else: zero_inputs = True
        if tx.validate_transaction(transaction, zero_inputs) == False:
            raise(ValueError("invalid transaction received"))
        # add the transaction to the mempool
        mempool.append(transaction)
        # place the transaction on the P2P network for further
        # propagation
        propagate_transaction(transaction)
    except Exception as err:
        logging.debug('receive_transaction: exception: ' + str(err))
        return False
    return True
def make_candidate_block() -> "dictionary || bool":
    """
    makes a candidate block for inclusion in the Helium blockchain.
    A candidate block is created by:
            (i)  fetching transactions from the  mempool and adding them to
                 the candidate blocks transaction list.
            (ii) specifying the block header.
    returns the candidate block or returns False if there is an error or if
    the mempool is empty.
    Executes in a Python thread
    """
    try:
        # if the mempool is empty then no transactions can be put into
        # the candidate block
        if len(mempool) == 0: return False
        # make a public-private key pair that the miner will use to receive
        # the mining reward as well as the transaction fees.
        key_pair = make_miner_keys()
        block = {}
        # create a incomplete candidate block header
        block['version']   = hconfig.conf["VERSION_NO"]
        block['timestamp'] = int(time.time())
        block['difficulty_bits']    = hconfig.conf["DIFFICULTY_BITS"]
        block['nonce']     = hconfig.conf["NONCE"]
        if len(bchain.blockchain) > 0:
             block['height'] = bchain.blockchain[-1]["height"] + 1
        else:
             block['height'] = 0
        block['merkle_root'] = ""
        # get the value of the hash of the previous block's header
        # this induces tamperproofness for the blockchain
        if len(bchain.blockchain) > 0:
             block['prevblockhash'] = bchain.blockheader_hash(bchain.blockchain[-1])
        else:
             block['prevblockhash'] = ""
        # calculate the  size (in bytes) of the candidate block header
        # The number 64 is  the byte size of the SHA-256 hexadecimal
        # merkle root. The merkle root is computed after all the
        # transactions are included in the candidate block
        # reserve 1000 bytes for the coinbase transaction
        block_size =  sys.getsizeof(block['version'])
        block_size += sys.getsizeof(block['timestamp'])
        block_size += sys.getsizeof(block['difficulty_bits'])
        block_size += sys.getsizeof(block['nonce'])
        block_size += sys.getsizeof(block['height'])
        block_size += sys.getsizeof(block['prevblockhash'])
        block_size += 64
        block_size += sys.getsizeof(block['timestamp'])
        block_size += 1000
        # list of transactions in the block
        block['tx'] = []
        # get the Unix Time now
        now = int(time.time())
        # add transactions from the mempool to the candidate block until
        # the transactions in the mempool are exhausted or the block
        # attains its maximum permissible size
        for memtx in mempool:
                # do not process future transactions
                if memtx['locktime'] > now: continue
                memtx = add_transaction_fee(memtx, key_pair[1])
                # add the transaction to the candidate block
                block_size += sys.getsizeof(memtx)
                if block_size <= hconfig.conf['MAX_BLOCK_SIZE']:
                     block['tx'].append(memtx)
                     remove_list.append(memtx)
                else:
                     break
        # return if there are no transactions in the block
        if len(block["tx"]) == 0: return False
        # add a coinbase transaction
        coinbase_tx = make_coinbase_transaction(block['height'], key_pair[1])
        block['tx'].insert(0, coinbase_tx)
        # update the length of the block
        block_size += sys.getsizeof(block['tx'])
        # calculate the merkle root of this block
        ret = bchain.merkle_root(block['tx'], True)
        if ret == False:
            logging.debug('mining::make_candidate_block - merkle root error')
            return False
        block['merkle_root'] = ret
        ###################################
        # validate the candidate block
        ###################################
        if bchain.validate_block(block) == False:
            logging.debug('mining::make_candidate_block - invalid block header')
            return False
        ###############################################
        # remove transactions from the mempool
        ###############################################
        remove_mempool_transactions(remove_list)
    except Exception as err:
        logging.debug('make_candidate_block: exception: ' + str(err))
    # At this stage the candidate block has been created and it can be mined
    return block
def make_miner_keys():
    """
    makes a public-private key pair that the miner will use to receive
    the mining reward and the transaction fee for each transaction.
    This function writes the  keys to a file and returns hash:
    RIPEMD160(SHA256(public key))
    """
    try:
        keys    = rcrypt.make_ecc_keys()
        privkey = keys[0]
        pubkey  = keys[1]
        pkhash  = rcrypt.make_SHA256_hash(pubkey)
        mdhash  = rcrypt.make_RIPEMD160_hash(pkhash)
        # write the keys to file with the private key as a hexadecimal string
        f = open('coinbase_keys.txt', 'a')
        f.write(privkey)
        f.write(' ')       # newline
        f.write(pubkey)
        f.write(' ')
        f.close()
    except Exception as err:
        logging.debug('make_miner_keys: exception: ' + str(err))
    return mdhash
def add_transaction_fee(trx: 'dictionary', pubkey: 'string') -> 'dictionary':
    """
    directs the transaction fee of a transaction to the miner.
    Receives a transaction and a miner's public key.
    Amends and returns the transaction so that it consumes the transaction fee.
    """
    try:
        # get the previous transaction fragments
        prev_fragments = []
        for vin in trx["vin"]:
            fragment_id = vin["txid"] + "_" + str(vin["vout_index"])
            prev_fragments.append(hchaindb.get_transaction(fragment_id))
        #  Calculate the transaction fee
        fee = tx.transaction_fee(trx, prev_fragments)
        if fee > 0:
            vout = {}
            vout["value"] = fee
            ScriptPubKey = []
            ScriptPubKey.append('SIG')
            ScriptPubKey.append(rcrypt.make_RIPEMD160_hash(rcrypt.make_SHA256_hash(pubkey)))
            ScriptPubKey.append('<DUP>')
            ScriptPubKey.append('<HASH_160>')
            ScriptPubKey.append('HASH-160')
            ScriptPubKey.append('<EQ-VERIFY>')
            ScriptPubKey.append('<CHECK_SIG>')
            vout["ScriptPubKey"] = ScriptPubKey
            trx["vout"].append(vout)
    except Exception as err:
        logging.debug('add_transaction_fee: exception: ' + str(err))
        return False
    return trx
def make_coinbase_transaction(block_height: "integer", pubkey: "string") -> 'dict':
    """
    makes a coinbase transaction, this is the miner's reward for mining
    a block. Receives a public key to denote ownership of the reward.
    Since this is a fresh issuance of heliums there are no vin elements.
    locks the transaction for hconfig["COINBASE_INTERVAL"] blocks.
    Returns the coinbase transaction.
    """
    try:
        # calculate the mining reward
        reward = mining_reward(block_height)
        # create a coinbase transaction
        trx = {}
        trx['transactionid'] = rcrypt.make_uuid()
        trx['version']  = hconfig.conf["VERSION_NO"]
        # the mining reward cannot be claimed until approximately 100 blocks are mined
        # convert into a time interval
        trx['locktime'] = hconfig.conf["COINBASE_INTERVAL"]*600
        trx['vin']  = []
        trx['vout'] = []
        ScriptPubKey = []
        ScriptPubKey.append('SIG')
        ScriptPubKey.append(rcrypt.make_RIPEMD160_hash(rcrypt.make_SHA256_hash(pubkey)))
        ScriptPubKey.append('<DUP>')
        ScriptPubKey.append('<HASH_160>')
        ScriptPubKey.append('HASH-160')
        ScriptPubKey.append('<EQ-VERIFY>')
        ScriptPubKey.append('<CHECK_SIG>')
        # create the vout element with the reward
        trx['vout'].append({
                            'value':  reward,
                            'ScriptPubKey': ScriptPubKey
                        })
    except Exception as err:
        logging.debug('make_coinbase_transaction: exception: ' + str(err))
    return trx
def remove_mempool_transactions(block: 'dictionary') -> "bool":
    """
    removes the transactions in the candidate block from the mempool
    """
    try:
        for transaction in block["tx"]:
            if transaction in mempool:
                mempool.remove(transaction)
    except Exception as err:
        logging.debug('remove_mempool_transactions: exception: ' + str(err))
        return False
    return True
def mine_block(candidate_block: 'dictionary') -> "bool":
    """
    Mines a candidate block.
    Returns the solution nonce as a hexadecimal string if the block is
    mined and False otherwise
    Executes in a Python thread
    """
    try:
        final_nonce = None
        save_block = dict(candidate_block)
        # Loop until block is mined
        while True:
            # compute the SHA-256 hash for the block header of the candidate block
            hash = bchain.blockheader_hash(candidate_block)
            # convert the SHA-256 hash string to a Python integer
            mined_value = int(hash, 16)
            mined_value = 1/mined_value
            # test to determine whether the block has been mined
            if mined_value < hconfig.conf["DIFFICULTY_NUMBER"]:
                final_nonce = candidate_block["nonce"]
                break
            # if a received block contains a transaction that is also in the
            # candidate block then terminate mining this block
            with semaphore:
                if len(received_blocks) > 0:
                    if compare_transaction_lists(candidate_block) == False:
                        return False
            # failed to mine the block so increment the
            # nonce and try again
            candidate_block['nonce'] += 1
        logging.debug('mining.py: block has been mined')
        # add block to the received_blocks list and process list
        received_blocks.append(save_block)
        ret = process_received_blocks()
        if ret == False:
            raise(ValueError("failed to add mined block to blockchain"))
    except Exception as err:
        logging.debug('mine_block: exception: ' + str(err))
        return False
    return hex(final_nonce)
def proof_of_work(block):
    """
    Proves whether a received block has in fact been mined.
    Returns True or False
    """
    try:
        if block['difficulty_bits'] != hconfig.conf["DIFFICULTY_BITS"]:
            raise(ValueError("wrong difficulty bits used"))
        # compute the SHA-256 hash for the block header of the candidate block
        hash = bchain.blockheader_hash(block)
        # convert the SHA-256 hash string to a Python integer to base 10
        mined_value = int(hash, 16)
        mined_value = 1/mined_value
        # test to determine whether the block has been mined
        if mined_value < hconfig.conf["DIFFICULTY_NUMBER"]: return True
    except Exception as err:
        logging.debug('proof_of_work: exception: ' + str(err))
        return False
    return False
def receive_block(block):
    """
    Maintains the received_blocks list.
    Receives a block and returns False if:
    (1)  the block is invalid
    (2)  the block height is less than (blockchain height - 2)
    (3)  block height > (blockchain height + 1)
    (4)  the proof of work fails
    (5)  the block does not contain at least two transactions
    (6)  the first block transaction is not a coinbase transaction
    (7)  transactions in the block are invalid
    (8)  block is already in the blockchain
    (9)  the block is already in the received_blocks list
    Otherwise (i)  adds the block to the received_blocks list
             (ii) propagates the block
    Executes in a python thread
    """
    try:
        # test if block is already in the received_blocks list
        with semaphore:
            for blk in received_blocks:
                if blk == block: return False
        # verify the proof of work
        if proof_of_work(block) == False:
            raise(ValueError("block proof of work failed"))
        # restore the Nonce to its initial value
        block["nonce"] = hconfig.conf["NONCE"]
        # validate the block and block transactions
        if bchain.validate_block(block) == False: return False
        for trx in block["tx"]:
            if block["height"] == 0 or block["tx"][0] == trx: flag = True
            else: flag = False
            if tx.validate_transaction(trx, flag) == False: return False
        if len(bchain.blockchain) > 0:
            # do not add stale blocks to the blockchain
            if block["height"] < bchain.blockchain[-1]["height"] - 2:
                raise(ValueError("block height too old"))
            # do not add blocks that are too distant into the future
            if block["height"] > bchain.blockchain[-1]["height"] + 1:
                raise(ValueError("block height beyond future"))
        # test if block is in the primary or secondary blockchains
        if len(bchain.blockchain) > 0:
            if block == bchain.blockchain[-1]: return False
        if len(bchain.blockchain) > 1:
          if block == bchain.blockchain[-2]: return False
        if len(bchain.secondary_blockchain) > 0:
            if block == bchain.secondary_blockchain[-1]: return False
        if len(bchain.secondary_blockchain) > 1:
            if block == bchain.secondary_blockchain[-2]: return False
        # add the block to the blocks_received list
        with semaphore:
            received_blocks.append(block)
        process_received_blocks()
    except Exception as err:
        logging.debug('receive_block: exception: ' + str(err))
        return False
    return True
def process_received_blocks() -> 'bool':
    """
    processes mined blocks that are in the in the received_blocks
    list and attempts to add these blocks to a blockchain.
    Algorithm:
     (1)  get a block from the received blocks list.
     (2)  if the blockchain is empty and the block has an empty prevblockhash
          field then add the block to the blockchain.
     (3)  Compute the block header hash of the last block on the blockchain.
          if blkhash == block["prevblockhash"] then add the block to the blockchain.
     (4)  if the blockchain has at least two blocks, then if:
             let blk_hash be the hash second-latest block of the blockchain,
             then if:
             block["prevblockhash"] == blk_hash
             create a secondary block consisting of all of the blocks of the blockchain
             except for the latest. Append the block to the secondary blockchain.
    (5)  Otherwise move the block to the orphans list.
    (6)  If the received block was attached to a blockchain, for each block in
        the orphans list, try  to attach the orphan block to the primary blockchain or
        the secondary blockchain if it exists.
    (7) If the received block was attached to a blockchain and the secondary blockchain
        has elements then swap the blockchain and the secondary blockchain if the
        length of the secondary blockchain is greater.
   (8)  if the receive block was attached and the secondary blockchain has elements
        then clear the secondary blockchain if its length is more than two blocks
        behind the primary blockchain.
    Note: Runs In A Python thread
    """
    while True:
        # process all of the blocks in the received blocks list
        add_flag = False
        # get a received block
        with semaphore:
            if len(received_blocks) > 0:
                block = received_blocks.pop()
            else:
                return True
            # try to add block to the primary blockchain
            if bchain.add_block(block) == True:
                logging.debug('receive_mined_block: block added to primary blockchain')
                add_flag = True
            # test whether the primary blockchain must be forked
            # if the previous block hash is equal to the hash of the parent block of
            # the block at the head of the blockchain add the block as a child of the
            # parent and create a secondary blockchain. This constitutes a fork of the
            # primary blockchain
            elif len(bchain.blockchain) >= 2 and  block['prevblockhash'] ==
                bchain.blockheader_hash(bchain.blockchain[-2]):
                logging.debug('receive_mined_block: forking the blockchain')
                ret = fork_blockchain(block)
                if ret == True: add_flag = True
            # try to add add block to the secondary blockchain
            elif len(bchain.secondary_blockchain) > 0 and block['prevblockhash'] ==
                bchain.blockheader_hash(bchain.secondary_blockchain[-1]):
                if append_secondary_blockchain(block) == True: add_flag = True
            # cannot attach the block to a blockchain add to the orphan list,
            else:
                if add_flag == False: orphan_blocks.append(block)
            if add_flag == True:
                if block["height"] % hconfig.conf["RETARGET_INTERVAL"] == 0:
                    retarget_difficulty_number(block)
                handle_orphans()
                swap_blockchains()
                propagate_mined_block(block)
            # remove any transactions in this block that are also
            # in the the mempool
            remove_mempool_transactions(block)
    return True
def fork_blockchain(block: 'list') -> 'bool':
    """
    forks the primary blockchain and creates a secondary blockchain from
    the primary blockchain and then adds the received block to the
    secondary block
    """
    try:
        bchain.secondary_blockchain = list(bchain.blockchain[0:-1])
        if append_secondary_blockchain(block) == False: return False
        # switch the primary and secondary blockchain if required
        swap_blockchains()
    except Exception as err:
        logging.debug('fork_blockchain: exception: ' + str(err))
        return False
    return True
def swap_blockchains() -> 'bool':
    """
    compares the length of the primary and secondary blockchains.
    The longest blockchain is designated as the primary blockchain and
    the other blockchain is designated as the secondary blockchain.
    if the primary blockchain is ahead of the secondary blockchain by at
    least two blocks then clear the secondary blockchain.
    """
    try:
        # if the secondary blockchain is longer than the primary
        # blockchain designate the secondary blockchain as the primary
        # blockchain and the primary blockchain as the secondary blockchain
        if len(bchain.secondary_blockchain) > len(bchain.blockchain):
            tmp = list(bchain.blockchain)
            bchain.blockchain = list(bchain.secondary_blockchain)
            bchain.secondary_blockchain = list(tmp)
        # if the primary blockchain is ahead of the secondary blockchain by
        # at least two blocks then clear the secondary blockchain
        if len(bchain.blockchain) - len(bchain.secondary_blockchain) > 2:
            bchain.secondary_blockchain.clear()
    except Exception as err:
        logging.debug('swap_blockchains: exception: ' + str(err))
        return False
    return True
def append_secondary_blockchain(block):
    """
    append a block to the the secondary blockchain
    """
    with semaphore:
        tmp = list(bchain.blockchain)
        bchain.blockchain = list(bchain.secondary_blockchain)
        bchain.secondary_blockchain = list(tmp)
        ret = bchain.add_block(block)
        if ret == True: swap_blockchains()
    return ret
def handle_orphans() -> "bool":
    """
    tries to attach an orphan block to the head of the primary or secondary blockchain.
    Sometimes blocks are received out of order and cannot be attached to the primary
    or secondary blockchains. These blocks are placed in an orphan_blocks list and as
    new blocks are added to the primary or secondary blockchains, an attempt is made to
    add orphaned  blocks to the blockchain(s).
    """
    try:
        # iterate through the orphan blocks attempting to append an orphan
        # block to a blockchain
        for block in orphan_blocks:
            if bchain.add_block(block) == True:
                orphan_blocks.remove(block)
                continue
            # try to append to the primary blockchain
            if len(bchain.blockchain) > 1 and len(bchain.secondary_blockchain) == 0:
                if block['prevblockhash'] ==
                  bchain.blockheader_hash(bchain.blockchain[-1]):
                    if block["height"] == bchain.blockchain[-1]["height"] + 1:
                        if fork_blockchain(block) == True:
                            orphan_blocks.remove(block)
                            continue
            # try to append to the secondary blockchain
            if len(bchain.secondary_blockchain) > 0:
                if block['prevblockhash']  ==
                    bchain.blockheader_hash(bchain.secondary_blockchain[-1]):
                    if block["height"] == bchain.secondary_blockchain[-1]["height"] + 1:
                        if append_secondary_blockchain(block) == True:
                            orphan_blocks.remove(block)
                            continue
            orphan_blocks.remove(block)
    except Exception as err:
        logging.debug('handle_orphans: exception: ' + str(err))
        return False
    return True
def remove_received_block(block: 'dictionary') -> "bool":
    """
    remove a block from the received blocks list
    """
    try:
        with semaphore:
            if block in received_blocks:
                received_blocks.remove(block)
    except Exception as err:
        logging.debug('remove_received_block: exception: ' + str(err))
        return False
    return True
def retarget_difficulty_number(block):
    """
    recalibrates the difficulty number every hconfig.conf["RETARGET_INTERVAL"]
    blocks
    """
    if block["height"] == 0: return
    old_diff_no   = hconfig.conf["DIFFICULTY_NUMBER"]
    initial_block = bchain.blockchain[(block["height"] -
         hconfig.conf["RETARGET_INTERVAL"])]
    time_initial  = initial_block["timestamp"]
    time_now      = block["timestamp"]
    elapsed_seconds = time_now - time_initial
    # the average time to mine a block is 600 seconds
    blocks_expected_to_be_mined = int(elapsed_seconds / 600)
    discrepancy = 1000 - blocks_expected_to_be_mined
    hconfig.conf["DIFFICULTY_NUMBER"] =  old_diff_no -
        old_diff_no * (20/100)*(discrepancy /(1000 + blocks_expected_to_be_mined))
    return
def propagate_transaction(txn: "dictionary"):
    """
    propagates a transaction that is received
    """
    if len(address_list) % 20 == 0: get_address_list()
    cmd = {}
    cmd["jsonrpc"] = "2.0"
    cmd["method"]  = "receive_transaction"
    cmd["params"]  = {"trx":txn}
    cmd["id"] = 0
    for addr in address_list:
        rpc = json.dumps(cmd)
        networknode.hclient(addr,rpc)
    return
def propagate_mined_block(block):
    """
    sends a block to other mining nodes so that they may add this block
    to their blockchain.
    We refresh the list of known node addresses periodically
    """
    if len(address_list) % 20 == 0: get_address_list()
    cmd = {}
    cmd["jsonrpc"] = "2.0"
    cmd["method"]  = "receive_block"
    cmd["params"]  = {"block":block}
    cmd["id"] = 0
    for addr in address_list:
        rpc = json.dumps(cmd)
        networknode.hclient(addr,rpc)
    return
def get_address_list():
    '''
    update the list of node addresses that are known
    AMEND IP ADDRESS AND PORT AS REQUIRED.
    '''
    ret = networknode.hclient("http://127.0.0.69:8081",
           '{"jsonrpc":"2.0","method":"get_address_list","params":{},"id":1}')
    if ret.find("error") != -1: return
    retd = json.loads(ret)
    for addr in retd["result"]:
        if address_list.count(addr) == 0:
            address_list.append(addr)
    return
def compare_transaction_lists(block):
    """
    tests whether a transaction in a received block is also in a candidate
    block that is being mined
    """
    for tx1 in block["tx"]:
        for tx2 in remove_list:
            if tx1 == tx2: return False
    return True
def start_mining():
    """
    assemble a candidate block and then mine this block
    """
    while True:
        candidate_block = make_candidate_block()
        if candidate_block() == False:
            time.sleep(1)
            continue
        # remove transactions in the mined block from the mempool
        if mine_block(candidate_block) != False:
            remove_mempool_transactions(remove_list)
        else: time.sleep(1)

Helium Mining Data Structures

mempool

mempool is a list containing transactions that have been received but are not in any mined block.

received_blocks

This is a list of blocks mined by other mining nodes and meant to be included in the receiving node’s blockchain.

orphan_blocks

This is a list of valid blocks which have been received from other nodes but which could not be appended to the receiving miner’s blockchain.

Blockchain

This is the local blockchain that is being maintained by this mining node. It is located in the hblockchain module. This blockchain is also referred to as the primary blockchain.

Secondary Blockchain

This is an alternate blockchain that is maintained by the miner when the primary blockchain is forked.

Helium Mining Functions

mining_reward

A miner who successfully mines a block is entitled to a reward which consists of the issuance of new helium coins. The mining reward determines the quantum of fresh currency awarded to the miner. This reward halves periodically. Both the initial reward and the halving period are set out in the hconfig module.

The initial reward is hconfig.conf["MINING_REWARD"] coins. This reward halves every hconfig.conf["REWARD_INTERVAL"] blocks.

receive_transaction

The function receive_transaction receives a transaction propagating on the Helium P2P network. This function checks to determine whether the transaction is in the mempool. The function discards the transaction if it is already in the mempool.

In the next step, receive_transaction verifies the validity of the transaction. If the transaction is valid, it is added to mempool; otherwise, the transaction is discarded. If the transaction is added to the mempool, it is sent for further propagation on the Helium network.

receive_transaction is meant to run in a Python thread.1

make_candidate_block

The purpose of make_candidate_block is to create a block that can be mined. If the mempool is empty, the function just returns.

make_candidate_block creates a partial header for a block and then draws transactions from the mempool and appends them to tx field of this block. Any number of algorithms can be used to draw transactions from the mempool. make_candidate_block uses a simple FIFO (First-In, First-Out) algorithm and keeps on drawing transactions until the maximum block size of 1 MB is almost reached or the mempool is exhausted. For each transaction, this function inserts a vout transaction fragment giving itself the transaction fee.

After make_candidate_block has finished its draw from mempool, it creates a coinbase transaction. The coinbase transaction is the first transaction in the tx list. For each block, this function creates a fresh public-private key pair that is used to create the coinbase transaction and the transaction fee fragment.

Finally, make_candidate_block computes the merkle root of the transactions and inserts it into the merkle field of the block header. It then tests the validity of the constructed block. If the block is valid, it removes the transactions in the block from the mempool. The candidate block is now ready to be mined.

make_miner_keys

This function creates a public-private key pair and saves these keys to a text file. make_candidate_block uses the public key to create the RIPEMD160(SHA256(public key)) hash that is inserted into the ScriptPubKey portion of the coinbase transaction and each transaction fee vout fragment.

add_transaction_fee

add_transaction_fee is called from make_candidate_block. It receives a transaction and the miner’s public key. This function calculates the transaction fee, updates the Chainstate, and creates a transaction fragment that directs the transaction fee to the holder of the private key corresponding to the public key.

make_coinbase_transaction

make_coinbase_transaction is called from make_candidate_block. It receives the height of the candidate block and the miner’s public key. This function computes the block reward and then creates and returns a coinbase transaction.

remove_mempool_transactions

This function removes transactions from the mempool and hence prevents transactions from being included more than once in different candidate blocks.

mine_block

mine_block receives a candidate block and initiates mining this block. The SHA256 hash of the nonce in the block header is computed. The block is said to be mined if the computed SHA-256 hash is less than the difficulty number in the Helium hconfig module. If the value is greater than or equal to the difficulty number, then the nonce is incremented and the computational result is compared once again to the difficulty number. This process continues until the block is mined or the computational loop is exited.

Once the block is mined, it is attached to the miner’s blockchain and then propagated on the Helium P2P network so that other miners can append it to their blockchains.

proof_of_work

Before a block received from another miner can be included in a local blockchain, the node must verify the integrity of the block and its transactions and also verify that the nonce supplied in the block header is less than the difficulty number in this node’s hconfig file. This function implements the proof of work for the received block.

receive_block

receive_block receives a block mined by some other miner. If the block is valid, the mining node adds the block to its received_blocks list. The block is invalid if any of the following conditions occur:
  1. 1.

    The block is invalid.

     
  2. 2.

    The block height is less than (blockchain height – 2).

     
  3. 3.

    The proof of work fails.

     
  4. 4.

    The block does not contain at least two transactions.

     
  5. 5.

    The first block transaction is not a coinbase transaction.

     
  6. 6.

    Transactions in the block are invalid.

     
  7. 7.

    Block is already in the blockchain.

     
  8. 8.

    The block is already in the received_blocks list.

     

If the received block is valid, it is added to the received_blocks list and the block is propagated on the Helium P2P network so that other nodes can attach it to their blockchains.

receive_block is meant to be executed as a Python thread.

process_received_blocks

process_received_blocks draws blocks from the received_blocks list and attempts to attach them to the blockchain. This function implements distributed consensus. Note that since miners are impelled to make a profit, they will always extend their longest blockchain. The algorithm for processing the received blocks is as follows:
  1. 1.

    Get a block from the received blocks list.

     
  2. 2.

    If the blockchain is empty and the block has an empty prevblockhash field, then add the block to the blockchain (this is a genesis block).

     
  3. 3.

    Compute the block header hash of the last block on the blockchain (blkhash)

     
if blkhash == block["prevblockhash"]
and then add the block to the blockchain.
  1. 4.

    If the blockchain has at least two blocks, let blk_hash be the hash of the block header of the second latest block in the blockchain, and then if

     
block["prevblockhash"] == blk_hash
create a secondary block consisting of all of the blocks of the blockchain except for the latest. Append the block to this secondary blockchain.
  1. 5.

    Otherwise, move the block to the orphan_blocks list.

     
  2. 6.

    If the received block was attached to a blockchain, then for each block in the orphan_blocks list, try to attach the orphan block to the primary blockchain or the secondary blockchain, if it exists.

     
  3. 7.

    If the received block was attached to a blockchain and the secondary blockchain has elements, then swap the primary blockchain and the secondary blockchain if the length of the secondary blockchain is greater.

     
  4. 8.

    If the received block was attached and the secondary blockchain has elements, then clear the secondary blockchain if its length is more than two blocks behind the primary blockchain.

     

process_receive_block is meant to run in a concurrent thread.

fork_blockchain

This function creates the secondary blockchain by forking the primary blockchain. fork_blockchain receives a block parameter which it attaches to the secondary blockchain.

swap_blockchains

swap_blockchains swaps the primary and secondary blockchains so that the primary blockchain is the longer blockchain.

handle_orphans

handle_orphans draws orphaned blocks from the orphans list and attempts to attach them to the primary blockchain. Orphaned blocks that are more than two blocks distant from the head of the blockchain are deleted.

remove_received_block

This function removes a block from the received_blocks list.

retarget_difficulty_number

This function is invoked every hconfig.conf[“RETARGET_INTERVAL”] number of blocks. It recalibrates the difficulty number. The purpose of this function is to ensure that blocks are mined every ten minutes on the average. This function implements the algorithm set out in Chapter 13.

Miscellany

The remaining functions in the hmining module handle concurrent mining and will be discussed in the next chapter.

Changes to the Helium Blockchain Module

In order to handle secondary blockchains, add the following top-level attribute to the hblockchain module:
"""
secondary block used by mining nodes
"""
secondary_blockchain = []

Helium Mining Unit Tests

We are now ready to run unit tests on the mining module. Copy the following code into a file called test_hmining.py and save it in the unit_tests directory. Execute the tests in your virtual Python environment with
$ pytest test_hmining.py -s
You should see 40 unit tests passing:
"""
   pytest unit tests for hmining module
"""
import hmining
import hblockchain
import tx
import hchaindb
import blk_index
import hconfig
import rcrypt
import plyvel
import random
import secrets
import time
import pytest
import pdb
import os
def teardown_module():
    #hchaindb.close_hchainstate()
    if os.path.isfile("coinbase_keys.txt"):
         os.remove("coinbase_keys.txt")
##################################
# Synthetic Transaction
##################################
def make_synthetic_transaction():
    """
    makes a synthetic transaction with randomized values
    """
    transaction = {}
    transaction['version'] = hconfig.conf["VERSION_NO"]
    transaction['transactionid'] = rcrypt.make_uuid()
    transaction['locktime'] = 0
    transaction['vin'] = []
    transaction['vout'] = []
    # make vin list
    vin = {}
    vin["txid"] = rcrypt.make_uuid()
    vin["vout_index"] = 0
    vin["ScriptSig"] = []
    vin["ScriptSig"].append(rcrypt.make_uuid())
    vin["ScriptSig"].append(rcrypt.make_uuid())
    transaction['vin'].append(vin)
    # make vout list
    vout = {}
    vout["value"] = secrets.randbelow(10000000) +1000
    vin["ScripPubKey"] = []
    transaction['vout'].append(vout)
    return transaction
def make_synthetic_block():
    """
    make synthetic block for unit testing
    """
    block = {}
    block["transactionid"] = rcrypt.make_uuid()
    block["version"] = hconfig.conf["VERSION_NO"]
    block["difficulty_bits"] = hconfig.conf["DIFFICULTY_BITS"]
    block["nonce"] = hconfig.conf["NONCE"]
    block["height"] = 1024
    block["timestamp"] = int(time.time())
    block["tx"] = []
    # add transactions to the block
    num_tx = secrets.randbelow(6) + 2
    for __ctr in range(num_tx):
        trx = make_synthetic_transaction()
        block["tx"].append(trx)
    block["merkle_root"] =  hblockchain.merkle_root(block["tx"], True)
    block["prevblockhash"] =rcrypt.make_uuid()
    return block
@pytest.mark.parametrize("block_height, reward", [
    (0, 50),
    (1, 50),
    (11, 25),
    (29, 12),
    (31, 12),
    (34, 6),
    (115,0 )
])
def test_mining_reward(block_height, reward):
    """
    test halving of mining reward
    """
    # rescale to test
    hconfig.conf["REWARD_INTERVAL"] = 11
    hconfig.conf["MINING_REWARD"] = 50
    assert hmining.mining_reward(block_height) == reward
    hconfig.conf["MINING_REWARD"] = 5_000_000
def test_tx_in_mempool(monkeypatch):
    """
    test do not add transaction to mempool if it is
    already in the mempool
    """
    hmining.mempool.clear()
    monkeypatch.setattr(tx, "validate_transaction", lambda x: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    syn_tx = make_synthetic_transaction()
    hmining.mempool.append(syn_tx)
    assert hmining.receive_transaction(syn_tx) == False
    assert len(hmining.mempool) == 1
    hmining.mempool.clear()
def test_tx_in_chainstate(monkeypatch):
    """
    test do not add transaction if it is accounted for in the
    Chainstate
    """
    hmining.mempool.clear()
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: {"mock": "fragment"})
    syn_tx = make_synthetic_transaction()
    assert hmining.receive_transaction(syn_tx) == True
    hmining.mempool.clear()
def test_invalid_transaction(monkeypatch):
   """
   tests that an invalid transaction is not added to the mempool
   """
   monkeypatch.setattr(hchaindb, "get_transaction", lambda x: False)
   monkeypatch.setattr(tx, "validate_transaction", lambda x: False)
   hmining.mempool.clear()
   syn_tx = make_synthetic_transaction()
   assert hmining.receive_transaction(syn_tx) == False
   assert len(hmining.mempool) == 0
def test_valid_transaction(monkeypatch):
   """
   tests that a valid transaction is added to the mempool
   """
   monkeypatch.setattr(hchaindb, "get_transaction", lambda x: False)
   monkeypatch.setattr(tx, "validate_transaction", lambda x, y: True, False)
   hmining.mempool.clear()
   syn_tx = make_synthetic_transaction()
   assert hmining.receive_transaction(syn_tx) == True
   assert len(hmining.mempool) == 1
   hmining.mempool.clear()
def test_make_from_empty_mempool():
    """
    test cannot make a candidate block when the mempool is empty
    """
    hmining.mempool.clear()
    assert hmining.make_candidate_block() == False
def test_future_lock_time(monkeypatch):
    """
    test that transactions locked into the future will not be processed"
    """
    monkeypatch.setattr(tx, "validate_transaction", lambda x: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    hmining.mempool.clear()
    tx1 = make_synthetic_transaction()
    tx1["locktime"] = int(time.time()) + 86400
    hmining.mempool.append(tx1)
    assert(bool(hmining.make_candidate_block())) == False
def test_no_transactions():
    """
    test that candidate blocks with no transactions are not be processed"
    """
    hmining.mempool.clear()
    tx1 = make_synthetic_transaction()
    tx1[tx] = []
    assert hmining.make_candidate_block() == False
def test_add_transaction_fee(monkeypatch):
    """
    tests that a transaction includes a miner’s transaction fee
    """
    monkeypatch.setattr(tx, "validate_transaction", lambda x: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    monkeypatch.setattr(tx, "transaction_fee", lambda x,y: 12_940)
    trx = make_synthetic_transaction()
    trx = hmining.add_transaction_fee(trx, "pubkey")
    vout_list = trx["vout"]
    vout = vout_list[-1]
    assert vout["value"] == 12_940
def test_make_coinbase_transaction():
    """
    tests making a coinbase transaction
    """
    ctx = hmining.make_coinbase_transaction(10, "synthetic pubkey")
    assert len(ctx["vin"]) == 0
    assert len(ctx["vout"]) == 1
    hash = rcrypt.make_RIPEMD160_hash(rcrypt.make_SHA256_hash("synthetic pubkey"))
    assert ctx["vout"][0]["ScriptPubKey"][1] == hash
def test_mine_block(monkeypatch):
    """
    test that a good block can be mined
    """
    block = make_synthetic_block()
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: False)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x: True)
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(blk_index, "put_index", lambda x,y: True)
    # make the mining easier
    hconfig.conf["DIFFICULTY_NUMBER"] = 0.0001
    assert hmining.mine_block(block) != False
def test_receive_bad_block(monkeypatch):
    """
    test that received block is not added to the received_blocks
    list if the block is invalid
    """
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: False)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: False)
    hmining.received_blocks.clear()
    block = make_synthetic_block()
    assert hmining.receive_block(block) == False
    assert len(hmining.received_blocks) == 0
def test_receive_stale_block(monkeypatch):
    """
    test that received block is not added to the received_blocks
    list if the block height is old
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    hmining.received_blocks.clear()
    for ctr in range(50):
        block = make_synthetic_block()
        block["height"] = ctr
        hblockchain.blockchain.append(block)
    syn_block = make_synthetic_block()
    syn_block["height"] = 47
    assert hmining.receive_block(syn_block) == False
    hblockchain.blockchain.clear()
def test_receive_future_block(monkeypatch):
    """
    test that received block is not added to the received_blocks
    list if the block height is too large
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x,y: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    hmining.received_blocks.clear()
    for ctr in range(50):
        block = make_synthetic_block()
        block["height"] = ctr
        hblockchain.blockchain.append(block)
    syn_block = make_synthetic_block()
    syn_block["height"] = 51
    assert hmining.receive_block(syn_block) == False
    hblockchain.blockchain.clear()
def test_bad_difficulty_number(monkeypatch):
    """
    test that the proof of work fails if the difficulty no
    does not match
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x,y: True)
    monkeypatch.setattr(hmining, "proof_of_work", lambda x: False)
    block= make_synthetic_block()
    block["difficulty_no"] = -1
    block["nonce"] = 60
    assert hmining.proof_of_work(block) == False
def test_invalid_block(monkeypatch):
    """
    test that received block is not added to the received_blocks
    list if the block is invalid
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x,y: False)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    hmining.received_blocks.clear()
    block = make_synthetic_block()
    assert hmining.receive_block(block) == False
    assert len(hmining.received_blocks) == 0
def test_block_receive_once(monkeypatch):
    """
    test that a received block is not added to the received_blocks
    list if the block is already in the list
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    hmining.received_blocks.clear()
    block1 = make_synthetic_block()
    hmining.received_blocks.append(block1)
    assert hmining.receive_block(block1) == False
    assert len(hmining.received_blocks) == 1
    hmining.received_blocks.clear()
def test_num_transactions(monkeypatch):
    """
    a received block must contain at least two transactions
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    hmining.received_blocks.clear()
    block1 = make_synthetic_block()
    block1["tx"] = []
    block1["tx"].append(make_synthetic_transaction())
    hmining.received_blocks.append(block1)
    assert hmining.receive_block(block1) == False
    assert len(hmining.received_blocks) == 1
    hmining.received_blocks.clear()
def test_coinbase_transaction_present(monkeypatch):
    """
    a received block must contain a coinbase transaction
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    hmining.received_blocks.clear()
    block1 = make_synthetic_block()
    block1["tx"] = []
    hmining.received_blocks.append(block1)
    assert hmining.receive_block(block1) == False
    assert len(hmining.received_blocks) == 1
    hmining.received_blocks.clear()
def test_block_in_blockchain(monkeypatch):
    """
    test if a received block is in the blockchain
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    block1 = make_synthetic_block()
    block2 = make_synthetic_block()
    hblockchain.blockchain.append(block1)
    hblockchain.blockchain.append(block2)
    assert len(hblockchain.blockchain) == 2
    assert hmining.receive_block(block2) == False
    hblockchain.blockchain.clear()
def test_block_in_secondary_blockchain(monkeypatch):
    """
    test if a received block is in the secondary blockchain
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    block1 = make_synthetic_block()
    block2 = make_synthetic_block()
    hblockchain.secondary_blockchain.append(block1)
    hblockchain.secondary_blockchain.append(block2)
    assert len(hblockchain.secondary_blockchain) == 2
    assert hmining.receive_block(block2) == False
    hblockchain.secondary_blockchain.clear()
def test_add_block_received_list(monkeypatch):
    """
    add a block to the received blocks list. test for a block
    coinbase tx
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    #monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x: True)
    monkeypatch.setattr(blk_index, "put_index", lambda x,y: True)
    block = make_synthetic_block()
    trx = make_synthetic_transaction()
    block["tx"].insert(0, trx)
    assert hmining.receive_block(block) == False
    assert len(hblockchain.blockchain) == 0
    hmining.received_blocks.clear()
    hblockchain.blockchain.clear()
def test_fetch_received_block(monkeypatch):
    """
    a block in the received_blocks list can be fetched
    for processing
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x: True)
    block = make_synthetic_block()
    hmining.received_blocks.append(block)
    assert hmining.process_received_blocks() == True
    assert len(hmining.received_blocks) == 0
    hmining.received_blocks.clear()
    hblockchain.blockchain.clear()
def test_remove_received_block():
    """
    test removal of a received block from the received_blocks list
    """
    block = make_synthetic_block()
    hmining.received_blocks.append(block)
    assert len(hmining.received_blocks) == 1
    assert hmining.remove_received_block(block) == True
    assert len(hmining.received_blocks) == 0
def test_remove_mempool_transaction():
    """
    test removal of a transaction in the mempool
    """
    block = make_synthetic_block()
    hmining.mempool.append(block["tx"][0])
    assert len(hmining.mempool) == 1
    assert hmining.remove_mempool_transactions(block) == True
    assert len(hmining.mempool) == 0
def test_fork_primary_blockchain(monkeypatch):
    """
    test forking the primary blockchain
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x: True)
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    block1 = make_synthetic_block()
    block2 = make_synthetic_block()
    block3 = make_synthetic_block()
    block4 = make_synthetic_block()
    block4["prevblockhash"] = hblockchain.blockheader_hash(block2)
    hblockchain.blockchain.append(block1)
    hblockchain.blockchain.append(block2)
    hblockchain.blockchain.append(block3)
    assert len(hblockchain.blockchain) == 3
    hmining.received_blocks.append(block4)
    assert len(hmining.received_blocks) == 1
    hmining.process_received_blocks()
    assert len(hblockchain.blockchain) == 4
    assert len(hblockchain.secondary_blockchain) == 0
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
def test_append_to_primary_blockchain(monkeypatch):
    """
    test add a received block to the primary blockchain
    """
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x: True)
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    block1 = make_synthetic_block()
    block2 = make_synthetic_block()
    block3 = make_synthetic_block()
    block4 = make_synthetic_block()
    block4["prevblockhash"] = hblockchain.blockheader_hash(block3)
    hblockchain.blockchain.append(block1)
    hblockchain.blockchain.append(block2)
    hblockchain.blockchain.append(block3)
    assert len(hblockchain.blockchain) == 3
    hmining.received_blocks.append(block4)
    assert len(hmining.received_blocks) == 1
    hmining.process_received_blocks()
    assert len(hblockchain.blockchain) == 4
    assert len(hblockchain.secondary_blockchain) == 0
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
def test_add_orphan_block(monkeypatch):
    """
    test add_orphan_block
    """
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    hmining.orphan_blocks.clear()
    monkeypatch.setattr(hblockchain, "blockheader_hash", lambda x: "mock_hash")
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y : True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x : True)
    monkeypatch.setattr(blk_index, "put_index", lambda x,y : True)
    block0 = make_synthetic_block()
    block1 = make_synthetic_block()
    block1["height"] = block0["height"] + 1
    block1["prevblockhash"] = "mock_hash"
    hblockchain.blockchain.append(block0)
    hmining.received_blocks.append(block1)
    assert len(hmining.received_blocks) == 1
    hmining.process_received_blocks()
    assert len(hmining.orphan_blocks) == 0
    assert len(hblockchain.blockchain) == 2
    assert len(hblockchain.secondary_blockchain) == 0
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
def test_swap_blockchain():
    """
    test swap the primary and secondary blockchains
    """
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    block1 = make_synthetic_block()
    block2 = make_synthetic_block()
    block3 = make_synthetic_block()
    block4 = make_synthetic_block()
    block4["prevblockhash"] = hblockchain.blockheader_hash(block3)
    hblockchain.blockchain.append(block1)
    hblockchain.secondary_blockchain.append(block2)
    hblockchain.secondary_blockchain.append(block3)
    hblockchain.secondary_blockchain.append(block4)
    assert len(hblockchain.blockchain) == 1
    assert len(hblockchain.secondary_blockchain) == 3
    hmining.swap_blockchains()
    assert len(hblockchain.blockchain) == 3
    assert len(hblockchain.secondary_blockchain) == 1
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
def test_clear_blockchain():
    """
    test clear the secondary blockchains
    """
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    block1 = make_synthetic_block()
    block2 = make_synthetic_block()
    block3 = make_synthetic_block()
    block4 = make_synthetic_block()
    block5 = make_synthetic_block()
    hblockchain.blockchain.append(block1)
    hblockchain.blockchain.append(block2)
    hblockchain.secondary_blockchain.append(block3)
    hblockchain.blockchain.append(block4)
    hblockchain.blockchain.append(block5)
    assert len(hblockchain.secondary_blockchain) == 1
    assert len(hblockchain.blockchain) == 4
    hmining.swap_blockchains()
    assert len(hblockchain.blockchain) == 4
    assert len(hblockchain.secondary_blockchain) == 0
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
def test_remove_stale_orphans(monkeypatch):
    """
    test to remove old orphan blocks
    """
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    hmining.orphan_blocks.clear()
    block1 = make_synthetic_block()
    block1["height"] = 1290
    block2 = make_synthetic_block()
    block2["height"] = 1285
    monkeypatch.setattr(hblockchain, "blockheader_hash", lambda x: "mock_hash")
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y : True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x : True)
    monkeypatch.setattr(blk_index, "put_index", lambda x,y : True)
    hblockchain.blockchain.append(block1)
    assert len(hblockchain.blockchain) == 1
    hmining.orphan_blocks.append(block2)
    assert len(hmining.orphan_blocks) == 1
    hmining.handle_orphans()
    assert len(hmining.orphan_blocks) == 0
    assert len(hblockchain.blockchain) == 1
    assert len(hblockchain.secondary_blockchain) == 0
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.orphan_blocks.clear()
def test_add_orphan_to_blockchain(monkeypatch):
    """
    test to add orphan block to blockchain
    """
    hmining.orphan_blocks.clear()
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(hblockchain, "blockheader_hash", lambda x: "mockvalue")
    monkeypatch.setattr(tx, "validate_transaction", lambda x, y: True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x: True)
    monkeypatch.setattr(blk_index, "put_index", lambda x, y: True)
    block0 = make_synthetic_block()
    block1 = make_synthetic_block()
    block1["height"] = 1290
    block2 = make_synthetic_block()
    block2["height"] = 1291
    hblockchain.blockchain.append(block0)
    hblockchain.blockchain.append(block1)
    assert len(hblockchain.blockchain) == 2
    hmining.orphan_blocks.append(block2)
    assert len(hmining.orphan_blocks) == 1
    block2["prevblockhash"] = "mockvalue"
    hmining.handle_orphans()
    assert len(hmining.orphan_blocks) == 0
    assert len(hblockchain.blockchain) == 3
    assert len(hblockchain.secondary_blockchain) == 0
    hblockchain.blockchain.clear()
    hmining.orphan_blocks.clear()
def test_add_orphan_to_secondary_blockchain(monkeypatch):
    """
    test to add orphan block to the secondary blockchain
    """
    hmining.orphan_blocks.clear()
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    #monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x, y: True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x: True)
    monkeypatch.setattr(blk_index, "put_index", lambda x, y: True)
    block0 = make_synthetic_block()
    hblockchain.blockchain.append(block0)
    block1 = make_synthetic_block()
    block1["height"] = 1290
    block2 = make_synthetic_block()
    block2["height"] = 1291
    block3 = make_synthetic_block()
    block3["height"] = 1292
    block3["prevblockhash"] = hblockchain.blockheader_hash(block2)
    hblockchain.secondary_blockchain.append(block1)
    hblockchain.secondary_blockchain.append(block2)
    assert len(hblockchain.secondary_blockchain) == 2
    hmining.orphan_blocks.append(block3)
    assert len(hmining.orphan_blocks) == 1
    hmining.handle_orphans()
    assert len(hmining.orphan_blocks) == 0
    assert len(hblockchain.secondary_blockchain) == 1
    assert len(hblockchain.blockchain) == 3
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.orphan_blocks.clear()
def test_append_to_secondary_blockchain(monkeypatch):
    """
    test add a received block to the primary blockchain
    """
    #monkeypatch.setattr(hblockchain, "validate_block", lambda x: True)
    monkeypatch.setattr(tx, "validate_transaction", lambda x,y: True)
    monkeypatch.setattr(hchaindb, "get_transaction", lambda x: True)
    monkeypatch.setattr(hchaindb, "transaction_update", lambda x: True)
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()
    block1 = make_synthetic_block()
    hblockchain.blockchain.append(block1)
    block2 = make_synthetic_block()
    block3 = make_synthetic_block()
    block4 = make_synthetic_block()
    block4["prevblockhash"] = hblockchain.blockheader_hash(block3)
    hblockchain.secondary_blockchain.append(block2)
    hblockchain.secondary_blockchain.append(block3)
    assert len(hblockchain.blockchain) == 1
    assert len(hblockchain.secondary_blockchain) == 2
    block4["height"] = block3["height"] + 1
    block4["prevblockhash"] = hblockchain.blockheader_hash(block3)
    hmining.received_blocks.append(block4)
    assert len(hmining.received_blocks) == 1
    hmining.process_received_blocks()
    assert len(hblockchain.blockchain) == 3
    assert len(hblockchain.secondary_blockchain) == 1
    hblockchain.blockchain.clear()
    hblockchain.secondary_blockchain.clear()
    hmining.received_blocks.clear()

Conclusion

At this stage, we have implemented a substantial portion of a Helium mining node. But our implementation of a mining node is not over. We have yet to implement portions of the hmining module as concurrent processes and interface the mining node with the external Helium peer-to-peer network. This is the matter that will be handled next.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.19.27.178