Back Matter

Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Appendix 1: Environment

A Python virtual environment is an isolated environment for the development of a Python project. This virtual environment contains the Python interpreter that is used for the project, the packages and dependencies of the project, as well as all of the project’s source code. In this appendix, I will show you how to set up a virtual environment for the Helium cryptocurrency project. The instructions that follow are for a Linux host environment. If you are using a Mac or Windows machine, the instructions are analogous and available on the Internet.

My Development Environment

This is my development environment for the Helium project. The host environment is Linux Mint release 19.3 Tricia, a 64-bit distribution running on a Lenovo ThinkPad. Mint is a distribution forked from Ubuntu. The windows manager is a Gnome version 2 fork, called Mate by the Linux Mint development team. Mint has a reputation for protecting privacy rights and espousing progressive politics.¹ I have created 24 virtual desktops on the laptop with Mate.² Typically, my IDE will be running in one virtual desktop. In another desktop, a terminal will be open with the Python interpreter ready to run the application. A third virtual desktop will have Git available at a terminal prompt. Another desktop will have a database client GUI open. A virtual desktop will have the Firefox browser pointing to the localhost address of the Helium application. LibreOffice will be open on another desktop for editing this book. A desktop may have VLC paused on a movie I am watching. A ribbon at the bottom of the screen displays icons for all of these virtual desktops. I can switch between these desktops by simply clicking one of these icons. This is the big picture. It is a very efficient development environment.

I periodically deploy Helium to a Linux Ubuntu server staging environment. Deployment is easy since the development and staging environments are closely related Linux distributions. My IDE is Visual Code and the version control system is Git. I use rsync to move files to the staging server. The staging server runs on a VPS (virtual private server). My laptop typically has a terminal with an active SSH (secure shell) connection to the VPS root shell.

This development environment is not very fancy, but it is dead simple and it works well.

Preliminary Steps

Check whether the pip3 tool is installed. pip3 (the Python Index Package Tool) is used to install, update, and remove Python packages and libraries.³

In your Linux operating system, you can drop into a command shell by opening a terminal window. This permits us to type in shell commands . In the exposition that follows, I shall denote a system command shell prompt with a $.

Open a terminal and do

$ which pip3 # returns the path where pip3 is installed

or:

$ pip3 --version

If pip3 is not installed, install it on Debian-derived Linux distributions , such as Mint and Ubuntu, as follows:

$ sudo apt-get update

$ sudo apt-get -y install python3-pip

This will install the latest version of Python. Test whether Python3 is installed:

$ which python3

or:

$ python3 -V

The first command returns the path where a Python version 3 is installed. The second command will display the version of Python that is installed.

We need a Python version that is 3.8 or greater. If Python is not present or we do not have a conforming version, we must install a valid Python version. We can install a specific version from the deadsnakes repository . To install version 3.8 of Python on your Ubuntu-derived Linux distribution, do

$ sudo apt install software-properties-common

$ sudo add-apt-repository ppa:deadsnakes/ppa

$ sudo apt-get update

$ sudo apt install python3.8

When executed from the terminal command line, this sequence of commands will install Python version 3.8 on your Ubuntu-derived distribution.

Create a Virtual Environment for the Project

virtualenv is a tool that creates isolated Python environments. virtualenv creates a directory tree and a virtual environment for our project. Install virtualenv in your host environment:

$ pip3 install virtualenv

Next, test that virtualenv has been successfully installed:

$ virtualenv --version

Now that we have installed virtualenv, we create a project directory called helium and create a virtual Python environment inside this directory:

$ mkdir helium

$ cd helium

$ virtualenv -p /usr/bin/python3.8 virtual

This sequence of commands will create a directory helium under your present directory. It will then create a virtual sub-directory inside the helium directory and install Python version 3.8 inside this virtual directory.

Note

You must have already installed Python version 3.8 into the host environment, as previously indicated.

The virtual folder will contain the Python interpreter, the pip library, as well as other modules and dependencies. pip3 will be used to install additional modules and packages inside the virtual environment. The name of the virtual environment (in this case helium) can be anything.

The preceding virtualenv command installs Python version 3.8 in the directory virtual/bin.

Now that we have created a virtual environment for the Helium project, we need to activate the environment in order to work in it. Drop down into the Helium directory and then enter

$ source virtual/bin/activate

The command-line prompt will prepend the string (virtual) to itself to indicate that we are in an active virtual environment. We can now install packages inside the environment with pip3, as usual, for example:

# do not install

(virtual) $ pip install requests

This command will install the requests module into the Helium virtual environment.

After we have finished working in the virtual environment, we deactivate it:

$(virtual) deactivate

We can freeze the virtual environment by entering the following command inside the root directory of the virtual environment (helium) as follows:

$(virtual) pip freeze > requirements.txt

This command will create a requirements.txt text file, which contains a simple list of all the installed modules in the current environment and their respective versions.

We can view the list of modules installed inside the virtual environment with

$(virtual) pip3 list

We can install the package versions listed in requirements.txt as follows:

$ (virtual) pip3 install -r requirements.txt

Appendix 2: Installing LevelDB for Helium

LevelDB is a key-value store implemented as a library. It was primarily developed by Sanjay Ghemawat during his tenure at Google.⁴ LevelDB is a lightweight, high-performance keystore. Its outstanding performance is primarily due to the simplicity of its construction and its limited design objectives.⁵ LevelDB is single threaded and only supports string keys and string values. Keys and values can be arbitrarily long.

LevelDB has three basic operations:

Put(key,value)

Get(key)

Delete(key)

LevelDB does not have any indexes; in lieu thereof, the keys in the store are maintained in a sorted lexicographic order.

In this appendix, we will install LevelDB and Plyvel. Plyvel is a Python interface module for the LevelDB library.

LevelDB Installation

Install the libsnappy compression development library :⁶

$ sudo apt-get install libsnappy-dev

Install git:

$ sudo apt-get install git-core

If git has been successfully installed, then the following command will display help on the terminal screen:

$ git

Change to the /tmp directory (or any other suitable directory) and clone LevelDB from its Github directory:⁷

$ git clone https://github.com/google/leveldb

We are now ready to compile the LevelDB source code. LevelDB uses cmake to compile the source and produce binaries. Install cmake:

$ sudo apt-get install cmake

Change to the tmp/leveldb directory and compile the source code:

$ cd leveldb

$ cmake -DCMAKE_BUILD_TYPE=Release .

$ cmake --build .

Install the LevelDB binary and include files:

$ cp --preserve=links libleveldb.* /usr/local/lib

$ cp -r include/leveldb /usr/local/include/

$ ldconfig

Build and Install Plyvel

The package Plyvel is a Python interface for the LevelDB library. Traverse to the root of your Helium virtual environment and activate it. Now we can install Plyvel:

$(virtual) pip install plyvel

We next test that plyvel has been successfully installed. Start the Python interpreter in the virtual environment:

$(virtual) python

At the interpreter command-line prompt, enter

>>> import plyvel

If the import generates an error, LevelDB has not been properly installed. It’s most likely that plyvel cannot find the levelDB library. This is a path issue.

You can also test the plyvel installation with

$(virtual) python -c 'import plyvel'

This command will not generate any output if the LevelDB and Plyvel installation were both successful.

A Plyvel Primer

Plyvel is remarkably easy to use . The following commands in the Python interpreter shell demonstrate its usage:

$(virtual) python

>>> import plyvel

# create a key-value store

>>> db = plyvel.DB('/tmp/testdb/', create_if_missing=True)

# put a few keys and values as byte strings

>>> db.put(b"farm", b"green acres")

>>> db.put(b"dog", b"wolf")

# get a key value

>>> db.get(b"farm")

>>> "green acres"

# try to retrieve a non-existent key

>> db.get(b"hens")

>> None

# delete a key

>>> db.delete(b"dog")

# close the database

>>> db.close()

# delete the database

>>> delete db

Iterating over the LevelDB Keyspace

Since LevelDB is a sorted keyspace , we can iterate over its keys:

>>> for key, value in db:

... print(key)

... print(value)

>>> b"farm"

>>> b"wolf"

...

We can also iterate over restricted key ranges:

>>> for key, value in db.iterator(start=b'key 6', stop=b'key 45'):

... print(key)

By default, Plyvel will start the iterator at the key b’key 6’ and stop at the key which is before b‘key 45’. This matches default Python behavior.

We can omit either the first argument or the last argument of db.iterator in order to iterate from the first key or iterate to the last key.

To iterate in reverse order, simply reverse the start and stop keys; do

>>> for key, value in db.iterator(start=b'key 45', stop=b'key '6'):

... print(key)

We can also iterate over keys which have a certain prefix:

>>> for key, value in db.iterator(prefix=b'fa'):

... print(key)

In this example, we iterate over keys which have the prefix bytes ‘fa’.

We can create iterators and use them as follows :

>>> iter = db.iterator()

...

>>> iter.close()

Appendix 3: Unit Tests with Pytest

Unit tests validate functions and methods. In particular, they verify that functions and methods return correct values for given inputs. We should always strive to provide maximal unit test coverage for our code. This will provide us with assurances of code correctness as well as quality.

For the Helium cryptocurrency application, we will use the pytest module for writing unit tests. pytest is the de facto Python standard for implementing unit tests.

Install pytest in your Helium virtual environment as follows:

$(virtual) pip install pytest

Running Unit Tests with Pytest

To run unit tests , traverse to a directory containing the tests, and execute pytest from the command line:

$(virtual) pytest test_rcrypt.py

This command will cause pytest to run all of the unit tests in the test_rcrypt.py file. The command-line option -v causes pytest to generate verbose output:

$(virtual) pytest test_rcrypt.py -v

If we are in a directory containing pytest files, we can run all of the unit tests in all of the pytest files in this directory with

$(virtual) pytest

$(virtual) pytest -v

In general, we can run all of the tests in a directory with

$(virtual) pytest path/to/directory/ -v

A pytest file is a Python file that contains pytest functions. pytest file names must be in either of these two forms: test_*.py or *_test.py (* is a wildcard, meaning any sequence of alphanumeric characters or underscores). For example, test_trig.py and trig_test.py are valid names for a pytest file.

A test function in a pytest file must begin with the prefix test_ or end with the suffix _test. For example, test_arithmetic and arithmetic_test are valid names for pytest functions.⁸

If we are debugging a single function or method, it is frequently very useful to only run a single test in a pytest file:

$(virtual) pytest path/to/test_file.py -k test_adder

This command executes the unit test called test_adder in the pytest file test_file.py. The parameter to k can be a sub-string instead of the whole name of a test function, for example:

$(virtual) pytest path/to/test_file.py -k arithmetic -v

This command will execute all of the pytest functions in the file test_file.py; those names contain the sub-string arithmetic.

Writing Pytest Tests

pytest uses the assert function to test whether a unit test passes or fails. assert() tests an expression and returns True or False. The following code shows the structure of a simple pytest unit test module with a single test in it:

# file test_functions.py

import pytest

def test_capital_case():

assert "python".upper == "PYTHON"

Since the expression “python”.upper() == “PYTHON” is True, the assert function returns True and the test passes. Similarly, the following test fails if x does not equal 0:

import pytest

import my_module

def test_arithmetic():

x = my_module.arithmetic_func(1)

assert x == 0

Note that a test function does not have any arguments (with the exception of fixtures and pytest parametrized tests, which are discussed subsequently).

Floating-Point Numbers and Tests for Approximation

In cases where we are handling floating-point numbers, it is typically necessary to test that a floating-point number is within a certain tolerance interval of another number. In pytest, we test for such an approximation as follows:

# my_module.py

def div(x, y):

return x/y

# test_sample.py module

import pytest

import my_module

def test_approx():

x = my_module.div(22,7)

assert x == pytest.approx(3.3, 1.5)

This tests whether the number x returned by the div function in my_module is within 1.5 of the reference number 3.3. The tolerance interval is 1.5.

Testing Exceptions

The following pytest code demonstrates how to test that an exception has been raised by a function:

# my_module.py

def div(x, y ):

return x/y

# test_my_module.py

import pytest

import my_module

def test_division():

with pytest.raises(Exception):

z = my_module.div(5, 0)

In this example, we are using the test_division function in the pytest file test_my_module.py to test the function div in my_module. The function div(x, y) throws an exception if the divisor is zero. This exception is trapped in the with block of test_division. Since the test correctly traps the exception, the test passes.

Setup and Teardown

If a pytest module contains any of the following functions: setup_module() or teardown_module() , then the setup_module function will be executed before any of the tests in the pytest module are run. Similarly, after all of the tests in the pytest module have been executed, the teardown_module function, if present, will be executed. The setup_module function allows us to perform initialization tasks before any of the tests are executed. Similarly, the teardown_module function performs cleanup tasks after the tests have run.

Here is an example:

# my_module.py

def init():

# some initialization tasks

database.start()

def finish():

# perform cleanup

database.stop()

# test_sample.py

import pytest

import my_module

# setup_module runs once in the test module

# before all tests

def setup_module():

my_module.init()

# teardown_module runs once in the test module

# after all tests have been executed

def teardown_module():

my_module.finish()

def test_func():

assert 1 == True

Pytest Fixtures

A fixture is a function which returns a predetermined value. Pytest fixtures are created by prefixing the fixture function with the decorator @pytest.fixture. A test function can use a fixture by specifying the fixture name as a function argument (in the test function).

Here is an example of using fixtures:

# test_sample.py

import pytest

# creates a fixture called db_connection

# simulates a valid connection to a database

@pytest.fixture

def db_connection():

return True

def test_is_connected(db_connection):

response = db_connection

assert response == True

Fixtures allow us to replace functions in our code with fixtures that return predetermined values.

Parametrized Test Functions

Parametrized test functions are one of the best features of pytest. This feature lets us test various scenarios easily. In other words, we can test the correctness of a large number of function inputs with ease.

The following example will demonstrate how this feature works:

# my_module.py

class Account:

def __init__(self, balance):

self.balance = balance

def deposit(self, amount):

self.balance += amount

def spend(self, amount):

self.balance -= amount

# test_sample.py

import pytest

import my_module

@pytest.mark.parametrize("deposit, spent, balance", [

(30, 10, 20),

(20, 2, 18),

(95, 25, 70),

(0, 0, 0)

])

def test_transactions(deposit, spent, balance):

my_account = my_module.Account(0)

#use a parametrized deposit

my_account.deposit(deposit)

# use a parametrized spend value

my_account.spend(spent)

# test the actual account balance against the

# parametrized balance

assert my_account.balance == balance

The decorator function is @pytest.mark.parametrize("deposit, spent, balance", [ … ]).

This parametrization has three variables: deposit, spent, and balance. Each tuple (deposit, spent, balance) is a particular scenario. In our case, there are four scenarios.

The test function test_transactions(deposit, spent, balance) uses the parameter values for each of the scenarios specified in the decorator function. This function uses the deposit and spent values and then compares the actual balance in the account object with the parametrized balance value.

Note that Python syntax requires that the parametrized test function test_transactions(deposit, spent, balance) follow immediately after the decorator.

Mocks

A mock replaces an attribute with a synthetic attribute that we have created or pretends that the attribute does not exist. The mocked attribute can be a function, method, dictionary value, an environmental variable, or any other top-level attribute in a module. Mocking permits us to substitute a mocked attribute for the real attribute. In pytest, mocking is accomplished with the monkeypatch() function .

Stubbing or mocking the return value of a function or method is a major use case of mocking. Suppose that we want to mock the function employee_record(id) in the employee module. Suppose further that this function has an id parameter and returns an employee name, then we can mock this function as follows:

monkeypatch.setattr(employee, "employee_record", lambda x: "alice smith")

The first monkeypatch function parameter is the module that contains the function that is to be mocked; the second parameter is the name of the function that is being mocked (as a string.)

Here we have mocked the employee_record function in the employee module to return “alice smith”.

The lambda function takes a number of symbolic parameters that is equal to the number of parameters in the mocked function. Since employee_record has one parameter, the lambda has one symbolic parameter x.

We can also write this function mock more concisely as

monkeypatch.setattr("employee.employee_record", lambda x: "alice smith")

Take a look at the following more complex mocked function implementation:

monkeypatch.setattr(block, "make_hash", lambda x, y: fake.hash(5,6))

This mocks the function make_hash in the block module. make_hash has two parameters; therefore, lambda has two symbolic parameters. The mock function returns the value produced by the function call fake.hash(5,6), where hash is a function in the fake module.

Suppose that we want to mock delete a top-level attribute in a module. The syntax for effecting this is

monkeypatch.delattr(module_name, "attribute")

For example,

monkeypatch.delattr(employee, "employee_record")

will mock delete the function employee_record in the employee module. Thus, in the context of the test environment, this attribute does not exist. We can also specify such an attribute deletion more concisely with

monkeypatch.delattr("employee.employee_record")

Monkeypatching dictionary values is also a prominent use case. Suppose that we have a dictionary called travel and a key in this dictionary which is called “airline”. We can mock the value for this key as follows:

monkeypatch.setitem(travel, "airline", mocked_value)

monkeypatch.setitem("travel.airline", mocked_value)

This will set the value of the airline key to mocked_value.

The following is the syntax to mock the deletion of a dictionary key:

monkeypatch.delitem(some_dictionary, "key")

or:

monkeypatch.delitem("some_dictionary.key")

We can also mock environmental variables. In the following example, we mock the value of an environment variable called COLOR:

monkeypatch.setenv(COLOR, value)

We can mock the deletion of an environment variable called "shell" as follows:

monkeypatch.delenv("shell")

The functions monkeypatch.setattr, monkeypatch.delattr, monkeypatch.delitem, and monkeypatch.delenv raise an exception if the target does not exist. The attribute that is being mocked must exist. This is default behavior. If you specify False as the last parameter to any of these functions, then an exception will not be raised.

Using Monkeypatching in Unit Tests

A test function that uses monkeypatching must pass monkeypatch as a parameter, for example:

def test_missing_difficulty_bit(monkeypatch):

"""

test for missing difficulty bits

"""

monkeypatch.setattr(tx, "validate_transaction", lambda x, y: True)

monkeypatch.setitem(block_1, "difficulty_bits", '')

assert hblockchain.add_block(block_1) == False

Note that in this example the dictionary key value for difficulty_bits is being mocked as an empty string.

Appendix 4: Making Concurrent Python Programs

A set of processes, tasks, or events is said to be concurrent if we do not know which one will finish first. A thread can be thought of as a block of code in a program that runs independently of other code in the program. We say that threads execute concurrently.

When a Python program begins execution, it runs in a main thread of execution. This thread may spawn additional threads in order to perform certain tasks. Hence, a program may consist of the main thread and several concurrently executing threads. For example, we could have a program where one thread is computing and printing Fibonacci numbers, and the second thread is searching for prime numbers and printing them. Because a CPU is extremely fast, it appears to a user that these two threads are executing simultaneously.

Aside from possibly increasing the speed at which a program is executed, one of the main benefits of implementing concurrency in a program is that it can improve program organization by partitioning code into separate tasks that can each run in a separate thread.

Thread Implementation in Python

To get up to speed with concurrency, we are going to write a concurrent Python program with two threads. The first thread will compute and print Fibonacci numbers, and the second thread will search for prime numbers and print them.

A Fibonacci sequence is a set of numbers where the first two numbers are 0 and 1. Each subsequent Fibonacci number is the sum of the previous two numbers. So our Fibonacci sequence looks like this:

0,1,1,2,3,5,8,13,21...

A prime number is a natural number that is only divisible by itself and 1.

In order to implement threads, our program has to import the threading module:

import threading

In order to implement Python concurrency, we need Python 3.6 or later.

The fibonacci function is

import threading

import time

def fibonacci():

ctr = 2

fib1 = 1

fib2 = 2

print("fibonacci number 0 = " + "0")

print("fibonacci number 1 = " + "1")

while True:

tmp = fib1 + fib2

fib1 = fib2

fib2 = tmp

print("fibonacci no: " + str(ctr) + " = " + str(fib2))

ctr += 1

time.sleep(1)

You will notice that this function is an infinite loop. After printing a fibonacci number, the function sleeps for one second and then continues its task when it wakes up.

The prime number function is

def prime():

number = 3

half = 3

while True:

for ctr in range(half):

divided = False

if ctr < 2: continue

if number%ctr == 0:

divided = True

break

if divided == False:

print("next prime number: " + str(number))

number += 1

half = int(number/2) + 1

time.sleep(1)

Like the fibonacci function, this is an infinite loop; it sleeps for one second after printing a prime number.

Our program starts and runs in a main thread. The Python code for the main thread is

if __name__ == "__main__":

print("starting program in main thread")

#Create the Threads

t1 = threading.Thread(target=fibonacci, args=(), daemon=True)

t2 = threading.Thread(target=prime, args=(), daemon=True)

# start the threads

t1.start()

t2.start()

#t1.join()

#t2.join()

time.sleep(20)

print("ending program")

The threading.Thread function creates a thread that can be executed. The first parameter is the name of the function that will be executed concurrently. The second parameter is a tuple of arguments that is passed to the function. Since our concurrent functions do not have any parameters, we specify empty tuples. The third argument daemon=True is optional. If this argument is included, the main program thread will terminate the concurrent threads t1 and t2 before it exits.

The two start functions initiate the execution of the t1 and t2 threads.

Put all of the preceding code into a Python script file named fibprime.py . Make sure that the program’s main thread code is at the bottom of this file and that you have included the threading and time modules. Now execute this script in your virtual environment:

$(virtual) python fibprime.py

You should see the fibonacci and prime numbers being printed on your terminal. The main thread creates the fibonacci and prime threads and then sleeps for 20 seconds. After awakening, it terminates the two threads and exits.

The join statement t1.join instructs the main thread not to terminate until the t1 thread has finished with its processing. To be precise, the thread to which the t1 thread is joined will not terminate until t1 has finished its work; similarly for the t2 thread.

Resource Access with Mutual Exclusion

A mutex synchronizes access to a shared resource. The following code demonstrates how to prevent improper concurrent access to a resource with a mutex :

from threading import Lock

balance = 0

hmutex = Lock()

def balance():

global balance

hmutex.acquire()

bal = balance

hmutex.release()

return bal

We must import the Lock module.

We create a lock object called hmutex. The hmutex.acquire and hmutex.release functions wrap around the resource that is to be protected with mutual exclusion. While hmutex is in an acquired state, no other concurrent thread will be able to access the balance until the lock is released. The portion of the code that is enclosed by the mutex is called a critical region.

It is important to release a lock after we have finished using the common resource. Failure to do so will result in resource starvation as threads that need the resource keep on waiting for the lock to be released.

The balance function can be written with a context manager; this ensures the release of the mutex:

def balance():

global balance

with hmutex:

return balance

The lock is automatically released once the code block under the context manager (with hmutex) has finished execution.

Deadlocks caused by improper resource locks are another dangerous pitfall in concurrent systems. Consider two concurrently executing threads T1 and T2, which each needs two resources R1 and R2. T1 acquires a lock on R2, and T2 acquires a lock on R1. T1 keeps on waiting for T2 to release its lock, and T2 keeps on waiting for T1 to release its lock. At this point, both threads are deadlocked.

Race Conditions

A data race occurs when two or more threads attempt to access a common resource at the same time. Data races typically occur infrequently and are difficult to debug. The classic exposition of a data race is the bank account bug. This is a canonical type of bug in concurrent systems. It’s worthwhile to examine this bug carefully. Take a look at the following pseudo-code:

balance = 0

function Deposit(amount) { balance = Balance() + amount }

function Balance() { return balance }

// Alice Bank Account:

function AliceAccount(amount) {

balance = Balance() + amount

}

// Alice:

AliceAccount(200)

// Smith

Deposit(1000)

Consider this scenario ; Alice decides to deposit 200 dollars, so she calls the function AliceAccount which executes on a concurrent thread (Alice thread). The balance() function is called and the balance of 0 dollars is read. At this point in time, Smith decides to deposit 1000 dollars in Alice’s account; the Deposit function is called on another thread. The Deposit function calls the Balance function to read Alice’s bank balance, which is 0 dollars. The thread now completes the deposit action, and at this point, Alice has a balance of 1000 dollars. Now the processor switches back to the Alice thread and resumes execution of the code and sets the final balance to 200 dollars. In other words, Smith’s deposit of 1000 dollars has been lost. This is a data race condition caused by improper concurrent access to the balance resource.

There are three ways to avoid a data race condition: (1) do not have shared variables, (2) design in such a way that only one function has access to a shared variable, and (3) allow many concurrent threads to access a shared variable, but only one at a time. The third approach is known as mutual exclusion, and the object that implements mutual exclusion is called a mutex.

Resource Access with Semaphores

A semaphore is a non-negative integer that synchronizes access to a shared resource. Semaphores are atomic, meaning that the operating system guarantees that an operation changing the value of a semaphore will not be interrupted.

A semaphore is initialized with a value which designates the maximum number of threads that can have concurrent access to the resource which is guarded by the semaphore. A semaphore is decremented each time a lock is acquired, and it is incremented each time a lock is released. If the semaphore has a value of 0, a lock cannot be acquired by any thread until some other thread releases its semaphore lock. It should be clear that a semaphore cannot exceed its initialization value.

The mutex object that we encountered in the last section is in actuality a semaphore initialized with 1.

The following code fragment demonstrates semaphore usage:

import threading

balance = 0

semaphore = threading.Semaphore()

def balance():

global balance

semaphore.acquire()

bal = balance

semaphore.release()

return bal

We obtain a semaphore object with threading.Semaphore. Since we have not specified a value when the semaphore is created, the default value of one is used. The acquire and release functions wrap around the resource that is to be protected.

It is important to release a semaphore after we are finished with it. The use of a semaphore context manager releases the semaphore automatically once the context block is exited:

def balance():

global balance

with semaphore:

return balance

Concurrent Asynchronous Functions

Consider the following design pattern that frequently arises in the context of concurrent applications. We have two concurrently executing functions: F() and G(). F() suspends execution waiting for G() to return a result to it. When F() gets the result, it resumes execution. F() is called an asynchronously executing function. We will see this pattern several times in Helium.

We will use the Python asyncio package to implement asynchronous, concurrent functions. To examine how asyncio works, let us implement our fibprime program so that a function called print_number prints the fibonacci or prime number after sleeping for a few seconds:

import asyncio

import threading

import time

import secrets

async def fibonacci():

ctr = 2

fib1 = 1

fib2 = 2

print("fibonacci number 0" + " = " + "0")

print("fibonacci number 1" + " = " + "1")

while True:

tmp = fib1 + fib2

fib1 = fib2

fib2 = tmp

message = await print_number(fib2, "F")

ctr += 1

time.sleep(1)

async def prime():

number = 3

half = 3

while True:

for ctr in range(half):

divided = False

if ctr < 2: continue

if number%ctr == 0:

divided = True

break

if divided == False:

message = await print_number(number, "P")

number += 1

half = int(number/2) + 1

time.sleep(1)

async def print_number(number: "int", number_type: "F or P" ):

#F: fibonacci number, P: Prime Number

nap = secrets.randbelow(3)

await asyncio.sleep(nap)

if number_type == "F":

print("fibonacci no: " + " = " + str(number))

else:

print("prime number no: " + " = " + str(number))

return "task executed"

async def main():

await asyncio.gather(fibonacci(), prime())

if __name__ == "__main__":

print("starting program in main thread")

asyncio.run(main())

print("ending program")

The important changes in the source code are emphasized. Copy the previous program into a file called fibprime2.py and execute it in your virtual environment:

$(virtual) fibprime2.py

The module asyncio must be imported. There is one cardinal rule with asyncio. If a function is to be executed concurrently with this module, then the function declaration has to be preceded with the keyword async.

The statement

asyncio.run(main())

instructs asyncio to create an event loop and call main(). All concurrent functions will execute under this loop.

The statement in main()

await asyncio.gather(fibonacci(), prime())

directs asyncio to execute the fibonacci and prime functions as concurrent threads under the control of the event loop. The await keyword directs the event loop to wait for these two functions to complete.

Now consider the statement

result = await print_number(number, "P")

in prime(). This statement suspends the execution of the prime function and passes control back to the event loop. Furthermore, it instructs the event loop to only return to it once the print_number function has finished. When print_number finishes execution, it passes the result back to the event loop and the event loop sends the result to prime(). prime() now has the value returned by print_number, and it resumes execution.

Appendix 5: Object Persistence with Pickle

This appendix introduces you to the Pickle module. Helium uses Pickle to serialize blocks to files. This enables blocks to be read into memory from files. Pickle persists a Python object by converting the object into a sequence of bytes and then storing this raw byte sequence to a file.

Aside from native Python types such as strings, numbers, lists, tuples, sets, and dictionaries, Pickle can convert almost any type of Python object into a sequence of bytes. This conversion is called serialization of the object.

Using Pickle

Here is an example of pickle usage :

# pickle_example.py

import pickle

dct = {

"prevblockhash": "31651d"

"version": "1.0"

"timestamp": 1580399346

"difficulty_bits": 20

"nonce": 0

"merkle_root": "2EECC5"

"height": 0

"tx": []

}

f = open('block_0.dat', 'wb')

pickle.dump(dct, f)

f.close()

As this example shows, we open or create a file called block_0.dat. The mode ‘wb’ means that the file is opened for writing in binary mode and the file will be truncated if it exists and created if it does not exist. Once the file has been opened, we use the pickle.dump function to serialize the dictionary dct to the file block_0.dat. Pickle will throw an exception if serialization fails.

We can use Pickle to retrieve the serialized object from the file data.pkl as follows:

f = open('block_0.dat', 'rb')

block_0 = pickle.load(f)

The file block_0.dat is opened in read binary mode. Then the function pickle.load is called with the file handle f as its argument. pickle deserializes the bytes in the file and recreates the block that was originally serialized.

Appendix 6: Helium Remote Procedure Calls

In this appendix, we are concerned with calling functions in remote modules. You should have familiarity with the content in Appendix 4 on concurrent Python programs.

A remote procedure call permits a node on a network to invoke a function on some other remote node and receive the result of this function execution. The request to execute the remote function is encoded as a JSON object and transmitted over the network to a JSON server that is running on the remote node. The server decodes the request and executes the function. Once the function has finished execution, the server packs the return value into a JSON object and sends the reply back to the RPC client that has made the request . Figure A6-1 shows this functionality at work.

Figure A6-1

JSON Remote Procedure Calls

A JSON request and response are both packaged in accordance with the JSON-RPC protocol. This is a very simple text-based protocol. We will use version 2 of this protocol. A request is packaged as the following JSON object:

{"jsonrpc": "2.0", "method": "add", "params": {"x": 5, "y": 23}, "id": 3}

As you will notice, this object is also a Python dictionary object. For all practical purposes, JSON objects and Python dictionaries are indistinguishable.

The particular request earlier is asking the remote server to invoke the function add to add two numbers 5 and 23. The method attribute in the JSON object specifies the function that is to be executed as a string type. The params attribute specifies the function parameters as a JSON object. Note that these are positional parameters. If the function does not have any parameters, we will not send a params attribute. The id parameter specifies an identification number that the server will use when it encodes its reply. When the RPC client receives a reply, it uses this id to identify the request to which the reply pertains. The jsonrpc attribute sets out the version of the JSON-RPC protocol that is being used in the encoding of the request.

An RPC server responds to the request to execute the add function by encoding its reply as the following JSON object:

{"jsonrpc": "2.0", "result": 28, "id": 3}

The result attribute is the return value emitted by the add function. The id attribute identifies the request to which the response pertains. The jsonrpc attribute specifies the protocol version number that has been used to encode the result.

Both Bitcoin and Helium use the JSON-RPC protocol to execute remote functions.

Making an RPC Server

In this section, we are going to create an asynchronous JSON-RPC server and demonstrate canonical client-server remote procedure call usage. This model will be used to implement peer-to-peer network functionality in Helium. A function call is synchronous if the caller waits for the function to return a value before proceeding to the execution of the next instruction. A function call is asynchronous if the caller does not wait for the called function to return a value but immediately continues with its processing after making the function call. The caller will process the function value returned (if any) when it arrives some time in the future.

Tornado is a highly scalable, asynchronous networking library.⁹ jsonrpcserver is a JSON-RPC server built on top of Tornado.¹⁰ Traverse to the root of your Helium environment and install both of these applications:

$(virtual) pip install tornado, jsonrpcserver

Next, install a JSON-RPC client that will be used to communicate with the server :

$(virtual) pip install "jsonrpcclient[tornado]"

Go to the root of your Helium virtual environment and make a directory named tmp in the Helium root. Copy the following program code to the file rpc_server.py and then save this code in the tmp directory . This is the code for our JSON-RPC server :

from tornado import ioloop, web

from jsonrpcserver import method, async_dispatch as dispatch

@method

async def fibonacci(m, n):

n = n+m

x1 = 0

x2 = 1

while True:

if x2 >= n: return {"method":"fibonacci", "result": x2, "error": "none"}

tmp = x2

x2 = x2 + x1

x1 = tmp

class MainHandler(web.RequestHandler):

async def post(self):

request = self.request.body.decode()

response = await dispatch(request)

print(response)

if response.wanted:

self.write(str(response))

app = web.Application([(r"/", MainHandler)])

if __name__ == "__main__":

app.listen(address="127.0.0.24", port="8081")

ioloop.IOLoop.current().start()

We have imported modules from Tornado and jsonrpcserver.

ioloop implements event loop functionality for Tornado, while the web module implements an HTTP framework. The method module handles functions which are callable through a remote procedure call, while async_dispatch handles asynchronous calls to these functions.

A function that is remotely callable must be prefaced with the decorator @method, and since the function returns its result asynchronously, the function definition must be prefaced with the async keyword.

The MainHandler class implements the asynchronous post method. This method handles a request by an RPC client for a function invocation. post invokes the function and packages function return value as a response which is sent back to the RPC client.

This RPC server has a callable method fibonacci.

The fibonacci function receives two numbers m and n. It sums these two numbers and computes the first fibonacci number that is greater than this sum. The fibonacci function returns a JSON object of the form:

{"method":"fibonacci", "result": fib_number, "error": "none"}

The app.listen function sets the IP address and port on which the server will listen. The ioloop statement creates an event-handling loop and starts the server.

Start the RPC server in your virtual environment with

$(virtual) python rpc_server.py

An RPC client is said to be synchronous if it makes a remote procedure call and then waits for a reply before proceeding. An asynchronous client does not wait for a reply and proceeds with other processing right after making the call. The client processes the reply when it arrives at some point later in time.

The program code for a synchronous JSON RPC client is

from jsonrpcclient.clients.http_client import HTTPClient

import json

try:

client = HTTPClient("http://127.0.0.24:8081")

print("RPC client is ready")

response = client.send('{"jsonrpc": "2.0", "method": "fibonacci",

"params":{ "m":100, "n":200}, "id":31 }')

print("have json fibonacci response from server: " + response.text)

val = response.text

val = json.loads(val)

print("result: " + str(val["result"]))

print("id: " + str(val["id"]))

print("fibonacci number is: " + str(val["result"]["result"]))

except Exception as error:

print(error)

This code instantiates an RPC client, connects to the RPC server at 127.0.0.24:8081, and then sends two synchronous requests encoded in the JSON-RPC protocol to the server.

Place this code in a file named rpc_client.py and save it in the tmp directory. With the RPC server listening, open a new terminal session, traverse to the tmp directory, and execute this client in your virtual environment with

$(virtual) python rpc_client.py

You should see the following terminal output :

RPC client is ready

have json fibonacci response from server: {"jsonrpc": "2.0", "result": {"method": "fibonacci", "result": 377, "error": "none"}, "id": 31}

result: {'method': 'fibonacci', 'result': 377, 'error': 'none'}

id: 31

fibonacci number is: 377

The RPC client can also be executed in asynchronous mode. The client RPC program code for this is

import json

from tornado import ioloop

from jsonrpcclient.clients.tornado_client import TornadoClient

try:

client = TornadoClient("http://127.0.0.24:8081/")

async def main():

response = await client.send('{"jsonrpc": "2.0", "method": "fibonacci",

"params":{ "m":100, "n":200}, "id":41 }')

print("have json fibonacci response from server: " + response.text)

ioloop.IOLoop.current().run_sync(main)

except Exception as error :

print(error)

Appendix 7: Simulated Blockchain Automation

The following program code implements a simulated Helium cryptocurrency blockchain that can be automatically created by a user. The user needs to only specify the desired height of the blockchain. This simulated blockchain makes full use of all of the Helium cryptocurrency modules. Furthermore, this synthetic blockchain is created using randomized values.

A simulated blockchain is useful for a number of purposes, including

1.
Creating integration tests
2.
Examining how blocks in a blockchain relate to each other
3.
Experimenting with blockchains that have different characteristics

In Chapter 15, a simulated blockchain is used by the Pytest code to test the functionality of the Helium network.

Simulated Blockchain Code Walkthrough

The simulated blockchain constructor requires access to all of the Helium modules including blk_index, hchaindb, hmining, hconfig, hblockchain, rcrypt, and tx. The Chainstate and blk_index databases must be in a running state. The startup function starts the Chainstate and blk_index databases.

Note

The final versions of the Helium modules blk_index, hchaindb, hmining, hconfig, hblockchain, rcrypt, and tx are required. See Appendix 10.

The following program code creates a simulated blockchain with 500 blocks.

The function make_blocks creates a blockchain. make_blocks receives a parameter that specifies the height of the blockchain. This synthetic blockchain simulates the actions of an arbitrary number of users who are conducting transactions on the network. The make_blocks function calls the make_random_transaction function to create Helium transactions with randomized values. Transactions and blocks are passed through the Helium modules for validation. Valid blocks are attached to the simulated blockchain. The Chainstate and blk_index databases are updated as required.

It must be noted that as transactions occur over time, the proportion of transaction fragments with very small helium amounts will increase. In an ordinary wallet, the wallet holder can consolidate these small values into one transaction fragment. The simulated blockchain does not do this. The effect of this fragmentation is to cause the blockchain to fail when the desired block height is large. This failure is strictly a consequence of the lack of a function to consolidate transaction fragments with very small helium outputs. I would invite you to write such a consolidation function; this exercise will provide you with a good exposure to blockchain mechanics.

I have used simulated blockchains with a height of 500 in testing.

Copy the following program code into a file called simulated_blockchain.py and save it in the unit_tests directory. Traverse to the unit_tests directory and execute the following command in order to build a simulated blockchain with a height of 500:

$(virtual) python simulated_blockchain.py

######################################################

# simulated blockchain constructor

######################################################

import blk_index as blkindex

import hchaindb

import hmining

import hconfig

import hblockchain

import rcrypt

import tx

import json

import logging

import pdb

import os

import secrets

import sys

import time

"""

log debugging messages to the file debug.log

"""

logging.basicConfig(filename="debug.log",filemode="w",

format='client: %(asctime)s:%(levelname)s:%(message)s', level=logging.DEBUG)

def startup():

"""

start the databases

"""

# start the Chainstate Database

ret = hchaindb.open_hchainstate("heliumdb")

if ret == False: return "error: failed to start Chainstate database"

else: print("Chainstate Database running")

# start the LevelDB Database blk_index

ret = blkindex.open_blk_index("hblk_index")

if ret == False: return "error: failed to start blk_index"

else: print("blkindex Database running")

def stop():

"""

stop the databases

"""

hchaindb.close_hchainstate()

blkindex.close_blk_index()

# unspent transaction fragment values [{fragmentid:value}]

unspent_fragments = []

######################################################

# Make A Synthetic Random Transaction For Testing

# receives a block no and a predicate indicating

# whether the transaction is a coinbase transaction

######################################################

def make_random_transaction(blockno, is_coinbase):

txn = {}

txn["version"] = "1"

txn["transactionid"] = rcrypt.make_uuid()

if is_coinbase == True: txn["locktime"] = hconfig.conf["COINBASE_LOCKTIME"]

else: txn["locktime"] = 0

# the public-private key pair for this transaction

transaction_keys = rcrypt.make_ecc_keys()

# previous transaction fragments spent by this transaction

total_spendable = 0

#######################

# Build the vin array

#######################

txn["vin"] = []

# genesis block transactions have no prior inputs.

# coinbase transactions do not have any inputs

if (blockno > 0) and (is_coinbase != True):

max_inputs = secrets.randbelow(hconfig.conf["MAX_INPUTS"])

if max_inputs == 0: max_inputs = hconfig.conf["MAX_INPUTS"] - 1

# get some random previous unspent transaction

# fragments to spend

ind = 0

ctr = 0

while ind < max_inputs:

# get a random unspent fragment from a previous block

index = secrets.randbelow(len(unspent_fragments))

frag_dict = unspent_fragments[index]

key = [*frag_dict.keys()][0]

val = [*frag_dict.values()][0]

if val["blockno"] == blockno:

ctr += 1

if ctr == 10000:

print("failed to get random unspent fragment")

return False

continue

unspent_fragments.pop(index)

total_spendable += val["value"]

tmp = hchaindb.get_transaction(key)

if tmp == False:

print("cannot get fragment from chainstate: " + key)

assert tmp != False

assert tmp["spent"] == False

assert tmp["value"] > 0

# create a random vin element

key_array = key.split("_")

signed = rcrypt.sign_message(val["privkey"], val["pubkey"])

ScriptSig = []

ScriptSig.append(signed)

ScriptSig.append(val["pubkey"])

txn["vin"].append({

"txid": key_array[0],

"vout_index": int(key_array[1]),

"ScriptSig": ScriptSig

})

ctr = 0

ind += 1

#####################

# Build Vout list

#####################

txn["vout"] = []

# genesis block

if blockno == 0:

total_spendable = secrets.randbelow(10_000_000) + 50_000

# we need at least one transaction output for non-coinbase

# transactions

if is_coinbase == True: max_outputs = 1

else:

max_outputs = secrets.randbelow(hconfig.conf["MAX_OUTPUTS"])

if max_outputs == 0: max_outputs = 6

ind = 0

while ind < max_outputs:

tmp = rcrypt.make_SHA256_hash(transaction_keys[1])

tmp = rcrypt.make_RIPEMD160_hash(tmp)

ScriptPubKey = []

ScriptPubKey.append("<DUP>")

ScriptPubKey.append("<HASH-160>")

ScriptPubKey.append(tmp)

ScriptPubKey.append("<EQ-VERIFY>")

ScriptPubKey.append("<CHECK-SIG>")

if is_coinbase == True:

value = hmining.mining_reward(blockno)

else:

amt = int(total_spendable/max_outputs)

value = secrets.randbelow(amt) # helium cents

if value == 0:

pdb.set_trace()

value = int(amt / 10)

total_spendable -= value

assert value > 0

assert total_spendable >= 0

txn["vout"].append({

"value": value,

"ScriptPubKey": ScriptPubKey

})

# save the transaction fragment

fragid = txn["transactionid"] + "_" + str(ind)

fragment = {}

fragment[fragid] = { "value":value,

"privkey":transaction_keys[0],

"pubkey":transaction_keys[1],

"blockno": blockno

}

unspent_fragments.append(fragment)

print("added to unspent fragments: " + fragid)

if total_spendable <= 0: break

ind += 1

return txn

########################################################

# Build some random Synthetic Blocks For Testing.

# Makes num_blocks, sequentially linking each

# block to the previous block through the prevblockhash

# attribute.

# Returns an array of synthetic blocks

#########################################################

def make_blocks(num_blocks):

ctr = 0

blocks = []

global unspent_fragments

unspent_fragments.clear()

hblockchain.blockchain.clear()

while ctr < num_blocks:

block = {

"prevblockhash": "",

"version": "1",

"timestamp": int(time.time()),

"difficulty_bits": 20,

"nonce": 0,

"merkle_root": "",

"height": ctr,

"tx": []

}

# make a random number of transactions for this block

# genesis block is ctr == 0

if ctr == 0: num_transactions = 200

else:

num_transactions = secrets.randbelow(50)

if num_transactions == 0: num_transactions = 40

num_transactions = 2

txctr = 0

while txctr < num_transactions:

if ctr > 0 and txctr == 0: is_coinbase = True

else: is_coinbase = False

trx = make_random_transaction(ctr, is_coinbase)

assert trx != False

block["tx"].append(trx)

txctr += 1

if ctr > 0:

block["prevblockhash"] =

hblockchain.blockheader_hash(hblockchain.blockchain[ctr - 1])

ret = hblockchain.merkle_root(block["tx"], True)

assert ret != False

block["merkle_root"] = ret

ret = hblockchain.add_block(block)

assert ret == True

blocks.append(block)

ctr+= 1

return blocks

###################################################

# make the simulated blockchain

###################################################

startup()

make_blocks(500)

print("finished synthetic blockchain construction")

stop()

Appendix 8: Helium Data Structures

In this appendix, we summarize the main data structures of the Helium cryptocurrency. Tokens in angle brackets refer to data types. Brackets as well as the token list denote a list.

Blockchain Structures

Primary Blockchain

list<block>

Secondary Blockchain

list<block>

Block

{

"prevblockhash": <string>

"version": <string>

"timestamp": <integer>

"difficulty_bits": <integer>

"nonce": <integer>

"merkle_root": <string>

"height": <integer>

"tx": <list>

}

Block Files

Block filename = "block_" + str(block Height) + ".dat"

Block heights start at 0. The genesis block has height 0.

Transaction Data Structures

Transactions

{

"transactionid": <string>

"version": <integer>

"locktime": <integer>

"vin": list<vin>

"vout": list<vout>

}

VIN

{

"txid": <string>

"vout_index": <integer>

"ScriptSig": list<string>

}

VOUT

{

"value": <integer>,

"ScriptPubKey": list<string>

}

ScriptSig

["signature", "public key"]

ScriptPubkey

["<DUP>", "<HASH-160>", PK_HASH, "<EQ_VERIFY>", "<CHECK-SIG>"]

Mining

Address List

list<remote node address >

Mempool

list<transactions>

Received Blocks

list<blocks>

Orphan Blocks

list<blocks>

Appendix 9: Helium Database Maintenance

This appendix contains Python files that rebuild the Chainstate and the blk_index databases from the Helium blockchain files. Before any of these maintenance functions is invoked, ensure that the corresponding database directory is deleted.

Chainstate Maintenance

Save the program code , infra, to the file build_chainstate.py.

Rebuild the Chainstate database as follows. Firstly, ensure that you are in your active Helium virtual environment. Traverse to the directory which will contain the database. Remove the chainstate directory, heliumdb, if it exists. Execute the Chainstate maintenance file:

$(virtual) python build_chainstate.py

###########################################################################

# build_chainstate.py: Builds the chainstate database from the blockchain files.

# Ensure that the directory containing the Chainstate is deleted.

###########################################################################

import hblockchain

import hchaindb

import pickle

import os.path

import pdb

def startup():

"""

start the chainstate database

"""

# start the Chainstate Database

ret = hchaindb.open_hchainstate("heliumdb")

if ret == False:

print("error: failed to start Chainstate database")

return

else: print("Chainstate Database running")

def stop():

"""

stop the chainstate database

"""

hchaindb.close_hchainstate()

def build_chainstate():

'''

build the Chainstate database from the blockchain files

'''

blockno = 0

tx_count = 0

try:

while True:

# test whether the block file exists

block_file = "block" + "_" + str(blockno) + ".dat"

if os.path.isfile(block_file)== False: break

# load the block file into a dictionary object

f = open(block_file, 'rb')

block = pickle.load(f)

# process the vout and vin arrays for each block transaction

for trx in block["tx"]:

ret = hchaindb.transaction_update(trx)

tx_count += 1

if ret == False:

raise(ValueError("failed to rebuild chainstate. block no: "

+ str(blockno)))

blockno += 1

except Exception as err:

print(str(err))

return False

print("transactions processed: " + str(tx_count))

print("blocks processed: " + str(blockno))

return True

################################

# start the chainstate build

################################

print("start build chainstate")

startup()

build_chainstate()

print("chainstate rebuilt")

stop()

blk_index Maintenance

Save the program code , infra, to the file build_blk_index.py.

Rebuild the hblk_index database as follows. Firstly, make sure that you are in your active Helium virtual environment. Traverse to the directory which will contain this database. Remove the hblk_index directory, if it exists. Execute the blk_index maintenance file:

$(virtual) python build_blk_index.py

#########################################################################

# build_blk_index: rebuilds the blk_index database from the blockchain

# Ensure that the directory containing this database is deleted first.

#########################################################################

import blk_index

import pickle

import os.path

import pdb

def startup():

"""

start/create the hblk_index database

"""

# start the hblk_index Database

ret = blk_index.open_blk_index("hblk_index")

if ret == False:

print("error: failed to start blk_index database")

return

else: print("blk_index Database running")

return True

def stop():

"""

stop the blk_index database

"""

blk_index.close_blk_index()

def build_blk_index():

'''

build the blk_index database from the blockchain

'''

blockno = 0

tx_count = 0

try:

while True:

# test whether the block file exists

block_file = "block" + "_" + str(blockno) + ".dat"

if os.path.isfile(block_file)== False: break

# load the block file into a dictionary object

f = open(block_file, 'rb')

block = pickle.load(f)

# process the transactions in the block

for trx in block["tx"]:

ret = blk_index.put_index(trx["transactionid"], blockno)

tx_count += 1

if ret == False:

raise(ValueError("failed to rebuild blk_index. block no: "

+ str(blockno)))

blockno += 1

except Exception as err:

print(str(err))

return False

print("transactions processed: " + str(tx_count))

print("blocks processed: " + str(blockno))

return True

###########################################

# start the hblk_index database rebuild

###########################################

print("start build hblk_index")

ret = startup()

if ret == False:

print("failed to open hblk_index database")

os._exit(-1)

build_blk_index()

print("hblk_index database rebuilt")

stop()

Appendix 10: Helium Source Code Listing

This appendix contains all of the source code pertaining to the Helium cryptocurrency.

Helium Configuration Module

"""

hconfig.py: module contains parameters that are used to configure Helium .

"""

conf = {

# The Helium version no.

'VERSION_NO': "1",

# The maximum number of Helium coins that can be mined

'MAX_HELIUM_COINS': 21000000,

# The smallest Helium currency unit

'HELIUM_CENT': 1/100000000,

# The maximum size of a Helium block in bytes

'MAX_BLOCK_SIZE': 1000000,

# The maximum amount of time in seconds that a transaction can be locked

'MAX_LOCKTIME': 30*1440*60,

# The maximum number of Inputs in a Helium Transaction

'MAX_INPUTS': 10,

# The maximum number of Outputs in a Helium Transaction

'MAX_OUTPUTS': 10,

# The number of new blocks from a reference block that must be mined before

# coinbase transaction in the previous reference block can be spent

'COINBASE_INTERVAL': 100,

# The number of blocks for which a coinbase transaction is locked

'COINBASE_LOCKTIME': 36,

# The starting nonce value for the mining proof of work computations

'NONCE': 0,

# Difficulty Number used in mining proof of work computations

'DIFFICULTY_BITS': 20,

'DIFFICULTY_NUMBER': 1/ (10 **20),

# Retargeting interval in blocks in order to adjust the DIFFICULTY_NUMBER

'RETARGET_INTERVAL': 1000,

#Mining Reward

'MINING_REWARD': 5_000_000,

# Mining reward halving interval in blocks

'REWARD_INTERVAL': 210000

}

The rcrypt Module

"""

The rcrypt module implements various cryptographic functions that are required by

the Helium cryptocurrency application

This module requires the pycryptodome package to be installed.

The base58 package encodes strings into base58 format.

This module uses Python's regular expression module re.

This module uses the secrets module in the Python standard library to generate

cryptographically secure hexadecimal encoded strings.

"""

# import the regular expressions module

import re

# imports from the cryptodome package

from Crypto.Hash import SHA3_256

from Crypto.PublicKey import ECC

from Crypto.PublicKey import DSA

from Crypto.Signature import DSS

from Crypto.Hash import SHA256

from Crypto.Hash import RIPEMD160

import base58

import secrets

# import Python's debugging and logging modules

import pdb

import logging

"""

log debugging messages to the file debug.log

"""

logging.basicConfig(filename="debug.log",filemode="w",

format='%(asctime)s:%(levelname)s:%(message)s', level=logging.DEBUG)

def make_SHA256_hash(msg: 'string') -> 'string':

"""

make_sha256_hash computes the SHA-256 message digest or cryptographic hash

for a received string argument. The secure hash value that is generated is converted

into a sequence of hexadecimal digits and then returned by the function.

The hexadecimal format of the message digest is 64 bytes long .

"""

#convert the received msg string to a sequence of ascii bytes

message = bytes(msg, 'ascii')

# compute the SHA-256 message digest of msg and convert to a hexadecimal format

hash_object = SHA256.new()

hash_object.update(message)

return hash_object.hexdigest()

def validate_SHA256_hash(digest: "string") -> bool:

"""

validate_SHA256_hash: tests whether a string has an encoding conforming to a

SHA-256 message digest in hexadecimal string format (64 bytes).

"""

# a hexadecimal SHA256 message digest must be 64 bytes long.

if len(digest) != 64: return False

# This regular expression tests that the received string contains only

# hexadecimal characters

if re.search('[^0-9a-fA-F]', digest) == None: return True

return False

def make_RIPEMD160_hash(message: 'byte stream') -> 'string':

"""

RIPEMD-160 is a cryptographic algorithm that emits a 20 byte message digest.

This function computes the RIPEMD-160 message digest of a message and returns

the hexadecimal string encoded representation of the message digest (40 bytes).

"""

# convert message to an ascii byte stream

bstr = bytes(message, 'ascii')

# generate the RIPEMD hash of message

h = RIPEMD160.new()

h.update(bstr)

# convert to a hexadecimal encoded string

hash = h.hexdigest()

return hash

def validate_RIPEMD160_hash(digest: 'string') -> 'bool':

"""

tests that a received string has an encoding conforming to a RIPE160 hash in

hexadecimal format

"""

if len(digest) != 40: return False

# This regular expression tests that received string only contains

# hexadecimal characters

if re.search('[^0-9a-fA-F]+', digest) == None: return True

return False

def make_ecc_keys():

"""

make a private-public key pair using the elliptic curve cryptographic

functions in the pycryptodome package.

returns a tuple with the private key and public key in PEM format

"""

# generate an ecc object

ecc_key = ECC.generate(curve='P-256')

# get the public key object

pk_object = ecc_key.public_key()

# export the private-public key pair in PEM format

p = (ecc_key.export_key(format='PEM'), pk_object.export_key(format='PEM'))

return p

def sign_message(private_key: 'String', message: 'String') -> 'string':

"""

digitally signs a message using a private key generated using the

elliptic cryptography module of the pycryptodome package.

Receives a private key in PEM format and the message that is to be

digitally signed.

returns a hex encoded signature string.

"""

# import the PEM format private key

priv_key = ECC.import_key(private_key)

# convert the message to a byte stream and

# compute the SHA-256 message digest of the message

bstr = bytes(message, 'ascii')

hash = SHA256.new(bstr)

# create a digital signature object from the private key

signer = DSS.new(priv_key, 'fips-186-3')

# sign the SHA-256 message digest.

signature = signer.sign(hash)

sig = signature.hex()

return sig

def verify_signature(public_key: 'String', msg: 'String', signature: 'string') -> 'bool':

"""

tests whether a message is digitally signed by a private key to which a

public key is paired.

Receives a ECC public key in PEM format, the message that is to to be verified

and the digital signature of the message.

Returns True or False

"""

try:

# convert the message to a byte stream and compute the SHA-256 hash

msg = bytes(msg, 'ascii')

msg_hash = SHA256.new(msg)

# signature to bytes

signature = bytes.fromhex(signature)

# import the PEM formatted public key and create a signature verifier

# object from the public key

pub_key = ECC.import_key(public_key)

verifier = DSS.new(pub_key, 'fips-186-3')

# Verify the authenticity of the signed message

verifier.verify(msg_hash, signature)

return True

except Exception as err:

logging.debug('verify_signature: exception: ' + str(err))

def make_address(prefix: 'string') -> 'string':

"""

generates a Helium address from a ECC public key in PEM format.

prefix is a single numeric character which describes the type of

the address. This prefix must be '1'

"""

key = ECC.generate(curve='P-256')

__private_key = key.export_key(format='PEM')

public_key = key.public_key().export_key(format='PEM')

val = make_SHA256_hash(public_key)

val = make_RIPEMD160_hash(val)

tmp = prefix + val

# make a checksum

checksum = make_SHA256_hash(tmp)

checksum = checksum[len(checksum) - 4:]

# add the checksum to the tmp result

address = tmp + checksum

# encode addr as a base58 sequence of bytes

address = base58.b58encode(address.encode())

# The decode function converts a byte sequence to a string

address = address.decode("ascii")

return address

def validate_address(address: 'string') -> bool:

"""

validates a Helium address using the four character checksum appended

to the address. Receives a base58 encoded address.

"""

# encode the string address as a sequence of bytes

addr = address.encode("ascii")

# reverse the base58 encoding of the address

addr = base58.b58decode(addr)

# convert the address into a string

addr = addr.decode("ascii")

# length must be RIPEMD-160 hash length + length of checksum + 1

if (len(addr) != 45): return False

if (addr[0] != '1'): return False

# extract the checksum

extracted_checksum = addr[len(addr) - 4:]

# extract the checksum out of addr and compute the

# SHA-256 hash of the remaining addr string

tmp = addr[:len(addr)- 4]

tmp = make_SHA256_hash(tmp)

# get the computed checksum from tmp

checksum = tmp[len(tmp) - 4:]

if extracted_checksum == checksum: return True

return False

def make_uuid() -> 'string':

"""

makes an universally unique 256 bit id encoded as a hexadecimal string that is

used as a transaction identifier. Uses the Python standard library secrets module to

generate a cryptographically strong random 32 byte string encoded as a hexadecimal

string (64 bytes)

"""

id = secrets.token_hex(32)

return id

Helium Blockchain Module

"""

hbockchain.py: This module creates and maintains the Helium blockchain

"""

import rcrypt

import hconfig

import hchaindb

import tx

import blk_index as blockindex

import json

import pickle

import pdb

import logging

import os

"""

log debugging messages to the file debug.log

"""

logging.basicConfig(filename="debug.log",filemode="w", format='%(asctime)s:%(levelname)s:%(message)s',

level=logging.DEBUG)

"""

A block is a Python dictionary that has the following

structure. The type of an attribute is denoted in angle delimiters.

{

"prevblockhash": <string>

"version": <string>

"timestamp": <integer>

"difficulty_bits": <integer>

"nonce": <integer>

"merkle_root": <string>

"height": <integer>

"tx": <list>

}

The blockchain is a list where each list element is a block

This is also referred to as the primary blockchain when used

by miners.

"""

blockchain = []

"""

secondary block used by mining nodes

"""

secondary_blockchain = []

def add_block(block: "dictionary") -> "bool":

"""

add_block: adds a block to the blockchain. Receives a block .

The block attributes are checked for validity and each transaction in the block is

tested for validity. If there are no errors, the block is written to a file as a

sequence of raw bytes. Then the block is added to the blockchain.

The chainstate database and the blk_index databases are updated.

returns True if the block is added to the blockchain and False otherwise

"""

try:

# validate the received block parameters

if validate_block(block) == False:

raise(ValueError("block validation error"))

# validate the transactions in the block

# update the chainstate database

for trx in block['tx']:

# first transaction in the block is a coinbase transaction

if block["height"] == 0 or block['tx'][0] == trx: zero_inputs = True

else: zero_inputs = False

if tx.validate_transaction(trx, zero_inputs) == False:

raise(ValueError("transaction validation error"))

if hchaindb.transaction_update(trx) == False:

raise(ValueError("chainstate update transaction error"))

# serialize the block to a file

if (serialize_block(block) == False):

raise(ValueError("serialize block error"))

# add the block to the blockchain in memory

blockchain.append(block)

# update the blk_index

for transaction in block['tx']:

blockindex.put_index(transaction["transactionid"], block["height"])

except Exception as err:

print(str(err))

logging.debug('add_block: exception: ' + str(err))

return False

return True

def serialize_block(block: "dictionary") -> "bool":

"""

serialize_block: serializes a block to a file using pickle.

Returns True if the block is serialized and False otherwise.

"""

index = len(blockchain)

filename = "block_" + str(index) + ".dat"

# create the block file and serialize the block

try:

f = open(filename, 'wb')

pickle.dump(block, f)

except Exception as error:

logging.debug("Exception: %s: %s", "serialize_block", error)

f.close()

return False

f.close()

return True

def read_block(blockno: 'long') -> "dictionary or False":

"""

read_block: receives an index into the Helium blockchain.

Returns a block or False if the block does not exist .

"""

try:

block = blockchain[blockno]

return block

except Exception as error:

logging.debug("Exception: %s: %s", "read_block", error)

return False

return block

def blockheader_hash(block: 'dictionary') -> "False or String":

"""

blockheader_hash: computes and returns SHA-256 message digest of a block header

as a hexadecimal string.

Receives a block those blockheader hash is to be computed.

Returns False if there is an error, otherwise returns a SHA-256 hexadecimal string.

The block header consists of the following block fields:

(1) version, (2)previous block hash, (3) merkle root

(4) timestamp, (5) difficulty_no, and (6) nonce.

"""

try :

hash = rcrypt.make_SHA256_hash(block['version'] + block['prevblockhash'] +

block['merkle_root'] + str(block['timestamp']) +

str(block['difficulty_bits']) + str(block['nonce']))

except Exception as error :

logging.debug("Exception:%s: %s", "blockheader_hash", error)

return False

return hash

def validate_block(block: "dictionary") -> "bool":

"""

validate_block: receives a block and verifies that all its attributes have

valid values.

Returns True if the block is valid and False otherwise.

"""

try:

if type(block) != dict:

raise(ValueError("block type error"))

# validate scalar block attributes

if type(block["version"]) != str:

raise(ValueError("block version type error"))

if block["version"] != hconfig.conf["VERSION_NO"]:

raise(ValueError("block wrong version"))

if type(block["timestamp"]) != int:

raise(ValueError("block timestamp type error"))

if block["timestamp"] < 0:

raise(ValueError("block invalid timestamp"))

if type(block["difficulty_bits"]) != int:

raise(ValueError("block difficulty_bits type error"))

if block["difficulty_bits"] <= 0:

raise(ValueError("block difficulty_bits <= 0"))

if type(block["nonce"]) != int:

raise(ValueError("block nonce type error"))

if block["nonce"] != hconfig.conf["NONCE"]:

raise(ValueError("block nonce is invalid"))

if type(block["height"]) != int:

raise(ValueError("block height type error"))

if block["height"] < 0:

raise(ValueError("block height < 0"))

if len(blockchain) == 0 and block["height"] != 0:

raise(ValueError("genesis block invalid height"))

if len(blockchain) > 0:

if block["height"] != blockchain[-1]["height"] + 1:

raise(ValueError("block height is not in order"))

# The length of the block must be less than the maximum block size that

# specified in the config module.

# json.dumps converts the block into a json format string.

if len(json.dumps(block)) > hconfig.conf["MAX_BLOCK_SIZE"]:

raise(ValueError("block length error"))

# validate the previous block by comparing message digests .

# the genesis block does not have a predecessor block

if block["merkle_root"] != merkle_root(block["tx"], True):

raise(ValueError("merkle roots do not match"))

if block["height"] > 0:

if block["prevblockhash"] != blockheader_hash(blockchain[-1]):

raise(ValueError("previous block header hash does not match"))

else:

if block["prevblockhash"] != "":

raise(ValueError("genesis block has prevblockhash"))

# genesis block does not have any input transactions

if block["height"] == 0 and block["tx"][0]["vin"] !=[]:

raise(ValueError("missing coinbase transaction"))

# a block other than the genesis block must have at least

# two transactions: the coinbase transaction and at least

# one more transaction

if block["height"] > 0 and len(block["tx"]) < 2:

raise(ValueError("block only has one transaction"))

except Exception as error:

logging.error("exception: %s: %s", "validate_block",error)

return False

return True

def merkle_root(buffer: "List", start: "bool" = False) -> "bool or string":

"""

merkle_tree: computes the merkle root for a list of transactions .

Receives a list of transactions and a boolean flag to indicate whether

the function has been called for the first time or whether it is a

recursive call from within the function.

Returns the root of the merkle tree or False if there is an error.

"""

try :

# if start is False then we verify have a list of SHA-256 hashes

if start == False:

for value in buffer:

if rcrypt.validate_SHA256_hash(value) == False:

raise(ValueError("tx list SHA-256 validation failure"))

buflen = len(buffer)

if buflen != 1 and len(buffer)%2 != 0:

buffer.append(buffer[-1])

# make the merkle leaf nodes if we are entering the function

# for the first time

if start == True:

tmp = buffer[:]

tmp = make_leaf_nodes(tmp)

if tmp == False: return False

buffer = tmp[:]

# if buffer has one element, we have the merkle root

if (len(buffer) == 1):

return buffer[0]

# construct the list of parent nodes from the child nodes

index = 0

parents = []

while index < len(buffer):

tmp = rcrypt.make_SHA256_hash(buffer[index] + buffer[index+1])

parents.append(tmp)

index += 2

# recursively call merkle tree

ret = merkle_root(parents, False)

except Exception as error:

logging.error("exception: %s: %s", "merkle_root",error)

return False

return ret

def make_leaf_nodes(tx_list: "list") -> "List or False":

"""

make_leaf_nodes: makes the leaf nodes of a merkle tree.

Receives a list of transactions. If the number of transactions is an

odd number, appends the last transaction to the list again so that we

can make a balanced tree.

Computes the hexadecimal string encoded SHA-256 message digest of each

transaction and appends it to a leaf node list that is initially empty.

Returns the leaf node list or False if there is an error

"""

try:

# cannot have an empty transaction list

if (len(tx_list) == 0): raise(ValueError("tx list zero length"))

# verify we have a list type

if type(tx_list) is not list: raise(ValueError("tx is not a list type"))

# verify the type of each transaction is a dict

for tx in tx_list:

if type(tx) is not dict: raise(ValueError("tx is not a dict type"))

# must have an even number of transactions

if len(tx_list) % 2 == 1 or len(tx_list) == 1:

tx_list.append(tx_list[- 1] )

# copy the transaction list

trx_list = list(tx_list)

# convert each transaction into a JSON string

tx = []

for transaction in trx_list:

tx.append(json.dumps(transaction))

# make the leaf nodes

sha256_list = []

for transaction in tx:

sha256_list.append(rcrypt.make_SHA256_hash(transaction))

except Exception as err:

logging.debug('make_leaf_nodes: exception: ' + str(err))

return False

return sha256_list

Helium tx Module

"""

The tx module defines the structure of Helium transactions and implements basic

transaction operations.

"""

import hconfig

import hblockchain as hchain

import hchaindb

import json

import rcrypt

import secrets

import pdb

import logging

"""

log debugging messages to the file debug.log

"""

logging.basicConfig(filename="debug.log",filemode="w",

format='%(asctime)s:%(levelname)s:%(message)s', level=logging.DEBUG)

"""

Each transaction is modeled as a dictionary :

{

"transactionid": <string>

"version": <integer>

"locktime": <integer>

"vin": <list<dictionary>>

"vout": <list<<dictionary>>

}

transactionid is the id of the the present transaction.

vin is a list of spendable inputs that are owned by the entity spending them.

Each spendable input comes from the vout list of a previous transaction.

Each vin is a list and each element is a dictionary object with the

following structure:

vin element:

{

"txid": <string>

"vout_index": <int>

"ScriptSig": <list<string>>

}

txid is the transaction id of a previous transaction. vout_index is an index

into this transaction's vout list .

vout is a list and each element in the vout list is a dictionary with the

following structure:

vout element:

{

"value": <integer>,

"ScriptPubKey": <list<string>>

}

"""

def create_transaction(transaction: 'dictionary', zero_inputs: 'boolean'=False)

-> 'bool':

"""

creates a transaction. Receives a transaction object and a predicate.

zero_inputs is True if the transaction is in the genesis block or if the transaction

is a coinbase transaction, otherwise zero_inputs is False.

Returns False if the transaction parameters are invalid. Otherwise returns True

"""

if validate_transaction(transaction, zero_inputs) == False: return False

return True

def validate_transaction(trans: "dictionary", zero_inputs: "boolean"=False) -> "bool":

"""

verifies that a transaction has valid values.

receives a transaction and a predicate.

zero_inputs is True if the transaction is in the genesis block or if the transaction

is a coinbase transaction, otherwise zero_inputs is False.

The following transaction validation tests are performed:

(1) The required attributes are present.

(2) The locktime is greater than or equal to zero.

(3) The version number is greater than zero.

(4) The transaction ID has a valid format.

(5) The vin list have positive length.

(6) The vin list is not greater than the maximum allowable length .

(7) The vin list has valid format.

(8) The vout list has positive length.

(9) The vout list is not greater than the maximum allowable length.

(8) The vout list elements have valid format.

(9) The The vout list implements a p2pkhash script.

(10) The reference to a previous transaction ID is valid.

(11) The total value spent is less than or equal to the total spendable value.

(12) The spent values are greater than zero.

(13) The index values in the vin array reference valid elements in the

vout array of the previous transaction.

(14) The transaction inputs can be spent

(15) The genesis block does not have any inputs

"""

try:

if type(trans) != dict:

raise(ValueError("not dict type"))

# Verify that required attributes are present

if trans.get("transactionid") == None: return False

if 'version' not in trans: return False

if trans['locktime'] == None: return False

if trans['vin'] == None: return False

if trans['vout'] == None: return False

# validate the format of the transaction id

if rcrypt.validate_SHA256_hash(trans['transactionid']) == False :

raise(ValueError("not SHA-256 hash error"))

# validate the transaction version and locktime values

if trans['version'] != hconfig.conf["VERSION_NO"]:

raise(ValueError("improper version no"))

if trans['locktime'] < 0:

raise(ValueError("invalid locktime"))

# genesis block or a coinbase transaction do not have any inputs

if zero_inputs == True and len(trans["vin"]) > 0:

raise(ValueError("genesis block cannot have inputs"))

# validate the vin elements

# there are no vin inputs for a genesis block transaction

spendable_fragments = []

for vin_element in trans['vin']:

if validate_vin(vin_element) == False: return False

tx_key = vin_element['txid'] + '_' + str(vin_element['vout_index'])

spendable_fragment = prevtx_value(tx_key)

if spendable_fragment == False:

raise(ValueError("invalid spendable input for transaction"))

else:

spendable_fragments.append(spendable_fragment)

# validate the transaction's vout list

if len(trans['vout']) > hconfig.conf['MAX_OUTPUTS'] or len(trans['vout']) <= 0:

raise(ValueError("vout list length error"))

for vout_element in trans['vout']:

if validate_vout(vout_element) == False: return False

# validate the transaction fee

if zero_inputs == False:

if (transaction_fee(trans, spendable_fragments)) == False: return False

# test that the transaction inputs are unlocked

ctr = 0

for vin in trans['vin']:

if unlock_transaction_fragment(vin, spendable_fragments[ctr]) == False:

raise(ValueError("failed to unlock transaction"))

ctr += 1

except Exception as err:

logging.debug('validate_transaction: exception: ' + str(err))

return False

return True

def validate_vin(vin_element: 'dictionary') -> bool:

"""

tests whether a vin element has valid values

returns True if the vin is valid, False otherwise

"""

try:

if vin_element['vout_index'] < 0: return False

if len(vin_element['ScriptSig']) != 2: return False

if len(vin_element['ScriptSig'][0]) == 0: return False

if len(vin_element['ScriptSig'][1]) == 0: return False

if rcrypt.validate_SHA256_hash(vin_element['txid']) == False:

raise(ValueError("txid is invalid"))

except Exception as err :

print("validate_vin exception" + str(err))

logging.debug('validate_vin: exception: ' + str(err))

return False

return True

def validate_vout(vout_element: 'dictionary') -> bool:

"""

tests whether a vout element has valid values

returns True if the vout is valid, False otherwise

"""

try:

if vout_element['value'] <= 0:

raise(ValueError("value is <= 0"))

# validate the p2pkhash script

if len(vout_element["ScriptPubKey"]) != 5:

raise(ValueError("ScriptPubKey length error"))

if vout_element["ScriptPubKey"][0] != '<DUP>':

raise(ValueError("ScriptPubKey <DUP> error"))

if vout_element["ScriptPubKey"][1] != '<HASH-160>':

raise(ValueError("ScriptPubKey <HASH-160> error"))

if vout_element["ScriptPubKey"][3] != '<EQ-VERIFY>':

raise(ValueError("ScriptPubKey <EQ-VERIFY> error"))

if vout_element["ScriptPubKey"][4] != '<CHECK-SIG>':

raise(ValueError("ScriptPubKey <CHECK-SIG> error"))

if len(vout_element["ScriptPubKey"][2]) == 0:

raise(ValueError("ScriptPubKey RIPEMD-160 hash error"))

except Exception as err :

print("validate_vout exception" + str(err))

logging.debug('validate_vout: exception: ' + str(err))

return False

return True

def prevtx_value(txkey: "string") -> 'bool':

"""

examines the chainstate database and returns the transaction fragment

corresponding to txkey:

{

"pkkhash": <string>,

"value": <int>,

"spent": <bool>,

"tx_chain": <string>

}

receives a transaction fragment key:

"previous txid" + "_" + str(prevtx_vout_index)

Returns False if the transaction if does not exist, or if the

value has been spent or if value is not a positive integer

otherwise returns the previous transaction fragment data

"""

try:

fragment = hchaindb.get_transaction(txkey)

if fragment == False:

raise(ValueError("cannot get fragment from chainstate"))

# return False if the value has been spent or if the value is

# not positive

if fragment["spent"] == True:

raise(ValueError("cannot respend fragment in chainstate: " + txkey))

if fragment["value"] <= 0:

raise(ValueError("fragment value <= 0"))

except Exception as err:

logging.debug('prevtx_value: exception: ' + str(err))

return False

return fragment

def transaction_fee(trans: 'dictionary', value_list: 'list<dictionary>' )

-> 'integer or bool':

"""

calculates the transaction fee for a transaction.

Receives a transaction and a list of chainstate transaction fragments

Returns the transaction fee (in helium_cents) or False if there is an error.

There is no transaction fee for the genesis block or coinbase transactions.

"""

try:

spendable_value = 0

spent_value = 0

for val in value_list:

spendable_value += val["value"]

for vout in trans['vout']:

spent_value += vout['value']

if spendable_value <= 0:

raise(ValueError("spendable value <= 0 "))

if spent_value <= 0:

raise(ValueError("spent value <= 0 "))

if spendable_value < spent_value:

raise(ValueError("spendable value < spent value"))

except Exception as err :

logging.debug('transaction_fee: exception: ' + str(err))

return False

return spendable_value - spent_value

def unlock_transaction_fragment(vinel: "dictionary", fragment: "dictionary")

-> "boolean":

"""

unlocks a previous transaction fragment using the p2pkhash script .

Executes the script and returns False if the transaction is not unlocked

Receives: the consuming vin and the previous transaction fragment consumed

by the vin element.

"""

try:

execution_stack = []

result_stack = []

# make the execution stack for a p2pkhash script

# since we only have one type of script in Helium

# we can use hard-coded values

execution_stack.append('SIG')

execution_stack.append('PUBKEY')

execution_stack.append('<DUP>')

execution_stack.append('<HASH_160>')

execution_stack.append('HASH-160')

execution_stack.append('<EQ-VERIFY>')

execution_stack.append('<CHECK_SIG>')

# Run the p2pkhash execution stack

result_stack.insert(0, vinel['ScriptSig'][0])

result_stack.insert(0, vinel['ScriptSig'][1])

hash_160 = rcrypt.make_SHA256_hash(vinel['ScriptSig'][1])

hash_160 = rcrypt.make_RIPEMD160_hash(hash_160)

result_stack.insert(0, hash_160)

result_stack.insert(0, fragment["pkhash"])

# do the EQ_VERIFY operation

tmp1 = result_stack.pop(0)

tmp2 = result_stack.pop(0)

# test for RIPEMD-160 hash match

if tmp1 != tmp2:

raise(ValueError("public key match failure"))

# test for a signature match

ret = rcrypt.verify_signature(vinel['ScriptSig'][1], vinel['ScriptSig'][1],

vinel['ScriptSig'][0])

if ret == False:

raise(ValueError("signature match failure"))

except Exception as err:

logging.debug('unlock_transaction_fragment: exception: ' + str(err))

return False

return True

Helium hchaindb Module

"""

Implementation of the Helium chainstate key-value store . This store lets us

access information about transaction fragments through a transaction key.

"""

import hconfig

import rcrypt

import plyvel

import logging

import json

import pdb

"""

log debugging messages to the file debug.log

"""

logging.basicConfig(filename="debug.log",filemode="w",

format='%(asctime)s:%(levelname)s:%(message)s', level=logging.DEBUG)

# handle to the Helium Chainstate Database

hDB = None

def open_hchainstate(filepath: "string") -> "db handle or False":

"""

opens the Helium Chainstate key-value store and returns a handle to

it. The database will be created if it does not exist. All of the

directories in filepath must exist.

Returns a handle to the database or False

"""

try:

global hDB

hDB = plyvel.DB(filepath, create_if_missing=True)

except Exception as err:

logging.debug('open_hchainstate: exception: ' + str(err))

return False

return hDB

def close_hchainstate() -> "bool":

"""

close the Helium Chainstate store

Returns True if the database is closed, False otherwise

"""

global hDB

try:

hDB.close()

if hDB.closed != True: return False

except Exception as err:

logging.debug('close_hchainstate: exception: ' + str(err))

return False

return True

def put_transaction(txkey: "string", tx_fragment: "dictionary") -> "bool":

"""

creates a key-value pair in the Chainstate Database

Receives a transaction key and a transaction_fragment.

The transaction key is: transactionid + "_" + str(vout_index)

The tx_fragment parameter received is:

{

"pkhash": <string>

"value": <int

"spent": <bool>

"tx_chain" <string>

}

Returns True if the key-value pair is created and False otherwise

"""

try:

# if transaction key already exists delete because

# the transaction fragment is going to be updated

encoded_key = str.encode(txkey)

hDB.delete(encoded_key)

keyvalue = json.dumps(tx_fragment)

# save the txkey-fragment pair to the store

hDB.put(encoded_key, str.encode(keyvalue))

except Exception as err:

print(str(err))

logging.debug('put_transaction: exception: ' + str(err))

return False

return True

def get_transaction(key: "string") -> "False or dictionary":

"""

Receives a transaction key (transactionid + "_" + str(vout_index))

for a transaction fragment

returns False or the key value. The transaction fragment returned has

form:

{

"pkhash": <string>

"value": <string>

"spent": <string>

"tx_chain" <string>

}

"""

try:

# get the transaction fragment corresponding to the transaction

# key, return False if the key does not exist

fragment = hDB.get(str.encode(key))

if fragment == None:

raise(ValueError("transaction fragment not found"))

fragment = json.loads(fragment.decode())

except Exception as err:

logging.debug('get_transaction: exception: ' + str(err))

return False

return fragment

def transaction_update(trx: "transaction")-> "bool":

"""

receives a transaction. Updates the chainstate database to

reflect the transaction. Sets previous transaction fragments

to indicate that they have been spent. Updates these fragments

to indicate the transaction id of the consuming transaction .

returns True or False if there is an error

"""

try:

# collect all of the outputs of previous transactions and set them to spent

# specify the transaction key consuming the previous transaction inputs

for vin in trx["vin"]:

# fetch a previous transaction fragment. It must exist

prev_tx_key = vin["txid"] + "_" + str(vin["vout_index"])

tx_fragment = get_transaction(prev_tx_key)

if tx_fragment == False:

print("vin fragment not found: " + prev_tx_key)

raise(ValueError("transaction fragment not found"))

# double spend error

# should be detected prior to writing to the Chainstate

if tx_fragment["spent"] == True:

print("fragment is double spend error: " + prev_tx_key)

raise(ValueError("transaction fragment double spend error: " +

prev_tx_key))

# set the spent values to True in the previous transaction

# fragment

tx_fragment["spent"] = True

# set the reference to the consuming transaction

tx_fragment["tx_chain"] = trx["transactionid"] + "_" + str(vin["vout_index"])

# save to HeliumDB

ret = put_transaction(prev_tx_key, tx_fragment)

if ret == False:

raise(ValueError("failed to update spent tx fragment"))

# put the fragments of the consuming transaction into the HeliumDB

ctr = 0

for vout in trx["vout"]:

tx_fragment = {}

txkey = trx["transactionid"] + "_" + str(ctr)

tx_fragment["pkhash"] = vout["ScriptPubKey"][2]

tx_fragment["value"] = vout["value"]

tx_fragment["spent"] = False

tx_fragment["tx_chain"] = ""

if put_transaction(txkey, tx_fragment) == False:

raise(ValueError("failed to insert consuming transaction fragment"))

ctr += 1

except Exception as err:

print(str(err))

logging.debug('transaction_update: exception: ' + str(err))

return False

return True

Helium blk_index Module

"""

Implementation of the Helium block index key-value store

lets us search for the block that contains a transaction

"""

import plyvel

import pdb

import logging

import json

import rcrypt

"""

log debugging messages to the file debug.log

"""

logging.basicConfig(filename="debug.log",filemode="w",

format='%(asctime)s:%(levelname)s:%(message)s',level=logging.DEBUG)

# handle to the Helium blk_index key-value store

bDB = None

def open_blk_index(filepath: "string") -> "db handle or False":

"""

opens the Helium blk_index store and returns a handle to

the database. Any directories in filepath must exist. The

database will be created if it does not exist.

Returns a handle to the database or False

"""

try:

global bDB

bDB = plyvel.DB(filepath, create_if_missing=True)

print("blk_index opened")

except Exception as err :

print("open_blk_index: exception: " + str(err))

logging.debug("open_blk_index: exception: " + str(err))

return False

return bDB

def close_blk_index() -> "bool":

"""

close the Helium block index store

Returns True if the database is closed, False otherwise

"""

try:

bDB.close()

if bDB.closed != True: return False

except Exception as err:

logging.debug("close_blk_index: exception: " + str(err))

return False

return True

def get_blockno(txid: "string") -> "False or integer":

"""

Receives a transaction id (not a transaction fragment id)

returns the block no. of the block that contains the transaction:

"""

try:

# get the block no, return False if the

# transaction does not exist in the store

block_no = bDB.get(str.encode(txid))

if block_no == None:

raise(ValueError("txid does not have a block no."))

block_no = block_no.decode()

block_no = int(block_no)

except Exception as err :

logging.debug('get_blockno: exception: ' + str(err))

return False

return block_no

def put_index(txid: "string", blockno: "integer") -> "bool":

"""

Returns True if the trainsactionid_block_no key-value pair is created

and False otherwise

"""

try:

if len(txid.strip()) != 64:

raise(ValueError("txid invalid length"))

if len(str(blockno).strip()) == 0:

raise(ValueError("blockno field is empty"))

if blockno < 0:

raise(ValueError("negative blockno"))

# save the transactionid-block no pair in the store

bDB.put(str.encode(txid), str.encode(str(blockno)))

except Exception as err:

logging.debug('put_blockno: exception: ' + str(err))

return False

return True

def delete_index(txid: "string"):

"""

deletes a (txid, blockno) pair from the blk_index store

"""

try:

bDB.delete(str.encode(txid))

except Exception as err :

logging.debug('delete_tx_blockno: exception: ' + str(err))

return False

return True

Helium Mining Module

"""

hmining.py: The code in this module implements a Helium mining node .

The node is listening for transactions propagating over the Helium peer-to-peer

network.

"""

import blk_index

import hconfig

import hblockchain as bchain

import hchaindb

import networknode

import rcrypt

import tx

import asyncio

import json

import math

import sys

import time

import threading

import pdb

import logging

"""

log debugging messages to the file debug.log

"""

logging.basicConfig(filename="debug.log",filemode="w",

format='%(asctime)s:%(levelname)s:%(message)s',

level=logging.DEBUG)

"""

address list of nodes on the Helium network

"""

address_list = []

"""

specify a list container to hold transactions which are received by the miner.

"""

mempool = []

"""

blocks mined by other miners which are received by this mining node

and are to intended to be appended to this miner's blockchain.

"""

received_blocks = []

"""

blocks which are received and cannot be added to the head of the blockchain

or the parent of the block at the head of the blockchain.

"""

orphan_blocks = []

"""

list of transactions to be removed from memcache because they are part of a

block that has been mined

"""

remove_list = []

semaphore = threading.Semaphore()

def mining_reward(block_height:"integer") -> "non-negative integer":

"""

The mining_reward function determines the mining reward.

When a miner successfully mines a block he is entitled to a reward which

is a number of Helium coins.

The reward depends on the number of blocks that have been mined so far.

The initial reward is hconfig.conf["MINING_REWARD"] coins.

This reward halves every hconfig.conf["REWARD_INTERVAL"] blocks .

"""

try:

# there is no reward for the genesis block

sz = block_height

sz = sz // hconfig.conf["REWARD_INTERVAL"]

if sz == 0: return hconfig.conf["MINING_REWARD"]

# determine the mining reward

reward = hconfig.conf["MINING_REWARD"]

for __ctr in range(0,sz):

reward = reward/2

# truncate to an integer

reward = int(round(reward))

# the mining reward cannot be less than the lowest denominated

# currency unit

if reward < hconfig.conf["HELIUM_CENT"]: return 0

except Exception as err:

print(str(err))

logging.debug('mining reward: exception: ' + str(err))

return -1

return reward

def receive_transaction(transaction: "dictionary") -> "bool":

"""

receives a transaction propagating over the P2P cryptocurrency network

this function executes in a thread

"""

# do not add the transaction if:

# it already exists in the mempool

# it exists in the Chainstate

# it is invalid

try:

# if the transaction is in the mempool, return

for trans in mempool:

if trans == transaction: return False

# do not add the transaction if it has been accounted for in

# the chainstate database.

tx_fragment_id = transaction["transactionid"] + "_" + "0"

if hchaindb.get_transaction(tx_fragment_id) != False:

return True

# verify that the incoming transaction is valid

if len(transaction["vin"]) > 0: zero_inputs = False

else: zero_inputs = True

if tx.validate_transaction(transaction, zero_inputs) == False:

raise(ValueError("invalid transaction received"))

# add the transaction to the mempool

mempool.append(transaction)

# place the transaction on the P2P network for further

# propagation

propagate_transaction(transaction)

except Exception as err:

logging.debug('receive_transaction: exception: ' + str(err))

return False

return True

async def make_candidate_block() -> "dictionary || bool":

"""

makes a candidate block for inclusion in the Helium blockchain.

A candidate block is created by :

(i) fetching transactions from the mempool and adding them to

the candidate blocks transaction list.

(ii) specifying the block header.

returns the candidate block or returns False if there is an error or if

the mempool is empty.

Executes in a Python thread

"""

try:

# if the mempool is empty then no transactions can be put into

# the candidate block

if len(mempool) == 0: return False

# make a public-private key pair that the miner will use to receive

# the mining reward as well as the transaction fees.

key_pair = make_miner_keys()

block = {}

# create an incomplete candidate block header

block['version'] = hconfig.conf["VERSION_NO"]

block['timestamp'] = int(time.time())

block['difficulty_bits'] = hconfig.conf["DIFFICULTY_BITS"]

block['nonce'] = hconfig.conf["NONCE"]

if len(bchain.blockchain) > 0:

block['height'] = bchain.blockchain[-1]["height"] + 1

else :

block['height'] = 0

block['merkle_root'] = ""

# get the value of the hash of the previous block's header

# this induces tamperproofness for the blockchain

if len(bchain.blockchain) > 0:

block['prevblockhash'] = bchain.blockheader_hash(bchain.blockchain[-1])

else:

block['prevblockhash'] = ""

# calculate the size (in bytes) of the candidate block header

# The number 64 is the byte size of the SHA-256 hexadecimal

# merkle root. The merkle root is computed after all the

# transactions are included in the candidate block

# reserve 1000 bytes for the coinbase transaction

block_size = sys.getsizeof(block['version'])

block_size += sys.getsizeof(block['timestamp'])

block_size += sys.getsizeof(block['difficulty_bits'])

block_size += sys.getsizeof(block['nonce'])

block_size += sys.getsizeof(block['height'])

block_size += sys.getsizeof(block['prevblockhash'])

block_size += 64

block_size += sys.getsizeof(block['timestamp'])

block_size += 1000

# list of transactions in the block

block['tx'] = []

# get the Unix Time now

now = int(time.time())

# add transactions from the mempool to the candidate block until

# the transactions in the mempool are exhausted or the block

# attains its maximum permissible size

for memtx in mempool:

# do not process future transactions

if memtx['locktime'] > now: continue

memtx = add_transaction_fee(memtx, key_pair[1])

# add the transaction to the candidate block

block_size += sys.getsizeof(memtx)

if block_size <= hconfig.conf['MAX_BLOCK_SIZE']:

block['tx'].append(memtx)

remove_list.append(memtx)

else:

break

# return if there are no transactions in the block

if len(block["tx"]) == 0: return False

# add a coinbase transaction

coinbase_tx = make_coinbase_transaction(block['height'], key_pair[1])

block['tx'].insert(0, coinbase_tx)

# update the length of the block

block_size += sys.getsizeof(block['tx'])

# calculate the merkle root of this block

ret = bchain.merkle_root(block['tx'], True)

if ret == False:

logging.debug('mining::make_candidate_block - merkle root error')

return False

block['merkle_root'] = ret

###################################

# validate the candidate block

###################################

if bchain.validate_block(block) == False:

logging.debug('mining::make_candidate_block - invalid block header')

return False

except Exception as err:

logging.debug('make_candidate_block: exception: ' + str(err))

# At this stage the candidate block has been created and it can be mined

return block

def make_miner_keys():

"""

makes a public-private key pair that the miner will use to receive

the mining reward and the transaction fee for each transaction.

This function writes the keys to a file and returns hash:

RIPEMD160(SHA256(public key))

"""

try:

keys = rcrypt.make_ecc_keys()

privkey = keys[0]

pubkey = keys[1]

pkhash = rcrypt.make_SHA256_hash(pubkey)

mdhash = rcrypt.make_RIPEMD160_hash(pkhash)

# write the keys to file with the private key as a hexadecimal string

f = open('coinbase_keys.txt', 'a')

f.write(privkey)

f.write(' ') # newline

f.write(pubkey)

f.write(' ')

f.close()

except Exception as err:

logging.debug('make_miner_keys: exception: ' + str(err))

return mdhash

def add_transaction_fee(trx: 'dictionary', pubkey: 'string') -> 'dictionary':

"""

add_transaction_fee directs the transaction fee of a transaction to

the miner.

receives a transaction and a miner's public key.

amends and returns the transaction so that it consumes the transaction fee.

"""

try:

# get the previous transaction fragments

prev_fragments = []

for vin in trx["vin"]:

fragment_id = vin["txid"] + "_" + str(vin["vout_index"])

prev_fragments.append(hchaindb.get_transaction(fragment_id))

# Calculate the transaction fee

fee = tx.transaction_fee(trx, prev_fragments)

if fee > 0:

vout = {}

vout["value"] = fee

ScriptPubKey = []

ScriptPubKey.append('SIG')

ScriptPubKey.append(rcrypt.make_RIPEMD160_hash(rcrypt.make_SHA256_hash(pubkey)))

ScriptPubKey.append('<DUP>')

ScriptPubKey.append('<HASH_160>')

ScriptPubKey.append('HASH-160')

ScriptPubKey.append('<EQ-VERIFY>')

ScriptPubKey.append('<CHECK_SIG>')

vout["ScriptPubKey"] = ScriptPubKey

trx["vout"].append(vout)

except Exception as err:

logging.debug('add_transaction_fee: exception: ' + str(err))

return False

return trx

def make_coinbase_transaction(block_height: "integer", pubkey: "string") -> 'dict':

"""

makes a coinbase transaction, this is the miner's reward for mining

a block. Receives a public key to denote ownership of the reward.

Since this is a fresh issuance of heliums there are no vin elements.

locks the transaction for hconfig["COINBASE_INTERVAL"] blocks.

Returns the coinbase transaction.

"""

try:

# calculate the mining reward

reward = mining_reward(block_height)

# create a coinbase transaction

trx = {}

trx['transactionid'] = rcrypt.make_uuid()

trx['version'] = hconfig.conf["VERSION_NO"]

# the mining reward cannot be claimed until approximately 100 blocks are mined

# convert into a time interval

trx['locktime'] = hconfig.conf["COINBASE_INTERVAL"]*600

trx['vin'] = []

trx['vout'] = []

ScriptPubKey = []

ScriptPubKey.append('SIG')

ScriptPubKey.append(rcrypt.make_RIPEMD160_hash(rcrypt.make_SHA256_hash(pubkey)))

ScriptPubKey.append('<DUP>')

ScriptPubKey.append('<HASH_160>')

ScriptPubKey.append('HASH-160')

ScriptPubKey.append('<EQ-VERIFY>')

ScriptPubKey.append('<CHECK_SIG>')

# create the vout element with the reward

trx['vout'].append({

'value': reward,

'ScriptPubKey': ScriptPubKey

})

except Exception as err:

logging.debug('make_coinbase_transaction: exception: ' + str(err))

return trx

def remove_mempool_transactions(block: 'dictionary') -> "bool":

"""

removes the transactions in the candidate block from the mempool

"""

try :

for transaction in block["tx"]:

if transaction in mempool:

mempool.remove(transaction)

except Exception as err:

logging.debug('remove_mempool_transactions: exception: ' + str(err))

return False

return True

async def mine_block(candidate_block: 'dictionary') -> "bool":

"""

Mines a candidate block.

Returns the solution nonce as a hexadecimal string if the block is

mined and False otherwise

Executes in a Python thread

"""

try:

final_nonce = None

save_block = dict(candidate_block)

# Loop until block is mined

while True:

# compute the SHA-256 hash for the block header of the candidate block

hash = bchain.blockheader_hash(candidate_block)

# convert the SHA-256 hash string to a Python integer

mined_value = int(hash, 16)

mined_value = 1/mined_value

# test to determine whether the block has been mined

if mined_value < hconfig.conf["DIFFICULTY_NUMBER"]:

final_nonce = candidate_block["nonce"]

break

with semaphore:

if len(received_blocks) > 0:

if compare_transaction_lists(candidate_block) == False:

return False

# failed to mine the block so increment the

# nonce and try again

candidate_block['nonce'] += 1

logging.debug('mining.py: block has been mined')

# add block to the miner's blockchain

with semaphore:

ret = bchain.add_block(save_block)

if ret == False:

raise(ValueError("failed to add mined block to blockchain"))

# propagate the block on the Helium network

propagate_mined_block(candidate_block)

except Exception as err:

logging.debug('mine_block: exception: ' + str(err))

return False

return hex(final_nonce)

def proof_of_work(block):

"""

Proves whether a received block has in fact been mined.

Returns True or False

"""

try:

if block['difficulty_bits'] != hconfig.conf["DIFFICULTY_BITS"]:

raise(ValueError("wrong difficulty bits used"))

# compute the SHA-256 hash for the block header of the candidate block

hash = bchain.blockheader_hash(block)

# convert the SHA-256 hash string to a Python integer to base 10

mined_value = int(hash, 16)

mined_value = 1/mined_value

# test to determine whether the block has been mined

if mined_value < hconfig.conf["DIFFICULTY_NUMBER"]: return True

except Exception as err :

logging.debug('proof_of_work: exception: ' + str(err))

return False

def receive_block(block):

"""

Maintains the received_blocks list.

Receives a block and returns False if:

(1) the block is invalid

(2) the block height is less than (blockchain height - 2)

(3) block height > (blockchain height + 1)

(4) the proof of work fails

(5) the block does not contain at least two transactions

(6) the first block transaction is not a coinbase transaction

(7) transactions in the block are invalid

(8) block is already in the blockchain

(9) the block is already in the received_blocks list

Otherwise (i) adds the block to the received_blocks list

(ii) propagates the block

Executes in a python thread

"""

try:

# test if block is already in the received_blocks list

with semaphore:

for blk in received_blocks:

if blk == block: return False

# verify the proof of work

if proof_of_work(block) == False:

raise(ValueError("block proof of work failed"))

# reset the nonce to its initial value

block["nonce"] == hconfig.conf["NONCE"]

# validate the block

if bchain.validate_block(block) == False: return False

if len(bchain.blockchain) > 0:

# do not add stale blocks to the blockchain

if block["height"] < bchain.blockchain[-1]["height"] - 2:

raise(ValueError("block height too old"))

# do not add blocks that are too distant into the future

if block["height"] > bchain.blockchain[-1]["height"] + 1:

raise(ValueError("block height beyond future"))

# test if block is in the primary or secondary blockchains

if len(bchain.blockchain) > 0:

if block == bchain.blockchain[-1]: return False

if len(bchain.blockchain) > 1:

if block == bchain.blockchain[-2]: return False

if len(bchain.secondary_blockchain) > 0:

if block == bchain.secondary_blockchain[-1]: return False

if len(bchain.secondary_blockchain) > 1:

if block == bchain.secondary_blockchain[-2]: return False

# add the block to the blocks_received list

received_blocks.append(block)

process_received_blocks()

except Exception as err:

logging.debug('receive_block: exception: ' + str(err))

return False

return True

def process_received_blocks() -> 'bool':

"""

processes mined blocks that are in the in the received_blocks

list and attempts to add these blocks to a blockchain.

Algorithm:

(1) get a block from the received blocks list.

(2) if the blockchain is empty and the block has an empty prevblockhash field

then add the block to the blockchain.

(3) Compute the block header hash of the last block on the blockchain (blkhash).

if blkhash == block["prevblockhash"] then add the block to the blockchain.

(4) if the blockchain has at least two blocks, then if:

let blk_hash be the hash second-latest block of the blockchain, then if

block["prevblockhash"] == blk_hash

create a secondary block consisting of all of the blocks of the blockchain

except for the latest. Append the block to the secondary blockchain.

(5) Otherwise move the block to the orphans list.

(6) If the received block was attached to a blockchain, for each block in the orphans

list, try to attach the orphan block to the primary blockchain or the secondary

blockchain if it exists.

(7) If the received block was attached to a blockchain and the secondary blockchain has

elements then swap the blockchain and the secondary blockchain if the length of the

secondary blockchain is greater.

(8) if the receive block was attached and the secondary blockchain has elements then

clear the secondary blockchain if its length is more than two blocks behind the

primary blockchain.

Note: Runs In A Python thread

"""

while True:

# process all of the blocks in the received blocks list

add_flag = False

# get a received block

if len(received_blocks) > 0:

block = received_blocks.pop()

else:

return True

# add it to the primary blockchain

with semaphore:

if bchain.add_block(block) == True:

logging.debug('receive_mined_block: block added to primary blockchain')

add_flag = True

# test whether the primary blockchain must be forked

# if the previous block hash is equal to the hash of the parent block of the

# block at the head of the blockchain add the block as a child of the parent and

# create a secondary blockchain. This constitutes a fork of the primary

# blockchain

elif len(bchain.blockchain) >= 2 and block['prevblockhash'] == bchain.blockheader_hash(bchain.blockchain[-2]):

logging.debug('receive_mined_block: forking the blockchain')

fork_blockchain(block)

if bchain.add_block(block) == True:

logging.debug('receive_mined_block: block added to blockchain')

add_flag = True

# add it to the secondary blockchain

elif len(bchain.secondary_blockchain) > 0 and block['prevblockhash'] == bchain.blockheader_hash(bchain.secondary_blockchain[-1]):

swap_blockchains()

if bchain.add_block(block) == True:

logging.debug('receive_mined_block: block added to blockchain')

add_flag = True

# cannot attach the block to a blockchain, place it in the orphans list

else:

orphan_blocks.append(block)

if add_flag == True:

if block["height"] % hconfig.conf["RETARGET_INTERVAL"] == 0:

retarget_difficulty_number(block)

handle_orphans()

swap_blockchains()

# remove any transactions in this block that are also

# in the the mempool

with semaphore:

remove_mempool_transactions(block)

propagate_mined_block(block)

return True

def fork_blockchain(block: 'list') -> 'bool':

"""

forks the primary blockchain and creates a secondary blockchain from

the primary blockchain and then adds the received block to the secondary block

"""

try:

bchain.secondary_blockchain = list(bchain.blockchain[0:-1])

bchain.secondary_blockchain.append(block)

# switch the primary and secondary blockchain if required

swap_blockchains()

except Exception as err:

logging.debug('fork_blockchain: exception: ' + str(err))

return False

return True

def swap_blockchains() -> 'bool':

"""

compares the length of the primary and secondary blockchains.

The longest blockchain is designated as the primary blockchain and the other

blockchain is designated as the secondary blockchain.

if the primary blockchain is ahead of the secondary blockchain by at least

two blocks then clear the secondary blockchain.

"""

try:

# if the secondary blockchain is longer than the primary blockchain

# designate the secondary blockchain as the primary blockchain

# and the primary blockchain as the secondary blockchain

if len(bchain.secondary_blockchain) >= len(bchain.blockchain):

tmp = list(bchain.blockchain)

bchain.blockchain = list(bchain.secondary_blockchain)

bchain.secondary_blockchain = list(tmp)

# if the primary blockchain is ahead of the secondary blockchain by

# at least two blocks then clear the secondary blockchain

if len(bchain.blockchain) - len(bchain.secondary_blockchain) > 2:

bchain.secondary_blockchain.clear()

except Exception as err:

logging.debug('swap_blockchains: exception: ' + str(err))

return False

return True

def handle_orphans() -> "bool":

"""

tries to attach an orphan block to the head of the primary or secondary blockchain.

Sometimes blocks are received out of order and cannot be attached to the primary

or secondary blockchains. These blocks are placed in an orphan_blocks list and as

new blocks are added to the primary or secondary blockchains, an attempt is made to

add orphaned blocks to the blockchain(s).

"""

try:

# iterate through the orphan blocks attempting to append an orphan

# block to a blockchain

if len(bchain.blockchain) == 0: return

# try to append to the primary blockchain

for block in orphan_blocks:

if block['prevblockhash'] == bchain.blockheader_hash(bchain.blockchain[-1]):

if block["height"] == bchain.blockchain[-1]["height"] + 1:

bchain.blockchain.append(block)

orphan_blocks.remove(block)

# remove this orphan blocks transactions from the mempool

with semaphore:

remove_mempool_transactions(block)

# try to append to the secondary blockchain

if len(bchain.secondary_blockchain) > 0:

for block in orphan_blocks:

if block['prevblockhash'] == bchain.blockheader_hash(bchain.secondary_blockchain[-1]):

if block["height"] == bchain.secondary_blockchain[-1]["height"] + 1:

bchain.secondary_blockchain.append(block)

orphan_blocks.remove(block)

# remove this blocks transactions from the mempool

with semaphore:

remove_mempool_transactions(block)

# remove stale orphaned blocks

for block in orphan_blocks :

if bchain.blockchain[-1]["height"] >=3:

if bchain.blockchain[-1]["height"] - block["height"] >= 2: orphan_blocks.remove(block)

except Exception as err:

logging.debug('handle_orphans: exception: ' + str(err))

return False

return True

def remove_received_block(block: 'dictionary') -> "bool":

"""

remove a block from the received blocks list

"""

try:

if block in received_blocks:

received_blocks.remove(block)

except Exception as err:

logging.debug('remove_received_block: exception: ' + str(err))

return False

return True

def retarget_difficulty_number(block):

"""

recalibrates the difficulty number every hconfig.conf["RETARGET_INTERVAL"] blocks

"""

if block["height"] == 0: return

old_diff_no = hconfig.conf["DIFFICULTY_NUMBER"]

initial_block = bchain.blockchain[(block["height"] - hconfig.conf["RETARGET_INTERVAL"])]

time_initial = initial_block["timestamp"]

time_now = block["timestamp"]

elapsed_seconds = time_now - time_initial

# the average time to mine a block is 600 seconds

blocks_expected_to_be_mined = int(elapsed_seconds / 600)

discrepancy = 1000 - blocks_expected_to_be_mined

hconfig.conf["DIFFICULTY_NUMBER"] = old_diff_no -

old_diff_no * (20/100)*(discrepancy /(1000 + blocks_expected_to_be_mined))

return

def propagate_transaction(txn: "dictionary"):

"""

propagates a transaction that is received

"""

if len(address_list) % 20 == 0: get_address_list()

cmd = {}

cmd["jsonrpc"] = "2.0"

cmd["method"] = "receive_transaction"

cmd["params"] = {"trx":txn}

cmd["id"] = 0

for addr in address_list:

rpc = json.dumps(cmd)

networknode.hclient(addr,rpc)

return

def propagate_mined_block(block):

"""

sends a block to other mining nodes so that they may add this block

to their blockchain.

We refresh the list of known node addresses periodically

"""

if len(address_list) % 20 == 0: get_address_list()

cmd = {}

cmd["jsonrpc"] = "2.0"

cmd["method"] = "receive_block"

cmd["params"] = {"block":block}

cmd["id"] = 0

for addr in address_list:

rpc = json.dumps(cmd)

networknode.hclient(addr,rpc)

return

def get_address_list():

'''

update the list of node addresses that are known

AMEND IP ADDRESS AND PORT AS REQUIRED.

'''

ret = networknode.hclient("http://127.0.0.69:8081",

'{"jsonrpc":"2.0","method":"get_address_list","params":{},"id":1}')

if ret.find("error") != -1: return

retd = json.loads(ret)

for addr in retd["result"]:

if address_list.count(addr) == 0:

address_list.append(addr)

return

def start_mining():

"""

assemble a candidate block

then mine this block

"""

while True:

candidate_block = make_candidate_block()

if candidate_block() == False:

time.sleep(1)

continue

# remove transactions in the mined block from the mempool

if mine_block(candidate_block) != False:

remove_mempool_transactions(remove_list)

else: time.sleep(1)

def compare_transaction_lists(block):

"""

tests whether a transaction in a received block is also in a candidate

block that is being mined

"""

for tx1 in block["tx"]:

for tx2 in remove_list:

if tx1 == tx2: return False

return True

#####################################

# start mining in a thread:

# $(virtual) python hmining.py

#####################################

if __name__ == "__main__":

#Create the Threads

t1 = threading.Thread(target=start_mining, args=(), daemon=True)

t1.start()

t1.join()

Helium Wallet Module

###########################################################################

# wallet.rb: a command-line wallet for Helium

###########################################################################

import hblockchain

import hchaindb

import rcrypt

import logging

import pdb

import pickle

logging.basicConfig(filename="debug.log",filemode="w",

format='%(asctime)s:%(levelname)s:%(message)s', level=logging.DEBUG)

# the state of the wallet

wallet_state = {

"keys": [], # list of private-public key tuples

"received": [], # values received by the wallet holder

"spent": [], # values transferred away by the wallet holder

"received_last_block_scanned": 0,

"spent_last_block_scanned": 0

}

def create_keys():

key_pair = rcrypt.make_ecc_keys()

wallet_state["keys"].append(key_pair)

def initialize_wallet():

wallet_state["received"] = []

wallet_state["spent"] = []

wallet_state["received_last_block_scanned"] = 0

wallet_state["spent_last_block_scanned"] = 0

def value_received(blockno:"integer"= 0) -> "list" :

"""

obtains all of the helium values received by the wallet-holder by examining

transactions in the blockchain from and including blockno onwards .

Updates the wallet state.

"""

hreceived = []

tmp = {}

try:

# get values received from the blockchain

for block in hblockchain.blockchain:

if block["height"] < blockno: continue

for transaction in block["tx"]:

ctr = -1

for vout in transaction["vout"]:

ctr += 1

for key_pair in wallet_state["keys"]:

if rcrypt.make_RIPEMD160_hash(rcrypt.make_SHA256_hash(key_pair[1]))

== vout["ScriptPubKey"][2]:

tmp["value"] = vout["value"]

tmp["blockno"] = block["height"]

tmp["fragmentid"] = transaction["transactionid"] + "_" +

str(ctr)

tmp["public_key"] = key_pair[1]

hreceived.append(tmp)

break

# update the wallet state

last_block = hblockchain.blockchain[-1]

if last_block["height"] > wallet_state["received_last_block_scanned"]:

wallet_state["received_last_block_scanned"] = last_block["height"]

for received in hreceived:

wallet_state["received"].append(received)

except Exception as err:

print(str(err))

logging.debug('value_received exception: ' + str(err))

return False

return

def value_spent(blockno:"integer"= 0):

"""

obtains all of the helium values transferred by the wallet-holder by

examining transactions in the blockchain from and including blockno.

onwards. Updates the wallet state.

"""

hspent = []

tvalue = {}

try:

# get values spent from blockchain transactions

for block in hblockchain.blockchain:

if block["height"] < blockno: continue

for transaction in block["tx"]:

for vin in transaction["vin"]:

for key_pair in wallet_state["keys"]:

prevtxid = vin["transactionid"] + "_" + str(vin["vout_index"])

fragment = hchaindb.get_transaction(prevtxid)

rcrypt.make_RIPEMD160_hash(rcrypt.make_SHA256_hash(key_pair[1]))

== fragment["pkhash"] :

tvalue["value"] = vin["value"]

tvalue["blockno"] = block["height"]

tvalue["fragmentid"] = vin["transactionid"] + "_"

+ str(vin["vout_index"])

tvalue["public_key"] = key_pair[1]

hspent.append(tvalue)

break

last_block = hblockchain.blockchain[-1]

if last_block["height"] > wallet_state["spent_last_block_scanned"]:

wallet_state["spent_last_block_scanned"] = last_block["height"]

for spent in hspent:

wallet_state["spent"].append(spent)

except Exception as err:

print(str(err))

logging.debug('value_sent: exception: ' + str(err))

return False

return

def save_wallet() -> "bool":

"""

saves the wallet state to a file

"""

try:

f = open('wallet.dat', 'wb')

pickle.dump(wallet_state, f)

f.close()

return True

except Exception as error:

print(str(error))

return False

def load_wallet() -> "bool":

"""

loads the wallet state from a file

"""

try:

f = open('wallet.dat', 'rb')

global wallet_state

wallet_state = pickle.load(f)

f.close()

return True

except Exception as error:

print(str(error))

return False

Helium Directory Server Module

'''

start a directory server on 127.0.0.69:8081

'''

import hmining

from tornado import ioloop, web

from jsonrpcserver import method, async_dispatch as dispatch

import ipaddress

import json

import threading

import pdb

import logging

# A Fake list of addresses for testing only

address_list = ["127.0.0.10:8081", "127.0.0.11:8081", "127.0.0.12:8081", "127.0.0.13:8081",

"127.0.0.14:8081", "127.0.0.15:8081", "127.0.0.16:8081", "127.0.0.17:8081",

"127.0.0.18:8081", "127.0.0.19:8081", "127.0.0.20:8081" ]

logging.basicConfig(filename="debug.log",filemode="w",

format='server: %(asctime)s:%(levelname)s:%(message)s', level=logging.DEBUG)

@method

async def get_address_list():

with hmining.semaphore:

ret = address_list

return ret

@method

async def register_address(address: "string"):

global address_list

try:

# validate IP address and port format: addr:port

if address.find(":") == -1: return "error-invalid address"

addr_list = address.split(":")

addr_list[1]=addr_list[1].strip()

if len(addr_list[0]) == 0 or len(addr_list[1]) ==

0: return "error-invalid address: " + address

if int(addr_list[1]) <= 0 or int(addr_list[1]) >=

65536: return "error-invalid address: " + address

_ = ipaddress.ip_address(addr_list[0])

addr = addr_list[0] + ":" + addr_list[1]

if addr not in address_list:

with hmining.semaphore:

address_list.append(addr)

return address

except Exception as err:

logging.debug('directory server::register address - ' + str(err))

return "error-register address"

class MainHandler(web.RequestHandler):

async def post(self):

request = self.request.body.decode()

logging.debug('decoded server request = ' + str(request))

response = await dispatch(request)

print(response)

self.write(str(response))

app = web.Application([(r"/", MainHandler)])

# start the network interface for a node on the local loop: 127.0.0.69:8081

if __name__ == "__main__":

app.listen(address="127.0.0.69", port="8081")

logging.debug('server running')

print("directory server started at 127.0.0.69:8081")

ioloop.IOLoop.current().start()

Helium Networknode Module

'''

netnode: implementation of a synchronous RPC-Client node that makes remote

procedure calls to RPC servers using the JSON RPC protocol version 2, and

a JSON-RPC server that responds to requests from RPC client nodes.

'''

import blk_index as blkindex

import hblockchain

import hchaindb

import hmining

import networknode

from tornado import ioloop, web

from jsonrpcserver import method, async_dispatch as dispatch

from jsonrpcclient.clients.http_client import HTTPClient

import ipaddress

import threading

import json

import pdb

import logging

import os

import sys

logging.basicConfig(filename="debug.log",filemode="w",

format='server: %(asctime)s:%(levelname)s:%(message)s', level=logging.DEBUG)

##################################

# JSON-RPC Client

##################################

def hclient(remote_server, json_rpc):

'''

sends a synchronous request to a remote RPC server

'''

try:

client = HTTPClient(remote_server)

response = client.send(json_rpc)

valstr = response.text

val = json.loads(valstr)

print("result: " + str(val["result"]))

print("id: " + str(val["id"]))

return valstr

except Exception as err:

logging.debug("node_client: " + str(err))

return '{"jsonrpc": "2.0", "result":"error", "id":"error"}'

#######################################

# JSON-RPC Server

#######################################

address_list = []

@method

async def receive_transaction(trx):

"""

receives a transaction that is propagating on the Helium network

"""

try :

hmining.semaphore.acquire()

ret = hmining.receive_transaction(trx)

hmining.semaphore.release()

if ret == False: return "error: invalid transaction"

return "ok"

except Exception as err:

return "error: " + err

@method

async def receive_block(block):

"""

receives a block that is propagating on the Helium network

"""

try:

ret = hmining.receive_block(block)

if ret == False: return "error: invalid block"

return "ok"

except Exception as err:

return "error: " + err

@method

async def get_block(height):

"""

returns the block with the given height or an error if the block

does not exist

"""

with hmining.semaphore:

if len(hblockchain.blockchain) == 0:

return ("error: empty blockchain")

if height < 0 or height > hblockchain.blockchain[-1]["height"] :

return "error-invalid block height"

block = json.dumps(hblockchain.blockchain[height])

return block

@method

async def get_blockchain_height():

"""

returns the height of the blockchain. Note that the first block has

height 0

"""

with hmining.semaphore:

if hblockchain.blockchain == []: return -1

height = hblockchain.blockchain[-1]["height"]

return height

@method

async def clear_blockchain():

"""

clears the primary and secondary blockchains.

"""

with hmining.semaphore:

hblockchain.blockchain.clear()

hblockchain.secondary_blockchain.clear()

return "ok"

class MainHandler(web.RequestHandler):

async def post(self):

request = self.request.body.decode()

logging.debug('decoded server request = ' + str(request))

response = await dispatch(request)

print(response)

self.write(str(response))

def startup():

'''

start node related systems

'''

try:

# remove any locks

os.system("rm -rf ../data/heliumdb/*")

os.system("rm -rf ../data/hblk_index/*")

# start the Chainstate Database

ret = hchaindb.open_hchainstate("../data/heliumdb")

if ret == False: return "error: failed to start Chainstate database"

else: print("Chainstate Database running")

# start the LevelDB Database blk_index

ret = blkindex.open_blk_index("../data/hblk_index")

if ret == False: return "error: failed to start blk_index"

else: print("blkindex Database running")

except Exception:

return "error: failed to start Chainstate database"

return True

app = web.Application([(r"/", MainHandler)])

######################################################

# start the network interface for the server node on

# the localhost loop: 127.0.0.19:8081

#####################################################

if __name__ == "__main__":

app.listen(address="127.0.0.51", port="8081")

logging.debug('server node is running')

print("network node starting at 127.0.0.51:8081")

###############################

# start this node

###############################

if startup() != True:

logging.debug('server node resource failure')

print("stopping")

sys.exit()

logging.debug('server node is running')

print('server node is running')

################################

# start the event loop

################################

ioloop.IOLoop.current().start()

Index

Advanced Encryption Standard (AES)

app.listen function

Asymmetric key

Authenticity

Basic attention token (BAT)

Binance Coin (BNB)

Bitcoin

Bitcoin Cash (BCH)

BitTorrent

blk_index database

code

interface functions

Pytests

blk_index database

blk_index maintenance

Block

Blockchain

attributes

block fields

blockheader_hash function

config module

crypto packages

dictionary data structure

exception error

hblockchain.py

module

Pytest primer

Python logger

rcrypt code

read_block function

unit tests

validate_block function

Blockchain maintenance

get_block

peer-to-peer network

Block checks

Brave browser

Brute-force attack

Caesar cipher

Canonical transaction processing

Canonical transaction structure

execution stack

Heliums

locktime

mechanics

unlocked

vin list

vout list

Cassandra

Chainstate database

code

interface functions

pk_hash

public-private key pairs

Pytests

structure

transaction key

tx_chain

tx Module

wallets

Chainstate maintenance

Ciphertext

Client-server model

close_blk_index function

close_chainstate function

close_hchainstate function

Coinbase transaction

Collision

Concurrent asynchronous functions

Concurrent Python program

asynchronous functions

code

fibonacci function

fibprime.py file

mutual exclusion

prime number function

race conditions

semaphores

threading.Thread function

Configuration

coinbase transaction

config directory

currency unit

DIFFICULTY_NUMBER

MAX_BLOCK_SIZE

maximum number, Helium coins

Max-Inputs parameter

MAX-LOCKTIME

Max-Outputs parameter

MINING_REWARD

nonce

REWARD_INTERVAL

version number

Confirmations

Consensus

Convertibility

Cryptanalysis

Cryptocurrency blockchain, transactions

pycryptodome

Cryptographic hash functions

DApp(s)

Darknet

delete_index function

denarius

Difficulty number

Digital advertising

Digital signature

Digraph

Distributed consensus algorithm

Distributed hash tables (DHT)

Double-spend

Elliptic Curve Digital Signature Algorithm (ECDSA)

employee_record function

Encryption key

Equation of exchange

ERC-20 token

Ethereum (ETH)

Ethereum Virtual Machine (EVM)

Euler’s theorem

Faucet

Fiat currency

fibonacci function

Flooding

Fractional banking

Fundamental theorem of arithmetic

Fungible

Games

Genesis block

get_blockno function

get_transaction function

Globally unique IDs

Go language

Gold standard

Great depression

Hashing power

Hash pointer

Hawala

hblockchain module

Helium

Helium blk_index module

Helium blockchain module

Helium configuration module

Helium cryptocurrency

Bitcoin

blockchains

configuration

SeeConfiguration

deadsnakes repository

Debian-derived Linux distributions

development environment

pip3 tool

Python installation/virtual environment

shell commands

transaction processing

virtualenv

Helium database maintenance

blk_index

Chainstate

Helium data structures

blockchain

Block files

Mining

Transactions

Helium directory server module

Helium hchaindb module

Helium mining module

Helium networknode module

Helium transaction processing

hblockchain module

local blockchains

p2pkhash stack processor

peer-to-peer network

tx module

unit tests

verification

Helium tx module

Helium wallet module

hmutex.acquire function

hmutex.release function

Huang, Qin Shi

Hyperinflation

Immutability

Initialization vector

Integrity

JSON-remote procedure calls (RPC) server

add function

app.listen function

code

fibonacci function

ioloop

MainHandler class

output

params

rpc_client.py

tmp directory

Tornado

Keep it simple, stupid (KISS)

Key distribution

Keynesian economics

Keynes, John Maynard

Key pair

Key-value store

LevelDB database

definition

installation

keyspace

Plyvel

Litecoin (LTC)

Locktime

make_blocks function

make_coinbase_transaction function

make_random_transaction function

Malicious nodes

Mempool

Merkle trees

balanced binary tree

blockheader_hash function

block transactions

cryptocurrency implementations

hblockchain module

leaf nodes

make_leaf_nodes

SHA256 hash

third layer

unit_tests directory

Message authentication codes (MAC)

Message digest

Mining

add_transaction_fee

Bitcoin

blockchain

distributed consensus algorithm

financial benefits

fork_blockchain

handle_orphans

hblockchain module

hmining module

make_candidate_block

make_coinbase_transaction

make_miner_keys

mempool

mine_block

mining_reward

miscellany

orphan_blocks

process_receive_block

proof_of_work

received_blocks

receive_transaction

remove_mempool_transactions

remove_received_block

retarget_difficulty_number

swap_blockchains

transaction fees

unit tests

Mocks

Monero

Monero (XMR)

Monetary theory (M1)

monkeypatch() function

Multisig transaction

Nakamoto Satoshi

Napster

Natural language processing (NLP)

Neolithic age

Network

address_list

address resolution

blockchain initialization

bootup scripts

client-server model

directory_server module

directory-server.py

hnetwork directory

JSON remote procedure calls

JSON-RPC server

networknode.hclient function

node process space

procedure

queries

receive_block

tasks

transaction propagation

unit_tests directory

NIST

Nonce

NoSQL database

Opcodes

open_blk_index function

open_chainstate function

open_hchainstate function

Operand

P, Q

Parametrized test functions

Pastry

Peer-to-peer network (P2P)

bidirectional channels

Bitcoin node

boots up

client-server networks

cryptocurrency network

DHT

directory server

flooding

IP addresses

Sybil attack

topology

transaction propagation

TTL parameter

Permission-less

Petrodollar

Pickle module

Plaintext

PostgreSQL

Pound sterling

prevblockhash value

prevtx_value function

prime number function

Private key

Promissory notes

Pseudo-random number generator (PRNG)

Public address

algorithm

checksum

generation

Public-key

Public-private

put_index function

put_transaction function

Pytest

assert function

fixture function

floating-point numbers/approximation

mocks

monkeypatching

parametrized test functions

setup_module() function

teardown_module() function

testing exceptions

unit tests

randomize_list function

rcrypt Module

Redis

Reliability engineering

Bitcoin

directory file constraints

hblockchain module

peer-to-peer architecture

serialize_block

simulated blockchain

Result stack

RIPEMD-160

RippleNet

Ripple (XRP)

RSA

Satoshi Nakamoto

ScriptPubKey script

ScriptSig scripts

secure hash algorithm (SHA)

Serpent

setup_module function

SHA-256

Simulated blockchain

fragmentation

make_blocks function

make_random_transaction function

simulated_blockchain.py

uses

simulated_network directory

Smart contracts

Solidity

Sovereign prerogative

Stablecoins

Supply chain

SWIFT

Symmetric encryption

teardown_module function

test_blockchain.py

test_division function

Testnet

Tether (USDT)

The Onion Router (TOR)

timestamp

Time-to-live (TTL) parameter

Totient function

Transaction confirmations

transaction_fee function

transaction_update function

Transaction validation

Tree data structures

balanced binary trees

bidirectional movement

diagram

nodes

Trigraphs

TRON (TRX)

Turing machine

Twofish

unit_tests directory

unlock_transaction_fragment function

update_transaction function

vin list

vout_index

Wallets

dictionary data structure

functions

interface functions

save_wallet function

value_spent

unit_tests directory

vin element

wallet.py

X, Y, Z

X.509

Footnotes

I would encourage you to use Linux in preference to Microsoft Windows and in this regard would refer you to Dr. Vandana Shiva’s critique of Bill Gates and his agenda. Dr. Shiva is a nuclear physicist and an eco-feminist. I would also refer you to Dr. Richard Stallman, inventor of the open source software movement and the inventor/maintainer of GNU C++. The C++ compiler is probably the most complex piece of software ever devised and programmed. Both authors are accessible on the Internet. I have not used Microsoft Windows in any form since 2002. This book was written using LibreOffice v. 6.07.

The Mate GUI makes it very easy to create this virtual desktop environment. Each desktop can have multiple windows open, and each desktop can be running multiple applications.

As this book is being written, we are in a transition period where Python 2.x versions are being deprecated, hence the usage of pip3. In due course of time, the identifier pip3 will be supplanted by pip.

https://github.com/google/leveldb , last accessed on January 20, 2020

The following article discusses the performance of LevelDB: http://highscalability.com/blog/2011/8/10/leveldb-fast-and-lightweight-keyvalue-database-from-the-auth.html , last accessed on January 20, 2020

These instructions are for a Debian-derived distribution, such as Ubuntu and Mint. If you are installing for Windows or Mac, refer to documentation on the Internet. The $ symbol represents the operating system terminal prompt.

The URL for the git LevelDB git repository is https://github.com/google/leveldb

A pytest file can contain functions which are not pytest functions, but these functions cannot be directly executed by pytest. However, a pytest function can invoke such functions.

See www.tornadoweb.org/en/stable/

See https://bcb.github.io/jsonrpc/tornado

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for Back Matter

Create new playlist

Sign In

Sign Up

Table of Contents for
Back Matter