sys

The sys library provides access to the execution system such as constants, variables, command-line options, streams (stdout, stdin, stderr), and more. Most of the features of the library are constants or lists. The streams can be accessed directly but typically we use the print() function, which sends data to the stdout stream by default.

The most commonly used functions in this library include the following. Listing 5-1 demonstrates these variables and the exit() function.

• sys.argv: list of arguments passed to the script from the command line

• sys.exit(r): exit the program returning the value r to the caller

• sys.modules: list of modules loaded (imported)

• sys.path: list of paths to search for modules – can be modified

• sys.platform: display the platform information such as Linux, MicroPython, etc.

• sys.stderr: standard error stream

• sys.stdin: standard input stream

• sys.stdout: standard output stream

• sys.version: the version of Python currently executing

Listing 5-1. Demonstration of the sys library features

# MicroPython for the IOT - Chapter 5
# Example use of the sys library
import sys
print("Modules loaded: " , sys.modules)
sys.path.append("/sd")
print("Path: ", sys.path)
sys.stdout.write("Platform: ")
sys.stdout.write(sys.platform)
sys.stdout.write("
")
sys.stdout.write("Version: ")
sys.stdout.write(sys.version)
sys.stdout.write("
")
sys.exit(1)

Notice we start with the import statement and after that, we can print the constants and variables in the sys library using the print() function. We also see how to append a path to our search path with the sys.path.append() function. This is very helpful if we create our own directories on the boot drive (SD drive) to place our code. Without this addition, the import statement will fail unless the code module is in the lib directory.

At the end of the example, we see how to use the stdout stream to write things to the screen. Note that you must provide the carriage return (newline) command to advance the output to a new line ( ). The print() function takes care of that for us. The following shows the output of running this script on a MicroPython board.

>>> import examples.sys_example
Modules loaded:  {'examples': <module 'examples'>, 'examples.sys_example': <module 'examples.sys_example' from 'examples/sys_example.py'>}
Path:  ['', '/flash', '/flash/lib', '/sd']
Platform: WiPy
Version: 3.4.0

Notice we see the data as we expect and that this example is running on a WiPy module. Do you see something strange in the import statement? Here, the code is placed in a new directory named examples and then copied the code module to that directory on my boot drive. Thus, we can use the directory name in the statement to find the code module and import it (examples/sys_examples.py). If you want to run this code on your own MicroPython board, you can write the code to a file such as sys_example.py, then copy the file to your boot drive or use ftp to copy the file. This allows you to keep your boot (flash) drive tidy and place you code in a common location.

Finally, notice that there are no command-line arguments. This is because we used an import statement. However, if we were to run the code on our PC providing command-line arguments, we would see them. The following shows the output of running this script on a PC.

$ python ./sys_example.py
Modules loaded:  {'builtins': <module 'builtins' (built-in)>, 'sys': <module 'sys' (built-in)>, ... '/sd']
darwin
Version: 3.6.0 (v3.6.0:41df79263a11, Dec 22 2016, 17:23:13)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]

uio

The uio library contains additional functions to work with streams and stream-like objects. There is a single function named uio.open() that you can use to open files (but most people use the built-in function named open()) as well as classes for byte and string streams. In fact, the classes have similar file functions such as read(), write(), seek(), flush(), close(), as well as a getvalue() function, which returns the contents of the stream buffer that contains the data. Let’s look at an example.

In this example, we first open a new file for writing and write an array of bytes to the file. The technique used is passing the hex values for each bye to the write() function. When you read data from sensors, they are typically in binary for (a byte or string of bytes). You signify a byte with the escape, x as shown.

After writing the data to the file, we then read the file one byte at a time by passing 1 to the read() function. We then print the values read in their raw for (the value returned from the read(1) call), decimal value, and hex value. Listing 5-2 shows how to read a file in binary mode using the uio.FileIo() class. The bytes written contain a secret word (one obscured by using hex values) – can you see it?

Listing 5-2. Demonstration of the uio library features

# MicroPython for the IOT - Chapter 5
# Example use of the uio library
# Note: change uio to io to run this on your PC!
import uio
# Create the binary file
fio_out = uio.FileIO('data.bin', 'wb')
fio_out.write("x5Fx9ExAEx09x3Ex96x68x65x6Cx6Cx6F")
fio_out.write("x00")
fio_out.close()
# Read the binary file and print out the results in hex and char.
fio_in = uio.FileIO('data.bin', 'rb')
print("Raw,Dec,Hex from file:")
byte_val = fio_in.read(1)  # read a byte
while byte_val:
    print(byte_val, ",", ord(byte_val), hex(ord(byte_val)))
    byte_val = fio_in.read(1)  # read a byte
fio_in.close()

This is like how you would use the normal built-in functions, which we saw in the last chapter. Listing 5-3 shows the output when run on the WiPy.

Listing 5-3. Demonstration of the uio library features (Pyboard)

>>> import examples.uio_example
Raw,Dec,Hex from file:
b'_' , 95 0x5f
b'xc2' , 194 0xc2
b'x9e' , 158 0x9e
b'xc2' , 194 0xc2
b'xae' , 174 0xae
b'	' , 9 0x9
b'>' , 62 0x3e
b'xc2' , 194 0xc2
b'x96' , 150 0x96
b'h' , 104 0x68
b'e' , 101 0x65
b'l' , 108 0x6c
b'l' , 108 0x6c
b'o' , 111 0x6f
b'x00' , 0 0x0

If you’re curious what the file looks like, you can use a utility like hexdump to print the contents as shown below. Can you see the hidden message?

$ hexdump -C data.bin
00000000  5f 9e ae 09 3e 96 68 65  6c 6c 6f 00              |_...>.hello.|
0000000c

ujson

The ujson library is one of those libraries you are likely to use frequently when working with data in an IOT project. It provides encoding and decoding of JavaScript Object Notation (JSON) documents. This is because many of the IOT services available either require or can process JSON documents. Thus, you should consider getting into the habit of formatting your data in JSON to make it easier to integrate with other systems. The library implements the following functions that you can use to work with JSON documents.

• ujson.dumps(obj): return a string decoded from a JSON object.

• ujson.loads(str): parses the JSON string and returns a JSON object. Will raise an error if not formatted correctly.

• ujson.load(fp): parse the contents of a file pointer (a file string containing a JSON document). Will raise an error if not formatted correctly.

Recall we saw a brief example of JSON documents in the last chapter. That example was written exclusively for the PC but a small change makes it possible to run it on a MicroPython board. Let’s look at a similar example. Listing 5-4 shows an example of using the ujson library.

Listing 5-4. Demonstration of the ujson library features

# MicroPython for the IOT - Chapter 5
# Example use of the ujson library
# Note: change ujson to json to run it on your PC!
import ujson

# Prepare a list of JSON documents for pets by converting JSON to a dictionary
vehicles = []
vehicles.append(ujson.loads('{"make":"Chevrolet", "year":2015, "model":"Silverado", "color":"Pull me over red", "type":"pickup"}'))
vehicles.append(ujson.loads('{"make":"Yamaha", "year":2009, "model":"R1", "color":"Blue/Silver", "type":"motorcycle"}'))
vehicles.append(ujson.loads('{"make":"SeaDoo", "year":1997, "model":"Speedster", "color":"White", "type":"boat"}'))
vehicles.append(ujson.loads('{"make":"TaoJen", "year":2013, "model":"Sicily", "color":"Black", "type":"Scooter"}'))

# Now, write these entries to a file. Note: overwrites the file
json_file = open("my_vehicles.json", "w")
for vehicle in vehicles:
    json_file.write(ujson.dumps(vehicle))
    json_file.write("
")
json_file.close()

# Now, let's read the list of vehicles and print out their data
my_vehicles = []
json_file = open("my_vehicles.json", "r")
for vehicle in json_file.readlines():
    parsed_json = ujson.loads(vehicle)
    my_vehicles.append(parsed_json)
json_file.close()

# Finally, print a summary of the vehicles
print("Year Make Model Color")
for vehicle in my_vehicles:
    print(vehicle['year'],vehicle['make'],vehicle['model'],vehicle['color'])

The following shows the output of the script running on the WiPy.

>>> import examples.ujson_example
Year Make Model Color
2015 Chevrolet Silverado Pull me over red
2009 Yamaha R1 Blue/Silver
1997 SeaDoo Speedster White
2013 TaoJen Sicily Black

uos

The uos library implements a set of functions for working with the base operating system. Some of the functions may be familiar if you have written programs for your PC. Most functions allow you to work with file and directory operations. The following lists several of the more commonly used functions.

• uos.chdir(path): change the current directory

• uos.getcwd(): return the current working directory

• uos.listdir([dir]): list the current directory if dir is missing or list the directory specified

• uos.mkdir(path): create a new directory

• uos.remove(path): delete a file

• uos.rmdir(path): delete a directory

• uos.rename(old_path, new_path): rename a file

• uos.stat(path): get status of a file or directory

In this example, we see how to change the working directory so that we can simplify our import statements. We also see how to create a new directory, rename it, create a file in the new directory, list the directory, and finally clean up (delete) the changes. Listing 5-5 shows the example for working with the uos library functions.

Listing 5-5. Demonstration of the uos library features

# MicroPython for the IOT - Chapter 5
# Example use of the uos library
# Note: change uos to os to run it on your PC!
import sys
import uos

# Create a function to display files in directory
def show_files():
    files = uos.listdir()
    sys.stdout.write("
Show Files Output:
")
    sys.stdout.write("	name	size
")
    for file in files:
        stats = uos.stat(file)
        # Print a directory with a "d" prefix and the size
        is_dir = True
        if stats[0] > 16384:
            is_dir = False
        if is_dir:
            sys.stdout.write("d	")
        else:
            sys.stdout.write("	")
        sys.stdout.write(file)
        if not is_dir:
            sys.stdout.write("	")
            sys.stdout.write(str(stats[6]))
        sys.stdout.write("
")

# List the current directory
show_files()
# Change to the examples directory
uos.chdir('examples')
show_files()

# Show how you can now use the import statement with the current dir
print("
Run the ujson_example with import ujson_example after chdir()")
import ujson_example

# Create a directory
uos.mkdir("test")
show_files()

While this example is a little long, it shows some interesting tricks. Notice we created a function to print out the directory list rather than printing out the list of files returned. We also checked the status of the file to determine if the file was a directory or not and if it was, we printed a d to signal the name refers to a directory. We also used the stdout stream to control formatting with tabs ( ) and newline ( ) characters.

You will also see how to use change directory to improve how we use the import statement. Since we changed the directory to examples, the import ujson_example will attempt to find that module (library) in the current directory first. This is a nice trick you can use in your own projects to deploy your code to your board in a separate directory, which means you can use this to deploy multiple projects to your board placing each in its own directory.

Now let’s see the output. Listing 5-6 shows the output when run on the WiPy. Take a few moments and look through the output to see how the functions worked. Also, note the output from the JSON example is shown since we imported that code in the middle of the script. Nice.

Listing 5-6. Demonstration of the uos library features (output)

>>> import examples.uos_example

Show Files Output:
        name    size
        main.py 34
d       sys
d       lib
d       cert
        boot.py 336
d       examples
        data.bin        15
        my_vehicles.json        377

Show Files Output:
        name    size
        sys_example.py  395
        uio_example.py  609
        ujson_example.py        1370
        uos_example.py  1163
        data.bin        15

Run the ujson_example with import ujson_example after chdir()
Year Make Model Color
2015 Chevrolet Silverado Pull me over red
2009 Yamaha R1 Blue/Silver
1997 SeaDoo Speedster White
2013 TaoJen Sicily Black

Show Files Output:
        name    size
        sys_example.py  395
        uio_example.py  609
        ujson_example.py        1370
        uos_example.py  1163
        data.bin        15
        my_vehicles.json        377
d       test

utime

The utime library is another popular library used in many projects. The library is used to get the current time, date, and working with time data such as calculating time differentials. Note that some of the functions only work with a real-time clock (RTC) installed either as a hardware extension or through a network time server. See Chapter 6 for more details. The following lists several of the more commonly used functions related to inserting delays in our scripts. Delays are helpful when we need to pause processing to allow a sensor to read or to wait for data from/to other sources. Another common use of this library is recording the date and time of an event or sensor data read.

• utime.sleep(seconds): put the board in a sleep mode for the number of seconds specified

• utime.sleep_ms(ms): put the board in a sleep mode for the number of milliseconds specified

• utime.sleep_us(us): put the board in a sleep mode for the number of microseconds specified

Let’s see a short example of how to use time delays. Here, we use a random function to sleep for a random period and to provide random values. Don’t worry about the RTC code; we will see more about that in Chapter 6.

Listing 5-7. Demonstration of the utime library sleep features

# MicroPython for the IOT - Chapter 5
# Example use of the utime library
# Note: This example only works on the WiPy
# Note: This example will not on your PC.
import machine
import sys
import utime

# Since we don't have a RTC, we have to set the current time
# initialized the RTC in the WiPy machine library
from machine import RTC

# Init with default time and date
rtc = RTC()
# Init with a specific time and date. We use a specific
# datetime, but we could get this from a network time
# service.
rtc = RTC(datetime=(2017, 7, 15, 21, 32, 11, 0, None))

# Get a random number from 0-1 from a 2^24 bit random value
def get_rand():
    return machine.rng() / (2 ** 24 - 1)

# Format the time (epoch) for a better view
def format_time(tm_data):
    # Use a special shortcut to unpack tuple: *tm_data
    return "{0}-{1:0>2}-{2:0>2} {3:0>2}:{4:0>2}:{5:0>2}".format(*tm_data)

# Generate a random number of rows from 0-25
num_rows = get_rand() * 25

# Print a row using random seconds, milleseconds, or microseconds
# to simulate time.
print("Datetime             Value")
print("-------------------  --------")
for i in range(0,num_rows):
    # Generate random value for our sensor
    value = get_rand() * 100
    # Wait a random number of seconds for time
    utime.sleep(int(get_rand() * 15))  # sleep up to 15 seconds
    print("{0}  {1:0>{width}.4f}".format(format_time(rtc.now()), value, width=8))

Notice there is a lot going on in this example than simply waiting a random number of seconds. This example shows you how to do that but also how to work with time data and formatting data. You typically must do this when logging data. Let’s walk through this a bit.

Aside from the RTC code, we create two functions: one to generate a random number using the machine library of the WiPy (this function is not available in the Pyboard firmware), and another to format the datetime in a user-friendly format.

Since the function creates a random number of 24 bits, we divide by the max value for 24 bits to give a value between 0 and 1. We can then multiply that by an integer to get a value for a range. For example, if we wanted a value between 0 and 6, we can use get_rand() * 6.

The format_time() function returns a string representing the datetime. Here we see some advanced formatting options to format the date with the correct digits per part. Specifically, 4 digits for year and 2 for month, day, hour, minute, and second. The :0>2 option tells the format() function to space the value over 2 positions (digits) with leading zeros. Cool, eh? Notice the last print() statement. Notice we use another trick to pass the width with a named parameter (width).

Finally, we get to the example code where we generate a random number of lines from 0 to 25, then loop for those iterations generating a random value (0-100) and then wait (sleep) for a random number of seconds (0-15) then print the new datetime and the value. This simulates a typical loop we would use to read data from sensors. As most sensors need time to read the value, the sleep function allows us to wait for that period.¹

Now that we know more about what this code is doing, let’s see an example of how it works. The following shows the output from running this code on the WiPy.

>>> import examples.utime_example
Datetime             Value
-------------------  --------
2017-07-15 21:32:12  014.1520
2017-07-15 21:32:25  003.5380
2017-07-15 21:32:28  044.8298
2017-07-15 21:32:35  099.0981
2017-07-15 21:32:41  086.8839
2017-07-15 21:32:47  083.8304
2017-07-15 21:32:55  083.0670
2017-07-15 21:32:59  064.3214

There are also built-in functions that are not part of any specific library and there are exceptions that allow us to capture error conditions. Let’s look at those before we dive into some of the more commonly used standard libraries.

machine

The machine library contains functions related to the hardware, providing an abstraction layer that you can write code to interact with the hardware. Thus, this library is the main library you will use to access features like timers, communication protocols, CPU, and more. Since this functionality is communicating directly with the hardware, you should take care when experimenting to avoid changing or even potentially damaging the performance or configuration of your board. For example, using the library incorrectly could lead to lockups, reboots, or crashes.

Since the machine library is a low-level hardware abstraction, we will not cover it in-depth in this chapter. Rather, we will see more of the hardware features in the next chapter. In the meantime, let’s explore another interesting gem of MicroPython knowledge by showing you how to discover what a library contains through the help function. For example, Listing 5-8 shows an excerpt of what is reported through the REPL console when we issue the statement help(machine) on the WiPy when connected to the Expansion Board. The help() function will display all the functions and constants that are available for use in the library. While it doesn’t replace a detailed explanation or even a complete example, it can be useful when encountering a library for the first time.

Listing 5-8. The machine Library Help

>>> import machine
>>> help(machine)
<module 'umachine'>object  is of type module
  __name__ -- umachine
  mem8 -- <8-bit memory>
  mem16 -- <16-bit memory>
  mem32 -- <32-bit memory>
  reset -- <function>
...
  Pin -- <class 'Pin'>
  UART -- <class 'UART'>
  SPI -- <class 'SPI'>
  I2C -- <class 'I2C'>
  PWM -- <class 'PWM'>
  ADC -- <class 'ADC'>
  DAC -- <class 'DAC'>
  SD -- <class 'SD'>
  Timer -- <class 'Timer'>
  RTC -- <class 'RTC'>
  WDT -- <class 'WDT'>
  PWRON_RESET -- 0
  HARD_RESET -- 1
  WDT_RESET -- 2
...

Notice there is a lot of information there! What this gives us most is the list of classes we can use to interact with the hardware. Here we see there are classes for UART, SPI, I2C, PWM, and more. Let’s contrast that to the same output from the Pyboard. The following shows the same excerpt of classes for the Pyboard.

  Pin -- <class 'Pin'>
  Signal -- <class 'Signal'>
  I2C -- <class 'I2C'>
  SPI -- <class 'SPI'>
  UART -- <class 'UART'>
  WDT -- <class 'WDT'>

Notice there are a few missing. We will explore some of these differences in the next chapter. It is always a good idea to check the output of help(machine) on a board you’re using for the first time. It may save you a lot of headaches trying to find support for hardware that doesn’t exist!

network

The network library on the Pyboard is used to install network drivers (classes for interacting with the networking abstraction) and configuring the settings. While the other libraries discussed are a little different from one board to another, this library is one that is quite different among the Pyboard and WiPy (and other) boards. In fact, the WiPy has the same library but contains different classes.

This is largely since the WiPy has built-in WiFi, it has its own mechanisms for working with networks. Recall we saw the network library for the Pyboard demonstrated briefly in Chapter 3 and we will see it used again later in the example chapters. That is, we learned there are only two network drivers for the Pyboard (and similar) boards: CC3K and WINNET5K. The WiPy network library contains three classes: WLAN, Bluetooth, and Server. Recall also we saw an example of the WLAN class in Chapter 3.

We will see more about the networking classes and the intriguing Bluetooth class in the next chapter.