Arduino constants

The constant kConsecutiveInferenceThresholds is redefined in the file arduino/constants.cc:

// The number of expected consecutive inferences for each gesture type.
// Established with the Arduino Nano 33 BLE Sense.
const int kConsecutiveInferenceThresholds[3] = {8, 5, 4};

As mentioned earlier in the chapter, this constant stores the number of consecutive positive inferences required for each gesture to be considered detected. The number depends on how many inferences are run per second, which varies per device. Because the default numbers were calibrated for the SparkFun Edge, the Arduino implementation needs its own set of numbers. You can modify these thresholds to make inference more difficult or easier to trigger, but setting them too low will result in false positives.

Capturing accelerometer data on Arduino

The Arduino accelerometer handler is located in arduino/accelerometer_handler.cc. It has the task of capturing data from the accelerometer and writing it to the model’s input buffer.

The model we are using was trained using data from the SparkFun Edge board. The Edge’s accelerometer provides a set of readings at a rate of 25 Hz, or 25 times per second. To work correctly, it needs to be fed data that is captured at the same rate. As it turns out, the accelerometer on the Arduino Nano 33 BLE Sense board returns measurements at a rate of 119 Hz. This means that in addition to capturing data, we need to downsample it to suit our model.

Although it sounds very technical, downsampling is actually pretty easy. To reduce the sample rate of a signal, we can just throw away some of the data. We look at how this works in the following code.

First the implementation includes its own header file, along with some others:

#include "tensorflow/lite/micro/examples/magic_wand/
  accelerometer_handler.h"

#include <Arduino.h>
#include <Arduino_LSM9DS1.h>

#include "tensorflow/lite/micro/examples/magic_wand/constants.h"

The file Arduino.h provides access to some basic features of the Arduino platform. The file Arduino_LSM9DS1.h is part of the Arduino_LSM9DS1 library, which we’ll be using to communicate with the board’s accelerometer.

Next, we set up some variables:

// A buffer holding the last 200 sets of 3-channel values
float save_data[600] = {0.0};
// Most recent position in the save_data buffer
int begin_index = 0;
// True if there is not yet enough data to run inference
bool pending_initial_data = true;
// How often we should save a measurement during downsampling
int sample_every_n;
// The number of measurements since we last saved one
int sample_skip_counter = 1;

These include a buffer we’ll be filling with our data, save_data, along with some variables for tracking our current position in the buffer and whether we have enough data to start running inference. The most interesting two variables, sample_every_n and sample_skip_counter, are used in the downsampling process. We’ll look at this more closely in a moment.

Next in the file, the SetupAccelerometer() function is called by the program’s main loop to get the board ready to capture data:

TfLiteStatus SetupAccelerometer(tflite::ErrorReporter* error_reporter) {
  // Wait until we know the serial port is ready
  while (!Serial) {
  }

  // Switch on the IMU
  if (!IMU.begin()) {
    error_reporter->Report("Failed to initialize IMU");
    return kTfLiteError;
  }

Because we’ll be outputting a message to indicate that everything is ready to go, the first thing it does is make sure that the device’s serial port is ready. It then switches on the inertial measurement unit (IMU), which is the electronic component that contains the accelerometer. The IMU object comes from the Arduino_LSM9DS1 library.

The next step is to start thinking about downsampling. We first query the IMU library to determine the board’s sample rate. When we have that number, we divide it by our target sample rate, which is defined in kTargetHz as part of constants.h:

  // Determine how many measurements to keep in order to
  // meet kTargetHz
  float sample_rate = IMU.accelerationSampleRate();
  sample_every_n = static_cast<int>(roundf(sample_rate / kTargetHz));

Our target rate is 25 Hz, and the board’s sample rate is 119 Hz; thus, the result of our division is 4.76. This lets us know how many of the 119 Hz samples we need to keep in order to attain the target sample rate of 25 Hz: 1 sample in every 4.76.

Because keeping a fractional number of samples is difficult, we use the roundf() function to round to the nearest number, 5. To downsample our signal, then, we need to keep one in every five measurements. This will result in an effective sample rate of 23.8 Hz, which is a close enough approximation that our model should work well. We store this value in the sample_every_n variable for use later.

Now that we’ve established the parameters of our downsampling, we give the user a message to inform them that the application is ready to go and then return from the SetupAccelerometer() function:

  error_reporter->Report("Magic starts!");

  return kTfLiteOk;
}

Next up, we define ReadAccelerometer(). This function is tasked with capturing new data and writing it to the model’s output tensor. It begins with some code that is used to clear its internal buffer after a gesture has been successfully recognized, cleaning the slate for any subsequent gestures:

bool ReadAccelerometer(tflite::ErrorReporter* error_reporter, float* input,
                       int length, bool reset_buffer) {
  // Clear the buffer if required, e.g. after a successful prediction
  if (reset_buffer) {
    memset(save_data, 0, 600 * sizeof(float));
    begin_index = 0;
    pending_initial_data = true;
  }

Next, we use the IMU library to check for available data in a loop. If there’s data available, we read it:

  // Keep track of whether we stored any new data
  bool new_data = false;
  // Loop through new samples and add to buffer
  while (IMU.accelerationAvailable()) {
    float x, y, z;
    // Read each sample, removing it from the device's FIFO buffer
    if (!IMU.readAcceleration(x, y, z)) {
      error_reporter->Report("Failed to read data");
      break;
    }

The accelerometer on the Arduino Nano 33 BLE Sense board is equipped with something called a FIFO buffer. This is a special memory buffer, located on the accelerometer itself, which holds the most recent 32 measurements. Because it’s part of the accelerometer hardware, the FIFO buffer continues to accrue measurements even while our application code is running. If it weren’t for the FIFO buffer, we might lose a lot of data, meaning we wouldn’t have an accurate record of the gestures being made.

When we call IMU.accelerationAvailable(), we are querying the accelerometer to see whether new data is available in its FIFO buffer. Using our loop, we continue to read all the data from the buffer until there is none remaining.

Next up, we implement our super-simple downsampling algorithm:

    // Throw away this sample unless it's the nth
    if (sample_skip_counter != sample_every_n) {
      sample_skip_counter += 1;
      continue;
    }

Our approach is to keep one in every n samples, where n is the number stored in sample_every_n. To do this, we maintain a counter, sample_skip_counter, which lets us know how many samples have been read since the last one we kept. For every measurement we read, we check whether it is the nth. If it isn’t, we continue the loop without writing the data anywhere, effectively throwing it away. This simple process leads to our data being downsampled.

If execution gets further than this point, we’re planning on keeping the data. To do this, we write it to consecutive positions in our save_data buffer:

    // Write samples to our buffer, converting to milli-Gs
    // and flipping y and x order for compatibility with
    // model (sensor orientation is different on Arduino
    // Nano BLE Sense compared with SparkFun Edge)
    save_data[begin_index++] = y * 1000;
    save_data[begin_index++] = x * 1000;
    save_data[begin_index++] = z * 1000;

Our model accepts accelerometer measurements in the order x, y, z. You’ll notice here that we’re writing the y value to the buffer before the x. This is because our model was trained on data captured on the SparkFun Edge board, whose accelerometer has its axes pointing in different physical directions to the one on the Arduino. This difference means that the SparkFun Edge’s x-axis is equivalent to the Arduino’s y-axis, and vice versa. By swapping these axes’ data in our code, we can make sure our model is being fed data that it can understand.

The final few lines of our loop do some housework, setting some state variables that are used in our loop:

    // Since we took a sample, reset the skip counter
    sample_skip_counter = 1;
    // If we reached the end of the circle buffer, reset
    if (begin_index >= 600) {
      begin_index = 0;
    }
    new_data = true;
  }

We reset our downsampling counter, make sure we don’t run off the end of our sample buffer, and set a flag to indicate that new data has been saved.

After grabbing this new data, we do some more checks. This time, we’re making sure that we have sufficient data to perform an inference. If not, or if new data was not captured this time around, we return from the function without doing anything:

  // Skip this round if data is not ready yet
  if (!new_data) {
    return false;
  }

  // Check if we are ready for prediction or still pending more initial data
  if (pending_initial_data && begin_index >= 200) {
    pending_initial_data = false;
  }

  // Return if we don't have enough data
  if (pending_initial_data) {
    return false;
  }

By returning false when there’s no new data, we make sure the calling function knows not to bother running inference.

If we got this far, we’ve obtained some new data. We copy the appropriate amount of data, including our new samples, to the input tensor:

  // Copy the requested number of bytes to the provided input tensor
  for (int i = 0; i < length; ++i) {
    int ring_array_index = begin_index + i - length;
    if (ring_array_index < 0) {
      ring_array_index += 600;
    }
    input[i] = save_data[ring_array_index];
  }

  return true;
}

And that’s it! We’ve populated the input tensor and are ready to run inference. After inference has been run, the results are passed into the gesture predictor, which determines whether a valid gesture has been spotted. The result is passed into the output handler, which we walk through next.

Responding to gestures on Arduino

The output handler is defined in arduino/output_handler.cc. It’s nice and simple: all it does is log information to the serial port depending on which gesture was detected, and toggle the board’s LED each time inference is run.

The first time the function runs, the LED is configured for output:

void HandleOutput(tflite::ErrorReporter* error_reporter, int kind) {
  // The first time this method runs, set up our LED
  static bool is_initialized = false;
  if (!is_initialized) {
    pinMode(LED_BUILTIN, OUTPUT);
    is_initialized = true;
  }

Next, the LED is toggled on and off with each inference:

  // Toggle the LED every time an inference is performed
  static int count = 0;
  ++count;
  if (count & 1) {
    digitalWrite(LED_BUILTIN, HIGH);
  } else {
    digitalWrite(LED_BUILTIN, LOW);
  }

Finally, we print some beautiful ASCII art depending on which gesture was matched:

  // Print some ASCII art for each gesture
  if (kind == 0) {
    error_reporter->Report(
        "WING:

*         *         *

 *       * *       "
        "*

  *     *   *     *

   *   *     *   *

    * *       "
        "* *

     *         *

");
  } else if (kind == 1) {
    error_reporter->Report(
        "RING:

          *

       *     *

     *         *

 "
        "   *           *

     *         *

       *     *

      "
        "    *

");
  } else if (kind == 2) {
    error_reporter->Report(
        "SLOPE:

        *

       *

      *

     *

    "
        "*

   *

  *

 * * * * * * * *

");
  }

It’s difficult to read now, but you’ll be rewarded with the output’s full glory when you deploy the application to your board.

Capturing accelerometer data on SparkFun Edge

The code that captures accelerometer data is located in sparkfun_edge/accelerometer_handler.cc. A lot of it is device-specific, but we’ll skip over the implementation details and focus on the important stuff.

The first step involved with capturing accelerometer data is configuring the hardware. The SetupAccelerometer() function kicks this off by setting various low-level parameters required by the accelerometer:

TfLiteStatus SetupAccelerometer(tflite::ErrorReporter* error_reporter) {
  // Set the clock frequency.
  am_hal_clkgen_control(AM_HAL_CLKGEN_CONTROL_SYSCLK_MAX, 0);

  // Set the default cache configuration
  am_hal_cachectrl_config(&am_hal_cachectrl_defaults);
  am_hal_cachectrl_enable();

  // Configure the board for low power operation.
  am_bsp_low_power_init();

  // Collecting data at 25Hz.
  int accInitRes = initAccelerometer();

You’ll notice a call to a function named initAccelerometer(). This is defined in the SparkFun Edge BSP’s accelerometer example, which is pulled down as a dependency when our project is built. It performs various tasks to switch on and configure the board’s accelerometer.

After the accelerometer is running, we enable its FIFO buffer. This is a special memory buffer, located on the accelerometer itself, which holds the last 32 datapoints. By enabling it, we’re able to continue collecting accelerometer measurements even while our application code is busy running inference. The remainder of the function sets up the buffer and logs errors if anything goes wrong:

  // Enable the accelerometer's FIFO buffer.
  // Note: LIS2DH12 has a FIFO buffer which holds up to 32 data entries. It
  // accumulates data while the CPU is busy. Old data will be overwritten if
  // it's not fetched in time, so we need to make sure that model inference is
  // faster than 1/25Hz * 32 = 1.28s
  if (lis2dh12_fifo_set(&dev_ctx, 1)) {
    error_reporter->Report("Failed to enable FIFO buffer.");
  }

  if (lis2dh12_fifo_mode_set(&dev_ctx, LIS2DH12_BYPASS_MODE)) {
    error_reporter->Report("Failed to clear FIFO buffer.");
    return 0;
  }

  if (lis2dh12_fifo_mode_set(&dev_ctx, LIS2DH12_DYNAMIC_STREAM_MODE)) {
    error_reporter->Report("Failed to set streaming mode.");
    return 0;
  }

  error_reporter->Report("Magic starts!");

  return kTfLiteOk;
}

When we’re done with initialization, we can call the ReadAccelerometer() function to get the latest data. This will happen between every inference.

First, if the reset_buffer argument is true, ReadAccelerometer() performs a reset of its data buffer. This is done after a valid gesture has been detected in order to provide a clean slate for further gestures. As part of this process, we use am_util_delay_ms() to make our code wait for 10 ms. Without this delay, the code often hangs when reading new data (as of this writing, the cause was unclear, but the TensorFlow open source project welcomes pull requests if you determine a better fix):

bool ReadAccelerometer(tflite::ErrorReporter* error_reporter, float* input,
                       int length, bool reset_buffer) {
  // Clear the buffer if required, e.g. after a successful prediction
  if (reset_buffer) {
    memset(save_data, 0, 600 * sizeof(float));
    begin_index = 0;
    pending_initial_data = true;
    // Wait 10ms after a reset to avoid hang
    am_util_delay_ms(10);
  }

After resetting the main buffer, ReadAccelerometer() checks whether there is any new data available in the accelerometer’s FIFO buffer. If there’s nothing available yet, we just return from the function:

  // Check FIFO buffer for new samples
  lis2dh12_fifo_src_reg_t status;
  if (lis2dh12_fifo_status_get(&dev_ctx, &status)) {
    error_reporter->Report("Failed to get FIFO status.");
    return false;
  }

  int samples = status.fss;
  if (status.ovrn_fifo) {
    samples++;
  }

  // Skip this round if data is not ready yet
  if (samples == 0) {
    return false;
  }

Our application’s main loop will continue calling, meaning as soon as there’s data available, we can move past this point.

The next part of the function loops through the new data and stores it in another, larger buffer. First we set up a special struct of type axis3bit16_t, designed to hold accelerometer data. We then call lis2dh12_acceleration_raw_get() to fill it with the next available measurement. This function will return zero if it fails, at which point we display an error:

  // Load data from FIFO buffer
  axis3bit16_t data_raw_acceleration;
  for (int i = 0; i < samples; i++) {
    // Zero out the struct that holds raw accelerometer data
    memset(data_raw_acceleration.u8bit, 0x00, 3 * sizeof(int16_t));
    // If the return value is non-zero, sensor data was successfully read
    if (lis2dh12_acceleration_raw_get(&dev_ctx, data_raw_acceleration.u8bit)) {
      error_reporter->Report("Failed to get raw data.");

If the measurement was obtained successfully, we convert it into milli-Gs, the unit of measurement expected by the model, and then write it into save_data[], which is an array we’re using as a buffer to store values that we’ll use for inference. The values for each axis of the accelerometer are stored consecutively:

    } else {
      // Convert each raw 16-bit value into floating point values representing
      // milli-Gs, a unit of acceleration, and store in the current position of
      // our buffer
      save_data[begin_index++] =
          lis2dh12_from_fs2_hr_to_mg(data_raw_acceleration.i16bit[0]);
      save_data[begin_index++] =
          lis2dh12_from_fs2_hr_to_mg(data_raw_acceleration.i16bit[1]);
      save_data[begin_index++] =
          lis2dh12_from_fs2_hr_to_mg(data_raw_acceleration.i16bit[2]);
      // Start from beginning, imitating loop array.
      if (begin_index >= 600) begin_index = 0;
    }
  }

Our save_data[] array can store 200 sets of three-axis values, so we set our begin_index counter back to 0 when it reaches 600.

We’ve now incorporated all of the new data into our save_data[] buffer. Next, we check whether we have enough data to make a prediction. When testing the model, it was discovered that around a third of our total buffer size is the bare minimum amount of data that results in a reliable prediction; therefore, if we have at least this much data, we set the pending_initial_data flag to false (it defaults to true):

  // Check if we are ready for prediction or still pending more initial data
  if (pending_initial_data && begin_index >= 200) {
    pending_initial_data = false;
  }

Next, if there is still insufficient data to run an inference, we return false:

  // Return if we don't have enough data
  if (pending_initial_data) {
    return false;
  }

If we got this far, there’s sufficient data in the buffer to run an inference. The final part of the function copies the requested data from the buffer into the input argument, which is a pointer to the model’s input tensor:

  // Copy the requested number of bytes to the provided input tensor
  for (int i = 0; i < length; ++i) {
    int ring_array_index = begin_index + i - length;
    if (ring_array_index < 0) {
      ring_array_index += 600;
    }
    input[i] = save_data[ring_array_index];
  }
  return true;

The variable length is an argument passed into ReadAccelerometer() that determines how much data should be copied. Because our model takes 128 three-axis readings as its input, the code in main_functions.cc calls ReadAccelerometer() with a length of 384 (128 * 3).

At this point, our input tensor is filled with fresh accelerometer data. Inference will be run, the results will be interpreted by the gesture predictor, and the result will be passed to the output handler to display to the user.

Responding to gestures on SparkFun Edge

The output handler, located in sparkfun_edge/output_handler.cc, is very simple. The first time it runs, we configure the LEDs for output:

void HandleOutput(tflite::ErrorReporter* error_reporter, int kind) {
  // The first time this method runs, set up our LEDs correctly
  static bool is_initialized = false;
  if (!is_initialized) {
    am_hal_gpio_pinconfig(AM_BSP_GPIO_LED_RED, g_AM_HAL_GPIO_OUTPUT_12);
    am_hal_gpio_pinconfig(AM_BSP_GPIO_LED_BLUE, g_AM_HAL_GPIO_OUTPUT_12);
    am_hal_gpio_pinconfig(AM_BSP_GPIO_LED_GREEN, g_AM_HAL_GPIO_OUTPUT_12);
    am_hal_gpio_pinconfig(AM_BSP_GPIO_LED_YELLOW, g_AM_HAL_GPIO_OUTPUT_12);
    is_initialized = true;
  }

Next, we toggle the yellow LED with each inference:

  // Toggle the yellow LED every time an inference is performed
  static int count = 0;
  ++count;
  if (count & 1) {
    am_hal_gpio_output_set(AM_BSP_GPIO_LED_YELLOW);
  } else {
    am_hal_gpio_output_clear(AM_BSP_GPIO_LED_YELLOW);
  }

After that, we check which gesture was detected. For each individual gesture, we light an LED, clear all the others, and output some beautiful ASCII art via the serial port. Here’s the code that handles the “wing” gesture:

  // Set the LED color and print a symbol (red: wing, blue: ring, green: slope)
  if (kind == 0) {
    error_reporter->Report(
        "WING:

*         *         *

 *       * *       "
        "*

  *     *   *     *

   *   *     *   *

    * *       "
        "* *

     *         *

");
    am_hal_gpio_output_set(AM_BSP_GPIO_LED_RED);
    am_hal_gpio_output_clear(AM_BSP_GPIO_LED_BLUE);
    am_hal_gpio_output_clear(AM_BSP_GPIO_LED_GREEN);

On a serial port monitor, the output will look like this:

WING:
*         *         *
 *       * *       *
  *     *   *     *
   *   *     *   *
    * *       * *
     *         *

A different serial output and LED are used for each gesture.

Running the example

We’ve now seen how the SparkFun Edge code works. Next, let’s get it running on our hardware.

To build and deploy our code, we’ll need the following:

• A SparkFun Edge board with the Himax HM01B0 breakout attached

• A USB programmer (we recommend the SparkFun Serial Basic Breakout, which is available in micro-B USB and USB-C variants)

• A matching USB cable

• Python 3 and some dependencies

Open a terminal window, clone the TensorFlow repository, and then change into its directory:

git clone https://github.com/tensorflow/tensorflow.git
cd tensorflow

Next, we’re going to build the binary and run some commands that get it ready for downloading to the device. To avoid some typing, you can copy and paste these commands from README.md.

make -f tensorflow/lite/micro/tools/make/Makefile 
 TARGET=sparkfun_edge magic_wand_bin

tensorflow/lite/micro/tools/make/gen/
 sparkfun_edge_cortex-m4/bin/magic_wand.bin

test -f tensorflow/lite/micro/tools/make/gen/sparkfun_edge_ 
  cortex-m4/bin/magic_wand.bin &&  echo "Binary was successfully created" || 
  echo "Binary is missing"

cp tensorflow/lite/micro/tools/make/downloads/AmbiqSuite-Rel2.0.0/ 
  tools/apollo3_scripts/keys_info0.py
  tensorflow/lite/micro/tools/make/downloads/AmbiqSuite-Rel2.0.0/ 
  tools/apollo3_scripts/keys_info.py

python3 tensorflow/lite/micro/tools/make/downloads/ 
  AmbiqSuite-Rel2.0.0/tools/apollo3_scripts/create_cust_image_blob.py 
  --bin tensorflow/lite/micro/tools/make/gen/ 
  sparkfun_edge_cortex-m4/bin/micro_vision.bin 
  --load-address 0xC000 
  --magic-num 0xCB 
  -o main_nonsecure_ota 
  --version 0x0

python3 tensorflow/lite/micro/tools/make/downloads/ 
AmbiqSuite-Rel2.0.0/tools/apollo3_scripts/create_cust_wireupdate_blob.py 
--load-address 0x20000 
--bin main_nonsecure_ota.bin 
-i 6 
-o main_nonsecure_wire 
--options 0x1

Attach the programmer to the board

To download new programs to the board, you’ll use the SparkFun USB-C Serial Basic serial programmer. This device allows your computer to communicate with the microcontroller via USB.

To attach this device to your board, perform the following steps:

1. On the side of the SparkFun Edge, locate the six-pin header.

2. Plug the SparkFun USB-C Serial Basic into these pins, ensuring that the pins labeled BLK and GRN on each device are lined up correctly.

You can see the correct arrangement in Figure 11-11.

Figure 11-11. Connecting the SparkFun Edge and USB-C Serial Basic (image courtesy of SparkFun)

# macOS:
ls /dev/cu*

# Linux:
ls /dev/tty*

/dev/cu.Bluetooth-Incoming-Port
/dev/cu.MALS
/dev/cu.SOC

# macOS:
ls /dev/cu*

# Linux:
ls /dev/tty*

/dev/cu.Bluetooth-Incoming-Port
/dev/cu.MALS
/dev/cu.SOC
/dev/cu.wchusbserial-1450

export DEVICENAME=<your device name here>

Run the script to flash your board

To flash the board, you need to put it into a special “bootloader” state that prepares it to receive the new binary. You can then run a script to send the binary to the board.

First create an environment variable to specify the baud rate, which is the speed at which data will be sent to the device:

export BAUD_RATE=921600

Now paste the command that follows into your terminal—but do not press Enter yet!. The ${DEVICENAME} and ${BAUD_RATE} in the command will be replaced with the values you set in the previous sections. Remember to substitute python3 with python if necessary:

python3 tensorflow/lite/micro/tools/make/downloads/ 
  AmbiqSuite-Rel2.0.0/tools/apollo3_scripts/uart_wired_update.py -b 
  ${BAUD_RATE} ${DEVICENAME} -r 1 -f main_nonsecure_wire.bin -i 6

Next you’ll reset the board into its bootloader state and flash the board.

On the board, locate the buttons marked RST and 14, as shown in Figure 11-12.

Figure 11-12. The SparkFun Edge’s buttons

Perform the following steps:

1. Ensure that your board is connected to the programmer, and the entire thing is connected to your computer via USB.

2. On the board, press and hold the button marked 14. Continue holding it.

3. While still holding the button marked 14, press the button marked RST to reset the board.

4. Press Enter on your computer to run the script. Continue holding button 14.

You should now see something like the following appearing on your screen:

Connecting with Corvette over serial port /dev/cu.usbserial-1440...
Sending Hello.
Received response for Hello
Received Status
length =  0x58
version =  0x3
Max Storage =  0x4ffa0
Status =  0x2
State =  0x7
AMInfo =
0x1
0xff2da3ff
0x55fff
0x1
0x49f40003
0xffffffff
[...lots more 0xffffffff...]
Sending OTA Descriptor =  0xfe000
Sending Update Command.
number of updates needed =  1
Sending block of size  0x158b0  from  0x0  to  0x158b0
Sending Data Packet of length  8180
Sending Data Packet of length  8180
[...lots more Sending Data Packet of length  8180...]

Keep holding button 14 until you see Sending Data Packet of length 8180. You can release the button after seeing this (but it’s okay if you keep holding it).

The program will continue to print lines on the terminal. Eventually, you’ll see something like the following:

[...lots more Sending Data Packet of length  8180...]
Sending Data Packet of length  8180
Sending Data Packet of length  6440
Sending Reset Command.
Done.

This indicates a successful flashing.

Tip

If the program output ends with an error, check whether Sending Reset Command. was printed. If so, flashing was likely successful despite the error. Otherwise, flashing might have failed. Try running through these steps again (you can skip over setting the environment variables).

Testing the Program

Start by pressing the RST button to make sure the program is running. When the program is running, the yellow LED will toggle on and off, once for each inference.

Next, use the following command to start printing the serial output of the device:

screen ${DEVICENAME} 115200

You should initially see the following output:

Magic starts!

You can now try to make some gestures. Hold the board up with one hand, with the components facing up and the USB adapter facing toward the left, as shown in Figure 11-13.

Figure 11-13. How to hold the board while performing gestures

Figure 11-14 presents a diagram showing how to perform each gesture. Because the model was trained on data collected when the board was attached to a wand, you might need a few tries to get them to work.

Figure 11-14. The three magic wand gestures

The easiest one to start with is “wing.” You should move your hand quickly enough that it takes around one second to perform the gesture. If you’re successful, the red LED should illuminate, and you should see the following output:

WING:
*         *         *
 *       * *       *
  *     *   *     *
   *   *     *   *
    * *       * *
     *         *

Congratulations, you’ve cast your first magic spell using the SparkFun Edge!

Next try the “ring” gesture, by tracing a clockwise circle with your hand (or the tip of your wand). Again, aim to take around a second to perform the gesture. You should see the following appear, as if by magic:

RING:
          *
       *     *
     *         *
    *           *
     *         *
       *     *
          *

For the final gesture, trace the corner of a triangle in the air. It’s best described by its ASCII art demonstration, shown here:

SLOPE:
        *
       *
      *
     *
    *
   *
  *
 * * * * * * * *

Like any good magic spells, you might have to practice these a bit before you can perform them perfectly each time. You can see video demonstrations of the gestures in README.md.

Making your own changes

Now that you’ve deployed the basic application, try playing around and making some changes. You can find the application’s code in the tensorflow/lite/micro/examples/magic_wand folder. Just edit and save, and then repeat the previous instructions to deploy your modified code to the device.

Here are a few things you could try:

• Experiment with the threshold values in constants.cc to make the gestures easier or more difficult to perform (at the cost of more false positives or negatives).

• Write a program on your computer that lets you perform tasks using physical gestures.

• Extend the program to transmit detection results via Bluetooth. There’s an example of how to do this in the Ambiq SDK, in AmbiqSuite-Rel2.0.0/boards/apollo3_evb/examples/uart_ble_bridge. When the magic wand application is built, the SDK is downloaded to tensorflow/tensorflow/lite/micro/tools/make/downloads/AmbiqSuite-Rel2.0.0.