Problem

You want to see what is happening under the covers when you compile and upload a sketch.

Solution

You can choose to display all the command-line activity that takes place when compiling or uploading a sketch through the Preferences dialog . Select File→Preferences (Linux, Windows) or Arduino→Preferences (macOS) to display the dialog box to check or uncheck the boxes to enable verbose output for compile or upload messages. You can also choose whether to enable compiler warnings and how verbose you want those warnings (None, Default, More, All).

Discussion

When you click on Compile or Upload, a lot of activity happens that is not usually displayed on-screen. The command-line tools that the Arduino IDE was built to hide are used to compile, link, and upload your code to the board.

First your sketch file(s) are transformed into a file suitable for the compiler (AVR-GCC) to process. All source files in the sketch folder that have the default Arduino (.ino) file extension are joined together to make one file. All files that end in .c or .cpp are compiled separately. Header files (with an .h extension) are ignored unless they are explicitly included in the files that are being joined.

#include "Arduino.h" (WProgram.h in previous releases) is added at the top of the file to include the header file with all the Arduino-specific code definitions, such as digitalWrite() and analogRead(). If you want to examine the contents of that file, change to the directory where Arduino was installed; from there, you can navigate to hardware/arduino/avr/cores/arduino to find the header files

On the Mac, right-click the Arduino application icon and select Show Package Contents from the drop-down menu. A folder will open; from the folder, navigate to Contents/Java/. You’ll be able to find the hardware/arduino/avr/cores/arduino folder there.

To make the code valid C++, the prototypes of any functions declared in your code are generated next and inserted.

Finally, the setting of the board menu is used to insert values (obtained from the boards.txt file) that define various constants used for the controller chips on the selected board.

This file is then compiled by AVR-GCC, which is included as part the Arduino IDE installation. It is in the hardware/tools/avr/bin folder under the Arduino installation (on macOS, the hardware directory is under Contents/Java/ folder within the Arduino app as described earlier).

The compiler produces a number of object files (files with an extension of .o that will be combined by the link tool). These files are stored in your temporary directory. In there, you’ll find directories such as arduino_build_137218 that contain all the build artifacts. You can determine your system temporary directory on Windows by checking the value of the TEMP environment variable: open a command prompt and run the command echo %TEMP%. On macOS, open a Terminal and run the command echo $TMPDIR. On Linux, you should be able to find build artifacts in /tmp.

The object files are then linked together to make a hex file to upload to the board. Avrdude, a utility for transferring files to the Arduino controller, is used to upload to the board.

The tools used to implement the AVR build process can be found in the hardware oolsavrin directory.

Another useful tool for experienced programmers is avr-objdumpwhich you can find in the hardware/tools/avr/bin folder under the Arduino install. It lets you see how the compiler turns the sketch into code that the controller chip runs. This tool produces a disassembly listing of your sketch that shows the object code intermixed with the source code. It can also display a memory map of all the variables used in your sketch. To use the tool, compile the sketch and navigate to the Arduino build folder (which will be a subdirectory of your temporary directory as described earlier). You can also find this by enabling and viewing verbose compiler output and looking for the directory name there. Navigate to the folder that contains the file with the .elf file extension. The file used by avr-objdump is the one with the extension .elf. For example, if you compile the Blink sketch you could view the compiled output (the machine code) by executing the following on the Windows command line (note the use of the PATH command to add the Arduino bin folder to the head of your PATH for this session only). You will need to change 706012 to the suffix of the arduino_build folder for the sketch you just compiled:

cd %TEMP%
cd arduino_build_706012
PATH "Program Files (x86)Arduinohardware	oolsavrin";%PATH%
avr-objdump -S Blink.ino.elf

On Linux, it will be more like this:

cd /tmp/arduino_build_700798/
PATH=~/arduino-1.8.9/hardware/tools/avr/bin/:$PATH
avr-objdump -S Blink.ino.elf

And for macOS:

cd $TMPDIR
cd arduino_build_97987/
PATH=/Applications/Arduino.app/Contents/Java/hardware/tools/avr/bin/:$PATH
avr-objdump -S Blink.ino.elf

It is convenient to direct the output to a file that can be read in a text editor. You can do this as follows:

avr-objdump -S Blink.ino.elf > blink.txt

You can add the -h option to add a list of section headers (helpful for determining memory usage):

avr-objdump -S -h Blink.ino.elf > blink.txt

For non-AVR boards, such as ARM-based boards or ESP8266, the Arduino core (a collection of tools, header files, and other files that support compilation on a particular hardware architecture) are stored under your home directory somewhere. On macOS, cores are stored in ~/Library/Arduino15/packages. On Windows, it’s %LOCALAPPDATA%Arduino15packages. On Linux, ~/.arduino15/packages. For example, at the time of this writing, the objdump file for SAMD boards is located in the arduino/tools/arm-none-eabi-gcc/4.8.3-2014q1/arm-none-eabi/bin/ folder beneath the packages folder:

PATH=~/.arduino15/packages/arduino/tools/arm-none-eabi-gcc/4.8.3-2014q1/bin:$PATH
arm-none-eabi-objdump -S Blink.ino.elf

If you have many cores installed, including cores from Adafruit, SparkFun, or the ESP8266 Community, there may be several ARM toolchains in there. The best way to determine which one goes with which core is to compile your sketch verbosely and look for the full path to the tools. When you see a command like arm-none-eabi-g++ -mcpu=cortex-m0plus or xtensa-lx106-elf-gcc -CC -E -P -DVTABLES_IN_FLASH scroll by, make note of the full path to the tool.

See Also

For information on the Arduino build process, see https://github.com/arduino/Arduino/wiki/Build-Process.

Problem

You want to be sure you have not run out of RAM. A sketch will not run correctly if there is insufficient memory, and this can be difficult to detect.

Solution

This recipe shows you how you can determine the amount of free memory available to your sketch. This sketch contains a function called memoryFree that reports the amount of available RAM on an AVR:

/*
 * Free memory sketch for AVR
 */

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  Serial.println(memoryFree());  // print the free memory
  delay(3000);
}

// Variable created by the build process when compiling the sketch
extern int *__brkval; // Pointer to last address allocated on the heap (or 0)

// function to return the amount of free RAM
int memoryFree()
{
  int freeValue; // This will be the most recent object allocated on the stack
  if((int)__brkval == 0) // Heap is empty; use start of heap
  {
    freeValue = ((int)&freeValue) - ((int)__malloc_heap_start);
  }
  else // Heap is not empty; use last heap address
  {
    freeValue = ((int)&freeValue) - ((int)__brkval);
  }
  return freeValue;
}

Discussion

The memoryFree function first declares a variable, freeValue, which is allocated on the stack because it’s local to this function. There are two primary areas of managed memory: the stack and heap. The stack resides at the end of free memory, and when your sketch calls functions, the stack usage grows (downward). The stack memory is used for function calls and storage for local variables. As your functions finish running, stack memory is released, and so stack usage shrinks. The heap resides at the beginning of free memory, and when your sketch (or a library) allocates memory (such as when you create a String object), the heap usage grows (upward). The stack grows toward the heap, and the heap grows toward the stack.

So, if you know the distance (in bytes) between the stack and heap, you know how much free memory Arduino has. The address of freeValue (&freeValue) is the address of the last byte of free memory (the start of the stack). The system variable __brkval contains the address of the first byte of free memory (aka the end of the heap), unless there is nothing currently allocated on the heap. In that case, it’s 0 (null), and won’t be of any help to us because there’s a lot of other things in memory before the heap. However, there’s another system variable, __malloc_heap_start that gives you the address of the start of the heap. If you know the heap is empty (0), use __malloc_heap_start instead.

For ARM, the sketch is a bit simpler. The general idea is the same, except for ARM, you can use the sbrk function, which is a low-level function for managing memory. If you call it with an argument of 0, it just returns the starting address of free RAM. The distance in bytes between the address of freeValue and the return value of sbrk(0) gives you the free RAM. sbrk is declared as a char pointer, so the sketch has to cast it to an int pointer to subtract it from the address of freeValue. The sketch uses reinterpret_cast rather than a regular cast [(int *)] because reinterpret_cast allows the use of conversions that may be considered unsafe in some configurations.

/*
 * Free memory sketch for ARM
 */

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  Serial.println(memoryFree());  // print the free memory
  delay(3000);
}

// Variable created by the build process when compiling the sketch
extern "C" char *sbrk(int incr); // Call with 0 to get start address of free ram

// function to return the amount of free RAM
int memoryFree()
{
  int freeValue; // This will be the most recent object allocated on the stack
  freeValue = &freeValue - reinterpret_cast<int*>(sbrk(0));

  return freeValue;
}

The number of bytes your code uses changes as the code runs. The important thing is to ensure that you don’t consume more memory than you have. Your stack and heap will grow as your program runs. If they crash into each other, you’re out of memory! If you make heavy usage of the heap, it will become fragmented, and there may be holes in it. The methods shown in this recipe will not take these holes into account, so you may have a little bit more memory than it reports.

Here are the main ways RAM memory is consumed:

• When you initialize constants or define a preprocessor macro:

  #define ERROR_MESSAGE "an error has occurred"

• When you declare global variables:

  char myMessage[] = "Hello World";

• When you make a function call:

  void myFunction(int value)
  {
     int result;
     result = value * 2;
     return result;
  }

  If you create recursive functions (functions that call themselves), you can end up with very high stack usage if your nesting level gets too deep and/or you have a lot of local variables.

• When you dynamically allocate memory:

  String stringOne = "Arduino String";

The Arduino String class uses dynamic memory to allocate space for strings. You can see this by adding the following line to the very top of the code in the Solution:

String s = "
";

and the following lines just before the delay in the loop code:

s = s + "Hello I am Arduino
";
Serial.println(s);           // print the string value

You will see the memory value reduce as the size of the string is increased each time through the loop. If you run the sketch long enough, the memory will run out—don’t endlessly try to increase the size of a string in anything other than a test application.

Writing code like this that creates a constantly expanding value is a sure way to run out of memory. You should also be careful not to create code that dynamically creates different numbers of variables based on some parameter while the code runs, as it will be very difficult to be sure you will not exceed the memory capabilities of the board when the code runs.

Constants and global variables are often declared in libraries as well, so you may not be aware of them, but they still use up RAM. The Serial library, for example, has a 128-byte global array that it uses for incoming serial data. This alone consumes one-eighth of the total memory of an old Arduino 168 chip.

See Also

A technical overview of memory usage is available at http://www.gnu.org/savannah-checkouts/non-gnu/avr-libc/user-manual/malloc.html.

Problem

You have a lot of constant numeric data and don’t want to allocate this to RAM.

Solution

Store numeric variables in program memory (the flash memory used to store Arduino programs).

This sketch adjusts a fading LED for the nonlinear sensitivity of human vision. It stores the values to use in a table of 256 values in program memory rather than RAM.

The sketch is based on Recipe 7.2; see Chapter 7 for a wiring diagram and discussion on driving LEDs. Running this sketch results in a smooth change in brightness with the LED on pin 5 compared to the LED on pin 3:

/* 
 * ProgmemCurve sketch
 * uses table in program memory to convert linear to exponential output
 */

#include <avr/pgmspace.h>  // needed for PROGMEM

// table of exponential values
// generated for values of i from 0 to 255 -> x=round( pow( 2.0, i/32.0) - 1);

const byte table[]PROGMEM = {
   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
   0,   0,   0,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,
   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   1,   2,   2,   2,   2,   2,
   2,   2,   2,   2,   2,   2,   2,   2,   2,   2,   3,   3,   3,   3,   3,   3,
   3,   3,   3,   3,   3,   3,   4,   4,   4,   4,   4,   4,   4,   4,   4,   5,
   5,   5,   5,   5,   5,   5,   5,   6,   6,   6,   6,   6,   6,   6,   7,   7,
   7,   7,   7,   8,   8,   8,   8,   8,   9,   9,   9,   9,   9,  10,  10,  10,
  10,  11,  11,  11,  11,  12,  12,  12,  12,  13,  13,  13,  14,  14,  14,  15,
  15,  15,  16,  16,  16,  17,  17,  18,  18,  18,  19,  19,  20,  20,  21,  21,
  22,  22,  23,  23,  24,  24,  25,  25,  26,  26,  27,  28,  28,  29,  30,  30,
  31,  32,  32,  33,  34,  35,  35,  36,  37,  38,  39,  40,  40,  41,  42,  43,
  44,  45,  46,  47,  48,  49,  51,  52,  53,  54,  55,  56,  58,  59,  60,  62,
  63,  64,  66,  67,  69,  70,  72,  73,  75,  77,  78,  80,  82,  84,  86,  88,
  90,  91,  94,  96,  98, 100, 102, 104, 107, 109, 111, 114, 116, 119, 122, 124,
 127, 130, 133, 136, 139, 142, 145, 148, 151, 155, 158, 161, 165, 169, 172, 176,
 180, 184, 188, 192, 196, 201, 205, 210, 214, 219, 224, 229, 234, 239, 244, 250
};

const int rawLedPin  = 3;           // this LED is fed with raw values
const int adjustedLedPin = 5;       // this LED is driven from table

int brightness = 0;
int increment = 1;

void setup()
{
  // pins driven by analogWrite do not need to be declared as outputs
}

void loop()
{
  if (brightness > 254)
  {
     increment = -1; // count down after reaching 255
  }
  else if (brightness < 1)
  {
    increment =  1; // count up after dropping back down to 0
  }
  brightness = brightness + increment; // increment (or decrement sign is minus)

  // write the brightness value to the LEDs
  analogWrite(rawLedPin, brightness);  // this is the raw value
  int adjustedBrightness = pgm_read_byte(&table[brightness]);  // adjusted value
  analogWrite(adjustedLedPin, adjustedBrightness);

  delay(10); // 10ms for each step change means 2.55 secs to fade up or down
}

Discussion

When you need to use a complex expression to calculate a range of values that regularly repeat, it is often better to precalculate the values and include them in a table of values (usually as an array) in the code. This saves the time needed to calculate the values repeatedly when the code runs. The disadvantage concerns the memory needed to place these values in RAM. RAM is limited on Arduino and the much larger program memory space can be used to store constant values. This is particularly helpful for sketches that have large arrays of numbers.

At the top of the sketch, the table is defined with the following expression:

const byte table[]PROGMEM = {
  0, . . .

PROGMEM tells the compiler that the values are to be stored in program memory rather than RAM. If you were to delete PROGMEM from the sketch on an Arduino Uno, you’d see the global variable use balloon from 13 bytes to 269 bytes (the sketch would not work, though, because pgm_read_byte will not work correctly without PROGMEM there). The remainder of the expression is similar to defining a conventional array (see Chapter 2).

The low-level definitions needed to use PROGMEM are contained in a file named pgmspace.h and the sketch includes this as follows:

#include <avr/pgmspace.h>

Serial.begin(9600);
while(!Serial); // for Leonardo and 32-bit boards
Serial.print("Address of table: 0x"); Serial.println((int)&table, HEX);

To adjust the brightness to make the fade look uniform, this recipe adds the following lines to the LED output code used in Recipe 7.2:

  int adjustedBrightness = pgm_read_byte(&table[brightness]);
  analogWrite(adjustedLedPin, adjustedBrightness);

The variable adjustedBrightness is set from a value read from program memory. The expression pgm_read_byte(&table[brightness]); means to return the address of the entry in the table array at the index position given by brightness.

See Also

Adafruit Industries’ Memories of an Arduino contains a lot of useful insights and techniques for working with Arduino memory.

See Recipe 17.4 for the technique introduced in Arduino 1.0 to store strings in flash memory.

Problem

You have lots of strings and they are consuming too much RAM. You want to move string constants, such as menu prompts or debugging statements, out of RAM and into program memory.

Solution

This sketch creates a string in program memory and prints its value to the Serial Monitor using the F("text") expression. The technique for printing the amount of free RAM is described in Recipe 17.2. This sketch combines both the ARM and AVR methods. If you are using something other than an ARM or AVR board, this sketch probably won’t compile correctly:

/*
 * Write strings using Program memory (Flash)
 */
 
void setup()
{
  Serial.begin(9600); 
}

void loop()
{
  Serial.println(memoryFree());  // print the free memory

  Serial.println(F("Arduino"));  // print the string
  delay(1000);
}

#ifdef __arm__
// Variable created by the build process when compiling the sketch
extern "C" char *sbrk(int incr); // Call with 0 to get start address of free ram
#else
extern int *__brkval; // Pointer to last address allocated on the heap (or 0)
#endif

// function to return the amount of free RAM
int memoryFree()
{
  int freeValue; // This will be the most recent object allocated on the stack
#ifdef __arm__
  freeValue = &freeValue - reinterpret_cast<int*>(sbrk(0));
#else
  if((int)__brkval == 0) // Heap is empty; use start of heap
  {
    freeValue = ((int)&freeValue) - ((int)__malloc_heap_start);
  }
  else // Heap is not empty; use last heap address
  {
    freeValue = ((int)&freeValue) - ((int)__brkval);
  }
#endif
  return freeValue;
}

Discussion

Strings are particularly hungry when it comes to RAM. Each character uses a byte, so it is easy to consume large chunks of RAM if you have lots of words in strings in your sketch. Inserting your text in the F("text") expression stores the text in the much larger flash memory instead of RAM.

If you remove the F from before the string, you’ll see the amount of free memory is lower than if you use it, at least on AVR. Depending on how the ARM compiler optimizes things, it may put strings in Flash without the F expression.

See Also

See Recipe 15.13 for an example of flash memory used to store web page strings.

Problem

You want to minimize RAM usage by telling the compiler that the value is constant and can be optimized.

Solution

Use const to declare values that are constant throughout the sketch.

For example, instead of:

int ledPin = 2;

use:

const int ledPin = 2;

Discussion

We often want to use a constant value in different areas of code. Just writing the number is a really bad idea. If you later want to change the value used, it’s difficult to work out which numbers scattered throughout the code also need to be changed. It is best to use named references.

Here are three different ways to define a value that is a constant:

int ledPin =  2;           // a variable, but this wastes RAM
const int ledPin =  2;     // a const does not use RAM

#define ledPin LED_BUILTIN // with a define, the preprocessor replaces
                           // ledPin with the value of LED_BUILTIN

pinMode(ledPin, OUTPUT);

Although the first two expressions look similar, the term const tells the compiler not to treat ledPin as an ordinary variable. Unlike the ordinary int, no RAM is reserved to hold the value for the const, as it is guaranteed not to change. The compiler will produce exactly the same code as if you had written:

pinMode(2, OUTPUT);

That said, if your sketch uses ledPin as though it were a constant (such as if you never modify it), the compiler is very likely to notice this and optimize it away, producing the preceding code even if you forgot to add const.

You will sometimes see #define used to define constants in older Arduino code, but const is a better choice than #define. This is because a const variable has a type, which enables the compiler to verify and report if the variable is being used in ways not appropriate for that type. The compiler will also respect C rules for the scope of a const variable. A #define value will affect all the code in the sketch, which may be more than you intended. Another benefit of const is that it uses familiar syntax—#define does not use the equals sign, and no semicolon is used at the end. One exception to this is if you need to perform conditional compilation of code, where the compiler ignores or includes code based on the values of one or more #defines (see Recipe 17.6).

See Also

See this chapter’s introduction for more on the preprocessor.

Problem

You want to have different versions of your code that can be selectively compiled. For example, you may need code to work differently when debugging or when running with different boards.

Solution

You can use the conditional statements aimed at the preprocessor to control how your sketch is built.

This example, using the sketch from Recipe 5.6, displays some debug statements only if DEBUG is defined:

/*
 Pot_Debug sketch
 blink an LED at a rate set by the position of a potentiometer
 Uses Serial port for debug if DEBUG is defined
 */

const int potPin = 0;    // select the input pin for the potentiometer
const int ledPin = 13;   // select the pin for the LED
int val = 0;             // variable to store the value coming from the sensor

#define DEBUG

void setup()
{
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);     // declare the ledPin as an OUTPUT
}

void loop() {
  val = analogRead(potPin);    // read the voltage on the pot
  digitalWrite(ledPin, HIGH);  // turn the ledPin on
  delay(val);                  // blink rate set by pot value
  digitalWrite(ledPin, LOW);   // turn the ledPin off
  delay(val);                  // turn LED off for same period as it was turned on
#if defined DEBUG
  Serial.println(val);
#endif
}

Discussion

This recipe uses the preprocessor used at the beginning of the compile process to change what code is compiled. The sketch tests if DEBUG is defined, and if so, the file it incorporates the debugging output. Expressions that begin with the # symbol are processed before the code is compiled—see this chapter’s introduction section for more on the preprocessor.

You can have a conditional compile based on the controller chip selected in the IDE. For example, the following code will produce different code when compiled for a Mega board that reads the additional analog pins that it has:

/*
 * ConditionalCompile sketch
 * This sketch recognizes the controller chip using conditional defines
 */

int numberOfSensors;
int val = 0;               // variable to store the value coming from the sensor

void setup()
{
  Serial.begin(9600);

#if defined(__AVR_ATmega2560__)   // defined when selecting Mega in the IDE
  numberOfSensors = 16;           // the number of analog inputs on the Mega
  #pragma message ( "Using 16 sensors" )
#else                             // if not Mega then assume a standard board
  numberOfSensors = 6;            // analog inputs on a standard Arduino board
  #pragma message ( "Using 6 sensors" )
#endif

  Serial.print("The number of sensors is ");
  Serial.println(numberOfSensors);
}

void loop() {
  for(int sensor = 0; sensor < numberOfSensors; sensor++)
  {
    val = analogRead(sensor);    // read the sensor value
    Serial.print(sensor); Serial.print(": ");
    Serial.println(val);         // display the value
  }
  Serial.println();
  delay(1000);        // delay a second between readings
}

The #pragma directive will display a message in the output area at the bottom of the IDE:

C:Sketchesconditional.ino:15:40: note: #pragma message: Using 16 sensors
        #pragma message ( "Using 16 sensors" )  
                                             ^

See Also

Technical details on the C preprocessor are available at https://gcc.gnu.org/onlinedocs/gcc-7.3.0/cpp/.