2.1.1 Using an ICSP for All Uploads

With this done, all sketches will now default to using the ICSP rather than the bootloader. This means that I no longer have to worry about remembering to change the programmer in the IDE, and, as a bonus, I will always overwrite the bootloader area and regain the use of that part of the Flash RAM for my own use. My Uno board will have an extra 512 bytes (1.5625% of the total) of flash for my programs, while my Duemilanove will regain an extra 2 Kb or 6.35% from the bootloader space.

Uploading with any ICSP device does still require you to press the Shift key when you click the upload button or to select Sketch ➤ Upload Using Programmer, even with this preference set.

I can still set the IDE to use a bootloader though. I just have to remember to select it from the Tools menu when I wish to create a sketch for a system that I cannot, or don’t want to, use the ICSP for uploads.

What’s an ICSP? Normally beginners would use a USB cable between the computer and the Arduino to upload programs. However, look at your Arduino and see if you can see a set of six pins in two rows of three. My Duemilanove has them beneath the reset switch. Those pins are where an ICSP can be plugged in to program the Arduino. Using one of these frees up the space taken by the bootloader program and gives you a bit more program space.

Unfortunately, it does prevent the Arduino from talking back to your computer using the Serial Monitor facility – Tools ➤ Serial Monitor. You will need an ICSP if you have to replace the ATmega328P on your board, and it comes with the default fuse settings. You will need to purchase one with an Uno bootloader already programmed in or use an ICSP to program your own.

My ICSP is from an eBay seller “finetech007” which no longer exists. There are lots of them if you search for “usbtiny isp” – here’s one example [www.ebay.co.uk/itm/USBTiny-USBtinyISP-AVR-ISP-programmer-for-Arduino-bootloader-Meag2560-uno-r3-CF/191780957944?epid=1138358692&hash=item2ca7092ef8:g:4-UAAOSwhvpd-fY7], identical to mine. (Sorry about the length of that URL!)

2.1.2 Change the Action of Home and End Keys

I’m reliably informed that this applies to Apple Mac users. It certainly has no effect on Windows 7 or Linux.

In the editor, when you press the Home key, the caret jumps to the very start of the sketch. When you press the End key, it jumps to the very end of the sketch. Apparently, this gets quite annoying when you expect the caret to be positioned at the start or end of the line you are editing. This sort of thing definitely needs changing!

When you next open the IDE and load a sketch, the Home and End keys should now do your bidding.

Issue 3715 on the GitHub issues page for the IDE has some interesting details about this preference. It only exists from version 1.6.6 onwards. Prior to that, it was called editor.keys.home_and_end_travel_far.

In the first incarnations of version 1.6.6, the setting was coded backward. Setting it to false meant that the Home and End keys sent the cursor to the start or end of the document. You had to set it to true to get it to go to the beginning or end of the current line.

Since August 28, 2015, that was fixed; and it now works as it should. You can find all the details at https://github.com/arduino/Arduino/issues/3715.

2.1.3 Setting Tab Stops

Now, you would think that an editor, for writing code, would at least allow you the ability to adjust the width of the tab stops and whether or not they are to be converted into spaces. Not so the Arduino IDE!

This causes tabs to indent four characters from the default of two characters. I don’t know about you, but I find two-character indents quite unreadable when looking at the structure of a sketch. I use four for just about everything I do. The preceding first line, which was not changed, determines if the IDE will convert tab characters into spaces. When set to true, the IDE will convert tabs to spaces, while false will leave the tab characters as they are, unchanged.

This makes editing in the IDE a little more comfortable, in my opinion.

2.3.1 Arduino Uno Example

The Arduino Wiki at https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5-3rd-party-Hardware-specification mentions, at least for IDE version 1.5.3 which appears to be the most recently documented, that

This file contains definitions and meta-data for the boards supported. Every board must be referred through its short name, the board ID. The settings for a board are defined through a set of properties with keys having the board ID as prefix.

What it doesn’t mention is how the system is supposed to know that “uno”, for example, refers to the Arduino/Genuino Uno device.

From the preceding listing, it is pretty obvious that the Uno’s short name must be “uno” as that is the prefix in use for every entry in this section of the file.

2.4.1 Build Recipes

The platform.txt file, as mentioned earlier, contains a large amount of meta-data that configures the IDE to be able to compile sketches and upload them, among other things, for the Arduino boards running with AVR microcontrollers. It does this using recipes. Having different recipes for all the different platforms allows the IDE to be used for a myriad of different devices.

A number of additional settings are defined within the $ARDINSToards.txt file based on the particular board chosen on the Tools ➤ Boards menu – see Section 2.3, “Boards.txt,” for those details – and the IDE global variables can be used within this file too. You can find more details on those variables in Section 2.2, “Globally Defined Paths.”

The compilation process can read source files written in plain C – these are the .c source files, C++ (.cpp), or even assembly language (.S). It has to know how to convert these files into object files (.o) which can be gathered together by the linker to create an executable file. The way this happens is by using the recipes within the platform.txt file.

You will notice, I hope, that the source format in each is case sensitive. Assembly language files must have an upper case .S extension.

2.4.2 Pre- and Post-build Hooks

These are identified by the recipe.hooks part. The next part determines which stage in the compilation the hook will be called. Prexxxxx and postxxxxx indicate that the pattern will be called before the appropriate stage or afterward.

In order that multiple hooks can be called at any stage, the NUMBER part is a sequence number which should be 1, 2, 3, 4, and so on – there’s one number for each hook to execute at a given stage in proceedings. The end of the recipe is always the word pattern.

If you find that you require ten or more hooks, then your NUMBER parts should be 01, 02, ..., 10, 11, and so on.

I noticed that some, but not all, variables do not get expanded. Using "{source_file}", for example, doesn’t expand, but {build.source.path} does. Also, the entire text after the equals sign becomes part of the message, not just the command’s output. The preceding code, on Linux, displays the text “echo Compiling sketch: /full/path/to/my/sketch/here” rather than just “Compiling sketch:

/full/path/to/my/sketch/here”. A similar thing happens with Windows.

The command used cannot be a built-in command and must be found on your system’s path ($PATH on Linux and MacOS, %path% on Windows.)

On Windows, the echo command is a built-in command. It cannot be found on %path% when the compilation is started, so the whole compilation fails because echo can’t be found. This is because of the way that the Java command exec, from Runtime.getRuntime(), works.

So how did the preceding echo command work for Windows? I created an echo.bat file and put it on my Windows %PATH%.

Commands to be executed in the hooks must be found on the $PATH; if not, they will not be executed, and the recipe will fail.

2.6.1 Arduino Sketch (*.ino) Preprocessing

These actions are performed by the arduino-preprocessor tool which lives on GitHub at https://github.com/arduino/arduino-preprocessor.

2.6.2 Arduino Sketch (*.ino) Build

This ends the compilation process. If the upload button is clicked in the IDE, rather than the compile (or verify) button, then the $TMP/<sketch_name>.ino.hex file is uploaded to the AVR microcontroller, using an Arduino-specific version of the avrdude tool which can be found on GitHub at https://github.com/arduino/avrdude-build-script.

If you have a bootloader installed on your ATmega328P, then you can use it to upload the files using avrdude. Normally, when using an ICSP to program your device, the bootloader will be overwritten when the chip is wiped. However, if you use the ICSP to upload the preceding file with the bootloader, then you effectively burn a bootloader as well as your application’s code into the Arduino board.

Either file can be uploaded using the bootloader – if it is still installed on your device – and after doing so, regardless of which of the two files you upload, the bootloader will still be present afterward.

All this “just works” and it makes life easy; however, what is the Arduino system hiding from you?

You can see all of this happening, before your very eyes, if you edit the preferences in the IDE to show verbose compiling and/or upload messages.

The following chapters describe the various files that your sketch ends up including when the compilation process has completed.

2.8.1 Header File avrpgmspace.h

This header file is included from $AVRINC, to allow pins_arduino.h, as described in the following, to create lookup tables within the program space – in flash memory, as opposed to in the scarce Static RAM (SRAM) on the device. It defines a number of typdefs and functions to copy data between the program space, in flash, and the variable space in RAM. It doesn’t make for very interesting reading I’m afraid.

2.8.2 Header File avrio.h

This file, from $AVRINC, sets up all the AVR-specific stuff for the appropriate AVR microcontroller that is in use on the Arduino board. The settings you chose in the IDE for Tools ➤ Boards will determine the specific device file that will be included. In the majority of cases, and for our purposes here, this will be the ATmega328P, and so this file simply causes the AVR definitions for that microcontroller to be read in from the file avr/iom328p.h.

This file also includes sfr_defs.h to set up numerous macros for memory address simplification, some functions to handle looping until a bit is clear (or set) and so on. These are not discussed here.

The AVRLib header files are to be found in $AVRINC, under your Arduino 1.8.5 installation, unless otherwise stated. These header files are not part of the Arduino IDE per se, but are supplied as part of the AVRLib, which the IDE uses.

2.8.3 Header File avrinterrupt.h

Because the init() function, as described in the following, sets up Timer 0 with an interrupt routine to keep track of the number of milliseconds (millis) that have passed since a sketch started, this header file is required. In essence, and among many checks, it creates the ISR() macro which allows you to create interrupt handlers when using AVR-specific C/C++ code. The Arduino Language uses a slightly different system, attachInterrupt(), for example, for external interrupts. Arduino interrupts will be described in Chapter 3, Section 3.5, “Interrupts.”

This file is also part of the AVRLib and is found in $AVRINC.

2.8.4 Header File binary.h

It looks strange, yes? The header is defining as many different binary style constants for every number between 0 and 255. Why does 0 get so many different constants while 255 only has one? This allows the programmer to specify zero in as many ways as there are leading zeros in the binary representation of the number zero. This applies to all the numbers, but once you reach 128, there are no more leading zeros, so those values only have a single constant defined.

Only one line of code? To do all that? Yes, one line of AVR code can correspond to numerous lines of Arduino code. This is another example of how the Arduino makes life easier for the beginner – which of the two preceding code sections is the easier to read and understand?

2.8.5 Header File WCharacter.h

This header file defines a number of inlined functions which can be used to determine if a character is numeric, alphanumeric, and so on. This is not specific to the Arduino software and will not be discussed further.

What are inlined functions?

Inlined functions get copied verbatim into the code where they are called. Normally, functions are set up in the executable once and called from many places. Inlining them improves runtime efficiency but at the expense of code size.

Then call the isAlphaNum() function frequently from your own code, rather than calling the isAlphaNumeric() function frequently.

2.8.6 Header File WString.h

The WString.h header file defines a C++ class named String. I have to admit to not seeing any Arduino code that uses this class, but maybe I haven’t been reading enough code. As this isn’t specifically Arduino code, even though the class has been written for Arduino, it will not be discussed further.

Okay, I lied. String is used by the Serial class, but deep down. Serial inherits from Stream which inherits from Print, and Print uses String internally.

Using this class will seriously increase the size of your sketches and may result in very difficult to diagnose runtime errors if too much dynamic memory is allocated – which this class does internally.

2.8.7 Header File HardwareSerial.h

This is the header file that defines the Serial interface whereby your Arduino board can talk back to your main computer over the USB cable. There’s a lot going on in this file, and it makes interesting reading to see how the Arduino library works.

If your device has less than 1024 bytes of RAM, then two buffers of 16 characters each are created. One is for serial receiving and the other is for serial transmission. The buffer sizes are increased to 64 characters if you have more than 1024 bytes of RAM available. The ATmega328P has 2048 bytes, so the larger-sized buffers are created.

The buffers are set up as what is known as circular buffers. They have a pointer to the first free character for insertion into the buffer and a pointer to the next character to be removed from the buffer. Hopefully, never the twain shall meet, but if they do happen to meet, the sketch will suspend for a while until some data has been removed from the buffer allowing the new data to be inserted.

Circular buffers are described in some detail at https://en.wikipedia.org/wiki/Circular_buffer.

Also set up in this file are a couple of interrupt routines which are called automatically by the AVR microcontroller whenever there is an empty transmit or a full receive buffer for the built-in USART device. The Serial class will, in the case of a transmission, read the next character from the Arduino transmit buffer and write it into the hardware register as appropriate to have it transmitted out via the USART. A similar process takes place for the receive interrupt. The hardware buffers on the AVR microcontroller are a single byte in size.

In the case where the AVR device has more than one USART, the Mega 2560 series, for example, then these are also set up from other header files included by this one. These additional hardware serial devices are not discussed further as the default Arduino board using the ATmega328P only has a single USART and they are all similar.

2.8.8 Header File USBAPI.h

This is a header specifically for devices which have built-in hardware USB features – these being boards based around the ATmega32U4 microcontroller which is on board the Arduino Leonardo, Pro Micro, Micro, and a few other models. As the ATmega328P doesn’t have hardware USB on board, this file will not be discussed any further.

2.8.9 Header File pins_arduino.h

The version of pins_arduino.h that is included is dependent on the Arduino board in use and, thus, the AVR microcontroller in use on that specific board. For the default board, the file is located in $ARDINST/variants/standard, using an ATmega328P, while the Adafruit Gemma board has its pins_arduino.h in $ARDINST/variants/gemma and uses an ATtiny85 device. The included file sets up the pin assignments for the appropriate device.

The board is chosen by the developer using the IDE’s Tools ➤ Boards menu option.

It is this header file that defines constants for the analogue pins, A0 which has the value 14 through A7 which is defined as 21, for example.

2.9.1 Enabling the Global Interrupt Flag

The function init() begins by enabling interrupts globally as shown in Listing 2-5. Arduino boards require interrupts to be enabled so that the millis() function can begin counting, once the appropriate timer (Timer/counter 0) is configured and started.

In the following walk-through of the source code for the init() function, and as with many other code listings in this book, the code that is not relevant to the ATmega328P has been removed to reduce the amount of source code listed and to avoid confusing this author!

void init()

{

    // this needs to be called before setup() or some

    // functions won't work there

    sei();                               ①

Listing 2-5

Setting interrupts on

    ①    Turns on global interrupts. This is required to make functions such as millis() and micros() work.

The code continues to enable Timer/counter 0 next.

2.9.2 Enabling Timer/counter 0

Timer/counter 0 is also used to provide 8-bit PWM (pulse width modulation for analogueWrite() ) on pins D5 and D6.

Listing 2-7 shows the source for the Timer 0 Overflow interrupt routine, which is separate from the code in the init() function. The remainder of the init() function walk-through follows later.

2.9.3 Timer/counter 0 Overflow Interrupt

The Timer 0 Overflow interrupt is used to update the millis() count. It does this every 1.024 milliseconds, and, as this is slightly over 1 millisecond, it accumulates these extra fractions; and when there are enough accumulated, the millis count gets incremented by an extra leap millisecond. This takes place roughly every 42 interrupt handler executions – it’s actually every 41.666 (recurring) executions, but you cannot have a fraction of an execution!

“What are MILLIS_INC, FRACT_INC, and FRACT_MAX?”, I hear you ask.

So, working backward, we see that the number of clockCyclesPerMicrosecond is 16e6/1e6 or 16. From that, we can then see that MICROSECONDS_PER_TIMER0_OVERFLOW is (64 * 256)/16, which gives us 1024.

This then allows MILLIS_INC to be calculated as 1024/1000 which is a solitary one, as this is integer division, not floating point. And, finally, FRACT_INC is 1024 – (1024/16) >> 3 or 24/8 which gives us 3.

FRACT_MAX is easy; it’s effectively 1000/8 or 125.

So, every 256 Timer/counter 0 clock ticks, we increment the number of millis by 1 and add 3 to the fractions accumulator, and if that is more than 125, we add an extra 1 to millis and reduce the fractions accumulator by 125, thus holding on to any spare fractions. Eventually, these will add up and generate another millisecond.

If you are wondering why we add 3 and check for 125, then consider that there are 24 microseconds spare each time through the interrupt handler – that’s 41.666 (recurring) to gain an extra millisecond. 125/3 is exactly the same value – 41.666 (recurring) – so it works out the same.

Is adding 3 and checking against 125 more efficient than adding 24 and checking against 1000? Yes, indeed, the former method fits a byte, while the latter requires a 16-bit value, and the ATmega328P is an 8-bit device without a 16-bit compare instruction.

2.9.4 Configuring Timer/counter 1 and Timer/counter 2

On the ATmega328P, Timer/counter 1 is a 16-bit timer; however, the Arduino system sets it up so that it appears as an 8-bit timer which makes it similar to Timer/counter 0 and Timer/counter 2.

Timer/counters 1 and 2 are used to provide PWM on four of the six pins that are PWM enabled on an ATmega328P.

Both timers have their prescaler set to divide the system clock by 64 and are set up in 8-bit Phase Correct PWM mode.

Timer/counter 1 provides PWM on pins D9 and D10, while Timer/counter 2 provides PWM on pins D3 and D11.

The function continues, now that all three timers are configured, to set up the Analogue to Digital Converter (ADC) so that the Arduino analogRead() function will work.

2.9.5 Initializing the Analogue to Digital Converter

According to the data sheet for the ATmega328P, the ADC (Analogue to Digital Converter) runs best, and most accurately, when it is running at a speed between 50 and 200 KHz. The system clock on the microcontroller is running at 16 MHz, so is a little on the speedy side.

In order to get the ADC into a valid speed range, it has its prescaler set to divide the system clock by 128. This puts the speed at 125 KHz, which is within the desired range specified by the data sheet.

The preceding code implies that even if you don’t want the ADC in your sketches, it is active and consuming additional power that might be better used keeping your batteries from running down!

This uses the AVRLib facility to turn off the power to the ADC clock and, in my opinion, is a lot more readable, and understandable, than the preceding one.

2.9.6 Disabling the USART

The final task for the init() function is to disable the Universal Synchronous/Asynchronous Receiver/Transmitter or USART for short.

This is left attached to Arduino pins D0 and D1 by the bootloader, and the two pins used need to be disconnected so that they can be reused for digitalRead() and/or digitalWrite() in sketches. On the ATmega328P, these digital pins are the physical pins 2 and 3.

This concludes the initialization that occurs at the start of every sketch and the walk-through of the init() function’s source code.