Nort_/D is a design collective that creates hardware and software for multitouch screens. The collective provides both instructions for creating touchscreens and also kits that include screens, cameras, and supports. Its aim is to make multitouch readily available in an open source fashion. Over the past few years, Nort_/D has created a few different multitouch systems: the CUBIT multitouch system, the TouchKit, and Tactility, which uses LCD screens. The TouchKit includes a Nort_/D Multitouch Screen, Nort_/D’s Multitouch Software (computer vision, calibration, API), and a camera (calibrated for use with the TouchKit).

The ofxTouchApp class is the main superclass for ofxTouch applications. It is a substitution for the typical openFrameworks ofBaseApp from which all applications usually inherit. The ofxTouchApp class adds to ofBaseApp multitouch-specific functionality. You’ll recall that all the event handlers in the ofxTouchApp class are sent a reference to this object to notify the application of the position, side, and order of the finger touching the surface.

In the vector<ofxTouchFinger> fingers vector, fingers stores all the blobs, which are areas detected in the video of your application that are moving together, that are detected and tracked on the touchscreen. They are stored as the ofxTouchFinger type, which is a simple class that the ofxTouchApp uses to store information about the location and movement of a finger on the touchscreen.

The class provides these event handlers to notify your application that a finger event has happened and pass the reference to the ofxTouchFinger instance representing the blob that triggered the event. The names are quite self-explanatory:

The ofxTouchApp class also has some convenience methods that make it easier for you to test your application with stored video files:

The other key component of the ofxTouch add-on is ofxTouchFinger, which is used to store information about finger events. These are the two methods useful for keeping track of the fingers:

To determine information about the finger, there are four integer values about the finger (x, y, width, and height) and a float value (radius).

Your application’s .h file will look something like this:

class testApp : public ofxTouchApp {

Note that instead of extending the ofBaseApp class, the application extends the ofxTouchApp class:

    public:
        bool bDragging;
        void setup();
        void update();
        void draw();
        void keyPressed  (int key);
        void keyReleased  (int key);
        void mouseMoved(int x, int y );
        void mouseDragged(int x, int y, int button);
        void mousePressed(int x, int y, int button);
        void mouseReleased();
        void fingerDragged( const ofxTouchFinger& finger );
        void fingerPressed( const ofxTouchFinger& finger );
        void fingerReleased( const ofxTouchFinger& finger );

};

In the main.cpp file of an ofxTouchApp, instead of seeing the standard call to ofRunApp(), you’ll see a call to ofxTouchAppRelay() wrapped inside the ofRunApp() call:

ofRunApp(new ofxTouchAppRelay(new testApp));

If you take a look at the ofxTouchApp.h file, you’ll see the following properties. If you read Chapter 14, the first two objects should be familiar to you.

Moving on, a very simple application might look like the following:

void fingerApp::setup() {
    setSetupMode( true );
    cwidth = 800;
    cheight = 800;

    //setSimulationMode( true );  //uncomment this to use mouse simulation
    setVideoPlayerMode( true );   //comment this out to run live from cam
    setVideoPlayerFile("touchkit-fingers.mov");
    bDragging = false;
    ofEnableAlphaBlending();
}


void fingerApp::update(){
    // nothing here
}


void fingerApp::draw() {

Since the fingers vector stores all the detected touch fingers, you can use this code to display the fingers:

    for( int i=0; i<fingers.size(); i++ ) {
        ofSetColor( 255, 255, 255);
        ofCircle( fingers[i].x, fingers[i].y, 4*fingers[i].radius );
        ofSetColor( 255, 0, 0 );
        ofCircle( fingers[i].x, fingers[i].y, 3*fingers[i].radius );
        ofSetColor( 255, 255, 255);
        ofCircle( fingers[i].x, fingers[i].y, 2*fingers[i].radius );
        ofSetColor( 255, 0, 0);
        ofCircle( fingers[i].x, fingers[i].y, 1*fingers[i].radius );
        ofSetColor( 255, 255, 255 );
        ofCircle( fingers[i].x, fingers[i].y, 0.5*fingers[i].radius );

    }
}

This will create a simple bull’s-eye at the location of each detected finger touch. There are some other excellent examples for download from the Nort_/D site at http://touchkit.nortd.com. For information on their other projects, look at nortd.com.

Another much smaller touchscreen device that you might be interested in exploring is the TouchShield developed by Liquidware (Figure 16-4). It plugs into the Arduino and lets you use small touchscreen devices both to send information to a listening laptop or desktop computer or to simply create a small device that relies entirely on the Arduino for its processing power. The TouchShield greatly simplifies working with resistive touch sensing and drawing.

Figure 16-4. The Liquidware TouchShield

To connect the TouchShield onto an Arduino Diecimila or Duemilanove, you simply mount it as shown previously in Figure 16-2 on the right. The TouchShield does quite different things than the standard Arduino components. In addition to handling touch data, the TouchShield also draws to its screen. The Liquidware team took a logical direction in how to best providing methods and a programming interface for the developer to draw to the screen: they emulated the Processing language and application structure.

The very simplest TouchShield application looks much like a Processing application. For instance, to draw a rectangle to the screen of the TouchShield, your application would look like this:

COLOR red = {255,0,0};
COLOR blue = {0,0,255};

void setup() {
    rect(20,20,80,80, red, blue);
}
void loop(){}

To upload your code to the TouchShield, you’ll need to use a slightly modified version of the Arduino IDE that the Liquidware team has put together and that is available for download on the Liquidware site. The only modifications that have been made are to allow for the uploading of code to the TouchShield. When you have an Arduino application that communicates with the TouchShield, you are in essence running two applications at the same time: one on the TouchShield and one on the Arduino. You could think of this metaphorically as being like the relationship between code running on your central processor unit communicating with code running on your graphics processor unit.

While the code for the TouchShield is quite similar to Processing, it is not actually Processing, and so it has a few slightly different methods:

To read the input locations and information about the user touching the screen, the TouchShield library defines the following methods and variables:

The TouchShield also defines quite a few different methods to draw on the canvas:

To set the color properties and line properties when you’re drawing, you can use the following methods (note that all the colors are passed as three values, red, green, and blue with values between 0–255):

One of the great strengths of open source software is that it allows developers to alter some of the internals of the program or add new features to support different functionality. Since the TouchShield is in a sense like an Arduino controller, in that it has its own processor that the Arduino IDE must compile and load code onto, you’ll need a version of the Arduino IDE that knows how to create and upload code from the TouchShield. Liquidware created a version of the IDE that adds this new board into the available configurations and didn’t change anything else, so you can use the TouchShield as well as all your other Arduino boards with this altered Arduino IDE. Those altered IDEs are available on the Liquidware site, http://liquidware.com. To work with the TouchShield, you’ll need to download and install the appropriate one for your operating system. Once you’ve done that, you’ll see the TouchShield appear in your Arduino IDE (Figure 16-5).

Figure 16-5. Selecting the TouchShield in the modified Arduino IDE

In the Board menu, you’ll notice a couple new boards, the TouchShield Slide and TouchShield Stealth. There are currently two different touch-enabled shields that Liquidware manufacturers; one is the TouchShield Slide and the other is the TouchShield Stealth. The workflow for uploading code to a TouchShield attached to a Duemilanove would look like this:

That’s all there is to it. Let’s take a look at a couple of examples.

The following example shows how to draw different shapes and characters to the screen:

#define SHAPE_SQUARE  0
#define SHAPE_CIRCLE  1
#define  SHAPE_X   2
#define  SHAPE_TRI   3

int nextShape = SHAPE_SQUARE;

int pMouseX;
int pMouseY;
char xposStr[4];
char yposStr[4];

void setup() {
    pMouseX = 0;
    pMouseY = 0;
}

void loop()
{

The getTouch() method polls the screen to see whether any touches have been registered. When you want to detect touches on the screen, you’ll want to do this each time the loop() is called:

    gettouch();

    if(pMouseX != mouseX || pMouseY != mouseY)
        touched();   // screen touched so draw new shape

    delay(50);

} //end loop

void touched()

{
    nextShape++;
    if(nextShape > 3) { // shape 3 is the last shape
        nextShape = 0;
    }

Just as in a Processing application, this clears the stage by overwriting everything:

    background(0);

Now the location of the touch is stored for comparison to the next time the loop() method is called:

    pMouseX = mouseX;
    pMouseY = mouseY;

The text() method writes characters to the screen, the first parameter are the characters to be written, and the next two are the x and y locations on the screen:

    text(mouseX, mouseX, mouseY+20);
    text(mouseY, mouseX+30, mouseY+20);

Depending on the value of the nextShape variable, a different shape is drawn to the stage:

    switch(nextShape)
    {
    case SHAPE_SQUARE:
        fill(0, 255, 0);
        rect(mouseX-20, mouseY-20, 30, 30);
        break;

    case SHAPE_CIRCLE:
        fill(0, 0, 255);
        ellipse(mouseX-10, mouseY-10, 15, 15);
        break;

    case SHAPE_X:
        stroke(255, 255, 255);
        line(mouseX - 10, mouseY, mouseX+10, mouseY);
        line(mouseX, mouseY - 10, mouseX, mouseY+10 );
        break;
    case SHAPE_TRI:
        fill(255, 255, 0);
        triangle(mouseX+10, mouseY-10, mouseX-10, mouseY+10,
              mouseX+20, mouseY+10);
        break;
    }
}

The application will look something like Figure 16-6.

Figure 16-6. Drawing a circle and a triangle to the TouchShield

Now that you’ve seen how to do simple drawing operations with the TouchShield, you can use the touch information and send it to the Arduino to control other objects. The following example allows the user to select the color of the circle they are drawing, draw a circle, and send those values to the Arduino controller. From there, the Arduino controller can, for example, use the x and y positions to set the rotation of a servo, relay those values to another listening controller, or, as shown in the next example, position a light.

In this example, though, those values will be used to position two servo motors that will shine a three-color LED with the color selected by the user. Before putting this code into the Arduino IDE, make sure that you have the TouchShield board selected, or the code won’t compile because the compiler will not have access to the right libraries:

COLOR curr;

Notice that the color is stored as a COLOR type. This is a struct that is defined in the TouchShield libraries and has red, green, and blue integer properties that can be read and written:

int redR[] = { 0, 0, 20, 20 };   // rectangle coordinates
int blueR[] = { 0, 20, 20, 40 };
int yellowR[] = { 0, 40, 20, 60 };
int greenR[] = { 0, 60, 20, 80 };
int whiteR[] = { 0, 80, 20, 100 };
int blackR[] = { 0, 100, 20, 120 };

COLOR red = {255,0,0};
COLOR green = {0,255,0};
COLOR blue = {0,0,255};
COLOR yellow = {255,255,0};

boolean changedColor = false;

void setup() {

    drawBackground();
    Serial.begin(9600);
    delay(3000);
    Serial.print('U'),
}

The drawBackground() method draws six boxes on the left of the TouchShield that the user can select. These will be used to set the color value of the color LED attached to the two servo motors:

void drawBackground()
{

Draw the first rectangle in the top left of the screen, first setting the fill using the fill() method. The fill() method takes the different values of the predefined color values included in the TouchShield API:

    fill(red.red, red.green, red.blue);
    rect(redR[0], redR[1], redR[2], redR[3]);

Now draw the other rectangles for blue, yellow, green, and white:

    fill(blue.red, blue.green, blue.blue);
    rect(blueR[0], blueR[1], blueR[2], blueR[3]);
    fill(yellow.red, yellow.green, yellow.blue);
    rect(yellowR[0], yellowR[1], yellowR[2], yellowR[3]);
    fill(green.red, green.green, green.blue);
    rect(greenR[0], greenR[1], greenR[2], greenR[3]);
    fill(white.red, white.green, white.blue);
    rect(whiteR[0],whiteR[1], whiteR[2], whiteR[3]);
}

void loop() {

The refresh rate on the TouchShield is quite a bit slower than a laptop computer. If you call the background() method on a loop and then do a relatively expensive drawing operation like the drawBackground(), then you’ll see the screen begin to flicker:

gettouch();
    check_touch();

    Serial.print(mouseX);
    Serial.print(mouseY);

If the color has changed, then send the values to over the hardware Serial port to the Arduino controller:

    if(changedColor) {
        Serial.print(curr.red);
        Serial.print(curr.green);
        Serial.print(curr.blue);
    }
}

void check_touch(){
    changedColor = true;
    // if the mouse is greater than 20, it's not on any of the rectangles
    if(mouseX > redR[0]) {
        return;
        changedColor = false;
    }
    if(mouseY < redR[2]) {
        curr = red;
    }
    else if(mouseY > blueR[2] && mouseY < blueR[4]) {
        curr = blue;
    }
    else if(mouseY > yellowR[2] && mouseY < yellowR[4]) {
        curr = yellow;
    }
    else if(mouseY > greenR[2] && mouseY < greenR[4]) {
        curr = green;
    }
    else if(mouseY > whiteR[2] && mouseY < whiteR[4]) {
        curr = white;
    }
    else {
        changedColor = false;
    }
}

In this example, you’ll see how the Arduino board and the TouchShield can communicate. The principle idea in this example is to use two servos together to create a hemispheric range for a light, that is, to enable an LED to cover an area that more or less would look like the top half of a sphere. The way you do that is by combining two servos and using one to provide 180 degrees of motion on the x-axis and the other to provide 180 degrees of motion on the y-axis. There are quite complex ways to do this, but there are simple ones that work almost as well. You can simply attach one servo to the other and then attach the LED to the second servo. This provides you with the range of motion to point the light. Using a “super-bright LED” means that you can attach a small object to a small servo and still create a good amount of light.

Figure 16-7 shows a diagram of how to connect the two servos together and how they would be wired to the Arduino.

Now, the Arduino code that receives the x and y values that will be used to set the two servos and the color values to set the LED. This code is for the Arduino, so don’t forget to make sure that you have the correct board selected in your Arduino IDE, or the code may not compile:

#include <Servo.h>

// create servo object to control a servo
Servo xServo;
Servo yServo;
int redPin = 5;
int greenPin = 6;
int bluePin = 11;

// variable to store the servo position
int xpos = 0;
int ypos = 0;

void setup()
{

Figure 16-7. Connecting the servos and LED to the Arduino

Here you start up the communication between the Arduino and the two servos by calling the attach() method of the Servo instance:

    xServo.attach(9);
    yServo.attach(10);

    Serial.begin(9600);
}

void loop()
{
    int tmpx, tmpy, tmpred, tmpblue, tmpgreen = 0;

Get all the available bytes from the TouchShield that have been sent over the Serial port. I’ve set my hardware up so this data is being sent from the TouchShield, but you could also use the TouchShield and have the data be sent from a Processing or oF application with which the user is interacting. The positions and colors will be stored in the five integers:

    if(Serial.available() > 4) {
        tmpx = Serial.read();
        tmpy = Serial.read();
        tmpred = Serial.read();
        tmpblue = Serial.read();
        tmpgreen = Serial.read();

After five values are received, set each respective color values, red, green, and blue, of the three color LED lights:

        analogWrite(redPin, int(tmpred));
        analogWrite(greenPin, int(tmpgreen));
        analogWrite(bluePin, int(tmpblue));

Now, set the location of each servo. Since the locations are being sent as integers, they’ll need to be cut down into smaller values to set the rotation of the servo.

       xpos = map(tmpx, 0,255, 0,180);
       ypos = map(tmpy,0,255,0,180);
       xServo.write(xpos);
       yServo.write(ypos);
       delay(15);

    }
}

You could use this same technique to position a camera, a microphone, or many other kinds of devices. The pairing of multiple servos is one of the basics of complex mechanical motion and is a very useful thing to look into. You might check out Robot Building for Beginners by David Cook (Apress). It is a good introduction to mechanics and robotics that is aimed at newcomers.

Another approach to this is to use a pan and tilt servo unit like the one shown in Figure 16-8, made by Trossen Robotics. It’s actually just two servo motors with a stronger mounting, which means you can attach larger lights or cameras to the servo.

Figure 16-8. The Trossen pan-tilt servo

The TouchShield is a wonderful tool for creating small touch-enabled applications. One thing to consider when working with the TouchShield is the relatively slow refresh rate of the drawing on the TouchShield. The code used with TouchShield is based on Processing, but the hardware will not be like a laptop or desktop computer. So, you’ll need to consider what drawing functions to call and when to call them to ensure that you don’t overtax the hardware. Another issue to consider is the power that the TouchShield requires. If you’re going to use the TouchShield as a mobile device, then you might want to consider using something like the Lithium Backpack that provides a few extra hours of battery power.

You can communicate with the Wii in two ways, either by plugging it into an Arduino using the Wii stick developed by Tod Kurt of Todbot or by using the Bluetooth connection on the Wii Remote. You can also connect the Nunchuck directly to the Arduino by cutting off the connector and using the wiring, as shown in Figure 16-9.

Figure 16-9. Two ways of connecting the Nunchuck to the Arduino: on the left, the Nunchuck Adapter and, on the right, the Nunchuk with the connector removed and wires connected directly to the Arduino

The following Arduino code uses the values from the Nunchuck to control a servo and also sends the accelerometer data from the Nunchuck to a Processing application:

#include <Wire.h>
#include <Servo.h>

// create servo object to control a servo
Servo xServo;
Servo yServo;

int angleX = 90;    // angle to move the X servo
int angleY = 90;    // angle for the Y servo

int refreshTime = 20;  // the time in millisecs between updates
int loop_cnt = 0;

int joy_x_axis;
int joy_y_axis;
int accel_x_axis;
int accel_y_axis;
int accel_z_axis;

int z_button;
int c_button;

void setup()
{
    Serial.begin(9600);
    xServo.attach(9);
    yServo.attach(10);
    nunchuck_init(); // send the initialization handshake to nunchuck
}

void loop()
{
    checkNunchuck();
    xServo.write(angleX);
    xServo.write(angleY);
    if( nunchuck_zbutton() ){  // light the LED if z button is pressed
        Serial.print(angleX,DEC);
        Serial.print(":");
        Serial.print(angleX,DEC);
        Serial.print(''),
    }
    delay(refreshTime);        // this is here to give a known time per loop
}

void checkNunchuck()
{
    if( loop_cnt > 5) {  // loop every 20msec, this is every 100msec

        nunchuck_get_data();
        nunchuck_print_data();

        angleX = map(accel_x_axis,70,185,0,180); // nunchuck range is ~70 - ~185
        angleY = map(accel_y_axis,70,185,0,180); // Arduino function maps this
                                                      // to angle 0-180
        loop_cnt = 0;  // reset
    }
    loop_cnt++;
}

The rest of the code defines the Nunchuck functions to initialize communication between the Arduino and the Nunchuck and to send and receive data from the Nunchuck. These methods all use the Wire library to read the data. Credit for these methods goes to Tod Kurt:

static uint8_t nunchuck_buf[6];   // array to store nunchuck data,

// initialize the I2C system, join the I2C bus,
// and tell the nunchuck we're talking to it
void nunchuck_init()
{
    Wire.begin();                    // join i2c bus as master
    Wire.beginTransmission(0x52);    // transmit to device 0x52
    Wire.send(0x40);        // sends memory address
    Wire.send(0x00);        // sends sent a zero.
    Wire.endTransmission();    // stop transmitting
}

// send a request for data to the nunchuck
// was "send_zero()"
void nunchuck_send_request()
{
    Wire.beginTransmission(0x52);    // transmit to device 0x52
    Wire.send(0x00);        // sends one byte
    Wire.endTransmission();    // stop transmitting
}

// receive data back from the nunchuck,
// returns 1 on successful read. returns 0 on failure
int nunchuck_get_data()
{
    int cnt=0;
    Wire.requestFrom (0x52, 6);    // request data from nunchuck
    while (Wire.available ()) {

The Wire library will receive a byte that needs to be converted to an integer, by using the nunchuck_decode_byte() method, defined later in this application:

        nunchuck_buf[cnt] = nunchuck_decode_byte(Wire.receive());
        cnt++;
    }
    nunchuck_send_request();  // send request for next data payload
    // If we recieved the 6 bytes, then go print them
    if (cnt >= 5) {
        return 1;   // success
    }
    return 0; //failure
}

This method prints out the input data received from the Nunchuck:

void nunchuck_print_data()
{
    static int i=0;
    joy_x_axis = nunchuck_buf[0];
    joy_y_axis = nunchuck_buf[1];
    accel_x_axis = nunchuck_buf[2];
    accel_y_axis = nunchuck_buf[3];
    accel_z_axis = nunchuck_buf[4];

    z_button = 0;
    c_button = 0;

The sixth byte of the data from the Nunchuck, nunchuck_buf[5], contains bits that indicate whether the z or c buttons have been pressed:

// so we have to check each bit of byte outbuf[5]
    if ((nunchuck_buf[5] >> 0) & 1)
        z_button = 1;
    if ((nunchuck_buf[5] >> 1) & 1)
        c_button = 1;

The sixth byte also contains the last bits of the accelerometer data for all the axes:

if ((nunchuck_buf[5] >> 2) & 1)
        accel_x_axis += 2;
    if ((nunchuck_buf[5] >> 3) & 1)
        accel_x_axis += 1;

    if ((nunchuck_buf[5] >> 4) & 1)
        accel_y_axis += 2;
    if ((nunchuck_buf[5] >> 5) & 1)
        accel_y_axis += 1;

    if ((nunchuck_buf[5] >> 6) & 1)
        accel_z_axis += 2;
    if ((nunchuck_buf[5] >> 7) & 1)
        accel_z_axis += 1;

    i++;
}

// encode data to format that most wiimote drivers except
// only needed if you use one of the regular wiimote drivers
char nunchuck_decode_byte (char x)
{
    x = (x ^ 0x17) + 0x17;
    return x;
}

// returns zbutton state: 1=pressed, 0=notpressed
int nunchuck_zbutton()
{
    return ((nunchuck_buf[5] >> 0) & 1) ? 0 : 1;  // voodoo
}

The next step would be to use the progress of the two servo motors in a Processing application. This could be easily extended to update another set of servo motors in a different location using a network, to drive complex graphics, to control sounds or video, or to implement any number of other approaches. For the sake of brevity, in this example the Processing application simply draws current positions of the servo motors:

import processing.serial.*;
Serial serial;  // The serial port
int firstAng = 0;
int secAng = 0;
byte[] inbyte = new byte[4];

void setup() {
    size(400, 400);
    String[] arr = Serial.list();
    if(arr.length > 0) {
        serial = new Serial(this, Serial.list()[0], 9600);
    }
}

void draw() {
    background(122);
    int i = 0;
    boolean changed = false;
    if(serial != null) {
        while (serial.available() > 0) {
            inbyte[i] = byte(serial.read());
            changed = true;
            i++;
        }
        if(changed) {
            firstAng = inbyte[0];
            secAng = inbyte[2];
        }
    }

    ellipse(40, 40, 50, 50);
    line(40, 40, 40+(25*cos(firstAng)), 40+(25*sin(firstAng)));
    ellipse(120, 40, 50, 50);
    line(120, 40, 120+(25*cos(secAng)), 40+(25*sin(secAng)));

}

The Nunchuck is a very cool tool, but the Wii Remote is a far more versatile tool because it doesn’t require you to have your input device tethered to a laptop. The Wii Remote can also communicate with your laptop or desktop over a Bluetooth connection. Once the remote and your computer are connected, you send OSC messages to an oF or Processing application and use the accelerometer data from the remote in your application.

Many times you’ll find yourself piecing together different libraries to create a working solution. In this case, creating a working solution that is truly cross-platform is difficult and involves using lots of libraries. Windows users should look into an application called GlovePIE, OS X users should look into an application called DarwiinOSC, and Linux users can utilize an application called WiiOSC.

Once the OSC messages are sent from the Wii controller to the oF application, you can use the ofxOSC add-on to read the messages and use them for drawing, controlling video, and so on:

#ifndef _TEST_APP
#define _TEST_APP

#include "ofMain.h"
#include "ofxOSC.h"

class remoteApp: public ofBaseApp{

public:

    void setup();
    void update();
    void updateOSC();
    void draw();

    void keyPressed(int key);
    void keyReleased(int key);
    void mouseMoved(int x, int y );
    void mouseDragged(int x, int y, int button);
    void mousePressed(int x, int y, int button);
    void mouseReleased();

    ofxOscReceiver    receiver;

These are the variables that will be store the values received from the Wii Remote:

float pitch, roll, yaw, accel;
};

#endif

Now on to the .cpp file, where the ofxOSC instance will read OSC messages from the application that handles the communication between the Wii Remote and the computer and then draws that data to the screen:

#include "remoteApp.h"
void remoteApp::setup(){

    receiver.setup( 9000 );
    ofSetVerticalSync(true);

}
void remoteApp::update(){

The call to the updateOSC() method ensures that the OSC listener grabs any new data in the OSC buffer:

    updateOSC();
}

void remoteApp::updateOSC() {

    // check for waiting messages
    while( receiver.hasWaitingMessages() ) {

Get the next message:

        ofxOscMessage m;
        receiver.getNextMessage( &m );

Check to make sure that the new message is from the Wii Remote. The getAddress() call here is using an OS X address. Depending on your operating system your address will look different:

        if ( strcmp( m.getAddress(), "/wii/1/accel/pry" ) == 0 ) {

            // verify the type
            if( m.getArgType(0) != OFXOSC_TYPE_FLOAT ) break;
            if( m.getArgType(1) != OFXOSC_TYPE_FLOAT ) break;
            if( m.getArgType(2) != OFXOSC_TYPE_FLOAT ) break;
            if( m.getArgType(3) != OFXOSC_TYPE_FLOAT ) break;
            // get the new score
            pitch = m.getArgAsFloat( 0 );
            roll = m.getArgAsFloat( 1 );
            yaw = m.getArgAsFloat( 2 );
            accel = m.getArgAsFloat( 3 );
        }
    }
}

void remoteApp::draw(){

Draw the background according to the pitch, and then use those values to create the background color of the application:

    ofBackground(pitch*255, roll*255, yaw*255);
    ofSetColor(yaw*255,pitch*255,roll*255);
    ofFill();

Draw the two circles using the pitch, yaw, and roll values sent from the Wii Remote:

    ofEllipse(pitch*ofGetWidth(), yaw*ofGetHeight(), 20, 20);
    ofEllipse(yaw*ofGetWidth(), roll*ofGetHeight(), 10, 10);

}

It’s important to remember that Wii data is, compared to the touchscreens discussed earlier, quite difficult data to work with; it’s erratic, and it’s often difficult to determine the exact position of a cursor as one would with a mouse or a pair of potentiometers. For this reason, accelerometer-based controls like the Wii Remote are sometimes not appropriate for building applications that require cursor control. However, there is another way to use the Wii Remote as an infrared pointing device. This is often more appropriate than accelerometer data because it is so much more precise. Accelerometer data can be used to position a cursor as well, but it requires some rather tricky calculations that, although certainly within the realm of possibility, are outside of the scope of this book.

There’s another possibility for working with the Wii Remote that provides other sorts of opportunities to work with interaction: in addition to accelerometers, the Wii Remote can also calibrate its position using an infrared sensor (Figure 16-10). To use this, you need to get four small IR lights and put them in front of the Wii Remote where it will be able to detect them. To create a 360-degree experience, you could use more IR lights and position them in multiple locations around the Wii Remote, though this is substantially more complex.

Figure 16-10. Creating an LED array to use with the Nunchuck

The first step in using the Wii Remote as a pointer is to read the IR data sent from the controller. This is sent as 12 floating-point numbers. The oscP5 code calls ir when new IR data is received and gives us 12 parameters. Those are four times x, y, and brightness values for each tracked LED. If the brightness value is bigger than 15, no point is recognized. To get usable x and y position values, you will want to pass them through a method that might look like this:

float ir[12]
void ir( float f10, float f11,float f12, float f20,float f21,
    float f22, float f30, float f31, float f32, float f40,
    float f41, float f42 ) { // first all the numbers
    ir[0] = f10;
    ir[1] = f11;
    ir[2] = f12;
    ir[3] = f20;
    ir[4] = f21;
    ir[5] = f22;
    ir[6] = f30;
    ir[7] = f31;
    ir[8] = f32;
    ir[9] = f40;
    ir[10] = f41;
    ir[11] = f42;

    points = 0;

We want every third value for the x position and every fourth digit for the y position:

    for (int i = 0; i < 4; i += 3) {
        if (ir[i+2] < 15) {
            x[points] = 0.5 - ir[i]; // average out the x values
            y[points] = 0.5 - ir[i+1]; // average out the y values
          points++;
        }
    }
}

The trickiest part of getting and using the infrared data is correctly parsing out the x and y values. Once you’ve done that, it’s quite easy to use them in an oF or Processing application. Over OSC, the IR data will be passed as 12 floating-point values, and you simply call this to have all the infrared data messages plugged into the ir array:

osc.plug(this,"ir","/wii/irdata");

This simple Processing application draws a rectangle to the screen at the locations of the infrared data:

import oscP5;

float ir[12];
OscP5 osc;

void setup() {
    size(800,600);

    // open an udp port for listening to incoming osc messages
       //from darwiinremoteOSC
    osc = new OscP5(this,5600);

    osc.plug(this,"ir","/wii/irdata");
    osc.plug(this,"connected","/wii/connected");
}

void ir(
float f10, float f11,float f12,
float f20,float f21, float f22,
float f30, float f31, float f32,
float f40, float f41, float f42
) {
    ir[0] = f10;
    ir[1] = f11;
    ir[2] = f12;
    ir[3] = f20;
    ir[4] = f21;
    ir[5] = f22;
    ir[6] = f30;
    ir[7] = f31;
    ir[8] = f32;
    ir[9] = f40;
    ir[10] = f41;
    ir[11] = f42;
}

void draw() {
    for(int i=0;i<12;i+=3) {

Every third value in the array is the size, and a size of either 0 or 15 indicates that IR point is not available:

    if(ir[i+2]<15 && ir[i+2]>0) {
        fill(255, 0, 0);

Now we’ll draw a rectangle at the point where the Wii Remote detected the light:

        rect(ir[i] * width, ir[i+1] * height, 5, 5);
    }
  }
}