Hour 9. Multi-Touch User Input

This hour covers the touchscreen: getting both single and multiple touch events from an Android device. Although Android supports a soft keyboard and a hardware keyboard (via Bluetooth or USB), we will concern ourselves only with touch-screen input because that is the most prevalent form of input on Android 4.0 devices.

Single-Touch Input

There are several ways to get user input on an Android device. The most common form of input on the newest devices is with a multi-touch-capable screen. Additionally, Android supports input via keyboard. The accelerometer and compass might be considered inputs as well. All devices that support Android 4.0 will have a touch screen and soft keyboard, but a Bluetooth keyboard may also be used for input. Most new games will feature touch input, although it would make sense for less interactive apps to make use of a keyboard. Because keyboard input is not going to be common in touch-enabled video games, we won’t add keyboard input to our “toolbox.”

“Listening” for Touch Events

To support touch input, you need to import OnTouchListener, a subclass below View, like so:

import android.view.View.OnTouchListener;

Next, a class must implement OnTouchListener to receive input events. The most common class to implement it will be your main Activity class (which we have been calling Main or Game). You could also create a custom new class designed solely to handle input, but I have found that to require some extra steps with little benefit. What we’re really going for is a Game class that already has a lot of reusable code in it that we can use to make new games fairly easily.

In the class definition, you will need to add implements OnTouchListener to the class definition. Here is how we will add touch input support to an Activity-based program:

public class Game extends Activity implements OnTouchListener

When adding implements OnTouchListener to a class, that class must implement a method called onTouch(). Here it is:

@Override public boolean onTouch(View v, MotionEvent event) {
    return false;
}

This is the default form of onTouch(), which returns false. This tells the calling class (which generates onTouch() events for us) that it should handle the input event because we did not do anything with it ourselves (speaking as the implemented method here). When you do handle the input event and want to conclude it, return false to tell the caller that no more input processing is needed.

The MotionEvent parameter contains all the info we need to discern where a touch event occurred. Most touch events will be treated as a MOVE event, with an additional event when you lift your finger, generating an UP event.

Single-Touch Input Demo

The complete source code for a program that demonstrates single-touch input is listed next. Figure 9.1 shows the program with a MOVE event registered (while touching or dragging a finger across the display), and Figure 9.2 shows the program after releasing the touch.

Figure 9.1. Single-touch input registers as a MOVE event.

Figure 9.2. Releasing the single-touch input produces an UP event.

package android.program;
import java.io.*;
import android.app.Activity;
import android.content.Context;
import android.content.res.AssetManager;
import android.graphics.*;
import android.os.Bundle;
import android.view.*;
import android.view.View.OnTouchListener;

public class Game extends Activity implements OnTouchListener {
    DrawView drawView;

    //input variables
    Point touch = new Point(0,0);
    String inputAction = "";

    @Override public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        requestWindowFeature(Window.FEATURE_NO_TITLE);
        drawView = new DrawView(this);
        setContentView(drawView);

        //turn on touch listening
        drawView.setOnTouchListener(this);
    }

    @Override public void onResume() {
        super.onResume();
        drawView.resume();
    }

    @Override public void onPause() {
        super.onPause();
        drawView.pause();
    }

    //touch listener event
    @Override public boolean onTouch(View v, MotionEvent event) {
        switch(event.getAction()) {
            case MotionEvent.ACTION_DOWN:
                inputAction = "DOWN";
                break;
            case MotionEvent.ACTION_MOVE:
                inputAction = "MOVE";
                break;
            case MotionEvent.ACTION_UP:
                inputAction = "UP";
                break;
        }

        //remember this touch position
        touch.x = (int)event.getX();
        touch.y = (int)event.getY();

        //notify touch handler that we used it
        return true;
    }

    public class DrawView extends SurfaceView implements Runnable
    {
        //game loop thread
        Thread gameloop = null;

        //surface holder
        SurfaceHolder surface = null;

        //thread-safe running var
        volatile boolean running = false;

        //asset manager
        AssetManager assets = null;

        //constructor method
        public DrawView(Context context) {
            super(context);

            //get the SurfaceHolder object
            surface = getHolder();

            //create the asset manager
            assets = context.getAssets();
        }

        public void resume() {
            running = true;
            gameloop = new Thread(this);
            gameloop.start();
        }

        public void pause() {
            running = false;
            while (true) {
                try {
                    gameloop.join();
                }
                catch (InterruptedException e) { }
            }
        }

        //thread run method
        @Override public void run() {

            while (running) {
                if (!surface.getSurface().isValid()) continue;

                //open the canvas for drawing
                Canvas canvas = surface.lockCanvas();
                canvas.drawColor(Color.BLACK);

                //draw circle at touch position
                Paint paint = new Paint();
                paint.setColor(Color.WHITE);
                paint.setTextSize(24);
                canvas.drawText("Touch screen to test single touch input",
                    10, 20, paint);
                canvas.drawText("Action: " + inputAction, 10, 50, paint);
                canvas.drawText("Position: " + touch.x + "," + touch.y,
                    10, 80, paint);

                if (touch.x != 0 && touch.y != 0)
                    canvas.drawCircle(touch.x, touch.y, 50, paint);

                //close the canvas
                surface.unlockCanvasAndPost(canvas);

                try {
                    Thread.sleep(20);
                }
                catch(InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

Multi-Touch Input

The latest version of the Android SDK (at the time of this writing) allows you to tether the emulator to a physical Android device to test multi-touch functionality. This sounds like an odd suggestion, but the real benefit that the emulator affords a developer is the capability to test code on a variety of devices with different screen resolutions. Having any Android device for development, even an older one (not running the target OS version) can be a benefit. You can, for instance, run your code on the emulator running an Android 4.0 “tablet” device, while using your tethered Android phone for multi-touch. The screen resolutions do not have to match, nor does the OS version have to be the same.

The multi-touch input system is a bit convoluted, but nothing we can’t handle. The way it used to work was by using a pointer identifier to represent each touch input source. Rather than return an array of input points, Android gave an identifier that was then used as an index. Fortunately, later versions of the SDK eliminated this strange behavior and switched to a simple indexed system.

Quick Example

Here is a very quick example that reports the position of multiple touch pointers using the log, just to give you the basic method calls at a glance. For multi-touch input, you need to iterate through each touch input, where the number of inputs is returned by MotionEvent.getPointerCount(). After that, you can use overloads of getX() and getY() with the index of the input to get its position.

@Override public boolean onTouch(View v, MotionEvent event) {
    int numPoints = event.getPointerCount();
    for (int n=0; n<numPoints; n++) {
        int x = (int)event.getX(n);
        int y = (int)event.getY(n);
        Log.d("Pointer", "Touch " + n + ": x=" + event.getX(n) +", " +
            y="+event.getY(n));
    }
    return true;
}

To see the output of the Log, you need to switch to the DDMS perspective. The normal editing environment is called the Java perspective. You should see the perspectives at the upper-right corner of the Eclipse window, as shown in Figure 9.3. This figure also shows the log output window at the bottom. You can use Log anytime you want to quickly display information without going through the trouble of outputting text to the screen using Context.drawText(). Most coders don’t normally work with Eclipse in this small of a window—it was resized for the figure.

Figure 9.3. Viewing the Log from the DDMS perspective in Eclipse.

Encapsulating Multi-touch Input

The simple example was pretty straightforward, but we can still improve on this code by molding it into a shape that is easier to reuse, requiring less Android work, so to speak, to get input in a game project. The quick example showed the basic data we would expect from multi-touch input, where several points of input are reported. What we want to do is get the values from multiple touch inputs and store those values so they can be used in any way we want during that game loop cycle (that is, during that “frame”).

First, we’ll create an array to keep track of touch points:

Point[] points;

We’ll assume that 5 is the maximum number of inputs. Realistically, you won’t need more than this for most games. The exception might be a larger tablet with a game that supports multiple players using the same tablet. Yes, you can do that—support up to five local players—but each player would get only one input, and they could interfere with each other by using two fingers.

Next, initialize the array:

points = new Point[5];
for (int n=0; n<5; n++) {
    points[n] = new Point(0,0);
}

Now, inside the onTouch() event, we can fill the points array with touch input point values that can be used at any time. It is more efficient this way, rather than going back to the MotionEvent parameter each time. You could probably create a global copy of MotionEvent, but all you need are the points!

To ensure that we don’t go out of the array’s bounds, we’ll get the number of touch points and adjust the total if needed. That is being handled by a global int called numPoints:

numPoints = event.getPointerCount();
if (numPoints > 5) numPoints = 5;

If an Android device supports more than five touch inputs at a time, you may want to support that device by modifying the code. This is just a quick example to show how it works!

Next, we grab the touch points out of the MotionEvent parameter:

for (int n=0; n<numPoints; n++) {
    points[n].x = (int)event.getX(n);
    points[n].y = (int)event.getY(n);
}

The Multi-touch Demo

The Multi-touch Demo is shown in Figure 9.4. This example shows how to read the touchscreen for up to five touch input points. The index number and position of each point is displayed, and a circle is drawn at each point, using a different color for each point.

Figure 9.4. The Multi-touch Demo.

package android.program;
import android.app.Activity;
import android.content.Context;
import android.content.res.AssetManager;
import android.graphics.*;
import android.os.Bundle;
import android.view.*;
import android.view.View.OnTouchListener;

public class Game extends Activity implements OnTouchListener {
    DrawView drawView = null;
    Paint paint = null;
    Point[] points;
    int numPoints=0;

    @Override public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        requestWindowFeature(Window.FEATURE_NO_TITLE);

        //create the view object
        drawView = new DrawView(this);
        setContentView(drawView);

        //create a paint object
        paint = new Paint();
        paint.setTextSize(24);

        //turn on touch listening
        drawView.setOnTouchListener(this);

        //create the points array
        points = new Point[5];
        for (int n=0; n<5; n++) {
            points[n] = new Point(0,0);
        }
    }

    @Override public void onResume() {
        super.onResume();
        drawView.resume();
    }

    @Override public void onPause() {
        super.onPause();
        drawView.pause();
    }

    //touch listener event
    @Override public boolean onTouch(View v, MotionEvent event) {

        //count the touch inputs
        numPoints = event.getPointerCount();
        if (numPoints > 5) numPoints = 5;

        //store the input values
        for (int n=0; n<numPoints; n++) {
            points[n].x = (int)event.getX(n);
            points[n].y = (int)event.getY(n);
        }

        return true;
    }

    public class DrawView extends SurfaceView implements Runnable {
        Thread gameloop = null;
        SurfaceHolder surface = null;
        volatile boolean running = false;
        AssetManager assets = null;
        int[] colors;

        public DrawView(Context context) {
            super(context);
            surface = getHolder();
            assets = context.getAssets();

            //create an array of colors
            colors = new int[5];
            colors[0] = Color.GREEN;
            colors[1] = Color.MAGENTA;
            colors[2] = Color.YELLOW;
            colors[3] = Color.RED;
            colors[4] = Color.CYAN;
        }

        public void resume() {
            running = true;
            gameloop = new Thread(this);
            gameloop.start();
        }

        public void pause() {
            running = false;
            while (true) {
                try {
                    gameloop.join();
                }
                catch (InterruptedException e) { }
            }
        }

        @Override public void run() {

            while (running) {
                if (!surface.getSurface().isValid()) continue;
                Canvas canvas = surface.lockCanvas();
                canvas.drawColor(Color.BLACK);

                paint.setColor(Color.WHITE);
                canvas.drawText("Multi touch input demo", 10, 20,
                paint);
                canvas.drawText("Number of inputs: " + numPoints,
                10, 50, paint);

                int y=80;
                for (int n=0; n<numPoints; n++) {
                    paint.setColor(colors[n]);

                    //display point values
                    String s = " " + n + ": " + points[n].toString();
                    canvas.drawText(s, 10, y, paint);
                    y += 30;

                    //draw a circle at each point
                    if (points[n].x != 0 && points[n].y != 0)
                        canvas.drawCircle(points[n].x, points[n].y,
                        50, paint);
                }

                surface.unlockCanvasAndPost(canvas);

                try {
                    Thread.sleep(20);
                }
                catch(InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

Summary

This hour taught you how to use the touch screen. You learned how to get single-touch input (which is similar to mouse input). You also learned how to get multiple touch input by detecting the number of touch inputs and reading the touch positions. The code to get touch input was fairly easy to write, but we could still improve on it to make it easier to use. We will keep this code handy for the game framework project coming up in a future hour.

Q&A

Q. User input is a more complex subject than it first appears to a programmer. From a design perspective, user input is crucial and not just a matter of getting touch coordinates. The user interface must be designed with fun in mind. The term for the study of fun in gameplay is called Funativity. Discuss this topic. You may want to search online references for more information.

A. Answers will vary.

Q. What types of games most benefit from single-touch input, and what types benefit most from multi-touch input? Discuss the differences among games that use the two methods. What are perhaps the most significant examples in the game market of each?

A. Answers will vary.

Workshop

Quiz

1. What is the primary interface class that gives access to touch input?

2. What method (usually called from Activity.onCreate()) initializes the touch input system?

3. Which event method reports touch input events?

Answers

1. OnTouchListener

2. SurfaceView.setOnTouchListener()

3. OnTouchListener.onTouch()

Activities

Write a pair of helper methods that encapsulate touch input for future use. Give them any name you want, but make them return the results of single touch and multiple touch inputs. You should be able to call these functions and receive back (as a return value) the coordinates, or null if no input was detected.

Hint: To make this work, some globals will need to be used.