Now that you’ve been introduced to the different tools that you can use to create interactive applications using hardware and software, this book will move on to some of the major themes in interactivity. Understanding how you can use sound, physical controls, video, and graphics in interaction is more than a matter of just looking at code; it also requires that you understand some of the theory and fundamentals behind each of these methods. This “Themes” part looks at some of the most commonly explored means of creating interaction between users and systems and shows you how to create examples of those themes in the appropriate technology. You can create some kinds of applications only in Arduino or openFrameworks, while in other cases, the technology that you use is entirely up to you.

Interaction design is more than just writing code; it’s also understanding how the approach that you’re taking to interaction creates meaning for the user and for your system. Good interaction requires good input and feedback, and will vary depending on the medium that you’re using to communicate. For example, audio presents much different challenges than video does, and integrating different mediums for input and output requires that you match up the strengths, weaknesses, and differences of the mediums creatively.

Throughout this part, you’ll be exploring different techniques for writing code, using sensors and controls, and integrating preexisting libraries into openFrameworks, Processing, and Arduino applications. Chapter 7, Sound and Audio, introduces working with audio in applications and shows you some of the techniques and libraries for processing signals, playing files, and generating sound. Chapter 8, Physical Input, introduces you to working with physical input via controls and sensors for Arduino. Chapter 9, Programming Graphics, shows you some more advanced techniques for creating vector graphics and basic physics on a screen with Processing and openFrameworks. Chapter 10, Bitmaps and Pixels, focuses on using bitmaps and videos and how to work with and manipulate individual pixels in openFrameworks and Processing. Chapter 11, Physical Feedback, introduces you to components and techniques that you can leverage to create physical feedback in your application with the Arduino controller. Finally, Chapter 12, Protocols and Communication, shows you how to work with networks and data protocols with all three of the tools.

By the end of this part, you will have a good introduction to several of the key methods of input and feedback that you can use in your own interactive designs.