CHAPTER 23

Sensors

Sensors are critically important building blocks of the digital world yet in their ubiquity are often invisible. Not only are they worth appreciating for the elegance and cleverness of their engineering, but also sensors are on their way to generating a truly unimaginable proportion of the planet's information. What held true for code is doubly true of sensors: Value judgments, future possibilities, and economic interests are represented in architectures that typically operate beneath the threshold of human consciousness.

Historical Roots

To understand the current sensor landscape, let us step back for a moment to see its antecedents. Originally, a variety of sensors were invented to augment human senses. Examples include the telescope, microscope, ear trumpet, hearing aids, and other devices. With the advent of electro-optics and electromechanical devices, new sensors could be developed to extend the human senses into different parts of the spectrum (e.g., infrared, radio frequencies, measurement of vibration, underwater acoustics, etc.). Where they were available, electromechanical sensors:

• Stood alone
• Measured one and only one thing
• Cost a lot to develop and implement
• Had inflexible architectures: They did not adapt well to changing circumstances

Let's discuss each of these points in turn. Sensors traditionally stood alone because networking them together was expensive and difficult. Shared technical standards were rare, so if one wanted a network of, say, offshore data buoys, the system of connections would be uniquely engineered to a particular domain. Someone connecting sensors of a different sort (such as surveillance cameras) would have to start from scratch, as would anyone monitoring road traffic.

In part because of their mechanical componentry, sensors rarely measured across multiple yardsticks. The oven thermometer measured only the oven temperature and displayed the information locally, if at all. The electric meter only counted watt-hours. Now it's common for a consumer Global Positioning System (GPS) unit, for example, to tell location, altitude, compass heading, and temperature, along with providing weather radio.

Sensors were not usually mass-produced, with the exception of common items, such as thermometers. Because supply was limited, particularly for specialized designs, the combination of monopoly supply and small order quantities kept prices high.

The rigid architecture was a function of mechanical devices' specificity. A vibration sensor was different from a camera was different from a humidistat. Humidity data, in turn, was designed to be moved and managed in a particular analog domain (a range of zero to 100%), while image recognition in the camera's information chain typically featured extensive use of human eyes rather than automated processing.

Ubiquity

Changes in each of these four facets combine to help create today's emerging sensor networks, which are growing in scope and capability every year. The many examples of sensor capability accessible to (or surveilling) the everyday citizen shows the limits of the former regime:

• Computers, which sense their own temperature, location, user patterns, number of printer pages generated, and so on.
• Thermostats, which are networked within buildings and now remotely controlled and readable.
• Telephones, the wireless variety of which can be understood as beacons, bar-code scanners, pattern-matchers (the Shazam application names songs from a brief audio sample), and network nodes.
• Motor and other industrial controllers, including drive-by-wire throttle linkages, automated tire-pressure monitoring, and airbags' accelerometers and high-speed actuators.
• Vehicle components, often including an on-board diagnostics module, a toll pass, satellite devices on heavy trucks, and theft recovery services such as LoJack, not to mention the inevitable mobile phone.
• Surveillance cameras (of which there are over 10,000 in Chicago alone, and more than 500,000 in London, England).¹
• Some hotel door handles and many minibars, which are instrumented and generate electronic records of people's and vodka bottles' comings and goings.
• Physical sensors, whether embedded in animals (radio-frequency identification [RFID] chips in both household pets and racehorses) or gardens (the EasyBloom plant moisture sensor connects to a computer via USB and costs only $50), or affixed to pharmaceutical packaging.

Note the migration from heavily capital-intensive or national-security applications down-market. A company called Vitality has even developed a monitoring system for something as simple as a pill bottle: If the cap is not removed when medicine is due, an audible alert is triggered or a text message could be sent.²

A relatively innovative industrial deployment of vibration sensors illustrates the state of the traditional field. In 2006, BP instrumented an oil tanker with “motes,” which integrated a processor, solid-state memory, a radio, and an input/output board on a single two-inch-square chip. Each mote could receive vibration data from up to 10 accelerometers, which were mounted on pumps and motors in the ship's engine room. The goal was to determine if vibration data could predict mechanical failure, thus turning estimates—a motor teardown every 2,000 hours, to take a hypothetical example—into concrete evidence of an impending need for service.

The motes had a decided advantage over traditional sensor deployments in that they operated over wireless spectrum. While this introduced engineering challenges arising from the steel environment of ships as well as the need for batteries and associated issues (such as the fact that the lithium in some batteries is a hazardous material), the motes and their associated sensors were much more flexible and cost effective to implement compared to hard-wired solutions. The motes also communicate with each other in a mesh topology: Each mote looks for nearby motes, which then serve as repeaters en route to the data's ultimate destination. Mesh networks are usually dynamic: If a mote fails, the signal is routed to other nearby devices, making the system fault-tolerant in a harsh environment. Finally, the motes could perform signal processing on the chip, reducing the volume of data that had to be transmitted to the computer where analysis and predictive modeling was conducted. This blurring of the lines between sensing, processing, and networking elements is occurring in many other domains as well.³

All told, there are dozens of billions of items that can connect and combine in new ways. The Internet has become a common ground for many of these devices, enabling multiple sensor feeds—traffic camera, temperature, weather map, social media reports, for example—to combine into more useful, and usable, applications.

A popular term for this state of affairs is “the Internet of Things.”⁴ As we saw earlier, network effects and positive feedback loops mean that considerable momentum can develop as more and more instances converge on shared standards. While we will not discuss them in detail here, it can be helpful to think of three categories of sensor interaction:

1. Sensor to people. The thermostat at the ski house tells the occupants that the furnace is broken the day before they arrive, or a dashboard light alerts the driver that the tire pressure on the car is low.
2. Sensor to sensor. The rain sensor in the automobile windshield alerts the antilock brakes of wet road conditions and the need for different traction-control algorithms.
3. Sensor to computer/aggregator Dozens of cell phones on a freeway can serve as beacons for a traffic-notification site, at much lower cost than helicopters or “smart highways.”

Current Examples

Less abstractly, the most mundane, low-tech activities of daily life are being transformed. Here are three: housecleaning, running, and parking.

Vacuuming

The Roomba robotic vacuum cleaner (see Figure 23.1) was introduced in 2002 by iRobot, an MIT spinout that got its start building military robots for cargo hauling and mine sniffing. Roombas randomly cover an area, sensing walls and furniture, before retreating to a predefined dock for recharging. They are not particularly powerful but do spare people the drudgery of one of housecleaning's least rewarding tasks. As of 2010, more than 5 million household robots had been sold, primarily the Roomba but also sibling units for floor washing, gutter cleaning, and other tasks.

The device found a ready audience as befits a classic disruptive innovation: It underperformed the existing market on traditional standards, such as suction or dust-bag capacity, but introduced an entirely different axis of competition: freedom from drudgery and an acceptably clean floor with zero effort. The Roomba quickly became a popular item on wedding registries.

FIGURE 23.1 Roomba Robotic Vacuum Cleaner
Photo courtesy of iRobot Corporation.

Once customers obtained their Roombas, a funny thing started happening: Many people named the appliances and described them in human terms. Seventy percent of people in a survey reported giving their robot a name, a gender, and a place in the family hierarchy. Broken robots were known to be “hospitalized.” The device does require maintenance: Fine dust can clog the sensors, for example, but people far preferred “grooming” their robots to using a traditional appliance.

People reported adjusting both their habits (picking up more carefully the day before a scheduled run) or rearranging furniture in response to the device. One person threw away a shag rug because the Roomba was “getting frustrated” trying to clean it. Promotion was common: Owners would take the Roomba to their parents' house to show it off, for example, but felt protective of the device because of the hazards of a new, not-yet-optimized environment.⁵

The outcome is counterintuitive: These robots, which are in essence a bunch of sensors and actuators networked through computing, evoke emotional responses in people far more than the Roombas can sense anything about the people. To oversimplify, the inanimate watcher/doer becomes an emotional presence in the human environment. The phenomenon is not new: Many people name their Roomba R2D2, others the Terminator, evoking robotic movie characters that generated large followings. In other cases, the vacuum becomes secondary: iRobot also sells educational robot programming kits, essentially Roombas without the primary functionality.

Running

Most people's everyday experience with mobile wireless data originated with smart devices like a BlackBerry pager or iPhone or an OnStar telematics system. The running of the Boston Marathon illustrates another scenario. Ever since 1996, each of up to 40,000 runners had a transponder about the size of a quarter on his or her shoe. Hardwired sensor mats keep track of runners' progress and prevent cheating. More recently, the system sends automatic e-mails to prearranged addresses, notifying fans of their friend's progress and projected finish time.⁶

In 2008, Nike teamed with Apple to link an accelerometer in the shoe (that counted footstrikes) to an iPod via a short-range wireless connection. (This arrangement was conceptualized as a PAN, or Personal Area Network. Bluetooth headsets are a classic example that uses the same technology as the Nike+.) The iPod supplies workout music and also provides audio feedback on time and distance markers, congratulations from Nike celebrity endorsers, and cumulative statistics. Maintaining personal fitness data became simpler, and then the sensors—more technically, the sensors' home PCs—became networked, enabling people from different cities to “race” against each other or simply to compare their personal performance on different days. As of mid-2011, Nike+ surpassed 425 million cumulative miles logged. The feedback provided by the electronic sensor is of an entirely different quality compared to a pedometer, paper mileage log, or other traditional device. In addition, the solitary activity becomes social; virtual teams compete to hit goals or beat other teams, often in conjunction with an actual event, such as a major marathon.

Parking

Parking meters^* are hardly glamorous and rarely connote particularly advanced technology, but networking them together provides unexpected benefits and consequences. A pilot deployment in San Francisco addresses many issues.

REAL-TIME INFORMATION San Francisco's population can nearly double in a normal weekday, from 700,000 to 1.3 million. Up to 30% of automobile traffic is estimated to be generated by people cruising, looking for an open spot. GPS sensors that record arrivals and departures for each spot are connected via wireless network to the city's parking authority, where the information is posted to a Web site: all of the available spots are mapped, and inventory levels are current.

SUPPLY AND DEMAND Given accurate, up-to-date knowledge of available inventory, the city then prices parking in such a way as to maintain 15% of all spots open at a given time. High prices will deter some drivers, while heavy fines for overstaying one's meter might shorten visits.

PAYBACK The sensor technology is clearly expensive to install, but so too are the millions of person-hours wasted each day. Variable congestion pricing has been implemented in other cities, such as Stockholm and Singapore, but only at toll stations and/or bridges. Moving the locus of congestion pricing to the extended parking infrastructure eliminates a choke point and theoretically can dynamically adjust pricing to reflect current supply and demand.⁷

Phones as Sensors

Several factors make the rapid deployment of “smart” phones (that get smarter every year) a powerful force for change. We have more to say about these devices in other Chapters 12 and 27, but purely from a sensor point of view, eight factors are relevant:

1. Massive deployment. On a planet of roughly 7 billion, there are more than 5 billion mobile phones in use.
2. Networked. For every camera, every accelerometer, and every geolocator in these phones, there is a radio transmitter inches away.
3. Powered. Getting power to a sensor network can be remarkably complicated. Those 5 billion sensor platforms are each attached to a human who presumably takes time to charge the device regularly.
4. Human deployed. Imagine having to build a network of surveillance cameras to cover miles of freeway or hundreds of city blocks. The cost is considerable, and the odds of having a major event occur out of coverage remain high. With humans in a subway explosion, or near an accident site, or in the presence of a strange odor, people will reach for mobile devices and start generating images, coordinates, and other information immediately.
5. A shared platform. Every mobile phone in the world can theoretically, if not economically, connect to any other.^* No other sensor platform in history has been so big and so interconnected. Thermometers, speedometers, scales, radar—all of these stood alone.
6. Read/write. In addition to sensing motion, or light, or bar codes, or sound, mobile phones can display results, processed versions of the input, authentication tokens, time/date stamps, and many other aspects of sensor-related information. Whereas a vibration sensor on a fence, for example, can send only what it measures, a camera that photographs a suspicious person or activity is connected to a display of image-recognition results, mug shots, or other relevant information.
7. Metadata enabled. Not only can a cameraphone record and display a picture, it can encode and decode information about the current and past photos of the same attraction or person, announcing who or what the subject of the photo is. Facebook photo recognition and augmented reality apps such as Word Lens (which translates text of signs) are familiar examples.
8. Mobile. Fixed sensor deployments required often highly sophisticated modeling to determine the location of sensors, networking infrastructure, power, security considerations, and other factors. Allowing the sensors to move can create gaps in coverage but also makes possible new kinds of approaches to data collection.

Looking Ahead

The whole planet is being instrumented from the bottom up—Weather Underground's network of more than 30,000 volunteer weather stations is a long-standing example—with help from some very expensive top-down infrastructure in the form of GPS along with open-access satellite and aerial imagery on Google Earth and elsewhere. Inexpensive and often redundant hardware, running on mobile and wireless platforms, creates new possibilities for discovery, for convenience, and also for the invasion of privacy. Given the speed of deployment and innovation, we will be seeing more questions than answers in the coming years.

Notes

1. Brian Palmer, “Big Apple Is Watching You,” Slate, May 3, 2010, www.slate.com/id/2252729/.

2. Llinca Nita, “Pill Bottle Caps to Call You via AT&T and Remind You to Take Your Medication,” Unwiredview.com, October 8, 2009, www.unwiredview.com/2009/10/08/pill-bottle-caps-to-call-you-via-att-and-remind-you-to-takeyour-medicine/.

3. Tom Kevan, “Shipboard Machine Monitoring for Predictive Maintenance,” Sensors Mag, February 1, 2006, www.sensorsmag.com/sensors-mag/shipboard-machine-monitoring-predictive-maintenance-715?print=1.

4. Inge Gronbaek, “Connecting Objects in the Internet of Things (IoT),” Telenor Research & Innovation Research Report (June 2008), www.telektronikk.com/volumes/pdf/2…/Tel_2-08_Page_109-120.pdf www.telenor.com/en/innovation/research/publications/reports/2008.

5. Ja-Young Song, Lan Guo, Rebecca E. Grinter, and Kenrik I. Christensen, “‘My Roomba Is Rambo’: Intimate Home Appliances,” in J. Krumm, G. D. Abowd, A. Seneviratne, and Th. Strang (eds.), Ubicomp 2007, (Berlin, Germany: Springer-Verlag, 2007), pp. 145–162.

6. Fred O'Connor, “RFID Helps the Boston Marathon Run,” InfoWorld, April 9, 2007, www.infoworld.com/t/networking/rfid-helps-boston-marathon-run-441.

7. Lisa Camner, “Car Talk,” Atlantic Monthly blog, May 24, 2010, www.theatlantic.com/personal/archive/2010/05/car-talk/56983/.

8. Joe Lindsey, “EpicMix Lets You Get Your Hero On,” Wired.com, August 31, 2011, www.wired.com/playbook/2011/08/epicmix-lets-you-get-your-hero-on/.

*The technology that allows a car to automatically parallel park operates on a different, but no less interesting, set of sensors and controllers. See, for example, the Lexus system at http://bit.ly/480EvM.

*Subject to the technical and economic limits on the number of possible combinations imposed by the various network layers.