CHAPTER 9

The Future of Event Processing

Complex event processing (CEP) will become part of the supporting technology service within the enterprise information infrastructure of the future. Some of the systems that use CEP as a supporting technology will evolve into global holistic event processing systems:

• Event processing becomes a supporting technology within the IT infrastructure
• Large-scale holistic event processing systems appear
• CEP disappears under the hood of these event-driven systems
• Businesses take a global view of information and events
• Five example areas where holistic event processing systems will emerge:

This chapter is about the kinds of holistic event processing systems that can be expected to emerge in the future and the role CEP will play in them. It is not about what will happen with absolute certainty. Rather, it is a vision of what may probably happen in the longer term and the reasons why that vision might turn out to be correct.

Taking Stock

At the present time, CEP is an event processing technology that is being marketed either as a standalone tool or as a visible component of a suite of business intelligence tools. Eventually, it will become a foundational technology within large application systems.

We believe CEP will become largely invisible to the average user of business event processing systems. This does not imply that CEP is somehow a failure; on the contrary. it will become an all pervasive essential technology—just as the TCP/IP network protocols are critical to our Internet systems today. The features of CEP described in Chapters 6 through 8, such as event patterns, abstract events, hierarchy definitions, and abstraction mappings, will be part of a technology arsenal used by technical support engineers and knowledgeable business specialists in building and using enterprise management systems. CEP technology will be available to users online through graphical interfaces and other user-friendly tools.

This is the final stage in the development of CEP technology, which we described in Chapter 4 in the section entitled “Ubiquitous CEP.” It is what we mean by disappearing under the hood of large application systems.

At present, the range of market areas for event-processing applications is expanding, as we described in Chapter 5. Sales of CEP in these markets are increasing. And new markets and new kinds of CEP products are appearing all the time. We are well into the stage of creeping CEP,1 and beginning to enter the next stage when CEP becomes an established information technology. From this position, we can look forward a few years and predict the longer-term future of CEP and of event processing in general.

First of all, present-day event processing applications are small event type space2 applications (e.g., the examples in Chapter 5). They deal with a small number of different types of event inputs and produce a small number of different types of results. Of course, they may be processing large numbers of events and high volumes of event throughput. But the point is that each of these applications is dealing with a limited variety of types of events, and they are specialized to produce a small number of types of results.

For most of the current generation of commercial event processing applications, small event type space processing is all that is needed. And it is worth noting that many of the present applications have become important components of various commercial information infrastructures. They will endure for many years. We shall refer to these kinds of event processing systems as specialized systems. There will always be a need for these systems, and they are not going to disappear!

But over time, a new category called holistic event processing systems will emerge from the melting pot of IT systems. Holistic systems will have a global span, processing event inputs from different types of sources from all over the world in order to produce a wide variety of different types of results. The event types will be a mix of structured and unstructured kinds of data, and they will be continually expanded to include more events from outside of the company (e.g., news feeds; market data feeds; Web click streams; feeds from trade associations and government agencies; communications from mobile phones such as text messages, social computing sources such as Twitter, Facebook, etc.; GPS data; perhaps new types of sensors; and of course data from application systems that do not currently emit events). The actual set of types of event inputs may be configurable dynamically during operation.

Essentially, holistic systems will be large event type space systems in contrast to today’s systems. They will employ many different, specialized technologies in their components (e.g., statistical analytic systems and predictive analytics, Monte Carlo simulation systems, constraint-based optimization systems, and rule engines) in order to satisfy the demands of different goals and objectives. CEP will be but one of these specialized technologies. Holistic systems will be open-ended, extensible event-processing systems.

This development in event processing is a natural consequence of increasing demands and expectations. Limiting the number of input event types not only helps to keep the event processing simple, but it also limits the results that the system can produce.3 Most systems will start life as small event type space systems. This is just a matter of limited planning and goals, and of course, economics. But in the future, as the number of different kinds of applications of event processing increases, large global businesses and governments will demand more results from these systems. Therefore, we can expect some event processing systems to evolve to a point where they will see simultaneous use for many different purposes. They will become not only global systems, but also holistic event processing systems.

You can already see this trend in some of the examples in Chapter 5. In the case of businesses, it is a trend toward developing management systems that encompass the total operation of the business. The idea is to achieve greater efficiency all round by using a single management system that has all aspects of the business within its reach.

There is a compelling example of broadening the space of input event types in the area of stock-trading systems. Some trading systems are now using event feeds from news aggregators, weather reports, and political news, together with sentiment attributes tagged onto each individual item of news, as additional input to trading algorithms. The idea is to factor in the old adage that “sentiment moves markets” into trading. One effect is to increase the number of types of event inputs beyond normal trading data.

Other kinds of businesses are now also exploring the use of public sentiment tagging and social media websites in the logistics that run their sales and marketing operations. The number of different types of events used in business operations is increasing, and some of these inputs would not have been predicted a short time ago!

The Evolution of Holistic Event Processing Systems

The term holistic event processing4 applies to a class of event processing systems that process many different types of event inputs and combine many event processing applications on a global scale. These systems will evolve gradually over time by combining lots of smaller, specialized event processing applications.

Holistic Event Processing System: An automated event processing system, usually global in scope, based upon the integration of a large number of diverse event processing applications, processing many different types of event inputs, and producing many different types of results.

How many EP applications and different types of events and event sources? There are no specific numbers at which point an event processing system passes some invisible boundary and “goes holistic”! Holistic systems are large event type space systems. They will often be implemented as a loosely coupled network of event processing applications running on separate engines, each application being specialized to a particular problem area. But other configurations are possible, such as a network of satellite processors doing event gathering at local areas and some preprocessing before passing events onto a large central system.

Holistic event processing is a term used to describe a scale of event processing greater than today’s systems. A holistic system will involve new technology for combining sets of event processing applications and for making predictions from combinations of outputs of sets of applications. Its outputs will go well beyond the functionality of current systems, due to the use of large sets of types of event inputs and the combination of widely diverse event processing applications. The predictive capabilities will definitely involve new technology. You will recognize a holistic event processing system when you see one. We will give examples.

The reason that holistic event processing systems will evolve is to achieve demands beyond the reach of existing individual event processing applications. These demands will require combining the results of many different event searches and event computations. Existing systems will be incapable of meeting these demands without their capabilities being extended. So event processing systems will be cobbled together, gradually one by one over time, to form a wider reaching system. Truly, Aristotle’s dictum will apply whenever such evolutionary compositions of applications are successful: “The whole is greater than the sum of the parts.”

Holistic event processing systems will not be planned or designed or built as such. They will result from evolutionary development, usually by haphazard extensions of existing systems and many trial-and-error experiments and failures.

The following points are to be emphasized:

• Holistic event processing systems will result from a series of evolutionary steps, incorporating sets of specialized EP applications over many years.
• An enabling force for holistic event processing will be the development of new methods of combining the results of different kinds of event processing using sets of event processors operating in different event type spaces to yield new types of results for which demand has arisen.
• Holistic systems will result from the changing requirements for right now information and predictions from different departments within a company or different communities within a nation. This is coupled with the need to meet economic constraints on the costs of developing new event processing systems. Instead of building new separate systems, old systems will be added to and extended beyond their original designs.
• A holistic system will produce predictions that result from abstracting the output events from sets of its component EP applications. Many of its outputs will share the results of common components. And predictions may take the form of abstract events.
• CEP concepts and tools will be employed in various roles during the evolution of a holistic EP system.
• CEP will disappear into the event processing infrastructure of holistic EP systems and will no longer be visible to the user. It will become simply an enabling technology.

What kinds of new results might be demanded? There are many examples. A retail company may want to track demographic and social trends so that it will be continuously aware of opportunities to expand its business or make acquisitions to introduce new directions in response to trends in styles and fashion. Pharmaceutical manufacturers may find advantages in tracking and predicting epidemiology events, perhaps on a worldwide scale. And of course large oil companies should be monitoring pretty much everything from the weather to politics!

Event patterns and event abstraction, supported by distributed networks of event processing engines, will be an important technology in combining widely different types of event inputs to yield understandable views from which decisions can be made and actions taken right now (consider Example 9.2).

There will be demand for holistic systems in areas well beyond business. One area is earth sciences where there is a pressing need for more accurate predictions of earthquakes, tsunamis, and other natural disasters, as well as for long-term global climate-monitoring systems. Another area is predictive systems in epidemiology, where the evidence of emerging disease epidemics is often misjudged because contributing events arise from diverse sources at different times and are not correlated to build an accurate picture of what is happening. Management of world resources, such as food and energy, is an area in which holistic event processing may evolve in support of monitoring and planning systems. And there are many areas of automation in government where holistic event processing will play a supporting role. We shall return to these example areas.

Note: Holistic event processing systems go beyond the concept of a federation6 of subsystems. A holistic system must incorporate technology for combining the events that are output by the separate CEP systems, for making predictions based upon the results, and for applying hierarchical abstraction to the resulting sets of events. It is not simply a single-access system to multiple sources. However, a federation of CEP systems may well be a step in the evolution of a holistic system.

Crossing Boundaries

The evolution of holistic event processing will have to cross boundaries. The first are the boundaries of individual CEP products and vendors, since holistic EP systems will involve combining and integrating many different CEP applications from many suppliers. Indeed, boundaries may exist within the same company between different business units with their own CEP applications and boundaries. The second are sociopolitical boundaries. In some cases, these kinds of EP systems may be seen as a threat to various social and political issues, such as privacy, the rights of the individual, and other matters beyond the scope of our discussion.

Crossing boundaries will depend upon progress on a number of technical and nontechnical enablers:

Progress on some of these enablers is already happening—for example, enabling infrastructures. The advent of cloud computing should be seen as a promising foundation for building the next-generation event processing infrastructures needed for holistic systems. And the proposed Internet of Things7 may turn out to be a very large-scale holistic system indeed, even if only parts of it such as global sensor networks come to exist.

Some efforts are also underway within official standards organizations that should eventually lay down basic standards for event processing, such as standards for event representations. As for technical innovation, that is happening all the time, although it is hard to predict.

Note: These enablers do not have to happen first before any progress towards holistic systems can take place. Development of the enablers and the holistic systems that they enable can, and most often do, take place simultaneously.

Indeed, it is hard to predict to what extent standards for event processing will be necessary. We guess that most standards will come after the first holistic systems emerge and become recognized as such, when the advantages of these systems become clear from the lessons learned. The evolution of holistic systems will be open-ended, opportunistic, and haphazard.

The Beginnings of Holistic Event Processing Systems

Today there are many examples of systems that may evolve into holistic event processing systems. This is a result of various pressures to expand the existing systems. As we described previously, some of these pressures include:

The first three pressures apply to many kinds of information systems. Example 9.2, describing a large national bank that plans to expand its lines of business beyond the domain of banking, is the result of these kinds of pressures within a single business organization.

Point 4 is an observation that powerful forces are beginning to arise in favor of achieving economic and humanitarian goals that can only be attained with the help of global event processing. Some of these we have already mentioned, such as a need for organizing warning and response systems to natural disasters like earthquakes and floods, worldwide food shortages, and disease epidemics. There are also pressures to build systems that predict these kinds of emergencies.

Some possible future holistic systems are already in the early stages of evolution today, and their evolutions are haphazard, as we predicted. This is happening:

• With little or no knowledge of CEP techniques
• With an implicit assumption that networking will be an adequate support platform for the huge amount of event inputs of many event types that are already available
• With little planning to how to support all the different kinds of event processing technologies that will be needed (e.g., event causality, correlation and abstraction, event hierarchies, communication between event processing systems, and the aggregation of the results of separate event processing systems)

Nobody is thinking about these emerging systems as being “holistic,” but that is what they will evolve to become!

Example 9.2, the large national bank that is planning to expand its businesses, is a possible example of an embryonic holistic event-processing system.

A compelling example is the United States Geological Survey (USGS) Natural Hazards website, described in Chapter 4. See the section on the creeping CEP stage of development, and Figures 4.4 and 4.5. The USGS NHSS website displays near real-time information from a large number of different types of event sources, including satellites, deep ocean buoys, surface buoys, and land-based remote automated weather stations (RAWS) around the United States,8 and in some cases around the globe, earthquake sensors, volcano monitors, water monitoring stations, forestry monitoring sensors, National Weather Service reports, and many other sources. All inputs are real time or near real time. Refresh and update rates for inputs vary from minutes to two hours. All in all, the actual number of input event sources to this facility must number in the tens of thousands.

The NHSS information is displayed on a dashboard with an interactive world map. There are graphical tools for manipulating the map, such as zoom and focus, and for displaying location specific information. For example, as a hurricane nears a shoreline, NHSS users can see its proximity to their current location and they can access real-time information on stream levels, wind speeds, and tide conditions for their location. The system also provides historical weather and natural hazard information for locations on the displayed map.

The goal of the USGS website is to construct a facility that provides national and local government authorities, fire and police departments, first aid planners, road work crews, and a host of other potential users (including private individuals) with an early warning system for potential natural disasters. As a system, it is clearly attempting to integrate a number of other systems. For example, the system of RAWS within the United States was developed to provide weather data to assist land-management agencies with monitoring air quality, rating fire danger, and providing information for research applications. But the RAWS system alone is not sufficient to make predictions about future hazards, even in the near term.

The USGS Natural Hazards Support System (NHSS) could evolve into a truly holistic event processing system, processing many thousands of event sources and producing many different types of predictive events. At present, users must make their own interpretations and conclusions from the information it displays. If, for example, they miss some of the information, their conclusions may be drastically wrong, as say in the case of trying to predict hazards that might result from earthquakes.9

The NHSS system needs to evolve to another level of service. One possible next step would be to add different types of event abstraction. Predictive models need to be incorporated into the system. These might enable inferences to be made from events appearing on the website. As new events appear, the predictions would be continuously updated. Data from previous events can be stored to test these models.

As Figure 4.5 in Chapter 4 illustrates, an event aggregation facility is needed that could correlate events at close locations from the present and recent past, together with terrain knowledge and historical data, to develop potential hazard warnings for possible imminent threats such as avalanches, mudslides, wildfires, high tidal waves, floods, and so on. Abstracting from different sources of events together with historical data to create alert events, and doing it for different kinds of hazard situations, would certainly put the system into the sphere of holistic applications.

But the types and numbers of event inputs for the continental United States are far more numerous and detailed than for the rest of the world, so at the moment the USGS NHSS is far from being the worldwide system it could be.

That said, the NHSS system is an event processing system that is evolving into a holistic system. It has all the ingredients: a large space of input event types, a large community of possible users requiring different types of information as outputs, and the need to process incoming events in real time. What is needed are predictive capabilities, political will, and economic backing. And its evolution may well be helped by the advent of new predictive methods in earth sciences.

Of course, there are many components to effective natural disaster warning systems beyond event processing predictive systems. Long-term planning includes building codes, shelters for the community, education, response systems involving police and the military, and many other facets, as in the case of the Japanese earthquake and tsunami of 2011.10,11 Such long-term planning takes the political will to do it. Even so, warning systems take time to react. And while the Japanese situation would have been much worse without the long-term planning, there was almost no time to react. Hawaii, in contrast, was much better prepared for the same event. Because of the National Oceanic and Atmospheric Administration (NOAA)’s Pacific warning system, Hawaii had time to react to its warnings and prepare for the tsunami.

Where would the necessary political will to develop the NHSS system in the United States come from? It is quite possible that if a natural disaster with large economic consequences happens in the United States, there could be political pressure upon various government agencies such as NOAA to develop a comprehensive disaster prediction and warning system. In response, those agencies might see the NHSS system as a promising starting point. The rest would depend upon funding.

In the following sections, we shall make a case for believing that several other examples of event processing systems are now in an early stage of evolution toward becoming holistic systems.

Future Air Travel Management Systems

Air travel systems today are a good example of business systems in chaotic bits and pieces all over the place—systems on the “garage floor,” so to speak. Here are the titles of some articles on the airline industry that have appeared over the past few years:

• “Increasing air travel, air freight, private aviation, light weight air limousine industry and proposed ‘flying cars’”
• “Climate effects of aviation”
• “Crowded air space”
• “Current air traffic control systems are outdated 50 year-old technology”
• “Safety, efficiency, scalability—an autonomous, globally integrated ATC system is needed”

The simple statistics describe the situation. In 2009, the world’s air transportation system flew 2.3 billion people safely on 35 million flights—the equivalent of between a quarter and a third of the entire world’s population. The air traffic across the north Atlantic itself was some 185,000 flights going in each direction. According to the U.S. Federal Aviation Administration, there were around 7,000 aircraft in the air over the United States at any given time in 2010.12

The current air traffic management systems (ATMs) have evolved from systems that were developed in the 1960s. Some of the components have been improved, and some new components have been added. But there are many separate systems in the air and on the ground that should interoperate but do not. The whole structure is creaking at the seams. As a consequence, there are plans by various government agencies in the United States, Canada, China, and Europe to put new air traffic management systems in place.

A future air traffic management system will be a holistic event processing system par excellence. It will no longer be ground-centric. The primary basis for air traffic control will be air-to-air communication between aircraft based upon satellite location. It will manage not only the movement of aircraft in the air across the nation, but also on the ground through airports. It will incorporate advanced planning to smooth the total flow of air traffic in the face of weather and other outside events. The system will manage all air traffic across the nation; eventually it will evolve into a worldwide ATM system.

A good example of the evolution toward future air traffic management systems is found in the United States’ plans for the NextGen13 system, which is a transformation of the entire National Airspace System (NAS). NextGen is planned to be developed in stages between 2010 and 2025. Various components will be developed and tested within the existing air traffic control system.

To understand the scope of the event processing involved in this system, we go into a bit more detail. NextGen has five core event driven components:

At the core of the NextGen system is an on-board aircraft control system, ADS-B (automatic dependent surveillance-broadcast technology), which is a satellite-based technology. Using ADS-B, each aircraft broadcasts its identification, position, and speed with once-per-second updates. ADS-B on-board transponders receive GPS signals and use them to determine the aircraft’s precise position in the sky. This and other data are then broadcast to other aircraft and to air traffic ground control. At the same time, each aircraft is receiving similar data from every other aircraft and ground station in its vicinity (see 9.1).

So a lot of different types of right now events are being received and generated by each aircraft. The goal of every aircraft using ADS-B is to give both pilots and air traffic controllers the same real-time display of the air traffic situation. If everyone is on the same page, so to speak, the result should be improved safety.

NextGen designers envisioned a future in which a series of event stream communications is used to define a flight. It will be a case of flight-deck computers exchanging streams of events with computers on the ground. Pilots and ground controllers, both using ADS-B, will simply confirm the flight paths that are planned by the system.

Obviously, NextGen itself will be a very large event processing system. Pilots and dispatchers will have the freedom to select their own flight paths, rather than follow a path on today’s railroad-like grid of predefined flight paths in the sky. Each airplane will transmit and receive precise information about the time at which it and others will cross key points along their paths.

The hope is that NextGen, with ADS-B in place, will shorten virtually every commercial airline route, save fuel and time, enable the amount of air traffic in the skies at any one time to be increased, and finally also reduce aircraft engine emissions.

There is a parallel development effort for a new European Union air traffic management system called SESAR14 or Single European Sky ATM Research, planned since 2004 and expected to be deployed by 2020. It will interoperate with NextGen. SESAR plans full integration of airport operations as part of ATM.

One of the issues with NextGen is how it will impact airlines, both from a financial and an event processing perspective. Nobody really knows yet!

On the financial side, figures anywhere up to $40 billion over the next 10 years have been mentioned. The costs to airlines for new equipment are estimated at half of this, which has been a stumbling block. Nobody wants to pay, not the government and not the airlines.15 But there are also estimates that the costs of doing nothing will be higher. In resolving the complicated politics of NextGen, political will must play a very large role!

Consider the future operations management system for a single airline by the time NextGen is operating. From an event processing perspective, extrapolating from the situation today, a single airline’s unified operational system will handle a very large number of different types of events coming from all the different systems that the airline will deal with. NextGen is only one of the event sources. Figure 9.2 schematically depicts a small part of the event processing for the operations of a large airline. By 2025 or thereabouts, the events could span everything from passengers and crew and airports to the core domain of flight operations.

This will be a holistic event processing system:

• Input-event types. Input-event types, like “NextGen events” (see Figure 9.1), include all the types of events flowing between each of the airline’s aircraft and other aircraft and all the NextGen subsystems during the course of a flight. And then there are all the other types of events involved in crew scheduling; passenger-booking system events; security; airport operations such as runway control; gate management; airline connection updates; and support systems such as aircraft maintenance, and the like . . . Plus there are more events from partner airlines. You can imagine the rest! There are hundreds of different types of input events.
• Output-event types. These are the types of events that drive an airline’s operations in right-now time. They span all operations and directives dealing with passengers, crew, aircraft, airports, flight schedules and delays, weather reports, catering supplies, and on and on. Again, there is a large number of varied output types.
• State. Finally, the state of the airline’s operations system itself—which will be a system distributed over many server clouds worldwide. This will contain regulations, the airline’s policy guidelines and other constraints on operations, event patterns that are being monitored right now, and reactive rules that codify the airline’s best practices. This state is being continuously updated.
• CEP. CEP technology will be used in a number of subsystems. Some obvious examples are:
  • Security. As we have seen in Chapter 5, event patterns play many roles in security and in command-and-control operations. CEP should be used in much the same roles in the security for all the operations of an airline.16
  • Airborne operations. These are event patterns for on-board prediction and early recognition of emerging air traffic situations during the operation of any flight. This part of the system is critical to the right-now operation of aircraft in an ever more demanding environment.
  • Flight planning. A library of common real-time air traffic patterns for all areas of operations will be a basic resource in planning the operation of each flight. These event patterns will become quite complex, including not only air-traffic event patterns but also the operations of airports, the patterns for handling passengers, weather patterns for areas en route, and many other factors.
  • Support-systems prediction. All airline operations are affected by supply chains and support systems, such as aircraft parts for maintenance, the operations and performance of subcontractors, airport operations, and so on. Conversely, many of the events that occur in airline operations affect the supply chain management systems that are running in other companies (suppliers, customers, freight forwarders, agents, etc.), so the airlines will need to send supply chain prediction events to their subcontractors and partners, too.
  • Sales and services. Passenger and cargo systems involved in sales and services.

Holistic airline operations-management systems will evolve from the current systems piece by piece. They will not be designed and built anew, but will evolve while in use. At every turn in their development, they will cross different boundaries. Various international and government regulations, technical IT boundaries as the component pieces are gradually unified, employee union issues, and the problems of collaborating with the existing operational systems of partners, or perhaps merging with other airlines. It will be a chaotic evolution.

Will such systems ever happen? If one reviews the development scenario we have just outlined, one would have to be skeptical. There is a very good chance that such an evolution will progress partway and stop at some point in the future. The airline would then be running on several separate unconnected EP systems, an intolerable situation! On the other hand, there will be increasing pressures for efficiency and interconnecting airline operations, as well as economies of scale that argue in favor of unified holistic operations-management systems for airlines. So we argue that whatever the outcome, the new systems that do emerge, even if they go only partway to what we have described, are quite likely to be holistic event processing systems.

Monitoring Human Activities

It is difficult to select just one example of a system for monitoring worldwide human activities to illustrate the role of event processing and CEP as an enabling technology. There are so many existing systems—and none of them are coping adequately with the problems they are intended to address. For example there are:

• Pandemic watch systems
• Drug-trafficking and money-laundering detection systems
• Basic resources monitoring systems (food, energy, water)

that are currently operated by separate organizations such as the United Nations (UN), the World Health Organization (WHO), and various national agencies. They all lack a basic event-driven IT infrastructure. These systems are now used to monitor events in different activity areas on a worldwide basis. Some of them drive enforcement systems as well—rather badly, in fact. All of them are event-driven IT systems, although the EP technology is hidden from view and is never mentioned at a level of operating the system. It is well recognized by the organizations concerned that these systems need to be improved. But the question is how to do it.

As a first step, there needs to be a lot more use made of event pattern detection and other CEP technology in these systems, because they are essentially tasked with doing event correlation, event-pattern detection, and abstraction to achieve their goals.

Longer term, as they develop, it is a good bet that these human activity monitoring systems are destined to become holistic event processing systems. Their scope of application will expand due to the pressures we have outlined previously.

For example, a worldwide drug-trafficking system has to monitor resources such as the movement of money, ships, planes, and people, while also factoring in all manner of demographic and economic events. So its input-event space certainly overlaps with the event spaces of the other systems.

Perhaps one day, far in the future, these systems may all finally evolve into a single global system, although many obstacles, both technical and political, will have to be overcome. Certainly, federation may be an evolutionary step. But abstracting combinations of the monitoring results and predicting longer-term outcomes cannot be achieved by simply federating the separate systems.

Here we choose to focus on future monitoring systems for outbreaks of infectious diseases as our example of holistic event processing in the area of monitoring human activities.

Pandemic Watch Systems

Early detection systems that watch for outbreaks of infectious diseases with the potential to grow into epidemics are in their infancy at the moment. Progress needs to be made toward developing epidemic warning systems that span the world. For example, here are some recent quotes from various WHO officials and reports:

• “Infectious diseases are now spreading geographically much faster than at any time in history.” (WHO report, August 22, 2007)
• “Over the last five years alone, WHO experts have verified more than 1,100 epidemics of different diseases.”
• “With more than 2 billion people traveling by air every year, an outbreak or epidemic in one part of the world is only a few hours away from becoming an imminent threat elsewhere.”
• “The question of a pandemic of a human form of avian influenza virus (H5N1) is still a matter of when, not if.”

As Dr. Larry Brilliant, an expert on disease pandemics, puts it:

. . . simulations of a Bird Flu pandemic today show that it will happen very quickly, from first observed case to global involvement, 3 weeks.17

There are many examples of infectious disease outbreaks where early detection systems would have helped curtail outbreaks and save lives, had they existed. Perhaps the latest is the 2009 outbreak of the H1N1 virus, which started on a pig farm in central Mexico sometime in 2008. It mutated into an infection that jumped from pigs to humans. Despite attempts to contain it in Mexico, it broke out in the United States early in 2009 and spread into a world epidemic during that year.18

But luckily H1N1 flu was a relatively mild form of this disease. There are far worse highly infectious diseases out there. The real problem is the speed with which they can travel between continents and populations these days.

This is clearly an area where holistic event processing can and will be used.

We can get a good idea of the kinds of event processing required in a pandemic watch system by studying existing systems. Notable among these are the Canadian Global Public Health Information Network (GPHIN), PROMED-mail,19 and the National Health Service (NHS) QFLU system for detecting flu outbreaks in the United Kingdom.

GPHIN

GPHIN was created in 1998 by Health Canada’s Laboratory Centre for Disease Control, in collaboration with WHO. It has since undergone continuous development. It is a fee-based electronic reporting service that continuously searches electronic media sources from around the globe. It gathers news reports related to public health and other topics from news wire services, including foreign language services such as Factiva and Al Bawaba (Farsi news reports), as well as thousands of website. The reports are processed and provided to subscribers. In some cases, email alerts to subscribers are triggered.

Input reports from the sources are subject to an automated translation step: The non-English articles are “gisted” into English using a machine translation engine. The purpose of the gist is to provide a report in English with the essence of the original article.

Inputs are then passed through an automated processing step that filters them for relevancy to public health issues, sorts them into eight categories, and then ranks them for importance. Those of high-enough rank are published to the GPHIN database. Articles with relevancy below the “publish” threshold are presented to a GPHIN analyst, who reviews the article and decides whether to publish it, issue an alert, or dismiss it. Those judged to be of “immediate concern” result in email alerts to subscribers.

Additionally, the GPHIN analyst team conducts more in-depth tasks, including linking events in different regions, identifying trends, and assessing the health risks to populations around the world.

Subscribers can access a stream of reports on the GPHIN website 24 × 7. Normally, about 4,000 reports are published per day, but in times of crisis this number has risen as high as 20,000. Subscribers receive alerts and can review the reports in one of several different languages (currently, Arabic, English, French, Russian, Simplified and Traditional Chinese, and Spanish).

GPHIN tracks a broad range of topics, such as disease outbreaks, infectious diseases, contaminated food and water, bioterrorism and exposure to chemical and radio nuclear agents,20 and natural disasters. GPHIN also monitors issues related to the safety of products, drugs, and medical devices. Events that may have serious public health consequences are given the highest rank.

GPHIN alerted the Western world to the Severe Acute Respiratory Syndrome (SARS) outbreak in 2002. This occurred in Guangdong Province in mainland China; it was detected and reported by GPHIN as early as November 27, 2002, three months ahead of the official WHO alert on February 25, 2003. Interestingly, GPHIN was alerted to monitor for SARS by an input that was a financial report about a pharmaceutical company’s increased sales of antivirals in the Guangdong Province. The report attributed the increased sales to the unusual outbreak occurring in that region.21

This example illustrates the need for epidemic warning systems to process a wide variety of different types of events. So, although its main function is clearly in the medical event area, the GPHIN system is tending to “go holistic” in the number of different types of input events.

In summary,

The human-in-the-loop aspect of GPHIN certainly slows it down.

The need to improve the speed of reporting and to extend the reach of the system beyond Internet sources is well understood. As one of GPHIN’s core implementers put it:

We need to increase the “grass roots” sources because that is where the “early warning” aspect lies. “Cream skimmers” like AP [the Associated Press news service] rarely venture to the local newspaper in some small village in rural China where some strange malady has struck.

Other alerting systems, such as the Program for Monitoring Emerging Diseases (PROMED-mail)22 and the United Kingdom’s QFLU have a similar underlying dependence upon event processing technology. PROMED-mail is completely based upon the Internet for information gathering and reporting. Interestingly, PROMED states that it maintains independence from governments on the grounds that independence helps it avoid issues of governmental delay or suppression of disease reporting for bureaucratic or strategic reasons. There are ongoing projects to make information flowing into PROMED-mail available on Google maps.23

GOARN

The Global Outbreak Alert & Response Network24 (GOARN) is a complementary system to GPHIN to respond to alerts. It was set up by member nations of the WHO in 2000 and now has a network of more than 120 partner nations throughout the world. The partners pool resources, including personnel, equipment, and laboratories.

GOARN’s outbreak response teams are assembled and mobilized to countries where a disease outbreak is occurring. The teams offer medical and logistics support to the national health authorities to coordinate and direct activities in response to an outbreak. The SARS outbreak was the first time that GOARN responded to an outbreak that was rapidly spreading internationally.

During the first six months of 2006, WHO/GOARN mobilized more than seventy operational interventions in response to avian influenza (AI) outbreaks and human transmission. Currently, GOARN responds to more than fifty outbreaks in developing countries each year. For example, a recent GOARN mobilization was in response to an outbreak of cholera in Haiti.25

Today the mission includes operating a “comprehensive event management system to manage critical information about outbreaks and ensure accurate and timely communications between key international public health professionals, including WHO Regional Offices, Country Offices, collaborating centres and partners in the Global Outbreak Alert and Response Network.” Exactly how this event-management system is implemented is not clear, nor is its response delay time, but it is almost certainly “human-in-the-loop.”

Future Epidemic Warning Systems

In the best of all worlds, there would be a single unified system for detecting and responding to epidemic outbreaks and other emergencies anywhere in the world. It could take many forms—for example, a federation of separate systems covering specific areas.

It would have to be a holistic event processing system correlating a wide range of inputs. Figure 9.3 shows some of the event processing in such a system. It is at best a sketch, but it illustrates that a holistic event processing system would be needed.

The range of input-event types spans everything from field messages, both text and voice, from mobile phones, PDAs, and other handheld devices to official alerts and reports from health agencies:

• Mobile phone and text messages. This class of inputs are events closest to the “grassroots” (i.e., possible locations of outbreaks). They will come from area inhabitants, field workers, and local health personnel and may be in multiple languages. In rural areas, mobile phones may be the only communication devices available. In fact, a recent study shows that the number of mobile phones in use worldwide has topped 5.0 billion, boosted by soaring demand in emerging markets India and China.26 So these events will probably be the most important in terms of early detection. But they are also the most in need of event filtering and correlation—in other words, quite advanced event processing at step one!
• Social networks. The adoption of this activity by the inhabitants of rural areas in, say, southeast Asia, is quite surprising and on the increase. The use of social networks such as Twitter by field workers and observers is a prominent feature in the planning of some of the future epidemic monitoring systems. It provides a scalable and rapid method of making field observations available to a very wide audience. And it is easily accessed using cell phones. But there are dangers in using this medium that we’ll discuss shortly!
• Newswires. These are often a second source of rumors and early news of emerging outbreaks. Again, their content is not reliable and needs careful review.
• Clinics and hospitals. Local-area medical facilities are also likely to produce early reports. Usually, these reports eventually find their way into reports from the area public health authorities, if such exist.
• Pharmacy sales. These can also be an indicator of an emerging epidemic, as in the case of SARS.
• Travel and immigration data. This information from airlines and border authorities will soon become important in tracking the spread of an epidemic and in predicting where medical resources may be needed.
• Reports of other monitoring networks. Networks such as GPHIN and PROMED-mail would probably act as confirmation to alerts issued by an effective warning system. The system would collaborate with other monitoring networks.

Output-event types vary from cautious early actions, such as adding a new event to a watch list or updating containment policies, to active actions, such as alerts for possible bioterrorism, to decisive emergency response actions, such as deploying field teams to the location of an outbreak or requesting border closures. This implies human-in-the-loop systems.

A broad range of input-event types is needed to correlate events from different sources and factor out unreliable reports. Moreover, an epidemic warning system has to pursue a careful line of decision making:

For these reasons, all the inputs types we have described (and perhaps others) will be needed.

As for response times, we should expect the first types of cautious outputs (such as updating watch lists) within twenty-four to forty-eight hours of initial observations from the field. Reaction times for alerts and mobilizations will depend upon the rapidity and volume of inputs and how they correlate. This again argues in favor of processing the widest possible range of types of input events.

There are many possible uses of CEP in future epidemic warning systems. Much of the preliminary processing of incoming reports will be automated. It is here that CEP may be used in roles that would have been assigned to human agents in the earlier systems. We can expect CEP to play important roles wherever the detection of patterns of events is required—for example, in the recognition of potential outbreaks. And patterns of events will be used in the processes for (1) updating watch lists and containment policies, (2) formulating testing plans, and (3) identifying conditions for issuing alerts. CEP is likely to play a role in the formulation of these rules as well.

Monitoring the Consequences

The effects of human activities on the environment in which we all live have been studied by earth scientists for several decades. They have also been a topic of contentious political debate between those who understand the results of the scientific work and those who choose to disbelieve them for one reason or another—usually motivated by short-term self interests.

The results are not good, and one of the “effects” is more extreme weather events. This is generally referred to as climate change. Former Vice President Al Gore talks about a “period of consequences.” Today, it is obvious that we are well into a period of suffering the consequences that Al Gore predicted.

The effects on the Earth and its environment are everywhere. They can clearly be seen in the new edition of The Times Comprehensive Atlas of the World:

We can literally see environmental disasters unfolding before our eyes. We have a real fear that in the near future famous geographical features will disappear forever.27

Changes in the atmosphere and the oceans are already well-documented. The ocean, for example, is 30 percent more acidic than it was in 1800, and this trend is having critical results on the human food chain.28

We have wired our planet with measuring devices, and all of them generate countless numbers of events. There are more than enough earth observation and forecasting systems (EOFS) in existence to track every one of the human activity effects in the minutest detail: sensors on satellites, aircraft, balloons, deep ocean buoys and surface buoys, polar ice cap stations, land-based observation stations, stream gauges in rivers, and on and on.

There are many international organizations and partnerships aimed at processing the events from the earth observation devices and making the results available to everyone. They have put up open-ended websites that host event processing toolsets, grid computing, numerical analysis, predictive models for forecasting hazards (e.g., floods, coastal surges, tsunamis, landslides, etc.), data fusion tools and other stuff. Anyone can join in and contribute.

Among these environmental monitoring projects we mention:

• Global Monitoring for Environment and Security (GMES)29, a joint initiative of the European Commission and the European Space Agency. It is aimed at achieving an autonomous and operational Earth observation capacity to be fully operational by 2014. It will pull together “all the information obtained by environmental satellites, air and ground stations to provide a comprehensive picture of the health of Earth.” The major unknown is how the information in all the collected events will be made available and what computing facilities will be available to study that information. That is not clear. The project seems to be following a path of chaotic evolution.
• Global Earth Observation System of Systems (GEOSS)30 is being built by the Group on Earth Observations (GEO), a partnership of 75 cooperating nations, including the United States, with a ten-year implementation plan from 2005 to 2015. It is a highly ambitious plan, shown in Figure 9.4, that may well fall short of some of its goals. The IT architecture to support this effort is not yet clearly specified. It will probably turn out to be an Internet portal with various tools that allow access to large quantities of data from all the measuring systems worldwide. If things turn out well, it may be based upon data representation standards that are yet to be determined.
• The Planetary Skin Institute (PSI),31 an R&D project founded jointly in 2009 by Cisco and NASA to “cut across institutional, disciplinary, and national boundaries and create a space for flexible pooling of assets and ideas between stakeholder networks.” It aims to develop systems for monitoring the demand for water, energy, food, and land, and the resulting environmental degradation and climate change. As part of its mission to advance global multi-stakeholder collaboration, PSI is committed to building a decentralized peer-to-peer network for research and innovation, anchored by eight regional hubs in Brazil, India, China, South Africa, Japan, Middle East, the European Union, and the United States.
• Various Advisory and Standards Panels sponsored by international organizations, devoted to environmental monitoring, and possibly overseeing the implementation of systems that contribute to the right-now availability of monitoring data, (e.g., the Global Climate Observing System (GCOS)32 and the Global Ocean Observing System (GOOS)33).

And there are many others. We cannot possibly list all the Earth monitoring projects that are ongoing at the moment.

Despite the official write-ups of GEMS, GEOSS, PSI, and the others, today’s trend is toward making the vast amounts of observational events and the data they contain accessible using very simple and widely available Web-based “pull” technology. That is, most of these projects plan to put the observational events on Internet portals that can deliver access to the information in a way that is optimized for PDAs and smartphones. The goal is instant access to everyone.

For example, a beta version of the PSI ALERTS platform is “an online cloud-based platform for near real-time global land change detection, and is available at no cost as a global public good to members of the public.” It “provides visualization, layering, and customization tools for identifying and characterizing land use change. Users can view changes in space and time on the ALERTS geospatial platform, and subscribe to personalized alerts that will notify them via email whenever land use changes are detected in geographical areas of interest.”34

All of these efforts are in the very good direction of dispelling disbelief and getting the true picture of our Earth’s dilemma out there for all to see and understand. The political will to take positive action to correct the environmental situation must be energized.

But there are problems with all these well-meaning plans:

• The various subsystems have typically operated in isolation from one another as a conglomeration of separate projects.
• They are run by different organizations and governments.
• There is a need for integration technology: standards for formats, protocols, and practices.
• These systems piggyback on existing networks—there is no scalable independent unifying event processing infrastructure. The plans rely on the Internet, and most projects are still being researched.
• The computing power needed may present a boundary. For example, Cisco estimates the global Internet traffic to be 63.9 exabytes per month in 2014.35 Also, some estimates guess that in 2010 Google used more than 450,000 servers worldwide at a cost on the order of $2 million per month in electricity charges.36

Perhaps NOAA’s website describes the issues as well as any (emphasis has been added):37

Today the many thousands of separate data systems in constant use usually don’t work together. Decision-makers and users at many levels—farmers making planting choices, emergency managers making evacuation decisions, companies evaluating prospective building sites, nations battling drought and disease, parents checking daily weather reports—all take advantage of data from satellite remote sensing, aerial surveys, land or ocean-based monitoring systems and a vast array of socio-economic information. But the Earth observation data being collected are just a fraction of what could be put to excellent, perhaps life-saving use in every region of the world.

Without comprehensive, integrated data sets, there are gaps in scientific understanding. Nature doesn’t work just on land, in the sea, or in the atmosphere, and taking the pulse of the planet requires an understanding of the intrinsic links of these Earth systems.

GEOSS is emerging to fill the gaps. With human ingenuity and the political will of 80 governments, GEOSS is a robust effort dedicated to building an integrated, comprehensive and sustained “system of systems” from many thousands of individual Earth observation technologies around the globe. This essential approach is as integrated as the planet that GEOSS is designed to observe, predict and protect.

NOAA pins our future hopes well and truly on the GEOSS consortium and planning. Indeed, the U.S. Environmental Protection Agency follows exactly the same plans.38

Any undertaking to monitor the consequences is going to be a truly holistic event processing project. A comprehensive supporting event processing infrastructure will be needed to make all of this worldwide event collection accessible in right-now time. It must process millions of events per second from a vast number of different types of sources, and then apply complex event pattern abstractions to produce higher-level events that allow humans to make decisions and take right-now actions.

Figure 9.5 does not do the project justice, but serves to illustrate parts of the input and output event type spaces and the holistic event processing that must be involved in supporting its implementation. The system would monitor event pattern constraints that detect trends toward situations that should not happen and should trigger alerts. It will execute rules that codify agreed policies and processes. The sets of constraints and rules would undergo continuous changes—a process that itself is challenging, since it will involve changes to processes that are currently executing.

The goal of a unified system: Continuous real-time estimates of long-term effects of human activity on the environment.

The most likely final implementation: A dynamic distributed network of millions of event processing engines, dynamic in the sense that some processors will enter or leave the network. It will evolve by a gradual federation of specialized event processing systems.

The magnitude of the holistic event processing that would be involved in a unified system for monitoring the consequences are enormous. This must raise doubts about whether such a system will ever evolve successfully from the bits and pieces that currently exist or are planned for the near future.

However, there are also arguments in favor of the evolution of such a system. First, there are commonalities between environmental monitoring and epidemic monitoring. For example, links between ocean temperatures and cycles such as El Niño and epidemics of infectious diseases such as malaria and dengue fever in many parts of the world have been established.39,40

Secondly, the quote from NOAA (above) states the case for such a system. GEOSS is only a first step. And understanding the links between separate earth monitoring systems is a step in earth science that will have to happen for there to be the political will to make a unified system a reality. Recall the enablers for holistic event processing systems:

Solving Gridlock in the Metropolis

Traffic congestion is a well-known hazard of large metropolitan areas the world over, and the economic costs have been the subject of both highway studies and political debate.

For example, in its study of 439 urban areas around the United States, the Texas Transportation Institute (Texas A&M University) found that American travelers are spending about 36 hours per year on average in traffic jams, and much more in the nation’s largest cities. Collectively, Americans spent nearly 500,000 years stuck in traffic in 2007—nearly 4.2 billion hours. In 2009 the cost of traffic congestion was estimated at $115 billion in wasted fuel and lost productivity, or $808 per traveler per year.41 Various government transportation secretaries, both in the United States and the European Union, have made speeches on the costs of gridlock in recent years.42

We can try building bigger highways and prioritizing “high occupancy vehicles,” or taxing cars entering the metropolis. But these “solutions” have all been tried without really solving the problem. What comes next?

One direction in traffic research is to take the human driver out of the loop altogether and automate the management of the traffic. Highway systems that control traffic lights and the flow of cars on the roads are already in service. They are event driven systems. A next step is to make the car part of these systems—so-called drive-by-wire cars that drive themselves and communicate with the traffic system and with one another to automate the management of traffic flow.

Building a drive-by-wire car depends upon solving four technical problems. The car must:

Arguably, two and a half of these problems are already solved: navigation and actuation have been solved completely, and sensors have been solved partially, but are improving fast. The main unsolved part is the motion planning.43

The other half of the problem is to automate the highways. Highways must provide navigation and traffic information in a form that the new guidance systems in cars can use.

Work toward building driverless cars is in various experimental stages in the United States, Europe, and Australia. So far, experimental pilot projects have combined magnetic sensors, forward-looking sensors, mobile wireless in cars, video cameras both in cars and on the highways, display technologies, and networks of computers spread out along the highway system. There have been several demonstrations of such systems around the world during the past fifteen years.

A totally automated traffic system, cars and highways, will certainly involve a lot of sophisticated event processing. It will entail using event patterns, real-time pattern matching, and event pattern abstraction to analyze traffic behavior and long-term trends. Hopefully, this use of event processing technology in automated traffic systems will contribute to reducing traffic congestion and increasing safety on the roads.44

The drive-by-wire car approach in the United States is an R & D project of the Department of Transportation based upon a dedicated wireless communication channel. Dedicated short-range communications45 (DSRC) is the communications media of choice because:

• It operates in a licensed frequency band.
• It is primarily allocated for vehicle safety applications by FCC Report & Order—February 2004 (75 MHz of spectrum).
• It provides a secure wireless interface required by active safety applications.
• It supports high-speed, low-latency, short-range wireless communications.
• It works in high vehicle speed mobility conditions.
• Its performance is immune to extreme weather conditions (e.g., rain, fog, snow, etc.).
• It is designed to be tolerant to multipath transmissions typical with roadway environments.
• It supports both intervehicle and vehicle ↔ infrastructure communications between cars and the highway system.

That’s a lot of events and event processing.

Note that DSRC46 is preferred over wifi because the proliferation of wifi handheld and hands-free devices that occupy the 2.4 GHz and 5 GHz bands, along with the projected increase in wifi hot spots and wireless mesh extensions, could cause intolerable and uncontrollable levels of interference that could hamper reliability and effectiveness. Also, car safety applications require response times measured in milliseconds. DSRC enables vehicles to receive safety messages (e.g., road blocks ahead) and immediately determine if they should respond.

“Cars that drive themselves—even parking at their destination—could be ready for sale within a decade,” is a quote from the director of research for General Motors Corp. in January 2008.

However, there is a question: Does the political will to “cross the boundaries” exist? The politics of traffic systems planning is complicated. Big money is involved!

For example, most experts agree that technology allowing cars to communicate with each other—and with the infrastructure—would make our highways safer and more efficient. But they disagree about who should have access to vehicles’ computer systems and at what level. “Right now, automakers rigidly control access to their automotive operating systems in a way that even Apple Computer might find constricting.”47,48

The European Union is actively researching drive-by-wire cars and infrastructure for an efficient and safe European road network. For example, the COM2REACT49 project is to develop new technology that allows a group of vehicles to exchange data automatically with each other and with traffic control centers. The vehicle-to-vehicle communication has been tested in several urban traffic environments. And one of the stated goals of the project is to bring the cost per vehicle down to €100.

Building on this project, the European Safe Road Trains for the Environment (SARTRE) project50 now aims at developing car-to-car communication so that cars can self-organize into platoons on highways at the usual rush hours times of the day. Each platoon has a professional driver in the lead and the other cars fall in behind and maintain distance by communicating with each other. Their drivers do nothing and can even take a nap during the trip.

An automated traffic system of the future will certainly involve holistic event processing. It will process event feeds from all surface traffic control systems (vehicles, video cameras, traffic lights, highway access controls, roadside sensors, etc.), police information systems, road works, media events, weather, airline and train schedules, and much else. It will factor in traffic simulator predictions for the area, all in right-now time. It must produce directives for the automated traffic control systems, congestion and hazards alerts and rerouting guides, as well as highway situation forecasts for the emergency authorities.

We attempt to sketch the scope of the holistic event processing required for controlling a driverless car within an automated traffic system for a large metropolitan area in Figure 9.6. There are a wide range of different types of input events to the area traffic management system from the highway sensors, communications with cars, police traffic systems, weather forecasts, and many other sources. Similarly, the output events range from immediate motion planning of the car and traffic positioning to route planning and predictions.

Monitoring Your Personal Information Footprint

One of the most challenging uses of CEP might turn out to be to track the use of personal information on the Internet and protect it from misuse. The Internet has become a way of life. We all use it daily for such things as email, online banking, and credit card purchases. Bits and pieces of our personal information are out there, and few of us know what happens to all of it.

Today we face a continuous erosion of privacy. More and more data on individuals are collected electronically and made available to anyone for a small price—usually without the individuals’ knowledge. Privacy laws do not keep pace with technology, and laws governing public records do not stop those records from being misused.

Personal Information Footprint: The totality of an individual’s personal data, including, e.g., name, address, parents’ names, social security number, passport number, bank accounts, medical records, Internet passwords, family history, and the like.

The problem: As individuals do more—shopping, talking, working—online, they leave private information behind in databases stored on servers that are not under the individual’s control. Most companies store proprietary data on networked servers connected to the Internet; Internet vendors are a prime example. Computer security experts struggle to develop technology and best practices to protect this information from unauthorized intruders or inadvertent leaks. It’s a losing battle!

The individual is powerless to control how his information footprint is used. The general situation is that once one’s personal information is “out there,” it may be subjected to many different uses.51 Commonly, it can make its way from one mailing list to another, be sold to online marketers, used for targeted advertising, or misused in ways that cause direct loss to the individual. In the worst case, personal information is used to impersonate the individual to apply for credit cards or to steal directly from the individual’s bank accounts. The costs of identity theft are high.52

In addition to the crooks, the individual must also worry about what happens to his information when it finds its way into a data fusion center.

As of July 2009 there are more than 72 state, local, and regional fusion centers in operation around the United States, and more in development. They are a relatively new tool for law enforcement, emergency management, and homeland security. Supposedly, fusion centers allow federal, state, and local public safety agencies to work side by side, collecting and sharing data from a wide variety of sources. Analysis of the data is intended to result in actionable intelligence that leads to safer communities.53

However, some fusion centers do not host the data, but rather refresh them regularly. That means their analysts are not subject to the Freedom of Information Act (FOIA) or being dragged into court for misuse of information.54,55 This is potentially disastrous for private citizens who are trying to pin down responsibility for mistaken information that is turning their lives upside down!

As with all technologies, event processing and CEP can be used for both legitimate and illegitimate purposes. On the negative side, for example, event processing can be used to implement spying programs that steal personal information. These days it is often used to spy on an individual’s activities and their communications for commercial and maybe even political ends.

The current methods of defense available commercially are, to say the least, inadequate. One can buy protections such as:

• Fraud prevention. A service that places fraud alerts on your data with the three major credit bureaus. Credit issuers are forced to contact you before opening an account in your name.
• ID theft detection. Software systems that send email alerts when triggered by events in public records, such as a change of address, a credit report inquiry, or a public record name change.
• Activity monitors. Programs that record emails, chats, instant messaging, website visits, Internet searches, programs that are run on your computer, keystrokes typed, files transferred, and screen snapshots. Activity monitors might help to warn an individual to avoid risky behavior, but they are more likely to be used by employers spying on their employees during working hours!

These are disparate commercial services, each covering only a very small piece of the individual’s information footprint. They do not solve the privacy problem. They cost more money than they turn out to be worth in most cases. They only deliver after-the-fact reporting. And they certainly don’t stop identity theft from happening!

Perhaps the best suggestion toward monitoring your information footprint may be to use alert services such as Google Alerts, which will continuously search the Web to track topics you’re interested in. Alert services can be configured to find out what information about you is being published on the Web. These services continuously search the Web for instances of personally identifying information such as your name, address, phone number, social security number, and so on. For example, when Google finds matches, it will send you an email with links. The downside may well be that you will be inundated with alerts that you know about or gave permission for.

The privacy of the individual is a continual battleground, because new tracking and collection methods are appearing all the time. It has become the subject for the formulation of new laws—e.g., requiring “opt out” features on website that collect data on visitors. But these laws are not likely to solve the problem.

On the positive side, holistic event processing systems probably represent the best line of attack toward building programs that can track and inform individuals as to what is happening to their information and proactively alert them to suspicious circumstances as they are happening and before damage is done.

Protecting Your Information Footprint: Holistic event processing systems that:

Figure 9.7 shows the scope of the kinds of event processing that could be needed to protect an individual’s personal data and enable that individual to keep track of what’s happening to the information. Possibly, event processing can be used to increase the individual’s awareness of his Internet behaviors and potential dangers thereof. Finally, it might be able to enable an individual to block uses of personal information. However, the potential for false alarms and burdensome demands upon an individual’s attention is obviously a possible drawback with such protection systems.

People have different reactions to the idea of a system that tracks and protects your information footprint. Some specialists in event processing think this may be the ultimate “police application,” and it will take fifty years to develop.

Others see a need for this kind of protection of the individual. They think it can be achieved in incremental steps, and that some tools for preventing personal information misuse—catch it as it is happening—can be built now; also some of the components already exist.

Summary: The Future of Complex Event Processing

These five examples of possible applications of holistic event processing are but a few of the possibilities. Some of the examples we have given may not happen. But some other large-scale future applications of event processing will evolve, and they will be holistic in the sense that they process large numbers of different types of event inputs, combine different kinds of event processing, and provide predictive capabilities for interpreting their outputs.

Complex event processing technology will be an essential component of holistic systems for enterprise management. Event patterns, rules, rules engines, and event hierarchies will be part of the technology available in these systems to technical support engineers and knowledgeable business specialists. But CEP will be under the hood and hidden by sophisticated graphical interfaces from the average user. In many cases, users will probably not know that their requests for information right now are being implemented underneath the user interface by event pattern detection engines—but of course, some users will also be specialists in event processing!

Who will be responsible for the evolution of holistic event processing systems? That is harder to predict than anything else about event processing! But one thing is likely true:

Some of the people who read this book will be contributors to the evolution of holistic event processing systems.

I encourage you to go forward with whatever application of, or new technology for, event processing catches your interest.

Of one thing I am certain. We have barely scratched the surface of discovering the ways events can be gathered and processed, and the results applied, right now, for the good of all of us here on this Earth.

Notes

1 See Chapter 4.

2 See Chapter 7 for a definition of event type spaces.

3 The expanding input principle; in Chapter 7, see the section entitled “Restricting the Types of Event Inputs May Not Be an Option.”

4 The term holism was introduced by the South African statesman Jan Smuts in his 1926 book, Holism and Evolution. Smuts defined holism as “The tendency in nature to form wholes that are greater than the sum of the parts through creative evolution.”

5 The architectures of some tsunami warning systems follow this model nowadays, except that the input events are gathered by deep ocean buoys, not cell phones.

6 L. M. Haas, E. T. Lin, and M. A. Roth, “Data Integration through Database Federation,” IBM Systems Journal, Vol. 41, No. 4, 2002, pp. 578–596.

7 http://en.wikipedia.org/wiki/Internet_of_Things

8 There are nearly 2,200 remote automated weather stations (RAWS) located throughout the United States (USGS, 2008).

9 The effects of the Richter 9.1 Sumatra-Andaman earthquake in December 2004 (http://en.wikipedia.org/wiki/2004_Indian_Ocean_earthquake) were underestimated, and those of the Richter 8.8 earthquake in the Pacific off the coast of Chile in February 2010 (http://en.wikipedia.org/wiki/February_2010_Chile_earthquake) were overestimated by current methods.

10 Max Fisher, “Why Japan Was Ready,” The Atlantic, March 14, 2011. www.theatlantic.com/international/archive/2011/03/why-japan-was-ready/72429

11 Japan Meteorological Agency, “What Is an Earthquake Early Warning?” www.jma.go.jp/jma/en/Activities/eew1.html

12 U.S. Federal Aviation Administration website. http://www.faa.gov/air_traffic/briefing

13 Automatic dependent surveillance-broadcast (ADS-B). http://en.wikipedia.org/wiki/ADS-B

14 European Commission Mobility & Transport, “What is the SESAR Project?” April 15, 2011. http://ec.europa.eu/transport/air/sesar/sesar_en.htm

15 Between 2007 and 2011 bills to finance the FAA for NextGen were stalled 18 times in the U.S. Congress: Jad Mouawad, “Untangling the Skies,” New York Times, April 2, 2011. http://query.nytimes.com/gst/fullpage.html?res=9F05EFDB1130F931A35757C0A9679D8B63&scp=1&sq=jad%20mouawad%202011%20untangling&st=Search

16 One of the criticisms of the NextGen plans has been the lack of planning for the security of its operations.

17 Larry Brilliant, “Larry Brilliant Wants to Stop Pandemics,” TEDtalks, July 2006. www.ted.com/talks/larry_brilliant_wants_to_stop_pandemics.html

18 2009 Flu Pandemic. http://en.wikipedia.org/wiki/2009_flu_pandemic#Epidemiology

19 Since October 1999, ProMED-mail has operated as an official program of the International Society for Infectious Diseases, a nonprofit professional organization with 20,000 members worldwide.

20 Radio nuclear agents are radioactive chemicals capable of causing injury.

21 Michael Blench, “Global Public Health Intelligence Network (GPHIN),” Proceedings of the 8th AMTA Conference, October 2008, pp. 299–303.

22 International Society for Infectious Diseases, ProMED-mail website. www.promed-mail.org

23 www.cdc.gov/eid/content/15/5/689.htm

24 WHO, Global Outbreak Alert & Response Network. www.who.int/csr/outbreaknetwork/en/

25 Pan American Health Organization, “GOARN Team Deployment to Haiti,” blog entry, November 24, 2010. http://new.paho.org/blogs/haiti/?p=1377

26 A study by Swedish telecom giant Ericsson, March 31, 2011, cited in “Mobile Phones in Use Worldwide Top 5.0 Billion: Study,” Physorg.com, July 15, 2010. www.physorg.com/news198405924.html

27 Mick Ashworth, editor-in-chief of the Times World Atlas, 2007.

28 Daniel Grossman, “UN: Oceans are 30 Percent More Acidic Than Before Fossil Fuels,” National Geographic Daily News (website), December 15, 2009. http://newswatch.nationalgeographic.com/2009/12/15/acidification

29 European Commission Enterprise and Industry, “GMES: Observing Our Planet for a Safer World.” http://ec.europa.eu/enterprise/policies/space/gmes

30 Group on Earth Observations website: www.earthobservations.org

31 Planetary Skin Institute website: www.planetaryskin.org

32 GCOS website: www.wmo.int/pages/prog/gcos/index.php

33 GOOS website: www.ioc-goos.org

34 University of Minnesota, College of Science and Engineering, “Computer Scientists Launch Beta Version of New System to Track Global Land Change,” http://cse.umn.edu/admin/comm/newsreleases/2010_12_07_planetary-skin.php

35 Cisco Systems, “New Peering Standards for Ethernet Exchanges: Simplify Interconnections and Enable New Revenues,” White Paper, 2010. p. 1. www.cisco.com/en/US/prod/collateral/routers/ps9853/c11-609224-00_ethernet_exchanges_wp.pdf

36 Randall Stross, Planet Google (New York: Free Press, 2008) p. 61.

37 NOAA, “Earth as a New Frontier: The World-Changing Capability of the Global Earth Observation System of Systems (GEOSS).” www.noaa.gov/eos.html

38 U.S. Environmental Protection Agency, “Global Earth Observation System of Systems (GEOSS) and the Group on Earth Observations (GEO).” www.epa.gov/geoss

39 The Board of Global Health and the Institute of Medicine, “Climate, Ecology, and Infectious Disease,” in Global Climate Change and Extreme Weather Events: Understanding the Contributions to Infectious Disease Emergence: Workshop Summary, www.ncbi.nlm.nih.gov/books/NBK45744/

National Academies Press (US); 2008. ISBN-13: 978-0-309-12402-7

40 The World Health Organization, “El Niño and its health impact.” www.allcountries.org/health/el_nino_and_its_health_impact.html

41 Texas Transportation Institute, “2009 Urban Mobility Report and Appendices.” http://mobility.tamu.edu/ums/report

42 For example, the E.U. telecoms commissioner at the time, Viviane Reding, said that 24 percent of driving time in Europe is spent in traffic jams, which could cost the E.U. economy €80 billion by 2010 (Agence France Presse, August 2008).

43 Shawn Langlois, “The Car of Tomorrow Will Drive Itself—and Fly,” MarketWatch, June 17, 2011. www.marketwatch.com/story/the-car-of-tomorrow-will-drive-itself-and-fly-2011-06-17

44 According to the National Highway Traffic Safety Administration (NHTSA), there are about 40,000 traffic deaths per year in the United States and a far higher number of serious injuries. www.nhtsa.gov

45 www.intellidriveusa.org

46 DSRC is similar to IEEE 802.11a except it operates in a 75 MHz licensed spectrum around 5.9 GHz, and it brings better wireless channel propagation with respect to multipath delay spread and Doppler effects caused by high mobility and roadway environments.

47 Bryan Laviolette, “Experts Debate How Cars Will Talk to Each Other,” The Detroit Bureau. www.thedetroitbureau.com/2010/08/experts-debate-how-connected-vehicles-will-talk-to-each-other, August 4, 2010.

48 The U.S. DOT will decide by 2013 about requiring DSRC receivers in vehicles or possibly develop another communications system.

49 The COM2REACT website: www.com2react-project.org

50 The SARTRE Project website: www.sartre-project.eu/en/Sidor/default.aspx

51 See, e.g., Robert L. Mitchell, “12 Tips for Managing Your Information Footprint,” Computerworld, January 27, 2009. www.computerworld.com/s/article/9125098/12_tips_for_managing_your_information_footprint

52 The U.S. government estimates state that 8.1 million U.S. adults were victims of identity theft in 2010, at a total cost of $374 billion. (California Office of Privacy Protection, “Identity Theft First Aid,” June 1, 2011). www.privacyprotection.ca.gov/identity_theft.htm

53 American Civil Liberties Union (ACLU), “Questions to Ask About Fusions [sic] Centers,” November 21, 2007. www.aclu.org/national-security-technology-and-liberty/questions-ask-about-fusions-centers

54 Fusion centers are often run by private companies under contract. For example, the largest collection of medical records in the United States is maintained by an insurance industry organization, the Medical Information Bureau (MIB).

55 For a list of the types of data a fusion center collects on private citizens with no criminal record, see: Charity & Security Network, “Texas Law Enforcement Memo: Beware Nonprofits Promoting Tolerance,” April 9, 2009. www.charityandsecurity.org/news/Texas_Memo_Beware_Nonprofits_Promoting_Tolerance