27.1   Introduction

During the past two decades, extensive research and development (R&D) on multisensor data fusion has been performed for the Department of Defense (DoD). By the early 1990s, an extensive set of fusion systems had been reported for a variety of applications ranging from automated target recognition (ATR) and identification-friend-foe-neutral (IFFN) systems to systems for battlefield surveillance. Hall et al.¹ provided a description of 54 such systems and an analysis of the types of fusion processing, the applications, the algorithms, and the level of maturity of the reported systems. Subsequent to that survey, Llinas and Antony² described 13 data fusion systems that performed automated reasoning (e.g., for situation assessment) using the blackboard reasoning architecture. By the mid-1990s, extensive commercial off-the-shelf (COTS) software was becoming available for different data fusion techniques and decision support. Hall and Linn³ described a survey of COTS software for data fusion (an update of which is provided in this handbook) and Buede^4,5 performed surveys and analyses of COTS software for decision support.

This chapter presents a new survey of data fusion systems for DoD applications. The survey was part of an extensive effort to identify and assess DoD fusion systems and activities. This chapter summarizes 79 systems and provides an assessment of the types of fusion processing performed and their operational status.

27.2   Recent Survey of Data Fusion Activities

A survey of DoD operational, prototype, and planned data fusion activities was performed in 1999–2000. The data fusion activities that were surveyed had disparate missions and provided a broad range of fusion capabilities. They represented all military services. The survey emphasized the level of fusion provided (according to the JDL model described in many chapters of this book, such as Chapter 3) and the capability to fuse different types of intelligence data. A summary of the survey results is provided here.

In the survey, a data fusion system was considered to be more than a mathematical algorithm used to automatically achieve the levels of data fusion described in, for example, Chapter 2. In military applications, data fusion is frequently accomplished by a combination of the mathematical algorithms (or fusion engines) and display capabilities with which a human interacts. Hence, the activities range from relatively small-scale algorithms to large-scale command, control, intelligence, surveillance, and reconnaissance (C4I) systems, which use specific algorithms—such as trackers—in conjunction with a sophisticated display of data from multiple intelligence (multi-INT) data types.

The objective in identifying the unique data fusion activities was to isolate the unique capabilities, both mathematical and display-related, of the activity. A master list was initiated, and the researcher applied expert judgment in eliminating activities for any of the following reasons: (1) obsolete systems, (2) systems that were subsumed by other systems, (3) systems that did not provide unique fusion capabilities, (4) systems that were only data fusion enablers, and (5) systems that emphasized visualization.

Table 27.1 lists the resulting 79 unique DoD activities with their primary sponsoring service or organization. The list is intended to be a representative, rather than exhaustive, survey of all extant DoD fusion activities. The R&D activities, as well as the prototypical systems, are shown in bold type.

27.3   Assessment of System Capabilities

A primary goal of the survey was to understand the JDL fusion capabilities of the current operational fusion activities, as well as to emphasize the R&D activities. Recognizing that the activities often provided more than one level of fusion, all of the activities were assigned one or more levels. For instance, a typical operational fusion activity, such as the global command and control system (GCCS), provides specialized algorithms for tracking to achieve level 1 (object refinement) in addition to specific display capabilities aimed at providing the necessary information from which the analyst can draw level 2 (situation refinement) inferences.

The capabilities were then counted for both the operational and nonoperational activities, and a histogram was generated (Figure 27.1). Note that level 2 fusion was divided into two components, given that many operational military data fusion systems are said to facilitate level 2 fusion through the display fusion of various INTs from which an analyst can draw level 2 inferences.

The majority of operational data fusion activities provide level 1 fusion. These activities include weapon systems, such as the advanced tomahawk weapons control system (ATWCS), and trackers, such as the processor independent correlation and exploitation system (PICES). A less common operational capability is algorithmic level 2 fusion, which is provided by some operational systems such as the all-source analysis system (ASAS).

The majority of the operational systems that are geared toward intelligence analysis have emerged from a basic algorithm to track entities of interest. In most cases, the trackers have operated from signals intelligence (SIGINT). Gradually, these systems evolved by adding a display not only to resolve ambiguities in the tracking, but also to bring in additional INTs. As a result, most of the systems provide an underlying algorithmic fusion, in addition to a display that accommodates multi-INT data.

Algorithmic fusion to achieve level 3 fusion is uncommon among the operational systems, and none of the operational systems provide level 4 fusion. These capabilities, however, are being developed within the R&D community and do exist in prototypical systems. In addition, R&D efforts are also focusing on level 1 fusion, but generally with new intelligence data types or new combinations of intelligence data types.

The activities were also analyzed for their capability to fuse data algorithmically from multi-INT types. The use of more than one intelligence data type is becoming increasingly critical to solving difficult military problems. Multi-INT data types collected on a single entity can increase the dimensionality of that entity. Common intelligence data types that are in use today include SIGINT, infrared intelligence (IR), imagery intelligence (IMINT), moving target indicator (MTI), measurement and signatures intelligence (MASINT), and a combination of two or more of the above or multi-INT data types.

The pie charts in Figure 27.2 illustrate the capabilities to fuse data algorithmically from multi-INT data types for the surveyed activities. There are two pie charts (operational and nonoperational) for each of four JDL levels of fusion. The pie chart in the upper left corner of the figure is interpreted as follows. All systems that provide algorithmic fusion to achieve level 1 fusion (object refinement) were tallied according to the intelligence data type(s) used to achieve the fusion. The chart shows that the majority of operational systems use only SIGINT to achieve level 1 fusion. By contrast, the pie chart in the upper right corner shows, for nonoperational systems, a wider variety in the usage of other intelligence data types.

Other conclusions from this figure are that level 2 fusion, which is achieved by operational systems, is primarily achieved using multi-INT data according to the pie chart in the lower left corner, second row. All nonoperational activities use multi-INT data to achieve level 2 fusion. Few operational systems automatically integrate multi-INT data. Most data fusion systems display multi-INT data—sometimes on a common screen. Selected R&D systems are tackling the algorithmic integration of multi-INT data.

A common belief is that the realm of military data fusion is marked by numerous duplicative activities, which seems to imply a lack of organization and coordination. In reality, the 79 activities in this study reflect various relationships and migration plans. A close examination of the pedigree, current relationships, and progeny demonstrates some commonality and a greater degree of coordination than is often apparent. In addition, the DoD has established several migration systems to which existing fusion systems must evolve. Examples of these are the GCCS and the distributed common ground station (DCGS).

Specific operational intelligence systems have been identified as migration systems; others are viewed as legacy systems that will eventually be subsumed by a migration system. For nonintelligence systems, the fusion capabilities are frequently highly specialized and tailored to the overall mission of the system, such as weapon cueing. In addition, the nonoperational systems are generally highly specialized in that they are developing specific technologies, but they also frequently leverage other operational and nonoperational activities.

Table 27.2 shows the division of these activities into several categories:

1. Migration systems that are converging to a common baseline to facilitate interoperability with other systems

2. Legacy systems that will be subsumed by a migration system

3. Government-sponsored R&D and prototypes

4. Highly specialized capabilities that are not duplicated by any other system

The R&D activities, as well as the prototypical systems, are shown in bold type.

In conclusion, the recent survey of DoD data fusion activities provides a snapshot of current and emerging fusion capabilities in terms of their level of fusion and their usage of various types of intelligence data. The survey also reflects a greater degree of coordination among military data fusion activities than was recognized by previous surveys.

References

1. Hall, D. L., Linn, R. J., and Llinas, J., A survey of data fusion systems, Proc. SPIE Conf. Data Struct. Target Classif., Orlando, FL, April 1991, 147B, 13–29.

2. Llinas, J. and Antony, R. T., Blackboard concepts for data fusion applications: Blackboard systems, Int. J. Pattern Recognit. Artif. Intell., 7(2), 285–308, 1993.

3. Hall, D. L. and Linn, R. J., Survey of commercial software for multi-sensor data fusion, Proc. SPIE Conf. Sensor Fusion: Aerosp. Appl., Orlando, FL, 1991.

4. Buede, D., Software review: Overview of the MCDA market, J. Multi-Criteria Decis Anal., 1, 59–61, 1992.

5. Buede, D., Superior design features of decision analytic software, Comput. Oper. Res., 19(1), 43–57, 1992.