22.2.1   Role of Data Fusion in Information Processing Systems

Data fusion is the process of combining data/information to estimate or predict the state of some aspect of the world.¹ Fusion provides fused data from the sources to the user and to resource management, which is the process of planning/controlling response capabilities to meet mission objectives (Figure 22.1).

In this section, refinements of the JDL data fusion model and dual extensions for resource management are described and applied in the DF&RM DNN software technical architecture. The motivation for this and earlier versions of the JDL data fusion model has been to facilitate understanding of the types of problems for which data fusion is applicable and to define a useful partitioning of DF&RM solutions.

The JDL data fusion model is a functional model. It was never conceived as a process model or as a technical architecture. Rather, it has been motivated by the desire to reduce the community’s perceived confusion regarding many elements of fusion processes. The model was developed to provide a common frame of reference for fusion discussions and to facilitate the understanding and recognition of the types of problems for which data fusion is applicable and also as an aid to recognizing commonality among problems and the relevance of candidate solutions.

As stated by Frank White, the original JDL panel chairman, much of its (JDL model) value derives from the fact that identified fusion functions have been recognizable to human beings as a model of functions they were performing in their own minds when organizing and fusing data and information. It is important to keep this human centric sense of fusion functionality since it allows the model to bridge between the operational fusion community, the theoreticians and the system developers.

The framework of the model has been useful in categorizing investment in automation and highlighting the difficulty of building automated processes that provide functionality in support of human decision processes particularly at the higher levels where reasoning and inference are required functions. The dual resource management levels described herein are offered as a starting place from which to evolve the same benefits for the resource management community.

The suggested revised data fusion function at the functional levels² are based on the entities of interest to information users (Figure 22.2) with revised fusion levels defined as follows:

• Level 0 (L0). Signal/feature assessment—estimation and prediction of signal or feature states

• Level 1 (L1). Entity assessment—estimation and prediction of entity parametric and attributive states (i.e., of entities considered as individuals)

• Level 2 (L2). Situation assessment—estimation and prediction of the structures of parts of reality (i.e., of relationships among entities and their implications for the states of the related entities)

  

  FIGURE 22.2
  Recommended revised data fusion model. (From Steinberg, A.N. and Bowman, C.L., Rethinking the JDL Fusion Levels, NSSDF JHAPL June, 2004.)

• Level 3 (L3). Impact assessment—estimation and prediction of the utility/cost of signal, entity, or situation states; including predicted impacts given a system’s alternative courses of action (COAs)

• Level 4 (L4). Process assessment—estimation and prediction of a system’s performance as compared to given desired states and measures of effectiveness

The JDL fusion levels provide a historically useful and natural batching of fusion nodes (i.e., into signals, entities, relationships, and impacts) that should be considered in a fusion system design to achieve the knee-of-the-curve in performance versus cost. The level 1 entities of interest are defined by the user as individual objects. Level 0 characterizes patterns in received data to estimate types and characteristics of features and signals. This may be done without necessarily posting a specific causative level 1 entity. Their relationships of interest (e.g., as defined by an ontology of their aggregates, attacks, cues, etc.) are the focus of level 2 processing. This level 2 processing can be represented with multiple predicates as described in Ref. 2, with single predicates (e.g., as defined by a taxonomy of entity classes) used for the entity attributes estimated in level 1. The use of the DNN architecture for level 2 and level 3 fusion provides the system designer with techniques to interlace the levels 2 and 3 processes and the functional partitioning for each DF&RM node. The processing at these DF&RM levels are not necessarily performed in sequential order and two or more levels may need to be integrated into common nodes to achieve increased performance albeit at higher complexity/cost.

The most significant change, from earlier versions^1,2 is the removal of process management from fusion level 4. Process management is now found within the new resource management levels, as suggested in Ref. 1. Process assessment is retained in the fourth level of JDL model for fusion (i.e., process assessment has always been included in the JDL level 4 fusion). The consideration of fusion process management as a resource management function provides a rich set of tools for segmenting and developing process management software using DNN resource management nodes much as they are applied for resource management process refinement (see Section 22.4). The restriction of level 4 fusion to process assessment provides the benefit for developing performance evaluation (PE) systems using the DNN data fusion architecture (see Section 22.3). Examples of levels 2 and 3 fusion processes are given in Sections 22.1 and 22.2, respectively. A discussion of how the DF&RM DNN architecture unifies numerous DF&RM models is given in Section 22.5. Before presenting these the role and significant components of the applications-layer DNN architecture are reviewed in Sections 22.2 and 22.3, respectively.

22.2.2   The Role for the DNN Architecture

Information technology and concepts are evolving rapidly in the global web. Paraphrasing Frank White, the evolution toward client services including web services, agent-based computing, semantic web with its ontology development offers the potential for a widely distributed information environment. Network services are receiving the bulk of the attention at present but the importance of the information services is becoming recognized. Information services center on content including data pedigree, contextual metadata, and contribution to the situational awareness. Data and information fusion is cited as a central, enabling technology to satisfy the inherent informational needs. From the fusion community perspective the ability to achieve automated fusion (e.g., belief function combination) is dependent on the data pedigree, metadata, context services, and the architecture standards that evolve to prescribe them. Thus, the user identifiable functional components of the fusion process as embodied in a fusion architecture are critical to providing affordable distributed information services.

The increasing demand for more automation of data fusion and response management functions operating in open distributed environments makes it imperative to halt the exponential increase in DF&RM software development costs and building of one-of-a-kind DF&RM software systems.

Architectures are frequently used as mechanisms to address a broad range of common requirements to achieve interoperability and affordability objectives. An architecture (IEEE definition from ANSI/IEEE Standard 1471-2000) is a structure of components, their relationships, and the principles and guidelines governing their design and evolution over time. An architecture should

• Identify a focused purpose with sufficient breadth to achieve affordability objectives

• Facilitate user understanding/communication

• Permit comparison, integration, and interoperability

• Promote expandability, modularity, and reusability

• Achieve most useful results with least cost of development

As such, an architecture provides a functional toolbox to guide software designers and organize the software patterns that they develop (thus easing reusability).

The DoD framework for architectures (Department of Defense Architecture Framework [DoDAF]) provides a common basis for developing architectures and defines standard presentation products for each role: operational, systems, and technical architectures. The DF&RM DNN architecture is a technical architecture as shown in Figure 22.3. The DoD framework promotes effective communications of operational analysis and systems engineering from design to operations. It is used to visualize and define operational and technical concepts, plan process improvement, guide systems development, and improve interoperability. The DoD architecture framework is complementary to the Defense Information Systems Agency (DISA) Technical Architecture Framework for Information Management (TAFIM), the Common Operating Environment (Figure 22.4), and the layered network enterprise (Figure 22.5). The DF&RM DNN architecture provides a unification and organization of the DF&RM algorithmic approaches at the applications layer as depicted in Figure 22.6.

The criteria for the selection of the DNN architecture for DF&RM at the applications layer includes the following:

Breadth. Permits representation and comparison of all aspects of fusion

Depth. Sufficient level of detail to represent and compare all approaches to fusion (e.g., individual fusion patterns)

Affordability. Provides engineering guidelines for the application of the architecture to fusion system development to achieve useful results while minimizing development costs

Understandability. Familiarity in community and facilitation of user understanding and communication of all fusion design alternatives

Hierarchical. Sufficient module structure to enable incorporation of legacy codes at varying levels of granularity (e.g., whole fusion node, data association, hypothesis scoring, estimation, tracking filters, attribute/ID confidence updating)

Levels of abstraction. Permits abstraction of fusion processes higher in the chain of command to balance depth versus breath as needed

Extendibility. Ease of extension to highly related processes, especially resource management, and to new applications supporting code reuse

Maturity. Completeness of architecture components and relationships yielding reduced risk in applications

22.2.3   Components of the DNN Technical Architecture for DF&RM System Development

As data fusion is the process of combining data/information to estimate or predict the state of some aspect of the world¹ and resource management is the process for managing/controlling response capability to meet mission objectives, it is important to remember the definitions of optimal estimation and control. Namely

The estimation problem consists of determining an approximation to the time history of a system’s response variables from erroneous measurements.⁵

An optimal estimator is a computational algorithm that processes measurements to deduce a minimum error estimate of the state of a system by utilizing: knowledge of system and measurements dynamics, assumed statistics of system noises and measurements errors, and initial condition information.⁶

The control problem is that of specifying a manner in which the control input should be manipulated to force the system to behave in some desired fashion. If a performance measure is introduced to evaluate the quality of the system’s behavior, and the control input is to be specified to minimize or maximize this measure, the problem is one of optimal control.⁵

The solution of the Riccati equation provided by estimation theory provides the solution of the quadratic optimal control problem as described in Figure 22.7. Data fusion contains the estimation process that is defined by the data preparation, and association function such as resource management contains the control process that is defined by the task preparation and planning functions. As such the estimation and control duality can be extended to DF&RM (Figure 22.8). Figure 22.9 depicts this for the original data fusion tree architecture developed over 20 years ago.^7,8

DF&RM systems can be implemented using a network of interacting fusion and management nodes. The dual DF&RM levels provide insight into techniques to support the designs for each. The key components in the DF&RM DNN architecture are depicted in Figure 22.10. The fusion and management node networks are interlaced to provide faster local feedback as well as data sharing and response coordination. A sample interlaced DF&RM node network is shown in Figure 22.11. The dual fan-in fusion and fan-out management networks are composed of DF&RM nodes whose dual processing components are described in Figure 22.12.

The Kalman estimate is the expected value of the state, x, given the measurements. This estimate is optimal for any symmetric and convex performance measure. The separation principle⁹ implies that the optimal filter gain matrix is independent of the parameters in the performance loss function (and dependent on the plant statistical parameters) whereas the optimal feedback control gain matrix is independent of the plant statistical parameters (and dependent on the performance objective parameters). This duality enables the more mature field of data fusion to rapidly advance resource management much as estimation did for control over 35 years ago, and suggests that fusion is driven by the data and management is driven by the performance objectives. This duality motivates the following description of the levels 1 through 4 fusion and management processes. The levels are partitioned due to the significant differences in the types of data, resources, models, and inferencing necessary for each level as follows:

These levels provide a canonical partitioning that should be considered during DF&RM node network design. Though not mandatory, such partitioning reduces the complexity of the DF&RM software. An example of the clustering of DF&RM nodes by the above 0/1/2/3 levels, which enables fast response feedback, is shown in Figure 22.13. This shows how interlaced fusion and management nodes can be organized by level into similar response timeliness segments. The knowledge management for the interactions with the user occurs as part of the response management processing as needed. Figure 22.14 shows a distributed level 1 fusion network over time that has individual sensor level 1 fusion nodes providing tracks to the ownship multiple sensor track-to-track fusion nodes. The ownship fusion track outputs are then distributed across all the aircraft for internetted cooperative fusion. Further batching of the processing of each fusion level (i.e., to reduce complexity/cost) can then be performed to achieve the knee-of-the-curve fusion network, as depicted in Figure 22.15. A dual knee-of-the-curve design process can be used for resource management network design.

All the fusion levels can be implemented using a fan-in network of fusion nodes where each node performs data preparation, data association, and state estimation. All the management levels can be implemented using a fan-out network of management nodes where each node performs response task preparation, response task planning, and resource state control. These levels are not necessarily processed in numerical order or in concert with the corresponding dual fusion level. Also, any one can be processed on their own given their corresponding inputs. The DF&RM function-processing levels are based on what the user defines as his entities and resources of interest. The five DF&RM dual processing levels are described in more detail below in their dual pairs per level:

• Resource signal management level 0. Management to task/control specific resource response actions (e.g., signals, pulses, waveforms, etc.)

• Signal/feature assessment level 0. Fusion to detect/estimate/perceive specific source entity signals and features

• Resource response management level 1. Management to task/control continuous and discrete resource responses (e.g., radar modes, countermeasures, maneuvering, communications)

• Entity assessment level 1. Fusion to detect/estimate/perceive continuous parametric (e.g., kinematics, signature) and discrete attributes (e.g., IFF, class, type, ID, mode) of entity states

• Resource relationship management level 2. Management to task/control relationships (e.g., aggregation, coordination, conflict) among resource responses

• Situation assessment level 2. Fusion to detect/estimate/comprehend relationships (e.g., aggregation, causal, command/control, coordination, adversarial relationships) among entity states

• Mission objective management level 3. Management to establish/modify the objective of levels 0, 1, 2 action, response, or relationship states

• Impact assessment level 3. Fusion to predict/estimate the impact of levels 0, 1, or 2 signal, entity, or relationship states

  

  FIGURE 22.14
  Sequential distributed fusion network over time, sensors, and platforms.

  

  FIGURE 22.15
  Fusion network selected to balance performance and complexity.

• Process/knowledge management level 4: Design, adjudication, and concurrency management to control the system engineering, distributed consistent tactical picture (CTP), and product relevant context concurrency

• Process assessment level 4: Performance, consistency, and context assessment to estimate the fusion system measures of performance/effectiveness (MOP/MOE)

22.2.4   DF&RM Node Processing

22.4.1   Level 2 Fusion DNN Application Example

An example of the correspondence of the fusion node functions across fusion levels is described next for the hypothesis selection function within data association for level 2 fusion. Across the five fusion levels, the declarations (e.g., detections) of entity features, entities, their relationships, their predicted activities, and their correspondence to truth are accomplished in the hypothesis selection function within data association.

For level 2 fusion the data association function can apply deterministic, MHT, or probabilistic approaches to pick the highest confidence relationships, multiple alternative relationships, or the probabilistic combination of the alternative relationships to estimate the relationship states (e.g., the estimation of the size, shape, count, etc., of a cluster, the strength and type of jamming signal, or other relationship state updates).

More specifically, if the relationship of interest is aggregation, then each entity association confidence to its feasible aggregates can be entered into the aggregate-to-entity feasible association matrix. This matrix can be searched to select the best aggregate association for each entity or an MHT set of aggregations can be selected to create alternative scenes from which alternative aggregate states (e.g., size, mean separation, count, etc.) are estimated. Alternatively a probabilistic weighting of the alternative aggregations can be used to estimate the aggregate states similar to what is done in level 1 entity-state estimation using the probabilistic data association filter (PDAF) approaches. In summary, for each level 2 node the labeling of entities with relationships (i.e., discrete association in the data association function) and the estimation of the relationship states are tailored for that node.

24.4.2   Level 3 Fusion DNN Application Example

Impact prediction is based on alternative projected states. This typically requires additional information on the planned actions and projected reactions for the entities (e.g., in the battlespace). The utility function is usually based on mission success criteria. Both of these (i.e., projected actions and utility function) are not necessary for levels 0–2 fusion (e.g., because levels 0–2 typically only predicts the fused state forward to the current time tag for the next data to be fused). Utility assessment in the estimation portion of level 3 processing is based on the associated projected reactions.

The data association function in each fusion node at all levels refers to a decision process that is not an estimation process. The role for this decision process for level 3 can be more easily understood by first describing its role in level 1 fusion processing and its extension to all fusion levels. The association process for level 1 is a labeled set covering function that provides selected associations (e.g., scored clusters or assignments of 1 or more with false or true labels) to be used for each entity-state estimation (e.g., deterministic or probabilistic). The existence of this function in the DNN fusion architecture enables the fusion designer to be enlightened about the alternative approaches that take advantage of a data association function at each fusion level. In addition, the architecture provides the organization for the data association software patterns that have been implemented in other applications.

Specifically for level 3 fusion, the DNN architecture exposes the software designer to consider a segmentation of the level 3 processing (e.g., by data source, entity type, location, etc.) and association decision processes that provide selected associations usually between alternative COAs for the host platform, other single entities, and groups of entities (e.g., just as in level 1). A sample of the level 3 data association functions are as follows:

• Hypothesis generation to determine the feasible COAs of the entities or aggregates of interest for impact prediction

• Hypothesis evaluation techniques such as probabilistic (e.g., max a posteriori, likelihood, etc.), possibilistic (e.g., fuzzy, evidential), symbolic, unified, and so on, besides hypothesis evaluation scoring algorithms

• Hypothesis selection techniques such as deterministic, MHT, etc. and search algorithms (e.g., Munkres, Jonker-Volgenant-Castanon [JVC], Lagrangian relaxation, etc.) for the COAs association to be used as the basis of impact prediction

In level 3 utility state estimation, the selected COAs can be used to generate alternative utility assessments (e.g., using an MHT approach) or averaged using probabilistic data association such as done in level 1 fusion entity-state estimation. Thus, the designer can extend his knowledge of level 1 fusion approaches to the corresponding types of data association approaches to be used as the basis for the utility (e.g., threat) state estimation. Note that level 3 fusion provides the impact assessment for the user and for automated resource management nodes. Specific resource management nodes may need to predict the fused state in alternative fashions with respect to alternative resource plans.

22.4.3   Level 4 DF&RM DNN Application Example

The difference between mission impact (utility) assessment (of levels 0–2 states) and the level 4 fusion system process assessment is that the former predicts the situation utility functions based on the observed situation and the latter estimates the DF&RM system MOEs/MOPs based on a comparison of the DF&RM system output to external sources of reality and the desired responses. Both can be done on- or off-line. For example, fusion performance assessment can provide the user an estimate of the trust in online fusion products. By considering PE as a data fusion function the designer receives the benefits of the DNN architecture that provides an engineering methodology for the PE system development. This includes an understanding of the alternative batching of the DF&RM outputs that defines the development of a PE node network (e.g., batching of the data to be evaluated over DF&RM level, time, nodes, and output data types) and the functional partitioning of the PE nodes. For example, the DNN architecture exposes alternative PE node functional alternatives such as consideration of track-to-truth association hypothesis generation, evaluation, and selection that can be deterministic, MHT, probabilistic, symbolic, or unified, and the organization of existing and the new reusable data fusion SW patterns. As another example, the PE node network design phase of the DNN process exposes the design trade where the level 2 fusion PE nodes can be based solely on the level 1 track-to-truth association or can be integrated with the level 1 PE node to determine the track-to-truth association based on both the entity-fused states and their relationships.

As another example, the DNN architecture enables a comparison of Wasserstein metric approach to PE with other approaches. The Wasserstein metric as proposed is used for association and for the PE MOPs estimation. The distance metric, d(x_i,y_j), used in Wasserstein can be tailored to one application MOP (e.g., with increased error costs for targets of interest). Alternatively, the d(x_i,y_j) can be the track-to-truth association score, and then the Wasserstein C_i,j used to compute the PE MOPs within the DNN architecture. The approach, as given in Hoffman and Mahler,¹¹ computes a deterministic association of track-to-truth with no false alarms or missing tracks when the number of tracks and truths are equal. However, unless the transportation matrix search is changed (to include the case when the number of tracks differ with the number of truths), the track-to-truth associations are not assignments with false tracks and missed truths; and when the number of tracks matches the number of truths, associations are forced for distant track-to-truth best matches. Solutions to these problems can be formulated within the DNN architecture. For example, the transportation matrix search can be changed to first use a different association distance metric to declare nonassociations and to incorporate track and truth existence over multiple time points. In addition, the DNN architecture exposes the issue that separating the association metric from the MOP can provide flexibility to the PE designer to enable an association metric that is akin to how an operator would interpret the fused results (e.g., for countermeasure response) with the MOP metric tailored to the mission requirements.

Finally, the DNN architecture enables PE node designs to be interlaced with system design resource management node networks, especially for automated system design methodologies. PE nodes tend to have significant interactions with their dual level 4 processes in resource management in that they provide the performance estimates for a DF&RM solution that are used to propose a better DF&RM solution. The new level 4 resource management function for optimizing the mapping from problem-to-solution space is usually referred to as system engineering. This dual resource management level 4 provides a representation of the system engineering problem that partitions its solution into the resource management node processes: design preparation (resolve design needs, conflicts, and mediation), design planning (design generation, evaluation, and selection), and design implementation. As with the use of the DNN for all the other DF&RM levels, the payoff is primarily for software implementations (e.g., automated system design), although some insight into user fusion and management processes is provided.

22.4.4   Dual RM DNN Application Example

It is expected that the dual management levels will serve to improve understanding of the management alternatives and enable better capitalization of the significant differences in the resource modes, capabilities, and types as well as mission objectives. Thus the less mature resource management field can be bootstrapped with the more mature data fusion field technology much as the duality between estimation and control enabled the solution of the Riccati equation to be applied to the formulation and solution of the quadratic optimal control problem in the 1960s. As with the data fusion levels, the resource management levels are also not necessarily processed in order and may need to be integrated with other resource management levels or data fusion levels such as when the locally linear separation principle does not apply.

The process management portion of the JDL fusion model level 4 can be implemented with a set of resource management node within various levels of a Resource Management node network. Process management of a DF&RM system involves as the adaptive data acquisition and processing to support mission objectives (e.g., DF&RM network management, DF&RM node process management, data management to drive adaptive DF&RM networks, data/information/response dissemination, and sensor replanning management). This enables the developer to integrate fusion process management software nodes into appropriate levels within the resource management node network that is interlaced with the fusion node network segmentation of the DF&RM problem. Each data fusion or resource management process management node can then be designed using the DNN resource management node components when it is appropriate to apply the separation principle.

Process management also includes the DF&RM model management function. Typically data mining and ontology development functions generate the models used for DF&RM. For fusion these models store prior knowledge of entity parametric and attribute signatures, entity relationships, and the entity-state evolution sequences (e.g., COAs). The association of data to models determines the levels 0–3 states where previous events instantiate alternative entity behavior and relationship patterns and arriving events further substantiate, cause deletions, or instantiate new hypothesized patterns.

Data mining abductive (hypothesis creation), deductive (hypothesis selection), and inductive (hypothesis validation) reasoning can determine and adapt DF&RM levels 0–3 model parameters on- or off-line. Data mining can be interlaced with fusion level processes to provide such modeling. The model management is performed in the interlaced resource management nodes.

Historically there has been an overlap of the data mining and fusion communities. As Ed Waltz et al.¹² states, The role for abduction is to reason about a specific target, conjecturing and hypothesizing to discover an explanation of relationships to describe a target (i.e., hypothesis creation). The role for induction is to generalize the fundamental characteristics of the target in a descriptive model. The next step is to test and validate the characteristics on multiple cases (i.e., hypothesis validation). The more deductive traditional data fusion processes (JDL levels 0–3) are based on these data mining models. A primary purpose of mining is to discover and model some as aspect of the world rather than to estimate and predict the world state based on combining multiple signals.

The model information on the feasible level 2 fusion relationships and level 3 impact pattern recognition can be captured using an ontology. The applicable ontology definitions according to Webster’s are as follows:

• Ontology. (1) The branch of metaphysics dealing with the nature of being, reality, or ultimate substance (cf. phenomenology); (2) particular theory about being or reality.

• Phenomenology. (1) The philosophical study of phenomena, as distinguished from ontology; (2) the branch of a science that classifies and describes its phenomena without any attempt at metaphysical explanation.

• Metaphysics. (1) The branch of philosophy that deals with first principles and seeks to explain the nature of being or reality (ontology); it is closely associated with the study of nature of knowledge (epistemology).

As stated by Mieczyslaw Kokar, An ontology is an explicit specification of a conceptualization: the objects, concepts, and other entities that are assumed to exist in some area of interest and the relationships that hold among them. Definitions associate the names of entities in the universe of discourse (e.g., classes, relations, functions, or other objects) with human-readable text describing what the names mean, and formal axioms that constrain the interpretation and well-formed use of these terms. (An ontology is) a statement of a logical theory. An agent commits to an ontology if its observable actions are consistent with the definitions in the ontology (knowledge level). A common ontology defines the vocabulary with which queries and assertions are exchanged among agents.¹³ Ontologies are used to capture entity relationships in level 2 fusion whereas level 1 fusion typically uses taxonomies for entity attributes. Taxonomy is a type of ontology that is a complete and disjoint representation in a set-theoretical format. Taxonomies have been used extensively to represent the entity attributes (e.g., entity classification trees) within level 1 fusion. A formal ontology provides

1. A shared lexicon of relevant terms

2. A formal structure capable of capturing (i.e., representing) all relations between terms within the lexicon

3. A methodology for providing a consistent and comprehensive representation of physical and nonphysical items within a given domain

As held by Eric Little,¹⁴ the ontological construction is ideally derived from a synergistic relation with user information needs. The user/task information needs inform and bound the ontological structure. The user needs can be developed from cognitive work analysis techniques, empirical, or ad hoc top-down or bottom-up approaches. The ontological development structures and validates one’s common-sense abstractions of the world. The development is usually top-down, rationally driven, and reflects the epistemologically independent structure of the existing world. When the fusion and management separation principle is being applied, process management may dynamically manage which ontological or data mining processes that allowed to be run on the fusion processor based on the current mission state.

By considering process management (i.e., for either fusion or management processes) as a resource management function the designer receives the benefits of the resource management architecture in his design decisions. For example, fusion process management (i.e., refinement) exposes the following design alternatives:

• Process management is part of the overall management of the processor that fusion resides on and may need to consider or coordinate with the demands on the processor by other external users of the processing.

• The fusion process management may need to take direction from or coordinate with the needs of the other resources to provide tailored and timely fused results.

• The fusion process management may need to be interlaced with a fan-out fusion process management network batched over time, data sources, or data types so as to divide and conquer the fusion process management function even if there is no coordination with any resource management function.

• Each fusion process management node may want to utilize software patterns (e.g., fusion process plan generation, evaluation, and selection followed by tasking/control) organized within the DNN architecture.

Similar advantages are afforded to the other resource management functions. This also includes the proposed level 5, user refinement which is an element of knowledge management. As stated by Eric Blasch at Air Force Research Laboratory (AFRL), user refinement provides an adaptive determination of who queries information, who has access to information (e.g., information operations), and adaptive data retrieved and displayed to support cognitive decision making and actions (e.g., altering the sensor display). The DNN provides insight into the development of the knowledge management and the user interface (e.g., displays, audio, etc.). Management functions within the overall resource management system and their integration with the data fusion network.

22.4.5   DF&RM System Engineering as a Level 4 Resource Management Problem

System engineering is a process for determining a mapping from a problem space to a solution space in which the problem is to build a system to meet a set of requirements. Fundamental to the system engineering process (as in all resource management processes) is a method for representing the structure of a problem in a way that exposes alternatives and is amenable to a patterned solution. The DNN architecture for resource management permits a powerful general method for system engineering (e.g., for DF&RM system engineering). It does so by providing a standardized formal representation that allows formal resource allocation theory and methods to be applied. As a resource management process, system engineering can be implemented as a hierarchical, recursive planning and execution process (e.g., as DNN level 4 resource management).

The Data Fusion Engineering Guidelines can be thought of as the design specification for a resource management process, the phases being levels in a hierarchical resource management tree. A network-structured resource management process is used to build, validate, and refine a system concept (which, in this application, may be a network-structured resource management or data fusion process).

To illustrate how design management and process assessment (e.g., PE) can be implemented as dual processes much like the lower levels of fusion and management. Figures 22.26 and 22.27 show examples of dual levels 1–3 fusion and management and corresponding dual level 4 fusion and management node network designs, respectively. The system engineering process shown in Figure 22.27 is a fan-out (top-down) resource management tree interlaced with a fan-in (bottom-up) data fusion tree, similar to that shown in Figure 22.26. Design goals and constraints flow down from the system level to allocations over successively finer problem partitioning.

22.5.1   Dasarathy Fusion Model

The Dasarathy fusion model partitions the levels 0–1 fusion problem into categories that are based on the data, feature, and object types of input/output (I/O) for each. These are shown boxed in bold in Figure 22.28 with the extensions to level 2 and 3 fusion and response functions.

22.5.2   Bedworth and O’Brien’s Omnibus Model

The Bedworth and O’Brien omnibus model, as depicted in Figure 22.29, applies the OODA model in a feedback process. The omnibus model can replace a selected portion of itself (i.e., they show all four OODA loop parts inserted into the Orient box).

The DNN can be viewed as extending this using its tree of interacting fusion (i.e., orient) or management (i.e., decide) nodes (e.g., segmenting the sensors and effectors from interlaced fan-in fusion and fan-out management processes). The distinctions of sensor, soft, and hard data fusion and sensor management (i.e., the labels of the arrows in the Figure 22.29) are not made in the DNN model. The DNN provides structure for the DF&RM functions to be implemented in each of the OODA boxes so as to provide a hierarchical modular architecture to achieve affordability objectives (e.g., enabling software reuse and pattern tool development through standard hierarchical functional components).

22.5.3   The Kovacich Fusion Taxonomy

The Kovacich fusion taxonomy can be viewed as an extension of one proposed by Drummond. The fusion local nodes are allowed to be sensor nodes or track nodes. The sensor nodes are fusion nodes that generate unfused sensor data and the track nodes are fusion nodes that fuse one or more sensor nodes or other track nodes. The matrix in Figure 22.30 specifies the four basic fusion network structures that are allowed.

Each fusion node consists of the following subfunctions:

• Alignment. Aligns input data to a common frame

• Association. Associates input data to existing entities

• Estimation. Estimates, predicts entity state

• Management. Decides entity existence

• Distribution. Distributes entities

This is similar to the DNN fusion node except the management and distribution functions are inserted into their fusion node as compared with the DNN that segments management from fusion and includes communications management within the resource management node network. Their general N node fusion network is built using a combination of centralized/hierarchical and distributed fusion node networks. In the centralized/hierarchical the local output of fusion nodes A and B are fused into a global product at fusion node C. In the distributed, the local output of fusion node A is fused locally into a global product at fusion node C1, which exchanges information with fusion node C2 that, in turn, fuses the local output of fusion node B, as shown in Figure 22.31. The DNN can be seen as extending the types of DF&RM node networks allowed.

22.5.4   The Endsley Model

Endsley’s model can be summarized as containing the following functions:

1. Perception

   1. Explicit objects, events, states, and values

2. Comprehension (what is)

   b. Implicit meanings, situation types

3. Projection (what if)

   1. Future scenarios, possible outcomes

   2. Intentions, COAs

The Endsley model has rough correspondence to JDL and DNN fusion levels 1, 2, and 3, respectively. The focus is on user situation awareness techniques and less on providing a methodology (e.g., functional partitioning) for computer automation. The Endsley model is summarized and compared with the 1998 version¹ of the JDL model in Figure 22.32. Note that this version shows process refinement as the level 4 fusion process. The DNN architecture segments this into process assessment as a fusion process and implements the process management portion of it as part of the resource management network of nodes.