Multiple cyber ranges and test beds are currently being used for testing and training (Davis and Magrath 2013), with more coming on‐line each month, often for educational purposes. Each of these ranges includes a modeling platform for representation of IT systems of interest. OPNET and Exata are key names in producing cyber M&S development platforms. For example, the Joint Communication Simulation System (JCSS), which leverages OPNET, is commonly used for long‐haul communication evaluation. Similarly, CORONA (Norman and Christopher 2013) uses OPNET for T&E experiments.

10.1 Background

The speed and combinatorial nature of the evolving cyber threat demands a more flexible modeling & simulation (M&S) approach using cyber ranges for system test and evaluation (T&E). One approach is to review the baseline process to conduct cyber‐range events. This includes the logical range construct, that provides event environments to be constructed in a location‐independent manner, along with its application to the various phases of cyber range events. We also describe how cyber M&S is used in cyber‐range events, and the need for using more intelligent and autonomous simulations in the event control plane.

The Department of Defense (DoD) (Defense Science Board 2013) is keen to ensure that fielded military systems are hardened to cyber exploits and resilient to cyberattacks. Hardening systems requires T&E of the processes and technologies related to countering cyberattacks. Training personnel is an essential component of making systems resilient to cyberattacks. The cost of using actual cyber assets or systems for T&E and training can be prohibitive in large scale, and therefore cyber M&S is used to reduce cost while doing cyber‐range events for testing and training. The foundational concepts for using cyber M&S in cyber‐range events are introduced here, along with applicable definitions and constructs.

The Chairman, Joint Chiefs of Staff, issued an instruction in February 2011 (Joint Chiefs of Staff 2014) mandating that all DoD exercises begin to include realistic cyberattacks into their war games. To paraphrase the threat,

If this level of damage can be done by a few smart people, in a few days, using tools available to everyone, imagine what a determined, sophisticated adversary with large amounts of people, time, and money could do.

Multiple efforts were started to coordinate cyber training, T&E. For example, the Joint Mission Environment Test Capability (JMETC) (Arnwine 2015) has the following goals for Cyber T&E:

• T&E must accurately and affordability measure cyberspace effectiveness and vulnerabilities of DoD systems (DoD Operatonal Test and Evaluation [DOT&E] 2015).
• Address the requirements for building Cyberspace T&E Process, Methodology, Infrastructure, and Workforce.

Cyber‐range events vary in complexity and their objectives, and cover a broad spectrum of event types. For example, some events are conducted for training cyber protection forces, and some are conducted for evaluation of people, process, and technology through large‐scale exercises, and yet others are conducted for developmental testing (DT) or operational testing (OT). Events may also be conducted for experimentation with technology or tactics, or to assess mission readiness. An early example of a cyber‐range used for academic studies is the DETER test bed (The DETER Testbed: Overview 2004). DoD Enterprise Cyber Range Environment (DECRE (DOT&E 2013)) is a federated set of ranges within DoD, comprised of the National Cyber Range (NCR), the DoD Cyber Security Range, the Joint Information Operations Range (JIOR), and the Joint Staff J6’s C4 Assessments Division (C4AD) (Table 10.1).

Table 10.1’s cyber ranges are used to support cyber‐range events such as Cyber Flag (Hansen 2008). In addition, the need for interoperability is being facilitated by the Cyber Range Interoperability Standards Working Group (CRIS WG) (Damodaran and Smith 2015).

10.2 Cyber Range Interoperability Standards (CRIS)

While Table 10.1 provides an overview of current cyber ranges and their capabilities within the DoD, standards for the interoperability of the cyber‐range events are still developing. Currently, the ground work is still being laid in terms of coordination bodies and authoritative data common to professional fields. The primary use cases considered by the standards effort are cyber‐range events, including training, exercises, and testing events. The reasons behind considering these use cases initially is that first, they represent a majority of the events, and second, the other use cases, such as mission rehearsals, are expected to be supported by the same standards with minor changes. A prominent standards effort is the CRIS WG (Damodaran and Smith 2015) that was formed by TRMC in collaboration with MIT Lincoln Laboratory in 2012 to foster interoperability of tools used to support cyber‐range events such as Cyber Flag (Hansen 2008), which developed a cyber‐range lexicon (Damodaran and Smith 2015), and a baseline process for cyber‐range events (Bakke and Suresh 2015).

10.3 Cyber Range Event Process and Logical Range

The purpose of cyber‐ranges is to conduct events. The events that are conducted will follow a baseline process, and therefore, it is beneficial to explore this process. While the overall flow of a cyber‐range event (Figure 10.1) follows a familiar sequence, the incorporation of a logical range provides for both event scalability and the opportunity to add scope, via the incorporation of other systems, to the T&E or training event.

Figure 10.1 describes the overall flow of a cyber‐event. At the top of Figure 10.1, the four distinct phases for most cyber‐range events: plan, deploy, execute, and analyze are shown. Activities in the planning phase, shown below “Plan” (Figure 10.1), include gathering and specification of event requirements such as event goals, design of all aspects of the event, and scheduling the event. The outputs from the planning phase are captured and stored for use in the remaining phases. A full‐fledged planning phase may not exist for some events that have a “persistent event environment” that is planned once, and subsequently deployed as many times as necessary using the event configuration of a previous event. It is normal for an event to iterate through one or more phases multiple times before proceeding to the next phase.

The deployment phase includes the deployment of the logical range (Figure 10.2). The logical range construct is a useful abstraction, and it has an implementation. The logical range provides for both event scalability and the opportunity to add scope, via the incorporation of other systems, to the T&E or training event. The cyber capabilities or infrastructure that are required for an event may not be available at a single site of a cyber‐range, and therefore, multiple sites of one or more cyber‐ranges may need to be interconnected to support an event. Alternatively, a single cyber‐range may be supporting multiple events simultaneously. The logical range construct is used to abstract away the details of where the range infrastructure elements, including specific assets and capabilities, are physically located. The logical range is created based on the requirements and designs developed during the planning phase. The logical range is self‐contained and is isolated from its surroundings. This isolation serves two purposes. First, the technologies that are deployed within the logical range do not escape into the real‐world causing damage. Second, this isolation prevents unintended exposure of the logical range.

A cyber‐range event is conducted using representative and operationally relevant cyberspace elements. Cyberspace is defined as “A global domain within the information environment consisting of the interdependent network of information systems infrastructures including the Internet, telecommunications networks, computer systems, and embedded processors and controllers” (CNSSI 2010). Therefore, cyberspace spans every system that has cyber elements, irrespective of whether a system is connected to a network. For example, a network router is in cyberspace, and so are all avionics components of an airplane. An operationally relevant and representative version of the actual environment with any additional features, including potentially one or more System‐Under‐Test (SUT), are deployed (Figure 10.2) on the logical range. This environment is called the event operating environment. The event plane includes the blue and red participants of the event who operate within the event operating environment as well as the operating environment. Since all participants in an event occupy the logical range, the logical range extends to, and includes, the remote terminals of any remote participants.

A control plane is deployed in the logical range for controlling the execution in the event operating environment. The participants in an event are organized in teams or cells. The control plane includes participants from all the teams operating on the control environment. A range support team, sometimes also called a green team, is responsible for the correct operation and availability of the infrastructure used to build the logical range. This team monitors the infrastructure, ensures its healthy operation, and makes any changes requested by the white cell on the infrastructure. The gray team is responsible for generating traffic within the operating environment. Simulated traffic provides the necessary background traffic and activities to conduct an event successfully. This team also monitors the health and status of the traffic generation systems they are responsible for. The white cell members in the control plane have complete authority on the entire logical range, participants, and the underlying infrastructure to make any changes at any time during the event. The instrumentation team installs appropriate probes in the operating environment, and is also responsible for the collection and archival of event data.

During the planning phase, several items are specified. These items include event goals, metrics, infrastructure required to support the event, scenarios to be executed during the event, the various environments of the event, cyber assets and capabilities needed for the event, the cyber ranges participating in the event, and tools required to support the event. The planning phase also results in the creation of plans and schedule for the various event milestones, asset and capability acquisition, as well as the plans for collecting data. The designs of the event operating environment, control and instrumentation environments, as well as the Mission Scenario Event List (MSEL) are also finalized during the planning phase. In the deployment phase, the elements of the event environment are deployed, verified, and validated. The execution of the event occurs during the execution phase, and event data is generated, and collected during the execution. The event data is used for monitoring the progress of the event as well as for analyzing the effectiveness of the event and report the event results to stakeholders.

The duration of event execution phase, typically a few days to a couple of weeks, is determined during the planning phase. The planning phase includes the specification, design, and scheduling of all activities related to the logical range, event operating environment, and the event phases. The planning phase may take anywhere from a few weeks to several months depending on how well the requirements are defined, ease with which the right assets and capabilities are acquired, reserved, and scheduled for the event, and the design complexity of the event environment. The activities during the planning phase are usually not conducted on the logical range. The bulk of these activities are conducted in a planning network prior to the construction of the logical range. However, sometimes updates to any of the plans or designs are done within the logical range in the control plane. The planning network can be the standard enterprise environment used for day‐to‐day business. The deployment phase may be a few days or months depending on the scale and complexity of the logical range, environment, and how widespread and integrated the use of automation tools are. The analysis phase may take a few days to months depending on the amount of event data generated by the event.

10.4 Live, Virtual, and Constructive (LVC) for Cyber

The event operating environment contains actual cyber assets and capabilities, or their models. The Live, Virtual, and Constructive (LVC) terminology (Henninger 2008a, b) used to classify kinetic‐range simulations can be used in cyber ranges as well, though these terms need re‐definition within the cyber context. In cyberspace, LVC simulations are used in events to provide the appropriate level of fidelity required for the event operating environment. Below, we define the LVC terminology briefly. For a more descriptive definition of these terms, please refer to (Damodaran and Smith 2015).

We assume the definitions of the terms Model and Simulation (Zeigler et al. 2000), and Emulation (SISO, 1999). A real‐world asset or system used to represent that asset or system, respectively, in a simulation is labeled as an “actual” asset or system. When a simulated model of an asset or system provides some of the operationally relevant interactive interfaces, protocols, or features of the actual asset, including partial or complete simulation of internals of an asset or system, it is called a “representative” model. It is possible to conduct emulation over the provided interfaces or protocols. When actual assets interact with representative models of protocol‐level fidelity representations of actual systems, where ease of (re)configuration, replication, restoration, and physical limitations make a representative system preferred over the actual one, there is no physical representation of the actual system. In this specific case, a representative model of the system only provides a cyber “attack surface,” through the protocol/packet interfaces, but not asset internals or attributes susceptible to other attacks. An agent that embodies the behavior of a human is a representative asset model, as long as the agent is capable of interacting effectively with actual assets or systems, or their representative models.

Some of the assets or systems used in simulations may not provide any of the operationally relevant interactive interfaces, or protocols, of the actual asset or system. Such a system or asset model is referred to as “limited.” A limited model may not provide relevant attack surfaces or participate in synthetic traffic generation. A model does not qualify as either limited, or representative model for an event if it does not provide any operationally relevant interfaces, protocols, or features for that event.

In looking at the differences among LVC cyber simulations using actual systems or assets or their representative, or limited models. Live (Cyber) Simulation: In this type of simulation, real‐world assets operate on/with real‐world systems and protocols. Even though actual assets are used, these are considered simulations because the scenario is simulated and attacks are not conducted against a live enemy. Examples include:

• Actual operators, actual network devices, actual machines, actual non‐emulated/simulated software.
• Packet, protocol, or frequency‐level attack and response launched by actual systems and/or live attackers.

Virtual (Cyber) Simulation: In this type of simulation, actual assets may interact with limited or representative system models, and limited or representative asset models may interact with actual systems.

Examples include:

• Asset emulators running on virtual machines.
• Automated response of a virtual machine to an attack.
• Replay of a logged live attack onto the live or virtual systems.
• Automated or semi‐automated attack simulators that replicate the actions of a live red team or real‐world threat.
• Simulated users (Assets) in a traffic generator using actual systems to generate traffic.
• Accurate (high‐fidelity) representations of system administrator GUIs.

Constructive (Cyber) Simulation: In this type of simulation, limited or representative asset models interact with limited or representative system models. The simulated systems are characterized by lower fidelity global/enterprise‐level networks and effects representations, and are not vulnerable to direct live or virtual exploits and manipulation.

Examples:

• Simulated internet‐scale traffic generation, background noise, and high‐volume gray‐space.
• Virus infection and worm propagation simulations.
• Asset representations with simulation interfaces that must be translated or bridged to connect with virtual and live assets.

While the terminology of LVC simulation is useful for distinguishing among different types of simulations, it is possible that, in practice, a “federated simulation” is the result of interaction among multiple simulations, often of different types. In such a situation, the type of the federated simulation is the lowest of the simulation types of the interacting simulations, where “live” is the highest type and “constructive” is the lowest type. For example, let us say a federated simulation results from the interaction of virtual simulations and constructive simulations. The type of the federated simulation is then constructive.

10.5 Applying the Logical Range Construct to System Under Test (SUT) Interaction

The logical range construct described in the previous section can be applied fruitfully to analyze problems in events. We describe one such interesting problem, namely, how the elements in an operational environment should interact with a SUT during an event. Often, SUT will be deployed in the event operating environment. However, sometimes it may not be feasible to extend the logical range to include the SUT due to logistical or other reasons. In this case, an SUT is placed external to the logical range. For example, Table 10.2 provides an analysis of how an SUT may interact with an event operating environment based on both its deployment location and possible simulation approaches.

As shown in Table 10.2, when the SUT is in the logical range’s operational environment, the SUT simulation state is embedded in the operational environment state; and the cause of an effect in the operational environment can be traced back to entities within the operational environment. In contrast, when the SUT is not in the operational environment of a specific logical range X (SUT may be in another logical range Y or not in any logical range), then, though the SUT simulation effects still correspond to the local operational environment state, entities within the operational environment lose their traceability to nonlocal effects. In this situation, the control plane has the responsibility to provide this information to the entities within the operational environment, as appropriate.

10.6 Conclusions

While M&S provides both a sanitary environment to test cyber effects and the requisite traceability for understanding threat behavior in an emulated attack, cyber M&S is still growing into the mature capability that current DoD stakeholders rely on in their current LVC simulations. In addition, the “right level” of cyber M&S is still being determined, in balancing the mix of actual assets and systems with limited and representative asset and system models for simulations.

Foundational concepts provided in this paper (e.g. logical ranges) (Figure 10.2) provide the tools, as shown in Table 10.2, for cyber M&S to grow beyond stand‐alone simulations; coupling ranges in terms of both model composition and the communication of effects. Due to the inherently dynamic cyber terrain, consisting of both moving targets (Okhravi et al. 2013a, b; Colbaugh and Glass 2012), and necessarily dynamic defense, the foundational work described here provides an extensible M&S approach for a rapidly developing system of systems domain.

10.7 Questions