CHAPTER 18

SCENARIOS IN AIR TRAFFIC CONTROL (ATC)

Perminder Sahota

Praxis Critical Systems, London, UK

CREWS-SAVRE is a process with supporting software tools. It was developed as part of the EU-funded Framework IV crews long-research project to support the systematic and domain-independent generation and walking through of scenarios described in Chapter 9. Design of the software scenario walkthrough tool was based on a cognitive principle familiar from prototyping—that people recognise items, such as events, better than they recall them with from memory. Therefore, CREWS-SAVRE automatically generates and presents candidate exceptions that stakeholders recognise as relevant and possible, helping people to generate new requirements and so reducing the risk of missing requirements.

This case study reports on an experience within Eurocontrol that used the CREWS-SAVRE walkthrough approach to help specifying the requirements for a state-of-the-art air traffic control system. The project was called CORA-2 (COnflict Resolution Assistant).

APPLICABILITY

The CREWS-SAVRE approach to scenario walkthroughs adopted for CORA-2 is part of the RESCUE process. This technique has been used in safety-critical domains such as warship development and air traffic control. We expect it to be suitable for safety-related projects in other domains.

Scenario walkthroughs are particularly useful for systems that have a large stakeholder involvement. The technique might be helpful in resolving the requirements for highly contentious projects even where safety is not a primary concern.

ROLES IN THE LIFE CYCLE

Scenarios were used in facilitated workshops with stakeholders to walk through each scenario. The aim of these workshops was to elicit and validate correct, complete, and testable requirements.

For each normal course scenario of a use case, the CREWS-SAVRE software tool automatically generated and presented alternative course (candidate exception) scenarios, which stakeholders were able to identify as relevant and possible and to generate new requirements for them. A detailed overview of the CREWS-SAVRE tools is given in this volume, Chapter 9, Systematic Scenario Walkthroughs, by Neil Maiden.

We used the normal course scenario of each use case to structure and present the requirements. Each requirement was positioned according to the event in the scenario, which the requirement either enabled (e.g. functional requirements) or constrained (e.g. performance and reliability requirements). This traceability structure was mirrored in the supporting requirements management tool, Rational Requisite Pro.

Scenario walkthrough workshops provided the context for stakeholders to discuss, negotiate, and resolve information. The structure and natural language of the scenarios provided a solid focal point for stakeholders to resolve requirements conflicts.

KEY FEATURES

Key features of the CREWS-SAVRE scenarios walkthrough approach included

• Automatic generation of scenarios from use cases, using the CREWS-SAVRE automated generator tool. Each different possible ordering of normal course events is presented in a different scenario.
• Automatic generation of alternative courses specific to the Air Traffic Control (ATC) domain, using the CREWS-SAVRE generator tool. The tool's algorithm generates candidate possible alternative courses expressed as ‘what-if’ questions for each normal course event by querying a database that implements a simple model of abnormal behaviour and state in socio-technical systems.
• Specific domain exceptions from the ATC can be imported into the tool and used by the generation algorithm to generate domain-specific scenarios.
• Scenario walkthroughs are conducted with stakeholders to elicit and validate a set of complete, correct, and testable requirements.

STRENGTHS

The CREWS-SAVRE tool automatically generates alternative course scenarios from use cases.

The CREWS-SAVRE tool for automatic scenario generation can be configured to produce scenarios and exceptions that are more domain-specific, and hence more meaningful to stakeholders.

The CREWS-SAVRE scenario presenter tool allows requirements and domain assumptions to be presented (in context) to stakeholders, making it easier for stakeholders to understand why they are necessary. This in turn allows stakeholders to validate the requirements and assumptions.

The CREWS-SAVRE presenter tool allows requirements to be captured and recorded in the presence of stakeholders during scenario walkthroughs, allowing direct validation by stakeholders.

The CREWS-SAVRE presenter (using the interactive style of the scenario presenter tool) provides a virtual storyboard style simulation. This allows people to think and communicate in the real context, and highlights missing requirements.

Specific guidelines for writing use cases are provided, allowing a consistent and managed approach to developing complex use case models.

A process supported by detailed guidelines allows successful scenario walkthroughs to be conducted in a pragmatic and proven way. This provides the requirements team with added confidence to make scenario walkthroughs effective.

Information elicited during the walkthroughs can be managed and structured within use cases, allowing a direct dynamic traceable relationship between requirements and use cases.

Scenario walkthroughs elicit more requirements than other elicitation methods.

Using scenarios helps reduce system complexity by focusing stakeholders' attention on critical areas. The controlled and structured scenario walkthrough workshops provide the right environment to discuss such critical areas.

The natural language format of scenarios and the informal approach of the walkthroughs allow users to adapt quickly to the workshop techniques.

WEAKNESSES

Use cases need to be stable and formally signed off to allow the CREWS-SAVRE tool to generate accurate and useful scenarios. Changes in the use cases content after scenarios have been generated may require the scenario generation process to be repeated, causing delay and expense. However, one lesson learnt from the CORA-2 project is that obtaining formal agreement and sign-off of use cases can be difficult, especially when assumptions are made, and changes occur randomly as knowledge of the domain and system progresses during the course of the project.

Missing important stakeholders can affect the quality of output of the scenario walkthroughs and risk missing key requirements. For scenario walkthroughs to be successful in obtaining requirement completeness, relevant stakeholders are required to be available during the workshops. On the basis of the CORA-2 project experience, it is not easy to involve stakeholders, especially when they are widely dispersed geographically.

Running scenario walkthrough workshops using several scenarios can be difficult to do and time-consuming. The workshops need to be carefully designed to maximise efficiency, and encourage stakeholder involvement.

The CREWS-SAVRE generator tool needs to be fine-tuned to the project to control the generation of abnormal events and states in alternative course scenarios. Generating too many exceptions makes workshops difficult to run. Without historical data, it was difficult during the project to know which exceptions would be most useful and which were not applicable to the project.

BACKGROUND

The cora-2 Project

CORA-2 will provide computerised assistance to air traffic controllers to resolve potential conflicts between aircraft. This need is predicted to become critical as increases in the number of aircraft and conflicts lead to increased workload for controllers, and the need for more complex resolutions of conflicts.

The CORA-2 project requirements team was multinational and multi-disciplined. It consisted of system engineers, human factors experts, requirement engineers, and air traffic controllers. The team produced an Operational Requirements Document for the CORA-2 system with approximately 400 requirements structured as 22 use cases. The document also contained use case and i* models (Yu 1997), and some completed VOLERE requirement attributes (Robertson and Robertson 1999).

The rescue Process

Scenario walkthroughs were one particular stream of the RESCUE (Requirements Engineering with Scenarios for a User-Centred Environment) process, which was developed for Eurocontrol by multi-disciplinary researchers from the Centre for HCI Design at City University in London.

RESCUE supports a concurrent engineering process in which different modelling and analysis processes take place in parallel to support the generation of complete and correct use case models and descriptions and, ultimately, complete, correct, readable, and testable requirements (Maiden and Jones 2003, Maiden, Jones and Flynn 2003). The RESCUE process was divided into two phases: ELABORATION and ANALYSIS. The elaboration phase consists of eight distinct processes. The analysis phase consists of a further four distinct processes as shown in Figure 18.1.

Scenario Walkthroughs

The Use Case Modelling stage of the RESCUE process is made up by four key stages to conduct effective scenario walkthroughs, shown in Figure 18.2. The different roles and time-scales are shown for the stages in the CORA project.

The following sections explain each stage in further detail.

Use Case Modelling

The use case modelling involved the investigation and modelling of the different system boundaries. Context diagrams and i* modelling was used to directly inform the development of the use cases (Jacobson, Booch, and Rumbaugh 2000). During this stage the team developed a use case model. The use case model consisted of 15 separate use cases with 13 actors. Figure 18.3 shows a snapshot of the final use case model.

FIGURE 18.1 rescue process (2001)

FIGURE 18.2 Stages for the scenario walkthrough process during the CORA project

FIGURE 18.3 cora-2 Use case model supplies use cases; the crews-savre tool generates scenarios from these

Use Case Descriptions

The team wrote detailed use case descriptions in a structured template derived from industry-best practices (e.g. Cockburn 2000). This template encouraged consistency and completeness across the set of use cases. An actual use case from the CORA-2 project is illustrated in Figure 18.4.

Authoring was guided by

• use case style and content guidelines from the CREWS-L'ECRITOIRE method (Ben Achour, Rolland, Maiden, and Souveyet 1999); see also Chapter 8 of this volume on the L'Ecritoire guidelines by Camille Salinesi);
• temporal semantics expressed as action-ordering rules, and
• an extensive lexicon of domain-specific nouns and verbs elicited from ATC domain experts.

Each use case description was written using the work from the other steams in the RESCUE process (System Modelling and Creativity Workshops) and through structured interviews with domain experts.

FIGURE 18.4 cora-2 Use case template

Further richness was added to the use cases through investigating different contexts that use cases can be applied to (i.e. air traffic environments), and also possible variations of events in the use cases. This was not part of the original scenario walkthrough process, but proved to be an important piece of additional work.

During the walkthroughs, stakeholders raised different requirements and critical conflicts for the same use case, but for different contexts in which the use case's normal course (what Cockburn calls the main success scenario) can occur.

For example, air traffic controllers play different roles when dealing with conflicts in the sky. There is often a planning controller and a tactical controller, who together resolve conflicts. Each role has specific responsibilities. However, one air traffic controller can simultaneously play both roles whilst resolving conflicts. To capture this important domain knowledge, scenarios describing air traffic controller interactions in these different contexts were explored in the walkthroughs.

Other examples included scenario walkthroughs for the same use cases being conducted in the context of the different times of the day, that is, specific scenarios provided different requirements when assuming light air traffic on a Sunday morning, and heavy traffic on a Monday morning. By understanding the different contexts a particular use case can be applied to, important information was captured.

On completion of the use case descriptions, a use case specification was produced and parameterised to generate scenarios automatically using the CREWS-SAVRE software.

A key feature of the CREWS-SAVRE generator tool was that it had been tailored to generate scenarios with what-if alternative courses that were specific to the ATC domain, through the addition of 140 classes of abnormal behaviour and state. This allowed scenarios that are more specific to the domain to be generated. The domain-specific abnormal classes were elicited prior to scenario generation during a workshop with four air traffic controllers. The results were modelled in UML and extended with rules to generate alternative courses from an ATC lexicon used to define normal course actions in the use cases. The ATC lexicon consisted of 140 agents, objects and actions, and was further elicited from two controllers. During elicitation sessions, City University staff helped the controllers develop over 500 ATC-specific rules to generate ATC-specific alternative courses for different ATC actions.

Scenario Walkthroughs

The CORA team organised and facilitated a series of walkthrough workshops, involving key stakeholders. The generated scenarios were presented using the CREWS-SAVRE presenter tool; an example is shown in Figure 18.5.

There are four specific parts to a CREWS-SAVRE scenario. The left-hand menu provides different functions for viewing the scenario and comments and requirements generated from it. The top-line buttons offer walkthrough functions (e.g. next or previous event, add/edit/delete event) and functions to add, edit, or delete comments and requirements (these functions were not available during the project). The left-hand main section describes the normal course event sequence for the scenario. Each event describes the start or end of an action, thus enabling a scenario to describe concurrent actions in this text-list form. The right-hand main section describes generated alternative courses for each normal course event, presented in the form of ‘what-if’ questions. Some generic requirement statements are provided on the Presenter for the team to reuse; during the workshops, these turned out to provide excellent triggers to start discussions, and helped to create actual requirements.

FIGURE 18.5 Scenario presenter tool

A series of walkthrough workshops were held, involving a wide variety of stakeholders. The scenarios were prioritised in order of importance. Depending on the type of scenario, specific stakeholders were invited (i.e. human factors stakeholders were invited to scenarios that occurred mostly of human-machine interactions). Each session lasted particularly two to three hours, and covered at least the scenarios for one use case (normal and alternative course). We used props (physical objects and artefacts) to encourage creative thinking amongst stakeholders and to help people think of different types of requirements (i.e. different requirement types and metrics, different ATC contexts).

Workshop Environment

The CREWS-SAVRE presenter tool encourages systematic requirement elicitation. It guides its users to walkthrough each normal course event, and each alternative course linked to that normal course event in turn. However, after the initial scenario walkthrough, the CORA team organised the workshops to take into account ergonomics issues about the style of facilitation, room layout, and so on. This is an important area for scenario workshops, and is not explicitly detailed in the process and guidelines provided. Workshops achieve more in less time if they are managed and facilitated properly (Cameron 2003).

The room layout chosen for the scenario workshops was structured according to RAD and JAD guidelines (Maiden and Jones 2003). Stakeholders sat around a U-shaped table with copies of the use case specification for the scenario being walked through. The facilitator stood inside this U-shaped table at the focal point of the interaction in the walkthrough; the layout is shown in Figure 18.6.

The facilitator controls the speed and focus of the walkthrough via the Scenario Presenter spreadsheet that is displayed permanently using a LCD projector. The screen is surrounded by prompts that encourage requirements discovery and creative thinking, and by a simple visual prompt of the repeating scenario walkthrough process that must be undertaken throughout the session. The prompts included ideas from earlier creativity workshops, different requirement types, and the ATC contexts that apply to the visible scenario.

To the left of the facilitator stands the scribe who records and manages information being stored on two large flipchart sheets. The first sheet records all requirements arising from the scenario walkthrough. It was divided into two parts—a description and rationale of the requirement, and the source of the requirement, in terms of the scenario event and alternative course ID that led to this requirement. To the right of the requirement flipchart stands a prompt list of the different types of requirement that can be specified in the CORA-2 requirements specification. The second flipchart records information about information emerging from the walkthrough that are not requirements. Such information includes assumptions, problems, and even changes to the scenario. The scribe was responsible for the accuracy and clarity of the information being recorded on the flipcharts, as it is this information that later passes through the quality gateway (Robertson and Robertson 1999) into the requirements specification.

FIGURE 18.6 Sketch of the scenario workshops during the CORA Project

The walkthrough process causes requirements to emerge by being acquired, discovered, or invented. The main purpose of each session is to walk through the scenario, identifying, describing, and documenting the first-cut requirements. Therefore, the first-cut requirement description is recorded informally in paper-based form. The first-cut requirements were then formally recorded in the project requirement management tool (Rational RequisitePro). This maintained traceability and placed the requirements under change control. The requirements were then quality checked and those that were deemed to have passed check were marked as ‘checked’ and were ready for final approval.

Owing to project time constraints, stakeholder availability, and the time required to carry out scenario walkthroughs, the CORA team walked through 10 scenarios in total.

RESULTS

Requirements versus Time

Structured scenario walkthroughs were more effective for discovering requirements than the brainstorming and interview techniques applied earlier in the project. For the CORA project, more requirements were generated during the three-week period of the scenario walkthroughs, than the previous 10 months of the project requirements starting (Figure 18.7).

FIGURE 18.7 Table showing the number of new requirements generated before and after the scenario walkthrough technique was introduced to the project

It is clear from these data that the scenario walkthroughs were more effective for requirement generation than whatever was done previously. However, it should be noted that the walkthrough workshops were intensive and lasted typically between three to five hours each. A more accurate measure of effectiveness would be obtained by comparing the actual time spent capturing requirements in the 10 months prior to the walkthroughs.

It may be suggested that if scenario walkthroughs are so cost-effective for requirement elicitation, then surely they should be introduced at the start of a project. But for the scenario walkthroughs to be successful, the generated scenarios must be accurate and relevant. This can only be achieved through solid engineering groundwork. The use cases in the CORA project were directly based on the work conducted during creativity workshops, as well as by system modelling using the i* methodology (Maiden, Jones and Flynn 2003).

Requirements Validation Using Use Cases

Scenario walkthrough interlinked the activities of eliciting and validating requirements.

During the scenario walkthroughs, existing requirements were incorporated into the scenario structures and paper copies were given to stakeholders during workshops. Stakeholders were then able to read requirements in the context of the underlying use case, making the requirements more readable and understandable. Requirements that were previously captured months ago from departed stakeholders could now be brought back to life and subsequently validated. Figure 18.8 shows a real snapshot of how requirements were integrated into scenarios.

Scenario walkthrough workshops provided the context for stakeholders to discuss, negotiate, and resolve information. The structure and natural language of the scenarios provided a solid focal point for stakeholders to resolve requirements conflicts.

Domain-Specific Scenarios

The experience of using CREWS-SAVRE scenarios in the CORA-2 project suggests that systematic walkthroughs of simple scenarios (not containing much domain knowledge) are more effective for discovering system requirements. The majority of the generated requirements were not associated with domain-specific alternative courses. The findings show that 33 of the 79 requirements linked with the domain-independent alternative courses were generated by just 5 of the 54 possible classes of ATC abnormal behaviour and state (Mavin and Maiden 2003).

FIGURE 18.8 Snapshot of the use case specifications given to stakeholders during the scenario walkthroughs. Stakeholders and facilitators used this to validate the existing requirements and avoid duplicating requirements

However, a majority of the stakeholders stated that they reacted better to seeing ATC domain-specific scenarios, as these were described in their shared language, making it easier for them to communicate effectively. These conflicting views probably merit further testing for this finding.

Requirement Types

Most of the generated requirements were functional. Figure 18.9 shows the totals of requirements by type (Maiden and Mavin).

One reason for the greater number of functional requirements came from the walkthrough facilitators, who mentioned “due to time constraints the emphasis was often on functional rather than non-functional requirements“. Non-functional requirements are more difficult to refine to a measurable level (but see Chapter 7 on Negative Scenarios by Ian Alexander in this volume) than functional requirements. During the time-boxed workshops, the majority of the non-functional requirements were elicited at high level only.

FIGURE 18.9 Totals of requirements by type discovered during the cora-2 scenario walkthroughs

Another possible factor contributing to the large proportion of functional requirements was the culture of the organisation, which relies on obtaining formal sign-off of functional specifications before proceeding with real life prototyping and simulation. Once the formal functional specifications are agreed, the organisation then reverse-engineers detailed non-functional requirements from the functional specification. There are obvious pros and cons with this approach, but we may simply note that whilst planning scenario walkthroughs, it is wise to take such issues into account.

Requirements Management

Scenario walkthroughs produce a mass of information, which needs to be managed to reap the benefits. The CORA team documented requirements within the scenarios themselves, and then reflected this structure in the supporting requirements management tool. This provided an effective method of analysing, documenting, and structuring the requirements in a format that all the different stakeholder groups (i.e. developers, testers, managers, and users) could understand.

Planning Workshops

It is important to be ready to act on unexpected events in any workshop. Most workshops achieve more in less time if they are managed and facilitated properly (Cameron 2003). An important lesson learnt from this case study is that it is not easy to generate or author the right use cases or scenarios first time. It is important for the stakeholders to familiarise themselves with the normal and alternative course scenarios before they are exposed to them during the walkthroughs. This will prevent long discussions, conflicting view, and confusion over some areas of the scenario during the time-boxed walkthroughs.

Formal review and change control process should be adhered to during scenario walkthrough process. The CREWS-SAVRE tool generated scenarios based on the normal course events of the use case. Changes to the use cases resulting from stakeholder discussion during the first walkthrough workshops resulted in the scenarios being regenerated.

How Many Scenarios Do We Need?

One issue for the CORA project was determining how many scenarios were needed to ensure that all the requirements had been captured or validated. This points to a dilemma underlying much of scenario-based RE:

• Scenarios are powerful because of the detail they contain, but “the devil lies in the detail”, as you never know when you have captured everything that matters. This creates a pressure to explore yet more variations and exceptions to see if they are important.
• But since scenario workshops are costly in time and effort, it is essential to limit the number of scenarios used.

Other stakeholder-centred processes such as those described by Ellen Gottesdiener in Chapter 5 and Karen Holtzblatt in Chapter 10 of this volume can help provide confidence that a set of scenarios and requirements is reasonably complete.

The Way Forward with Scenario Walkthroughs

This case study helped show positive effectiveness that CREWS-SAVRE scenario walkthroughs has on a large-scale project. Important findings were captured, analysed, and implemented to improve the use of scenario walkthroughs.

From this case study and follow-on analysis work, the authors of the CREWS-SAVRE scenario walkthrough process have recommended five key lessons for using scenarios and use cases to discover requirements from stakeholders. These, along with other research findings, are currently being implemented in the next version of the process (Mavin and Maiden 2003).

• Make your walkthroughs domain-specific
• Impose structured walkthroughs
• Stakeholders own the scenarios
• Encourage exploration with scenarios
• Combine different walkthrough styles.

Work on CREWS-SAVRE's processes and tools is continuing. Details of the latest developments can be seen at www.soi.city.ac.uk/artscene.

KEYWORDS

Alternatives

CREWS-SAVRE

Exceptions

Requirement Elicitation

Requirement Validation

Scenarios

Scenario Walkthroughs

Use Cases

REFERENCES

Ben Achour, C., Rolland, C., Maiden, N.A.M., and Souveyet, C., Natural language studies on use case authoring, Proceedings of the 4th IEEE Symposium on Requirements Engineering, IEEE Computer Society Press, 1999, pp. 36–43.

Cameron, E., Facilitation Made Easy, Kogan Page Limited, 2003.

Cockburn, A., Writing Effective Use Cases, Addison-Wesley, 2001.

Jacobson, I., Booch, G., and Rumbaugh, J., The Unified Software Development Process, Addison-Wesley Longman, 2000.

Limbachia, N., Scenario-based Requirements Engineering at Eurocontrol, Final Year Project Thesis, City University, 2002

Maiden, N.A.M., SAVRE: Scenarios for acquiring and validating requirements, Journal for Automated Software Engineering, 5, 419–446, 1998.

Maiden, N.A.M. and Jones, S., An Integrated User-Centred Requirements Engineering Process, Version 4, RESCUE Process Document, 2003.

Maiden, N.A.M. and Mavin, A., Determining socio-technical systems requirements: experiences with generating and walking through scenarios, RE '03 Conference, Monterey CA, IEEE Computer Society Press, 2003.

Maiden, N.A.M., Jones, S., and Flynn, M., Innovative Requirements Techniques Applied to ATM (Air Traffic Management), Budapest, 23–27, 2003.

Robertson, S. and Robertson, J., Mastering the Requirements Process, Addison-Wesley, 1999.

Sutcliffe, A.G., Maiden, N.A.M., Minocha, S., and Manuel, D., Supporting scenario-based requirements engineering, IEEE Transactions on Software Engineering, 24(12), 1072–1088, 1998.

Yu, E., Towards modelling and reasoning support for early-phase requirement engineering, Proceedings of the 3^rd IEEE International Symposium on Requirements Engineering (RE '97), Annapolis, USA, 1997, pp. 226–235.