CHAPTER 24

WHAT SCENARIOS (STILL) AREN'T GOOD FOR

Ian Alexander¹ and Neil Maiden²

¹Scenario Plus, London, UK

²Centre for HCI Design, City University, London, UK

Scenarios are extremely versatile and useful in systems development—and in other places, but they do not offer help in every situation, and may even be misleading if over-enthusiastically applied. This short chapter suggests some possible limits to scenario use—at least until we gain a better understanding. We hope this may restrain the over-enthusiastic, give readers a better feel for what scenarios really can do well, and perhaps indicate some open areas where researchers and innovative practitioners can explore and develop new techniques.

CONTINUOUS BEHAVIOUR

Scenario modelling assumes that behaviour can be divided up into meaningful episodes or chunks, such as transactions: you go and get some cash from the teller machine. But some systems provide continuous behaviour that just doesn't fall into that pattern. As Michael Jackson writes, possibly damning with faint praise,

Consider a couple of examples:

• System A is a message router for a large organisation. A huge stream of traffic of all kinds arrives from all over the world, is split into its components, and sent on to the addressees of the individual messages. Another stream travels in the other direction all the time.
• System B is a weather forecasting model. A huge mass of reports pour in from weather ships, coastal stations, and satellites. The data are fed into the model, and the system simulates the weather up to several days ahead. Forecasts are sent to customers.

Now, even in these intentionally awkward examples, you can identify more or less standard situations to consider separately as scenarios. In Systems A and B, you still need to cope with cases like failure, diagnosis, repair, test, restart, upgrade, statistics, and logging, although the main case is just one enormous routing or forecasting algorithm. In System B, you also have some stories to tell about customers who ask for rapid delivery of high definition video pictures for their TV channels, or faxed weather maps of The Lake District to print out in monochrome at a fuzzy 60 dots per inch for hillwalkers. Scenarios will therefore be of some help, but they probably cannot address the core elements of the system in any detail.

VERY LARGE SYSTEMS

Scenarios seem to be rather ineffective when applied top-down on large system hierarchies. We simply don't know how to apply scenarios consistently throughout the many years and across all the contractual boundaries of the biggest system engineering projects. It would be nice to claim that a new aircraft carrier could be specified, designed, built, and tested entirely (pace the many non-functional requirements and constraints) on the basis of scenarios, but it wouldn't be true. The huge search space implied by the set of all the end-to-end scenarios for such a system, multiplied up by all the details, exceptions, and combinations that would be created by all the subsystems and sub-subsystems, would today be utterly unmanageable. That is not even to mention the complexity of navigating the many contractual interfaces other than via traditional lists of textual requirements.

In practice (therefore), such big systems do not yet use scenarios wholeheartedly.

FRAGMENTARY MODELS

Another issue for large projects is that we do not yet have what a jet engine maker would call ‘whole engine model’ understanding. There are CAD/CAM models, static thermodynamic models, noise models, partial models of what happens in the high-pressure turbine when the pilot moves the throttle—but putting all these pieces together is another matter.

A complete system scenario model would enable us to inject any set of conditions, and get a simulated or animated set of results. Ultimately (not at the start of a project), such a model would be completely mathematical; but a complete model even at high level and expressed in text or diagrammatic form would be very useful also.

The power and convenience of a tool like David Harel and Rami Marelly's Live Sequence Chart engine as described in Come, Let's Play (Harel and Marelly 2003; see also the discussion in Chapter 1 of this volume, in the section on Simulation) for small interactive software developments might one day be available to explore all aspects of system behaviour, performance, and reliability for systems of all kinds and sizes.

EPISODIC, ALLUSORY

As hinted above, scenarios are inherently episodic and allusory. They try to grasp the essence of complex situations and systems by giving examples, by describing significant cases, and by hinting at important aspects of system behaviour. They lead to system design by induction, rather than by defining rules; they are in a word not very specification-like. Theorists like Michael Jackson who believe that systems should be specified as a whole with some kind of comprehensive model are inevitably uncomfortable with the use of scenarios as specifications: they must feel that the RUP (see Peter Haumer's account in Chapter 12) is a serious mistake, and its popularity one of the stranger vagaries of fashion.

DOMAIN-SPECIFIC?

Are scenarios domain-specific? Do you need (much) domain knowledge in scenarios? The answer may be yes, at least in Work and Business Models. If you try to proceed directly to system usage scenarios without making any such models, trouble may ensue. On the other hand, there is certainly a school of thought that scenarios are simple human things that can be described and understood regardless of the domain. Perhaps the truth is somewhere between these extremes. Scenarios do help bridge the gap between domain specialists and developers; but they may not be totally successful in this attempt.

WHICH REPRESENTATION?

If the range of approaches described in this book makes anything clear, it must be that there are many possible scenario representations. We do not today know how to select the best representation (storyboard, video, simulation, use case, user story, acting out, etc.) for a specific project. In practice, most current usage is of textual scenarios—stories or sequences of steps described in words—but it is far from certain that this is optimal in all cases.

Interestingly, there are some applicable rules from other domains. For example, psychology theory says that errors are best found when expressed in the third-person form (the operator does X, somebody else does Y, then a third person does Z) as this form results in unthreatening statements that people can evaluate more objectively.

We can at present only wonder whether there are equivalent rules governing the use of graphics, animations, and other more colourful ways of presenting scenarios. It's tempting to suppose that the use of non-technical and non-exclusively verbal scenarios might help with the specification of systems, especially where stakeholders are from widely varying backgrounds with different cultures and languages. There is some evidence that analytic models are poorly understood by non-engineers. Psychologists joke that business executives treat all diagrams as flowcharts. If there is any truth in that, there is plenty of room for research into the applicability of different scenario representations.

OPEN-ENDED

Scenarios give less guidance to developers than many more prescriptive sorts of models or ways of writing requirements. Conversely, scenarios often imply that a specific time-ordering is mandatory where a goal or task model might be vague about time constraints—or might state them explicitly (this must be synchronised with that, those can be performed in any order). When several concrete scenarios overlap, or when a Use Case documents numerous alternatives, it may be a moot point which is the ‘right’ modelling approach. Scenario use increases uncertainty by giving many choices, so people may be left not knowing what they should explore. This is so even when scenarios and Use Cases are analytic and supposedly generic. When scenarios illustrate concrete situations as examples of what a system might possibly do, different developers could take quite different solution approaches.

TACIT KNOWLEDGE

Scenarios only capture knowledge that people can consciously express, or at least work out by stepping through operational tasks with the help of a requirements engineer. Some kinds of knowledge, such as recognising faces or doing skilled work, are inherently tacit: ‘we know more than we can tell’ (Polanyi 1966, page 4). It is indeed possible to build facial recognition systems, but as Polanyi himself said, that just gives us something else to point at. We can't hope to make all tacit knowledge explicit, but ‘we can communicate … provided we are given adequate means of expressing ourselves’ (Polanyi 1966, page 5). The development of such means of expression may encompass facilitation techniques, new types of model, diagram, and animation, and may well include domain-specific techniques.

NON-FUNCTIONAL REQUIREMENTS (NFRs)

Scenarios can, with some effort (see Chapter 7, Negative Scenarios and Misuse Cases by Ian Alexander, in this volumes) be used to capture and at least indirectly document some kinds of requirements other than those directly describing system functions. But we ourselves document and indeed discover most of our NFRs by populating templates (e.g. Scenario Plus 2004, Volere 2004) and applying other conventional requirements approaches. When you know a system constraint or required quality, you simply don't need to go to the trouble of writing scenarios about it: it's quicker and more sensible just to write it as a requirement.

SUMMARY

We don't mean to sound negative—we think scenarios are wonderful and indispensable in process modelling and system development. But scenarios don't do everything—certainly not now; and there are some things they will never do. Of course all these limitations are part of the challenge for scenario researchers, and opportunities for the future.

REFERENCES

Harel, D. and Marelly, R., Come, Let's Play: Scenario-Based Programming Using LSCs and the Play-Engine, Springer-Verlag, 2003

Jackson, M., Problem Frames: Analyzing and Structuring Software Development Problems, Addison-Wesley, 2001.

Polanyi, M., The Tacit Dimension, Doubleday & Co, 1966; Reprinted Peter Smith, Gloucester, MA, 1983.

Scenario Plus: templates from website at http://www.scenarioplus.org.uk, 2004.

Volere: templates from website at http://www.volere.co.uk, 2004.