10
Design of Cooperative Systems in Complex Dynamic Environments
Conservatoire National des Arts et Métiers
Conservatoire National des Arts et Métiers
We describe a general method we use to design cooperative environments in real situations. We develop the notion of environmental resources and show how cognitive simulation can be used to design and test the new working environments. This general approach is centered around two work situations (air traffic control and emergency center) where cooperation can be viewed as the key point for task accomplishment.
INTRODUCTION
Understanding cooperative activity and developing methodologies to design and test cooperative environments (groupware, cooperative expert systems, team behavior) is crucial today in the field of cognitive engineering. There are numerous obstacles to an understanding of cooperative activity: Cognitive processes underlying cooperation are difficult to directly investigate (i.e., mechanisms of intent recognition, mutual knowledge, recursivity of mutual representations, modal knowledge, multimodality in speech acts, difficulty in formalizing collaboration procedure, and so on). From a designer or human factor point of view, the difficulties are even greater because it is not enough to understand cognitive processes, it is also necessary to be able to predict new behaviors in relation to the “to be designed” environment.
The design process must also include analysis of “borderline situations” where operators are in exceptional situations such as high workload, incidental or stressful contexts. We know that in such situations, operators not only change their cognitive strategies (through cognitive regulations) but they also may not use the same resources from their working environment.
To be really operational, the cognitive modeling of working activities must incorporate some of these factors. The models we are working with must explain how, why, and when behavior changes when operators are dealing with degraded situations or when the designer changes some aspects of functional resources. The current trend in cognitive modeling is to favor “internal” models where representations and control structures are independent of external factors (i.e., GOMS model, CLG, TAG, ACT, SOAR.). Although the advantage is clear cut (gain in generality), the approach is detrimental from the designer’s point of view because it is extremely difficult to take into account real environmental constraints and hence predict realistic behavior. This difficulty may not only depend on the soundness of cognitive models but also on theoretical frames : in real work situations the cognitive representations used by operators are often associated with the external devices that constitute the whole environment. For example, Hutchins (1991) showed how memory processes in a cockpit are dynamically distributed among human agents and external representational devices. The difficulty of making a clear distinction between representations that are internal to the operator and those that are deeply rooted in the environment has been shown in other domains such as word processing (Pavard, 1987; Scavetta & Pavard, 1989), music composition (Marmaras, Pavard, & Xanthoudakis, 1987) and aircraft radar control (Bressolle, 1992).
Our approach will start with this premise and will address the issue of how to articulate these “internal” models of cognition with external constraints (work environment). In other words, are we able to handle the vicariance of human behavior at the level of the internal models (the current tendency) alone or do we have to work out a “pragmatic layer” (by analogy with the linguistic domain) whose purpose is to explain how external functional resources are incorporated and operationalized in the operator’s mental representation? We will show how cognitive simulation appears to be an efficient paradigm to 1) combine in a unified formalism internal and external representations and 2) model situations where complexity (representational and operational) cannot be localized at the individual level but rather is distributed over several interacting entities (agents or devices). A case study where complex interactions arise between operators and an environment and where human behavior is highly dependent on environmental resources is illustrative of this issue and will be developed.
COOPERATION AS A COGNITIVE REGULATION MECHANISM
Cooperation is a special feature of collective activity oriented towards a specific goal. It can take on many forms (Zachary & Robertson, 1990). For example:
- agents can share goals and accomplish them without any cooperative activity,
- agents can negotiate their contributions to cooperative activity,
- agents can have different goals and still cooperate,
- agents may have coordinated activity without any cooperative goals and so on.
These different forms of cooperation are not mutually exclusive but they may alternate during the course of a dialogue.
From the modeling (internal) point of view, communication can be seen as a resource allocation problem. The basic mechanisms can be stated as follows:
- agents are resource limited,
- having limited resources and goals to fulfill, they must allocate cognitive resources to what they estimate to the most efficient activity (perception, problem solving, hypothesis evaluation, situation assessment).
- cooperation between agents can be seen as an answer to the resource limitation problem but cooperative actions are also resource consuming and must be evaluated before being initiated.
This approach, if well formalized, can yield a dynamic view of human cognition and is a classical frame to analyze specific problems such as relationship between cognition and workload or modeling of human errors (Cacciabue & Kjaer-Hansen, 1992; Woods, Pople, & Roth, 1990).
In order to be more predictive in a realistic context, models need to include more information about the trade-off between the “advantages” and “cost” of cooperation such as :
- cooperation (by sharing goals) implies mutual knowledge but mutual knowledge is resource consuming (for example it requires intention perception),
- cooperation calls for availability from colleagues but availability depends on workload.
As we will see from the work analysis, these regulations are highly dependent on the characteristics of the work environment, and the theoretical issue is where, in the general architecture of models, do we implement these kinds of “interactions” between external constraints and cognitive regulations which are decisive to the final operationality of human environment interaction.
In the next section, we describe the notion of environmental resources in two examples.
ENVIRONMENTAL RESOURCES
Environmental resources are defined as all the potential resources (information, informational supports, etc.) that are available in the operator’s environment that can be used to fulfill a task. Different environmental resources may be used at different times during the work process.
In the domain of air traffic control, for example, environmental resources are very dense and at times hard to identify exhaustively. The basic tools an air traffic controller uses are the radar screen, where he can visualize the position and speed of flights and flight strips (small pieces of paper on which aircraft characteristics and destinations are written). These strips in fact are used to code much more information than the basic information printed on the paper (Shapiro, Hughes, Randall, & Harper, 1991). For example, controllers can actively organize these strips on the flight progress board to spatially code dynamic information such as vertical speed, position, abnormal flight, as well as the state of problem solving (what has been done and what remains to be done) (Figure 1). The resources specifically associated with the flight progress board can be defined as the mobility, writability of strips, visual accessibility of this information by agents, and gestual modality in strip transmission between controllers (new information or abnormal information can be expressed by controllers through gesture).
FIGURE 1. Example of environmental resources provided by air traffic control environment: the flight progress board for air traffic controllers
The positioning of paper strips on the board is a prime means of memorizing the dynamic state of the world. The progress board is also extensively used by controllers to update their mutual knowledge.
The following example illustrates the role of active organization of the strips on the flight progress board (Figure 2). In what is called a terminal sector (in the vicinity of an airport area) two different kind of spatial organization of strips can be found. The first orders the strips as a function of category of flight (towards the airport area or leaving the airport area - even if the flight did not take off from this airport). Alternatively, strips can be ordered in terms of the location of the flights in the critical sector, with the airport as a reference point.
FIGURE 2. Two different external representations used by air traffic controllers
As a function of traffic complexity and environmental constraints, the controller can use a propositional style of external representation (left) or a more analogical, topographical oriented organization of the progress board (right).
In some situations the inherent complexity of the flights configuration in the sector makes one type of representation predominate. However the selection of a specific external representation is highly dependent on contextual features of the situation. For example, even though traffic complexity is not solely determined by the number of flights at hand, this factor may constrain the choice of representation because of space on the flight board: Too many strips at one time may make the use of a topographic representation unfeasible. In other words the controller uses the flight board as an external memory and can choose the most relevant organization and information format at a time, but contextual feature elements may highly restrain the span of this choice (Figure 3). Furthermore, all this strip information is also used to update mutual knowledge (Bressole, 1992) which is a prerequisite for an efficient cooperation.
As a second example, take the cooperation between agents situated close together in a task of cooperation in an emergency room (Pavard, Benchekroun, & Salembier, 1990). A detailed work analysis showed that in this situation, cooperation is extremely dependent on the notion of agent mutual knowledge and mutual knowledge is made possible in this case by environmental resources such as “proximity of agents,” and “possibility of direct visual contact.” Proximity of agents makes passive and active information acquisition possible. Information acquired in this case can be used later for cooperation. Possibility of direct visual contact is also one of the most important factors in conversation regulation: Agents can listen to all communication (when they are not themselves busy) and they can adjust the strength (illocutionary force) of their request in relation to the estimated (by visual perception) urgency of the request.
FIGURE 3. Effect of traffic complexity and number of flights to control at a given time on the selection of relevant external representation by air traffic controllers.
The controller’s choice of representation may vary as a function of changes in context.
ENVIRONMENTAL RESOURCES AND THE DESIGN OF A NEW WORKING ENVIRONMENT
Changing the working environment requires being extremely aware of the consequences of the new environmental resources on the organization of work, problem solving strategies, management of risks, and so on. For example, as shown below, introducing an electronic mail system in a group of cooperating people (even if this new tool is merely added to the previous situation) can radically alter communication regulation strategies and how the work is really done. Similarly in air traffic control, going from paper strip to electronic display cannot be without extensive investigation of how it will affect the resources implemented in the traditional system.
Another type of difficulty arises from the fact that working conditions usually evolve over time. Workload may increase for a short period of time (burst of external calls in an emergency center) or may start to evolve in an unexpected way (abnormal air-flight trajectory). To handle such situations operators change their cognitive strategies (or regulate their activities). These regulations have been extensively studied in domains such as air traffic control (Sperandio, 1972) or aircraft piloting (Hutchins, 1992). The main point is that in these situations, operators or cooperating agents may use different environmental resources to fulfill their tasks.
At this point, it is important to realize that without specific work analysis on critical situations such as high workload or unexpected event scenarios, it is very difficult to assess all the resources an environment provides1 and, furthermore, it would be difficult to anticipate how a new working environment could provide the required functionalities. The methodology we propose has been elaborated through different complex interface design projects that attempt to overcome the difficulties described earlier. It is mainly based on the identification of environmental resources, the understanding of how operators use these resources both in nominal and exceptional situations, and finally, the modification of the environment and a simulation of cognitive regulation mechanisms.
Methodology
From a practical point of view, our methodology is based on the following steps:
Work analysis
Identification of the cognitive regulation mechanisms in the situation. Cognitive regulation mechanisms arise because operators have limited cognitive resources (memorization, attention, etc.). Therefore, they must adapt (regulate) their cognitive mechanisms to allocate their resources in an optimal way, which depends on many factors such as previous experience, knowledge of interface functionalities, environmental factors, and so on.
Cognitive regulation modeling on a rule-based system
From previous work analyses, a collection of typical scenarios are identified and stored (typical scenarios are segments of activities, usually a few seconds or a few minutes, that characterize how operators behave in response to particular tasks or outside disturbances). Strategies observed on the previous step are formalized taking into account the role of environmental factors. The model is based on speech act theory (to handle the pragmatic components of communication in relation to environment resources) and on information processing theories (to handle cognitive resource limitation constraints).
Confrontation with video scenarios
We test the model with previously monitored scenarios. The model must be able to describe the evolution of cognitive strategies in relation to the different environmental constraints (workload, stress, etc.).
Simulation of what the scenario would become in the new situation.
Rules are modified in order to take into account the new working environment and, with the same input (external events, communication intentions, etc.), we visualize what the initial scenarios would become.
A CASE STUDY OF COOPERATIVE ACTIVITY: COMMUNICATION REGULATION IN AN EMERGENCY CENTER
The emergency center (SAMU 91) is in charge of all medical emergency calls in the Southern Paris area (one million people). Currently the emergency team is made up of two operators (nurses) and two physicians. The task, referred to as “phone call regulation” consists in assessing the level of emergency of phone calls, diagnosis, and decision making about sending the right medical help (counseling, ambulance, etc.). This task is a complex one because it implies decision making under uncertainty and time constraints, management of ambulance resources, up-date of log books, management of patient stress, and so on. The object of the study was the design of a cooperative computer tool (groupware) that would improve communication efficiency between the members of the emergency team, both in normal and overload situations (peaks of emergency calls either due to statistical phenomena or accidents involving many patients).
Work analysis and identification of regulation mechanisms through cooperation
During an emergency call, more than one agent may have to cooperate, either due to simple information requests or a process of distributed diagnosis between operators and physicians. This collaboration process is usually facilitated by collaboration awareness within the group: Each agent tries to be aware of all ongoing events, even if these are managed by colleagues.
This organization makes cooperation at a high cognitive level possible (shared cognition) where any agent can handle a phone conversation, even if it is not initiated by him/her. Several other regulation mechanisms have been identified (Pavard, Benchekroun, & Salembier, 1990) and we focus on two because of their stability over many situations.
Communication regulation through less constrained agents (operators or physicians)
The task requires that any phone call must be answered as soon as possible. Three configurations are possible :
a) the operator is available: He/she will take the phone call.
b) the operator is not available but one of his/her colleagues is available and will take the phone call.
c) no operator is available: In this degraded mode, operators use a regulation mechanism based on an evaluation of the level of emergency of all their phone calls. Less urgent calls are put on a waiting list.
Communication regulation through turn-taking protocols.
Conversational analysis has clearly pointed to the importance of turn-taking in ordinary conversation. The key notion is that conversation between people is structured by alternating dialogues which are controlled by social and organizational rules (Procter, 1991). Turn-taking not only reflects the structure of group communication but also helps sustain it. In our situation, turn-taking plays an important role because it is the process by which cognition is distributed through agents. The particularity of this mode of communication is that responsibility for organizing shared cognition is distributed across participants and not controlled by any external structure.
Communication Modeling
The theoretical framework for interpreting communication is inspired by the theory of speech acts (Searle, 1969), the theory of intentionality (Searle, 1985) and the formal model of illocutionary acts of Allen, Cohen, and Perrault (Cohen & Perrault, 1979). In this approach, the action of talking corresponds not only to the desire to transmit information but also to modify the universe of thought of the interlocutor (non-explicit). Communication can be categorized into three parts:
- the preparation of the request (illocutionary act): A needs a certain piece of information, and he believes that B has it. A will then prepare his request according to what he knows about B (i.e., B knows it, now, not busy, and so on),
- the formulation of the request (locutionary act): The request itself, that is the words employed to convey the message,
- the impact of the request on the interlocutor (perlocutionary act): B hears the A’s request, interprets it, and eventually responds. For personal reasons B may decide not to satisfy A’s request. In this case, the communicative act intended by A is either an explicit failure (i.e., B disregards the information completely) or will have an impact in the future (i.e., the locutionary act affects B’s universe of thought).
This theoretical framework is nevertheless not sufficient to formalize communication in real situations because, as we pointed out for work analysis, speech acts are preceded by a turn-taking process where the listener’s availability is assessed. Availability is evaluated through many non verbal channels such as gesture, posture, facial expressions, and so on. These channels are supported by what we call “environment resources” and constitute what is called elsewhere “collective space” (Procter, 1991) and are responsible for the “social presence” by which members of the group signal their intentions, monitor, detect, and resolve ambiguities and conflicts. As a function of this evaluation, the speech act will be either canceled or reinforced.
Three situations can then occur:
- B is perceived as available, then A will perform the speech act.
- B is not perceived as available, then A can delay his request or eventually increase his illocutionary strength in order to interrupt B’s conversation.
- A’s judgment of B’s availability is erroneous and the communication process is canceled.
As an example, we take the situation where operator A wants to make a request to operator B about proposal P:
P : “Which physician is working next Sunday ?”
To formalize the communication process, we use the following rules :
illoc_rule :
want (A, request (A,B,P,Level_i)),
believe (A,availability (B,Level_j)),
Level_i>Level_j,
retract (want (A,request (A,B,P,Level_i))),
assserta (request(A,B,P,Level_i))).
This illocutionnary rule states that if A wants to make a request to operator B about the proposal P (A supposes that B has the information), and if the degree of B’s availability is greater than the emergency of the request, then A will transform his intention to communicate into a speech act.
Another rule is necessary to make a distinction between illocutionary act and its perlocutary effects, as in the Scarle model.
perloc_rule_1 :-
request (A,B,P,Level_i))),
availability (B,Level_j)),
Level_i>Level_j,
assserta(believe (B,A,request (A,B,P,Level_i))).
The perlocutary effect is bound by a time constraint: Listener B must allocate time to receive the verbal message from A (not shown in this example for reasons of simplicity). This duration is picked up from real scenarios monitored during work analysis. The recognition by B of A’s intention is not enough to complete the communication process: B must also assess the request in the context of his own activity. If the request is taken into account, listener B will answer. For this purpose, he will again allocate time to answer and interrupt the current task in order to pursue it later:
perloc_rule_2 :-
believe (B,A,request(A,B,P,Level_i)),
availability (B_Level_j)),
Level_i>Level_j,
interrupt_current_task (B),
want (B,inform (B,A,Answer,Level_i)).
Communication rules are written in a formal language (Prolog in this case) from the results of work analysis and tested in relation to the typical scenarios we identified from the work analysis. The second step in the modeling process consists in the specification of the new rules related to the electronic mail environment. It is then possible to “replay” the original scenarios in relationship to the new communication rules (Benchekroun et al., 1990).
Comparison with video scenarios
Real communication scenarios are interpreted in terms of the model described above. In the following example (Figure 4), many communication intentions are not followed by speech acts (see broken arrows in Figure 5).
For example, Ph gets an outside phone call at time 1 that should be handled by operator P15 (personal call).
P15 already has an interlocutor.
Ph tries to inform him unsuccessfully.
Ph must then allocate cognitive resources to pick up the end of P15’s conversation if he wants to transfer this personal phone call to him.
Two other cases of unsuccessful communication arise in this example:
- P15 tries to interrupt R15 at time 3’ and must wait till the end of R15’s communication (time 5).(who speaks to whom) is distributed over the different
- Between Ph and R15 at time 7.
In this case, Ph decides to transfer the phone call to P15 instead of R15. operators and rule
All these communication regulations are handled well by our model provided that there is a good description of phone call priorities (depending on their degree of emergency) and provided that operators clearly manifest their availability in relation to the degree of emergency of the calls they are handling (Grice’s sincerity axiom). As mentioned before, in these situations, the control of communication governed. Communication patterns and the global efficiency of communication in the group is very sensitive to environmental resources such as the possibility to visually assess interlocutor availability, the possibility of “staying in the loop” by listening to other conversations, and so on.
FIGURE 4. Real communication between three agents (R15 and RH are physicians, P15 and PH are operators, SP is an external call).
Verbal communications are represented by full arrows, whiie intentions of communication (i.e., which are not followed by a verbalization) by broken arrows. External calls are shown as telephone icons. Shaded areas indicate temporal discontinuity in communications.
FIGURE 5. The simulation of the communication scenario shown in Figure 4 in the new environment (hand written graphic E-mail).
Groupware communications are represented by double arrows. Intentions of communication is always completed; in this case the interlocutor is busy; a message is left in the listener’s E-mail box.
In the next section, we will show how a computational model of communication can be used to predict how a given communication pattern can be improved by introducing new groupware.
Simulation of communication in the new working environment
The rational principles behind the groupware we designed are based on the fact that the communication rules for turn-taking used in normal situations stay unchanged during a peak of activity (when many phone calls arrive during a short period of time): Operators try to interrupt each other, looking for cooperation, but, as everybody struggles to handle their own phone calls, they are no longer ready to cooperate (Salembier, Pavard, Benchekroun, de Medeiros, & Denier, 1992). The simulation of this process is shown in Figure 5.
Due to a burst of phone calls, operators cannot transfer them immediately to the right agent.
A phone waiting list then occurs at all workstations. In order to overcome this situation, we introduced a new communication medium: a hand writing graphic network. Operators can write information manually (on a digitizing tablet) and send it to any other operator in the group. Turn-taking rules can then be based on the new information seen on the computer screen of the agent who receives the message. By this process, communication can be improved without changing the organizational principles of the team. The simulation of the impact of this new environment leads to a modification of communication rules. For example, it is necessary to add rules to modelize the situation where operator A wants to interrupt operator B who is already answering a phone call. Operator A will then decide to send operator B a message on the network :
com_rule :-
want (A, inform (A,B,P,Level_n)),
activity (B,phone,_),
send_E_mail (A,B,P).
This rule states that if the goal of operator A is to inform B of a proposal P (emergency level n) and if B is already answering a phone call, then A will send B an E-mail, whose content is P. The predicted consequences of introducing this new groupware can be seen in Figures 5 and 6 C.
Figure 5 (a real scenario), shows that P15 reads on his screen the information sent by Ph (time 3’) concerning a personal phone call that will be handled at time 4. The overall impact of the groupware can be evaluated in Figure 6 C. This last step allows for exploration of different kinds of environmental resources. For example, multimodal information interfaces can be tested virtually. In the above case, it is also possible to compare the functional operationality of audio and visual alarms when electronic messages are sent to other agents: auditory and visual cues can be hierarchized in relation to their attentional effects. Interaction between perceptual modalities can also be modeled easily (visual information can be read on the screen at the same time as a phone call is handled, but two conversations cannot take place at the same time).
FIGURE 6. Modeling of a cooperative activity in a group of agents (two in this case) answering phone calls in an emergency center.
A: A burst of phone calls rings on the same telephone line
B: In the present environment (without electronic tools), there is an accumulation of calls because the operator cannot transmit phone calls to the other agents (they are supposedly busy themselves).
C: Simulation of the previous scenario within the new environment (with voice, visual, and electronic communication media).
CONCLUSION
This chapter addresses the issue of using cognitive simulation as a tool to anticipate human-environment interactions when we design new working situations. In the context of cooperative activities, cognitive information processing is tightly connected to functional environmental resources.
Thus, the first priority is to have a model of how internal processes are “tuned” in relationship to functional properties of the environment and the task at hand (what could be called the ecological dimension of the cognitive simulation).
Too much emphasis on the “internal cognitive processes” may lead to difficulties in generalizing observations to a new situation. Cognitive simulation emerges as a good paradigm to handle operator knowledge not only at the individual level but also for interactions between operators or between operators and the environment.
REFERENCES
Benchekroun, T.H., Pavard, B., Salembier, P. & de Medeiros, E. (1990, October). Analyse et modélisation des interactions dans un centre de régulation des appels d’urgence. Paper presented at the XXVIème congrès de la SELF. Montréal.
Bressole, M.C. (1992). Intention perception and cooperation in the process of air traffic regulation. Unpublished master’s thesis, University of Toulouse le Mirail.
Cacciabue, P.C., & Kjaer-Hansen, J. (1992). Cognitive modelling and human machine interactions in dynamic environments. Le Travail Humain., 56, 1–26.
Cohen, P.R., & Perrault, C.R. (1979). Elements of a plan based theory of speech acts. Cognitive Science, 3, 3–20.
Hutchins, E. (1992) How a cockpit remembers its speed (Tech. Rep.). San Diego: University of California, Distributed Cognition Laboratory.
Marmaras, N., Pavard, B., & Xanthoudakis, H. (1987). Les changements de reprèsentation induits par les systèmes informatiques: Le cas de la synthèse musicale [Representational changes induced by computer systems]. Revue of Academy of Sciences of URSS, 8, 120–133.
Pavard, B. (1987). La conception des systèmes de traitement de texte [Text editors design]. Intellectica, 1, 37–68.
Pavard, B., Benchekroun, T.H., & Salembier, P. (1990, October). Simulation des mécanismes de régulation collective dans un environnement complexe [Simulation of collective regulation mechanisms in a complex environment]. Paper presented at Ergo-IA’90. Biarritz, France.
Procter, R. (in preparation). Supporting Distributed Cognition in Groupware.
Salembier, P., Pavard, B., Benchekroun, T.H., de Medeiros, E. & Denier, J.P. (1992). Cognitive engineering as a tool to design human-computer interfaces in complex environments. Proceedings of the International World Space Congress (pp. 75–80). Washington, DC: International Astronomical Foundation.
Scavetta, D. & Pavard, B. (1989, March) Le traitement de texte professionnel: vers un modèle cognitif intégrant les caractéristiques de l’outil de travail [Professional text processing: toward a cognitive model including working tool features]. Paper presented at Colloque sur l’Interactivité. Paris.
Searle, J.R. (1969). Speech acts. Cambridge, UK: Cambridge University Press.
Searle, J.R. (1983). Intentionality: An essay in the philosophy of mind. Cambridge: Cambridge University Press.
Shapiro, D.Z., Hughes, J.A., Randall, D., & Harper, R. (1991). Visual representation of database information: The flight data strip in air traffic control. University of Lancaster.
Spérandio, J.C. (1972). La régulation des modes opératoires en fonction de la charge de travail chez les contrôleurs de trafic aérien [Strategy regulation in relation to air traffic controllers’ workload]. Le Travail Humain, 40, 249–256.
Woods, D.D., Pople, H., & Roth, E.M. (1990). The cognitive environment simulation as a tool for modeling human performance and reliablity (Tech. Rep. No. NUREG/CR-5213). Washington, DC: U.S. Nuclear Regulatory Commission.
Zachary, W.W., & Robertson, S.P. (1990). Introduction to cognition, computation, and cooperation. In S.P. Robertson, W. Zachary, & J.B. Black (Eds.), Cognition, computing, and cooperation (pp.1–19). Norwood, NJ.: Ablex.
1Hutchins uses the concept of “opportunistic use of the structure of the environment” in a way similar to ours (Huichins, 1992).