6
BRINGING KNOWLEDGE TO BEAR IN CONTEXT

CONTENT, ORGANIZATION AND ACTIVATION OF KNOWLEDGE

As the name suggests, this chapter deals with practitioners’ knowledge of the technical system. This part of knowledge factors is more or less familiar ground for decision theorists, expert system designers, and technical experts. But in the arena of cognitive analysis, knowledge factors is a broader category. It also includes the organization and application of the knowledge, that is, the extent to which the knowledge can be used flexibly in different contexts. It includes an examination of the types of processes that “call to mind” specific items of knowledge relevant to the situation at hand. In other words, the category encompasses the way that practitioners bring knowledge to bear effectively in their cognitive work. This work includes decision making and problem solving, but also what are usually considered the relatively mundane aspects of daily practice that involve application of knowledge.

To deal with each of these issues the practitioner was employing a great deal of knowledge (in fact, the description of just a few of the relevant aspects of domain knowledge important to the incident would occupy several pages). The individual actions of the practitioner can each be traced to specific knowledge about how various physiological and pharmacological systems work; the actions are grounded in knowledge. The question for us in this case is how the knowledge is organized and how effectively it is brought to bear.

Significantly, the issues in this case are not separate but interact in several ways important to the overall state of the patient. Briefly, the high glucose value indicated diabetes out of control; when the blood sugar is high, there is increased urine output as the glucose draws water into the urine. The diuretic given in the emergency room added to the creation of urine. Together, these effects create a situation in which the patient’s intravascular volume (the amount of fluid in the circulatory system) was low. The already damaged heart (prior heart attack) is also starved for oxygen (low arterial oxygen tension). The patient’s pain leads to high blood pressure, increasing the strain on the heart.

There is some evidence that the practitioner was missing or misunderstanding important features of the evolving situation. It seems (and seemed to peer experts who evaluated the incident at the time; cf., Cook et al., 1991) that the practitioner misunderstood the nature of the patient’s intravascular volume, believing the volume was high rather than low. The presence of high urine output, the previous use of a diuretic (furosemide, trade name Lasix) in the emergency room, and the high serum glucose together are indications that a patient should be treated differently than was the case here. The high glucose levels indicated a separate problem that seemed to be unappreciated by the practitioner on the scene. In retrospect, other practitioners argued that the patient probably should have received more intravenous fluid and should have been monitored using more invasive monitoring to determine when enough fluid had been given (e.g., via a catheter that goes through the heart and into the pulmonary artery).

But it is also apparent that many of the practitioner’s actions were appropriate in the context of the case as it evolved. For example, the level of oxygen in the blood was low and the anesthetist pursued several different means of increasing the blood oxygen level. Similarly, the blood pressure was high and this, too, was treated, first with nitroglycerin (which may lower the blood pressure but also can protect the heart by increasing its blood flow) and then with nifedipine. The fact that the blood pressure fell much further than intended was probably the result of depleted intravascular volume which was, in turn, the result of the high urinary output provoked by the diuretic and the high serum glucose level. Significantly, the treatment in the emergency room that preceded the operation made the situation worse rather than better.

In the opinion of anesthesiologist reviewers of this incident shortly after it occurred, the circumstances of this case should have brought to mind a series of questions about the nature of the patient’s intravascular volume. Those questions would then have prompted the use of particular monitoring techniques before and during the surgical procedure.

This incident raises a host of issues regarding how knowledge factors affect the expression of expertise and error. Bringing knowledge to bear effectively in problem solving is a process that involves:

content (what knowledge) – is the right knowledge there? is it incomplete or erroneous (i.e., “buggy”);

organization – how knowledge is organized so that relevant knowledge can be activated and used effectively; and

activation – is relevant knowledge “called to mind” in different contexts.

Much attention is lavished on content but, as this incident demonstrates, mere possession of knowledge is not expertise. Expertise involves knowledge organization and activation of knowledge in different contexts (Bransford, Sherwood, Vye, and Rieser, 1986). Moreover, it should be clear from the example that the applications of knowledge go beyond simply matching bits of knowledge to specific items in the environment. The exact circumstances of the incident were novel, in the sense that the practitioner had never seen precisely this combination of conditions together in a single patient, but we understand that human expertise involves the flexible application of knowledge not only for familiar, repetitive circumstances but also in new situations (Feltovich, Spiro and Coulson, 1989).

When analyzing the role of knowledge factors in practitioner performance, there are overlapping categories of research that can be applied. These include:

mental models and knowledge flaws (sometimes called “buggy” knowledge),

knowledge calibration,

inert knowledge, and

heuristics, simplifications, and approximations.

MENTAL MODELS AND “BUGGY” KNOWLEDGE

Knowledge of the world and its operation may be complete or incomplete. It may also be accurate or inaccurate. Practitioners can only act on the knowledge they have. The notion of a mental model, that is a mental representation of the way that the (relevant part of the) world works is now well established, even if researchers do not agree on how such a model is developed or maintained. What is clear, however, is that the function of such models is to order the knowledge of the work so as to allow the practitioner to make useful inferences about what is happening, what will happen next, and what can happen. The term mental model is particularly attractive because it acknowledges that things in the world are related, connected together in ways that interact, and that it is these interactions that are significant, rather than some isolated item of knowledge, discrete and separate from all others.

Of course the mental model a practitioner holds may be incomplete or inaccurate. Indeed, it is clear that all such models are imperfect in some ways – imprecise or, more likely, incomplete. Moreover, mental models must contain information that is nowhere in textbooks but learned and refined through experience. How long it takes the sublingual nifedipine to work, in this incident, and how to make inferences back from the occurrence of low blood pressure to the administration of the nifedipine earlier is an example. When practitioner mental models are inaccurate or incomplete they are described as “buggy” (see Gentner and Stevens, 1983; Rouse and Morris, 1986; Chi, Glaser, and Farr, 1988, for some of the basic results on mental models).

For example, Sarter and Woods (1992, 1993) found that buggy mental models contributed to problems with cockpit automation. A detailed understanding of the various modes of flight deck automation is a demanding knowledge requirement for pilots in highly automated cockpits. Buggy mental models played a role in automation surprises, cases where pilots were “surprised” by the automation’s behavior. The buggy knowledge created flaws in the understanding the automatic system’s behavior. Buggy mental models made it hard for pilots to determine what the automation was doing, why it was doing it, and what it would do next. Nearly the same problems are found in other domains, such as anesthesiologists using microcomputer-based devices (Cook, Potter, Woods, and McDonald, 1991b).

Significantly, once the possibility of buggy mental models is recognized, it is possible to design experiments that reveal specific bugs or gaps. By forcing pilots to deal with various non-normal situations in simulator studies, it was possible to reveal knowledge bugs and their consequences. It is also possible to find practitioners being sensitive to these gaps or flaws in their understanding and adapting their work routines to accommodate these flaws. In general, when people use “tried and true” methods and avoid “fancy features” of automation, we suspect that they have gaps in their models of the technology. Pilots, for example, tend to adopt and stay with a small repertoire of strategies, in part, because their knowledge about the advantages and disadvantages of the various options for different flight contexts is incomplete. But these strategies are themselves limiting. People maybe aware of flaws in their mental models and seek to avoid working in ways that will give those flaws critical importance, but unusual or novel situations may force them into these areas.

It is not clear in the incident described whether the practitioner was aware of the limitations of his mental model. He certainly did not behave as though he recognized the consequences of the interdependent facets of the problem.

Technology Change and Knowledge Factors

All of the arenas where human error is important intensively use technology. Significantly, the technological strata on which such domains are based are more or less in constant flux. In medicine, transportation and other areas, technological change is constant. This change can have important impacts on knowledge factors in a cognitive system. First, technology change can introduce substantial new knowledge requirements. For example, pilots must learn and remember the available options in new flight computers, learn and remember how to deploy them across a variety of operational circumstances – especially during the rare but difficult or critical situations, learn and remember the interface manipulations required to invoke the different modes or features, learn and remember how to interpret or where to find the various indications about which option is active or armed and the associated target values entered for each. And here by the word “remember” we mean not simply being able to demonstrate the knowledge in some formal way but rather be able to call it to mind and use it effectively in actual task contexts.

Studying practitioner interaction with devices is one method for understanding how people develop, maintain, and correct flaws in mental models. Because so much of practitioner action in mediated through devices (e.g., cockpit controls) flaws in mental models here tend to have severe consequences.

Several features of practitioner interactions with devices suggest more general activities in the acquisition and maintenance of knowledge.

1. Knowledge extension by analogy: Users transfer their mental models developed to understand past devices to present ones if the devices appear to be similar, even if the devices are internally dissimilar.

2. There is no cognitive vacuum: Users’ mental models are based on inferences derived from experience with the apparent behavior of the device, but these inferences may be flawed. Devices that are “opaque” and that give no hint about their structure and function will still be envisioned as having some internal mechanism, even if this is inaccurate. Flaws in the human-computer interface may obscure important states or events or incidentally create the appearance of linkages between events or states that are not in fact linked. These will contribute to buggy mental models of device function.

3. Each experience is an experiment: Practitioners use experience with devices to revise their models of device operations. They may do this actively, by deliberately experimenting with ways of using the device or passively by following the behavior of the device over time and making inferences about its function. People are particularly sensitive to apparent departures from what “normal”.

4. Hidden complexity is treated as simplicity: Devices that are internally complex but superficially simple encourage practitioners to adopt overly simplistic models of device operation and to develop high confidence that these models are accurate and reliable.

Device knowledge is a large and readily identified area where knowledge defects can be detected and described. But characteristics of practitioner interaction with devices have parallels in the larger domain. Thus, the kinds of behaviors observed with devices are also observed in use of knowledge more generally.

Knowledge Calibration

Closely related to the last point above are results from several studies (Sarter and Woods, 1993; Cook et al., 1991; Moll van Charante et al., 1993) indicating that practitioners are often unaware of gaps or bugs in their mental models. This lack of awareness of flaws in knowledge broadly the issue of knowledge calibration (e.g., Wagenaar and Keren, 1986).

Put most simply, individuals are well calibrated if they are aware of the accuracy, completeness, limits, and boundaries of their knowledge, i.e., how well they know what they know. People are miscalibrated if they are overconfident (or much less commonly underconfident) about the accuracy and compass of their knowledge. Note that degree of calibration is not the same thing as expertise; people can be experts in part because they are well calibrated about where their knowledge is robust and where it is not.

There are several factors that can contribute to miscalibration. First, the complexity of practice means that areas of incomplete or buggy knowledge can remain hidden from practitioners for long periods. Practitioners develop habitual patterns of activity that become well practiced and are well understood. But practitioners may be unaware that their knowledge outside these frequently used regions is severely flawed simply because they never have occasion to need this knowledge and so never have experience with its inaccuracy or limitations. Practitioners may be able to arrange their work so that situations which challenge their mental models or confront their knowledge are limited. Second, studies of calibration indicate that the availability of feedback, the form of feedback and the attentional demands of processing feedback, can affect knowledge calibration (e.g., Wagenaar and Keren, 1986). Even though flaws in practitioner knowledge are being made apparent by the failure, so much attention may be directed to coping with failure that the practitioner is unable to recognize that his or her knowledge is buggy and so recalibration never occurs.

Problems with knowledge calibration, rather than simply with lack of knowledge, may pose substantial operational hazards. Poor calibration is subtle and difficult for individuals to detect because they are, by definition, unaware that it exists.

Avoiding miscalibration requires that information about the nature of the bugs and gaps in mental models be made apparent through feedback. Conversely, systems where feedback is poor have a high propensity for maintaining miscalibrated practitioners. A relationship between poor feedback and miscalibrated practitioners was found in studies of pilot-automation interaction (Sarter and Woods, 1993) and of physician-automation interaction (Cook and Woods, 1996b). For example, some of the participants in the former study made comments in the post-scenario debriefings such as: “I never knew that I did not know this. I just never thought about this situation.” Although this phenomenon is most easily demonstrated when practitioners attempt to use computerized devices because such devices so often are designed with opaque interfaces, it is ubiquitous.

Knowledge miscalibration is especially important in the discussion of error. Failures that occur in part because of miscalibration are likely to be reported as other sorts of failures; the absent knowledge stays absent and unregarded. Thus problems related to, for example, poorly designed devices go unrecognized. Significantly, the ability to adequately reconstruct and examine the sequence of events following accidents is impaired: the necessary knowledge is absent but those involved in the accident are unaware of this absence and will seek explanations from other sources.

Activating Relevant Knowledge in Context: The Problem of Inert Knowledge

A more subtle form of knowledge problem is that of inert knowledge, that is knowledge that is not accessed and remains unused in important work contexts. This problem may play a role in incidents where practitioners know the individual pieces of knowledge needed to build a solution but are unable to join the pieces together because they have not confronted the need previously. (Note that inert knowledge is a concept that overlaps both knowledge and attention in that it refers to knowledge that is present in some form but not activated in the appropriate situation. The interaction of the three cognitive factors is the norm.) Thus, the practitioner in the first incident could be said to know about the relationship between blood glucose, furosemide, urine output, and intravascular volume but also not to know about that relationship in the sense that the knowledge was not activated at the time when it would have been useful. The same pattern can occur with computer aids and automation. For example, some pilots were unable to apply knowledge of automation successfully in an actual flight context despite the fact that they clearly possessed the knowledge as demonstrated by debriefing, that is, their knowledge was inert (Sarter and Woods, 1993).

We tend to assume that if a person can be shown to possess a piece of knowledge in one situation and context, then this knowledge should be accessible under all conditions where it might be useful. But there are a variety of factors that affect the activation and use of relevant knowledge in the actual problem solving context (e.g., Bransford et al., 1986). But it is clear that practitioners may experience dissociation effects where the retrieval of knowledge depends on contextual cues (Gentner and Stevens, 1983; Perkins and Martin, 1986). This may well have been the case in the first incident. During later discussion, the practitioner was able to explain the relationship between the urine output, hyperglycemia, diuretic drugs, and intravascular volume and in that sense possessed the relevant knowledge, but this knowledge was not summoned up during the incident.

Results from accident investigations often show that the people involved did not call to mind all the relevant knowledge during the incident although they “knew” and recognized the significance of the knowledge afterwards. The triggering of a knowledge item X may depend on subtle pattern recognition factors that are not present in every case where X is relevant. Alternatively, that triggering may depend critically on having sufficient time to process all the available stimuli in order to extract the pattern. This may explain the difficulty practitioners have in “seeing” the relevant details when the pace of activity is high and there are multiple demands on the practitioner. These circumstances are typical of systems “at the edge of the performance envelope.”

The problem of inert knowledge is especially troubling because it is so difficult to determine beforehand all the situations in which specific knowledge needs to be called to mind and employed. Instead, we rely on relatively static recitals of knowledge (e.g., written or oral examinations) as demonstrations of practitioner knowledge. From a cognitive analysis perspective, what is critical is to show that the problem solver can and does access situation-relevant knowledge under the conditions in which tasks are performed.

Oversimplifications

One means for coping with complexity is the use of simplifying heuristics. Heuristics amount to cognitive “rules of thumb”, that is approximations or simplifications that are easier to apply than more formal decision rules. Heuristics are useful because they are easy to apply and minimize the cognitive effort required to produce decisions. Whether they produce desirable results depends how well they work, that is, how satisfactorily they allow practitioners to produce good cognitive performance over a variety of problem demand factors (Woods, 1988). In all cases heuristics are to some degree distortions or misconceptions – if they were not, they would not be heuristics but rather robust rules. It is possible for heuristics that appear to work satisfactorily under some conditions to produce “error” in others. Such heuristics amount to “oversimplifications.”

In studying the acquisition and representation of complex concepts in biomedicine, Feltovich et al. (1989) found that some medical students (and even by some practicing physicians) applied knowledge to certain problems in ways that amounted to oversimplification. They found that “bits and pieces of knowledge, in themselves sometimes correct, sometimes partly wrong in aspects, or sometimes absent in critical places, interact with each other to create large-scale and robust misconceptions” (Feltovich et al., 1989, p. 162). Broadly, oversimplifications take on several different forms (see Feltovich, Spiro, and Coulson, 1993):

seeing different entities as more similar than they actually are,

treating dynamic phenomena statically,

assuming that some general principle accounts for all of a phenomenon,

treating multidimensional phenomena as unidimensional or according to a subset of the dimensions,

treating continuous variables as discrete,

treating highly interconnected concepts as separable,

treating the whole as merely the sum of its parts.

Feltovich and his colleagues’ work has important implications for the teaching and training. In particular, it challenges what might be called the “building block” view of learning where initially lessons present simplified material in modules that decompose complex concepts into their simpler components with the belief that these will eventually “add up” for the advanced learner (Feltovich et al., 1993). Instructional analogies, while serving to convey certain aspects of a complex phenomenon, may miss some crucial ones and mislead on others. The analytic decomposition misrepresents concepts that have interactions among variables. The conventional approach can produce a false sense of understanding and inhibit pursuit of deeper understanding. Learners resist learning a more complex model once they already have an apparently useful simpler one (Spiro et al., 1988).

But the more basic question associated with oversimplification remains unanswered. Why do practitioners utilize simplified or oversimplified knowledge at all? Why don’t practitioners use formal rules based, for example, on Bayesian decision theoretical reasoning? The answer is that the simplifications offered by heuristics reduce the cognitive effort required in demanding circumstances.

It is easier to think that all instances of the same nominal concept … are the same or bear considerable similarity. It is easier to represent continuities in terms of components and steps. It is easier to deal with a single principle from which an entire complex phenomenon “spins out” than to deal with numerous, more localized principles and their interactions. (Feltovich et al., 1989, p. 131)

This actually understates the value of heuristics. In some cases, it is apparent that the heuristics produce better decision making over time than the formally “correct” processes of decision making. The effort required to follow more “ideal” reasoning paths may be so large that it would keep practitioners from acting with the speed demanded in actual environments. When the effort required to reach a decision is included and the amount of resource that can be devoted to decision making is limited (as it is in real world settings), heuristics can actually be superior to formal rule following. Payne, Bettman, and Johnson (1988) and Payne, Johnson, Bettman, and Coupey (1990) demonstrated that simplified methods produce a higher proportion of correct choices between multiple alternatives under conditions of time pressure than do formal Bayesian approaches that require calculation. Looking at a single instance of failure may lead us to conclude that the practitioner made an “error” because he or she did not apply an available, robust decision rule. But the error may actually be ours rather than the practitioners when we fail to recognize that using such (effortful) procedures in all cases will actually lead to a greater number of failures than application of the heuristic!

There is a more serious problem with an oversimplified view of oversimplification by practitioners. This is our limited ability to account for uncertainties, imprecision, or conflicts that need to be resolved in individual cases. In the incident, for example, there are conflicts between the need to keep the blood pressure high and the need to keep the blood pressure low. As is often the case in this and similar domains, the locus of conflict may vary from case to case and from moment to moment. The heart depends on blood pressure for its own blood supply, but increasing the blood pressure also increases the work it is required to perform. The practitioner must decide what blood pressure is acceptable. Many factors enter into this decision process. For example, how precisely can we predict the future blood pressure? How will attempts to reduce blood pressure affect other physiological variables? How is the pressure likely to change without therapy? How long will the surgery last? Will changes in the blood pressure impact other systems (e.g., the brain)? Only in the world of the classroom (or courtroom) can such questions be regarded as answered in practice because they can be answered in principle. The complexity of real practice means that virtually all approaches will appear, when viewed from a decision theoretical perspective, to be oversimplifications – it is a practical impossibility before the fact to produce exhaustively complete and robust rules for performance. (The marked failure of computer based decision tools to handle cases such as the incident presented in this section is evidence, if more were needed, about the futility of searching for a sufficiently rich and complicated set of formal rules to define “good” practice.)

In summary, heuristics represent effective and necessary adaptations to the demands of real workplaces (Rasmussen, 1986). When post-incident cognitive analysis points to practitioner oversimplification we need to examine more than the individual incident in order to determine whether decision making was flawed. We cannot simply point to an available formal decision rule and claim that the “error” was the failure to apply this (now apparently) important formal decision rule. The problem is not per se that practitioners use shortcuts or simplifications, but that their limitations and deficiencies were not apparent. Cognitive analysis of knowledge factors therefore is extended examination of the ways practitioners recognize situations where specific simplifications are no longer relevant, and when (and how) they know to shift to using more complex concepts, methods, or models.

ANALYZING THE COGNITIVE PERFORMANCE OF PRACTITIONERS FOR KNOWLEDGE FACTORS

The preceding discussion has hinted at the difficulty we face when trying to determine how buggy mental models, oversimplifications, inert knowledge, or some combination contributed to an incident. The kinds of data available about the incident evolution, the knowledge factors for the specific practitioners involved in the incident, the knowledge factors in the practitioner population in general, are critical to our understanding of the human performance in the incident. These sorts of high precision data are rarely available without special effort from investigators and researchers. The combination of factors present in Incident 1 was unusual, and this raises suspicion that a buggy mental model of the relationship between these factors played a major role. But the other characteristic flaws that can occur under the heading of knowledge factors are also likely candidates.

Given the complexities of the case, oversimplification strategies could be implicated. The congestive heart failure is usually associated with increased circulating blood volume and the condition is improved by diuretic therapy. But in this case high blood glucose was already acting as a diuretic and the addition of the diuretic drug furosemide (which occurred in the emergency room before the anesthesia practitioner had contact with the patient) probably created a situation of relative hypovolemia, that is too little rather than too much. The significance of the earlier diuretic in combination with the diabetes was missed, and the practitioner was unable to recognize how this situation varied from typical for congestive heart failure.

Inert knowledge may have played a role as well. The cues in this case were not the ones that are usually associated with deeper knowledge about the inter-relationships of intravascular volume, glucose level, and cardiovascular volume. The need to pay attention to the patient’s low oxygen saturation and other abnormal conditions may well have contributed to making some important knowledge inert.

Beyond being critical of practitioner performance from afar, we might ask how the practitioners themselves view this sort of incident. How clearly does our cognitive analysis correspond to their own understanding of human performance. Interestingly, practitioners are acutely aware of how deficient their rules of thumb may be, how susceptible to failure are the simplifications they use to achieve efficient performance. Practitioners are actually aware that certain situations may require abandoning a cognitively less effortful approach in favor of more cognitively demanding “deep thinking.” For example, senior anesthesiologists commenting on the first incident shortly after it occurred were critical of practitioner behavior:

This man was in major sort of hyperglycemia and with popping in extra Lasix [furosemide] you have a risk of hypovolemia from that situation. I don’t understand why that was quietly passed over, I mean that was a major emergency in itself … this is a complete garbage amount of treatment coming in from each side, responding from the gut to each little bit of stuff [but it] adds up to no logic whatsoever … the thing is that this patient [had] an enormous number of medical problems going on which have been simply reported [but] haven’t really been addressed.

This is a pointed remark, made directly to the participant in a large meeting by those with whom he worked each day. While it is not couched in the language of cognitive science, it remains a graphic reminder that practitioners recognize the importance of cognition to their success and sometimes distinguish between expert and inexpert performance by looking for evidence of cognitive processes.