CHAPTER3
Models
This chapter presents several ways of modeling the problem and how to use these modeling frameworks:
•Dyads
•Importance of representation
•Tools for living and tools for learning
3.1DYADS
Short Definition: High-functioning AT often has two first-class users, the typical end-user (the actual person using the AT to accomplish actions) and a person in the role of a caregiver. The caregiver is needed to help configure and personalize the complex technology as well as monitor the efficacy and update the data/personalization of the application.
Longer Description: A unique aspect of software and systems for people with cognitive disabilities is that, while the focus is on the end-user, the person with cognitive disabilities, its design and evaluation must involve their caregiver. In fact, it may be taken as an axiom (Carmien, 2007; LoPresti et al., 2008) that every system is used by and must accommodate a dyad—the end user and a caregiver. In MAPS design the end user was described as a dyad, one person with two roles, the person with cognitive disabilities and a caregiver, requiring two interfaces using one set of data. Typically, the caregiver assists in the setting up and maintenance of assistive technology systems, as they often are too difficult for the person with cognitive disabilities to setup and keep up to date. Also pertinent and contributing to the success or failure of a design being adopted or abandoned are the lesser stakeholders discussed above. Often the motivation and goals of these different stakeholder roles can be divergent and even conflict (Carmien, 2011).
IT-based AT will often have two end-users. There are several reasons for this; one is that often the preparation of the underlying data is too complicated for typical end-users. Examples of this could be creating multimedia prompt sets for a MAPS like prompting system or programming a route for a guidance application; also, the user interface (in contrast to the content) may need to be customized using an interface too difficult for the end-user. Both of these are really deep customization issues (see Section 3.2). The other reason is that it is often useful and perhaps necessary to provide a monitoring or active help function to the application that will appropriately bring the caregiver into the situation; it is critical that it does not flag too many false positives as that might lead to abandonment.
The other user, the nominal “end-user,” must have an interface and set of functionalities that is appropriate to their needs. Interestingly enough, the design and implementation this part of the system is often an easier task. As a result, these types of interfaces have one database or set of static data and two quite different interfaces. In the case of Fidemaid (Carmien, 2009) (see Figure 3.2), a prototype system designed to support independence in elders in their financial activities, the caregiver interface allowed trusteed familiar caregivers, with the permission of the end-user, to monitor and set alarms for suspect transactions.
3.1.1CANONICAL PAPER
Carmien, S. and A. Kintsch (2006). Dual user interface design as key to adoption for computationally complex assistive sechnology. (Carmien and Kintsch, 2006)
3.1.2AT EXAMPLES
The MAPS system (Carmien, 2004b) is a very clear example of the dyad approach (Figure 3.1). The end-user interface was basically a multimedia player on a PDA. This was in 2004; currently, the platform would be a smartphone. The end-user (a young adult with cognitive disabilities) can start and step through uploaded scripts to complete a task. One version of the PDA prompter allowed the user to choose one of several sub-scripts at certain points in the task.
Figure 3.1: Left: the PDA prompter and controls; right: the caregiver’s script editor. From Carmien (2006b).
The primary caregiver interface supported non-commuter experts creating, editing, and reusing scripts, which are sets of sequential multimedia (pairs of an image and a verbal instruction). The caregiver interface allowed the user to load pre-segmented task templates and populate the steps with their own images and recordings of prompt text. The caregiver could store and retrieve their own photos and recordings to use (and reuse) in script composition and editing. It also supported previewing the complete script.
A secondary interface for the caregiver was implemented in a follow-up project to MAPS, LifeLine (Gorman, 2005), that used tests imbedded in the uploaded scripts that tracked the performance of the task by transmitting the status of the script to a server and under certain conditions could send a SMS to the caregiver’s cell phone.
The Fidemaid system (Carmien, 2009) was a research prototype to support elders in personal finance management. It was motivated to support elders living independently as long as possible and of the primary reasons for making the decision whether or not to move from their own home to supported living are health and medication issues and financial self management ability, as well as successful accomplishment of ADLs and IADLs (see Figure 3.2). The concept was to give them tools to easily see the status of their personal finances, both at the moment and over a span of time extending into the immediate future. Another support provided was a way to make decisions about major expenses by providing personalized comparisons between not making the purchase and the changes in discretional income that making the purchase would require. This prototype was based on research that discovered a lessening ability to make comparisons by casting both conditions into similar representations (Mather, 2006; Karlawish, 2008)
Figure 3.2: Fidemaid’s (left) end-user’s interface and (right) caregiver’s intervention triggers. From Carmien (2009).
In Fidemaid’s case the part of the system supporting dyads was the intervention trigger pane of the application setup. This allowed the end-user and caregiver to decide when and how to notify a familiar (Spanish for family based rather than professional caregiver) about potential problematic situations and incidents.
3.1.3THE ASSISTANT SYSTEM
The ASSISTANT system, as discussed in various sections to follow, was designed so that, should the route planning be too complex for the end-user, a caregiver could do the planning for them. To do this the system provided a role of a “caregiver,” or in the Spanish localized version the “cuidador,” that allowed the end-user to delegate to a chosen, trusted person the ability to plan and send routes to the PND (see Figure 3.3). The system enabled multiple end-users for a caregiver and multiple caregivers for an end-user. This allowed the end-user to provide access to their data in a controlled fashion, ensuring the privacy and trustworthiness of the process.
Figure 3.3: Left, the end-user’s PND; right the caregiver’s assignment tab on the route editor. From Assistant Project (2015).
Not all support systems will require this two-interface approach, but often designers try and fit all the system’s functionality into one interface. In both ASSISTANT and MAPS requests were made to put both the script/route design into the handheld device; issues with screen size, information complexity, and network robustness lead to the dual interface approach. Putting all the functionality into one interface may lead to an end-user user interface that is too complex to be used by the end-user, especially with respect to navigation within the system. The dual interface approach should, in my experience, be always at least considered, especially in the requirements gathering and initial design. This decision should be supported by all stakeholders, not just the technologist, and especially experts, clinicians, and family.
3.1.4CONCLUSION
Not all AT systems that are IA based depend on the dyad approach, but many do. One important take-away from this section is for the designer to ask herself, “What part do caregivers or other intimate stakeholders take in the work practices that this system will require?” Even if the inter-twinement is not as fundamental as in the above examples, the answer to the question will always improve the interface and functionality of the final system.
3.2THE IMPORTANCE OF REPRESENTATION
Short Definition: Representation is the translation of components or strategies in problem solving to make the problem more tractable. “Efforts to solve a problem must be preceded by efforts to understand it” (Simon, 1996).
Longer Description: Solving a problem may simply mean representing it so as to make the solution transparent. Here is an example, a two-person game.
1.Take the numbers from 1 to 9 (1, 2, 3, 4, 5, 6, 7, 8, 9).
2.Players alternate and take one of the numbers.
3.The player who can add exactly three numbers in her/his possession to equal 15 wins.
Nickerson discusses this problem in his book, Mathematical Reasoning: Patterns, Problems, Conjectures, and Proofs (Nickerson, 2010):
Sometimes the right representation can greatly reduce the amount of cognitive effort that solving a problem requires. A representation can, for example, transform what is otherwise a difficult cognitive problem into a problem, the solution of which can be obtained on a perceptual basis. A compelling illustration of this fact has been provided by Perkins (2000). Consider a two-person game in which the players alternately select a number between 1 and 9 with the single constraint that one cannot select a number that has already been selected by either player. The objective of each player is to be the first to select three numbers that sum to 15 (not necessarily in three consecutive plays). A little experimentation will convince one that this is not a trivially easy game to play. One must keep in mind not only the digits one has already selected and their sum, but also the digits one’s opponent has picked and their running sum. Suppose, for example, that one has already selected 7 and 2 and it is one’s turn to play. One would like to pick 6, to bring the sum to 15, but one’s opponent has already selected 6 along with 4. So, inasmuch as one cannot win on this play, the best one can do is to select 5, thereby blocking one’s opponent from winning on the next play. In short, to play this game, one must carry quite a bit of information along in one’s head as the game proceeds.
Figure 3.4: Tic-Tac-Toe → The same game based on Nickerson (2010).
One could reduce the memory load of this game, of course, by writing down the digits 1 to 9 and crossing them off one by one as they are selected. And one could also note on paper the current sum of one’s own already selected digits and that of those selected by one’s opponent. Better yet, as Perkins points out, the game can be represented by a “magic square”—a 3 × 3 matrix in which the numbers in each row, each column, and both diagonals add to 15. With this representation, the numbers game is transformed into tic-tac-toe. The player need only select numbers that will complete a row, column, or diagonal, while blocking one’s opponent from doing so. There is no need now to remember selected digits (one simply crosses them out on the matrix as they are selected) and no need to keep track of running sums.
If you give this task (with timed turns) to a graduate student and a 12-year old, with the graduate student doing the number version and the 12-year old the Tic-Tac-Toe isomorph (Figure 3.4), the graduate student often doesn’t have a chance.
Other representational “tricks” include this tree (Figure 3.5) to decode Morse code without memorizing the system.
Figure 3.5: Morse Code decipher tree.
Another representational system leveraging everyday metal capacity is the Memory Palace (Yates, 1966). The memory palace is an age-old system, dating from Roman times through the Renaissance and featured in the current BBC television version of the Sherlock Holnes mysteries, for storing almost unlimited amounts of information and retrieving it easily (Ericsson, 2003) via a visualized house or location.
3.2.1CANONICAL PAPER
Simon, H.A. (1996). The Sciences of the Artificial. (Simon, 1996)
3.2.2AT EXAMPLE
The MAPS system used images to prompt steps in a script on a hand-held PDA (e.g., HP IPAQ). At first the intention was to use icons to represent objects and actions for the young adult with cognitive disabilities to be guided with. However, learning from the special education consultant in the L3D lab (Kintsch, 2002), and following the lead of the AbleLink commercial product (AbleLink, 2002), real and specific pictures were considered as better for recognition by the person with cognitive disabilities. A research literature search did not produce a tested confirmation of this assumption so the MAPS team decided to perform an experiment (Carmien and Wohldmann, 2008) to test this hypothesis. What follows is a summary of the paper referenced above.
In the process of developing MAPS, it became clear that the type of image displayed on the screen of the hand-held computer could affect strongly the success of the attempted task. The images used by computationally based augmentative and alternative communication systems (AAC) (Beukelman and Mirenda, 1998) and prompting task support systems can be a weak link in providing AT support for people with disabilities that require their use. When an image is not instantly recognizable as referencing an object in the world, then there are immediate consequences to the person attempting to use it successfully to accomplish a given task.
The two classes of systems that use images as supports, AAC and prompting support, have similar goals but differ with respect to how the image is used. Specifically, AAC images are intended to replace or supplant communicative ideas, which can range from simply pointing to an object in the world (“a loaf of bread”) to encompassing an entire concept (“buy a movie ticket”) (Beukelman and Mirenda, 1998). Computationally supported multimedia task support systems (Davies et al., 2002), on the other hand, have the ability to use a recording of a verbal prompt to provide the “verb” required to perform the task. Because of this difference, the emphasis of the present study was to determine the type of image necessary for successful identification of an object in the world.
The studies reviewed supported the notion that picture matching with objects is mastered before the use of images to communicate, and the matching skill is separate from the communicative skill. Several studies support the position that images out of context or, more properly in no context (on a white background), are more conductive to object matching. Other studies emphasized that picture recognition is a learned skill (as well as being developmentally based) and in support of this discussed various studies of different cultures approach to object representation and matching.
A common “rule of thumb” used by assistive technologists is that in order to obtain a match between a representation and the real-world object being depicted by the representation, the image fidelity must be inversely related to the level of cognitive ability. That is, lower abilities result in a greater need for fidelity of the image (e.g., Snell, 1987). Here the term “image fidelity” refers to a hierarchy of representations, from its actual physical form, to a model or photograph of the actual object, to a photograph of a specific brand of object, and finally to written words. It is not identical to iconicity or guessability, as the iconicity of a representation is a relationship between the observer, the representation, and the “target” object (e.g., Wilkinson and McIlvane, 2002). While research has pointed out that the iconicity of a representation is an idiosyncratic phenomenon, at least in specific instances, the authors posit that the need for higher fidelity is only loosely connected with cognitive ability—varying not only from person to person but also possibly from culture to culture (Huer, 2000).
The trials were designed to determine how the type of representation (icons, photos of objects in context, photos of objects in isolation) displayed on a hand-held computer affected recognition performance in young adults with cognitive disabilities. Participants were required to match an object displayed on the computer to one of three pictures projected onto a screen. The experiment was designed to test the opinion widely held by occupational therapists and special education professionals that there is an inverse relationship between cognitive ability and the required fidelity of a representation for a successful match between a representation and an external object. Despite their widespread use in most learning tools developed for people with cognitive disabilities, our results suggest that icons are poor substitutes for realistic representations.
Fifteen high-school students (9 males, 6 females, m = 15.8 years old) from the special education classes in the Boulder Valley School District were invited to participate in this experiment. Informed consent from both the student and his or her legal guardian was obtained. Students were selected on the basis of their IQ scores and their ability to use the MAPS script guidance system, as well as on their need for such a mnemonic and executive cognitive orthotic to accomplish activities of daily living (ADLs) independently. There was a wide range of cognitive disabilities, including, but not limited to, autism, Down syndrome, alcohol-related neurological disorders, and fetal alcohol syndrome. IQs (using the WAIS scale) ranged from 43–85 (m = 58.1). In addition, 15 age-matched high-school students (9 males, 6 females, m = 15.9 years old) with no cognitive disabilities were selected from classes in the Boulder Valley School District.
A laptop computer was to run a program written using Visual Basic V6 and Embedded Visual Basic. A projector displayed three target images onto a white screen. The computer was connected to a wireless router that established a local area network (LAN), which was connected to a hand-held Compaq IPAC model 3670 computer, with a 2¼ × 4 inch screen. There were three large push buttons (6 inches in diameter), in three colors (yellow, red, and green), attached to the laptop.
Figure 3.6: Image /icon recognition experiment setup. From Carmien and Wohldmann (2008).
Participants were tested individually in a quiet room. Each trial consisted of matching a picture displayed on the hand-held computer to one of three pictures displayed on the white screen by pressing one of three buttons that were placed in front of each picture on the white screen. Response time was measured as the time that participants took to press one of the three buttons after the onset of the image on the white screen.
Each image (called the cue) was scaled to fit on the hand-held computer (280 × 210 pixels), and the images displayed on the white screen included the correct target, a target lure (an item that could possibly make for a suitable substitute in place of the target), and non-target item (an item that was neither the target nor a near substitute for the target, but had a similar shape or size to the target) (Figure 3.6). For example, if the target image was jar of spaghetti sauce, then the target lure might be a jar of salsa and the non-target might be a jar of peanut butter. Each target could be represented on the hand-held computer in one of three ways. Specifically, the picture was either shown in isolation (a single target object shot against a white background), in context (a view of the target item on a grocery shelf), or was represented as an icon from the set of Mayer-Johnson picture-symbol icons (Mayer-Johnson, 2001) that are typically used in instruction for this specific population of young adults.
A short break was given after every 10 trials were completed, and for each participant the experiment consisted of 2 blocks of 30 trials with 10 of each picture type (icon, isolated, context) in a block. Each block had 10 sets of targets paired with one of the three kinds of cue images on the PDA. The 30 trials were randomly scattered across a block. In this way every participant was presented with every kind of cue for a given target.
The experiment began with a demonstration to show how the task was to be performed. After this initial practice phase the hand-held computer was set up for the trial. The hand-held computer had two states—currently showing the target image and ready to show the next target. The participant activated these states by touching the screen. Once the set of pictures had been selected, the hand-held showed the “ready” image. The participant was then asked to touch the screen when ready and the hand-held transmitted a signal to the laptop, triggering the laptop to both import the images that the participant would see on the projector screen and to send a signal back to the hand-held to display the target image on its screen. The images appeared simultaneously on the hand-held computer screen and on the white screen. When the participant pressed one of the three buttons, the computer obtained information identifying the image and a code that indicated whether that image was a perfect match, a lure match, or a non-target match.
Two dependent measures were included in the analyzes. First, response time was calculated as the time that it took participants to press one of the three buttons after the pictures were displayed on the white screen. Only accurate trials were included in the response time analyzes. Second, accuracy of the response was calculated and reported as proportion correct. The overall design for this experiment was a mixed factorial including participant type (cognitively disabled, cognitively typical) as a between-subjects variable and block (first, second) and target type on the hand-held computer (isolation, context, icon) as within-subjects variables.
A repeated measure ANOVA was conducted on both response time and accuracy. Two types of analyses were conducted on each measure. The first type included all three target types (isolation, context, icon) for comparison. The second type compared icons to the average of realistic images (i.e., to the average of those in isolation and in context) to determine how suitable icons are for representing real-world items. One typical subject was removed from the analysis because his accuracy performance was more than four standard deviations away from the mean of his group.
Both the first and second analyses of response time yielded a significant main effect of participant type. Typical participants showed faster response times compared to cognitively disabled participants. No effects involving picture type were significant.
In both types of analyses conducted, partially correct (i.e., the selection of a target lure) trials were included because target lures would make a suitable substitute for the target item and because there were very few correct trials from the participants with cognitive disabilities. Thus, correct trials were given a score of 1, correct target-lures were scored as .5, and non-targets were scored as 0.
As expected, the first type of analysis yielded a significant main effect of participant type. Specifically, typical participants were more accurate on the matching task compared to those with cognitive disabilities. In addition, participants were more accurate when the object was represented realistically either in context or in isolation than when it was represented as an icon; the main effect of picture type was significant.
In the second analysis comparing icons to realistic images, the main effect of participant type was also significant. Again, typical participants were more accurate than those with cognitive disabilities. In addition, the main effect of picture type (icon, realistic) was significant, with, overall, higher accuracy on realistic images compared to icons. Furthermore, the interaction of participant type and picture type was marginally significant. Participants with cognitive disabilities were less accurate when the pictures were represented as icons than when they were represented realistically, and although typical participants showed a similar trend, the difference in accuracy between icons and realistic images was smaller. This finding suggests that icons may be poor substitutes for realistic representations, especially for people with cognitive disabilities.
There was no significant difference between the two types of realistic images (isolated and in context). This was unexpected, as existing research (Braun, 2003) tends broadly to emphasize context and contextual cues as significant in object recognition. There are at least two possible explanations for our findings. First, the images called “in context” might not be in context in the way that the other studies used the word. Second, in the case of selecting discrete objects in the world on the basis of a small two-dimensional image, context may not be very pertinent.
Regarding the second explanation, it might be that what many of the other studies were looking for was the “meaning” of the images, especially with respect to communicative intent (for AAC purposes). For this specific use of images, and especially in a use environment where the image used is intended to be very specific to the goal, such as in a supermarket (Carmien, 2006b), the fidelity of the image is much more important than its arrangement. In this case of the MAPS system it is easy to change the images to reflect the specifics of the task on a day-to-day basis (i.e., the image could be changed for each instance of the prompter script that fit the user, goal and task). In any case, our results, while not conclusively supporting the conjecture that drove the experiment, suggest that further exploration of this topic would be helpful in building an effective computational task support system.
Based on the results, icons and photographic images depict real-world objects in different ways. This difference, whatever it may be, is enough to lead to errors in matching those representations to their real-world referent. The results imply that making “generic” scripts out of icons would not be a good strategy for creating scripts to guide a person with cognitive disabilities in daily tasks. Given that knowledge, the current practice of assembling prompts out of icons needs to change toward using realistic photos. For some participants and objects a generic photo may suffice, for others a generic photo may either not be available or not have rich enough detail to afford easy matching. This leads to a requirement of computationally enhanced prompting systems to support the easy use of caregiver-taken photos in the creation of scripts.
3.2.3CONCLUSION
The importance of representation on interfaces is critical to success in applications designed for AT/IA. This, as discussed above, is an expression of the cognitive abilities of the target end-users. Also, and perhaps not so obviously, is the choice of navigation and controls for the functions of the application. Often, novice designers will try to express their creativity by designing new icons for actions like getting help or starring a action, this can lead to failure in two ways: (1) a programmer is not a trained graphic designer and the results, while pleasing, may be incomprehensible to the end-user; and (2) introducing novel icons to replace ones that the end users, especially young adults, may already understand, creates another cognitive hurdle to surmount. Reuse successful interface elements and think (and do trials with end-users) before injecting improvements. In fact, this is a useful guide line for all user interface design (Lewis and Rieman, 1993).
3.3TOOLS FOR LIVING AND TOOLS FOR LEARNING
Short Definition: Tools for learning are systems that support changing the user to (re)gain skills. Tools for living are systems and devices that support the end user in doing tasks that they cannot do. Tools for learning are used and (hopefully) abandoned; tools for living are carefully fitted to the user and typically used for the rest to the user’s life (Carmien and Fischer, 2005).
Longer description: Tools for learning support people in (re)learning a skill or strategy with the objective that they will eventually become independent of the tool. Tools for learning afford an internalization of what was (if it existed previously at all) an external ability/function; tools for learning often serve a scaffolding function (Pea, 2004).
In contrast, tools for living are external artifacts that empower human beings to do things that they could not do by themselves. One use of a tool for living is supporting distributed cognition; that is, it serves as an artifact that augments a person’s capability within a specific task without requiring the individual to internalize the sub-tasks conducted by the artifact (e.g., a hand calculator). Tools for living can be tailored for specific tasks and for specific people.
CLever’s (“Cognitive Levers: Helping People Help Themselves”) (CLever, 2004) goal was to create more powerful media, technologies, and supportive communities to support new levels of distributed cognition (Carmien et al., 2005). This support is designed to allow people with disabilities to perform tasks that they would not be able to accomplish unaided. The objective is to make people more independent by assisting them to live by themselves, use transportation systems, interact with others, and consistently perform normal domestic tasks such as cooking, cleaning, and taking medication.
CLever identified and explored a fundamental distinction in thinking about the empowerment and augmentation of human beings (Engelbart, 1995) and the change of tasks in a tool-rich world by identifying two major design perspectives.
1.Tools for Living: grounded in a “distributed cognition” perspective, in which cognition is mediated by tools for achieving activities that would be error prone, challenging, or impossible to achieve.
2.Tools for Learning: grounded in a “scaffolding with fading” perspective leading to autonomous performance by people without tools.
Internal and External Scripts: The differentiation between tools for living and tools for learning is closely related to a similar fundamental issue: the interplay and integration between internal and external scripts (Carmien et al., 2007). Internal scripts are chunks of mastered acquired behavior that can be executed without the need for external support. External scripts are instructions that afford the accomplishment of more complex tasks by triggering internal scripts to execute the externally cued steps. Tools for living exploit the interplay and integration between internal and external scripts. In contrast, the goal of tools for learning is to acquire new internal scripts, thereby becoming independent of external scripts.
3.3.1TOOLS FOR LEARNING
Looking at the relationships between humans and the artifacts they use, tools or systems can be seen as primarily either supporting human adaptation or providing support for humans to affect change in their environments. Thinking about systems and artifacts in this way affords insights into distributed intelligence, the design and use of artifacts, and educational decisions about learning and teaching of skills with and without tool support. Some artifacts support people by providing guidance and then gradually removing assistance until the skills become internal scripts, accessible without external assistance. Examples of such tools for learning are: (1) bicycles with training wheels (see Figure 3.7); (2) toddlers’ walkers; (3) wizards (environments that interactively guide users through the steps that comprise the process of setting up computer applications) used in many computational environments; (4) experienced teachers providing help and assistance to learners; and (5) caregivers assisting people with disabilities.
Figure 3.7: Learning to ride a bicycle with training wheels. An example of a tool for learning. Courtesy of s_oleg/Shutterstock.com.
Optimally, a tool for learning uses scaffolding of some sort to provide enough support in the beginning of the tool use to actually compete a task without so much frustration that the user quits, an example of which are systems to teach programing (Wood et al., 1976) or the scientific approach (Guzdial, 1994). Similarly, it should have some sort of graduation point where the tool is no longer needed to do the task, an example of this is the removal of the training wheels on a bicycle.
Tools for Living
Some artifacts enable users to perform tasks that are impossible for them to do unaided. Often, no matter how many times the task is accomplished with the aid of a tool, the user has no greater ability to do the task unaided than she or he did initially. Examples of these tools for living include ladders, eyeglasses, the telephone, screen readers for blind people, visualization tools, and adult tricycles (see Figure 3.8). No matter how many times people use the phone to talk to friends across town, their native ability to converse over long distances unaided remains the same as before they used the telephone. Tools for living allow people with disabilities to perform tasks that they would not able to accomplish unaided, and therefore allows these people to live more independently.
Figure 3.8: Adult tricycle: an example of a tool for living. Courtesy of Belize Bicycle, Inc.
Tools for living require a tight fit between the user and the device (e.g., eyeglasses and eye examination). They also are used forever (exceptions being laser surgery and progressive loss of vision).
Trade-Off Analysis: As with any category scheme, there are overlaps and differences of interest and use (Table 3.1 summarizes some of those). The categorization of devices depends only, to a small extent, on the intentions and viewpoints of designers and developers, but their use is much more profoundly determined by the cultural values of the use situations and the societal organizations that determine them. The inappropriate use of some tools for living has been identified as causing a sort of “learned helplessness” in that the ease and accessibility of using some of these tools inclines the user to not expend the energy and time to acquire these skills internally. Examples of this learned helplessness are the use of hand calculators and spell checkers as tools for living, thus blocking the acquisition by the users of arithmetic and spelling skills, not unlike the above-mentioned example of complaints about reading ruining native human abilities. Much has been written and debated about the use of calculators, spell checkers, and other cognitive tools in education and whether they should be used as tools for living or tools for learning and how these decisions may be different in the case of children who suffer from permanent disabilities such as dyslexia.
Over-reliance on Tools for Living: Does an over-reliance on tools for living lead to learned helplessness and deskilling, ruining the user’s native abilities by making them dependent on the tool? The cautionary point of The Time Machine (Wells, 1905) was that the Eloi had lost any power in their world by having too much done for them. It is this fear that leads to a moral judgment about the use of such tools. Asimov (1959) tells the thought-provoking story “The Feeling of Power,” in which a scientist who is actually capable of doing simple math problems on paper without the aid of a computer is identified by the military of that distant century and how the military tries to make use of his “newfound skills.” Perhaps the resistance to them is also tied to the 19th century belief that mental deficits were illnesses that reflected moral vices or immoral lifestyles. The logic goes something like this: a calculator is a “crutch” that allows a person to forgo the effort associated with true learning (displaying the moral vice of laziness). There is little doubt that the inappropriate use of these devices can lead to “learned helplessness” (e.g., providing screen readers to children with dyslexia has in some cases prevented them learning techniques that may have largely ameliorated the effects of their developmental disability (Olsen, 2000)). However, accepting that as a genuine concern does not necessarily lead us to the conclusion that all such technology is bad for humans. The daily life of all of us is filled with an almost infinite number of tools for living.
3.3.2CANONICAL PAPER
Carmien, S. and G. Fischer (2005). Tools for living and tools for learning, (Carmien and Fischer, 2005).
3.3.3AT EXAMPLES
An other example of a tool for living, but having some attributes of a tool for learning and more situated in AT for IA, was the MAPS project. This is a good example because of the parts that were designed from each tool’s perspective highlights the tool’s approach well.
Prompting has long been used as a technique to aid people with cognitive disabilities to live more independently. With the advent of computationally based prompting systems, the image and verbal prompts that lead a user through task completion can now be highly personalized, and the need to memorize the steps to a task is off-loaded to the computerized prompter (in this case, an HP IPAQ h6315), thus changing the task from memorizing scripts of prompts to accessing the appropriate script and following the prompts as they are presented. Our prompting system consists of the caregiver’s script editor, which supports the creation and storage of scripts consisting of pairs of images and verbal prompts, and a handheld prompter, which is used by the person with cognitive disabilities and presents the previously made scripts.
MAPS as a Tool for Living
MAPS was designed as a tool for living inspired by the Visions system (Baesman, 2000), a stationary prompting and scheduling system based on PCs using speakers and stationary touch screens to prompt a user with cognitive disabilities through a complex domestic task such as cooking. Part of the Visions system supplied sets of cards that assisted away-from-the-system tasks such as grocery shopping. Although Visions worked, it had problems with re-configuration, and the computationally implemented in-home video/audio prompting component was not able to go with the user out of the home in which it was installed (other than the sets of cards). As a result of the problems with re-configuration and generalizability issues, the MAPS user interface was made for two (end) users: the actual user and the caregiver. As a tools-for-living artifact, the initial setup was critical, and special care was paid to functions that aided the caregiver in conforming to the needs and abilities of the prompter user. The initial configuration supports specifying image and verbal prompt styles: image aspect, verbal prompt complexity, and word choice. The configuration wizard also parameterizes the capturing and correction of errors: how to detect that a script was not being followed, how to detect the problem that caused the error, and what to do to correct the error (some users can assist with self-correction, some will need human intervention (e.g., activating the MAPS cell phone connected to a caregiver, and some will require emergency intervention (e.g., calling 911)).
Because the target population has a limited number of possible “operations,” the set of available scripts should not vary greatly, and the same scripts will be used over and over. Moreover, many scripts for outings will be constructed by reusing sub-scripts (e.g., the steps to “get from getting ready to going out to the closest bus stop” will be repeatedly used). What will change is the timing and, to a small degree, the content of the scripts. For example, if MAPS were equipped with a GPS and is networked so that when the user gets to the bus stop, the arrival of a specific bus will trigger the prompt to get on the bus (Sullivan, 2005) (see references to the ASSISTANT system which implemented this). As a tool for living, MAPS is expected to support a specific, small set of tasks and, in this mode, not provide support for autonomous generalization of the skills through which the user is being guided.
Training in using MAPS, just as training in using any tool for living, consists of a simple set of instructions in the use of the tool, not extended to any overview or understanding in the principles involved in doing the prompted through task itself. Similar to instructions for the care and use of contact lenses, in which there is no attempt to explain optics, training in using MAPS does not involve anything about the domain of the tool other than pragmatic ATM-style instructions. This is typical for the change in the task that using tools for living involves. The task now is the use of the tool, not the difficult cognitive tasks of figuring out what to do and when to do it. This task re-mapping is a typical result of bringing a tool for living into play. Another example of this is the change in the details for arithmetic calculator users, who are now experts on using the buttons to input and control the calculator rather than memorizing complicated algorithms for calculating square roots. In this aspect the AT/ IA versions of Tools for Living implement many of the concepts of distributed cognition (see Section 2.3).
Concerns about the robustness of the tool, and concerns about the target population (e.g., what if the user loses it, what if it breaks while the user is in a bad part of town at night?) are part of our research agenda. Any design that allows and encourages a somewhat fragile population into a situation that may be somewhat of a threat in the case of breakdown has a moral responsibility to ensure an elegant decay/failure in such cases. LifeLine (Gorman, 2005) aims to address these concerns by monitoring the use of MAPS and involving a caregiver if necessary (Carmien et al., In Press). These issues were also addressed in the ASSISTANT system (see Section 2.2).
Creating Tools for Living: A crucial part of MAPS is the script design environment used by caregivers to design effective external scripts that trigger the internal scripts to allow people with cognitive disabilities to live more independently. Critical to successful design of a tool for living is the initial fit of the device; examples include visits to an optometrist for eyeglass fitting or to a clinic for prosthesis fitting—devices that require precise alignment to the user to operate. In the case of the MAPS system, the hand-held user needs images that have the right affordances (Norman, 1993) to allow the correct selection of the desired object. Similarly, the verbal prompts need to have the right level of words and grammar to fit the needs and abilities of the person with cognitive disabilities that is using the prompter. Generic icons and computer-generated verbal prompts will typically not be as effective as ones tailored to the user specifically (Snell, 1987). Figure 3.9 shows the caregiver’s script editor in use: the caregiver builds prompts by selecting from sets of pictures and recorded prompts, and creates scripts from sequential prompts The finished script can then be loaded into the prompter for use by the person with cognitive disabilities in accomplishing the desired task. Because this initial fit is so critical to the use of the tool for living, generic script templates, which have been previously used successfully, are provided for caregivers to use as models in producing specific scripts.
Figure 3.9: MAPS script editor sections. From Carmien (2006b).
Learning to Use Tools for Living: Tools for living will be of no help if people are unable to use them. In the context of MAPS, the system explored the interpretation of specific external scripts. In designing and using tools for living, two key elements of which are the initial fit and the correct leverage of existing skills. In the case of MAPS, the caregiver and assistive technologist need to understand what would be the best modality and style for the effective delivery of scripts. Perhaps the user is able to map standard icons to articles in the real world for a shopping script; this would make the creation of scripts much easier than those for the person with cognitive disabilities who needs specific pictures of the exact target item. Similarly, some users would respond best to prompts from a familiar authority figure (i.e., mom), whereas for others this might be totally wrong, due to typical teenage power struggles (the user’s own voice doing the prompting might be best). Having determined the appropriate modality for the tool, the caregiver needs to assess and fit the prompting structure of the script for the internal scripts that the user may have. For instance, one user may need only an overarching prompt of “Go to your bus stop” whereas another may need to have this segmented into three steps: “Walk down the street to the corner,” “Cross the street,” and “Turn left and walk to the bus stop in the middle of the block,” each one of these steps corresponding to an internal script possessed by the user that may be successfully triggered by an external script in the prompting sequence.
Another perspective on using tools for living is that once the task that was too difficult to do is remapped to a task that is within the domain of existing skills, there still remains the need to acquire those skills that are needed for the remapped task. An example in the case of MAPS would be replacing the whole task of performing a janitorial job at a restaurant, requiring a series of executive (when to do the subtask) and mnemonic (the specifics of the subtask) actions that constitute the job, with the task of using the controls on the prompter to sequentially trigger the existing internal scripts that would then constitute the whole job. An example from another domain might be replacing the task of memorizing the Qur’an with reading the Qur’an. In both examples there is the need to learn the use of the tool for living, and this learning with the same external support constitutes a tool for learning. In the case of learning the use of the MAPS prompter, studies have observed young adults with cognitive disabilities taking about a half hour; in the case of learning reading, this would take perhaps many thousands of hours.
MAPS as a Tools for Learning
MAPS is also designed to be used as a tool for learning; even when used in the tools-for-living mode, it will afford some tools for learning functionality. MAPS is designed to accommodate several types of scaffolding fading. The first and most crude form of fading is based on a review of the logs of user behavior that MAPS keeps (with awareness of the privacy issues involved): the caregiver can consequently manually edit a given script, folding several prompting steps into one “reminder” prompt. Next, MAPS could evaluate usage and autonomously, in an algorithmic fashion, compress a script based on logs in a batch mode. Finally, MAPS could dynamically expand or contract the steps required to perform a given task based on past behavior, user input, and immediate history. Neither of these were actually implemented in the project, but were simulated by hand and an algorithm generated with an aim of later implementation. MAPS is designed to provide scaffolding to augment memory and cognition; as a memory prompt tool, it may just need to “nudge” the user in the correct direction upon determining that an error condition is about to happen (a probabilistic guess, but no harm done if the guess is wrong), Suchman (2007) discussed this in the second edition of her seminal work, Plans and Situated Actions.
3.3.4CONCLUSION
Problems in using the wrong tool include, in an example using a tool for learning in place of a tool for living, spending inappropriate effort in trying to learn the task instead of using a tool for living to just make progress. This trade-off is based on the amount of time/resources needed to learn the task (e.g., using a calculator versus learning numerical analysis in a computer science engineering school to do cube roots). Similarly using a tool of living in the place where a tool for learning is more appropriate can make a task frustratingly slow and motivate abandonment. An example of this misapplication of class of tool is giving a system to compensate for dyslexia to a younger person where it might be more appropriate to provide a set of exercises so that they can internally compensate for their reading problems, one that they can scaffold and graduate from (Olsen, 2000).
Problems in using the tool wrong include deskilling inappropriately, an example of which is losing the ability of doing simple arithmetic in your head as a result of using a calculator exclusively. (A problem I had after undergraduate engineering school, which I worked successfully on by not bringing out my calculator or pencil for simple problems.) Another example is losing a mental overview map of a route by overreliance on GPS.
The most easily recognized tools for living—eyeglasses and prosthetic limbs—are clear examples of the properties of these tools. Supporting tasks that will never be doable without them, precise fitting, and never graduated from (i.e., used forever).
There are, as mentioned above, many stratagems and tools for learning compensatory skills for moderate dyslexia (Olsen, 2000). Similarly, rehabilitative systems for re-mapping muscle functions in stroke victims are in contrast to AT provided later when the brain is not so amenable to exploiting its plasticity (Perry et al., 2009).