Chapter 8

Good Reasons for Bad Testing Performance: The Interactional Substrate of Educational Testing^*

Douglas W. Maynard

Indiana University

Courtney L. Marlaire

Marquette University

Children who experience difficulties in school or at home may be referred to a diagnostic clinic and there take a battery of examinations, including some that test their educational level and learning abilities. In analyzing the administration of a variety of test instruments, we argued that the results of these examinations are collaborative productions (Marlaire & Maynard, 1990).¹This is contrary to the stimulus–response model of the testing relationship, which presumes that examiners are neutral conduits of prespecified items to which examinees respond with correct or incorrect answers reflecting individual levels of ability. Videotapes and transcripts of actual exam episodes show that each part of a “testing sequence” is assembled in the socially organized interaction between examiner and examinee.

Whereas the previous analysis utilized excerpts from a variety of testing instruments,² in this chapter we concentrate on a single subtest, called blending, of the Woodcock–Johnson Psychoeducational Battery, which is designed to measure both aptitude and ability in a variety of learning-related functions. The blending subtest involves the clinician breaking up words into components and speaking them to the child, whereupon the child must reconstitute the sounds as the appropriate word. According to the test manual, the purpose is to measure a child’s ability to verbalize whole words after hearing syllabic and phonemic components that the examiner presents sequentially. The subtest by itself does not determine how clinicians will assess the child’s ability; rather, the blending score, along with those from other subtests, becomes part of a cluster that indicates “broad cognitive ability” and “reading aptitude.”

By investigating this one subtest intensively, our purpose is to deepen our understanding of the test process as a collaborative accomplishment. It seems that participants interactively assemble the individual parts of a “testing sequence”—that is, that each question or answer as a performance is routinely embellished with the minutiae of actions exhibiting the participants’ efforts to work together to produce the utterance. More than that, this sequence itself is an embedded detail of an organized substrate of nonvocal as well as vocal activities. We refer to the interactional substrate of educational testing as consisting of those skills of the clinician and child that allow them to arrive at an “accountable” test score. By “accountable” test score, we mean one that is taken as objective, verifiable, valid, properly achieved, and so on, where that achievement depends on an organization of concerted practical actions that constitutes the participants’ interaction. Actual testing, then, is nothing other than the interactional achievement of which it consists, and from which test scores can be extracted. Although the capacity to engage in these practical actions is not itself being tested, the possibility of displaying those abilities which are subject to examination utterly depends upon them.³

Our title is meant to suggest that the interactional substrate of educational testing may be a factor that interferes with the proper standardization and, hence, validity of testing scores. However, we do not mean to describe the testing process in an ironic manner—that is, to somehow debunk the exams⁴ and show the ineptitude of administrators and consequent unfairness to children. If anything, our adult subjects were impressive for the amount of professional expertise they deployed in questioning and probing children. Rather, our title obviously adverts to Garfinkel’s (1967) classic study, “Good Organizational Reasons for Bad Clinic Records,” where it is demonstrated that apparently incompetent recordkeeping has an organizational basis in the reflexivity of documents to the courses of action that clinicians engage in their everyday routines. In our investigations, we “came upon” a seemingly vast amount of behavior that might threaten the ideal of standardized test-giving practice.⁵ However, detailed analysis reveals just the opposite of unprofessional, sloppy, or uncontrolled conduct, for the giving and receiving of test items depends on interactional systematicity and already orderly modes of collaborative behavior.

PRELIMINARY

We videotaped three clinicians who were individually paired (according to a system of rotation) with 10 children ranging in age from 3 to 8 years. We chose to study the Woodcock–Johnson Blending Subtest for two reasons: It is short enough to analyze intensively from beginning to end, and we have two episodes involving different clinicians and children. We have transcribed Episode 1 (with Clinician 1 and Child 1) and Episode 2 (with Clinician 2 and Child 2) in detail, using conventions adapted from the work of both Gail Jefferson (1974) and Charles Goodwin (1981) (see Appendix). In excerpts from these episodes and in the text, we refer to the children and clinicians with abbreviations (CL1 = Clinician 1, CH1 = Child 1, CL2 = Clinician 2, CH2 = Child 2).

Initially, three aspects of the interactional substrate stand out. They are exhibited in the first excerpt:⁶

(1) Episode 1:1

1.	CL1:	This is kind of a game, and this game (0.2) means you
2.		have to figure out what I’m saving. I’m gonna say a
3.		word. One part at a time. And you have to tell me what
4.		word I’m saying. Okay? Listen to this one.
5.	CH1:	Kay
6.	CL1:	Fing? ger.
7.	CH1:	Finger.
8.	CL1:	Goo::d. You know this game, don’t you.
9.		(0.4)
10.	CL1:	Okay. Listen to this word. Win dow.

First, there is co-orientational work on the part of clinician and child; both posture themselves in characteristic ways while engaging in the examination (see Fig. 8.1). While we will eventually discuss this matter in some detail, we can note here that the clinician manages several tasks through what may be called displays of “administrativeness”: paying attention both to the child and to reading and scoring tasks. Co-orientation by the child involves displays of “recipiency” (Heath, 1984) or shows of readiness to receive a testing item. When such displays are absent, the clinician regularly issues a co-orientational summons, such as “listen to this one” (line 4) or “listen to this word” (line 10). A second aspect of the interactional substrate consists of instructional sequences that prepare the child for an upcoming series of tasks. At lines 1–2, the clinician formulates what the subtest task is; at lines 2–4, assigns jobs to specific parties; and at lines 6–8, rehearses a testing sequence. The transition from instruction to actual testing is regularly marked with some proposal about the child’s understanding (line 8, “You know this game, don’t you”).⁷ In the context of neurobehavioral examination, Lynch (1984, pp. 71–72) also notes the importance of “prefatory” instructional components and notes that the transition to testing involves a “stripping away” of these components.⁸

FIG. 8.1. Displays of co-orientation.

Finally, then, are the testing sequences to which we have already referred. Both rehearsal and actual testing appear to incorporate a three-part sequence:⁹

Part 1; testing prompt:	Fing? ger.	(Line 6)
Part 2; reply:	Finger.	(Line 7)
Part 3; acknowledgment:	Goo::d.	(Line 8)

This sequence can be either elaborated or collapsed according to the contingencies of actual interaction.

An elaborated test sequence results when a clinician initiates “repair” or correction¹⁰ with respect to the child’s answer:

(2) Episode 2:18

1.	CL1:	Can dee
2.	CH1:	Can dee
3.		(0.1)
4.	CL1:	Can you sayitfast?
5.	CH1:	CANdee
6.	CL1:	Good.

The basic sequence here consists of the prompt (line 1), the correct answer (line 5), and the acknowledgment (line 6). However, the correct answer is arrived at by way of the child first offering a reply (line 2) whose cadence closely mimics the prompt. Thus, at line 4, the clinician asks for a speeding up of the reply, which she models through very swiftly speaking the latter part of the request, “say it fast.” McHoul (1990, pp. 365, 372) suggests that repair initiations of this sort deal with “procedural” rather than substantive or informational aspects of answering, although it is clearly the case here that the procedural issue relates to the substance of the answer. That is, the child’s answer may be regarded as incorrect if the “parts” are there but not spoken in the correct modality. In any case, repairs like this exhibit how instructions provided at the outset of the subtest are inherently incomplete. That is, it may be impossible to anticipate all the ways in which the child might have to be cued to provide appropriate answers when the test is actually in progress; accordingly, clinicians introduce instructive activities according to contingencies that are local to the in vivo performance of particular test items.¹¹ Resulting largely from clinicians’ initiations of repair and correction, then, elaborated testing sequences show further aspects of the interactional substrate. It is through such sequences that, when children do not at first provide what a question intends, clinicians, in various ways, can seek a better, more appropriate, and ultimately correct answer.

A collapsed test sequence is visible in the clinician’s elimination of the third-turn acknowledgment. When a clinician first elides the third turn, this may launch a chained series of collapsed sequences.¹² This happens with our second clinician–child pair (see lines 7–16):

(3) Episode 2:9

1.	CL2:	I want you to tell me what the word is. I’m gonna say
2.		f:in:: ger: What word did I say.
3.		(0.4)
4.	CH2:	Finger?
5.	CL2:	Okay. Good. You’ve got these. Win: dow.
6.	CH2:	(0.2) Window?
7.	CL2:	Good. (1.0) Muh: ther:
8.	CH2:	Mother?
9.	CL2:	(1.4) Ta: hl::.
10.	CH2:	Table?
11.	CL2:	(1.0) .hhh Can: dee
12.	CH2:	Candy?
13.	CL2:	(1.2) Rho: dah:.
14.	CH2:	Road.
15.	CL2:	(1.4) Soh: puha.
16.	CH2:	Soap.

The clinician’s last use of an acknowledgment (line 7) occurs after the child correctly provides the answer “window” (line 6). Subsequent to “mother” (line 8) and for the next 10 items, the clinician provides only a prompt. Thus, it seems that rather than using an acknowledgment to indicate the completion of a sequence, the clinician does so by pausing and then producing a new prompt. There is a simultaneity here (the closing of one sequence and the opening of another) that is similar to what Schegloff (1986, pp. 130–131) describes as the “interlocking” of sequences. It is only when the child gives an incorrect answer that this chaining or interlocking stops, and the clinician once again uses an acknowledgment.

FURTHER ASPECTS OF THE INTERACTIONAL SUBSTRATE

So far, we have described the interactional substrate in terms of sequences—co-orientational, instructional, and testing. It is through such sequences that participants engage in much of the work of actual examination. However, these sequences only scratch the surface of the interactional substrate. We wish to deepen our analysis in two ways. First, we consider another expanded test sequence and, among other things, show that it is important to situate that sequence within the local history of the coordinated activity comprising the test so far. Second, and at greater length, we will examine nonvocal, embodied activity of the exam’s participants.

Local History of Testing Sequences

The episode we wish to examine occurs just after excerpt (2) just mentioned. In this next episode, the child is ultimately scored as answering incorrectly. We will consider how the interactional substrate may contribute to such an outcome.

(4) Episode 1:23

1.	CL1:	Roh::duh.
2.		(1.2)
3.	CH1:	Rohduh.
4.	CL1:	Inkay. Roh::°duh
5.		(3.8)
6.	CH1:	Roh::
7.		(0.2)
8.	CL1:	Can you say it fast?
9.	CH1:	Rohduh
10.	CL1:	Oka::y.

Several things are of note here. For one, after CH1’s first try at an answer (line 3), CL1 initiates repair with regard to that answer (line 4), but also modifies her own prompt by softening her pronunciation of the last sound (“duh”; the degree sign indicates this softening), which may exhibit her awareness that the source of the incorrect answer may be her original prompt whose sound bursts are distinct and forceful. Thus, it is not only the child who may have to do some in situ learning (as we just noted). In the course of giving an exam, the clinician may modify her own behavior in relation to responses of the child, acquiring expertise as she goes.

More to our concern is the possibility that the child’s first answer reflects her just-acquired sense of speed. Having been successful at the speeding-up strategy on the previous word (“candy”), CHI appears to employ the same strategy here, saying “rohduh” (line 3) very quickly. Her focus on the speed of pronunciation may deflect attention from something crucial that differentiates this word from the previous ones, its syllabic structure. Moreover, if CH1 is preoccupied with the rapidity issue, CL1 reinforces this. After the repair initiation (at line 4), CH1 looks away from the clinician and slows down her pronunciation of the word by lingering on the initial sound (line 6), whereupon CL1 requests repair (line 8) in a way that again signals the need for a faster tempo. This time, CH1’s answer (line 9) is much like her first try (line 3); CL1 subsequently produces an acknowledgment (line 10) and scores the reply as incorrect.

The syllabic difference between the prompting word here and earlier ones is important. The subtest, we noted, starts with the rehearsal item “finger.” Next, in order, are “window,” “mother,” “table,” “candy,” and then the prompt for “road.” Thus, while all of the previous items are two syllables, this is just one. The significance of this is that, as Sacks (Lecture 12, Fall 1967, pp. 8–9) has argued, the “position of an item on a list is relevant to hearing what that item is.”¹³ Hence, in at least two ways—that is, with regard to the signaled concern for speed and the indicated syllabic structure of words, the local history of preceding talk and action provide a context for the child to mishear and even err in this particular segment. Just as in the courtroom Pollner (1979) studied, “… the developing session may act as its own socialization agent … every transaction in the ongoing activity may become fraught with instructional possibilities” (p. 235). Here, to the extent that the child is learning what this exam is from within the interactive practices of which it consists, she arguably has been socialized to produce a wrong answer. It is not the clinician’s incompetence nor the child’s inabilities that allow this, however, but rather the structures of interaction.

A final matter of note here is, in the acknowledgment turn, a significant change from what has gone before. Previously, when the child provided correct answers, the clinician produced the term good. Here, when the child seems to err, the clinician uses okay (line 10). This is a subtle shift but can occur regularly in accordance with whether the child is right or wrong, despite the fact that clinicians are not to provide evaluative feedback to children (Marlaire & Maynard, 1990, p. 96).¹⁴ Thus, it is not only that any given testing sequence has a history, it may also be historicized in certain ways. In a manner similar to how astronomers discover a pulsar (Garfinkel, Lynch, & Livingston, 1981), clinicians must extract from the “foliage” of their and their subjects’ embodied interactional practices, (a) an object that can be heard as an assessable answer and as achievedly produced according to proper methodic procedure, and (b) an assessment (correct or incorrect) that would stand as any competent clinician’s determination in the circumstances of the answer’s production. Accordingly, we have noted that when the clinician responds to the child’s candidate answers, she may modify her own behavior, and thereby show an awareness that an improperly given prompt may have misled the child. We have also shown how the design of the exam itself, which the clinician incorporates as a particular sequential pattern in the giving of testing prompts, may provide the occasion for a subject’s mishearing. Nevertheless, clinicians’ initiations of repair and correction work to obtain further utterances from the child that are the official exhibits for professional inspection.¹⁵ Thus, no matter what the cues to the child from the foliage of embodied practices that comprise the exam’s historicity, an extracted incorrect answer ultimately devolves to the child’s inability as its source (Drew, 1981, pp. 259–260; Marlaire, 1990) ¹⁶ As Lynch (1984) has remarked about neurobehavioral examinations, “Failures during the collaborative activity of testing were transformed into failure of the patient’s performance of the test” (p. 80).¹⁷

Nonvocal and Embodied Behavior: The Clinician

So far, we have concentrated on vocal aspects of these testing encounters, but the interaction is considerably more complex in the ways that clinician and child comport themselves bodily and do so collaboratively. We will demonstrate how the two participants finely tune their embodied behaviors in a concerted way, by discussing the separate tasks to which the parties attend, and then analyzing how their complementary management of these tasks is part of an overall activity structure of which, as we indicated earlier, the three-part testing sequence is only a part.

The clinician has to manage at least four jobs, and does so through co-orientational practices that we have glossed as displays of “administrativeness.” She must read the test items, give them to the child, listen to the child’s answers, and score correct and incorrect answers. Thus, a clinician sits with the examination instrument between herself and the child, has a score sheet on the table in front of her, and holds a pencil or pen, which she uses for scoring. In patterned ways, the clinician moves her head and gaze in such a way as to be noticeably attending to the child, the instrument, and the writing that she does on the sheet. How the testing sequence is coordinated with this nonvocal behavior is mapped in detail on the next transcript segment. Understanding the excerpt may necessitate consulting the transcription conventions for gaze behavior in the Appendix. In the segment, notice in particular the “X’s,” which mark where the clinician brings her gaze to the child:

(5) Episode 1:5

The points at which the clinician’s gaze arrives on the child are various, but still show a clear pattern: at line 2, the point is one word before the testing prompt; at line 6, it is just before the prompt itself; at line 9, it is during the prompt but before its second component: and at line 11, it is at the initial sound of the prompt. Invariably, in both of our blending subtests, the clinician’s gaze will have reached the child at least by the end of the testing prompt.

This shows a readiness to receive the child’s answer, and once the clinician’s gaze is directed to the child, it stays there for at least the beginning of the child’s response. Regularly, the clinician withdraws her gaze before a candidate answer is finished, whereupon she looks at the testing instrument (observe the commas, which mark withdrawal of gaze, at lines 3, 8, 10, and 12). This enables the clinician to be reading the next item from the instrument and sometimes scoring the answers while producing a third-turn acknowledgment or assessment.¹⁸ Thus, while one version of the third turn in a testing sequence is that it completes the sequence and aids with the transition between test items, we would suggest that the clinician’s gaze (and head movements), in anticipating or expecting that a complete answer is underway, indicate and help achieve the completion and transition process before an assessment appears. Moreover, because at least two tasks—the child answering and the clinician beginning to read the next item—are done in partial simultaneity, the glancing away of the clinician figures in the pacing of the exam.

Nonvocal and Embodied Behavior: The Child

Complementary to the clinician’s displays of administrativeness are the child’s displays of “recipiency.” As Heath (1984) described them, displays of recipiency are ways of showing a readiness for talk; in the context of examination, they exhibit a readiness for receiving the testing prompt.¹⁹ Displays of recipiency on the part of the child are required at each moment when the clinician is ready to deliver a prompt. The child provides them as seemingly natural accompaniments to the activity underway. That they are required, effortful acts becomes evident when a clinician takes notice of the child’s comportment and produces a co-orientational request, which we illustrated in episode (1) at lines 4 and 10.

Children display recipiency through at least three types of behavior: keeping the torso in an upright position, facing the clinician with the head, and directing gaze toward the clinician, at least by the time that a prompt is given (see Fig. 8.1). The interactional substrate is very finely tuned with respect to co-orientation. In the aforementioned excerpt, for instance, the clinician appears very sensitive to the slightest alteration in the child’s recipiency. Thus, in excerpt (5), during the rehearsal prompt (line 3), its answer (line 3), and the clinician’s evaluation (line 4), CH1 keeps her gaze fully on CL1. As CL1 says, “You know this game don’t you,” however, CH1 looks down at the test booklet. Then, before going on to the initial testing prompt, CL1 issues the co-orientational summons (“Listen to THIS word,” line 6) to which we earlier referred. During this summons, CH1 brings her gaze back to the clinician so that she is looking fully at CL1 before the latter produces the prompt (line 6). Furthermore, CH1 continues to keep her gaze and head fully directed toward CL1 during the subsequent three testing sequences (from lines 6 through 11). This expression of attentiveness, in fact, seems to solicit the evaluation or acknowledgment that the clinician provides (e.g., at lines 9 and 11). However, when answering the prompt for “table” (line 12), CH1 begins to withdraw her gaze, also moving her head down and to the right. Immediately, two things happen. CL1 eliminates the evaluation or acknowledgment, and she pauses (also clicking her tongue, line 13) for a shorter duration than she did with preceding prompts (compare silences at line 9, 11, and 13). It appears, then, that CL1 regards the child’s movements as an attentional lapse to be remedied with a faster pace, which is achieved in part through moving more quickly to the next test item.²⁰

Further evidence that clinicians may deal with problems in co-orientation by eliminating the third turn of the testing sequence derives from a similar happening in Example 2.²¹

(6) Episode 2:9

Initially, CH2’s gaze behavior here is similar to that of CH1. She brings her eyes to rest on the clinician just before the prompt is given, as during the rehearsal sequence involving “finger” (line 2), or she brings her gaze to the clinician on the second and last component of the prompt for “window,” line 9). After CH2 answers each of these prompts, CL2 provides the third part of the testing sequence, an acknowledgment. Next, however, during the prompt for “mother” (line 12), CH2, rather than looking at CL2, gazes downward at the test booklet, and continues doing so even as she answers (line 13). Then, while scoring and reading, CL2 simply pauses (line 14) and then produces the next prompt at line 15 (“table”).

At this distinct moment and in this precise way is the series of collapsed sequences, which we examined previously, begun. This is different from Episode 1, where, after the third-turn acknowledgment was eliminated, CHI answered the next item incorrectly, the clinician asks her to repair the candidate answer and (when the child does so), says “good.”²² CH2, in similar circumstances, answers correctly and continues doing so until the prompt for “about” (later in Episode 2). In dealing with CH2’s incorrect answer to “ah: bow: t:,” CL2 reintroduces a third-term acknowledgment, “Okay.” It seems, then, that continuing to engage the collapsed form of a testing sequence depends on the child obtaining correct answers. Also noticeable in Episode 2 is that throughout the series of collapsed testing sequences, CH2 mostly gazes away from CL2, only momentarily looking at her during the prompts for “table” (excerpt (6), line 15), “candy” (line 18), and “soap” (not in excerpt (6)). Mostly, CH2 is gazing at the test booklet, the table, and even the floor. As compared with CH1, she appears much more listless and less solicitous of the third-turn acknowledgment.

Recipiency and Alignment Toward the Testing Activity

Both interviews suggest that the components of recipiency are ways of enacting an alignment toward the proposedly central activity of testing. On the one hand, clinicians respond to gross head and bodily movements with co-orientational summonses. For example, in Test 1, after the child’s try at “road,” CL1 next gives the prompt for “soap”:

(7) Episode 1:38

As CH1 answers (line 3), she withdraws her gaze and then also moves her head down and to the right. At that moment, CL1 issues a co-orientational summons (line 4). Then, during the latter part of that utterance, CH1 returns her gaze to the clinician. She keeps it there, except for a brief moment during the silence at line 8, until the end of a rehearsal sequence (lines 11–14). When CH1 repeats the word “soap” (at line 14), she again withdraws her gaze. CL1 almost immediately produces a co-orientational summons (line 16), whereupon CH1 moves her head in a semicircle and ends up redirecting her head and gaze at CL1 (lines 16–17) in time to receive the prompt for “pillow” (line 18).

In both of our episodes, every time the child either moves her head or body out of the base recipiency position, such summoning activity occurs. Figure 8.2 shows a prototypical example of the child looking away; it is just that turn of the head to which the clinicians responds with “listen to this one.” Thus, asking a child to “listen” is not an indiscriminate action whereby clinicians offhandedly remind children to pay attention; rather, it squarely corresponds with a child’s change in alignment toward what the clinician regards as the focal activity. It seems, therefore, that such gross movements enact an unacceptable alignment that threatens the interactional substrate. Clinicians, from within the midst of that substrate, work to preserve it through explicit vocal and nonvocal summoning behavior.

On the other hand, children can distance themselves from the central activity in less extreme forms, withdrawing gaze while maintaining proper displays of bodily and head recipiency. Rather than using an orientational request and explicitly asking for the child’s return to attentiveness, clinicians may “speed up” or even “rush through” (Schegloff, 1982) the evaluation parts of testing sequences and in that way manipulate components to command the child’s orientation. As the child continues to answer correctly, the two parties concertedly achieve “rapid pacing” as an orderly feature of their interaction.

FIG. 8.2. Child’s head and gaze averted; Clinican: “Okay. Listen to THIS word.”

DISCUSSION

The interactional substrate of educational testing underlies the accountable production of test results. While both participants contribute to and collaboratively organize the substrate, the involved skills are not themselves subject to measurement, except indirectly. The substrate consists of such practical activities as prompting with test items, answering, initiating repair and correction of prompts and answers, doing the repair and correction, acknowledging, evaluating, and engaging other vocal and nonvocal, embodied practices so as to effect the test as an official and valid enterprise.

One way of approaching the interactional substrate is through the issue of standardization. Examinations are to be administered in a uniform manner, and environmental factors are to be controlled so that the test truly measures the child’s ability. Metaphorically speaking, the interactional substrate is like an environmental factor that has not been adequately investigated, much less controlled. We explore this metaphor with two main points, the learning that clinician and child experience during the test, and the feedback that clinicians may be giving their subjects.

Learning in the Midst of the Test

Even though children are given instructions and participate in a rehearsal of the subtest, excerpt (1), in which CL1 asked CH1 to say her answer “fast” shows that instructions can never be complete. In other words, for the child, there is in-process instructing and learning, even though she is supposed to know the “rules of the game” beforehand. This is true for the clinician as well. CL1’s self-repair on the prompt for “road” in excerpt (2) is evidence that she may notice her own performance as a source of trouble. In short, the participants to a test are like “Agnes,” the transsexual whom Garfinkel (1967) studied and who, from within the interior of the process of passing as a “natural, normal” female, simultaneously was learning what that passing entailed as a practical accomplishment. Or, the participants are like the judges and defendants in traffic court, who “live within” the very order of courtroom affairs and features that they simultaneously manage as a practical task (Pollner, 1979). From within the interior of the exam experience, children and clinicians learn what they should do to give, receive, and answer test items properly and correctly.

An implication of participants being attuned to what the in situ experience can teach them is that the child can acquire presumptions about a subtest and how to perform. When a clinician says to “say it fast” on one item, the child may use that strategy like an algorithm to be applied to any next answer. If a series of items show a pattern or characteristic (such as prompting words being one or two syllables), the child may use her knowledge of that pattern to fashion subsequent responses. Our point here is not that this is uniformly dysfunctional for the child. Indeed, what is perhaps being tested is the child’s ability to adjust to the nuances of different testing prompts. However, if errors introduced in a child’s answer reflect orderly processes that are not adequately controlled or held constant in the design of the test, it may bias the results (cf. Schaeffer, 1991). Following our metaphor, the interactional substrate, as an organization of practical activities, may be a systematic source of such influence on testing outcomes.

Feedback

Although permitted to provide comments of a generalized nature (“you’re doing fine,” for example), clinicians’ feedback is very circumscribed because of the possibility that it might influence the child’s performance. For instance, the Woodcock–Johnson (of which the Blending exercise is a subtest) specifically admonishes the clinician, “Be careful that your pattern of comments does not indicate whether answers are correct or incorrect” (cf. Mehan, Hertweck, & Meihls, 1986, pp. 96–97). Yet we see that clinicians, perhaps inadvertently, regularly do give such feedback, in at least two major ways.

First, clinicians and children, in being sensitive to one another’s activities and especially displays of recipiency, produce and alter their behaviors moment-to-moment in a contingent fashion. Thus, differences occur with respect to pacing and rhythm both within and across particular dyads engaged in the examination.

Indeed, the interactive structure of the testing sequence seems to be a product of collaborative pacing practices, although such pacing and structure are also responsive to the perceived accuracy of a child’s replies. Collapsed sequences only seem to occur in the context of correct answers, and the full three-part sequence is re-invoked in an environment of errant answering and/or difficult testing prompts. Thus, the type of sequence employed can provide feedback to children on their performance or the difficulty of prompting items.

A second way that feedback occurs is in clinicians’ altering their third-turn acknowledgments between “good” (when an answer is correct) and “okay” (when incorrect). Recall that in episode 1, the clinician pronounced the term “good” after correct answers to “finger,” “window,” “mother,” “table,” and “candy.” When the child errs on “road,” however, CL1 says “okay.” Also, clinicians may give encouraging nonvocal signals when a child answers correctly, while remaining more taciturn when she errs. In Episode 1, CL1 smiles and/or nods at lines 3, 9, 11, 13, and 17, whereas no such gestures are present subsequent to the “rohduh” episode. The pattern is operative throughout the episode.

Smiling is not just a matter of the clinician’s style or independent method of encouraging a child. It seems that the child, when giving her answers, smiles expectantly at the clinician. That is, there may be something of a smiling sequence in operation, where the clinician’s smile is a response to the child’s initiation, and the absence of the clinician’s smile may be a noticeable absence (Schegloff & Sacks, 1974), or one from which it is possible for the child to draw inferences about her own performance.

That nonvocal nodding and smiling behavior is a crucial aspect of the interactional substrate that can have consequences for the child’s performance is lent credibility from Goodwin and Goodwin’s (1987) consideration of the interactive structure of assessments in ordinary conversation. When speakers reference an “assessable” object, they may provide a characterization that proposes how their recipients are to understand and appreciate that object. Recipients, who are highly attuned to what Goodwin and Goodwin (1987) call the “participation possibilities” that assessments invoke, may reciprocate by producing an evaluation complementary to the initial characterization. Therefore, assessments are a conversational nexus of organized, collaborative actions. In the testing situation, a child’s answers are assessable objects. Clinicians, in vocal and nonvocal ways, regularly evaluate these answers in third-turn acknowledgments. If these assessments work at all like they do in conversation, children can inspect them to decide what they might implicate for their own concurrent and subsequent activities. On the part of clinicians, then, even slight alterations between “good” and “okay,” smiling and not smiling, or going relatively slow or fast, may provide children with the opportunity to infer how well they are doing.

If children are making such inferences, we do not know exactly what they are, nor how they might influence subsequent behavior. However, it is possible that the famous “Pygmalion effect” (Rosenthal & Jacobson, 1968; Rubovits & Maehr, 1973) or transmission of what becomes a self-fulfilling expectation, is, to the extent that it is real, brought about through the organization of activities in the interactional substrate of educational exams. More generally, our point is that until we know more about this substrate, there may be unknown and uncontrolled influences on testing behavior that are sources of distortion in the scores that children receive.

CONCLUSION

By discussing issues of standardization, we do not mean to engage in an ironic impugning of educational testing. We cannot claim that these examinations are unstandardized, and by no means were our clinician–subjects incompetent or unprofessional. Rather, our aim was to simply describe and analyze an infrastructure of collaborative actions that make generating accountable test scores possible. The interactional substrate is like a scaffolding on which clinicians depend in order to obtain access to measurable, quantifiable abilities. It is something that underlies other information-generating interviews, such as the survey (Schaeffer, 1991; Suchman & Jordan, 1990). After all is said and done, the reportable score that is the child for the purposes of educational placement, or the attitude that is registered in some poll, depends on this infrastructure, or substrate, or scaffolding of organized, practical skills. Ultimately, interviews as real-world accomplishments are inseparable from the substrate or scaffolding of skills through which participants make both the process and its products observable in their specificity. The clinical signs of deficit, just as the codable manifestations of subjective regard on social issues, emanate from what Lynch (1984, p. 81) has referred to as a “primordial grounding in the life world” (cf. Cicourel, 1982).

Despite our initial lack of interest in the effectiveness of test administration, problematic aspects of the testing immediately stood out as we reviewed our tapes. That is, we saw “mistakes” on the part of the clinician as well as the child, many of which appear not to stem from the inabilities or incompetencies of the clinician, or of the child, but precisely from the interactional competence they both exhibit. That competence resides in a set of skills through which they bring off official testing activities, such as providing a “stimulus” and a “response.” Thus does it appear, in our paraphrase of Garfinkel (1967), that there can be good interactional reasons for bad testing performance.

APPENDIX

Transcription Conventions

For the audio portion of transcripts, conventions are adapted from Gail Jefferson’s (1984) system (for example, see published version in J. Maxwell Atkinson & John Heritage, 1984, Structure of Social Action, pp. ix–xvi). Following is a transcription key for gaze behavior; these conventions are adapted from Charles Goodwin (1981), Conversational Organization: Interaction Between Speakers and Hearers (New York: Academic Press).

Gaze Transcription

REFERENCES

Adelman, H. S. (1979). Diagnostic classification of LD: A practical necessity and a procedural problem. Learning Disability Quarterly, 2, 56–62.

Atkinson, J. M., & Heritage, J. (Eds.). (1984). Structures of social action: Studies in conversation analysis. Cambridge, MA: Cambridge University Press.

Cicourel, A. V. (1982). Interviews, surveys, and the problem of ecological validity. American Sociologist, 17, 11–20.

Cicourel, A. V., Jennings, K. H., Jennings, S. H. M., Leiter, K. C. W., MacKay, R., Mehan, H., & Roth, D. R. (Eds.). (1974). Language use and school performance. New York: Academic Press.

Coles, G. S. (1978). The learning-disabilities test battery: Empirical and social issues. Harvard Educational Review, 48, 131–340.

Drew, P. (1981). Adults’ corrections of children’s mistakes. In P. French & M. MacLure (Eds.), Adult-child conversation (pp. 244–267). London: Groom Helm.

Garfinkel, H. (1967). Studies in ethnomethodology. Englewood Cliffs, NJ: Prentice-Hall.

Garfinkel, H., Lynch, M., & Livingston, E. (1981). The work of discovering science construed with materials from the optically discovered pulsar. Philosophy of the Social Sciences, 11, 131–158.

Goodwin, C. (1981). Conversational organization: Interaction between speakers and hearers. New York: Academic Press.

Goodwin, C., & Goodwin, M. H. (1987). Concurrent operations on talk: Notes on the interactive organization of assessments. IPRA Papers in Pragmatics, I, 1–54.

Grimshaw, A. (1980). Mishearings, misunderstandings, and other nonsuccesses in talk: A plea for redress of speaker-oriented bias. Sociological Inquiry, 50, 31–74.

Heap, J. L. (1980). What counts as reading: Limits to certainty in assessment. Curriculum Inquiry, 10, 265–291.

Heath, C. (1984). Talk and recipiency: Sequential organization in speech and body movement. In J. M. Atkinson & J. Heritage (Eds.), Structures of social action (pp. 247–265). Cambridge, MA: Cambridge University Press.

Holstein, J. A. (1983). Grading practices: The construction and use of background knowledge in evaluative decision-making. Human Studies, 6, 377–392.

Jefferson, G. (1974). Error correction as an interactional resource. Language in Society, 2, 181–199.

Jefferson, G., & Schenkein, J. (1977). Some sequential negotiations in conversation: Unexpanded and expanded versions of projected action sequences. Sociology, 11, 87–103.

Lloyd, R. (1991, August). Initiating interrogative talk. Paper presented at the annual meeting of the American Sociological Association, Cincinnati, OH.

Lynch, M. (1984). Turning up signs in neurobehavioral diagnosis. Symbolic Interaction, 17, 67–86.

Marlaire, C. L. (1990). On questions, communication, and bias: Educational testing as “invisible” collaboration. In J. A. Holstein & G. Miller (Eds.), Perspectives on social problems (Vol. 2, pp. 233–260). Greenwich, CT: JAI Press.

Marlaire, C. L. (1992). Professional idealizations and clinical realities. Current Research on Occupations and Professions, 7, 59–77.

Marlaire, C. L., & Maynard, D. W. (1990). Standardized testing as an interactional phenomenon. Sociology of Education, 63, 83–101.

Maynard, D. W. (1984). Inside plea bargaining: The language of negotiation. New York: Plenum.

Mazeland, H. (1986). Some aspects of the organization of repair in lessons. Paper presented at the International Conference on Discourse in Institutions, Dourtmund University, Germany.

McHoul, A. (1978). The organization of turns at formal talk in the classroom. Language in Society, 7, 183–213.

McHoul, A. (1990). The organization of repair in classroom talk. Language in Society, 19, 349–377.

Mehan, H. (1973). Assessing children’s language using abilities: Methodological and cross-cultural implication. In M. Armer & A. D. Grimshaw (Eds.), Comparative social research: Methodological problems and strategies (pp. 309–343). New York: Wiley.

Mehan, H. (1978). Structuring school structure. Harvard Educational Review, 48, 32–64.

Mehan, H. (1979). Learning lessons. Cambridge, MA: Harvard University Press.

Mehan, H., Hertweek, A., & Meihls, J. L. (1986). Handicapping the handicapped: Decision-making in students’ ethical careers. Stanford, CA: Stanford University Press.

Pollner, M. (1979). Explicative transactions: Making and managing meaning in traffic court. In G. Psathas (Ed.), Everyday language: Studies in ethnomethodology (pp. 227–255). New York: Irvington Publishers.

Rosenthal, R., & Jacobson, L. (1968). Pygmalion in the classroom. New York: Holt, Rinehart, & Winston.

Rubovits, P. C., & Maehr, M. L. (1973). Pygmalion black and white. Journal of Personality and Social Psychology, 25, 210–218.

Sacks, H. (1967). Unpublished lectures. Irvine, CA: University of California.

Schaeffer, N. C. (1991). Conversation with a purpose—or conversation? Interaction in the standardized interview. In P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, & S. Sudman (Eds.), Measurement errors in surveys (pp. 367–391). New York: Wiley.

Schegloff, E. A. (1982). Discourse as an interactional achievement: Some uses of “uh-huh” and other things that come between sentences. In D. Tannen (Ed.), Analyzing discourse: Text and talk (pp. 71–93). Washington, DC: Georgetown University Press.

Schegloff, E. A. (1986). The routine as achievement. Human Studies, 9, 111–151.

Schegloff, E. A., & Sacks, H. (1973). Opening up the closings. Semiotica, 8, 298–327.

Schegloff, E. A., Jefferson, G., & Sacks, H. (1977). The preference for self-correction in the organization of repair in conversation. Language, 53, 361–382.

Suchman, L., & Jordan, B. (1990). Interactional troubles in face-to-face survey interviews. Journal of the American Statistical Association, 85, 232–241.

Shuy, R. W., & Griffin, P. (Eds.). (1978). The study of children’s functional language and education in the early years. Final report to the Carnegie Corporation of New York. Arlington, VA: Center for Applied Linguistics.

Sinclair, J. M., & Coulthard, R. M. (1975). Toward an analysis of discourse: English used by teachers and pupils. New York: Oxford University Press.

Ysseldyke, J. E. (1986). The use of assessment information to make decisions about students. In R. J. Morris & B. Blatt (Eds.), Special education: Research and trends (pp. 8–27). Elmsford, NY: Pergamon.

^*Originally published in Qualitative Sociology, 15, 177–202, 1992. Reprinted with permission.

  ¹   See also Cicourel et al. (1974); Heap (1980); Holstein (1983); Mehan (1973); Mehan (1978); Mehan, Hertweck, and Meihls (1986).

  ²   Included here were the Woodcock–Johnson Psychoeducational Battery, the Brigance Diagnostic Inventory of Early Development, the Ongoing Developmental Assessment Tool, and the Psychoeducational Profile. For descriptions of these, see Marlaire and Maynard (1990, p. 85). Special Education assessments rely, by legal mandate, on the use of more than one testing instrument in order to minimize test bias in the reported results. In addition, clinicians choose specific tests on the basis of what they judge to be the target “problem” and/or characteristics (such as the age, emotional or physical disability) of the subject.

  ³   See also Lynch’s (1984) discussion of how the questions in a mental status exam work to “turn up signs” of neurological disease: “An interactional context first had to be created before any signs were available for interpretation” (p. 67).

  ⁴   There is a critical literature concerning these tests, which casts doubt on whether questions and answers do, in fact, offer an indication of subjects’ generalized cognitive abilities. Critics (e.g., Adelman, 1979; Coles, 1978; Ysseldyke, 1986) maintained that these exams are unable to discriminate accurately among identified “learning problems,” primarily for reasons of construct validity. That is, it is claimed that research fails to show that the tests accurately tap the attributes they are said to measure. This concern, however, is not ours, except as what we identify as the “interactional substrate” interferes with the possibility for standardizing the administration of the test, and hence its facility for unbiased measurement.

  ⁵   For a discussion of the idealization of the testing and assessment process as a feature of the academic/scientific literature, see Marlaire (1992).

  ⁶   Until discussing nonvocal activities in more detail, we use transcripts in simplified form.

  ⁷   Although the clinician’s comment, “You know this game, don’t you,” might propose that the child is familiar with the subtest (indeed, there were several instances in the larger corpus of videotaped examinations where children displayed prior knowledge of a specific subtest), clinicians use the “game” metaphor consistently enough across the corpus to suggest more general functions. First, they use the metaphor to engage the child in an activity (such as testing) that otherwise might not be inviting. Second, clinicians thereby indicate that the test idiom has certain game-like qualities. In the blending subtest, for instance, the clinician presents it as a puzzle (“… you have to figure out what I’m saying”). It may be, then, that the consequentiality of the activities is thereby obscured.

  ⁸   One of the effects of stripping away prefatory components during actual testing, Lynch (1984) argued, is to “trivialize,” or press into the background, the interactional accomplishment on which successful adherence to the structure of testing sequences depends. In our terms, “testing” as an observable/reportable activity is truncated from its reflexivity to the interactional substrate, including the instruction and coaching which precede the use of testing sequences proper. It is by way of such truncation that it is possible to analyze correct and incorrect answering as the child’s success or failure at the skill being measured.

  ⁹   In characterizing testing sequences as three-part units, we are following a literature in which a variety of researchers (McHoul, 1978, 1990; Shuy & Griffin, 1978; and Sinclair & Coulthard, 1975) identified and described what Mehan (1979) called an “instructional sequence” (pp. 52–53):

1.  Initiation.	Teacher:	(Holding up card) This is the long word.
		Who knows what it says?
2.  Reply.	Student:	Cafeteria.
3.  Evaluation.	Teacher:	Cafeteria, Audrey. good for you!

In these terms, “testing sequences” are similar to the instructional sequence, except that the third turn is officially to be neutral rather than evaluative. However, we wish to note that there is debate over whether the minimal units of interaction, including instructional sequences, are two or three parts. Schegloff and Sacks (1973) argued that the base unit of sequence construction is an adjacency pair. This adjacency pair can be expanded through the use of pre-sequences (see, e.g., the discussion in Maynard, 1984, pp. 86–87), or through insertion sequences such as those occupied with repair (Schegloff, Jefferson, & Sacks, 1977). If instructional or testing sequences followed the two-part format (question and answer), then the evaluative component is an add-on of some kind. In contrast, Jefferson and Schenkein (1977) proposed that a three-turn sequence (such as appeal, acceptance/rejection, and acknowledgment) may be a base conversational unit or “action sequence,” which also can be “expanded,” and the third turn is an integral part of that unit. (Participants’ orientation to its use for controlling the interactional “so what” of the prior two parts may engender considerable jockeying to be in position to produce the third turn.) We cannot settle here whether instructional and testing sequences are two or three parts. Our strategy, in following the literature on classroom interactions and referring to a three-part testing sequence, is heuristic. Our analysis is meant to be consistent with the phenomena of interaction (such as pacing, as will be discussed), however, they are ultimately parsed in and as the achievements of organized practices such as sequence parts. Although, in the body of our text, we discuss the three-part testing sequence, at relevant points we footnote how these phenomena are to be understood in relation to a hypothetical two-part sequence.

¹⁰   On repair and repair sequences, see Schegloff, Jefferson, and Sacks (1977). For lengthy and technical studies on the operation of repair in classroom lessons, see Mazeland (1986) and McHoul (1990).

¹¹   See Drew’s (1981) illuminating discussion of the instructive features of adults’ corrections of children’s mistakes in ordinary conversation. On how the organization of repair relates to giving instruction in classroom talk, see McHoul (1990).

¹²   If the testing sequence is only two parts—question and reply—then the chained series would be simply a number of contiguous and complete sequences rather than a collection of “collapsed” units.

¹³   Thus, there can be a sequential basis for the sort of thing Grimshaw (1980) identified as “partial” understandings and “mishearings.” As one example, Sacks (1967, p. 8) provided the following:

A:	I went out a lot then. One weekend I went to hear Pete Seeger, the next weekend
	I went to hear Joan Baez, and the next weekend I went to hear Wayne Morse.
B:	Who’s Wayne Morse.
A:	Wayne Morse. The Senator.
B:	Oh. Wayne Morse.

B, displaying recognition of Wayne Morse in the last utterance, evidently “knew” who he was all along. However, hearing the first two persons (Seeger and Baez) in the list to be singers, B may have been listening for another singer, and therefore did not originally identify the politician.

¹⁴   Note that during the “rehearsal” in excerpt (6), which involves CL2 and CH2, the clinician (at line 6) produces “okay” and “good” in tandem (followed by “you’ve got these”). Evaluations, then, sometimes involve more than just one component.

¹⁵   Lynch (1984, p. 78) notes that the use of repair during a mental status exam is also a method for “objectifying” the patient’s response and providing a clinician with an opportunity to closely scrutinize it.

¹⁶   This works both ways, of course. That is, an extracted “correct” answer is seen to reflect the child’s ability, no matter what the contribution of the clinician in coaching it from the child. On how a correct answer can he coached, see Marlaire and Maynard (1990, p. 88). For a treatment of the way that teachers can “clue” children into producing a correct answer during classroom lessons, see McHoul (1990, pp. 355–362).

¹⁷   Lynch (1984) discussed how, in administering “mental status exams,” clinicians came to see patients’ expressions of hositility, unresponsiveness, or disinterest as symptoms of clinical syndromes, whereas we are pointing to the manner in which a variety of accommodative behaviors of both clinician and child, contingently produced according to the exigencies of asking and answering exam questions, dissipate as clinicians render a judgment of correct or incorrect in regard to a child’s response. For a phenomenon similar to Lynch’s (1984), see Lloyd’s (1991) discussion of how therapists, during examinations designed to determine if sexual abuse has occurred, treat children’s lack of responsiveness to questions as evidence of that abuse.

¹⁸   If instructional sequences are two parts, it may be that the addition of an acknowledgment is “holding” the connection between contiguous sequences while the clinician completes her scoring and reading in preparation for vocalizing the next test item.

¹⁹   The complementarity between clinician and child is more complicated than our discussion here might indicate. We have already noted that part of the clinician’s work is listening to and thus being in a position of recipiency with regard to the child’s answering. This is a momentary aspect of the clinician administering the exam, which is why we generally refer to her shows of co-orientation as “administrativeness.” Correspondingly, because the children’s answering is done from a basic position of waiting for the test prompts, we gloss their co-orientational displays as “recipiency.”

²⁰   Immediately after this, CL1 returns to using the third turn of the testing sequence. This seems to occur because of the repair sequence that is inserted after CH1’s reply (line 14 in excerpt 5) to the prompt for “candy” (line 13). CH1 had mimicked the deliberateness with which CL1 gave that prompt. After CH1, responding to the repair initiation (line 15), “says it fast” (line 16), CL1 once more gives an evaluation (line 17).

²¹   Note that if the instructional sequence is two parts, it is not that the third turn is being eliminated, but that an extraneous component, which can be employed to manage the transition from one testing sequence to another, or to provide evaluative feedback when a child seems solicitous of it, can be dispensed with.

²²   See footnote 19 and excerpt (5), lines 13–17.