19

Eye Tracking in Media Studies

Theory, Method, and Its Exemplary Application in Analyzing Shock-Inducing Advertisements

Stephanie Geise

Although the method of eye tracking does not yet constitute a standard instrument of media studies, in recent years it has been repeatedly used for analyzing the reception of media and its effects, and its implementation in research designs is increasing. The improvements in eye-tracking technology, especially the availability of easier to handle, faster, and more accurate eye trackers, as well as the decline of acquisition costs, have inspired eye-tracking studies in the field of media studies. Eye movements have now been established as indicators for drawing inferences regarding psychophysiological processes, such as activation or visual attention, as well as connected cognitive processes – for example, interest in certain contents, motivations, and like or dislike (Duchowski, 2007; Hammoud, 2008). The inherent fascination of eye tracking is “eye catching”: as a technology-based, empirical observation method, eye tracking allows deeper insights into the otherwise hidden reception process of visual perception. Eye tracking can, however, do more than display when and how recipients perceive which visual stimuli visually. According to previous findings, the analysis of eye movements also allows inferences on active perceptive and cognitive processes when visually perceiving photos, posters, advertisements, newspaper pages, Internet sites, or movies.

A particular scientific potential can be seen in the combination of eye tracking with other empirical methods of communication and media science. If one focuses, for example, on the effects of media reception and if the presentation of visual stimuli is accompanied by a questionnaire aimed at capturing changes in cognitive structures, eye tracking helps to identify what information items have actually been perceived by the recipients, that is, what could have caused the potential effects.

Against this background, this chapter explores the founding theories and applications of eye tracking as a method for media studies. The object of analysis – the process of visual perception – is hereby understood as a multilayered phenomenon, to which justice can be done only by a theoretical foundation that addresses all of these layers. Researching with the apparatus-based method, eye tracking therefore necessitates: (1) familiarity with the technical and methodological requirements, (2) psychophysiological knowledge on the human eye and its movements, as well as a deeper understanding of (3) cognitive processes of visual perception and information processing. These interdisciplinary facets constitute the theoretical basis on which the implications for operationalization, analysis, and interpretation of the eye movement data are established.

An in-depth discussion of all the relevant theoretical, methodological, and empirical results is, however, beyond the scope of this chapter. After providing a synopsis of the theoretical and methodological foundations of the method, this chapter will instead focus on the application of this method in media studies, which will be illustrated via a case study.

Eye Tracking: The Method

When Duchowski (2007, p. 3) notes that

he points to the origins of the research field of human eye movement analysis. Eye tracking allows for the apparatus-based recording of eye movements. This is what characterizes eye tracking as a unique approach, since it is the only empirical method to date to provide an exact reconstruction and analysis of this process. With this research focus, eye tracking differentiates itself from other physiological measurements of the visual perception apparatus that examine not the extra-ocular muscle movement of the eye, that is, the eye movements, but single parameters such as the change of pupil size (“pupillometry”; Backs & Walrath 1995), movements of the eyelids (“eyeblinks”; Stern, Walrath, & Goldstein 1984) or changes in the accommodation condition (Piccoli, Perris, Gratton, & Grieco, 1985). Furthermore, in the recording and analysis of eye movements via eye tracking the visual stimulus or fixation target is, by definition, recorded and integrated into the analysis (Duchowski, 2007).

Eye-tracking research can therefore be specified as apparatus-based, fixation-dependent analysis of individual eye movements, which enables the researcher to qualitatively and quantitatively describe, analyze, and interpret them. Through this, eye tracking provides rather precise indications as to what content recipients foveally fixate on during the reception of the visual stimulus (i.e., focus with their fovea centralis, the eye's center of concise visual perception), with what intensity they visually turn toward the presented content or information, in what temporal and contextual order the observations are made, and where no visual attention is allocated (Gale, 2003). In contrast, eye tracking provides no or only little potential for clarification on what content recipients perceive during the reception of the visual stimulus in the peripheral or para-foveal perception space. Additionally, it cannot reveal why, or with what intention or motivation, recipients turn toward visual stimulus areas, what they think and feel when they turn toward them, or what subsequent emotional and cognitive processes are connected to the reception of the respective visual stimulus areas.

Despite these limitations, and because of its potential to cast light on the process of visual perception, eye-tracking methodology can be applied in a vast number of research fields, from physiologically based studies that explore the fundamental mechanisms of visual perception to advertising research and usability studies. For example, eye tracking has been used to study the interaction of eye and eyelid movements (Gandhi & Katnani, 2011) and the saccadic decision-making process (Ludwig, 2011), and has been implemented in research that aims to explain eye movement control during reading (Rayner, 1998; Vitu, 2011) and language processing in dialogues (Kreysa & Pickering, 2011). Another research field, connected to information technology and computer science, operationalizes eye tracking to study the possibilities of optimizing human–computer interaction, such as in hardware or software usability studies (Ehmke & Wilson, 2007). For media studies the method possesses enormous potential with regards to the reconstruction and analysis of visual perception and processing and can be utilized in many ways in the examination of questions directly originating from the reception process (Geise, 2011; Geise & Schumacher, 2011); for example:

1. The distribution of attention with regards to duration and frequency of defined elements (“areas of Interest”) of a visual stimulus, for example, a web page, a newspaper page, an election poster (Adam, Quinn, & Edmonds, 2007; Goldberg, Stimson, Lewenstein, Scott, & Wichansky, 2002), a movie or a video sequence (d'Ydewalle, Desmet, & Van Rensbergen, 1998; Hughes, Wilkens, Wildemuth, & Marchionini, 2003).
2. The allocation of visual attention: its common structure as well as its specific patterns on a media stimulus in relation to its graphical design and visual presentation (Geise, 2011; Holmqvist, Holsanova, Barthelson, & Lundqvist, 2003; Holmqvist & Wartenberg, 2005; Josephson, 2005; Stenfors, Morén, & Balkenius, 2003).
3. The structure of human–computer interaction and its specific patterns during the use of websites, in relation to their visual design (Grier, 2004; Jacob & Karn, 2003), especially in usability research (Goldberg & Wichansky, 2003; Manhartsberger & Zellhofer, 2005).
4. The temporal and spatial characteristics of functionally different phases of reception such as orientation, scanning, or reading (Bucher & Schumacher, 2006; Holmqvist et al., 2003).
5. Processes of visual perception and cognitive information processing in the reception of complex visual or hypertextual offerings (Holsanova, 2011).
6. Processes of visual perception and cognitive information processing, also with regards to the analysis of disorders in information processing and understanding, for example when using multimodal e-learning programs (Dogusoy & Cagiltay, 2009; Shukla, Wen, White, & Aslin, 2011; van Gog & Scheiter, 2010; Zambarbieri, 2003).
7. Perception, attention, and interaction patterns in virtual worlds such as Second Life (Triesch, Ballard, Hayhoe, & Sullivan, 2003; Walter, 2009) or while playing with a computer (Joensson, 2005; Kickmeier-Rust, Hillemann, & Albert, 2011).

Nevertheless, the productive potential of the method is far from exhausted. In the field of media studies, in particular, the implementation of eye-tracking methodology still is an exception; and thus, a limited number of eye-tracking studies exist addressing the “classical questions” of mass media, their perception and effects. However, most studies implementing eye tracking share the same theoretical assumption: visual perception is understood as a fundamental precondition off any cognitive or emotional effect that a visually perceived stimulus can induce. Following this idea, the recipient's actual eye movements can be segmented into a set of relevant indicators that reflect the medium's inherent effect potential. Taking the influence of individual predispositions as well as situational factors and stimulus-specific features on eye movements into account (Radach, Lemmer, Vorstius, Heller, & Radach, 2003), the visual characteristics of the stimulus have to be seen as highly important in the perception process, as they hold the potential to modify or even control the recipient's eye movements. Implementing these ideas in a scientific setting, one can propose that differences in the nature of the visual stimulus – for example, visual versus textual communication mode, unimodal versus multimodal, or dynamic versus static content – should have observable effects on the recipient's eye movements that are, via eye tracking, recordable and reconstructable. Nevertheless, it has to be considered that eye movements only reflect intentional selection and perception processes (Theeuwes, 1993). While the sensorial visual perception is functionally operated by the eyes, the seeing itself has to be understood as a cognitive construction that is guided by our brain: “The eye is not innocent, because depiction cannot be explained by perception alone; sight is informed by . . . the cognitive” (Lopes, 1996, p. 32).

Thus, researching with eye-tracking methodology involves more than utilizing an eye tracker as a specific technical device; setting up the research design necessitates a deeper knowledge of the technical operating mechanisms of eye-tracking processes and their methodological requirements as well as knowledge about visual perception and information processing. This is no less true for the processing and analysis of the collected data. The next section briefly summarizes the most important fundamentals of apparatus-based recording of visual perception through eye tracking.

Structural Components of Human Eye Movements: Fixations, Saccades, Micro-Movements

“We have the subjective impression of an immediate, full detail, pictorial view of the world. We are prone to forget that this impression is, in a very real sense, an illusion” (Findlay & Gilchrist, 2003, p. 2). This statement not only alludes to the subjectivity and selectivity of perception, but also to the fact that the human vision system by no means delivers a permanent and complete perception, even if we have this impression. Although the retina is nearly completely covered by photoreceptors, an individual can see well only with a small part of it: the fovea centralis, an area of most focused sight only about 0.5 mm wide, covers only around 1% of the field of vision. In order to achieve precise visual perception the eye is controlled by extra-ocular muscle movements so that the object of interest is in the sight of the fovea and can be perceived with maximum resolution.

Eye-tracking research has identified several factors that influence human visual perception and eye movements (Henderson, 2003). For example, our visual attention – and, therefore, our eye movements – is attracted to suddenly appearing objects, visually salient pieces of information, and distinctive and “new” visual stimuli – even if these materialize only in the peripheral field of view. Thus, eye movements are assigned the central task of preparing foveal perception and of providing the necessary orientation information from the peripheral field of sight (Findlay & Gilchrist, 2003).

Figure 19.1 Visualized scan paths of five randomly chosen subjects of a Dior advertisement poster (2010): eye-tracking measurement of 4 seconds; scan path of each person in one color (analysis with Tobii Studio)

Every eye movement can be structured as a scan path that is constituted through at least three basic structural components (see Figure 19.1): (1) the fixations as centered focusing of the fovea on a visual object; (2) the saccades, which serve through the orientation of the eye as a preparation for foveal fixation (and usually result in a fixation); and (3) micro-movements (especially micro-saccades), which are mostly undirected and serve physiological fixation control in a situation of spatial movement. In addition, vestibular eye movements¹ are also to be expected. Particularly important for eye-tracking measurements is the differentiation of fixations, where the gaze rests for about 300 milliseconds – and which are recorded by the eye tracker – and saccades, where the eye “jumps” extremely quickly to the next fixation destination (Hoffman & Subramaniam, 1995). Saccades are not directly recorded by the eye tracker, but are technologically reconstructed later, as during these jumps top speeds of up to 1000 degrees per second are measurable. In fact, saccades are the fastest movement the human body is capable of (Fuchs & Kaneko, 1981; Gilchrist, 2011). On average, about three to five saccades happen every second. Thus every day the eye is moved in tens of thousands of saccades occur for the most part unconsciously although they are generally directable. Even during a fixation, however, the eye is not completely at rest: more or less undirected micro-movements can be detected, which happen several times a second (Duchowski, 2007; Gilchrist, 2011). The illusion of a continuous optical perception of our environment is supported by two factors: (1) the extraordinary integration and construction ability of our brain; and (2) the high adaptability of the human vision system. There are, however, strong interdependencies between eye movements, as a “fundamental feature of vision” (Findlay & Gilchrist, 2003, p. 1), and the entirety of visual information processing. For most of the common eye trackers micro-movements are not detectable – they are recorded as part of the corresponding fixation. However, as a micro-movement's function is primarily to reposition the fovea, for most questions of communication and media science they are of subordinated interest anyhow and can be neglected.

In sum, the scan path cannot be assumed to be linear and sequentially stringent, it is best conceived as a loose, more or less intended zigzag pattern. This structure becomes evident once the scan path has been recorded by eye tracking and graphically reproduced. A visualized result of an eye-tracking analysis of several subjects, translated into fixation points and saccadic lines, is shown in Figure 19.1. As mentioned, the micro-movements that are summarized in the fixation points are not depicted.

The information intake of the recipient occurs during the fixations, which occupy around 90% of the whole viewing time of visual perception. Paradoxically, although fixations (or, to be more precise, the indicators derived from them) play a central role in the analysis of eye movement, fixations in themselves do not constitute a movement of the eye as such. A fixation is defined as the condition where the eye is at a relative standstill compared to a perceived object. The minimum fixation length is often assumed to be around 100 milliseconds; average fixation lengths are a multiple of that. During the approximately 30–90 milliseconds that a saccade lasts no intake of information is possible. Here, the phenomenon of saccadic blindness occurs: around 30 to 40 milliseconds prior to and up to 120 milliseconds after the start of a saccade² – for very short saccades, this is during the first fixation phase – visual perception capabilities are extremely limited (Gilchrist, 2011; Hoffman, 1998). Saccades only allow the recipient a rough orientation in the perception process (and even that is a matter of discussion). For Findlay and Gilchrist (2003, p. 7) saccades can nevertheless be considered an active system: as not only are they a visually controlled motor response but their operation also controls the input of visual sampling, their involvement with vision takes the form of a continuously cycling loop. Before eye movements are started, the information that is visually perceived on the peripheral or foveal field of the retina has to pass numerous stages of visual information processing, which start on the retina itself but also involve higher cognitive processes, especially in the visual cortex (Bullier, 2001). Accordingly, visual perception and visual information processing are obviously more than just the inner reflection of the world that is visually represented outside: vision and cognition interact in a highly intimate way.

For this reason – and from which the explanatory power of eye-tracking research derives – the recorded eye movements allow inferences on active perceptive and cognitive processes during visual media stimulus perception. The most apparent inference is the connection between eye movements and visual attention, as the scan path offers information about where visual attention is absorbed and for how long (Deubel & Schneider, 1996; Just & Carpenter, 1976). Posner (1995) argues that the visual attention of a recipient is “bound” during the observation of a fixation target (attentional engagement). Prior to a saccade, and thus a new positioning of the fovea, the visual attention has first to be shifted from the original fixation target and redirected (attentional disengagement) (Posner, 1995). Only this redirection of visual attention can lead via saccadic control to the next fixation target: without attentional disengagement, there would be no saccade (Findlay & Gilchrist, 2003).

Exogenous and Endogenous Control of Visual Perception

While the fovea is centrally directed onto an object during fixation and temporally stabilized in this position, directly afterwards – due to saccadic movements – a new positioning follows: When visual attention is directed to a new area, saccades reposition the fovea. Here, the interaction of attention, visual perception, and processing can (in simplified terms) be interpreted as an attentional feedback loop. Fixations thus do not occur arbitrarily but can rather be seen as intended manifestations of the conscious or unconscious interest or motivation to look at certain objects more elaborately (McLeskey, Levie, & Fleming, 1982). In this process (visual) attention presents itself as a central predictor for the rhythm of saccades and fixations (Proulx, 2007; Wright, 1998), with individual stimulus-specific and situational factors modulating the scan path (Godijn & Theeuwes, 2003). An object suddenly appearing in the field of vision will, for example, automatically receive attention and thus focusing, even if the perception remains irrelevant for a subsequent course of action. This affective intuitive mode of external gaze direction is also known as exogenous control or stimulus-driven perception (Proulx, 2007). Exogenous reactions to perceptions due to stimulus reaction schemas are considered relatively uncontrolled (Schneider & Maasen, 1998; Yantis, 2002); they prove to be highly invariant across individuals. As this bottom-up process seems to reflect automatic, evolutionary, and biologically programmed mechanisms that are deeply rooted in human perception behavior, the bottom-up perception often is interpreted as one basic “component of natural human vision” (Duchowski, 2007, p. 162).

Bottom-up perception can be placed in contrast to endogenous control, or goal-driven perception – a consciously intended perception process (Godijn & Theeuwes, 2003). The latter is based on specific action or perception goals, or the expectation of them. Already the early scan path analyses of Yarbus (1967) had shown prototypically, that scan paths can diverge significantly depending on individual expectations or requirements. A complete theoretical foundation of the interaction between visual attention and eye movement therefore exceeds solely affection-motivated viewing behavior and incorporates higher cognitive functions into the analysis (Gordon, 2004) – as, for example, the intention of a visual search task. Visual perception is thus not only a simple bottom-up process, as was believed in the early stages of visual perception research. Instead, understanding vision as a complex interaction of endogenous and exogenous perception processes that are guided by cognitive mechanisms of intentional strategic control (Proulx, 2007) seems to be more appropriate.

What Fixations and Saccades Tell Us

Eye tracking has a great potential to objectively uncover the structure and hierarchical sequence of visual perception as well as the allocation of visual attention. Based on this, it also allows conclusions on higher cognitive tasks. However, it cannot directly reveal the processes happening within the recipient. For these mental processes, the scientist remains a passive observer and can only partially uncover what the recipient is doing, why he or she is doing it, and what he or she is thinking and feeling while doing it. Thus, although fixations and saccades can tell a lot about visual perception and information processing (see below), their deeper interpretation requires further contextualization. Eye movements are unspecific in their meaning unless the eye-tracking data are connected to the specific research context.

However, in line with previous findings, eye movement data allow some general conclusions on the allocation, depth, and order of visual and cognitive processing steps, as strong interdependencies exist between eye movement and visual information processing. One of the central premises of the interpretation of eye-tracking data is that visually perceived stimuli are instantaneously processed. According to this immediacy assumption, fixation times are often used as a measure for the depth of cognitive processing. The processing length is consequently identical to the fixation length: “The immediacy assumption posits that an attempt to relate each content word to its referent occurs as soon as possible” (Just & Carpenter, 1980, p. 341; see also Carpenter, 1988, 1991; Carpenter & Robson, 1999). Moreover, as demonstrated by studies by Just and Carpenter (see, for example, Just & Carpenter, 1980) on reading behavior, fixated objects are instantaneously processed and the cognitive process lasts exactly as long as the recorded fixation (eye–mind assumption) (Just & Carpenter, 1976, 1980). The fixation length is thus equivalent to the duration of central processing. Finally, conclusions on the hierarchy of mental processing can be drawn from the analysis of the scan paths, as the sequence of fixations allows a reconstruction of the order of central processing steps (eye–mind sequence assumption).

Various overviews of different physiological parameters describe eye movements as a highly sensitive indicator for processes of mental demand (Carpenter, 1988; Carpenter & Robson, 1999). Also, the general existence of a momentarily occurring “looking without seeing” – as can be observed in “dreamy behavior” or “lost in thought staring” – does not in a narrow sense constitute a limitation of these eye – mind premises. Finally, the immediacy assumption and the eye–mind assumption do not have to prove true for every single fixation but for the sum of all fixations. And last but not least, most of the eye-tracking research settings make a “dreamy behavior” during the reception process relatively unlikely.

However, all of the presented principles have to be adapted to the particular research context and have to be critically evaluated against the background of the specific research questions and the implemented methodological setting. In the course of the data interpretation, for example, one usually assumes that the fixation and observation length can be interpreted as an indicator of cognitive processes or respectively processing performance. However, it remains to be asked which specific cognitive processing tasks occur during the fixation of individual visual information items and which cognitive interdependencies exist: “Although a given cognitive event might reliably lead to a particular fixation, the fixation itself does not uniquely specify the cognitive event” (Hayhoe, 2004, p. 268).

Additionally, although the majority of stated premises are in line with the fundamentals of visual communication and its perception (Geise, 2011), researchers should critically reflect that most insights are based on studies concerning the analysis of visual perception while reading. While only a few strategies for scene viewing have been discerned, in contrast to reading behavior, there seem to be no established eye movement canon for particular visual objects and images (Duchowski, 2007). As the perception and information processing of visual images may function differently in reading and in real-life scene perception some theoretical implications and/or their limitations may require further adaptations. For example, while a linear, left-to-right scan path would be expected during reading in the Western world, it could not be predicted during image perception, where the holistic reception process of images would rather result in a zigzag structure (see Figure 19.1). From this, three methodological consequences can be derived: (1) there is a need for research that addresses theoretical and methodological premises and their specific explanatory power in terms of the reception of image content; (2) the study of (visual) media content may need further theoretical and/or methodological adjustments; and (3) in this modification process all implemented adaptations have to take the specific research context into account.

Additionally, it is important to note that scan path implications, and thus the research parameters derived for their study (see Table 19.1), have not yet been fully explored and consolidated. Thus, the interpretation of eye tracking-data should be considered critically against the background of the specific research questions and/or research design at hand, especially since few studies address methodological challenges (as the above mentioned) when utilizing eye tracking. Depending on context, even seemingly “unambiguous” indicators, such as, for example, the interpretation of observation lengths, can at the very least be rather confusing. This is another reason why a commonly shared standard of research indicators and their interpretation is developing only slowly (Granka, Feusner, & Lorigo, 2008).

Table 19.1 Overview of the indicators of eye-tracking analysis

Indicator	Implication of the indicators
FIXATIONS AND FIXATION-BASED INDICATORS
Time to first fixation	Provides evidence on the activation potential of an area of interest; short time to first fixation signifies high activation potential
Observation length of fixation	Shows how intensively or how long the eye keeps a specific area of interest fixated, and thereby the allocation of attention. Longer observation length makes a higher degree of information input probable.
Fixation count	Measure on how many fixations are attributed to a certain area of interest. A high fixation count signifies a high degree of interest or activation potential; it can, however, also be a sign for a (too) high complexity of this area.
SCAN PATH AND SCAN-PATH-BASED INDICATORS
Scan path position	Identification of typical entry points into the observation process; provides information on activation potential or degree of activation of a stimulus area and – derived from that – optimization potential for scan path control
Saccadic amplitude/length	Indicator for the sensitivity of peripheral perception and the attention window of para-foveal perception
Distribution of fixations	Measure for different modes of information processing, task-connected skills of a recipient, the conciseness of a target object as part of the visual stimulus or the difficulty of a searching task
Backtracks	Often used in reading research; most times interpreted as a processing difficulty
Transition matrix	Provides evidence on typical scan paths or irritations in the stimulus
Scan path duration	Measure to describe the duration of a global visual search or solution to a task
Scan path length	Comparable to scan path duration; measure to describe the duration of a global visual search or solution to a task
SACCADES AND SACCADE-BASED INDICATORS
Fixation saccadic ratio	Indicator for the requirements on information processing
Saccadic count	Refers to the quality of visual search and visual processing
Saccadic duration	Allows conclusions concerning visual attention and mental presence/fatigue of the recipient
Saccadic velocity	Provides evidence of vigilance and cognitive strain
Saccadic latency	Indicator for the “visual reaction time”; is reflected in the influence of arousal mechanisms on scan path control
Saccadic length	Indicates extended orientation reactions or stimulus areas with higher visual conciseness

Nevertheless, a few key variables can be identified that can serve as significant indicators of human eye movement analysis. Table 19.1 provides an overview of these indicators, which are of special interest to questions of communication and media science (for a more detailed overview, see, for example, Ehmke & Wilson, 2007; Goldberg & Kotval, 1999; Jacob & Karn, 2003). It is noteworthy that the majority of the existing “aspects of eye movements” are conceptualized via the level of allocation of visual attention (Radach et al., 2003). It is thus implicitly assumed that the allocation of attention can be interpreted as a function of the motivation of the recipient, his or her interests, goals and capabilities, which can, in turn, be significantly modified by factors such as the perception situation or stimulus-specific characteristics. As visual attention can be controlled endogenously as well as exogenously, the possibility exists – particularly in early phases of visual perception (Bucher & Schumacher, 2006; Geise, 2011) – that visual attention is also activated independently of the individual perception intention through properties of the stimulus (thus also, stimulus-driven) (Godijn & Theeuwes, 2003). On the basis of this assumption individual stimulus-specific and situational factors modulate the allocation of visual attention – the structure of (exogenous) visual perception should (at least partially) be interpretable as a function of the stimulus.

In this interpretation, however, it has to be taken into account that the common distinction of perception being unambiguously either exogenous or endogenous has to be questioned (Bucher & Schumacher, 2006). Although this distinction offers some explanatory power, it neglects possible interrelations between exogenous and endogenous perception control: Where does endogenous control of visual perception start? Where does exogenous control end? Where do the two interact? Given the interdependency of endogenous and exogenous perception, what does that mean for visual perception and its processes? How might that impact its empirical analysis? How can one differentiate between them analytically or, to be more concise, which elements constitute themselves in a scan path on the basis of stimulus-driven perception and which on the basis of individual preferences? Potentially, the empirical “inability” to capture factors of endogenous control (for example, inherent influences of an individual's characteristics) has to be attributed to the fact that existing measurement methods may not be capable to sufficiently record the interdependencies between endogenous and exogenous control during visual perception and information processing.

With the theoretical and methodological foundation set we will now explore the method through its implementation in a case study – a study on shock communication. We will also critically reflect on the opportunities, challenges, and limitations the method presents.

Setting Up an Eye-Tracking Study: A Case Study on Shock-Inducing Advertisements

Eye-tracking technologies which allow the concise recording of an recipient's eye movements are commonplace today – including table-mounted eye-tracking systems that operate through video-based measurements of the corneal reflex of the retina. (Duchowski, 2007; Hammoud, 2008). These technologies strive to influence subjects' behavior as little as possible and to work without hindering and obtrusive devices. For most of these technologies, the optical impression is equivalent to a modern PC monitor (as, for example, in the Tobii system; see Figure 19.2). A monitor-integrated eye tracker like this, which offers a “natural experience for the subjects” (Hammoud, 2008, p. 3), was used for the case study at hand. It remained close to undetectable for the subjects by operating completely without physical contact, creating a relatively naturalistic, though still experimental, setting. While more invasive systems – where subjects carry the eye-tracking technology on their bodies – usually exhibit slightly higher measuring accuracy, accuracy is not the only decisive factor when deciding on a particular eye-tracking system. The type of stimulus material is also critical: the noninvasive processes whereby a camera is integrated into a screen are especially appropriate with media that can be depicted and observed on a screen. If, on the other hand, printed or bounded magazines or newspapers are to be examined, relatively more invasive head-mounted systems may be required. In our case, the treatment consisted of full-page, shock-inducing advertisements, a kind of stimulus for which passive table-mounted systems are particularly well suited.

Figure 19.2 Stationary table-mounted eye tracker Tobii T120 in a data-recording situation during a study on the reception of shock advertisement (University of Hohenheim study)

Regardless of the technology used, the eye-tracking process starts with the individual localization of the eyes of the subject: for this purpose the subject sits on a stationary chair in a slightly dimmed eye-tracking laboratory and looks at the eye-tracking screen, which is placed on a table in front of him or her at a distance of 40–50 cm. Although most passive eye trackers allow for small movements of the body or head, the subject should move close enough to the edge of the table so that he or she lightly touches it, and the body is stabilized by putting the arms on the table-top. In order to adjust the individual vision characteristics of the subject to the eye-tracking system as well as to the stimulus material – which has been fed into the eye-tracking software – an initial calibration procedure is conducted. Subjects are asked to focus on five to seven points and to follow them as they move across the screen (for additional details on the adjustment and fine tuning of the calibration with different eye-tracking systems, see Duchowski, 2007; Hammoud, 2008).

Subjects' eyeglasses or contact lenses do not usually negatively influence the calibration process and, after a short acclimatization phase, the actual eye-tracking process can be started. First, the already uploaded stimulus is presented to the subject for a preset time on the eye-tracking monitor – in our case advertisements with shock-inducing motives. After a brief introduction to the process and individual calibration of the eye tracker, the subjects were exposed to a treatment series of 11 advertisement motives including some aiming for a “shock effect,” in the sense of a visualized, intended violation of the norm (see Figure 19.3).

Here, the test scenario, that is, the context in which the subject is confronted with the stimulus, plays an important role. The early studies of Yarbus (1967) have shown that the task given to a recipient looking at a visual stimulus exerts an influence on the reception situation and thus on his or her viewing behavior. The explicit or implicit task provided to the recipient may cause significant differences in the explanatory power, the processing, and the interpretation of the results. If subjects, for example, are given advice to seek out certain information, this will induce an intentional visual search, while the task to “just get an overview” may trigger holistic perception patterns. In our study, subjects were asked to “just look” at the images; the instruction was: “In the following you will see a series of advertisements. Please just have a look at them.” The methodological setting also has to incorporate the decision about the reception time given to the subject. Here as well, the decision is mainly dependent on the research question. In cases where the average natural reception time of the stimulus is known – as was the case for our advertisements – and if the simulation aims for realistic, “cursory” reception, it may be a good idea to restrict the exposition time to this time span. In most cases, the exposition time can be set down to the millisecond in the eye-tracking system and can be easily varied across stimuli or subjects.

During the reception, infrared diodes record the eye movements by the millisecond by measuring the corneal reflex, the reflection of the infrared diode in the eye, or more specifically in the eye's cornea. In order to produce the corneal reflex, during the experiment the subject's eye is exposed to weak infrared light emitted by the diodes (near-infrared light (NIR)). Also, integrated highly sensible LED camera(s) record the subject's eyes at the same time. The analysis software of the eye tracker (in our example Tobii Studio) then processes the recorded information to allow a reconstruction of the scan path. Our Tobii T120 eye tracker works on a frequency of 120 Hertz, which means that every second 120 values are captured. Thus, fixation lengths from a duration of 8 milliseconds and above are registered, allowing for quite accurate measurements through the fixation coordinates and their respective time stamps (start and end times). Shorter fixations remain “invisible” to the eye tracker.

Figure 19.3 Original stimulus: Eastpak advertisement poster from the Built to Resist campaign (2009)

In a second step, during the output phase – that is, the visualization of the data through the analysis software – these raw data were filtered according to the structure of the individual scan path and interpreted as fixations and saccades (Figure 19.3; on modeling approaches see Duchowski, 2007). Saccades, which are actually invisible to the eye tracker, are not recorded, but rather constructed in hindsight between the fixation points, when the gap in the scan path is completed through the saccade pattern. Figure 19.4(a) shows an example of such a reconstruction of the scan path: a visualized result of the recorded eye movements of the subject in our case study, translated into fixations (visualized as the “bubbles”) and saccades (visualized as the “lines”) at a standstill.

Figure 19.4 (a) Gaze plot: scan paths of 12 randomly selected subjects projected onto the stimulus with an exposure time of 4 seconds; (b) heat map: visualization of the allocations of the attention of 12 subjects in an “attention landscape” projected onto the stimulus with an exposure time of 4 seconds

Another typical analysis of the reconstructed eye movements aggregates, for example, the number of fixations (or, alternatively, their combined lengths) as a fixation indicator by intensity and time into hot spots of a heat map and superimposes this map in the visual display on the original stimulus (Figure 19.4(b)). Heat maps show the gradual, aggregated fixation density as a cluster of fixations and attention landscapes for a given (selected) reception time. A high fixation density leads, as Figure 19.4(b) shows, to a hotspot overlapping the zombie's face, which denotes an intensively gazed upon area. This depiction of eye movement is quite close to the actual perception space: in reality as well, the foveal visual attention is not directed at discrete geometrical points but rather at larger or smaller areas of the visual stimulus. In our example, the objective recording of the recipient's eye movements enables us to answer research questions that are connected to the visual perception process, for example: What are the area's recipients focused on first? In what order were these areas perceived? How long was visual attention focused on which areas? Do shared or individual patterns of perception occur? Where are the areas that were not perceived at all? Conducting a first eyeball analysis, one can already deduce at first sight from the heat map and gaze plot that, for example, the shock-inducing area (the zombie's bloody face and neck) has not only been fixated first, but also most intensely, and that atypically even the Eastpak logo at the bottom of the advertisement received some fixations.

Of course, for further analysis (for example, regarding differences in viewing patterns in relation to age, sex, involvement or shock resistance) the recorded data have to be compressed and aggregated. With regards to the eye-tracking indicators discussed above, we first have to decide, on the basis of the specific research question, which indicators are to be operationalized in what way. In our case study, which analyzes visual perception, processing, and evaluation of the shock-inducing advertisement, it can, for example, be assumed that – regardless of a recipient's sociostructural characteristics – a shock-inducing stimulus area carries a very high visual salience and is comparatively intensively perceived foveally. Research findings that emotionalizing stimulus elements affect visual attention and therefore eye movements point in a similar direction. Independent of other factors, recipients have been shown to look at emotionalizing elements first and for longer than at “neutral” elements (Alpers, 2008; Kissler & Keil, 2008). Here, indicators such as time to first fixation, observation length, and the type and number of backtracks (i.e., a sudden saccadic change in direction of more than ±90° in relation to the preceding saccade) can be utilized for empirical testing of such research assumptions: while time to first fixation provides insights into the potential of an area to activate a recipient's visual attention – the shorter the time to the first foveal contact, the higher the visual attention potential – a measure of how intensively or how long a recipient foveally fixated a particular area is provided by the observation length of fixations. It thus allows insight into a person's visual attention allocation as well as his or her connected information-processing activities. For example, a backtrack can be seen as an indicator for specific irritations in the scan path. In our case study, such a repetitive visual “drift” toward a shock-inducing stimulus element and a resulting accumulation of backtracks was interpreted as an expectation-incongruent perception pattern that induces surprise and thus results in repeated fixations.

Nonetheless, the researcher also has to take into account that most eye-tracking data are in the first place unspecific and open to ambiguous interpretation – as in many other research fields where psychophysiological measurements are utilized to draw inferences regarding mental processes. The visual characteristics of the stimulus, the recipient's goals and intentions, as well as the meaning with which the stimulus is associated, influence where recipients look and how their eye movements are structured. In order to determine and understand what recipients are looking at, their visual information processing should be connected with their mental structures, their prior experiences, and their knowledge.

Analysis and interpretation of eye movements therefore require contextualization to gain specific insights into the reception, processing, and effect mechanisms of the underlying visual perception. For most questions of communication and media science, employing the eye tracking method on its own may therefore not be appropriate; this is also true for our case study dealing with the visual perception, processing, and evaluation of shock-inducing advertisements. More complex perception and effect analyses require that eye tracking be combined with other empirical methods of communication science (Geise & Schumacher, 2011). By recording additional perception and reception data and using them for eye movement data analysis, difficulties in the interpretation of the data – especially with regard to cognitive operations – can be reduced. Likewise, post-receptive questionnaires, considered an established instrument of social science, allow the collection of facts, knowledge, opinions, attitudes, or evaluations. Qualitative interviews, in particular, have the potential to complement the quantitative perception data by post-receptive oral statements of subjective patterns of meaning or sense. In our case study, we supplemented eye tracking with qualitative interviews. This method combination offers two major advantages within our research context: (1) the inclusion of the subjective perspective of the recipient as an additional source of information for the interpretation and evaluation of the current eye-tracking findings; and (2) the ability to draw conclusions on more profound interpretations, on subjective meaning, individual sentiments, associations, motives, or thoughts. As in our case, this can make sense if it is the researcher's intention to compare the eye movements to the subjective opinion of the recipient concerning his or her perception or his or her individual experience with the stimulus.

In contrast to a standardized quantitative survey, the qualitative interview is structured more openly. It is more flexible and more details can be captured that are of importance to the interviewee (Lamnek, 2010, p. 311). Most of the research questions, where a method combining the analysis of eye movement data and of the underlying subjective pattern of meaning and sense seems appropriate, are based on the idea that the interpretation of the recipient's (viewing) behavior and actions requires a deeper understanding of his or her thoughts, feelings, and intentions.

Here, the distinction of intentional versus nonintentional viewing behavior poses a huge challenge, as in this context it cannot be plausibly argued that the subjective patterns of meaning can explain the actual scan path. In a narrow sense of the word, one cannot even describe viewing behavior as an action, as not only does it happen to a large degree unconsciously (the prerequisites of intentionality could just be assumed for the phases of endogenous, consciously intended gaze control), but it is also controlled by the stimulus. It is therefore not the case that the documentation, reconstruction, interpretation, and explanation of the constitution process of social reality captured by the interviews (Lamnek, 2010) reflect the actual viewing behavior or the process of visual perception. In contrast, as in our case study, post-receptive qualitative interviews can be operationalized to deepen understanding of the recipient's individual processing, classifying, interpreting, and evaluating of the observed content against the background of his or her individual relevance system. Thus, if the eye-tracking data reveal that a particular visual element has proven to be salient or relevant for the recipient, the qualitative interview data can specify his or her associated meanings, spontaneous thoughts, reports of pleasure or liking – and thus, the researcher can learn more about the underlying cognitive processes when perceiving the tested visual stimulus.

In our case study on the perception, interpretation, and evaluation of shock-inducing advertisements, this fact was considered of central importance. Therefore, after exposure to the advertisements, a post-receptive qualitative guideline interview followed. Specific questions on the process of visual perception or the scan path were not asked. Instead, the understanding, the interpretation, the construction of meaning, the classification, and the ethical-moral evaluation of the treatment was queried. Here, we asked whether and which taboos were evoked by the shock advertisements, whether any taboos were violated, what spontaneous associations and feelings were generated, and whether and what incentives for action were given. The results of the qualitative interview thus did not deliver additional information on what information was foveally fixated upon or why it was fixated upon. However, additional insight was provided on how the foveally fixated information was perceived by the recipient, how it was understood, associated, and interpreted. In addition, the results of the qualitative interviews were able to provide some indication on the identification of certain types of recipients (e.g., shock-affine, shock-neutral, and shock-averse recipients).

Figure 19.5 exemplifies the gaze plot of a shock-averse recipient who declared in the guideline interview “not being able to see blood.” As he/she put it, “honestly, that may sound childish, but it's the case, I just cannot look at it, basta.” After the first three fixations on the shock motive (and after less than half a second) the subject directed his/her gaze onto the black screen and let it stay there for about two seconds.

Although the comparative analysis of scan paths could only be conducted exploratively in this study with regards to significant differences between reception types, the findings provide a good foundation for a follow-up study. It remains to be critically reflected that the two methods, eye tracking and qualitative survey, are approaches from two divergent angles – a true synchronization of the data is neither methodologically nor theoretically plausible. It is the combination of insights from two subsequent perception and information-processing levels that can complement each other in their explanatory power.

Methodological Limitations and Challenges

When it comes to the practical application of the method of eye tracking in the research process, eye tracking can in many ways enrich empirical media studies. As we tried to illustrate with the case study, it possesses an enormous explanation potential, especially in the examination of questions directly connected to the process of visual perception and visual information processing. Eye tracking offers insights that other empirical methods cannot provide. As a first research interest, it reveals the structure and hierarchical sequence of visual perception and the allocation of visual attention. The second research interest, which is based on this, is that eye-tracking data allow conclusions on higher cognitive tasks, especially the underlying processes of visual information processing. Thus, even though it is not capable of unveiling processes happening within the recipients (which can be compensated for through method combination only to a limited degree), the researcher can draw some reflected inferences on internal processes, as visual perception and cognitive information processing are highly interrelated. Through eye tracking, the observation and analysis of media reception take on a new quality. Especially in combination with other methods, conscious and unconscious reception processes can be reconstructed in a way that is very close to the processes themselves.

Figure 19.5 Screenshot of the scan path analysis of a shock-averse recipient in the analysis software Tobii Studio shows “avoidance behavior”

Consequently, there are many interdisciplinary fields of media studies where eye-tracking research can provide new insights. One large research field in which it could be of particular interest is e-learning and multimodal education; here, eye tracking could deepen the understanding of multimodal perception processes (for example, simultaneously watching and listening to a video stream) and their implications for learning (Chuang & Liu, 2011; Dogusoy & Cagiltay, 2009; Slykhuis, Wiebe, & Len, 2005; van Gog & Scheiter, 2010; Zambarbieri, 2003). Also, it could be a valuable tool for teachers, who study classroom dynamics by viewing them on video. Film or television studies could use eye-tracking methodology to analyze the perception modes of films (d'Ydewalle, Desmet, & Van Rensbergen, 1998), and especially the interrelation between video and sound information, for example, to detect points of maximum and minimum attention in relation to the soundtrack. For media studies, the eye tracking of visual perception and the processing of (pictorial) art may constitute another interesting field of research (Buswell, 1935; DeCarlo & Santella, 2002; Holsanova, 2011; Molnar, 1981; Wooding, 2002; Yarbus, 1967). Additionally, media studies could utilize (head-mounted) eye tracking to examine the audience's viewing and attention patterns in different media reception contexts, such as in classrooms, conferences, films, concerts, or sports events, as a way to detect and indicate variation in the quality of the attention in each case.

At the same time, eye tracking is still challenging. The methodological implementation of eye tracking is relatively complex and the researcher can only refer to a limited number of standardized procedures. Also, while various questions concerning the relationship between visual perception and cognition are not yet satisfactorily answered, commonly established standards for the operationalization and interpretation of eye-tracking indicators are only slowly evolving. Thus, from a methodological perspective, eye tracking offers as many limitations as challenges. Therefore, even if the researcher's focus is predominantly directed on the empirical results of eye-movement research, a deeper knowledge about the methodological requirements and technical functions of eye tracking, as well as about the underlying processes of visual perception and visual information processing is a fundamental basis for conducting eye-tracking studies, creating research designs, tracking the eye movements in an adequate setting and, last but not least, for data analysis and data interpretation.

NOTES

1 Vestibular eye movements (also known as pursuit movements) are eye movements that prevent a “shifting” or drifting of sensory information on the retina. They appear as a result of the movement of the body, the movement of a perceived object, or the movement of a relevant part of the visual environment, and help to continuously fix on an object (as a quick saccade-like following movement or as a slow sliding movement of the eyes, i.e., smooth pursuit) or to maintain a stable retina image (optokinesis). Vestibular eye movements are adaptation movements that (to simplify) do not structure the scan path, but rather stabilize and enable it (especially fixations) during movement.

2 Those values vary substantially: according to Diamond, Ross, and Morrone (2000), for example, the suppression of intake and processing of visual information already start around 75 milliseconds prior to a saccade and continue after its end for approximately 50 milliseconds.

REFERENCES

Adam, P., Quinn, S., & Edmonds, R. (2007). Eyetracking the news: A study of print and online reading. St. Petersburg, FL: Poynter Institute.

Alpers, G. W. (2008). Eye-catching: Right hemisphere attentional bias for emotional pictures. Laterality, 13(2), 158–178.

Backs, R. W., & Walrath, L. C. (1995). Ocular measures of redundancy gain during visual search of color symbolic displays. Ergonomics, 38, 1831–1840.

Bucher, H.-J., & Schumacher, P. (2006). The relevance of attention for selecting news content: An eye-tracking study on attention patterns in the reception of print and online media. Communications, 31(3), 347–368.

Bullier, J. (2001). Integrated model of visual processing. Brain Research Review, 36, 96–107.

Buswell, G. T. (1935). How people look at pictures: A study of the psychology and perception in art. Chicago: University of Chicago Press.

Carpenter, R. H. (1988). Movements of the eyes. London, UK: Pion.

Carpenter, R. H. (1991). Eye movements: Vision and visual dysfunction. Boca Raton, FL: CRS.

Carpenter, R. H., & Robson, J. (1999). Vision research: A practical guide to laboratory methods. Oxford, UK: Oxford University Press.

Chuang, H.-H., & Liu, H.-C. (2011). Effects of different multimedia presentations on viewers' information-processing activities measured by eye-tracking technology. Journal of Science Education and Technology, 21(2), 1–11.

DeCarlo, D., & Santella, A. (2002). Stylization and abstraction of photographs. ACM Transactions on Graphics, 21(3), 769–776.

Deubel, H., & Schneider, W. X. (1996). Saccade target selection and object recognition: Evidence for a common attentional mechanism. Vision Research, 36(12), 1827–1837.

Diamond, M. R., Ross, J., & Morrone, M. C. (2000). Extraretinal control of saccadic suppression. Journal of Neuroscience, 20(9), 3449–3455.

Dogusoy, B., & Cagiltay, K. (2009). An innovative way of understanding learning processes: Eye tracking. In J. Jacko (Ed.), Human–computer interaction: Interacting in various application domains (pp. 94–100). Berlin, Germany: Springer.

Duchowski, A. T. (2007). Eye tracking methodology: Theory and practice. London, UK: Springer.

d'Ydewalle, G., Desmet, G., & Van Rensbergen, J. (1998). Film perception: The processing of film cuts. In G. Underwood (Ed.), Eye guidance in reading and scene perception (pp. 357–368). Oxford, UK: Elsevier.

Ehmke, C., & Wilson, S. (2007). Identifying Web usability problems from eye-tracking data. In L. J. Ball, M. A. Sasse, C. Sas, T. C. Ormerod, A. Dix, P. Bagnall, & T. McEwan (Eds.), People and computers XXI: HCI – but not as we know it: Proceedings of HCI 2007 (pp. 119–128). London, UK: British Computer Society.

Findlay, J. M., & Gilchrist, I. D. (2003). Active vision: The psychology of looking and seeing. Oxford, UK: Oxford University Press.

Fuchs, A. F., & Kaneko, C. R. S. (1981). A brain stem generator for saccadic eye movements. Trends in Neurosciences, 4, 283–286.

Gale, A. G. (2003). Eye movements in communication and media applications. In J. Hyöna, R. Radach, & H. Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye movement research (pp. 729–732). Amsterdam, Netherlands: Elsevier Science.

Gandhi, N. J., & Katnani, H. A. (2011). Interaction of eye and eylid movements. In S. P. Liversedge, I. D. Gilchrist, & S. Everlin (Eds.), The Oxford handbook of eye movements (pp. 323–338). Oxford, UK: Oxford University Press.

Geise, S. (2011). Eyetracking in der Kommunikations- und Medienwissenschaft: Theorie, Methode und kritische Reflexion [Eye tracking in communication and media science: Theory, method and critical reflection]. SCM: Studies in Communication & Media, 2 149–263.

Geise, S., & Schumacher, P. (2011). Eyetracking. In T. Petersen & C. Schwender (Eds.), Die Entschlüsselung der Bilder: Methoden zur Erforschung visueller Kommunikation (pp. 349–371). Cologne, Germany: Herbert von Halem.

Gilchrist, I. D. (2011). Saccades. In S. P. Liversedge, I. D. Gilchrist, & S. Everlin (Eds.), The Oxford handbook of eye movements (pp. 85–94). Oxford, UK: Oxford University Press.

Godijn, R., & Theeuwes, J. (2003). The relationship between exogenous and endogenous saccades and attention. In J. Hyöna, R. Radach, & H. Deubel(Eds.), The mind's eye: Cognitive and applied aspects of eye movement research (pp. 3–26). Amsterdam, Netherlands: Elsevier Science.

Goldberg, J. H., & Kotval, X. P. (1999). Computer interface evaluation using eye movements: Methods and constructs. International Journal of Industrial Ergonomics, 24(6), 631–645.

Goldberg, J. H., Stimson, M. J., Lewenstein, M., Scott, N., & Wichansky, A. M. (2002). Eye tracking in Web search tasks: Design implications. Paper presented at the symposium on “Eye tracking research & applications” (ETRA) in New Orleans, LA, March 25–27.

Goldberg, J. H., & Wichansky, A. M. (2003). Eye tracking in usability evaluation: A practitioner's guide. In R. Radach, J. Hyönä, & H. Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye movement research (pp. 493–516). Amsterdam, Netherlands: Elsevier Science.

Gordon, I. E. (2004). Theories of visual perception. New York, NY: Wiley

Granka, L., Feusner, M., & Lorigo, L. (2008). Eye monitoring in online search. In R. I. Hammoud (Ed.), Passive eye monitoring: Algorithms, applications and experiments (pp. 347–372). Berlin, Germany: Springer.

Grier, R. A. (2004). Visual attention and web design. Unpublished doctoral dissertation, University of Cincinnati, Cincinnati, OH.

Hammoud, R. I. (2008). Passive eye monitoring: Algorithms, applications and experiments. Berlin, Germany: Springer.

Hayhoe, M. M. (2004). Advances in relating eye movements and cognition. Infancy, 6(2), 267–274.

Hendersen, J. M. (2003). Human gaze control during real-worlds scene perception. Trends in Cognitive Science, 7(11), 498–504.

Hoffman, J., & Subramaniam, B. (1995). The role of visual attention in saccadic eye movements. Attention, Perception, & Psychophysics, 57(6), 787–795.

Hoffman, J. E. (1998). Visual attention and eye movements. In H. Pashler (Ed.), Attention (pp. 119–154). Hove, UK: Psychology Press.

Holmqvist, K., Holsanova, J., Barthelson, M., & Lundqvist, D. (2003). Eye movements in the processing of print advertisements. In J. Hyöna, R. Radach, & H. Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye movement research (pp. 657–670). Amsterdam, Netherlands: Elsevier Science.

Holmqvist, K., & Wartenberg, C. (2005). The role of local design factors for newspaper reading behavior – an eye-tracking perspective. Lund University Cognitive Studies, 127, 1–21.

Holsanova, J. (2011). How we focus attention in picture viewing, picture description and mental imagery. In K. Sachs-Hombach & R. Trotzke (Eds.), Bilder, Sehen, Denken. Zum Verhältnis von Begrifflich-Philosophischen und Empirisch-Psychologischen Ansätzen in der Bildwis-senschaftlichen Forschung (pp. 291–313). Cologne, Germany: Herbert von Halem.

Hughes, A., Wilkens, T., Wildemuth, B., & Marchionini, G. (2003). Text or pictures? An eye-tracking study of how people view digital video surrogates. In E. Bakker, M. Lew, T. Huang, N. Sebe, & X. Zhou (Eds.), Image and video retrieval: Second International Conference, CIVR 2003, Urbana–Champaign, IL, USA, July 24–25, 2003 (Vol. 2728, pp. 1–6). Berlin, Germany: Springer.

Jacob, R. J. K., & Karn, K. S. (2003). Eye tracking in human–computer interaction and usability research: Ready to deliver the promises. In J. Hyona, R. Radach, & H. Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye movements (pp. 573–605). Amsterdam, Netherlands: Elsevier Science..

Joensson, E. (2005). If looks could kill – an evaluation of eye tracking in computer games. Unpublished master's dissertation, Royal Institute of Technology, Stockholm.

Josephson, S. (2005). Eye tracking methodology and the Internet. In K. Smith, S. Moriarty, G. Barbatsis, & K. Kenney (Eds.), Handbook of visual communication: Theory, methods and media (pp. 63–80). London, UK: Routledge.

Just, M. A., & Carpenter, P. A. (1976). Eye fixations and cognitive processes. Cognitive Psychology, 8(4), 441–480.

Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354.

Kickmeier-Rust, M., Hillemann, E., & Albert, D. (2011). Tracking the UFO's paths: Using eye-tracking for the evaluation of serious games virtual and mixed reality – new trends. In R. Shumaker (Ed.), Virtual and mixed reality – new trends (pp. 315–324). Berlin, Germany: Springer.

Kissler, J., & Keil, A. (2008). Look – don't look! How emotional pictures affect pro- and anti-saccades. Experimental Brain Research, 188(2), 215–222.

Kreysa, H., & Pickering, M. (2011). Eye movements in dialogue. In S. P. Liversedge, I. D. Gilchrist, & S. Everlin (Eds.), The Oxford handbook of eye movements (pp. 943–960). Oxford, UK: Oxford University Press.

Lamnek, S. (2010). Qualitative Sozialforschung [Qualitative sociological research]. Weinheim, Germany: Beltz.

Lopes, D. M. (1996) Understanding pictures. Oxford, UK: Oxford University Press.

Ludwig, C. J. H. (2011). Saccadic decision-making. In S. P. Liversedge, I. D. Gilchrist, & S. Everlin (Eds.), The Oxford handbook of eye movements (pp. 413–424). Oxford, UK: Oxford University Press.

Manhartsberger, M., & Zellhofer, N. (2005). Eye tracking in usability research: What users really see. In Usability Symposium 2005: Empowering Software Quality: How Can Usability Engineering Reach These Goals? (pp. 141–152). Vienna: Oesterreichische Computer Gesellschaft.

McLeskey, J., Levie, W. H., & Fleming, M. L. (1982). Selective attention to visual stimuli: The need to combine theory and research. Journal of Special Education Technology, 5, 23–32.

Molnar, F. (1981). About the role of visual exploration in aesthetics. In H. Day (Ed.), Advances in instrinsic motivation and aesthetics (pp. 385–413). New York, NY: Plenum.

Piccoli, B., Perris, R., Gratton, I., & Grieco, A. (1985). Visione e Lavoro. Nota II: Metodo per lo Studio della Funzionalita Visiva in Lavoro. Medicina del Lavoro, 76(3), 238–245.

Posner, M. I. (1995). Attention in cognitive neuroscience. In M. S. Gazzaniga (Ed.), The cognitive neurosciences (pp. 615–624). Cambridge, MA: MIT Press.

Proulx, M. (2007). The strategic control of attention in visual search: Top-down and bottom-up processes. Saarbrücken, Germany: VDM.

Radach, R., Lemmer, S., Vorstius, C., Heller, D., & Radach, K. (2003). Eye movements in the processing of print advertisements. In J. Hyöna, R. Radach, & H. Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye movement research (pp. 609–632). Amsterdam, Netherlands: Elsevier Science.

Rayner, K. (1998). Eye movements in reading and information processing: 20 Years of Research. Psychological Bulletin, 124, 372–422.

Schneider, W., & Maasen, S. (1998). Mechanisms of visual attention: A cognitive neuroscience perspective [Special issue]. Visual Cognition, 5(2), 257–286.

Shukla, M., Wen, J., White, K., & Aslin, R. (2011). SMART-T: A system for novel fully automated anticipatory eye-tracking paradigms. Behavior Research Methods, 43(2), 384–398.

Slykhuis, D., Wiebe, E., & Len, A. (2005). Eye-tracking students' attention to PowerPoint photographs in a science education setting. Journal of Science Education and Technology, 14(5), 509–520.

Stenfors, I., Morén, J., & Balkenius, C. (2003). Behavioural strategies in Web interaction: A view from eye-movement research. In J. Hyöna, R. Radach, & H. Deubel (Eds.), The mind's eye: Cognitive and applied aspects of eye movement research (pp. 633–644). Amsterdam, Netherlands: Elsevier Science.

Stern, J. A., Walrath, L. C., & Goldstein, R. (1984). The endogenous eyeblink. Psychophysiology, 21(1), 981–985.

Theeuwes, J. (1993). Visual selective attention: A theoretical analysis. Acta Psychologica, 83(2), 93–154.

Triesch, J., Ballard, D. H., Hayhoe, M. M., & Sullivan, B. T. (2003). What you see is what you need. Journal of Vision, 3(1), 86–94.

van Gog, T., & Scheiter, K. (2010). Eye tracking as a tool to study and enhance multimedia learning. Learning and Instruction, 20(2), 95–99.

Vitu, F. (2011). On the role of visual and oculomotor processes in reading. In S. P. Liversedge, I. D. Gilchrist, & S. Everlin (Eds.), The Oxford handbook of eye movements (pp. 731–750). Oxford, UK: Oxford University Press.

Walter, V. (2009). Eyetracking in second-life: Eine explorative Analyse zur Wahrnehmung in virtuellen Welten. Frankfurt: Volker Walter.

Wooding, D. S. (2002). Fixation maps: Quantifying eye-movement traces. Paper presented at the symposium on Eye Tracking Research & Applications (ETRA) in New York, NY.

Wright, R. D. (1998). Visual attention. Oxford, UK: Oxford University Press.

Yantis, S. (2002). Stimulus-driven and goal-directed attention control. In V. Cantoni, M. Marinaro, & A. Petrosino (Eds.), Visual attention mechanisms (pp. 125–134). New York, NY: Springer.

Yarbus, A. (1967). Eye movements and vision. New York, NY: Plenum.

Zambarbieri, D. (2003). E-tracking: Eye tracking analysis in the evaluation of e-learning systems. In D. Harris, V. Duffy, M. Smith, & C. Stephanidis (Eds.), Human-centred computing: Cognitive, social and ergonomic aspects (pp. 617–621). Mahwah, NJ: Lawrence Erlbaum.