23

Clicks and Bricks

Methods and Strategies for Measuring the Effects of Similar Virtual and Physical Activity

Lillian C. Spina-Caza

Research on young people and interactive virtual technology (IVT)¹ use is abundant and diverse, emerging from numerous fields of interest. These fields include cognitive science, human–computer interaction (HCI) and its subfield of child–computer interaction (CCI), video game studies, developmental and ecological psychology, and literacy, communication, and media studies – to name just a few. Conversations range from those suggesting that new technology environments can enhance creativity, promote media literacy and information technology (IT) proficiency, and empower young people to be programmers in their own right, to those more circumspect about moving activity to virtual environments and the impact of losing touch with the physical world (Healy, 1998).

One common thread running throughout much of the literature is the call for young people to be creators and not just consumers of computer culture, producers and not just participants in the virtual worlds they inhabit. It is a call that places the notion of creative agency front and center in explorations of IVT use. What sorts of creative or contributive activities young people can undertake, and what they perceive they are able to accomplish when playing video games, interacting with online content, or using mobile applications, are critical to understanding the effects of virtual activity on a developing person. Knowing more about how young people perceive themselves and their own creative abilities when interacting with virtual environments can also inform and improve the design of interactive technologies overall.

Another recurring thread in the research and literature on IVT use is the call for empirical research to address the benefits of moving activity from physical to virtual settings. Across disciplines there is agreement that studies examining the effects of virtual interactions are long overdue. Roussou, Oliver, and Slater (2008), for example, who use activity theory to evaluate behavior in virtual environments, note that “To date, very few efforts exist that explore the value of interactive virtual environments and applications, especially the added value that these can bring to children's learning” (p. 141). Livingstone (2006), who conducts large-scale, multinational, surveys of children and Internet use, writes, “little work has examined, let alone demonstrated unequivocally, the positive benefits this has for educational achievement” (p. 220).

The research methods and strategies presented in the studies that follow address both of these threads, and are also influenced by activity theory (AT), which has its roots in developmental psychology and education. AT emphasizes the connections between human consciousness, activity, and environment, and recognizes the role human agency and creativity play in a person's ability to realize goals and intentions. According to Nardi (1996), who researches activity theory in the context of HCI and interaction design, AT views human beings as “active beings in control of their tools for creative purposes rather than as automatons whose operations are to be automated away, or nodes whose rights to privacy and dignity are not guaranteed” (p. 87).

While AT provides a solid framework for research concerned with youth and technology use, it has some limitations. For example, by focusing on collective activity oriented toward motives and objects, AT does not examine subjective experience or how self-perceptions might be changed in the performance of certain types of activity, in this instance creative activity. It also does not tell us whether there might be a relationship between mode or how an activity is performed – with physical or virtual objects and artifacts – and the effects on a person's perceptions of his or her own creative agency.

Kaptelinin (1996), whose work also examines activity theory and interaction design, observes that “Virtual realities present a problem to activity that probably cannot be solved without enriching activity theory's basic principles with new ideas” (p. 64). Tikhomirov (1999), who notes human creativity has been dramatically altered in the context of computer use, points out that “The content of individual human experience includes not only mastered social experience, but also personal experience, the experience of an activity's realization” (p. 358). As he concludes, “Studies of creative activity demand an analysis of the relations between activity and personality in a specific creative act” (p. 350).

The research approaches discussed in this chapter presuppose the importance of activity theory to our understanding of IVT research and design. The three studies presented here were conducted to broaden our knowledge of virtual activity by employing various qualitative and quantitative analyses to describe what happens during Internet and video game play, and what the effects might be on a developing person. Some of the research strategies used included the coding of visual data analysis as children played at two popular websites; observational and cooperative inquiry techniques were used to explore similar physical and virtual play activity with a group of students in an after-school program; and an experimental field study that included physical and virtual conditions and pre- and post-test surveys was conducted to examine the effects of similar physical and virtual activities on young people's perceptions of their own creative agency.

The chapter also introduces the creative agency model (CAM), which offers a way to operationalize or measure self-perceptions that emerge in the performance of activity in virtual environments. CAM suggests we adopt a bioecological approach to the study of virtual activity and affordance use to better understand how this activity might influence human development over time. It supports iterative, multimethod designs that employ observation, interpretation, and experimentation to examine the role environmental artifacts and objects play in goal attainment, as well as the subjective experience and possible changes to concepts of self that emerge in the performance of virtual activity. No other studies that we are aware of bring together activity, affordances, and perceptions to the study of virtual play. Combined, these strategies contribute to our understanding of virtual activity and allow us to explore the connections between mode of play, gender, and perceptions.

Review of Research Approaches to Traditional and New Media Studies

Early studies of virtual interactions took a traditional media effects approach, emphasizing content analysis and the link between antisocial or aggressive behavior and violent content in video games. Surveys and controlled laboratory experiments are two common strategies applied to study the effects of media viewing on behavior. Some of the other issues addressed using these kinds of approaches are children's health and well-being – identifying, for instance, links between childhood obesity and video game addiction (Anderson & Bushman, 2001; Wimmer & Dominick, 2006) – Internet safety and access, the need for critical analyses of web content, the interplay of technology and identity, or the erosion of children's agency imposed by consumer culture (Cross, 2009; Grimes & Shade, 2005; Livingstone, 2003a, 2003b, 2006; Mazzarella & Atkins, 2010; Turkle, 1995, 2005; Willett, 2009).

A compelling body of pro-social research, however, also started to emerge in the 1990s, emphasizing the positive role video games and Internet use might assume in health communication, education, behavioral, and social change (Flanagan, 2006; Gee, 2007a, 2007b; Lieberman, 1997, 1999). Lieberman's research (1997, 1999) finds that video games promoting healthy messages show positive results for asthma and diabetes management, as well as smoking prevention. In her research, Flanagan (2006) examines activist games and Internet environments that embrace social values. One such game – Peter Packet – encourages players to use Internet technology to both learn about and solve the problems faced by youth in less developed countries, including issues such as “education, clean drinking water, and AIDS” (p. 494).

In both video game and Internet research, this more celebratory branch of research typically positions young people as active media content contributors who create fan fiction, produce videos for YouTube, and “mod” or modify video games as they dynamically shape their own experiences in virtual environments (Gee, 2007a, 2007b; Jenkins, 2007; Lenhart & Madden, 2005; Marsh, 2005; Mazzarella, 2005; Mazzarella & Atkins, 2010). The Internet is touted as a space for cultural and civic or democratic participation, and a safe place for self-expression (Center for Media Literacy, 2008; Cordes & Miller, 2000; Kellner & Share, 2005; Livingstone, 2003a, 2006; Mazzarella, 2005).

One prominent strand of research in video game studies, introduced in Gee's (2007b) seminal book What Video Games Have to Teach Us About Learning and Literacy, identifies how virtual play can also be supportive of creativity and knowledge construction overall. Good games, Gee asserts, allow players to be designers in their own right, providing players with software that allows them to modify games by building “new maps, levels, and scenarios, and sometimes even change the rules” (2007a, p. 136). He argues that the feelings of agency and control experienced in video game play lead to “good learning,” and that gaming literacy gives young people the competitive skills that are critical in a global, high tech world.

A number of HCI and social science research methods and strategies are used to explore such issues tied to young people and IVT use, including – but not limited to – content analysis, action research, participatory research, field studies, case studies, lab experiments, survey research, basic research, applied research, and normative writings. Another research method successfully applied to studies of IVT use, elaborated upon later in this chapter, adapts research methods associated with the collection, coding, and analysis of verbal streams of data. It is a method that is consistent with media content analysis used to determine, for example, the relative frequency of television violence (Geisler, 2004).

The impact of computerization and the virtualization of human activity has resulted in a transformation of a person's “whole system of motives, stable meanings of personality, and personality's goals” (Tikhomirov, 1999, p. 353). To understand the breadth and depth of the effects of IVT use on young people, a multifaceted, mixed method approach needs to be applied to the study of virtual activity. The purposes of such research are to:

• examine the ways in which activity unfolds in virtual environments over time;
• identify opportunities for creative or contributive interactions that support healthy development;
• understand the roles both IVT affordance recognition and use play in supporting or thwarting creative activity; and
• learn how subjective experiences and concepts of self are shaped by the performance of virtual activity.

The following sections present several research strategies to explore virtual activity and its impacts on young people's perceptions of their own creative agency. The various methods and approaches outlined here provide useful ways to develop a better understanding of virtual environments overall.

Experimenting With Mixed Research Methods to Study Complex Virtual Interactions

Mixed research methods were used in the following three studies to explore young people's interactions with web content and video games, and to measure perceptions of play that emerged in comparable virtual and physical activity. The first study applied methods adapted from verbal data analysis (Geisler, 2004) to investigate and compare how play activity unfolded at two popular children's websites. This study was critical in moving the focus of the research beyond activity to address perceptions.

The second study was conducted as an observational study to further investigate children's perceptions of virtual play. It extended the first study by comparing various physical play activities with similar virtual activities, in this case focusing on video game play instead of Internet interactions. Most importantly, this study explored whether mode of play impacts perceptions of creativity and control, and suggested the design for a third study.

The third and final study, conducted as a field experiment, was both inspired and informed by the two earlier studies. Following up on the findings of the second study, it was specifically designed to measure whether or not changes to perceptions emerge in the performance of comparable physical and virtual play tasks. The third study also led to the development of the creative agency model, which provides a useful framework for future research.

Study 1: A Tale of Two Websites

One of the overarching goals of the first study was to develop methods to measure and assess how children engage in play online at popular commercial websites. Another object of this early study was to identify virtual activities that encouraged the expressive and constructive sorts of play recognized as being both supportive of healthy development, and also the kinds of play children say they prefer when interacting with technology (Druin, Bederson, Boltman, Miura, Knotts-Callahan, & Platt, 1999). To accomplish this, play activity at two commercial websites – Millsberry.com and Webkinz.com – was recorded and coded for analysis (see Figures 23.1 and 23.2).

The author's three daughters – ages 7, 9, and 12 at the time – participated in the initial study. The websites selected were ones at which the children were already playing. Two of the three girls gave consent to have play sessions recorded using screen capture technology, and all three made lists of the things they liked to do when playing at the two sites, and took part in discussions and demonstrations of the various activities available. Working with one's own children in research is not unusual. Linguists, for example, have observed their own children to study how language skills develop. Child development theorists also study the behaviors of their own children. Jean Piaget (1962, 1970), for instance, worked with his children to explore the implications of childhood play on a developing child, and the ways in which young people both perceive and organize their experiences.

Figure 23.1 Screen capture of Millsberry.com

Figure 23.2 Screen capture of Webkinz.com

The research methods used in the initial observational study were inspired by approaches derived from the analysis of verbal data (Geisler, 2004). Verbal data analysis (VDA) methods are compatible with qualitative research approaches, and provide a way to identify units of analysis, calculate frequencies of specific types of activities, and explore behavioral patterns over time. Once data are collected, they can be segmented into units appropriate for analysis to identify the level at which phenomena of interest occur. For this initial study, however, VDA methods were applied to the analysis of visual data and virtual activity.

As indicated earlier, to analyze visual data, video screen capture technology was used to record streams of activity as the girls played at the two websites studied. Several sessions of online play were recorded for both children. Basic video editing software was utilized to segment visual data into discrete activity units or “a-units” for coding purposes; a-units were correlated to the time the girls spent engaging in a particular virtual activity. A-units are similar to the “play frames” described by literacy researchers Neuman and Roskos (1992), insomuch as these are used to segment “play that is bound by location and a particular focus of interaction” (p. 213). Three coding schemes were developed to explore play as it unfolded in real time.

The coding schemes created for this study considered the three types of interactions Druin et al. (1999, p. 66) identify as most important to young people: a variety of play options, an ability to interact with others, and “expressive tools that enable them to tell stories or build things.” The three schemes developed – coding for range of activities, coding for social activities, and coding for engaging activities – were designed to:

• calculate the range of activities a young person participated in during play, categorized as “high,” “medium,” or “low” based on the number of activities offered on the active screen or menu bar and the number selected;
• identify opportunities for social activity using site-based communication tools (coded as “social-responsive,” “social-unresponsive,” or “non-social”) based on whether the child made contact, attempted to contact, or was unable to contact another player online; and
• classify the types of activity engaged in with greater frequency using the terms “search,” “select,” “design,” and “direct” to provide descriptive information about the types of interactions that occur in web play environments.

Definitions of activity in virtual environments suggested by the work of Parés and Parés (2001) were useful for developing the third coding scheme. The authors describe activity in virtual worlds as explorative, manipulative, and contributive. Explorative refers to the ability to navigate within a virtual environment, manipulative indicates the ability to influence or control elements within that environment, and contributive suggests the ability to add to or modify the environment as a whole.

Building on Parés and Parés, the terms “search,” “select,” “design,” and “direct” were used to describe activities children participated in at both Millsberry.com and Webkinz.com. If a child navigated or clicked around the website looking for something to do it was coded “search.” If she chose an arcade game to play, or picked out clothing or food items for a virtual pet, the activity was coded “select” If she decorated a T-shirt or customized an avatar it was coded “design.” Activities that allowed for more creative control (i.e., composing a song or making virtual movies using site apps) were coded as “direct.” Activities coded as either “design” or “direct” corresponded to practices that are considered to be creative or contributive.

The third coding scheme was particularly useful for understanding how interactions in web environments unfold over time. As indicated in the temporal charts of engaging activities, the terms “search” and “select” were used more frequently than “design” or “direct” (see Figures 23.3 and 23.4). Video capture recordings show study participants engaging mostly in game play at both websites, and interactions in virtual arcades or games of chance, typically involve searching for things to do and/or selecting games to play to earn virtual cash to buy (select) things (food, clothing, toys, furniture, etc.) for their virtual avatars or pets.

Figure 23.3 Temporal chart for an 8-year-old girl engaged in 22 activities at Webkinz.com during a play session lasting 17 minutes 22 seconds. No activities are coded as “design” and only three are coded “direct.”

Figure 23.4 Temporal chart for a 10-year-old girl engaged in 14 activities at Webkinz.com during a play session lasting 17 minutes 53 seconds. No activities are coded as “design” or “direct.”

Overall, the three coding schemes developed for this study offer useful ways to explore patterns that emerge during online play and can tell us more about:

• what children do when they visit popular websites;
• whether children play alone or with others;
• what activities support creative or contributive sorts of play; and
• whether similar patterns of play occur across different websites.

In addition to coding screen capture data, informal interviews were conducted with participants to gather more information about Internet play at Webkinz.com and Millsberry.com. Study participants were also asked to make a list of their favorite activities, which indicated a preference for activities that allowed them to be creative. Surprisingly, these kinds of activities were not engaged in with any frequency, at least not during the play sessions recorded and coded for the study. For additional information about the findings of this early studysee Spina-Caza (2010).

The apparent disconnect between what the children said they liked to do and what they actually did might not be that unusual for two reasons. First, as Druin (2002, p. 2) observes, “Children are extremely honest in their feedback and comments concerning technology, but much of what they say may be in their actions and therefore, needs to be interpreted within the context of concrete experiences.” And, second, as Hayes (2008, p. 103) found out in a study conducted on the topic of content creation, girls do not engage in such activity as frequently as boys:

As this study was conducted with girls only, it was clear that future studies of this kind would need to include both boys and girls in order to better understand the role of gender in virtual play.

Observations of online play in this first study suggest that a child's subjective experience, or his or her perceptions of what happens when interacting with virtual environments, may be key to our understanding of how IVT use impacts human development. Wachs (1991), who argues that environment and experience are not necessarily synonymous, notes that a “child's perception of the environment, rather than objective reality may be critical in influencing developmental outcomes” (p. 52). The need for research exploring how perceptions might be altered by virtual interactions is underscored by Bronfenbrenner (2005), developmental psychologist and founder of Head Start, who argues that perceptions emerging from activity “can contribute in powerful ways to shaping the course of development in the future” (p. 5).

To expand upon this early research we decided to further explore perceptions of creativity and agency as these were formed in the context of virtual play. To this end, a follow-up study was conducted over several weeks with 15 children in grades 4 through 8 attending a 4-H after-school program.² In this study, youngpeople's interactions with comparable real and virtual objects were observed to develop an understanding of how transferring physical activity to virtual environments can influence feelings of creativity and control.

Study 2: Redefining the Problem Space, Extending the Research Focus

The second study used a participatory research strategy inspired by Druin (1999) called cooperative inquiry, an approach often taken when designing technologies for children with children on a more or less equal footing. According to Druin (1999), while cooperative inquiry is unique as it involves children, it is grounded in HCI research and theories such as participatory design, contextual inquiry, and situated action:

In this study, a similar approach was used to advance our understanding of how young people perceive their interactions with virtual technology. Study participants were invited to be our research partners and to think about the different types of games they played, and whether these games and toys – both physical and virtual – allowed them to be more or less creative, or more or less in control over play. Handwritten observations describing free play activity, informal interview notes, and a “wrap-up survey” were the only instruments used to collect play data.

The hands-on board games and construction toys used in the study were matched with similar video games for the PC and Wii platforms. For example, both Kabookii (Ubisoft, 2007) for the Wii – a video game based on the popular board game Cranium – and the actual board game were used in the study. Physics games were also used, including Professor Heinz Wolff's Gravity (Deep Silver, 2008), a virtual marble run, and the Chaos Tower, a physical construction toy and marble run (see Figures 23.5 and 23.6).

Several key observations emerged from the second exploratory study. During the informal interviews that were conducted in the workshops, the physical board games and construction toys were identified as being most supportive of creative sorts of play. Interestingly, video games were not perceived as offering opportunities to be creative. When asked which games afforded greater control over play activity, participant responses were not as cut and dried. Eight of 13 children interviewed (two boys and six girls) reported feeling more in control when playing with physical objects, and five (two girls and three boys) responded feeling more in control during video game play.

At the end of the six workshops, participants were asked to fill out a “wrap-up survey” identifying which games or toys they perceived allowed them to be more creative, and which afforded more control over play activity. Responses, however, were mixed. The surveys show a large number of study participants felt play with the Chaos Tower was more creative and that they had more control over play activity. The PC version of the Gravity video game was also identified by nearly as many participants as allowing for both creativity and control. Interestingly, study participants who played Gravity using the Wii felt both less creative and less in control of play.

Figure 23.5 Chaos Tower physical marble run

Figure 23.6 Screen capture of Professor Heinz Wolff's Gravity, a virtual marble run video game (PC version)

While this second study did not specifically look for gender differences, observation notes indicated girls and boys played differently with the physical and virtual games and toys. A few boys freely shared hints to help others figure out how to solve puzzles or what to do in video games. Boys also took charge of physical, hands-on building activities, and were more likely to improvise how they built the Chaos Tower. Girls were more apt to construct the Chaos Tower according to the written directions, and also found themselves in the role of helpers or, at times, excluded from play. One 10-year-old girl was observed:

The purpose of this second, participatory study was to gain a better understanding of how physical and virtual play unfold in naturalistic settings. It was also conducted to inform the design of a larger, more controlled field experiment. The observations made in this study suggested the following changes to a future study to avoid potential confounds:

• already tested, valid survey instruments would be used to measure creativity and agency perceptions to see if results support the exploratory findings;
• subjects would be randomly selected to participate in either a physical or a virtual treatment, but not both;
• the virtual task would be performed using a PC, not the Wii, since perceptions of play using the Wii controller were not as positive and could confuse results;
• an equal number of girls and boys would be invited to take part in the study and be grouped by gender.

The second study allowed us to investigate children's interactions with comparable physical and virtual games and toys in a naturalistic research setting; to identify, with young people, perceptions of their own creative abilities in the contexts of virtual and physical play environments; and to bring into focus key issues associated with moving activity from physical to virtual spaces. Most importantly, it was useful for designing a research plan for the third, experimental, field study. The third study, which further extends this research, used a pre- and post-test design to identify self-perceptions as these emerged through the performance of physical and virtual play. Unlike the first two studies, the third study also considered the role gender plays in perceptions of IVT use.

Study 3: A Conceptual Framework for Virtual Studies

Activity theory (AT) identifies creativity and agency as critical components of how knowledge is constructed and meaning is produced; both are also associated with learning through play and consequently warrant further study in the context of youth and IVT use. The third study, conducted as a field experiment, extended the earlier studies and examined the connections between mode and gender on creativity and agency perceptions as these emerge through physical and virtual activity. To this end, the creative agency model, or CAM, was developed to guide virtual research exploring the effects of IVT use on young people.

The CAM provided a conceptual framework for the overall experimental design of the third study. It built on existing research and theory by offering a formula to express creativity and agency as emergent factors of both subjective and objective aspects of human activity. Bronfenbrenner's (1981, 2005) formula for understanding development in context, or D = f(PE),³ which posits human development is a process or a joint function of a person's interactions with his or her environment, was the inspiration for the CAM formula presented here:

This formula does three things: first, it identifies creativity (C) and agency (A) as core dimensions of human development that can be measured; second, it defines C and A as both functions of virtual activity (VrAc) and emergent properties of performance perceptions (PfPc); and third, it recognizes gender (g) as a key dimension.⁴

The CAM, combined with self-perception measures such as the two instruments used for this study, can help us identify how technology environments might encourage the creative, contributive, and child-directed sorts of play that are associated with psychological development and cognitive growth. It also underscores the role environmental affordances play in these processes. The term “affordance,” as it is used here, refers to “a three-way relationship between the environment, the organism, and an activity . . . centered on the notion of an organism acting in an environment: being in the world” (Dourish, 2004, p. 118).

The CAM formula proposes that it is through activity performance that human agency and creativity transpire. It posits that a person's perceptions of his or her own abilities, which emerge in the performance of virtual activity, are crucial to human development in general. CAM acknowledges all activity is situated in specific environments, but focuses attention on the role environmental affordances play in facilitating or constraining activity in virtual settings. Perceptions are placed at the center of the model (Figure 23.7) because the ways in which young people perceive their own creative abilities or agency in the performance of either virtual or physical activity may, as pointed out in the first study, be more critical to development than objective reality (Bronfenbrenner, 2005; Wachs, 1991).

Figure 23.7 Creative agency model (CAM)

The CAM model offers a potential way to conceptualize creative agency perceptions as these emerge through activity performance. It also provides a useful formula for measuring whether or not young people's feelings about their own creative abilities increase or decrease when participating in comparable physical or virtual activities.

Measuring Perceptual Shifts in Physical and Virtual Play

In addition to using the CAM framework to guide this research, the experimental study also utilized existing psychometric measures to quantify perceptual shifts based on activity performance. The design of the field study centered on two treatment conditions, a physical/video task and a virtual/web task. Subjects participating in the physical/video treatment (Figure 23.8) were invited to make movies using physical affordances (video camera, tripod, puppet stage, and handheld puppets), while subjects assigned to the virtual/web treatment (Figure 23.9) made movies using virtual affordances (the Webkinz Studio application at Webkinz.com).

Figure 23.8 Fourth-grade study subjects participate in the physical/video treatment

The children who took part in the study were randomly assigned to either the physical/video or the virtual/web treatments and were grouped by gender. Data was collected using the Khatena–Torrance Creative Perception Inventory (Khatena & Torrance, 1998). The KTCPI is composed of two self-report survey instruments: “What Kind of Person Are You?” (WK) and “Something About Myself” (SAM). Both instruments include dimensions generally associated with creativity and agency, and were administered as pretests three weeks prior to the experimental treatments. The same surveys were later administered a second time as post-tests, immediately following treatments or task completion.

The field experiment included three phases of data collection. In Phase I, the two survey instruments were administered to all of the study cohorts, including the control or wait list group, during the regular morning homeroom period by 4th- and 5th-grade teachers. The researcher met with all of the teachers and provided them with verbal and printed instructions to assist them in the administration of the KTCPI instruments to students.

Figure 23.9 Fourth-grade study participants making movies in the virtual/web treatment

In Phase II, cohorts, grouped by grade level and gender, were asked to make short movies using video production equipment in a physical environment, or a web movie-making application in a virtual production environment at Webkinz.com. In order to insure task interventions were comparable, physical sets were purposely selected and created to resemble virtual settings. An artist was commissioned to paint two backgrounds for the tabletop set to closely match two of the backgrounds in the Webkinz Studio application (see Figures 23.10 and 23.11). To make the physical and virtual conditions as similar as possible, hand puppets were selected for the physical treatment that closely matched the virtual characters.

In each condition study participants were able to write and direct their own movies (see Figures 23.12 and 23.13). In the physical/video condition subjects took turns at being director. Directors were given control over running the video camera and could direct puppet actors in movies based on their own scripts. In the virtual/web condition each participant could also direct virtual actors in movies based on the skits they had written. Both the physical and virtual movie-making activities were videotaped for research purposes and provided additional information that could be used during the analysis process and discussion of results.

In Phase III the control and treatment groups were expected to repeat the two KTCPI survey instruments. Post-task measures included brief, informal exit interviews designed to assess what types of affordances participants recognized as being available to them when performing either the physical or virtual tasks.

Figure 23.10 Physical studio tabletop stage set

Figure 23.11 Virtual Webkinz Studio set

Figure 23.12 Fifth-grade study participant writes her movie script

Figure 23.13 Fifth-grade participants use Webkinz Studio to make their movies

Independent Variables

The three independent variables in this study included:

• the physical condition (video movie-making task);
• the virtual condition (web movie-making task); and
• gender: students were randomly assigned to groups by grade level, isolating for gender.

Dependent Variables

The dependent variables in this study were two self-report measures typically used for measuring creativity perceptions and drawn from the Khatena–Torrance Creative Perception Inventory:

• the forced-choice attitude survey “What Kind of Person Are You?” (WK), developed by Torrance (Khatena & Torrance, 1976, 1998);
• the autobiographical creativity assessment inventory “Something About Myself” (SAM), created by Khatena (Khatena & Torrance, 1976, 1998).

Self-reports measuring perceptions and attitudes are often used to measure creativity (Fishkin & Johnson, 1998). The KTCPI is one such measure, consisting of two different inventories listed above. SAM is an autobiographical instrument based on “the rationale that creative functioning is reflected in the personality characteristics of the individual” (Khatena & Torrance, 1998, p. 6). According to Khatena (1977, p. 517) autobiographical instruments can be used to measure self-perceptions:

WK is a self-report inventory used to identify creatively oriented individuals. It requires subjects to reflect on and to choose from various creative and noncreative characteristics that most closely apply to them. The WK inventory is “based upon the rationale that the individual has a psychological self, whose structures have incorporated creative and non-creative ways of behaving” (Khatena & Torrance, 1998, p. 10). Most important to the virtual research conducted in the field experiment, the WK instrument is described as a composite of sub-selves triggered by an individual's interactions with his or her environment.

The experimental study had a wait list/control group that closely mirrored the experimental groups, and early plans included pre- and post-testing of both experimental and control groups. A decision was made, however, to conduct post-tests following the two experimental treatments, as it was important to determine whether or not we could establish direct links between the treatments and changes to self-perceptions. Among other things, this decision resulted in shorter time intervals between pre- and post-tests. While longer time intervals are recommended, shorter intervals are useful in some instances. As Buddenbaum and Novak (2001, p. 128) point out, “Studies using a very short time interval will measure short-term effects. For longer-term effects, a second set of observations at some later time may be added or the interval between pretest and posttest may be increased.” Future research conducted using the CAM model will consider longer-term effects.

To supplement the pre- and post-test data collected, qualitative data were also gathered to identify what affordances were used in the movie-making process, and to point out the possibilities and the constraints associated with these affordances as they are articulated in children's own words, or depicted by their actions during the experimental treatments. Several key questions associated with affordance use can be addressed using this type of qualitative analyses. For example:

1. Is the ability (or inability) to recognize and use affordances in physical and virtual environments connected to increases or decreases in creativity or agency perceptions?
2. Are there gender differences associated with affordance use and creativity and agency perceptions?
3. If gendered differences are linked to affordance recognition and use, do the differences support the findings of the current study?
4. What contributions can such findings make to the study of children's IVT use?

Qualitative data collected for this third study includes videotape recordings of participants performing the physical and virtual tasks, and the final movies made during the experimental study. These data have yet to be analyzed. However, some of the things we will be examining include participant statements, off-hand comments, and questions recorded in the videotaped observations. Verbal data analysis techniques will be used to identify any phenomena connected to creativity or agency perceptions and gendered practices.

In addition to the coding of verbal data, visual data will be segmented into activity units that can also be coded to identify, among other things, the frequency and range of affordance use. For example, in their finished movies, how many times did study participants recognize and use the affordances available to them in the physical/video or virtual/web treatments? Did they alter the framing or the types of shots used to make their movies? Did they add music or change set backgrounds? Did girls recognize and use affordances more frequently than boys?

The importance of conducting qualitative analysis of the physical and virtual treatments is threefold: (1) qualitative findings can confirm or complement the significant findings reported in the experimental study; (2) new findings may be revealed that enrich the experimental study; and (3) unexpected findings may come to light that necessitate further investigation.

The study of complex systems requires more complex, multilevel approaches to research exploring new media technology, particularly for studies attempting to capture or measure subjective experience or perceptions. In this study, we measured creativity and agency perceptions as these emerged in physical and virtual play. Our preliminary quantitative data analysis indicates perceptions are related to mode. Significant results were reported for both paired samples t tests and independent samples tests, indicating a possible relationship between how an activity is performed and self-perceptions. Positive effects were noted for three of the four creativity dimensions we studied and for one agency dimension, but only in the virtual condition, and all of these were gendered.

Overall, girls who took part in this study showed a positive increase in creativity perception scores across three dimensions, while boys demonstrated an increase for only a single agency-related variable. Negative effects, however, were found for two of the other agency dimensions in the virtual/web condition only, though these do not appear to be gendered.

Secondary analyses of the qualitative data collected for this research will shed more light on the various phenomena identified in the pre- and post-test results. The triangulation of data in mixed method approaches can verify or confirm that notable trends are not simply the result of study methods or limitations, and can also provide critical information to improve the design of future studies.

The Importance of Isolating for Gender

As the research presented here demonstrates, studies that explore young people and IVT use would be remiss not to consider gender as a contributing variable. Gender differences exist in how children interact with the physical environment and are also found in studies of Internet and video game use. Research in gender and literacy practices, for example, indicates that childhood objects and toys, the characteristics of different play settings, and the activities performed in these settings all influence how children interact with each other and their environment (Cassell, 1998; Heath, 1983; Jenkins, 2006; Laurel, 2003; Thorne, 1993).

Research on computers, Internet use, and video games indicates gender differences have an influence on IVT use. Colley (2003), for example, who examines gender differences associated with how computers are perceived in school settings, explains, “Girls approach computers as tools for accomplishing tasks, while boys approach them as technology for play mastery” (p. 673). Cassell (2002, p. 4) notes that girls not only participate in different types of virtual play from boys, but are also often treated differently in the context of computer use.

According to Heeter and Winn (2008), video game play, when it occurs publicly, becomes gendered. The author's note that at school, where computer screens are visible to classmates, young people “compare progress and performance” (p. 283). Hayes (2008), who studies the connections between content creation and IT proficiency of middle-school-age girls, recommends girls not work in mixed gender groups, and suggests researchers design experiments “in which girls can explore gaming and related creative practices without male competition” (p. 106).

Due to the many instances in which gender differences emerge for how young people approach both physical and virtual play, and because it appears that girls not only play differently from boys but also alter their behavior with technology when boys are around, it is imperative that studies of IVT use also isolate for gender to avoid mixed-gender confounds.

Complex Virtual Interactions Call for Complex Methodological Approaches

The research approaches described here bring various strategies to the study of virtual activity. An early study, influenced by literacy research and studies of popular children's websites, was conducted to identify opportunities for informal learning and creativity in Internet play environments. It contributed useful ways for measuring and assessing online play environments using visual data analysis techniques. While the study concluded that opportunities for creative and contributive play exist at commercial websites, children's perceptions of what they did when playing online differed from their recorded actions. The initial study suggested that subjective experience, in the form of self-perceptions, plays a crucial role in our understanding of virtual interactions.

A second research project, inspired by the first, extended the scope of the earlier study by exploring perceptions emerging from both traditional and virtual play activity. It took a participatory approach that embraced participants as research partners, as do many studies in the field of human–computer interaction. The overarching purpose of this study was to inform the design of a larger, experimental study.

The third study, conducted as a field experiment, employed both quantitative and qualitative data collection strategies. It, too, explored similar physical and virtual play activity. Most importantly, this study produced the creative agency model (CAM), a useful framework for understanding how self-perceptions that emerge through physical and virtual activity are impacted by variables such as gender and affordance recognition. Together, these various methods offer complementary research tools that can expand our understanding of how activity unfolds in virtual environments, as well as the overall impact of transferring activity from physical to virtual settings on how people play, develop, and learn.

Finally, several fundamental questions are raised by research contrasting physical and virtual modes of play:

1. What do we want to measure, and how will we measure it?
2. Will we use or adapt existing measurement tools or create new ones?
3. What can we learn by identifying perceptions of creativity and agency as these emerge in the performance of physical and virtual activity?
4. What else ought we to be measuring?

To answer these and similar questions we need to consider the goals of such research. Are we trying to determine which physical activities transfer best to virtual environments? Are we developing or advocating the use of virtual technology, including video games, Internet sites, and mobile applications, for informal or formal learning? Are we trying to positively influence design to develop technology that supports, extends, or enhances human activity? Are we making and marketing new technology to young people? And, if the purpose is purely a commercial one, can we enrich IVT design so that it considers young people as active participants in shaping virtual culture and supports their goals and desires?

Whatever questions are posed, whatever research methods are selected, adopting approaches that are informed by the performance of activity in ecologically sound contexts can provide more accurate information about which types of environmental affordances – physical and/or virtual – produce positive creativity and agency perceptions that are believed to be vital to human development in the long run.

NOTES

1 The phrase “interactive virtual technology” (IVT) was used in this research because it is more inclusive of activity that happens in Internet or Web environments, video games, and/or other digital play spaces. “Information technology” (IT) and “information communication technology” (ICT), while often used to describe technology in the context of computers and communication, or the management of information, are terms that can be all-encompassing; however, they do not necessarily conjure up playful activity. “Virtual reality” (VR) or “virtual reality environment” (VRE) were also considered, although these terms are sometimes associated with specialized, highly immersive, 3-D environments, and simulations, which are (at least for now) less accessible. IVT embraces a variety of digital activities, and as activity is the focus of this research, serves the purpose well.

2 4-H is a youth program sponsored by the United States Department of Agriculture that focuses on “hands-on learning programs in the areas of science, citizenship, and healthy living” according to the organization's website (retrieved August 1, 2013, from http://www.4-h.org/about/youth-development-organization/).

3 Bronfenbrenner later revised the formula to Dt = f(t − p)(PE)(t − p) to account for developmental changes at the time (t) in which developmental outcomes are observed, and the period (p) during which interactions between person and environment “operate to produce the outcome existing at the time of observation (t − p)” (Johnson, 2008, p. 4). (The variables associated with Bronfenbrenner's formulas for development are reproduced as they appear in the sources, in roman type.)

4 The flexibility of CAM allows gender (g) and other independent variables such as socioeconomic status, race, or ethnicity to be factored into the equation to determine whether these, too, might play a role in perceptions of IVT use. Other types of activity can also be examined or compared to determine impact on creativity and agency perceptions (i.e., physical activity or mixed reality activity using tangible or computational objects).

REFERENCES

Anderson, C. A., & Bushman, B. J. (2001). Effects of violent video games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A meta-analytic review of the scientific literature. Psychological Science, 12(5), 353–359. doi: 10.1111/1467–9280.00366

Bronfenbrenner, U. (1981). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.

Bronfenbrenner, U. (2005). Making human beings human: Bioecological perspectives on human development. Thousand Oaks, CA: Sage.

Buddenbaum, J. M., & Novak, K. B. (2001). Applied communication research. Ames, IA: Iowa State University Press.

Cassell, J. (1998). Storytelling as a nexus of change in the relationship between gender and technology: A feminist approach to software design. In J. Cassell & H. Jenkins (Eds.), From Barbie to Mortal Kombat: Gender and computer games (pp. 298–326). Cambridge, MA: MIT Press.

Cassell, J. (2002). Genderizing HCI. In J. Jacko & A. Sears (Eds.), The handbook of human–computer interaction (pp. 402–411). Mahwah, NJ: Lawrence Erlbaum.

Center for Media Literacy. (2008). Literacy for the 21st century (2nd ed.). Retrieved August 1, 2013, from http://webspace.ship.edu/hliu/etextbook/theory/doc/media%20literacy_v02.pdf

Colley, A. (2003). Gender differences in adolescents' perceptions of the best and worst aspects of computing at school. Computers in Human Behavior, 19(6), 673–682. doi: 10.1016/S07475632(03)00022-0

Cordes, C., & Miller, E. (Eds.). (2000). Fool's gold: A critical look at computers in childhood. College Park, MD: Alliance for Childhood. Retrieved August 1, 2013, from http://www.allianceforchildhood.org/fools_gold

Cross, B. (2009). Mimesis and the spatial economy of children's play across digital divides: What consequences for creativity and agency? In R. Willet, M. Robinson, & J. Marsh (Eds.), Play, creativity and digital cultures (pp. 125–144). New York, NY: Routledge.

Dourish, P. (2004). Where the action is: The foundations of embodied interaction. Cambridge, MA: MIT Press.

Druin, A. (1999). Cooperative inquiry: Developing new technologies for children with children. In CHI 99: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 592–599). doi: http://doi.acm.org/10.1145/302979.303166

Druin, A. (2002). The role of children in the design of new technology. Behaviour & Information Technology, 21(1), 1–25. doi: 10.1080/01449290110108659

Druin, A., Bederson, B., Boltman, A., Miura, A., Knotts-Callahan, D., & Platt, M. (1999). Children as our technology design partners. In A. Druin (Ed.), The design of children's technology (pp. 51–72). San Francisco, CA: Morgan Kaufmann.

Fishkin, A. S., & Johnson, A. S. (1998). Who is creative? Identifying children's creative abilities. Roeper Review, 21(1), 40–40.

Flanagan, M. (2006). Making games for social change. AI and Society, 20(4), 493–505. doi: 10.1007/s00146-006-0048-3

Gee, J. (2007a). Good video games and good learning: Collected essays on video games, learning and literacy. New York, NY: Peter Lang.

Gee, J. (2007b). What video games have to teach us about learning and literacy (rev ed.). New York, NY: Palgrave Macmillan.

Geisler, C. (2004). Analyzing streams of language: Twelve steps to the systematic coding of text, talk, and other verbal data. New York, NY: Pearson/Longman.

Grimes, S., & Shade, L. (2005). Neopian economics of play: Children's cyberpets and online communities as immersive advertising in NeoPets.com. International Journal of Media & Cultural Politics, 1(2), 182–198. doi: 10.1386/macp.1.2.18 1/1

Hayes, E. (2008). Game content creation and it proficiency: An exploratory study. Computers & Education, 51(1), 97–108.

Healy, J. M. (1998). Failure to connect: How computers affect our children's minds – for better and worse. New York, NY: Simon & Schuster.

Heath, S. B. (1983). Ways with words: Language, life, and work in communities and classrooms. Cambridge, UK: Cambridge University Press.

Heeter, C., & Winn, B. (2008). Gender, identity, play style, and the design of games for class-room learning. In Y. B. Kafai, C. Heeter, J. Denner, & J. Sun (Eds.), Beyond Barbie and Mortal Kombat: New perspectives on gender and gaming (pp. 281–300). Cambridge, MA: MIT Press.

Jenkins, H. (2006). Complete freedom of movement: Video games as gendered play spaces. In K. Salen & E. Zimmerman (Eds.), The game design reader: A rules of play anthology (pp. 330–363). Cambridge, MA: MIT Press.

Jenkins, H. (2007, February 16). From YouTube to YouNiversity. Chronicle of Higher Education, 53(24), B9. Retrieved August 1, 2013, from http://www.writingwithvideo.net/readingsReferences/chronicle.html

Johnson, E. S. (2008). Ecological systems and complexity theory: Toward an alternative model of accountability in education. Complicity: An International Journal of Complexity and Education, 5(1), 1–10. Retrieved August 1, 2013, from http://ejournals.library.ualberta.ca/index.php/complicity/article/view/8777

Kaptelinin, V. (1996). Activity theory: Implications for human–computer interaction. In B. A. Nardi (Ed.), Context and consciousness: Activity theory and human–computer interaction (pp. 103–116). Cambridge, MA: MIT Press.

Kellner, D., & Share, J. (2005). Toward critical media literacy: Core concepts, debates, organizations, and policy. Discourse: Studies in the Cultural Politics of Education, 26(3), 369–386.

Khatena, J. (1977). The Khatena–Torrance Creative Perception Inventory. Gifted Child Quarterly, 21(4). doi: 10.1177/001698627702100410

Khatena, J., & Torrance, E. (1976). Manual for Khatena–Torrance creative perception inventory. Chicago, IL: Stoelting.

Khatena, J., & Torrance, E. (1998). Khatena–Torrance Creative Perception Inventory: Instruction manual. Bensenville, IL: Scholastic Testing Service.

Laurel, B. (2003). SimSmarts: An interview with Will Wright. In B. Laurel (Ed.), Design research methods and perspectives (pp. 253–259). Cambridge, MA: MIT Press.

Lenhart, A., & Madden, M. (2005). Teen content creators and consumers. PEW Internet & American Life Project. Retrieved August 1, 2013, from http://www.pewinternet.org/Reports/2005/Teen-Content-Creators-and-Consumers.aspx

Lieberman, D. A. (1997). Interactive video games for health promotion: Effects on knowledge, self-efficacy, social support, and health. In R. L. Street, W. R. Gold, & T. Manning (Eds.), Health promotion and interactive technology: Theoretical applications and future directions (pp. 103–120). Mahwah, NJ: Lawrence Erlbaum.

Lieberman, D. A. (1999). The researcher's role in the design of children's media and technology. In A. Druin (Ed.), The design of children's technology. San Francisco, CA: Morgan Kaufmann.

Livingstone, S. (2003a). The changing nature and uses of media literacy. Retrieved August 1, 2013, from http://eprints.lse.ac.uk/13476/1/The_changing_nature_and_uses_of_media_literacy.pdf

Livingstone, S. (2003b). Children's use of the Internet: Reflections on the emerging research agenda. New Media & Society, 5(2), 147–166. doi: 10.1177/1461444803005002001

Livingstone, S. (2006). Drawing conclusions from new media research: Reflections and puzzles regarding children's experience of the Internet. Information Society, 22, 219–230.

Marsh, J. (2005). Introduction: Children of the digital age. In J. Marsh (Ed.), Popular culture, new media and digital literacy in early childhood (pp. 1–8). New York, NY: Routledge Falmer.

Mazzarella, S. R. (2005). Introduction: It's a girl wide web. In S. R. Mazzarella (Ed.), Girl wide web: Girls, the Internet, and the negotiation of identity (pp. 1–12). New York, NY: Peter Lang.

Mazzarella, S. R., & Atkins, A. (2010). Community, content, and commerce: Alloy.com and the commodification of tween/teen girl communities. In S. R. Mazzarella (Ed.), Girl wide web 2.0: Revisiting girls, the Internet, and the negotiation of identity (pp. 263–282). New York, NY: Peter Lang.

Nardi, B. A. (1996). Studying context: A comparison of activity theory, situated action models, and distributed cognition. In B. A. Nardi (Ed.), Context and consciousness: Activity theory and human–computer interaction (pp. 69–102). Cambridge, MA: MIT Press.

Neuman, S., & Roskos, K. (1992). Literacy objects as cultural tools: Effects in children's literacy behaviors in play. Reading Research Quarterly, 27(3), 202–225. Retrieved August 1, 2013, from http://www-personal.umich.edu/∼sbneuman/pdf/LiteracyObjects.pdf

Parés, N., & Parés, R. (2001). Interaction-driven virtual reality application design a particular case: Elball del fanalet or lightpools. Presence, 10(2), 236–245. doi: 10.1162/105474601750216830

Piaget, J. (1962). Play, dreams and imitation in childhood (C. Gattegno & F. M. Hodgson, Trans.). New York, NY: Norton.

Piaget, J. (1970). Genetic epistemology (E. Duckworth, Trans.). New York, NY: Norton.

Roussou, M., Oliver, M., & Slater, M. (2008). Exploring activity theory as a tool for evaluating interactivity and learning in virtual environments for children. Cognition, Technology and Work, 10(2), 141–153. doi: 10.1007/s10111-007-0070-3

Spina-Caza, L. (2010). When girls go online to play: Measuring and assessing play and learning at commercial websites. In S. Mazzarella (Ed.), Girl wide web 2.0: revisiting girls, the Internet, and the negotiation of identity (pp. 223–244). New York, NY: Peter Lang.

Thorne, B. (1993). Gender play: Boys and girls in school. New Brunswick, NJ: Rutgers University Press.

Tikhomirov, O. K. (1999). The theory of activity changed by information technology. In Y. Engeström, R. Miettinen, & R. Punamäki (Eds.), Perspectives on activity theory (pp. 347–359). Cambridge, UK: Cambridge University Press.

Turkle, S. (1995). Life on the screen: Identity in the age of the Internett. New York, NY: Simon & Schuster.

Turkle, S. (2005). The second self: Computers and the human spirit (20th anniversary ed.). Cambridge, MA: MIT Press.

Wachs, T. D. (1991). Environmental considerations in studies with nonextreme groups. In T. D. Wachs & R. Plomin (Eds.), Conceptualization and measurement of organism: Environment interaction (pp. 44–67). Washington, DC: American Psychological Association.

Willett, R. (2009). Consumption, production, and online identities: Amateur spoofs on YouTube. In R. Willet, M. Robinson, & J. Marsh (Eds.), Play, creativity and digital cultures (pp. 54–67). New York, NY: Routledge.

Wimmer, R. D., & Dominick, J. R. (2006). Research in media effects. In Mass media research: An introduction (8th ed.). Retrieved August 6, 2013, from http://rogerwimmer.com/mmr/wimmerdominick8e.htm