Introduction

It is apparent that digital media and new communication technologies are not only influencing our online interactions and the teaching of literacy skills, but also influencing our written communication. However, the effects of digital technology are far more extensive than just affecting written language: it is also shaping literacy skills in quite innovative ways. If we consider the multimodal features of the digital world, and our ability to extract meaning not just from traditional print (books) but also through visual (photographs, videos, animations) and audio materials (music, audio narration, sound effects), then we can see how literacy skills are being extended. Literacy occurs online through a variety of media including text messaging and social networking sites with tweeting seen as a new literacy practice (Greenhow & Gleason, 2012). So how have these new communication tools shaped our understanding and definition of literacy? In contrast to some of the more traditional definitions of literacy, which focus primarily on reading, spelling and the written exposition of ideas, the term ‘digital literacies’ is encouraging us to think beyond such narrow definitions and to recognize a broader skill-set that includes acquiring and evaluating information within a multimodal environment. While we have long been surrounded by visual imagery through television, films, videos and computers, images rather than words are increasingly used as a communication tool. Pictures are passed from mobile phone to mobile phone with very little text accompaniment and are easily understood particularly by the young. Indeed an emoticon, voice over or tune may be used to enhance the meaning of the image being shared rather than a conventional written description of the image. This develops greater confidence and ability in dealing with many of the multimodal aspects of digital environments.

This use of digital technology has clearly influenced our understanding of literacy skills. Given the rise in sales of electronic books, and touch-screen tablets coupled with the digital resources to enhance reading, the traditional definitions of literacy are perhaps outdated and possibly even redundant in the digital age. The term digital literacy has been considered, discussed and embraced by many academics and practitioners within recent years leading to extended definitions of digital literacy, visual literacy, television literacy and, within many HEIs, information literacy (see Buckingham, 2006). Yet digital literacy is much more than a functional matter of learning how to perform online searches or to navigate through web pages. It involves a whole new skill set centred around extracting information, evaluating sources and deciphering written and visual representations online. Being digitally literate involves a multidimensional and interactive approach to learning and the ways in which learners are developing new digital and visual literacy skills is the focus of this chapter.

Engaging with New Forms of Literacy

Traditional views of literacy provide the fundamental building blocks of the educated person but Web 2.0 technologies demand more from individuals who would call themselves literate in a twenty-first century world. We have previously seen how the nature of online communication is rapidly changing, with an increase in one-to-one and one-to-many types of online interactions, and a growth in the participatory media culture that is now emerging – in the form of blogging, social networking, podcasting and texting (Jenkins, 2006).

Engagement with digital technologies has therefore produced a new, lively community of readers and writers engaged with new tools for digital communication. The digital divide between literacy as an ‘academic’ practice and literacy as a ‘social’ practice remains evident (Lankshear & Knobel, 2003). Yet digital literacy is not a new concept, and the ability to assess, retrieve and comprehend information within a digital landscape has intrigued academics for the past 20 years. However, education has still focused on teaching literacy skills using traditional methods and has shown a reliance on print rather than screen.

Traditional views of literacy focus primarily on text-based reading and writing activities within an educational context, thus emphasizing the importance of those basics skills or building blocks commonly associated with print-based text. In the case of more traditional alphabetic skills, literacy still consists of the mastery of letters and phonemes so that a person is able to encode and decode print. Yet learners need to demonstrate certain skills in: deciphering complex images and sounds as well as the syntactical subtleties of words (Lanham, 1995, p. 200). One argument is that a focus simply on the acquisition of alphab etic skills alone remains insufficient for providing the current school cohort with essential literacy skills. Teaching traditional literacy skills and a reliance on printed texts may be outdated and restrictive, a point clearly articulated in the following argument:

Literacy is wider than reading and writing in traditional print genres … the ability to decode information in various orthographic formats, including digital media, to make and take meaning from it and to encode information into those formats to communicate ideas to others.

(Plester and Wood, 2009, p. 1109)

This argument is further emphasized in a quote by Krause (2013) who clearly advocates how the focus on relying on traditional print as a means for literacy instruction can often restrict and marginalize learners:

Traditional conceptualizations of literacy as reliant on print forms of text are outdated and unresponsive to the dynamic changes of the 21st century, thus, creating a potentially marginalized group of learners.

(Krause, 2013, p. 237)

Rather than marginalize different groups of learners, as Krause suggests, we need to teach students a much broader range of literacy skills beyond those simply provided by traditional printed texts. That is, our current teaching practices need to acknowledge how literacy is embedded not simply within textbooks but across a growing range of media texts, including print (books), visual (photographs, videos, animations) and audio (music, audio narration, sound effects). Our society is saturated with different forms of digital media and knowing how to access, evaluate and apply information is necessary for success in the workplace and at school and this requires learning a new literacy skill set. In essence we need to reskill the current generation of learners.

So Which Literacy Skills are Required to Become a Digital Native?

Changes in literacy practices appear synonymous with the introduction of new digital technologies. For example, our early engagement with the Internet, often defined as Web 1.0 technology, required us to use a version of the web containing static and often linear forms of information, that is material to be downloaded or delivered to the Internet user. Within this context, literacy skills often relied on our ability to read, identify and select material online within a fairly structured and linear way. Often skills required for online literacy appear analogous to those required for reading printed text. With the introduction of Web 2.0 technologies, these skills have changed and information is available in a more participatory, dynamic and social place with an emphasis on uploading information for communication, collaboration and discussion as well as on reading information in a non-linear, and highly interactive way (Sharples, 2010). Accessing digital information, especially within the context of Web 2.0 formats, now requires a greater reliance on interactivity, collaboration and communication rather than simply deciphering written texts online (Greenhow & Gleason, 2012). For example, children using the Internet need to learn how to locate and select appropriate material. In essence, this involves learning how to use browsers, hyperlinks and search engines, and to read information effectively online. While the reading and comprehension of written material still applies, the way in which this information is retrieved, interpreted and delivered to others is changing as a result of the rapid increase in our engagement with these digital technologies (Buckingham, 2007). There is an implicit expectation that learners will embrace a wide range of skills, including cognitive, motor, sociological and emotional skills, in order to function most effectively within this new highly interactive digital landscape (Kress, 2010).

Contrasts between the characteristics of written communication on the page and on screen have been carefully explored (Kress, 2003, 2010; Merchant, 2007). Arguably, using new forms of digital technology to assist with promoting online communication with a remote audience provides a further dimension to conceptualising literacy work in the classroom, opening up new possibilities for young writers (Merchant, 2005, 2007). Some researchers (Gee, 2003; Jenkins, 2006; Lankshear & Knobel, 2003; Lankshear, Knobel, & Curran, 2013) contend that any social activity that involves the use of digital technology, such as browsing the Internet, transferring images or playing video games, promotes some literacy practices that are often ignored by teachers or educators.

There is also some recognition that technologies are changing how writing is produced, received and delivered to an often much wider range of audiences. Many popular literacy practices show a commitment to more inclusive, collaborative and participative forms of written communication. Written information plays a central part in many screen-based forms and some of the most popular of these, such as email, texting and blogs, are conducted predominantly through the written expression of ideas where traditional conventions are often ignored or adapted for convenience. The previous chapter has shown how text message abbreviations succeed admirably in promoting new conventions for written language. As learners engage within a broad spectrum of applications such as social networking sites (SNSs), collaborative authoring, information sharing and multiplayer games, they are using their written language to exchange ideas and communicate effectively. There are claims that students’ text-based interactivities on social networking sites, such as Facebook, represent a new dimension of social literacy practices (Davies, 2012; Gee, 2003). There is much educational potential and some suggestion that practices involving digital literacy can fruitfully bridge gaps between people’s home and school learning lives (Davies & Merchant, 2009; Willett, Robinson, & Marsh, 2009).

The Multimodal Landscape

The ability to function within a multimodal environment is a crucial aspect of becoming a digital native. Paivio’s (1986) dual-route model is often used to explain issues of transfer and integration of meaning between words, images and other symbol systems. In this model verbal and non-verbal information is held in the dual-coding system, which suggests that the two systems are intricately connected. However, this surfeit of images does not necessarily lead to deeper understanding. Eye-movement studies of primary school children’s abilities to integrate text and images have shown that this does not take place automatically and the higher the prior knowledge of the topic, the more able the children were to integrate material from different modalities. Mason, Tornatora, and Pluchino (2013) assumed that such prior knowledge allowed the child to exhibit more strategic behaviour and integrate verbal and pictorial information. This ability to integrate material or information across different modalities has been associated with improved academic performance. Students assume that they are able to comprehend images well since they can be processed so much faster than written or audio text (Schroeder, et al., 2011). This is demonstrated by the following quote from a middle-school pupil: Seeing is beeter [better] than hearing, seeing is believing (Hibbing & Rankin-Erikson, 2003, p. 765). This is a dangerous assumption on every level. For example, Hibbing and Rankin-Erikson have demonstrated that children with low-reading ability have difficulty in creating and using images because they are so focused on their attempts to decode words. The cognitive load of such decoding for a poor reader does not leave spare cognitive capacity to integrate the pictorial and textual information.

So while the multimodal textual landscape of digital literacy is widely established (Kress, 2010) extracting meaning from a range of different representations can be a highly complex process. This involves the conversion of information between one form of representation to another, which requires understanding of relationships between the different external representations (Mayer, 2003). This ability has been termed ‘representational competence’ (Kozma & Russell, 1997). Without such competence, like the young readers in Hibbing and Rankin-Erikson’s study, learners will fail to stitch together the information available to them in these multiple representations. Learners need to be familiar with the visual conventions used in such representations, which are prerequisite for extracting the idea that is being represented (Ainsworth, 2006).

However, this shift towards digital literacy is not without its critics. Concerns are raised about learners being so immersed in web-based technologies in their broader lives that they have difficulties engaging in more conventional forms of study and educational practices, such as academic reading and writing essays (Lea & Jones, 2011). The complex inter-relationship between literacy and technology may well have the potential to disrupt traditional academic practices, especially among those teachers who are less familiar with digital literacies and feel that literacy should be confined solely to the realm of writing (Barton, 1994; Kress, 1997). Despite concerns about the threat to conventional standards of literacy, the major shift in the use of digital technologies to enhance reading and writing practices raises important questions about our conventional definitions of literacy, and more importantly, its current role within education, society and culture.

Visual Literacy and Visual Representations

While digital literacy skills are almost second nature to many of those considered to be digital natives (Prensky, 2001), the ability to extract written information online, as we have alluded to above, is just one aspect of this skill set. Educators are recognizing the importance of helping students develop visual literacies in order to survive and communicate within a highly complex visual world. A key skill is the learner’s ability to read intuitively and freely read and interpret information in visual-graphical forms by deciphering and interpreting symbols and other visual representations. Much can be gained from understanding how individuals use these visual representations, particularly for the purpose of promoting collaboration, discussion and problem-solving (Salomon & Perkins, 1989).

Visual literacy is often defined as the ability to evaluate, apply or create conceptual visual representations using digital formats. One way in which we can help support learning and aid digital literacy is through the use of visual-based external representations. Diagrams, graphs, and pictures are a few typical types of these external representations. But external representations are much more than memory aids. External representations are so intrinsic to the tasks, that they guide, constrain and even determine the pattern of cognition. Arguably, external representations may facilitate not just inference or problem-solving but also enhance learners’ conceptual understanding within a given domain (Suwa & Tversky, 2001). These representations may be static forms of representations (e.g., diagrams) or more advanced dynamic forms of representations that display pictures, animations and sounds graphs simultaneously.

How do these visual representations work? External visual representations capture and structure knowledge using: physical symbols, objects or dimensions (e.g., beads of abacuses, dimensions of a graph) which have a physical configuration (e.g., spatial relations of written digits, visual and spatial layouts of diagrams, physical constraints in abacuses) (Zhang, 1997, p. 179). We use these representations to support many of our day-to-day tasks from the mundane grocery shopping list, assessing changes in the popularity of political parties through graphs and tables, or driving to an unfamiliar destination with the help of a road map or, more often these days, an audio voice-over from our satellite navigation system. However, it is their role as powerful representational aids to understanding, problem-solving and learning, that is the focus here (see, for example, Larkin & Simon, 1987; Novak, 1990; Zhang, 1997). There is also some suggestion than encouraging the use of more than one single visual representation can support students’ learning because they will: capture a learner’s interest and, in so doing, play an important role in promoting conditions for effective learning (Ainsworth, 1999, p. 131). The use of multiple forms of representations can often be found within the classroom, particularly in supporting students’ leaning of complex concepts in mathematics, science and geometry. For instance, using a combination of graphs, equations and tables to develop students’ understanding of complex arithmetical functions can often aid learning and deepen understanding (see Ainsworth & Van Labeke, 2004).

These visual representations can be identified, analysed and interpreted through our perceptual systems and integrated into a range of different learning activities across many subject areas. This encourages, and may promote, active learning and a deeper level of processing academic information. Certainly, access to these representations can support students’ academic performance, especially when attempting to solve difficult cognitive tasks. For instance, Larkin and Simon (1987) argue that diagrammatic representations often aid learning because learners can recognize features easily and make inferences based on those diagrams while Stenning and Oberlander (1995) argue that diagrammatic representations such as Euler circles can help learners to make inferences in logical reasoning tasks. Therefore, our ability to use representations as another form of literacy can have a crucial impact on other aspects of the learning process.

Why should the use of these, often highly complex, visual representations support learning? The argument for the value of external representations for cognitive learning is that they reduce working memory load. By acting as an external memory store such representations leave spare working memory capacity to conduct those mental operations often considered essential for problem-solving (Newell & Simon, 1972; Tversky, 2001). They also serve as visuo-spatial retrieval cues for long-term memory. In addition to providing working memory capacity external representations facilitate higher order thinking skills by making the problem both external and visible. The manipulation of symbols and objects in space allows judgements and inferences to take place when solving problems. The use of symbols and objects lies at the heart of pedagogical practice, for example, early years mathematics teachers used simple counters and more complex representational objects such as Dienne’s rods to instil the wonders of number into their young charges’ minds. If we consider the task of calculating, counting or solving simple arithmetic problems, learners often perform better if they have access to these external representations (such as an abacus) to help support mental calculations. Even on more difficult scientific problems, such as interpreting velocity and speed, students can often draw a velocity–time graph to help aid or scaffold their understanding. Having access to such physical representations reduces memory load and provides concrete representations to aid performance. In this instance, the choice of representation can influence an individual’s conception of a problem and hence the ease of finding a solution (Suthers & Hundhausen, 2003)

How Can Visual Representations Support Learning?

In order to assess how external representations may impact on students’ thinking and learning it is important to address how these visual representations are being used. Much of what we do inside and outside the classroom involves us fitting our thinking to a standard format even when we create the representation. For example, there is standard format for representing data in a graph, particularly within science or geography which all students are expected to follow. The creation of individual personal representations is seldom called for, particularly in formal education, although cognitive or mind maps are one such example.

Using a preformed visual representation may help to support learning. When learners are asked to work through a task using the prestructure of an external representation, as a background for extending their own thinking, they need to integrate their own externalizations into the given prestructure. In such cases, the structure of the representation can act as a cognitive tool to help guide students through their own intuitive learning process (e.g., Baker & Lund, 1997, Pfister & Mühlpfordt, 2002). The key issue for education is to ensure that the type of representation is matched to the demands of the situation because it is only then that performance and understanding improve. If the type of external representation appears vague or difficult to interpret, then its value as an education tool remains somewhat limited. Identifying the correct type of representation is therefore crucial for learning success. Ainsworth (2006) identified a number of cognitive prerequisites for ensuring that external representations can support and improve learning. She suggests that learners need to:

1. understand the representation
2. understand the relation between the representation and the task
3. understand how to select the most appropriate representations
4. recognize how to, when appropriate, construct their own form of external representation of any given task.

As we know: learning complicated scientific concepts, interacting with multiple forms of representation such as diagrams, graphs and equations can bring unique benefit (Ainsworth, 2008, p. 191). Yet relying solely on pre-existing representations may have its limitations. As Ainsworth (2006) argues, it is not simply the ability to interpret and understand any given representation that remains important but the ability to generate our own form of visual representation. Scientists themselves do not simply use words to express complex scientific ideas but also rely heavily on diagrams, graphs, videos and photographs to make important discoveries and explain their, often highly complex, findings. Students can also benefit from a similar process (Ainsworth, Prain, & Tytler, 2011). There is growing evidence to suggest that encouraging learners to construct their own set of representations, such as creating diagrams or graphs, can aid their conceptual understanding. However, as Ainsworth, Prain, and Tytler note, students are often asked to interpret other visualizations and rarely rely on generating their own visual representations while solving scientific problems. Yet, it is suggested that allowing students to generate their own representations may in fact lead to a deeper and more conceptual level of understanding and will help them to translate difficult concepts into more explicit forms of knowledge. For example, Grossen and Carnine (1990) explored learners’ ability to solve logic problems and found that problems were solved much more effectively when they were asked to develop their own form of representation rather than relying on a pre-existing representation devised by another. Furthermore, modelling through pictorial representations can provide a really useful structure to support the scientific discovery and learning of many students who find scientific concepts difficult to grasp (Leenaars, van Joolingen, & Bollen, 2013).

Such metacognitive processes, that is ‘thinking about your own thinking’, are supported by drawing diagrams or developing representations, allowing learners to constantly monitor their own progress in what they are learning and ensure that the representation is closely tied to the task domain. Drawing pictorial representations is thought to support thinking and reasoning processes. One example for the effective use of external visual representations within education is that of learning about complex scientific concepts. If we consider the complexities involved in grasping concepts, such as velocity, time, speed and distance, particularly within physics, these can often cause great difficulty without the use of external representations to guide learners’ understanding.

A further way of developing our own representations is through our ability to generate self-explanations while attempting to solve problems. Ainsworth and Loizou (2003), for example, found that being presented with a diagram rather than text promotes a greater level of self-explanation leading to a greater incidence of students’ learning. Those in the text condition produced significantly fewer self-explanations and showed limited benefits to their overall learning when compared to those who were encouraged to generate self-explanations when presented with visual representations or diagrams.

While representations may facilitate learning in some instances they may inhibit the learning process especially if these representations are inconsistent with the demands of the task (Ainsworth, Bibby, & Wood, 2002). While certain representations, for example, graphs, diagrams or even animations may work for one person, they may not work for another, so recognizing how individual learner characteristics may constrain or facilitate the use of representations is crucial. We also know from research on perception that such visuo-spatial displays, especially vague and ambiguous ones, can be interpreted and reinterpreted in many different ways. Familiarity with the representation is crucial for learning – a learner who fails to understand the relevance of any representation will undoubtedly fail to use this to support their learning activity. Similarly, if learners misinterpret representations, then their ability to solve any given task may suffer as a result because the representation will act as a further distraction from the task in hand.

While visual representations remain an important aspect of digital literacy, particularly within the current digital landscape, these skills need to be embedded firmly into our digital lives. As Friere (1994) asserts, the development of any literacy takes off when it speaks to the needs of the individual and is clearly exemplified by the rapid assimilation of digital technologies into the fabric of the lives of those of us that are less than 25 years of age. These skills are not encapsulated in an older generation of teachers but they are being developed in the digitally literate younger generation and we would argue Internet users and gamers are at the forefront of these new literacy practices (Gee, 2003; Underwood, 2007). Prensky (2005) sums this up when he expresses his amazement that the debate on education standards in the United States has not taken into account the fundamental fact that in a digital world the students themselves have changed. According to Prensky, the younger generation are digital natives while we, the teachers, are mere tourists. It was further argued that digital natives’ brains are likely to be physically different from those of digital tourists as a result of the digital input they received while growing up.

Risks, Skills and Opportunities

Although the evidence suggests digital literacy skills are an important aspect of our student’s education, particularly how they extract information from a range of digital resources from images, graphs and text, there are potential risks in promoting digital literacy within the classroom. As we have seen, there is a lack of a clear definition of the exact skill set required to become digitally literate. Neither is there any clear advice of how digital literacy skills should be embedded within classroom practices. Nonetheless, learners need to be taught how to navigate effectively within a multimodal digital environment and to extract information from visual representations. The ability to read information through visual forms is crucial for learning within this digital age, and visual representations can act as a cognitive tool to help guide students through their own intuitive learning process. However, students need to be taught how to interpret visual representations effectively. This may have a small impact on the able but, without explicit teaching, for the less able there is a very real possibility of the digital divide becoming an unbridgeable chasm. There are some encouraging examples of embedding digital literacy into the curriculum. For instance, there are clear examples of the use of blogs to promote digital literacy activities and peer-collaboration within the classroom context (Huffaker, 2005; Mullen & Wedwick, 2008) and some suggestion that these digital literacy skills can fruitfully bridge gaps between people’s home and school learning lives (Davies & Merchant, 2009).

Conclusions

The need for the ability to work within a multimodal digital landscape remains clear (Kress, 2010) and we are aware that many children deal with written and visual representations prior to receiving any formal training or education. They are highly experienced in surfing the net, using images and audio to share and express ideas as well as interpreting visual pictures or diagrams. For many, these skills are second nature. Their engagement with new technologies is having a profound effect on their developing literacy skills. To be digitally literate, students’ need to embrace the multimodality of their digital environments and such skills embrace much more than simply reading or decoding text on a digital screen. The required skill set often goes beyond the more traditional values of functional literacy to include collaboration, communication and synthesizing ideas in a structured and often non-linear fashion. This can involve using hyperlinks or web pages to identify written information and creating and authoring digital resources through video, multimedia, and audio recordings – core literacy skills needed within a rapidly changing and highly digital world. Using written texts online, while still an essential skill, represents just one aspect of digital literacy and learners need to be taught how to incorporate visual representations to support their learning activities and to work with multimodal representations beyond that of the written word.