CHAPTER 5

Learning With Emerging Technologies

At the heart of any evaluation of emerging learning technologies is, of course, the phenomenon of learning. Related to learning are the many activities we call teaching—helping others to learn. While teaching and learning are not the same, they are related. In this chapter we lay out the relationship between learning and learning technologies to develop evaluation criteria for emerging technologies that can support learning, teaching, assessment, and the management of learning activities.

The use of technology in learning dates back to at least the 13th century, when medical lectures were delivered at the University of Bologna, and moves forward to the recent use of brain implants, which can improve memory by up to 30 percent (Bates 2014; Hamzelou 2017). Lecturing (from the Latin, “to read”) began as a copying technology. It started with monks reading a single book out loud to other monks who wrote down what they heard to produce additional copies of a manuscript. Although lecturing and reading have persisted until the present day, many new learning technologies present alternative ways to deliver or access information. Just as the invention of reading changed how the human brain functions, the heavy use of digital devices is also changing our brains. This has prompted literacy expert Maryanne Wolf (2018) to call for “biliterate” education for young people, equivalent to learning two languages, where both reading-based and digital-based approaches to thinking are taught in schools. More on that idea later.

There is no question that the field of learning “has become very complex, with different foci, founders and proponents, schools, and disciplinary approaches” (Qvortrup and Wiberg 2013). At the same time, we agree with JD Dillon (2017) that “at no point … is L&D expected to throw away everything it’s already doing.” In one of the first critiques of the impact of computer technology on human knowledge, Dreyfus and Dreyfus (1986) used philosopher Gilbert Ryle’s differentiation of “knowing-that” and “knowing-how” to argue that human problem solving and knowledge application depends on context, rather than a process of searching through everything we know for the right answer to a given problem. Biggs and Tang (2011) make the distinction between surface learning (based on memorization) and deep learning (based on understanding). Recently, Frank (2017) argued for the separation of mimetic learning (memorization and recall) from transformative learning (learning to be creative, intuitive, and feeling based), suggesting that digital adaptive tutoring programs strongly favored mimetic learning at the expense of transformative learning. How do we make sense of all these different ways of learning and their interactions with emerging learning technologies?

One way is through rigorous experimentation, which is what Nobel laureate Daniel Kahneman (2011) has done with a lifetime of careful research on human thinking. He reconciles many of the distinctions we’ve listed with his dual-process theory, confirming that, in terms of how we think, we are two selves in one body. One self is fast thinking, intuitive, and heuristics-based, whereas the other is slow thinking, rational, and analytical. The two systems within one brain shown here are adapted from Canadian philosopher Joseph Heath’s 2014 book, Enlightenment 2.0:

• System 1. The Experiencing Self: Fast, Intuitive, Heuristic

– unconscious, automatic

– rapid, computationally powerful, massively parallel

– associative

– pragmatic (contextualizes problems in the light of prior knowledge and beliefs)

– does not require the resources of central working memory

– functioning is not related to individual differences in intelligence

– low effort

– prone to biases and errors.

• System 2. The Remembering Self: Slow, Rational, Analytical

– linked to language and reflective consciousness

– slow and sequential

– linked to working memory and general intelligence

– capable of abstract and hypothetical thinking

– volitional or controlled—responsive to instructions and stated intentions

– high effort

– prone to losing focus due to fatigue or interruption of attention.

The implication of Kahneman’s research is that we need to consider both systems when designing instructional technology for workplace learning (or education-based learning). It may turn out that the exciting, stimulating, multimedia approach of augmented and virtual reality works best for training the experiencing self, while a more formal, careful presentation or reading approach (whether online or in a classroom) works best for instructing the remembering self. Kahneman’s research may put an end to the controversy over which approach to learning is better, as both approaches will be needed in the future. And, in spite of its detractors, classroom technologies and direct instructional strategies, such as presentations, may provide important scaffolding for the development of rational thinking and memory work appropriate for specific kinds of learning tasks (Wood, Bruner, and Ross 1976; Heath 2014).

For instructional designers, the problem is that the heuristics used by the experiencing self are prone to behavioral biases and systematic errors. So, despite managers acting fast and feeling right, their decisions may need to be supervised by the more rational remembering self. For example, according to Lejarraga (2010), “Entrepreneurs learning from descriptive sources (e.g. market analyses, industry reports, or records of entrepreneurial ventures) about the potential payoff distribution of a given venture make more Type I errors (selecting poor ventures), while those learning from self-experience make more Type II errors (forgoing promising ventures).” Instructional designers need to take this kind of information into account when designing learning materials. Moving forward, learning and talent development departments need to understand and develop instructional strategies for both kinds of thinking (Campbell 2015).

The Rise of Work-Based Learning Technologies

The importance of work-based learning has grown with the ever-increasing pace of technological change. One estimate of the critical importance of continuous and lifelong learning is that “technology skills have to be updated every three years in order to have continued relevance” in today’s economy (Grand-Clement 2017). Although a baseline of knowledge is necessary for any job or profession, the need to always know specific information has decreased in importance, while the ability to find information when needed and assess its quality is now a critical skill for adult learning. In this section we offer our views of the process of learning and relate how learning technologies are being used and might soon be used at each stage of the learning process.

Lachman (1997) proposed the following definition of learning: “Learning is the process by which a relatively stable modification in stimulus–response relations is developed as a consequence of functional environmental interaction via the senses … rather than as a consequence of mere biological growth and development.” We like Lachman’s definition for several reasons. Besides emphasizing that learning is a process of relatively stable modifications in response to stimuli, we agree with his emphasis on the fact that learning takes place within a context and necessarily involves the senses. While we elaborate the process of workplace learning in more detail than does Lachman, we think his definition of learning makes for a good starting point in our discussion. At the same time, our concepts of learning are changing. The conventional view of the process of learning that has dominated classroom teaching was articulated by Robert Gagné (1985) more than 35 years ago when he defined learning as “a change in human disposition or capacity that persists over a period of time and is not simply ascribable to processes of growth.” He listed these nine steps as “events of instruction” (Gagné, Briggs, and Wager 1992):

1.   Gain attention.

2.   Orient the learner.

3.   Stimulate recall of prior knowledge.

4.   Present content material.

5.   Provide learner guidance.

6.   Elicit performance “practice.”

7.   Provide informative feedback.

8.   Assess if lesson objectives have been learned.

9.   Enhance retention and transfer.

While the process outlined by Gagné describes a conventional direct instruction approach, much has changed thanks to the cognitive revolution in psychology and the tremendous advances made in neurology over the past 20 years. A more updated view of processes involved in learning is provided by How People Learn II: Learners, Context and Cultures, perhaps the most comprehensive review of the latest research on learning available today. This collaboration of the National Academies of Sciences, Engineering, and Medicine (2018) identifies five learning strategies for which there is good evidence of effectiveness:

• Retrieval practice. The act of retrieving information enhances learning it, and the ability of a learner to retrieve and use knowledge again in the future is enhanced.

• Spaced practice. Spaced practice (compared with massed practice such as cramming) distributes learning events over extended periods of time and shows greater positive effects across learning materials and stimulus formats, for both intentional and incidental learning.

• Interleaved and varied practice. Practicing skills in different ways and mixing different activities in the same learning session promotes better learning.

• Summarizing and drawing. Producing a verbal description that distills the most important information from a set of materials and creating a diagram that portrays important concepts and relationships both enhance learning.

• Constructing explanations. Techniques of elaborative interrogation, self-explanation, and teaching others all have been shown to facilitate learning.

None of the above resembles passive students listening to lectures, but active involvement in the act of learning at all times. While learning involves memory, learned memories are reconstructed in our brain each time we remember something (National Academies 2018). Learning is a dynamic and ongoing process of connecting memories and a current problem or context, not simply the storage of information for later wholesale retrieval. But, near verbatim reproduction of presented materials or texts has been the gold standard for success in passing assessments for a very long time.

Marshall McLuhan noted in the 1960s, “We look at the present through a rear-view mirror. We march backwards into the future” (McLuhan and Fiore 1967). It is not surprising, then, that the first vendors of digital learning technologies created standard school-based instructional applications, such as the delivery of textbooks, notetaking, courses, lectures, and assessments, as well as class management procedures, such as taking attendance, daily planning, recording grades, and producing report cards. Learning management systems (LMSs) and learning content management systems (LCMSs) simply bundled most of these functions into one big, and usually expensive, program. Content was presented onscreen in much the same way that a teacher would put content on a blackboard or show an occasional movie. Innovations such as classroom response systems (such as clickers) and presentation software (such as PowerPoint) freshened the classroom experience but did little to change the relationship between learners and instructors.

Before the Association for Talent Development (first known as the American Society for Training Directors and then the American Society for Training & Development) was founded in 1943, most training in industrial settings was either done on the job or close to the job as vestibule training. During and after World War II we saw the beginnings and growth of purpose-specific training classrooms. There, training, usually provided and managed by the human resources department, became more formal and aligned to designated job skills to support specific business units. The end of the 1990s saw the growth of classroom-based and online course–based corporate universities, which were more aligned to wider organizational goals. Instructors followed a stepped curriculum and held the status of workplace learning and performance professionals, usually reporting to a chief learning officer (Abel 2008; Meister 1998). Training also took place in seminars and conferences, which were both extensions of the classroom metaphor for training.

But with the arrival of mobile and immersive technologies, learning could take place anytime, anywhere (Udell 2012). Since the beginning of the new millennium, learning departments, talent development programs, and corporate universities have scrambled to catch up to the implications of the newest technologies. The change is best summarized by Prieto, Dimitriadis, and Asensio-Pérez (2014) who observed that learning environments (including classrooms), “are becoming messy, complex socio-technical ecosystems of resources.”

Given this latest shift to learning within “complex socio-technical” environments, traditional methods for evaluating learning technologies must also change. From the 19th century to the 1960s, the gold standard of scientific research methods has been experimental evaluation, with an emphasis on control through double-blind studies, a random selection of subjects, and the use of rigorous statistical procedures to come to conclusions with a specified degree of confidence (usually 5 percent or 1 percent). Since World War II we have also seen the rise of ethnographic evaluation, with participant observation, careful recording of both behaviors and contexts, and “thick description” of results (Geertz 1973). Starting in the 1980s, the emerging wave of evaluation methods has been environmental evaluation based on network analysis, ecological interdependencies, complexity theory, and emergence. On the horizon (and already here in a few instances) are algorithmic evaluations by computers using big data, machine learning, and artificial intelligence for pattern recognition and correlation of connected variables. While examples of all four approaches can be found in the latest research on work-based learning, it is clear that a more complex and ecological view of the meaning of enterprise learning and its new technologies is pushing us to rethink how we approach the evaluation of learning technologies.

Faced with constant change, an explosion of available information, and myriad resources, the instructor in an enterprise has moved from being principally a source and presenter of knowledge to a guide and curator for individual and group learning. At the organizational level, the instructor has become a conductor who orchestrates teams to learn to operate in the most optimal way for a business (Prieto et al. 2011; Dillenbourg 2013). Further complicating the matter, employees are increasingly moving toward “self-regulated workplace learning,” reducing the roles of learning professionals altogether (Siadaty et al. 2012).

Clearly, we are in a transition period between two approaches to delivering training to employees. While new technologies are now used in the corporate world to support self-directed learning, it is still the case that “passive learning … consisting largely of sitting down and then consuming pre-packaged content in bulk that’s presented formally by an educator” remains the norm (Hinchcliffe 2017). In our opinion, this is about to change. Workplace learning is increasingly being facilitated through new teaching strategies, including “modelling, coaching, questioning, scenario building, organizing and sequencing of workplace experiences, encouraging interpersonal interactions, helping to identify learning conditions, and teaching in the use of learning strategies” (Snoeren, Niessen, and Abma 2015). Emerging learning technologies are having effects that are only going to accelerate.

What has changed in workplace learning is the realization that going digital allows for possibilities that have never existed before, and that some of these new opportunities improve the competitiveness of a business. In the design world, “affordances” are the qualities or features of an object or an environment that allow an individual to perform an action (Norman 1988). The new affordances of digital technologies allow for many other possibilities to support teaching and learning than those found in a traditional classroom. As vendors learn to innovate and the learning and development market accepts possibilities beyond the usual “tell and test” procedures common in the era of in-class instructors and three-ring binders, new features have gradually been introduced to emerging learning technologies, such as:

• mobility, making anytime learning possible from any location with access to the Internet

• networking to anywhere in the world allowing access to vast amounts of information

• social media that allows peer-to-peer commentary on user-created content

• collaboration tools that support working with others

• location-based (using GPS) applications that permit algorithms to understand and use contextual cues

• games and gamification techniques that improve motivation

• sophisticated search software that supports do-it-yourself learning

• cloud computing that allows content to be retrieved from any location

• self-tracking and immediate feedback

• sensors to collect and store massive amounts of data on individuals and groups

• artificial intelligence and data to personalize content and interactions with software.

As a society, we are only now realizing the new possibilities that the abilities of digital devices can bring to the design of learning technologies. Few of these affordances were possible in the classroom, and all are still in early stages of development. And, they all interact together, creating a new reality for learning leaders called intertwingularity (Nelson 1974). The term refers to the complexity of the interrelations of human knowledge whereby the cross-connections among topics cannot be easily divided into simple categories. Thus, learning technologies are intertwingled, and cannot be treated as separate, unrelated technologies for learning. The skill set needed to operate in this new environment requires managers and instructional designers to step up their game in terms of their own personal learning and collaboration with others (Woodill, Udell, and Stead 2014).

At the same time, there is lots of resistance to change, and vendors never want to get too far ahead of the market in their offerings. Despite what learning and development departments or vendors want, many employees, especially younger ones, have discovered these new affordances and often bypass official channels to use them. This trend is only going to accelerate as the capabilities of digital devices, especially smartphones and global networking, rapidly improve, and the working populations shift to include even more new entrants.

The Workplace Learning Process

Williams (2010) describes three key elements of work-based learning that distinguish it from learning in an educational setting:

• learning is acquired in the midst of action and dedicated to the task at hand

• knowledge creation and utilization are collective activities where learning becomes everyone’s job

• learners demonstrate a learning-to-learn aptitude, which frees them to question the underlying assumptions of practice.

For these reasons, a workplace learning experience is often more innovative and variable than a school-based learning experience.

At the same time, we need to recognize that, for many employees, workplace learning is tacit, undocumented, rooted in repetitive body movements and “muscle memory,” and often not spoken about explicitly. It is just something that happens as we work and absorb job procedures from the environment and those around us, and continually practice them. It is a social and collective process where knowledge is both co-constructed by, and held within, a local group of workers. It is often tribal, where the knowledge created holds its value within the tribe, sometimes with little usefulness beyond group boundaries.

If we step back and take a system view of learning, the description of the process for Western culture might look like this: Because human beings exist in the world, all teaching and learning necessarily takes place within a context or environment. The process starts with getting a motivated learner’s or learners’ attention, followed by a stimulus or experience that, combined with memories, results in a modification to the person’s usual response to that stimulus. That modification, which is also stored in memory, can be strengthened through practice, spacing, and other techniques for improving retention. As this happens, data can be collected using a variety of technologies and used to assess whether learning has taken place. These data and assessments can be further examined using the techniques of learning analytics, which can form the basis of reports or feedback and be used to justify the issuance of credentials that attest to the fact that accumulated learning has taken place. That learning is now available to be combined with new contexts, problems, and experiences in an iterative loop (Figure 5-1).

Figure 5-1. Stages of a Learning Process Involving Current Western Cultural Practices

At each stage of the learning process, digital technologies can be introduced to support or take over parts of the process. Some of these technologies are being offered with great fanfare right now, while others are on the horizon and will likely appear and become commonly used in the next five to 10 years. The central question for this book is how do we evaluate these emerging technologies?

One of the most accepted definitions of technology was formulated by sociologist Read Bain in 1937: “Technology includes all tools, machines, utensils, weapons, instruments, housing, clothing, communicating and transporting devices and the skills by which we produce and use them.” A learning technology is anything that can be used to support learning, teaching, assessment of learning, or the administration of learning or teaching practices.

Today, by “learning technologies” we mostly mean digital information and communications technologies (ICT), but textbooks, classrooms, blackboards, desks, school buildings, conferences, lecterns, and analog audiovisual equipment also qualify as learning technologies, even if they are not digital. By “emerging learning technologies” we mean mostly digital devices, products, or services that have newly arrived or are on the horizon for likely implementation in the next five years.

The most commonly talked about new technologies currently on offer from commercial vendors and research organizations such as universities and corporate labs include 3-D printing, cloud computing, simulations, 360-degree virtual worlds, enhanced search algorithms, artificial intelligence, machine learning, computer vision, augmented reality, adaptive tutoring systems, wearable devices, the Internet of Things, and robotics. We believe that other, even more exotic or nascent technologies, such as quantum computing and embodied digital devices will be available or coming in the next 10 to 20 years and may also assist in learning. For example, while much work needs to be done to make quantum computing viable for everyday use, it is possible to try out several simple quantum computing apps right now (Captain 2018; Russon 2018). Whatever the changes, the world of learning and talent development will surely look much different 10 years from now.

The Workplace as Context

As embodied human beings living in a specific location, learning always has a context. Sometimes the context is simply the place or time in which we find ourselves, while in other instances we are in a specifically designed learning environment; a place (for example, a learning lab, classroom, or simulated environment) that has been deliberately set up to facilitate learning, and which, therefore should be seen as another learning technology. Sometimes this place is virtual, as in a set of goggles that tries to reproduce the sensations of a realistic locative experience. Other times this place is augmented, with a mixed reality overlay that blends digital media with a physical space or location.

Describing the physical environment as a technology is not the only aspect of context to be considered in any analysis of learning (Hinton 2014). Also relevant are:

• the affordances of the individual physical elements in an environment

• the semiotics of both natural and designed elements including color, space, sounds, and visualizations

• the semantic functions of language as an environment, both as text and spoken words

• the personal context that people bring to their environments including embodiment, identity, situations, interests, and past narratives

• the social dimensions when other people are in an environment

• digital information that is available to augment the immediate information in a specific setting

• mapping systems of meaning by combining these elements.

Understanding and using context to support learning is often called situated or situational learning and can be facilitated in several ways using emerging multimedia technologies (Tretiakov and Kinshuk 2003). Yusoff, Zaman, and Ahmad (2010) describe a mixed reality technology supported by 3-D graphics, animation, text, narration, and music to give biomedical students some exposure to the regenerative concept and in-vitro processes of animal tissue culture, but note that there is little research on the effectiveness of this approach. Batson (2011) argues that adults learn best with “situated learning, a humanistic view of learning that envisions learning in real life occurring constantly, outside of the classroom as well as in the classroom” and advocates for the use of e-portfolios as the best way to document this type of learning. Leinonen and colleagues (2013) use contextual inquiry and participatory design to develop a number of new technology-enhanced techniques for the construction industry, including wearable point of view cameras, RFID tags, and augmented reality headsets—all integrated into the workflow of a job site. These are all examples of setting up a learning technology as a situated environment.

One related emerging technology to watch is “context modelling for learning” (Yin et al. 2015), whereby several standardization efforts have been undertaken to develop learner models and learning objects embedded in a learning context. One example of context modeling is the IMS Global Learning Consortium’s (2018) Learner Information Package, which defines the context for ubiquitous learning as the “learner’s state,” “the educational activity state,” “the infrastructure state,” and “the environment state.” Each state has its own set of variables and its data can be collected for future analysis and used within machine learning algorithms. This approach is an example of how context is being turned into data to be used later in analytics and personalization.

The Learner as Context

Before individual learning can take place, there needs to be a motivated learner willing to learn. Employees are part of the context of a workplace, and each one brings a personal learning orientation, past work and social history, embodied characteristics such as physical abilities and intelligence, and various degrees of motivation to get a job done. Motivation can be supplied through personal interests, a set of goals that orient the learner, fears, desires, or simple enjoyment of learning.

Yet, a 2013 Gallup poll reported that almost 70 percent of Americans who work are “actively disengaged in their work.” This lack of engagement costs the U.S. economy billions of dollars annually (Parlavantzas 2015). Technologies like simulations, games, virtual reality, and other attractors are all attempts to re-engage employees to improve productivity and profits. Rollag and Billsberry (2012) contend that improved information technology over the past decade has led to a shift from passive to active learning, where, independent of any developmental or organizational input from an L&D department, employees simply look up the information they need, when they need it. Learning has definitely become more learner-centric, which means that, more than ever, instructional technology designers need to take the characteristics and interests of learners into account when they develop technology-enhanced learning.

With this self-directed approach to learning taking hold, learners have altered the context and inserted their motivations and desires into the conversation. Providing ways for the employees to increase their connection and engagement with the business through easy-to-use learning technologies may unlock performance improvements previously unrealized or neglected.

The Social Context of Workplace Learning

Collective or group learning is not the same as individual learning. In North America, individual learning has traditionally been favored over group learning in workplace learning research and practice. This is a mistake because learning is first and foremost a social enterprise. As noted Russian psychologist Lev Vygotsky (1978) stated, as individuals we always relate to a “shared social world.”

Collective workplace learning, also known as organizational learning, often occurs within and among teams, and “involves the exchange of facts and concepts, experimenting with ideas, joint reflection on them, and the collective restructuring and fine-tuning of them” (Andres 2013). Organizational learning is the process of constructing and storing new knowledge within organizations—or reconstructing existing knowledge to improve the functions of individuals within the organization or the organization as a whole—and is embedded in an organization’s culture.

Collaborative learning and knowledge building have many different facets, including mutual teaching, exposure to multiple perspectives, division of tasks, pooling results of actions, brainstorming, critiquing, negotiating, compromising, and agreeing (Stahl 2004). In fact, group learning may take precedence over individual learning in that it supplies the cultural background, the motivational support, and the interactive experiences to facilitate individual learning (Stahl 2006).

Workplace climate, especially positive supervision and encouragement from other members of one’s workgroup, can influence motivation. It can also change how workers view emerging learning technologies introduced into the workplace. A 2012 study by Cheng and colleagues found that “employees’ perceived managerial support and job support had a significant impact on their perceived usefulness of the e-learning system for individual learning, and that perceived organizational support had a significant influence on the perceived usefulness of the e-learning system for social learning.”

Varieties of Technology-Enhanced Learning

There are at least four broad ways to learn in a workplace:

• incidental or accidental learning (from everyday experiences that have not specifically been designed for learning)

• self-directed learning

• directed learning or instruction

• collective or social learning.

Let’s look at each in terms of how they can be influenced by technology.

Incidental Learning

We are always learning simply by being alive and experiencing the world as we encounter it. Sometimes those experiences are painful and unexpected, but learning still takes place—a process referred to in the literature as “incidental learning.” For example, railway engineers can learn from derailments or train collisions even though they are not planned. Of course, the experience is very different if you are involved in such an incident compared with investigating it later or reading about it.

Lukic, Littlejohn, and Margaryan (2012) developed a framework for learning from incidents in the workplace, especially in a health, safety, and environmental context, which can improve organizational safety and productivity. A critical factor in the success of learning from incidents is the level of engagement of employees and their ability to challenge the status quo within the organization. Here, reporting technologies, such as documenting safety concerns by posting digital photos of hazardous conditions online, can improve safety through group learning.

Self-Directed Learning

Methods used by employees for self-directed learning include the development of good search skills; being comfortable linking to a variety of information sources, including audio and video; networking with peers and mentors as an information source and for .collaborative learning; being members of learning communities (also known as communities of practice); self-tracking of body movements and performance measures; and simply following employee interests. Increasingly, digital devices become part of workplace learning processes along with other material elements, a concept referred to as digital materiality (Pink, Ardèvol, and Lanzeni 2016). Embedding digital games for learning along with a manual assembly process for manufacturing is one example. Games can be used for motivation, microlearning, performance support, and simulations for learning teamwork (Cela-Ranilla et al. 2014).

Direct Instruction

Direct instruction refers to the explicit teaching of information, concepts, or skills using presentations or demonstrations of the required learning materials. It includes lectures, tutorials, laboratory sessions, discussion groups, audiovisual presentations, conferences, seminars, workshops, apprenticeships, internships, assigned readings, supervised practice, organized games, simulations, field trips, triggered locative learning, and other forms of arranged learning experiences. Mostly, it uses language in the form of text or narration to convey information, and is usually based on a documented curriculum that is being followed by the person or people who are giving the direct instruction.

Systematic direct instruction may use specific behavioral objectives to facilitate assessment and measurement of results. This approach has been also termed instructionism, and while it can have an impact on learning it is now seen as inadequate for learning in a knowledge economy (Sawyer 2014).

Direct instruction is often driven by measurable behavioral objectives, but much learning cannot be measured quantitatively or even qualitatively, and tacit knowledge can remain hidden until a person is called upon to enact a specific skill or remember a specific piece of information in a particular context. Similarly, direct instruction is often not seen as a way of exciting or motivating learners about content that needs to be learned, which is why it is sometimes referred to pejoratively as just-in-case learning.

Although direct instruction has fallen somewhat out of favor with the movement away from classrooms toward mobile learning, there is substantial empirical evidence supporting its effectiveness, at least for material that needs to be committed to memory, such as emergency instructions or legally binding regulations. A recent meta-analysis of a half-century of research (328 studies) on the effectiveness of direct instruction was very supportive of its effectiveness (Stockard et al. 2018). As noted earlier, direct instruction may be the best way to teach representational language-based knowledge, by supporting and motivating learners to pay attention to material that they otherwise might not learn.

What if a company wants to teach something very specific to their employees? Lots of canned learning programs are offered and sometimes mandated with varying degrees of success. One of the strategies for increasing motivation to follow a specific program of learning is the use of games and gamification techniques to develop employee skill sets and build competence in different tasks. Games are generally immersive, engaging, and motivating—all ingredients for attracting and holding learner attention. Moreover, with most games learners are active and doing something. They usually have a storyline that moves the learner on to completion of a training session.

Games can be a form of direct instruction if they have been specifically designed for learning. According to Karl Kapp (2018): “Learning games provide context, engagement, challenge and the thrill of mastery.” Kapp (2013) also writes, “there is solid research and overwhelmingly compelling evidence that games can and do teach a variety of subjects effectively.” Also known as serious games, many have shown good results in terms of motivation and knowledge acquisition. For example, the Facebook game FarmVille has been used to teach accounting skills (Krom 2012).

One of the criticisms of direct instruction is that it mostly has been used to teach lower-order thinking by emphasizing the memorization and regurgitation of information. Much work has been done on the development of higher-order cognitive skills through the publication of taxonomies of behavioral objectives, such as Bloom’s (1956) taxonomy (more recently revised by Anderson and Krathwohl in 2001). Beyond the higher-order thinking skills has been a rich literature on meta-learning or learning how to learn. Finally, systems thinking allows the widest view of the learning enterprise and how it relates to larger issues (Meadows 2008).

Collective or Social Learning

Collective learning can happen at several levels. At the lowest level, we can all make contributions to a group effort. That group effort can then be coordinated so the group works together without conflict with one another. Beyond that, groups that work together can engage in collaboration as a form of learning. Finally, collective action also can result in social movements that get things done. The results of collective learning can be stored within the memories of individuals in an organization or archived through online storage methods. Collective learning can become part of the culture and social cohesiveness of any organization.

Collective learning is situated within the context and culture in which it occurs (Lave and Wenger 1991). It involves social interaction in the building of a feeling of community, as well as the beliefs and practices that become part of an organization. As newcomers join the organization they start at the edges of what is known and gradually learn how things are done as they become enculturated within the organization. This legitimate peripheral participation is a normal process of becoming part of a group.

E-mail and social media have been cited as examples of learning technologies that support collective learning, but they are mostly based on individual-to-individual communications, or supporting small local groups of friends. We mentioned the nasty side of social media in the last chapter, and large platforms such as Twitter and Facebook are now trying to correct and improve on these unpleasant and dangerous side effects of open, anarchistic communications software without limits.

There are dozens of examples of emerging technologies that seek to improve collective learning and internal communications within organizations, but this market is still in flux, without any clear winners dominating the sector. Commonly used communications, collaboration, and collective learning software include Dropbox, Google Docs, Slack, and Basecamp. But there are many more productivity tools available at a variety of price points, and more are added every month.

Technology-Enhanced Assessment

Even before the hiring process starts, emerging assessment technologies are being used for talent identification and evaluation of the suitability and potential abilities of prospective employees. Chamorro-Premuzic and colleagues (2016) state, “From smartphone profiling apps to workplace big data, the digital revolution has produced a wide range of new tools for making quick and cheap inferences about human potential and predicting future work performance.”

Early developers of learning assessment software produced simple tests and quizzes, mostly based on the types of questions typically found in school-based teacher-constructed tests. From the beginning of the use of computers in education and training, vendors have touted the ease with which computers can calculate results for a wide variety of question types. Drag-and-drop, multiple choice, multiple response, numeric, choices on pull-down lists, ranking, text matching, graphic matching, true/false, yes/no, fill in the blank, find a hotspot, and Likert scale items are all ways to measure recall of material. Essay answers and simple explanations can now be “marked” by computer, with about the same reliability as human markers. And, files can be uploaded to instructors for later manual perusal.

Memorization is one of the principal ways that we consider whether something has been learned, and testing for retention in memory is the major form of computer-based assessment. Practice is a technique to move content from short-term sensory memory to working memory, and then to long-term memory. Because of this emphasis on memory, some of the earliest technologies for learning were drill and practice programs, where a computer simply presented examples of content over and over, evaluating answers for correctness based on what was stored in the memory of the computer, and repeating exercises when wrong answers were detected.

More advanced assessment techniques that don’t just rely on memory are now available. These assessment techniques can be used to record observations, maintain online portfolios, record pathways and decisions in working through learning materials, trace digital bread crumbs (Schwartz 2010) to show where a learner has been online, track the processes learners follow to solve problems, and embed nonobvious assessments (also known as stealth assessments) in games and simulations. All these techniques and more are possible with current technology.

Procedures such as note-taking and repetitive games were also a favorite of early developers, perhaps because they were relatively easy to program and could be run with the limited capabilities of early computers. While memorization of content has its place in the repertoire of learning processes, learning by doing, inquiry learning, and experiential learning are alternative approaches to learning that have greatly been helped by the improved power of new digital technologies. Assessment technologies for these forms of learning are mostly the provision of methods of recording and recalling performance, such as the use of e-portfolios that can later be assessed and validated by subject matter experts or workplace supervisors.

Coming next are personalized adaptive assessments in which tasks are automatically matched to the real-time performance of individual learners. The focus of the assessment will shift to understanding the current learning situation; this understanding will then be used to provide personalized materials, monitor progress, give performance support, and evaluate ongoing effectiveness. These advances allow on-the-fly adaptation of what is presented to each learner, high-quality feedback for the learner and the instructor, and the gleaning of expert knowledge on common learner errors and misunderstandings. This knowledge can then be used for diagnostics, guidance to the learner, and adaptive tutoring on the areas of difficulty (Masters 2015). In effect, the technologies of assessment become interconnected, forming an ecosystem with different sources of data feeding into the profile of each learner, which is continuously updated whenever the learner is online (Behrens and DiCerbo 2014).

Critics of this approach to assessment describe it as mimetic, designed to encourage the regurgitation of standardized answers. While there is a case to be made for learning specific materials, such as in regulatory compliance training, this use of computer technology is not seen as transformative, whereby the learner becomes more reflective, creative, and thoughtful through the use of adaptive technologies (Frank 2017). But, as we noted at the beginning of this chapter, the choice between memorization and experiential approaches to learning should not be an either/or proposition. Both approaches to training will be needed and used in the future.

Technology for the Management of Workplace Learning

Traditionally, instructors and managers of learning and development departments managed the administration of workplace learning sessions with paper-based methods: records of attendance and participation, assessment results, and content nicely organized in books or binders. Computer-based technology substantially changed all that, starting in 1978, with the development of spreadsheets such as VisiCalc (the first “killer app” for microcomputers) for maintaining records and calculating assessment results. Spreadsheets as workplace learning management tools were followed by virtual learning environments or learning management systems, which were developed in the 1980s and improved throughout the 1990s (for a detailed history of these systems see the Wikipedia article, “History of virtual learning environments”). These systems launched courses, presented learning content, tracked student attendance and participation, administered basic quizzes and tests, and recorded and reported on assessment results. They were often high-priced, cumbersome, and difficult to use, but were seen as necessary for the administration of learning and development.

Learning management systems were not designed to support informal, just-in-time learning. Nor were they designed to gather data on all types of learning activities in real time as those activities happen. Only by tracking the clickstream of online activities and interactions can learning administrators get a good handle on the actual daily learning activities of students or employees. Antonelli (2017) advocates tracking “elements like topics searched and shared socially, time viewing learning videos, engagement with corporate content, and tie these with people analytics from HR systems.” Tracking, of course, must be carried out with the informed consent of those being tracked.

In the new organizational and learning analytics approaches, large amounts of data are collected from entire populations, rather than small samples, enabling researchers to make correlations between inputs and outputs in the learning process. Most learning analytics data are gathered automatically, as learners interact with digital tools and information. Learning management for emerging technologies must break out of the browser-based prison and reach out to a wide variety of disparate data sources. The newly minted standard for such collection and tracking, xAPI, does just this, but it is up to learning experience designers and the learning management team to ensure that learning activities are used to provide data for engagement tracking and assessment. This leads us to the next key component in understanding this ecosystem—where to put all this new data.

Data Collection

With the information explosion, the knowledge available in the world today is “too big to know” and the amount of available information is increasing exponentially (Weinberger 2014). In addition, the raw data generated by information technology is overwhelming in terms of sheer quantity. Even experts are unable to use or process all the data available in their own fields.

The collection of data and its use in analytics is an emerging trend in workplace learning. While most learning platforms purport to “do analytics,” the term means many different things depending on the kinds of data collected and the applications using the data. Emerging learning analytics platforms can collect data that can be used in six different kinds of analysis (descriptive, inferential, pattern recognition, visual, predictive, or prescriptive) applied at two different levels: individual and organizational.

Analytics can apply to individuals and are generally referred to as learning analytics. Analytics that provide information at organizational levels are called institutional or organizational analytics. Compared with most LMSs, emerging analytics platforms (such as those based on the xAPI protocol) can collect many new data points that can be used for both learning analytics and organizational analytics, thus opening the potential for new big data approaches to organizational analytics, learning analytics, and data-driven decision making. Adaptive learning, personalized recommendations, and AI-driven personal assistants all feed off large data sets and are enhanced immensely by having access to good data. However, to take advantage of these types of emerging technologies, the pumps must be primed, so to speak. We’ll cover more on that in the next chapter on dependencies.

As these new platforms record user activities, they also store relevant contextual data, making search results more meaningful to users, as well as enhancing insights for researchers, administrators, and other interested stakeholders. These data points provide a gold mine for analysis by CIOs and other C-suite executives, who can use the data to better understand how to improve learning materials without relying on subjective anecdotal feedback, as well as be used in many other analytical applications. Adding contextual metadata and analytics to the use of content enhances existing resources, encourages collaboration, and provides valuable insight into how learning materials can best be deployed.

Knowledge flow is greatly improved by networking, because newer search engines can find resources from anywhere, including applications like Google Drive, Pinterest, OneNote, OneDrive, Dropbox, training apps, connected repositories, professional learning environments, and learning management systems. Once identified, a persistent link can be maintained to the content that is found without moving or making a copy of it. Newer programs can store the data points that come from the search results, such as names of folders, files, lessons, or metadata tags. This makes every search result easier to use, easier to keep and organize, and easier to share, as well as trackable for future analysis. Data can be visualized immediately using a built-in dashboard.

For data-driven learning systems, all this data and metadata can be available for individual users as well as larger entities (for example, a linked program, a learning object repository, or a database), generating even more data points. If metadata is anonymized and not traceable to a specific user once aggregated, privacy of individual users can be maintained.

Analytics

Generating such a rich set of data points allows for the subsequent use of the mass of data collected for both organizational analytics (providing organizations information to support operational and financial decision making) and learning analytics (providing organizations, teachers, and students with information to support achieving specific learning goals). These data points could also be used as data sets for use by authorized external analytics software or authorized researchers and analysts.

On the organizational analytics side, the data collected can be used by companies to understand learning materials, analyze trends, generate summary statistics, correlate variables, visualize patterns, provide logical explanations of patterns, evaluate system performance, and predict future behaviors of groups. Once sufficient data is collected, various models can be produced that CIOs can use to formulate or evaluate training policies and instructional approaches. Forms of modeling can include user knowledge modeling, user behavior modeling, user experience modeling, and domain modeling, including human ecosystem modeling. Using APIs, emerging analytical platforms can link to many external modeling and analytics systems.

On the learning analytics side, a learning analytics program can be used to identify and track a variety of user interactions, sessions, and the completion of tasks within sessions, as well as record any other learner activity. Data can be used as an early alert system by sensing if an employee is having difficulties, and individual data can be used to personalize advising, recommendations, adaptive learning, and cognitive tutoring. Other uses of data include the application of various machine learning techniques and algorithms for capturing, processing, indexing, storing, analyzing, and visualizing data. Once enough data is collected on the “learning trails” that learners follow (Walker 2006), it can be classified, analyzed in clusters, and used for “attribution modeling” (Deshpande 2017), a form of association rule mining that connects various inputs to outcomes (Borne 2014).

Sometimes, associations can be surprising. For example, when a sales analyst at Walmart looked at associations between products, he found that many customers who bought diapers also bought beer. He realized that it was stressful to raise kids, and parents impulsively decided to purchase beer to relieve their stress. So, he bundled diapers and beer together, and sales of both skyrocketed (Choi 2016).

A learning analytics program can be configured with a set of standard reports or it can be used with other data analytics software and reporting programs through external APIs. It can connect to and cross-reference data with external human resources management systems, align data with standards and objectives, and work with other systems that collect data, such as most learning management systems or learning record stores. Newer learning analytics programs can collect their own data, as well as aggregate and integrate data from other learning platforms and databases or content repositories.

As data scientist and author Thomas Dinsmore (2016) notes, “At a high level, the analytics value chain includes three major components: steps that acquire data, steps that manage data, and steps that deliver insight. Delivering insight to human or machine users is the critical link in the chain; a system that successfully acquires and manages data but does not deliver insight has failed.” By delivering such a huge variety and amount of data points, and integrating both internal and external data resources, a learning analytics program enables educational insight in many different ways.

Evaluating Emerging Technologies for Their Support of Learning

By now it should be apparent we don’t believe that learning is mostly about the memorization of information or procedures. There are many different ways to learn and many types of learning. We introduced the work of Nobel laureate Daniel Kahneman at the beginning of this chapter, noting that we each have two kinds of selves: the experiencing self that is fast, intuitive, and based on the use of heuristics, and the remembering self that is relatively slow, rational, and analytic. Both need to be considered when we look at how people learn. We believe that new technologies, such as augmented and virtual reality, are effective because they cater to the experiencing self, while at the same time we believe that well-designed e-learning programs may work best for the remembering self.

Of course, from an instructional design point of view, there are many ways to design learning activities using new technologies. But, some technologies are effective for one kind of learning but not another. This is something you’ll need to evaluate.

We have emphasized that workplace learning is substantially different from learning that takes place in educational settings such as K–12 classrooms or colleges. Emerging technologies need to be evaluated for their ability to support workplace learning, and not based on the assumptions of classroom-based learning.

We believe that much learning takes place in groups through cooperation, collaboration, and communication among members of teams and organizations. There is usually a social aspect to all individual learning, in that we learn little without the support and example of other people. Certain technologies are more supportive than others of collective learning, and any new technology should be evaluated for the support it gives to group processes.

Finally, the latest learning technologies offer a wide range of assessment techniques that need to be evaluated. The latest assessment techniques involve data collection and several types of analytics. Any emerging technology should be evaluated for its ability to provide data for analytics, especially if it uses artificial intelligence and learning algorithms.

Using the BUILDS framework, these are the kinds of questions you should ask about the support any new technology offers to learning in organizations:

1.   Does this technology support both intuitive (experiential) and rational (instructional) ways of learning?

2.   Is this technology flexible enough to support many different kinds of learning activities?

3.   Has this technology been specifically designed for workplace learning?

4.   Does this technology support individual and group learning? Does it embrace collaborative and collective learning?

5.   Does this technology allow for a wide range of assessment tools, including support for organizational analytics and learning analytics?

Looking Ahead

The use of technologies for workplace learning has a long history and will continue to evolve as new technologies emerge. These need to be evaluated in terms of their usefulness for training employees on a case-by-case basis, depending on the types of learning that each technology supports. The very definition of learning is shifting and changing with the advent and use of these new technologies. It’s likely that this change and the pace will widen the gap between true academic venues, such as K–12 and higher education, and workplace learning environments.

With more advanced tracking and assessment opportunities available in the emerging technology space, we should be able to more accurately record and report on our learners’ activities. We hope this improvement and the iterative changes in curriculum and content it will enable will lead to more engaged and therefore better performing employees.

At the same time, the increase in data gathering and analytics adds layers of complexity to the new technologies that use these new capabilities. They also increase the number of dependencies that programmers and administrators need to deal with, increasing both the fragility of the software and the risk of failure. How to deal with dependencies is the subject of our next chapter.