Qualitative research on information and communication technology (ICT) covers a wide terrain, from studies examining the skills needed for reading, consuming, and producing information online to the communication practices taking place within social media and virtual environments. Rapid proliferation of ICT has fostered experimental studies examining a number of quantifiable outcomes associated with its uses, but qualitative research has built on our understanding of how, when, why, and for whom experiences with ICT shape participation, relationships, and learning. Researchers still have much to explore in order to understand what Shaw and Hargittai (2018) have described as the “pipeline of online participation” (p. 143). Persistent questions emerge about how the overlapping layers of Internet access, socioeconomic level, geographic location, educational background, privilege, position, and power might converge to create more opportunities for equitable participation in the online consumption, production, and sharing of knowledge. The methods provided by qualitative research provide the necessary analytical tools and theoretical frameworks to explore these emerging issues. This entry begins with an overview of three current areas of qualitative research on ICT and is then followed by a discussion of the methodological challenges of ICT research. Finally, new demands on qualitative researchers are explored.

Current Areas of Inquiry in ICT Research

Current work on ICT tackles research questions stemming from three main focal areas: (a) the skills needed for reading, consuming, and producing information for the Internet; (b) the factors impacting system‐wide implementation of ICT initiatives in schools; and (c) the affordances of network‐based interactions that take place across geographically distal contexts (for syntheses, see Suggested Readings).

Considerations of Qualitative Methods in ICT Settings

Conventional qualitative data collection methods are commonly used in qualitative research on ICT, such as conducting interviews and observations, collecting field notes, amassing relevant documents, and disseminating surveys (Miles & Huberman, 1994). These methods, however, are based on face‐to‐face, physical settings and require new considerations in many ICT settings. Researchers committed to crossing, as Leander (2008) suggests, the “online/offline, virtual world/real world, and cyberspace/physical space binaries” (p. 37) must retool and repurpose these methods of data collection and analysis in order to capture interactions and information flows across digital as well as physical spaces.

Researchers must consider the interaction between the products of digital communication (interaction transcripts, Weblog archives, videos, Web sites, etc.) and the processes of the production and consumption cycle (access, orientation, affiliation, collaboration, interactivity, etc.). In practice, this translates into the creative addition of new data collection and analytic techniques. Archived data can include typical downloaded materials such as transcripts of synchronous chats and asynchronous posts, but they also extend to an array of new archives that layer in visual and aural modes: network analyses, social media communications, and screen capture data.

Researchers of social networking and gaming activities often gather transcripts and screenshots of the social interaction and activity that take place in various forums (online gaming, fan fiction, multilingual networking, instant messaging, listservs, chatrooms, and blogs) and triangulate this interactional information with other online activity, including participants' personal Web sites and updates, player manuals, and online guidebooks (see Black, 2008; Leander, 2008; Lam, 2009; Steinkuehler & King, 2009).

Examining the processes of ICT activity also presents new challenges in qualitative research because the local is no longer dictated by physical spaces. When participants are distally located—whether in naturally occurring interactions or in projects that involve international communication—researchers seek other ways to understand the perspectives and contexts of participants across wide geographic areas. To this end, interviews can be conducted utilizing e‐mail, listservs, instant messenging, and video conferencing.

A final consideration of qualitative methods in ICT is the continued enrichment of research in this field that is offered by its interdisciplinary flexibility and creative mixing of new ideas. Communication in ICT is increasingly multimodal and focused not just on transmission of ideas but also on meaning making across and within multiple sign systems. Stornaiuolo, Smith, and Phillips (2017) offer a transliteracies framework that foregrounds analytical tools that describe, and explicitly investigate, not just the material and social aspects of communication, but also the power relationships, normative assumptions, and multiple perspectives that surround interaction. This complex interplay of relationships is echoed in the concept of playscapes put forward by Abrams, Rowsell, and Merchant (2017), who argue that researchers must understand that these “entangled practices involve the interweaving of human, material, semiotic, and discursive practices” (p. 1).

New Demands in Qualitative Research in ICT

New demands are placed on researchers in the dynamic and expanding contexts in which ICT takes place within and across the multiple contexts of home, school, and Internet activity. Issues related to ethics, privacy, and expertise intertwine in ways that continue to require novel approaches.

First, conducting ethical research within ICT poses a number of new challenges. Busher and James (2015) explore these “ethical conundrums” (p. 178) that arise as the once‐separate domains of online communication and face‐to‐face contact have, in many contexts, become interlocking hybrid communities. To keep subjects safe and research findings trustworthy, Busher and James urge researchers to take additional steps—such as iterative reflection on ethical issues—at every stage of the study, not just at the inception. They also argue that the sites of research should not be framed as physical locations, but rather as multiple physical, multimodal, and social interactional spaces that people occupy.

Deepening qualitative inquiry will demand expertise beyond traditional methods and require time investment as well as a larger shift toward collaborative research teams that can make use of distributed expertise. Such collaborations have already been conducted in online intercultural communication projects, in which teachers and researchers design and implement studies across distally located sites (O'Dowd, 2015). Collaborative research has also taken place in single‐site ethnographic work, in which large teams of researchers conduct home and school visits on larger numbers of focal participants by using the same theoretical lens and framing questions (Leander, 2008). Research teams can address some of the practical issues that can become problematic, such as ensuring representation and coverage of an area of inquiry when the information needed is not conveniently located in one physical space.

Issues related to privacy also pose new challenges. In online‐only contexts in naturally occurring interactions, recruiting participants and obtaining informed consent can be challenging. Verification of participant identity is a challenge, which in Black's (2008) study on fan fiction was addressed by verifying participants' self‐described identity through triangulating information gleaned from self‐reports, consistencies in their presence in other areas of the Internet, and signed written consent. Privacy must also be established to protect individuals who might appear in the communication, but who are not themselves research participants (Busher & James, 2015). One plausible solution to this dilemma was provided by Lam (2009), who asked her focal participants to inform their online contacts about the research prior to recording any instant messaging sessions.

Leander (2008) documents further challenges associated with ethical online research, including maintaining the privacy of what he calls “incidental data” (p. 59), that is, interactions and activities documented by the researcher but which the participants might wish to exclude from viewing by others. Leander therefore employed a series of precautionary measures, including obtaining consent through Web‐based forms, allowing participants to view and delete sections of the transcripts of files before releasing them to the researchers, and developing longer‐term, collaborative relationships with the participants.

Conclusion

Qualitative research is ideal for exploring new realms in which there is little previous research. With technology‐mediated interaction developing so rapidly, qualitative research has been well suited to exploring ICT practices that occur in such realms, and will likely continue to be in the future.

SEE ALSO: Intercultural Interaction; Mobile‐Assisted Language Learning; Multilingualism and the Internet; New Literacies of Online Research and Comprehension; Teaching Culture and Intercultural Competence

Qualitative sociolinguistics research (QSR) is an approach that focuses on the ways in which people in communities use language to accomplish things in the world, and how they represent themselves to the world. QSR, in contrast to quantitative research, seeks to examine and explain the richness of language use in particular contexts with an overall aim of understanding the relationships between language and the social world. As such, qualitative sociolinguistics research employs a range of data‐collection and analytic tools and approaches that can lead to a “socially sophisticated approach to language variation that systematically takes into account the behaviors and motivations of speakers” (Chambers, 2003, p. 8).

Given the increasing complexity and interconnectedness of today's world, QSR places central importance on understanding how differences in language both reflect and help to create differences in, on the one hand, how individuals and groups use language, and, on the other, how they are evaluated or judged as a result of their language use by others. While QSR describes a particular research orientation, researchers in this area employ several related research methods, in particular ethnographic methods and discourse analytic approaches.

Formulating Qualitative Sociolinguistic Research

Qualitative research, by its name, implies a concern for the details and particularities of a research problem; in the case of sociolinguistic research, this entails a focus on the details and particularities of language use in context (Ramanathan & Atkinson, 1999). Key to any qualitative sociolinguistics research is a definition of language use and a definition of context. The concept of context can be understood in terms of a set of concentric circles of influence on a given language situation in which language is used for particular communicative functions. Beginning with the most local context, sociolinguists are interested in understanding how the moment‐to‐moment actions of one speaker in a particular time and place affect the form of what is said, the meaning that is intended, and the manner in which it is interpreted by others in the interaction. Social interactions are also framed by relationships among group members, which approaches such as social network analysis (Milroy, 1987) and communities of practice (Eckert & McConnell‐Ginet, 1992) take as their starting point. Discerning and explaining differences in how language use both creates and reflects social groupings lies at the heart of sociolinguistics research. Through careful in‐depth analyses of a wide range of situations and associated language uses, a picture of how language works to structure society, and human relations within it, can be drawn.

QSR, like all other forms of research, starts with a particular question or issue. Consonant with the field of sociolinguistics, the research question or issue typically focuses on a social concern: for example, understanding how individuals argue and misunderstandings occur, understanding how children and adults acquire the tools to communicate in their own or in new communities, or understanding how particular contexts serve to support or hinder individuals with different language practices when it comes to achieving their communicative and social goals. Topics or areas of concern in the last 20 years have included differences in interaction both within groups and between members of groups based on social class, race, ethnicity, and gender; bilingualism, code switching, and translanguaging; language planning and language policies; and educational and workplace practices. In each of these cases, research also often attempts to understand and explain inequality in terms of how individuals' language is evaluated.

Traditions in Qualitative Sociolinguistics Research

The use of qualitative methods in sociolinguistics generally belongs to one of three related approaches. The first derives from Dell Hymes's (1974) formulation of the ethnography of communication (also called the “ethnography of speaking”). The second is John Gumperz's (1982a, 1982b) work in the area of interactional sociolinguistics (see also Gumperz & Hymes, 1972). The third approach examines language socialization processes and focuses on the development of language use appropriate to a given community (Schieffelin & Ochs, 1986).

Hymes, coming from the tradition of linguistic anthropology, was interested in understanding how speakers accomplish social goals in given social situations in particular speech communities. His contributions include the concept of communicative competence and the characterization of language use in nested contexts, namely, speech situation, speech event, and speech act. In addition, Hymes introduced the SPEAKING model as a mnemonic (Setting and Scene, Participants, Ends or goals, Acts, Key, Instrumentalities, Norms, and Genre) for examining all aspects of speech events that affect the form language takes and the associated language and social functions it fulfills. This model, moreover, provided qualitative researchers with a methodological tool or a frame for focusing on data to collect, and allowed for systematic and easily comparable analyses of situated language practices with a particular focus on the macrolevel of social situations and settings in which language is used (Copland & Creese, 2015).

Seminal works in the ethnography of communication tradition include Philips's (1983) study of how language differences between Warm Springs Indian and Anglo children in schools—for example, the organization of interactional sequences and the use of silence—often made Anglo teachers perceive Native American children's behavior as reticent or noncommunicative rather than polite and deferential, as the students intended. Likewise, Heath's (1983) study of the literacy practices of three social groups revealed how values associated with particular language practices are socially constructed according to cultural norms and expectations. Eckert (2000) blended qualitative and quantitative methods to study the social structure of a typical high school. Her description and analysis of student cliques, “jocks,” and “burn‐outs” allowed her to demonstrate how language serves to structure social groups. What these studies have in common is their connection to the roots of ethnography: long‐term participant observation in order to come to an understanding of language use from an insider (emic) perspective.

The second research tradition—interactional sociolinguistics established by John Gumperz (1982a, 1982b)—focuses on the face‐to‐face immediate context of interaction with the goal of understanding how each turn at talk influences subsequent turns at talk in ways that link any particular conversation to broader social categories. Such microlevel analyses of language use allow researchers to trace the linguistic and associated social origins of tensions that may exist across speakers that reflect broader patterns of inequality and that can lead to miscommunication. For example, Gumperz (1982a) analyzes how Indian and British participants in three short conversations had radically different perceptions of the attitudes and intentions of the Indian participants, based on the intonation patterns they used in simple questions: While the Indians intended to convey respect and deference, the British perceived their interlocutors as “surly and uncooperative” because the communicative value of rising as opposed to falling intonation patterns on simple questions are reversed for speakers of these two different English‐speaking speech communities. While this perspective focuses on a careful analysis of the linguistic form, it further requires the researcher to link the analysis of language features such as intonation to socially defined relationships between speakers.

Research from a language socialization perspective (Ochs & Schieffelin, 1984; Schieffelin & Ochs, 1986; Ochs, 1988) also examines language in use contextually and qualitatively, drawing on work from linguistic anthropology. Here the primary focus of research is on how individuals—in particular children in their native communities—come to use language appropriately (i.e., in line with community norms). This strand of research is cross‐cultural and also rests on the tradition of ethnographic observation and careful analysis of interactions, primarily caregiver and young child interactions. Such work, recently extended to include a focus on life‐long socialization into new communities, focuses on how individuals come to be “speakers of culture” (Ochs, 2002).

Observing Language Use in Context

To engage in qualitative sociolinguistics research means to spend time carefully observing and analyzing language in context in order to understand the language practices of individuals, the motivations or intentions underlying those practices, and how these language practices relate to society's norms or expectations about language. Johnstone (2000) traces this tradition of documenting language use to the field methods used by dialectologists in the 19th century who were interested in describing the linguistic characteristics of speakers of regional varieties. In contrast to quantitative sociolinguistics research, qualitative sociolinguistics research involves in‐depth and often long‐term observation of individuals and groups interacting naturally in society. Whereas a researcher using quantitative methods may use statistical correlations to show, for example, the complex connections between gender and social groups, a qualitative researcher may approach a similar question by analyzing both routine practices and “telling” examples to demonstrate how language constructs identity. For example, Ek (2009) analyzes how a young Guatemalan American woman negotiates her identity in contexts (home, church, and school) that represent and structure conflicting expectations with respect to ethnicity and gender. While quantitative researchers' data and analyses address questions of how often or how much a particular language practice occurs and is associated with social groupings, qualitative analyses are better suited to questions of how and why people engage in these practices (Johnstone, 2000).

Issues and Tools in Data Collection

There are several important issues related to collecting data on people's language practices. The first applies to both quantitative and qualitative methods, and was recognized early on by sociolinguist William Labov as the observer's paradox. The dilemma arises from the fact that language use is highly situated and dependent both on speakers' perceptions of what their interlocutors know and need to know, and on how they wish to present themselves to their interlocutors. Therefore, when an observer enters the social scene, their presence changes the dynamic of how speakers use language, and data (language practices) are altered simply because the observer is there to record them. For example, if participants perceive the researcher as an outsider, they will likely adjust their speech, by simplifying their grammar and lexis (foreigner talk), or by altering the form or content of what they say in subtle ways to accommodate the researcher's presence. This might affect the production of exactly the kinds of in‐group linguistic markers or expressions of shared understandings that sociolinguists are interested in documenting.

There are several ways of counteracting the effect of the observer's paradox. Consonant with ethical issues related to working with human subjects, the preferred method is to be forthright about the purpose of the study, and to spend enough time interacting with people in the research setting so that the researcher's presence becomes familiar and participants' language use reverts mostly to what it would have been before the researcher arrived. This method, however, generally presupposes long‐term engagement with the community, and does not resolve the observer's paradox as much as require researchers to locate themselves in the study and recognize that they have become part of the social scene they are investigating. Likewise, if researchers study a group of which they are already members or “insiders,” they will have better direct access to the group's unfiltered language practices, though their insider status may cause other issues for their ability to analyze and interpret the data.

The tools for collecting qualitative sociolinguistic data are relatively straightforward: researchers generally rely on observation or participant observation supported by some combination of questionnaires, interviews, field notes, and recordings. The questionnaire or survey is a common instrument in both quantitative and qualitative studies since it can efficiently gather a large number of responses in a short period of time. The responses are easy to tabulate and present a straightforward set of results. Qualitative surveys often include Likert‐scale (e.g., a five‐point scale indicating agreement with a particular statement) and some open‐ended items (Dörnyei, 2003). The limitation of questionnaires for language research is that they measure linguistic perceptions or beliefs rather than actual language practices, and include unrecorded adjustments to those perceptions based on the concept of the observer's paradox. It is also deceptively difficult to design good questionnaire items that will be interpreted in the same way by all respondents.

The sociolinguistic interview (Becker, 2013), like the survey, gathers elicited data by directly posing questions to participants. While interviewing is not as efficient as questionnaires, good interviewing techniques can yield rich insights about participants, in particular regarding their own attitudes and perspectives on language use in the particular context under study. Field notes and recordings both serve to document naturalistic data, that is, in situ language practices that are not directly elicited by the researcher. Taking field notes is a common practice in ethnography and is unobtrusive, although researchers are restricted by how much they can write down and are not able to go back and recheck the accuracy of their notes. Audio and video recordings have the advantage of allowing researchers to transcribe interactions and gain a fuller picture of language use. However, recording devices can be obtrusive—although less so today in many contexts, with digital devices so prevalent and much smaller in size. Nonetheless, as with all other forms of data collection, recording equipment always both focuses and therefore limits the data that can be gathered for analysis.

Recent Trends in Qualitative Sociolinguistic Research

Since the late 1990s, a number of new approaches to qualitative sociolinguistic research have emerged, each stemming from the traditions outlined above, but drawing new insights from theorizing and research in related fields. Most notable is the influence of sociocultural theory and, in particular, the community of practice model from cognitive science (Lave & Wenger, 1991), which has been taken up to examine language practices (Bucholtz, 1999; Eckert, 2000). With the introduction of this perspective a number of concepts, originally part of the work of either Hymes or Gumperz, have received greater attention. In particular, with the introduction of the community of practice as an alternative to the concept of the speech community, the concepts of identity, ideology, and power have led to a shift in focus from understanding the normative behavior of individuals in groups to examining language practices at the margins, that is, language practices that individuals engage in to resist, subvert, or change community‐wide practices. These studies examine the ambivalence, tensions, and contradictions that define intragroup as well as intergroup differences (Rampton, 1999). Furthermore, there is an increased interest in the sociolinguistics of multilingualism (Bell, 2014). The multilingual focus stems from the recognition that many of original questions and the face‐to‐face encounters that traditional sociolinguistics analyzed are increasingly happening in complex multilingual, multimodal, and even digital spaces (Horner & Weber, 2018).

Recent work has moved away from a normative or essentialist view of language practices and aims to examine and explain the multiple and fluid ways in which language is used by particular groups of people in particular contexts. One such recent focus centers around the concepts of superdiversity (Arnaut, Blommaert, Rampton, & Spotti, 2015) and translanguaging (Otheguy, García, & Reid, 2015). These two concepts expand the concept of language to focus on language practices of multilinguals, and the concept of context as one characterized by multilingual language use. Nevertheless, these new approaches continue to employ the core data‐collection techniques and insights from earlier research, while at the same time incorporating a different range of theoretical frames for analysis, frames that draw on a critical perspective of how language and society mutually structure one another.

In addition, there is a caveat worth noting with respect to the “quantitative versus qualitative” distinction: since the mid‐1990s, following criticisms by variationist sociolinguists as well as anthropological linguists on work grounded in a single tradition, combined or mixed‐methods approaches have emerged, which aim to provide richer explanations of observed differences in language use among members of communities under study. Those richer explanations, coming from a more in‐depth examination of social contexts in which language use occurs, have been referred to as the third wave in sociolinguistic research (see Eckert, 2008).

SEE ALSO: Anthropological Linguistics

Introduction

There has been a plethora of ongoing debates in the field of applied linguistics on many aspects of quantitative research method (QRM) in the last few decades. Despite the validity and significance of the debates and their contribution to sharpening our understanding of the concepts, QRM remains one of the most widely practiced approaches in social sciences in general, and in applied linguistics in particular (Lazaraton, 2005; Mackey & Gass, 2016). Although such debates are important and necessary, this entry will focus on explaining the major features of QRM in as nontechnical terms as possible.

Most scholars would agree with the classic definition of the term “research” as a systematic approach to answering questions (Hatch & Farhady, 1982, p. 1). There are three key terms in this definition: a question, a systematic approach, and an answer. The controversies in the field center not on the definition of the term “research” but on different interpretations of the key terms in the definition. For instance, the nature of the question and the way it should be formulated, the approach to be taken to answer the question, and the quality of the answer to the question are all issues upon which no wide agreement exists among scholars. The differences in the interpretations of these concepts stem from different theoretical orientations that scholars adhere to because advocates of each theory believe in a number of assumptions that justify their interpretations (Kinn & Curzio, 2005). Therefore, to put the issue in an appropriate context, first an explanation of the theoretical assumptions of QRM is given and then a description of how the three key terms in the definition function in practice is discussed.

Theoretical Assumptions

Generally, an inquiry is performed within a particular school of thought commonly referred to as a paradigm. This paradigm is believed to have four cornerstones: ontology, epistemology, methodology, and axiology (Onwuegbuzie & Leech, 2003; Creswell, 2007; Lochmiller & Lester, 2017). There are several popular paradigms such as positivism, constructivism, and relativism, each with some axiomatic principles that scholars follow, or should follow, in practice. Since QRM is rooted in the positivistic paradigm, a brief definition of the key concepts and their treatment in this paradigm will help clarify the characteristics of QRM within this framework.

Ontology raises questions about the nature of reality and the nature of human beings in the world. According to positivism, there is a real world, the reality of which is expressed in terms of the relationships among variables, and the extent of these relationships can be measured in a reliable and valid manner using a priori operational definitions. Positivists believe in the verifiability principle, which states that something is meaningful if and only if it can be verified by direct observation. Thus, ontologically, positivism is a school of thought that focuses on observable and measurable phenomena and their interrelationships.

Epistemology asks how we know the world and what the relationship between the inquirer and the known is. Epistemologically, positivism places a premium on objective observation of the “real world” out there. Positivists contend that the researcher (i.e., knower) and the object of the study (i.e., known) are independent. As such, researchers should remain objective and impartial in studying phenomena (Cohen, Manion, & Morrison, 2011).

Methodology focuses on the best means of acquiring knowledge about the world. Methodologically, positivists use deductive reasoning which is a system for organizing known facts in order to reach a conclusion. Conclusions are true only if the premises upon which they are based are true. That is, positivists emphasize the importance of a priori hypotheses and theories. The researcher manipulates at least one variable and then measures its effect on another variable, controlling potentially extraneous variables. Through such a procedure, valid cause–effect relationships are believed to be established and generalized as laws.

Axiology deals with the ethics and asks how moral a person, a researcher, should be in the world. Every ontology or epistemology has its own ethical stance toward the world and the self of the researcher. Positivists believe that inquiries should be value free. In other words, the researcher's values, interpretations, feelings, and musings have no place in the positivist's view of scientific inquiry. Positivists prefer rhetorical neutrality, involving an exclusively formal writing style using impersonal voice and specific terminology.

Putting the above‐mentioned principles together, a working definition for QRM can be formulated as an inquiry where an impartial researcher investigates cause–effect relationships between the variables in a world of real objects. This derived definition conceptually matches textbook definitions where scholars contend that in QRM numerical data are collected on the variables and analyzed using mathematical methods (Aliaga & Gunderson, 2006). Following this definition, the three key concepts of “question,” systematic approach, and “answer” in QRM will be detailed below.

Formulating the Research Question

QRM starts with a question which is formulated about the relationship between at least two variables. A variable is any attribute that changes from person to person (height), place to place (size), or time to time (temperature). Variables can be concrete, that is, directly observable and measurable such as height and weight; abstract, that is, not directly observable or measureable but inferred from observations and measurements such as intelligence and language ability; discrete or categorical, that is, either/or type such as left or right handedness, male or female; and continuous, that is, can take any value such as weight and language ability. Research questions can be formulated about the relationship between the variables to indicate either a cause–effect relationship or just togetherness between them:

Example 1: What is the effect of using multimedia tasks on the learners' listening comprehension (LC) ability?

Example 2: What is the relationship between the students' LC and reading comprehension abilities?

The first question is of a cause–effect type where the researcher sets out to investigate whether using multimedia tasks will have any effect on the learners' LC ability. The second question, however, is of correlational type where the researcher intends to find an existing relationship between the two variables. No instruction or manipulation is planned to find a cause–effect relationship since the researcher appears on the scene after the fact and has no role in changing any one of the variables.

An important point in formulating a research question is defining the variables under investigation as clearly as possible in both theoretical and operational forms. To provide a reasonable theoretical definition, an in‐depth review of literature needs to be performed. Operational definition, on the other hand, is to suggest a clearly measurable procedure for the variable in question. For example, LC ability may be defined theoretically as the ability to understand oral messages in real‐life situations, and operationally as one's performance on a test which is designed to measure one's ability to understand oral messages.

Theoretical and operational definitions of a variable given in a particular research context may not be acceptable to all readers because they are specific to that research design. For instance, there may be different definitions of LC ability in theory and different measures of LC in practice. However, a working definition should be provided by the researcher that is most reasonable in the context of research. Despite the potential disagreement among scholars, it is an advantage of QRM that the researcher and readers share common definitions of the variables.

In addition to defining the variables, the researcher should determine the functions that variables are designed to serve in the process of research. Most common functions that a variable can serve in QRM are: independent, dependent, moderator, control, intervening, and extraneous. An independent variable is the main or the cause variable which is under the control of the researcher to be manipulated, while the dependent variable is the variable that depends on, or changes as the result of, the manipulation of the independent variable. For instance, in the example above, using multimedia tasks is the independent variable and performance on a measure of LC ability is the dependent variable because performance on this measure depends on the quality, quantity, and effectiveness of the multimedia tasks designed by the researcher.

A moderator variable is the variable that may influence the outcome of the dependent variable without being necessarily manipulated. For instance, within the context of this example, the researcher may hypothesize that males and females may benefit differentially from the multimedia instruction. In such a case, gender, as a moderator variable, may change the outcome of research. A control variable, on the other hand, is a variable whose effect is controlled or eliminated by the researcher so that the variation of this variable does not influence the outcome of research. Suppose the researcher believes that familiarity with computers may have an effect on the achievement and performance of the learners. Therefore, to control the effect of familiarity with computer, the researcher selects participants who are absolutely unfamiliar, or equally familiar for that matter, with computer technology. In this case, any effect due to computer familiarity is removed from the research design, that is, controlled.

An intervening variable is the variable that intervenes between the independent and the dependent variables. It is often left unobserved and unmeasured because it is very often an abstract variable. In our example, due to the effect of the independent variable, that is, multimedia instruction, the participants learn and improve their LC ability. Although this learning process can neither be observed nor measured, its effect is manifested in the measurement of the performance on an LC test. In other words, the learning process is situated between the independent and the dependent variables and intervenes with the outcome of the research without being observed or measured. And finally, an extraneous variable is a variable whose presence is neither observed nor accounted for in the research design. Such variables may influence and contaminate the outcome of research without the researcher's awareness. In the example above, many variables such as age, educational background, learning style, motivation, and so forth may have some effect on the students' learning. In short, any variable that is not controlled by the researcher and may influence the outcome of research is called an extraneous variable.

When the research question is formulated with well‐defined variables, it is converted into a research hypothesis to be tested. A hypothesis is a tentative statement about the outcome of research and can take two forms: null and alternative. A null hypothesis, symbolized as H0, is generally stated in the form that the manipulation of the independent variable will not have an effect upon the dependent variable. The alternative hypothesis, on the other hand, symbolized as H1, stipulates an effect, either positive or negative, of the independent variable on the dependent variable. For instance, null and alternative hypotheses for the example above would be:

Selecting a null or an alternative hypothesis is often based on a review of past research findings. If there is strong evidence about the relationship between the two variables, the researcher might decide to choose the alternative hypothesis to confirm or reject the previous findings. However, when there are controversial reports on the relationship between the variables under investigation, researchers usually go with null hypothesis. Possible forms of hypotheses are illustrated in Figure 1.

Following a Systematic Approach

When the research hypothesis is formulated, the researcher attempts to choose a systematic approach, technically called a research design, to test the research hypothesis. Many factors—including the nature of the research question, the type and number of variables, the number and groups of participants in the research, and the type of collected data—interact to form an efficient design that will optimize the outcome of the research. In fact, there is no single best way to determine the design of the research (Clark‐Carter, 2018). However, research designs can be broadly divided into two major groups of experimental and nonexperimental where cause–effect relationships are investigated in the former and correlations in the latter. Depending on the factors mentioned above, experimental designs are classified into three categories of true, quasi‐, and pre‐experimental. A true experimental design is the most rigorous of all and has the following principles:

1. Subjects are selected randomly.
2. Two groups of experimental and control are formed.
3. A pretest is administered to both groups.
4. A treatment is given to the experimental group and a placebo to the control group.
5. A posttest is administered to both groups.

In true experimental research, subjects are selected randomly. In random selection, every individual is given an equal chance to be included in the research and no bias is exercised since the selected sample should represent, as closely as possible, the population to which the findings of the research are intended to be generalized. Then the participants are divided into a control group and an experimental group. In the example above, the researcher should make certain, as far as possible, that the two groups have the same LC ability before starting the experiment. Therefore, even after random selection which minimizes prior differences among the selected groups of participants, a pretest is given to both groups and their performance is compared before the experiment. When the equivalence of the groups is statistically established, the experimental group receives the treatment, in this case multimedia tasks on LC, and the control group receives a placebo, or an irrelevant treatment, to make sure that both groups are treated similarly. After a predetermined length and intensity of giving the treatment, a posttest is given to both groups to see the effect of treatment on the participants' LC ability. A schematic representation of a true experimental design is given in Figure 2.

It should be mentioned that not all QRM designs meet all these requirements. It is often very difficult to set up a true experimental design, especially in an educational context. If any one of the requirements of the true experimental design is not met, the researchers use pre‐experimental designs. Further, quasi‐experimental methods are designed to compensate for the weakness of pre‐experimental designs. Of course, the farther away a researcher moves from meeting the requirements of the true experimental design, the weaker the claims about the cause–effect relationship between the variables will be. In the following section, some of the most common types of pre‐ and quasi‐experimental designs will be presented. For the purpose of illustration, the symbol G will be used to refer to a “group,” R to “random,” T for “treatment,” and X for a score.

The simplest design includes only one group of subjects, one treatment, and one test given to the subjects. This design, referred to as a “one‐shot case study,” is represented in Figure 3.

This design can be used as a base to build up more complex designs. For instance, we can improve this design by adding one more group to make it a posttest‐only design, as illustrated in Figure 4.

To make this design more rigorous, pretests are given before the treatment and the resulting design is a two intact group pretest–posttest design, as illustrated in Figure 5.

Pre‐experimental designs are frequently used in educational contexts in general, and in applied linguistics in particular because conducting true experimental designs is extremely impractical in educational contexts. For instance, the two intact group pretest–posttest design, where a particular treatment—for example, multimedia tasks—is given to one group and no treatment to the other group, and both groups receive pretest and posttest, is very close to a true experimental design. The only missing parameter is the random selection of the subjects. It is simply not practical to select a group of subjects randomly from among the subjects who have already been placed in different classes on the basis of certain criteria.

To compensate for this shortcoming, quasi‐experimental designs are developed where one group of participants functions as both the treatment and control group. A common example of such a design is time series design. In time series design one intact group of subjects is selected and exposed to the treatment on several occasions and tested after each treatment. In one form of time series design, the subjects receive a pretest before the experiment. Then they receive a few sessions of treatment in a row as if they were in the experimental group, and a posttest. Then they follow their regular instruction without receiving the treatment as if they were in the control group, and a posttest. This design, called repeated time series design, is schematically represented in Figure 6.

There are two likely outcomes for this design. If the average score of the subjects increases from the pretest to the first posttest, it indicates that the treatment has had a positive effect on the performance of the subjects (black line) provided that there is no considerable increase from posttest 2 to posttest 3 indicating that the increase in the first part of the experiment was due to the treatment. However, if there is a consistent increment in the scores from posttest 1 to posttest 2 (dotted line), it implies that the treatment does not have a specific effect and the trend of increase is the natural result of education.

Another version of this design is to present the treatment and test in alternating sessions. This design, called alternate time series design, is depicted in Figure 7.

If the treatment is effective, the scores on the tests would show a rising and falling pattern, as shown in the graph, implying that the rise of the scores is due to the treatment. If the treatment does not have any effect, the score pattern will not show a systematic rise or fall.

Factorial Designs

The designs mentioned above all involved one independent variable and one dependent variable. The test scores are obtained at different points in time either prior to or after the treatment. More often than not, the research project involves more than one variable. It is a quite common practice to include a moderator variable or allow more than one level of observation for the independent variable. In such cases, factorial designs are employed. In fact, any simple design can be expanded into a factorial design by adding more levels to the independent variable or adding a moderator variable. For instance, in the example provided before, suppose the researcher speculates that boys and girls will respond differently to receiving multimedia tasks regarding LC ability. In such a case, a moderator variable is added to form a posttest‐only factorial design, as shown in Figure 8.

The design can be made more rigorous by adding a pretest before the treatment to form a pretest–posttest factorial design, shown in Figure 9.

This is very similar to intact group pretest–posttest design, with the difference that in the factorial design the groups are male and female. Another example of factorial design would be to introduce two levels of the independent variable. For example, giving multimedia tasks in class versus offering them through the Internet will form two groups for the same independent variable. Each group will receive multimedia tasks in a different form, but the dependent variable is the same, that is, improving LC ability.

A few points are worth mentioning here. First, all the above‐mentioned research designs, whether one way or factorial, involve only one dependent variable, that is, one measurable variable. Therefore, they are called univariate designs. When there are two or more dependent variables in the study, the design will be more complex and is called multivariate design (Meyers, Gamst, & Guarino, 2013). Second, not all QRM falls into the category of experimental designs. QRM could also be descriptive or correlational. When the focus of research is on investigating the existing relationship between the variables without any intervention or manipulation, it is referred to as correlational research and the design is called bivariate since two sets of scores are obtained from each subject. In correlational research the selection of independent and dependent variables is arbitrary because none of the variables is manipulated. For instance, to investigate the relationship between students' language ability and their academic success, a correlational design is used by correlating scores on a measure of language ability and a measure of academic success obtained from each subject.

Following a version of experimental or nonexperimental designs, the researcher collects data on the variables. Data collection is an important stage in QRM because the validity of the findings of the research will depend very much on the quality of the collected data. Therefore, great care should be exercised in selecting appropriate instruments for data collection.

Answering the Research Question

In order to answer the research question, the collected data should be analyzed. Data analysis in QRM is performed by utilizing a body of techniques called statistics. There are good guidebooks on how to apply statistical techniques using computers. SPSS is probably the most commonly used statistical package, with good guides specifically written for data analysis in applied linguistics and SLA research (Larson‐Hall, 2016).

The final stage in quantitative research is to interpret the findings and discuss the implications of the findings for improving the theory and their applications to practice. The validity of the findings depends on the validity of the research. Despite the objectivity in QRM, researchers should observe some reservations in making conclusions about their findings because statistical significance does not necessarily lead to a firm law. Human wisdom should always be applied to the interpretation of the findings. Strong statements should be avoided because the findings may not be perfectly valid.

There are two types of interrelated validity in QRM: internal and external. Internal validity refers to the extent to which the findings are due to the process of research and not due to other variables, and external validity refers to the extent to which the findings are generalizable to the population from which the samples are drawn. To maximize the internal validity, researchers focus on the unique effect of the treatment by controlling as many variables as possible that might have an effect on the dependent variable. However, the more controlled the context of research, the farther away it will be from the real world and the findings will lose generalizability to the population because the context of population will be different from the context of experiment. That is, the higher the internal validity, the lower the external validity or generalization of the findings to the population. On the other hand, to maximize the external validity, the context of research should approximate the context of the real world. In such a case, the effect of the independent variable will be blurred due to the intervention of so many uncontrolled extraneous variables, thus decreasing the internal validity. Therefore, depending on the context of research and the importance of the findings, researchers are recommended to keep a balance between the internal and the external validity of the findings.

Further, some factors, such as history (i.e., what is happening to the participants outside the context of experiment), mortality (i.e., the attrition in the number of participants during the experiment), and test effect (i.e., the effect of the pretest on the posttest), to name just a few, are considered threats to the validity of the findings of research. As Kinn and Joan Curzio (2005) state, in many of the 450 research articles they examined, the methodological and statistical procedures used in the quantitative research to inform recommended practices were not convincing. This implies that only in rare cases can researchers meet all the rigorous and vigorous requirements of QRM, especially in the contexts of human and social sciences. Therefore, researchers are strongly recommended to interpret the findings with caution and reservation.

SEE ALSO: Interviews in Qualitative Research; Mixed Methods