I spend hours searching when I could be reading or analyzing or writing . . . I’m still trying to resolve this issue – (I’m too ashamed to tell my professor at _______) – here I’m in a LIS beginning course training for some aspect of librarianship – and don’t know how to use the searches efficiently! The tutorials are there on the website, but they are not user-friendly.

Where the problem starts

It’s very common for university professors to lay the blame for the inadequate critical thinking and writing skills of their students on poor training in elementary and high schools. Such criticism is not entirely justified. For one thing, most pre-university students lack the mental maturity even to be able to perform sophisticated critical thought. For another, the leap to higher education has always been a huge one requiring a lot of catch-up. (That’s why they call it “higher” education.)

There is evidence, as well, that pre-college educational institutions are looking for new ways to meet the demands of the information age. Culp et al. (2003) found that, in 20 years of study of the use of information technology in pre-college institutions, a fundamental shift had occurred, from seeing technology as a possibly useful adjunct to instruction, to viewing it as a “tool of transformation, which promised, simply by its presence and capabilities, to bring change to how teachers teach, how schools are organized, and how students work together and learn” (ibid., p. 20). The new emphasis on 21st Century Skills (Partnership for 21st Century Skills, 2009), and recent online curricula designed to promote information literacy (such as Scholastic’s Expert Space: www.scholastic.com/expertspace) show promise for the future.

Unfortunately, the new emphasis on technology in learning, while generating great excitement, has not translated into widespread development of student information handling skills. McEuen pointed out that, while pre-college students are using technology widely even in their schools, college students in his qualitative study “reported that their high school experience encompassed the implementation of basic skills (e-mail, Internet browsing, and word processing) at best” (2001, p. 16). The extensive study by Flanagan, Metzger and Hartsell (2010) shows that children up to age 18 are more trusting of Internet sources than they should be. Development of information abilities to required pre-university levels, from gathering information to evaluating it, seems not to be on the agenda of many high school curricula.

Duke and Ward summarize the problem as follows: “All too often, teachers are merely presenters of ready-made information rather than facilitators of knowledge construction, and their students are passive consumers of other people’s ideas rather than active participants in the co-construction of knowledge” (2009, p. 254).

Clearly, if the pre-college education system isn’t embracing the more sophisticated tools of the modern information age, then any hope of instructing pre-college students in the nature of the new information environment, the technology needed to discover information, and the skills required to evaluate it properly, is doomed to fail.

This is not to criticize pre-university instruction as much as to say that, with the world having changed dramatically since 1989 (birth of the WWW), education is still grappling with how to get computers into student hands. We have not begun to address significantly the information skills of the pupil in today’s classroom. Further, we have yet to understand the extent to which our information world has been changed by technology. Our pre-university students are ill-equipped with the information-handling skills needed to move into the workforce, let alone to meet the demands of college and university.¹

Williams and Wavell (2007) found that, while there were some common points between secondary teachers’ conceptions of information literacy and those found in information literacy literature, there were notable differences, leading to deficiencies. Using the language of information literacy, which we have chosen to describe as ability with “research processes,” they write:

Beyond limitations in understanding the nature and complexity of information literacy (or “research processes”), an understanding that improved among the teachers involved as the study went on, Williams and Wavell found that most teachers believed the demands of the curriculum prevented them from helping their students become more skilled. In other words, content trumped process to such an extent that there was no room for development of research skills that would foster learning when their students entered higher education.

Is the complaint of university professors valid? Do incoming students lack the skills to do adequate research or even to handle information well? One of the largest studies of entering university students was done in Quebec, Canada (Mittermeyer and Quirion, 2003). It surveyed just over 3000 students, finding that less than 36 percent of them understood such research foundations as the characteristics of scholarly journals, the difference between library catalogs and bibliographic databases, search terminology constructions that would eliminate non-essential words, the use of controlled vocabularies in databases, identification of a journal citation, and issues regarding the ethical use of Internet information. The researchers concluded that “a significant number of students have a limited knowledge, or no knowledge, of basic elements characterizing the information research process.”

Several other researchers have found the same. Kennedy et al. (2008) surveyed more than 2000 incoming Australian university students who demonstrated that, while they were highly technology-aware in using a core of tool types (computers, cell phones, e-mail), their knowledge of, and facility with, academic tools for research were limited. The researchers commented: “Moreover, it is recognized that core technology based skills do not necessarily translate into sophisticated skills with other technologies or general information literacy.”

The First Year Information Literacy in the Liberal Arts Assessment (2008), studying students in several American and Canadian institutions, found for 2006 and 2007 that incoming university students had weak understanding of many foundational information handling and research skills (though this weakness was ameliorated when information literacy instruction was done). For example, less than half of new students understood the function of a Boolean “or” search, and most could not identify a citation to a journal article or a portion of a book. Only about half had used library catalogs and less than a quarter had used journal databases (though almost everyone used search engines).

We may, in fact, have created a misnomer in referring to our young people as the “Google Generation.” The University College London (UCL) CIBER Group (2008) study, a combination of literature review and live study of information seeking, found that today’s so-called Google Generation shows no uniformity in the use of technology for information seeking. Less than a third of UK teenagers have a deep interest in IT technology. Moreover, “there is no evidence in the serious literature that young people are expert searchers, nor that the search skills of young people has improved with time” (ibid., p. 22). Little effort was being devoted by these students to evaluating information for accuracy, quality or relevance.

The UCL study concludes:

More recently, the Ontario (Canada) Confederation of University Faculty Associations surveyed 2000 professors about their freshman students’ skills and found deterioration, over earlier studies, in freshman ability to handle university studies (Ontario (Canada) Confederation of University Faculty Associations, 2009). Of the five most often stated deficits, two stand out: “Lack of required writing, mathematical and critical thinking skills” and “poor research skills as evidenced by an overreliance on Internet tools like Wikipedia as external research sources” (ibid., p. 2).

The British study, Higher Education in a Web 2.0 World (Melville, 2009) pointed out serious gaps in the information literacy of school age and university students. The top two fundamental issues the report identified as most needing to be addressed with urgency were the digital divide and information literacy. The study’s conclusion: “Information literacies,including searching, retrieving, critically evaluating information from a range of appropriate sources and also attributing it – represent a significant and growing deficit area” (ibid., p. 6).

So the answer is generally, yes, the students who enter our colleges and universities as first year undergraduates are often poorly equipped to handle research and information tasks at the higher education level.

University students and information skills - an overview

True or false?: College and university students over time develop reasonably good research skills simply through practice. Any students who do not are likely unmotivated or require remedial instruction. Many professors I know would answer, “True.” And they would be wrong. The research unequivocally tells us that the answer is “No.” In fact, most students perpetuate both their lack of understanding of the information world and its tools and their assumption that there is little to learn about doing research. Students do not develop significantly better research skills by the experience of doing research. While they tend to overestimate their research ability, they go on performing at a level far lower than their programs of study expect of them.

Massey-Burzio’s (1998) study of undergraduates at Johns Hopkins University (an institution with a strong emphasis on research writing) found that students continued to have trouble understanding how libraries are organized, tended to retrieve too much information and not know what to do with it, chose the wrong databases for the information they need, tended to stick to one or two favorite databases, and much preferred searching for books to finding journal articles. Her telling conclusion is that “library patrons think that using a library does not require all that much skill development and knowledge, . . . [and] they are, therefore, unwilling to invest time and energy into developing those skills and knowledge” (ibid., p. 215).

This unwillingness of students to recognize the need for skill development is often seen in the literature. Andretta et al. (2008), for example, found that most students view information literacy instruction as a waste of time, assuming that they know everything because they are technologically literate.

The technological abilities of today’s students seem to offer little help when it comes to doing academic research. McEuen (2001) reported on a study within the Association of Colleges of the South that found that students had strong self-reported skills in technology use. But when it came to “using a database system to set up and access useful information” (ibid., p. 14), the self-reported scores went down dramatically, with only 31.4 percent of students believing they had average to expert skills (see also Lippincott, 2005).

The supposed positive relationship between developing technological ability and research process skills is increasingly being debunked as an optimistic but unfounded assumption. Katz and Macklin (2007) show strong evidence that frequency of student use of information technology does not correlate with information technology skills. Katz (2007) discussed the release of preliminary research from the Educational Testing Service (ETS) showing that “while students may be tech savvy when it comes to entertainment, they may not have the critical thinking skills to perform the kinds of information management and research tasks necessary for academic success.” Using a measure of ICT (Information and Communications Technology) Literacy, which is akin to information literacy, ETS found that only 44 percent of students could identify a research statement that met the demands of an assignment. Only 35 percent correctly narrowed a topic. In a database search only 50 percent used a strategy that minimized irrelevant results.

Alexius Macklin, Associate Professor of Library Science at Purdue University, added this comment in Katz’s report:

Overall, on tests of these skills, students earned about half of the points they should have (Educational Testing Service, 2006).

Pan et al. (2007) found that college students are more influenced by the position of results in a list than perceived relevance of results for the research being done. That could well mean that they are more likely to choose the early results of a search engine or database search, rather than sifting through results to glean the most useful information.

Students seem unable to understand how the world of academic information functions, and for good reason. Jill Jenson, an associate professor at University of Minnesota, explains:

The picture is no better when it comes to higher level skills such as effective evaluation and use of sources. Wang and Artero (2005), in a study of Internet use among 647 students, found that 40 percent believed that information found through an Internet search engine was as reliable as that in books and journals, while a further 33 percent were undecided on the issue. Though 78 percent reported that they evaluated Web resources before using them, 58 percent indicated that they would use a piece of information so long as it fitted their point of view. The authors concluded that their subjects were creating their own highly subjective evaluation criteria. “Although the students in this study judged that they had critically evaluated Web information, their responses to the survey questions showed that they were not equipped with sufficient knowledge and skills to critically evaluate Web resources” (ibid., p. 80).

It has been overwhelmingly demonstrated in multiple studies that students continue to struggle, year after year, to meet the research project demands of their professors. Head and Eisenburg (2009), in an investigation of 86 second year to senior undergraduates at several American universities, found significant levels of frustration with research. Though most students had developed personal procedures to complete research projects, it was often at the cost of wasted time and poor results. Head and Eisenburg concluded:

Take note that the subjects of this study were not first year students. Their experience of research processes seems to have done little to improve either their skills or their understanding of those processes.

Information literacy of senior undergraduate/graduate students

It is a well-accepted element of academic mythology that undergraduates may begin their studies with gaps in research ability but their skills become more fully developed as they complete their assignments (the “information literacy by osmosis” myth). While students do, inevitably, become somewhat more skilled as time goes on, the notion that they are competent researchers by the time they reach their senior undergraduate year has been soundly refuted by a significant number of studies.

Knight-Davis and Sung (2008) investigated 957 undergraduate writing samples from electronic writing portfolios at Eastern Illinois University. While the number of papers even having reference lists rose from 39 percent to 51 percent over the four papers submitted per student, the number of citations did not. Fewer than 10 percent of the papers that did have reference lists had more than 10 citations in them.

Weetman (2005) specifically studied the “osmosis theory” in a survey of faculty at De Montfort University, UK. Among those faculty who responded, fully 93 percent believed that students should have achieved recognized standards of information literacy by the time they had completed their undergraduate studies. The study concludes:

Maughan (2001) presented surveys administered to senior undergraduates at the University of California-Berkeley in 1994, 1995, and 1999 which showed that students consistently overestimated their research ability, while, of eight discipline-specific groups of students studied, five showed failing scores even on measures of lower-order information literacy. His study concluded: “students think they know more about accessing information and conducting library research than they are able to demonstrate when put to the test” (ibid., p. 83).

Not only are these students not developing library skills, but they are also deficient in assessing the information they do find. Kuh and Gonyea (2003) studied data gathered from over 300,000 participants in the College Student Experiences Questionnaire, 1984–2002. While more students were using databases in the later years, almost 20 percent of senior students indicated that they never make judgments about the quality of information they acquire for academic work. The researchers concluded: “This is an unacceptably high number of students about to graduate from college who, by their own report, are underprepared to live and work in an information-rich world” (ibid., p. 266).

Alison Head’s (2007) research may seem a bit more encouraging, in that her study of upper level undergraduate students found them reducing their use of Google as a first choice when starting research, in preference to course readings and even library resources. But her results are telling nevertheless as she argues that: “Most students were confused by what college-level research entails.” About 60 percent of her subjects struggled with narrowing topics and making them manageable, while the same percentage admitted being overwhelmed by the number of resources available to them. Interestingly, the greatest frustration was reserved for the perceived lack of guidance from professors regarding the conduct of quality research (supported by an actual lack of helpful instruction in assignment handouts studied). We may thus assume that these students will graduate having received less than complete guidance from their professors and lacking research skills they should have had.² Clearly there is a disconnection between what professors think they are communicating and what their students are actually able to understand.

Head’s (2008) study of junior and senior undergraduates in the humanities and social sciences found ongoing struggles with the research process:

A complicating study is that of Rosenblatt (2010), who found that upper level undergraduates were indeed able to meet the bibliographic requirements of their professors but were not able to synthesize gathered resources into the bodies of their research papers. Yet, when we look more closely at the nature of those “bibliographic requirements,” we find that expectations of the cooperating professor were low (three articles, one book and one presentation text). Presumably most students could keyword search specified databases and obtain the five resources required. This low level of expectation, however, simply highlights the fact that a number of professors have come to believe that their students are only capable of doing minimal work in compiling bibliographies. Thus the professorial requirements are minimal as well.

Paradoxically, considering the frustrations they experience, it is a well-documented phenomenon that students consistently overestimate their own research ability. For example, Massey-Burzio’s (1998) study found significant discrepancies between high self-perceptions of students at Johns Hopkins University and their extensive ignorance of actual research skills. Ren (2000) found that the positive views of students regarding their perceived research ability had no significant relationship with their actual self-efficacy unless they received research training. This adds support to the earlier study of Fischoff (1986) who demonstrated that, while overconfidence regarding research ability is the norm among students, it increases in proportion to their lack of actual knowledge of the research process.

If we view yesterday’s senior undergraduate as tomorrow’s new graduate student, we can see a continuum in the information literacy problem in grad schools. Graduate students also overestimate their abilities while showing significant gaps in their information discovery and handling facility. Many participants in a 2003 study of 330 beginning graduate law students believed that their research skills were well advanced, while they failed dramatically in an actual test of skills (Anon, 2004). Perrett (2004) found that 81 percent of incoming graduate students in several disciplines required further information literacy instruction in order to meet educational standards, though many of them had self-rated their skills as good or excellent.

Such results are no surprise to university reference librarians who regularly observe significant gaps between personal assessment and actual skills, from freshman to grad school levels. Kuruppu and Gruber (2006) as well as Gallacher (2007) have found evidence of the same overestimation of ability even in more advanced graduate students and faculty.

Gross and Latham (2009) point to the field of competency theory as a possible explanation. Competency theory, as discussed by Kruger and Dunning (1999) and Ehrlinger et al. (2008) demonstrates that the less competent someone is at a skill, the more likely he or she is to overestimate their ability. It is also the case that the less competent will generally not learn how better to estimate ability from further trial and error. Lack of skill, in fact, leads directly to inability to discern lack of skill. Thus the confidence of students who believe they already know how to do research is not at all incongruent with their less than adequate skills when actually tested.

In the face of growing use of Internet search engines by students, research is consistently demonstrating that 45 percent or more of students, even graduate students right to the doctoral level, use search engines such as Google and Google Scholar as their predominant search tools in research (for example, Griffiths and Brophy, 2005; Liu and Yang, 2004, p. 26; University College London (UCL) CIBER Group, 2008). This demands that we question further whether or not students do well searching even with Google. Evidence is available, once again, to show that they do not (Griffiths and Brophy, 2005).

Graduate students in carrying out their research projects tend to reveal a strong level of uncertainty. George et al. (2006) studied the information-seeking behaviors of 100 graduate students from a wide range of disciplines. One interesting finding was the extent to which students initially sought the advice of professors and peers on the best way to begin research projects. Professors often responded with suggested authors or particular approaches students might take. About 40 percent of students sought out librarians for help with search strategies and shaping their topics.

Most of these students used the World Wide Web as a significant tool for information seeking, though less than half saw themselves as specifically searching for scholarly papers and articles as opposed to more general websites. The students expressed personal challenges with information overload, search strategy and evaluation of resources. Many pursued a pattern of following up on citations in materials they located (citation chaining), though this method, while useful, does not ensure comprehensiveness.

The same study revealed that 42 percent of graduate students reported that a lack of knowledge of tools and resources, along with a skills gap in using them effectively, was limiting their research success. From previous studies demonstrating that upper level students consistently overestimate their research ability, the self-declared 42 percent may be much higher in practice. The strong use of basic Internet search engines, despite their uneven results, is another indicator that convenience and searching ease are trumping search sophistication, such as we find in library databases.

The information literacy gap appears to exist across graduate disciplines. A telling study by Randall, Smith, Clark and Foster (2008) demonstrates haphazard, confused and inconsistent research methods among students doing doctoral research across a number of disciplines. Other than the mining of existing bibliographies, it appears that none of those subjects had genuinely sophisticated skills in locating information. Few of them were using bibliographic managers to organize their resources, and there seemed to be a general air of trial and error in all of their research methods.

Gallacher (2007) reported widespread inadequacies of research ability in studies of incoming law students in seven institutions and saw little evidence that the research training available to law students was succeeding. His conclusion:

The LexisNexis International Workplace Productivity Survey (2010) reserved a portion of its study for lawyers worldwide. Six of ten such lawyers agreed that the quality of their work is at times impaired by the sheer volume of information they encounter daily. If they lack the information handling skills they need, the problem is surely going to be exacerbated.³

Lippincott and Kuchida (2005) found that MBA graduates continue to struggle with information needs in the business world. “Of concern was the lack of differentiation between information skills and technological abilities and the lack of understanding of the complex nature of information used to make important business decisions.”

Brown (2005), studying molecular biology graduate students, determined that, while they were reading a selection of key journals on a regular basis, these students relied on bioinformatics databases much more than they did on key journal databases. Approximately half of those surveyed in this study were not users of common databases for scientific information, such as SciFinder Scholar, Web of Science, Biological Abstracts, and Zoological Record.

The information literacy of faculty members

The world of information has changed dramatically in the past twenty years, and so have the now highly technological tools of research. Faculty members, busy with their own disciplines and tending to keep up with advances in their field primarily by reading specific journals and maintaining a strong network of colleagues, often find that the ever-changing tools of research are leaving them behind. When academics actually need to do literature reviews, it may well be that the complexities, and downright strangeness, of the databases they have to use prove to be a barrier. Many of them, particularly in the sciences, are turning to Google Scholar, which, though seemingly easy to use, has a simplistic search interface and confusing result displays.

Over the past while, I have had a number of professors come to me for private (and I do mean “private”) sessions to help them through their research tasks. They are generally uncomfortable and embarrassed, but they also reveal a level of anxiety in the midst of their attempted bravado (“I should know this, but I’ve been so busy lately; when will they stop changing these databases every year?” and so on). While academic faculty members are frustrated with the shoddy research done by students (Gilchrist, 2007), they often fail to see that the winds of their own research requirements have changed and their ship is listing badly.

Massey-Burzio (1998) found that, while a number of faculty members studied at Johns Hopkins University admitted that they lacked certain research skills, they did not see the problem as sufficient to pursue further. Could this be in part because of embarrassment about admitting that they have not kept up with technology? There is, in fact, a degree of professorial resistance to upgrading their own research skills. Feldman and Sciammarella (2000), for example, found that only 35 percent of 425 faculty members at the City University of New York had attended their own library’s seminars on technological resources for research.

Hall (1999) describes the introduction of a well-received faculty upgrading program. From her experience with the program, she comments about faculty:

More professors now appear to be waking up to the onward march of technology, and academic librarians regularly find themselves providing upgrading to those faculty members willing to admit to inadequacies in their skills with research technology. Meanwhile, however, many students are still being given assignments that fail to recognize the power of the newer research tools.

Shen (2007) surveyed social science professors at the University of Wisconsin-Madison. While a number of them were making use of electronic resources, there was high reliance on Google and free databases on the Internet. In their use of all databases, these academics expressed struggles with doing the difficult task of computer searching, viewing their search engines as not intelligent enough to provide what was being sought except through a lot of brainstorming and guesswork.

Stoan (1991) sought, decades ago, to provide a corrective to the consensus view of librarians that most academics seem to rely more on personal networks and citation gleaning than on databases. While professional academics appear haphazard in their research methodology, each of them is part of an environment in which skills and expertise are furthered by a network of collaboration and reliance on key publications. The fact, argued Stoan, that good products come out of their research shows that it is effective. Bibliographic tools, in fact (fairly primitive when Stoan was writing) produce inconsistent results, and most academics rarely need to produce a full literature review from scratch.

All of Stoan’s arguments remain true today, yet they fail to address the issue that, while immersed in their world of current journal issues and academic collaboration, today’s average faculty members seem unable or unwilling to impart a knowledge of sophisticated bibliographic tools to their students. These students, lacking a professional skill and knowledge base themselves, absolutely must rely on bibliographic tools even if their professors do not. And when faculty members need to expand their scope by using the same tools, they often lack the advanced skills to optimize their searches.

A study entitled Higher Education in a Web 2.0 World concludes with a statement regarding university professors:

The bottom line: information illiteracy in academia

Admittedly, students never were very adept at handling information and doing research. Professors for decade upon decade have been well used to getting slap-dash, mediocre research papers showing limited analytical thinking and sporting too few references. But the rise of information technology has added complex new wrinkles to the problem.

Many of today’s students go first to Google or Wikipedia for most any kind of research project (Head and Eisenberg, 2009), the latter being a tool that a goodly number of their professors warn them against using. But students do not care, because they know that Wikipedia is where the information is. Students will use books in their research, if they can find them, but rarely journals. Student search skills, despite the seemingly technologically advanced nature of today’s young people, are poor to dismal, whether using Google or a proprietary journal database.

But this only scratches the surface of the problem. As Jenson (2004) has made plain, the web-based nature of most students’ research experience turns the notion of such things as journal volumes, issue numbers, and dates into something quite alien. The average undergraduate cannot distinguish, from their citations, the difference between a journal article and an essay in a book. Students, addressing most research information electronically, see it as content rather than understanding it as a collection of distinct entities like books, journals, and so on.

Clearly, many students, whether undergraduate or graduate, have trouble formulating research questions/thesis statements, identifying their information needs, locating relevant information by use of good search strategies, evaluating the information they’ve found, and applying it effectively to the research problem at hand. This is what we mean by ability with research processes. Its missing presence in student curricula is both mystifying and disturbing.

Perhaps even more seriously, faculty members as a whole, as we will see, do not appear to have noticed the gap, nor do they have either the strategies or, apparently, the time to do much to improve the research abilities of their students. In fact, a significant proportion of professors themselves are rapidly becoming less able to do their own research, because the skills required to handle today’s information environment are radically different from what they were even 15 or 20 years ago.

The world of information has changed in ways that we are only beginning to appreciate. Sadly (and dangerously), at the very moment in which the information age has finally arrived, those who would best be able to walk into that world, skilled and confident, are finding themselves ill-equipped, and their teachers often unable to provide them with the abilities they need.