Introduction

Many researchers have found the concepts of serendipity and chance (or accident) useful for analyzing discovery in science and technology. Here are a few examples. Royston M. Roberts published in 1989 a book entitled Serendipity: Accidental Discoveries in Science. Kantorovich and Ne’eman in 1989, and van Andel in 1994 published papers on serendipity. In 2004, there was published, for the first time in English, a long book by Merton and Barber on serendipity. This book had originally been written in 1958, but did not appear in print until 2002, when Il Mulino published an Italian translation. In 2010, Norrby published Nobel Prizes and Life Sciences, and chapter 2 of this book is entitled “Serendipity and Nobel Prizes.” Anguera de Sojo et al. (2014) discuss “Serendipity and the Discovery of DNA.” What they deal with is not the discovery of the double helix structure of DNA by Watson, Crick, Franklin, and Wilkins in 1953, but the much earlier discovery of the molecule itself, originally called nuclein. This discovery was made by the Swiss chemist Miescher in 1869.

This paper fully endorses the importance which has been given to the role of serendipity and chance in scientific discovery. However, the concept of serendipity is not entirely clear, and indeed has been used in different senses by different authors. Moreover, the relationship between serendipity and chance (or accident) is also in need of clarification. I will therefore begin by discussing in the next section the concepts of serendipity and chance.

Serendipity and Chance

There is one definite fact about the term “serendipity” – it was coined by Horace Walpole in a letter to his friend Horace Mann, dated January 28, 1754. However, Merton and Barber (2004) show that the subsequent history of the term was tortuous and complicated. It has been, and still is, used in a variety of different senses. In this paper, I will use a definition of serendipity which, according to Merton and Barber (2004: 112), was first formulated by Edward Solly in 1880 and is cited in the Oxford English Dictionary. According to this definition, serendipity consists in “looking for one thing and finding another.” This definition does have some basis in Horace Walpole’s original letter, for he says there: “you must observe that no discovery of a thing you are looking for comes under this description” (Merton and Barber 2004: 2). However, as we shall see, serendipity is sometimes used in broader senses, which also have some basis in Horace Walpole’s original letter.

The classic instance of serendipity, as it has just been defined, is Columbus’s discovery of America. Columbus was looking for a sea route to the East Indies obtained by sailing west. However, what he actually found was a new continent, whose existence was unknown to European geographers.

The next step in clarifying serendipity in the sense to be used in this paper is to distinguish between serendipitous discoveries and accidental or chance discoveries. A discovery can be said to be accidental or to involve chance if some accidental or chance occurrence plays an important role in the discovery. The classic instance of an accidental or chance discovery is Fleming’s discovery of penicillin. Fleming was carrying out a routine experiment when one of his Petri dishes became contaminated with a mold. Fleming noticed that the colonies of staphylococcus aureus (a pathogenic bacterium), which would normally have covered the Petri dish, appeared to have dissolved in the vicinity of the mold. He concluded that the mold must be producing an anti-bacterial agent, which he later named penicillin. There can be no doubt that chance (or accident) played a large part in this discovery, but was the discovery serendipitous?

In his 1944 account of the discovery of penicillin, Fleming begins by sketching the background of his researches, first into septic wounds and then into lysozyme. He then writes (4):

It was however, fortunate that, with the background I have briefly sketched, I was always on the lookout for new bacterial inhibitors, and when I noticed on a culture plate that the staphylococcal colonies in the neighborhood of a mold had faded away I was sufficiently interested in the antibacterial substance produced by the mold to pursue the subject.

(Fleming 1944: 4; my italics)

At the time Fleming was performing a routine experiment, but he says explicitly that he was always on the lookout for new bacterial inhibitors, and attributes his further investigation of the mold to this fact. Thus Fleming found something he was looking for, and so his discovery, according to the above definition, was not serendipitous. Despite this, Fleming’s discovery of penicillin, and indeed accidental discoveries in general, are often described as serendipitous.

Van Andel also quotes from Fleming, but suggests (1994: 639) a compromise position according to which Fleming’s discovery is described as an instance of pseudoserendipity. The concept of pseudoserendipity comes from Roberts, who defines it as follows:

I have coined the term pseudoserendipity to describe accidental discoveries of ways to achieve an end sought for, in contrast to the meaning of (true) serendipity, which describes accidental discoveries of things not sought for.

(1989: x)

This point of view is expressed also in the title of Roberts’s book Serendipity: Accidental Discoveries in Science. My own view, however, is that it makes for greater clarity to distinguish the two concepts “serendipitous discovery” and “accidental discovery” rather than running them together in a portmanteau concept of “pseudoserendipity.” One reason for this view is that many serendipitous discoveries are not in fact accidental discoveries. The standard example of Columbus discovering America is an illustration of this. Columbus’s discovery did not involve any chance events or accidents like the contamination of a Petri dish. Anyone who followed Columbus’s strategy of sailing west in order to discover a sea route to the East Indies would inevitably, if his ship hadn’t accidentally sunk, have discovered America. America was simply there blocking the path, and any mariner following Columbus’s search strategy would inevitably have run into it.

Let us further illustrate this idea by another discovery, which is acknowledged by all to have been serendipitous, namely Röntgen’s discovery of X-rays. When he made the discovery, Röntgen was carrying out an investigation into whether cathode rays could penetrate the glass walls of the cathode ray tube in which they were produced. However, when he switched on his cathode ray tube, he noticed that a barium platinocyanide screen more than a meter from his apparatus started to glow with a shimmering light. He investigated the matter further and concluded that the glow was caused by some rays produced by the cathode ray tube which were different from cathode rays. These rays he named X-rays, because their nature was unknown.

Now Röntgen’s discovery was clearly serendipitous. He was looking for one thing (cathode rays which penetrated the walls of a cathode ray tube), and found another (the new and mysterious X-rays produced by the cathode ray tube). It also seems that accident played a key role in the discovery. Röntgen happened to notice the glowing of the barium platinocyanide screen. But why was there a barium platinocyanide screen in the laboratory? The answer is that this was the fluorescent screen which Röntgen had planned to use as a detector of any cathode rays which penetrated the glass walls of the cathode ray tube. Now, suppose Röntgen had failed to notice the glowing of the screen. He would have continued his experimental protocol and used the screen to attempt to discover cathode rays penetrating the glass walls, and, in this investigation (if carried out systematically as it would have been), Röntgen would have undoubtedly discovered X-rays. So, although there was an accidental observation in the discovery of X-rays just as there was in the discovery of penicillin, the two cases are different. The accidental observation was essential to the discovery of penicillin which would not have occurred without it, but the accidental observation was only incidental to the discovery of X-rays which would almost certainly have occurred without it.

For these reasons, I favor distinguishing between serendipitous and accidental or chance discoveries. Of course it could be the case that a serendipitous discovery is also accidental, or, at least, involves chance elements. An example of this is Perkin’s discovery of the first synthetic dye (Roberts 1989: 66–70). Perkin set out to try to synthesize quinine from coal tar products. He first used toluidine for his starting point, but did not meet with success. He then tried using aniline as his starting point, and was no more successful. He obtained, as a product of his chemical reactions, a black solid. Another investigator might have thrown this away, but Perkin noticed that the water or alcohol used to wash it out of the flask turned purple. This gave him the idea that the black solid might yield a purple dye: an idea which proved to be correct.

This is clearly an instance of serendipity. Perkin was looking for one thing (a method for synthesizing quinine), and found another (the first artificial dye). However, the case of Perkin differs from those of Columbus and Röntgen. As I have argued, it was almost inevitable that Columbus and Röntgen would have made their discoveries just by following the search strategy which they had laid down for themselves in advance. This was not so in the case of Perkin. He, like Fleming, needed to have the good judgment to change his search strategy in quite a radical way. Perkin when he obtained an unpromising solid black residue had to start investigating the new idea that it might yield a purple dye. He could just as easily have thrown it down the drain, and started another attempt to synthesize quinine. However, Perkin also benefited from a lucky chance. As Roberts says (1989: 67): “Actually, the aniline Perkin used contained small amounts of toluidine, which was essential to the formation of the purple dye.” This all agrees surprisingly well with Walpole’s original definition of serendipity, for Walpole says of the three princes of Serendip (quoted from Merton and Barber 2004: 2): “they were always making discoveries, by accidents and sagacity, of things which they were not in quest of.” Perkin discovered something he was not in quest of partly by accident (the aniline he used was contaminated with toluidine), but also by sagacity. He showed considerable sagacity in reaching the conclusion that his black solid residue might yield a purple dye.

That concludes my main analysis of the concept of serendipity, and its relation to chance (or accident). However, it will be useful to make one further point about the concept. The word “serendipity” suggests something strange and mysterious. Yet there is really nothing mysterious about many instances of serendipity as it has been defined in this paper. Suppose I have mislaid my pen and search for it in a drawer filled with miscellaneous objects. The pen is not there, but, at the bottom of the drawer, I find an old notebook which had gone a-missing more than a year before. This is an example of serendipity since I was looking for one thing, and found another. However, there is nothing strange or mysterious about it. Some of the famous examples of serendipity such as Columbus’s discovery of America, or Röntgen’s discovery of X-rays, are scarcely more mysterious than my everyday example of the pen and the notebook.

Implications of Serendipity and Chance for the Individual Researcher

There are a number of clear implications of serendipity and chance for the individual researcher who wants to make a discovery in science or technology. First of all, such a researcher should always keep an eye open for something unexpected, and reflect as to whether this unexpected occurrence might have some significance. This was already stressed by the pre-Socratic philosopher Heraclitus, who wrote: “If one does not expect the unexpected one will not find it out” (Kirk and Raven 1957: 195). Second, a researcher should always have a flexible attitude toward his or her research plans, and be prepared to change them in the light of unexpected developments. This implication comes out very clearly from the cases of Fleming and Perkin. Fleming was conducting a fairly routine investigation of the pathogenic bacterium staphylococcus aureus, when he noticed the Petri dish contaminated by a mold. Now such contaminations were a frequent occurrence in the bacteriological laboratories of the time. Normally the plates so contaminated were of no further use, and were simply discarded. However, Fleming had the insight to realize that this contaminated plate had unexpected features which were worthy of further investigation, and he had the flexibility to abandon his planned investigation, and to start a new and different investigation of the contaminated plate. The case of Perkin is very similar. Perkin was trying to synthesize quinine, but instead obtained an unpromising black solid. However, noticing the unexpected fact that this black solid turned water and alcohol purple, Perkin had the flexibility to give up his attempt to synthesize quinine, and to start a new research program designed to synthesize an artificial purple dye. So, to sum up, the implication of serendipity and chance is that the individual researcher should have the flexibility to alter his or her plans for research in the light of unexpected developments.

At first sight this may seem just a matter of training researchers to acquire some psychological characteristics. However, this is to ignore the importance of the social and economic environment in which the researcher is working. This environment was very favorable for Fleming. He was the deputy of the head of the laboratory (under Sir Almroth Wright). Sir Almroth had a great admiration for Fleming’s abilities as researcher, and did not interfere in any way with Fleming’s research. Thus Fleming was free to carry out whatever research he liked. Other researchers are not in such a happy position. Perkin, for example, at the time of his discovery, was a student of the Royal College of Chemistry (now part of Imperial College London). His supervisor was a distinguished professor of chemistry (von Hofmann), who had come to London from Germany. He set Perkin the task of trying to synthesize quinine. Quinine was the only effective treatment for malaria, but it was an expensive natural product, and there could be considerable commercial advantages in finding how to make it in the chemical laboratory. When Perkin decided that it would be more interesting to try to create an artificial dye instead of synthesizing quinine, he was afraid that von Hofmann would not approve. Luckily von Hofmann was away on a visit to Germany at the time. So Perkin carried out his new research program secretly in a hut in his garden with the help of his brother Thomas and a friend, Arthur Church. However, if Perkin had been working in an official laboratory, under the watchful eye of von Hofmann, he might not have been able to pursue his research plans, and create the first synthetic dye. All this shows that a consideration of the psychology of the individual researcher is not enough, and we must also examine policy implications about the way in which research should be organized and funded. We will make a start with this investigation in the next section.

Policy Implications of Serendipity and Chance

The policy implications of serendipity and chance in scientific and technological discovery are clear. Research and its funding should be organized so that the researcher is encouraged to adopt a flexible attitude toward his or her research plans, and to be prepared to alter those plans in the light of unexpected developments. I will call this the principle of encouraging flexibility. Unfortunately, this principle is negated rather than adopted by most existing systems of research funding. In this section, I will give a particular striking example of a research funding system which negates the principle of encouraging flexibility,¹ and in the next section, I will discuss the problem in more general terms in the context of globalization.

The funding scheme, which I will now describe, was introduced in 2001 by the Arts and Humanities Research Board (AHRB), one of the leading UK funding bodies. The scheme was designed to provide some research leave for researchers to pursue their research free from teaching and administrative duties. In September 2001, the AHRB issued a Guide to the Research Leave Scheme (AHRB 2001). This guide explains the details of the new scheme in a document of 28 pages. It has to be said that such bureaucratic documents do not in general make very enjoyable reading. However, I will try to show that the judicious mind can find much of interest in the pages of AHRB (2001).

Let me begin by describing the application procedure, which is very complicated. At the end of AHRB (2001), there is a copy of the application form with a further three pages of guidance notes on how to fill it in. The heart of the application is of course the description of the research which the candidate wants to carry out. However, the candidate is certainly not allowed to use his or her initiative in describing the research. The description must be given in terms of a scheme devised by the AHRB. The relevant section begins: “You should read the guidance notes provided before completing this section,” and the applicant is then told to use the following sub-headings:

• Research question(s)
• Aims and objectives
• Research context
• Research methods
• Timetable for completion within the period of leave (please refer to guidance notes for definition of completion)
• Plans for public dissemination (e.g., publication, exhibition, performance)

Of course the applicant might very well think that the scheme devised by the AHRB is not an appropriate one for describing his or her research, but then “rules are rules.” Moreover the instructions conclude with the following instruction: “Please complete the word-count box provided: if you exceed the word limit, your application will be deemed ineligible for funding and will be returned to you.”

Filling in an application form is, however, only the beginning. The candidate has to find two external assessors who are then sent a complicated form with guidelines about how to fill it in. They have to use this form to give an assessment of the candidate’s application. After that “The external assessors’ reports together with your application form will be sent for assessment to peer reviewers” (AHRB 2001: 13) because, as we would expect, “The Board is committed to the principle of peer review in its Advanced Research competitions” (AHRB 2001: 12).

So far the AHRB has introduced quite a lot of complexity into the application procedure. However, the most striking innovation of the AHRB is not here, but rather in what happens after the completion of the period of research leave. This is described in the sections of AHRB (2001) dealing with monitoring, and we will now give a brief account of the new procedures introduced.

Addressing the candidate, AHRB (2001: 19) says:

You must submit an end-of-award report within three months of the end of the period covered by the award. … In the report you will be asked to provide a self-assessment of the extent to which you have met the original aims and objectives of the research … The report will normally be assessed through peer review.

The more detailed instructions require that the scheme of research from the original application be attached to the end-of-award report and referred to when completing the end-of-award report. The end-of-award report with its attachment is then sent for peer review so that the peer reviewers can judge to what extent the original plan has been carried out.

What is striking here is that the AHRB are imposing a particular strategy for carrying out research. This strategy has two parts. (1) A detailed plan for future research must be drawn up. Moreover, this plan must use categories imposed by the AHRB, and not categories selected by the researcher himself or herself. (2) This plan must be strictly adhered to. We can summarize this strategy as that of strict adherence to earlier detailed research plans. It is immediately obvious that this completely contradicts the principle of encouraging flexibility.

The AHRB makes clear that they fully endorse this strategy of strict adherence when they go on to describe how the peer reviewers will be required to assess the end-of-award report, and also the penalties which will fall on the heads of those researchers who have bad end-of-award reports. The peer reviewers are required to assess the end-of-award report using the following three categories: satisfactory, problematic, and unsatisfactory. The authors of AHRB (2001) go on to say:

unsatisfactory – indicates a project that … has failed to conduct the research as agreed at the time of the award (and any subsequent agreed changes to the plan of research), and which therefore does not meet the regulations and the aims and objectives of the particular scheme of awards. (20)

They then list the penalties to which the unfortunate researcher who has an unsatisfactory end-of-award report will be subjected. A selection from these penalties runs as follows:

If your end-of-award report is assessed as unsatisfactory:

• we will write to you, and to your host institution, to inform you of the status of the report.
• you will be debarred from making applications to any of the AHRB’s research schemes for two years … We will write to you, and to the Head of your institution, confirming the penalty …
• we will keep a record of the unsatisfactory assessment on file, and this will be taken into consideration when you make further applications to the AHRB. (20)

These penalties are very harsh, particularly for a young researcher. Such a researcher, on receiving an unsatisfactory assessment, is debarred from applying for any research funding from the AHRB for two years, and, since the unsatisfactory report will be kept on file, it becomes very unlikely that he or she will get any further research funding from the AHRB even after two years. As the AHRB might well be one of the few sources for research funding, this is a heavy blow to the young researcher’s hopes of a research career. In addition, there will be a letter to the Head of the researcher’s institution confirming the penalty, and this makes it unlikely that the unfortunate researcher will get further research leave from that institution, as well as damaging promotion prospects. The curious thing about these heavy penalties is that they are quite compatible with the researcher having worked hard during the period of research leave, and even with the researcher having done what is generally agreed to be brilliant research. This is because one of the criteria for getting an unsatisfactory assessment is explicitly stated to be failure “to conduct the research as agreed at the time of the award.”

It is now an easy matter to show that if the research of Fleming and Perkin had been funded by a scheme similar to AHRB (2001) (which was not of course the case!), then, if the two researchers had followed the requirements of the scheme, Fleming would have failed to discover penicillin, and Perkin would have failed to discover the first synthetic dye. Let us take the case of Fleming first.

Fleming made his discovery when engaged in a piece of fairly routine research into the staphylococcus bacterium. Staphylococci are responsible for a variety of infectious diseases – some quite serious. The most virulent form of the bacterium is the golden-colored staphylococcus aureus. There are also staphylococci with other colors, such as white, which are much less virulent. In 1927 or 1928, Fleming read a paper claiming that colonies of staphylococci changed color if they were kept at room temperature for several days. Fleming decided to check the claims of the paper he had read by conducting a program to investigate color changes in staphylococci. His procedure was very simple. He prepared colonies of staphylococci in Petri dishes, and left these dishes on his bench, examining them every few days to see if changes in the color of some of the staphylococci could be observed.

It should be pointed out that this research program was in line with a well-known principle introduced by Pasteur in the nineteenth century, that of oxygen attenuation. Pasteur discovered that many pathogenic bacteria lose some of their virulence if exposed to oxygen for lengthy periods. The bacteria might in some cases be treated with oxygen gas, but in others a simple exposure to air for a protracted period could suffice. Pasteur used this principle to create vaccines. So Fleming’s research was a typical instance of what Kuhn would call normal science. If successful, the research might establish another instance of Pasteur’s well-known principle of oxygen attenuation.

When Fleming noticed the contaminated plate, however, he immediately abandoned this rather routine bit of research, and devoted his energies to a quite new research program, namely that of investigating the mold which had contaminated his Petri dish, and which was apparently producing a bacterial inhibitor which acted on staphylococcus aureus. As we have already mentioned, Fleming’s position in the laboratory meant that he was quite free to pursue whatever research he liked. Suppose, however, that he had been obliged to obtain funding for his research through a funding scheme similar to AHRB (2001). For his first, rather routine piece of research into color change in staphylococci, he would have had to fill in a complicated application form and find two assessors to support him. Both Fleming and his assessors would have had to spend some time on this form-filling, which would have been time deducted from their research. Fleming’s application would then have been sent to a set of peer reviewers who would have had to take time away from their own research in order to review it. However, let us suppose that Fleming was finally awarded the research funds and began his research into color changes into staphylococci. Now, however, Fleming is financed by a research grant like AHRB (2001), which, as we have seen, imposes on researchers the strategy of strict adherence to earlier detailed research plans by threatening heavy penalties to anyone who fails to follow this strategy. If Fleming had been pressurized by such a grant-giving body to follow their directives, he would have continued his routine normal science investigation of color changes in staphylococci, and failed to discover penicillin.

The case of Perkin is of course just the same as that of Fleming. Perkin was given the task of trying to synthesize quinine by his professor (von Hofmann). However on the appearance of a black residue which stained water and alcohol purple, Perkin decided to change his research program to that of trying to synthesize a purple dye. Perkin was worried that von Hofmann would not approve of this plan, but luckily von Hofmann was away in Germany, and so Perkin went ahead with his plans, carrying them out secretly in a hut in his garden. But suppose Perkin had been financed by a research grant like AHRB (2001). His change of plan would then have risked the heavy penalties which, as we have seen, that scheme imposes on any researcher who does not carry out his original plan. As we remarked these penalties are particularly harsh for a young researcher whose whole research career might be ended by their imposition. Thus in an AHRB (2001) regime, Perkin might not have dared to change his research plans and so would have failed to discover the first artificial dye.

What is so remarkable here is that a leading UK research funding body should devise a research funding scheme based on principles which would probably have prevented two of Britain’s most famous scientists making their most famous discoveries. This clearly shows that the role of serendipity and chance in scientific discovery has very important policy implications, and that at least some members of important funding bodies are quite unaware of these implications. Still, it could be that AHRB (2001) is just one isolated case. I will therefore, in the next section, discuss the problem in more general terms in the context of contemporary global science.

The Effects of Globalization on Research

Globalization has transformed the world economy, and it has similarly transformed the field of research. In the late 1960s and early 1970s, research groups were characteristically located in a single country, and often within a single university in that country. Nowadays, however, members of the same research group are typically scattered throughout the world. They can of course still communicate on a day-to-day basis by email, and meet regularly at international conferences. Virtual presence via Skype at conferences is now becoming more and more common. The communication possibilities of modern technology have been rendered effective by the adoption of a common language for research (English).

Another factor which favors the globalization of research is the increasingly easy availability of information via the Internet. In the late 1960s and early 1970s, it was very difficult for anyone to do research who did not live near a large research library, containing the necessary papers and books. This tended to concentrate research groups in a few major universities. Now, however, researchers with an Internet connection and an iPad can download nearly all the papers they need for their research, and even much of the data they require. There are still some blocks caused by attempts to preserve copyright legislation, and government secrecy, but it looks as if even these blocks will be removed in the coming years. So we already have a research community dispersed throughout the globe, but yet in constant communication with each other; and this community is likely to extend and become more and more the basis of research in the future.

Because it is based on the advances of contemporary technology, globalization is well grounded, and much more likely to increase than diminish. This does not mean, however, that we have to accept all the features of contemporary globalization as inevitable and fully determined by technology. There are both bad and good features of contemporary globalization, and, it may be perfectly possible to develop globalization in such a way that the good features are strengthened, while the bad features are eliminated or at least minimized.

In general we can say that, paradoxically, two exactly opposite tendencies can be detected in contemporary globalization. One is a good tendency and the other a bad tendency. The good tendency is toward a lively and beneficial variety. The bad tendency is toward a dull and harmful uniformity. These two tendencies can be illustrated by the example of food production and consumption under globalization. The tendency toward a lively and beneficial variety is clearly marked. In the past, the members of a particular nation would, in general, eat only dishes of the national cuisine, whatever that was, and would be completely unfamiliar with the cuisines of other countries. Nowadays, in the big cities at least, most people can enjoy a variety of different national cuisines in restaurants, and obtain the ingredients needed to make these dishes in their local supermarket. This is surely an example of a lively and beneficial variety.

However, there is also a tendency in the opposite direction through the increasing prevalence of fast food consisting of a limited number of dishes (the same throughout the world) delivered by a limited number of multinational chains. This is dull uniformity, and also harmful. Standard fast food is well known to be harmful to the health because of its high content of sugar, salt, and animal fats, and the large-scale production of fast food is also damaging to the environment.² The increasing international diffusion of standard fast food is a consequence of globalization, but not an inevitable one. It is a feature of contemporary globalization which could be eliminated.

Research production would seem to be very different from food production, and yet, strange to say, the two tendencies at work in the globalization of food also seem to be at work in the globalization of research. Let us start with the good tendency. The increasing number of researchers, and their wide diffusion throughout the globe, makes it possible for a considerable number of different research programs to be pursued in any field. Particularly in the light of serendipity and chance, the pursuit of a large number of different approaches to any given problem makes it more likely that significant scientific discoveries will be made. This was realized by Bacon as early as 1620, when he wrote: “For then only will men begin to know their strength when instead of great numbers doing all the same things, one shall take charge of one thing and another of another” (Bacon 1620: 293). Globalization of research could potentially encourage the pursuit of a considerable number of different research programs in any field, and this would certainly be a lively and beneficial variety.

Unfortunately the opposite tendency toward a dull and harmful uniformity also exists in globalized research. This occurs when a small group (the trendsetters) start a particular research program, and everyone else simply imitates them by working on the same research program. The situation becomes one of “great numbers doing all the same things.” Possibly the research program which sets the trend does lead to some good results, but these would have been discovered anyway, even if pursuit of the program had been confined to a relatively small group of researchers. What is lost is the valuable results, arising directly or through serendipity and chance, which might have been produced by some groups pursuing a variety of alternative research programs.

But why does the tendency to a dull and harmful uniformity arise in globalized research? I will next argue that there are two factors principally responsible for this tendency, namely (1) the assignment of research funding by a competition between projects which is decided using anonymous peer review (I will refer to this, for brevity as “assignment by peer review”), and (2) the establishment of an alleged hierarchy of excellence among research groups, journals, and so on, and the use of this alleged hierarchy to evaluate research. I will now show that these two factors lead, in any specific area of research, to the predominance of one particular research program, and the elimination of alternatives, that is, to the exact opposite of what is desirable.

Let us start then by considering the assignment of funding by anonymous peer review. It will be useful to see how this would have affected the research in the four striking cases of serendipity and chance which we discussed earlier. Now Columbus had to raise funding for his expedition which resulted in the discovery of America. He applied first to the King of Portugal who turned him down. Davis (1973: 13) explains why as follows:

Columbus failed to get Portuguese support because the king could muster advisers with a sound knowledge of world geography. The plans of Columbus were based on nonsensical geographical notions that grossly understated the distance westward from Europe to Japan, which was his intended destination, and the Portuguese experts exposed them.

It is perhaps unfair to describe Columbus’s geographical notions as “nonsensical.” Columbus based his estimate of the circumference of the Earth on the writings of Aristotle and Ptolemy which were generally accepted at the time. The Aristotelian-Ptolemaic paradigm in astronomy was only overthrown by the Copernican revolution which started in 1543, 51 years after Columbus’s first voyage of 1492. However, Columbus modified Ptolemy’s geography in the light of Marco Polo’s travels. These had shown the extent of Cathay (China), and the existence of an island Zipangu (Japan) off the coast of China. Columbus calculated that Zipangu must be 4000 miles to the west of Portugal. This estimate agreed with a map produced by Toscanelli in 1474, and with Behaim’s globe of 1492. Columbus reckoned that it would take him about 33 days to sail from the Azores to Zipangu, and, ironically, he sighted land after 35 days of sailing.

But although Columbus’s geography was not nonsensical, the Portuguese experts knew better. This is a most instructive example because the experts who criticized Columbus were indeed correct. Yet Columbus’s expedition was worthwhile all the same because of serendipity. This shows clearly that, once we take account of the important role of serendipity and chance in scientific discovery, then experts are no longer able to judge whether a proposed research project will prove fruitful or not.

The case of Perkin is also interesting from this point of view. The original project was to synthesize quinine. In fact we now know that this is a very difficult task, which could not have been carried out at the time (1856). Synthetic quinine has never been produced industrially as a substitute for naturally occurring quinine, and there has even been a debate about when the first laboratory synthesis of quinine occurred. One side argues for 1944, and the other for 2001. In the light of all this, experts in 1856 might well have argued correctly that the project for synthesizing quinine was not feasible with the methods available at the time, and so should not be funded. However, if the project had not been carried out, the first artificial dye would not have been discovered.

The other two cases are rather different. They were both rather routine pieces of research which might well have been refused funding on the grounds that they would not produce any very new or exciting results. Fleming’s research was into a possible instance of the well-established principle of oxygen attenuation of pathogenic bacteria. Röntgen set out to discover whether cathode rays penetrated glass. A contemporary expert might very reasonably have argued that this was rather a boring project, and unlikely to produce any result of interest. After all, at that time (1895), cathode rays had been studied for over 20 years by means of cathode ray tubes. The expertise acquired made it unlikely that cathode rays would penetrate the glass walls of the tubes used, and, even if some did, this could almost certainly be corrected by making the glass wall a bit thicker. Yet this seemingly dull and routine research project yielded one of the most striking discoveries ever made in physics.

Let us now return to the question of whether the policy of assigning funds to research projects assessed by anonymous peer review is a good one. How likely is it that peer reviewers, even if they are experts in the field, will deal well with the very hard problem of assessing the potential of research projects? My own view is that they are not likely to make a very good job of it, because considerations of serendipity and chance show that no one really knows in advance what research projects are going to succeed. In particular, peer reviewers, even if experts in the field, do not really know which research projects are likely to succeed, but they do often have quite strong prejudices about the matter. These prejudices, taken in aggregate, lead, as I will now show, to sub-optimal decision-making.

The root of the problem is what I will call researcher narcissism. This is a condition which affects nearly all researchers (including the author of the present paper). It consists in an individual researcher believing quite strongly that his or her approach to research in the field is the best one, and the one most likely to produce good results; while the other approaches are less good and less likely to produce any good results. The existence of researcher narcissism is not surprising. Most researchers will spend some time thinking about which approach to adopt to their research, and, when they opt for a particular approach, there will be reasons for their decision. Moreover, once having made the decision, it tends to get reinforced by the fact that they mix a great deal with others working along similar lines, all of whom are convinced that what they are doing is right. In addition, any researcher has a strong interest in his or her approach proving to be the successful one. If it is successful, the researchers working on it will be more likely to get promotions and perhaps prizes. Their PhD students are more likely to get jobs, and so on. Conversely, if their approach to research proves unsuccessful, all these desirable consequences are much less likely to follow, and they are likely to see the rewards going to those working on a rival research program. Now it is a universal characteristic of humans to believe what is in their interest, and, in the present case, this amounts to the belief of researchers that they have adopted the best approach to research in the subject.

Let us assume therefore that we have community of peer reviewers who are all researchers who suffer from researcher narcissism. How will this affect the way in which research projects are selected for funding by peer review? Because of researcher narcissism, a peer reviewer is likely to take a favorable view of a research project which adopts the same general approach to research as that of the peer reviewer, and an unfavorable view of a research project which adopts a different approach. Now let us suppose that peer reviewers are chosen more or less at random. Very few of these peer reviewers will, by definition, be working on a minority research program. If the research project adopts the approach of this minority research program, therefore, it is likely to be viewed with disfavor by the majority of the peer reviewers and so won’t be funded. Conversely a research project which adopts a majority or mainstream approach is much more likely to be funded. Hence, funding based on assessment of research projects by peer review will lead to most of the funding going to whatever is the majority research program, and minority research programs receiving little or no funding. Thus the system results in the dull and harmful uniformity, which was mentioned earlier.

The tendency is reinforced if attention is paid to the alleged hierarchies of excellence in research. The existence of such hierarchies is hardly in doubt since league tables for institutions, journals, and so on are published regularly. The problem, however, with laying too much weight on such tables is that, even if they are entirely correct at present, which is much to be doubted, it by no means follows that they will continue to be in the future. Often approaches to research are very fruitful for a while, but are then superseded by new approaches. Newton may have been the most brilliant mathematician in the world in the 1670s, but in the eighteenth century Newton’s approach to the calculus was definitely superseded by the approach developed in continental Europe. Those who stuck to the Newtonian approach because of its prestige were onto a loser.

Let us consider a country (X say) whose research groups are relatively low down in the alleged hierarchies of research excellence. The government of X will now ask: “How can we get our researchers to improve?” The usual answer is the following. A group of American universities are at the top of the league at the moment. We must therefore get our researchers to imitate them. In particular we must rate very highly and reward those of our researchers who succeed in publishing in those journals controlled by these top universities. Now these journals will select papers by anonymous peer review, and so to get a publication in them, it will be necessary to participate in the research program supported by that journal, that is, the research program supported by the group at the top of the league. Hence, such research evaluations, which are very common, will result in the researchers of X imitating the research which is being done in the groups at the top of the league. This again reinforces the tendency to a dull and harmful uniformity. Of course, the considerations of serendipity and chance show that common government policies of the type we have just described are quite wrong, and the researchers of X would be better advised to pursue research programs which are different from those being followed by groups who are currently at the top of the league. There could indeed be intellectual and cultural traditions specific to X, which might suggest approaches different from those of researchers at the top of the league, and these different approaches could very well lead to interesting results. Yet instead of building on valuable local traditions, governments often just encourage imitation of whatever those currently at the top of the league are doing.

Having pointed out some features of present-day research organization which push in the undesirable direction of a dull and harmful uniformity, the question naturally arises of whether there are alternative policies which could be introduced and which would encourage lively and beneficial variety instead. Now this is obviously a complicated problem, which cannot be dealt with at an appropriate length in the present chapter. However, a couple of preliminary suggestions can be made. The first suggestion is that there should be a shift away from attempting to make any immediate evaluation of a researcher’s work. The history of science shows over and over again that developments which in the long run are seen as major advances can be judged initially by contemporary researchers to be valueless. I give three examples of this typical phenomenon in Gillies (2008: 14–27). Very often 20 or 30 years need to elapse before a sound conclusion as to the value of a piece of research can be reached. Of course it is perfectly reasonable to check that researchers are working, but this can easily be done by counting their publications. The mistake is to evaluate a publication in a high-status journal as being of higher quality than one in a low-status journal. It could well be that in 20 or 30 years, the publication in the low-status journal will be considered as the one which really made the important breakthrough. This policy would allow local traditions with their journals to flourish, adding to the variety of the system.

The second suggestion is to set aside some funds to encourage the formation of international research teams. If we suppose that the members of such a team receive their main funding from their home university or research institute, the extra funding would be needed only to pay for communication, through travel and workshops, and for the publication of results. These items are not very costly. The formation of such international networks could be very helpful to researchers who are trying to develop minority approaches. In each individual country, a minority approach is liable to be crushed, but, if there are minorities who adopt the approach in many countries, together they can form an international group of researchers of a viable size. In this way a minority approach, which would be liable to disappear in every individual country, could flourish at the global level.

Conclusions

In this paper I have argued that the role of serendipity and chance in scientific and technological discovery does have important policy implications. It shows that researchers should be encouraged to have the flexibility to change their research plans in the light of unexpected developments, and that in every area of research a variety of different approaches and research programs should be encouraged. Existing systems of research organization, and, in particular, the method of assigning research funding on the basis of anonymous peer review, have exactly the opposite effect. These contemporary systems of research organization are therefore in need of considerable changes. A couple of suggestions have been made for changes, which could prove beneficial, but more study of this problem is clearly needed.

References

1. AHRB. 2001. “Guide to the Research Leave Scheme.” Arts and Humanities Research Board.
2. Anguera de Sojo, A., J. Ares, M.A. Martínez, J. Pazos, S. Rodríguez, and J.G. Zato. 2014. “Serendipity and the Discovery of DNA.” Foundations of Science 19: 387–401.
3. Bacon, F. 1620. Novum Organum. English translation in The Philosophical Works of Francis Bacon, ed. R.L. Ellis and J. Spedding, 212–387. London: Routledge, 1905.
4. Davis, R. 1973. The Rise of the Atlantic Economies. London: Weidenfeld and Nicolson.
5. Fleming, A. 1944. “The Discovery of Penicillin.” British Medical Bulletin 2: 4–5.
6. Gillies, D. 2008. How Should Research Be Organised? London: College Publications.
7. Kantorovich, A., and Y. Ne’eman. 1989. “Serendipity as a Source of Evolutionary Progress in Science.” Studies in the History and Philosophy of Science 20(4): 505–529.
8. Kirk, G.S., and J.E. Raven. 1957. The Presocratic Philosophers. Cambridge: Cambridge University Press.
9. Merton, R.K., and Barber, E. 2004. The Travels and Adventures of Serendipity: A Study in Sociological Semantics and the Sociology of Science. Princeton, NJ: Princeton University Press.
10. Norrby, E. 2010. Nobel Prizes and Life Sciences. Singapore: World Scientific.
11. Pollan, M. 2006. The Omnivore’s Dilemma: The Search for a Perfect Meal in a Fast-Food World. London: Bloomsbury.
12. Roberts, R.M. 1989. Serendipity: Accidental Discoveries in Science. Hoboken, NJ: John Wiley & Sons, Inc.
13. Schlosser, E. 2001. Fast Food Nation: What the All-American Meal Is Doing to the World. London: Penguin.
14. van Andel, P. 1994. “Anatomy of the Unsought Finding: Serendipity: Origin, History, Domains, Traditions, Appearances, Patterns and Programmability.” British Journal for the Philosophy of Science 45(2): 631–648.