4.1.3.1 Tags or Keywords?

It could be said that tags are reminiscent of the social bookmarking sites, where they were key elements in the thematic structuring of content. From a certain viewpoint, they represent traces of the several loans that Mendeley makes from previous academic social sites to configure its web service. However, the role of tags in Mendeley is not so important as in CiteUlike and BibSonomy. In Mendeley, documents are classified beforehand following a structural scheme, and the tags are used simply as navigational tools that broaden the retrieval possibilities of the system. In addition, many of these tags are automatically added from the author’s keywords, so analysis of these labels would result in confusion as it is not easy to distinguish between users’ tags and authors’ keywords. A recent analysis found 6.13 tags per document with a median of 3 in computer science articles (Pimasoli, 2014). These results describe an elevated ratio of tags, despite the majority of them being supplied by the author him or herself.

4.1.3.2 Profiles, the Students’ Network

Besides being a reference management tool, Mendeley also acts as a document sharing site. This means that each user can upload their own publications to be read and downloaded by the rest of members. However, this functionality causes some confusion because many users consider that section to be a personal library as well, and they post no authored papers. It is not possible to know how often this is the case, but it could influence further comparisons with other document sharing sites. Each profile contains identification data such as affiliation, location and discipline. This last is mandatory when a profile is created, as well as the academic position. In addition, the profile shows several statistics that describe the performance of the profile in the network. Thus the number of publications is the number of documents uploaded to the profile, while ‘Readers’ refers the number of users that have included some paper from that profile in their libraries. Other indicators (followers/followings) describe social interactions with other users following the updates of their profiles with new papers and other activities. Each profile is informed of this activity through a Dashboard, where he or she is alerted to the recent activity of their followers and groups.

Figure 4.3 shows the evolution of the number of users in Mendeley since 2009. The WayBack Machine of Archive.org and the Mendeley blog were used to extract the more approximate figures. The addition of users to Mendeley follows a linear trend with an average annual growth (CAGR = 63 per cent) since 2009. The incorporation of new profiles is similar each year, while the addition of new users following a pause in the rate may be noted, and the 2011–13 period shows a mild deceleration. In this sense, although Mendeley increases its members each year, the rate is rather slow and this could be due to signs of exhaustion.

A more detailed analysis of Mendeley’s profiles was carried out to study the disciplinary, geographic, institutional and academic post distribution of its users. Mendeley has a search interface to find other colleagues and follow them. However, this mechanism is rather rudimentary and only retrieves profiles with an exact match and is lacking in wildcards or other types of operators. In addition, it only indexes the names of the profiles, so it is not possible to obtain profiles by affiliation, location, discipline, etc. Thus, to extract a representative and random sample of profiles, more than 3,000 of the most frequent names in the world were selected to query the system. In total, 1,029,998 profiles were retrieved from Mendeley in May 2015, a very large sample that could well be a quarter of the total number of profiles created in the system (four million approximately) (Mendeley blog, 2015).

From a disciplinary view, the distribution of profiles and their publication in Mendeley is quite similar to the distribution of papers and groups. This brings together the document coverage and the users’ population and suggests that the taking up of these sites is affected by the contents that are generated in them. Figure 4.4 shows that the network is mainly populated by users from Biological Sciences (16 per cent), Medicine (12 per cent) and Computer and Information Science (10 per cent), a distribution very similar to the distribution of papers in the catalogue. According to the publications of these profiles, Biological Sciences (20 per cent) is still the research area that posts most papers, followed by Medicine (13 per cent) and Computer and Information Science (13 per cent). This is influenced by the number of profiles, which determines the number of publications. However, the rate of publications per profile presents a different figure. Now, it is Physics (2.69) that is the discipline posting the most publications host, along with Material Science (2.05) and Earth Sciences (1.8). This proportion is closer to the usual average of papers by disciplines observed in other academic social sites (Ortega, 2015) and scientific databases (Ioannidis, Boyack, & Klavans, 2014; Ruiz-Castillo and Costas, 2014).

From the sample, only 141,895 (13.7 per cent) users include publications in their profiles, which demonstrates that the possibility to share documents in Mendeley is not entirely utilized by all users. As was said above, Mendeley is somewhat confusing according to the publications that are listed in the profiles, because some users include not authored papers by themselves, thinking that it is an extension of their personal library. Figure 4.5 plots the number of users against the number of papers. The distribution shows a skewed trend (γ=2.61) in which a large proportion of users publish a small number of papers, while only a small group of members post a large number of publications. Thus, only 20 per cent of users upload more than ten papers and just 152 profiles post 500 papers, the limit for uploading papers in each profile. The average number of papers is 24.9 per user.

Similar behaviour describes the distribution of followers by profiles, although the exponent (γ=3.33) in this case is more pronounced (see Figure 4.6). This means that the imbalance in the number of followers is even more so among the users of Mendeley. Thus 692,275 (67 per cent) profiles do not have any followers and from those remaining, 94 per cent have less than ten followers. However, only two profiles have more than 1,000 followers. The average of followers in Mendeley is around 1. This indicator of social networking informs us that the Mendeley network is still sparse and only a small number of users are following the careers of other members. This could be due to several facts. The first is that this tool was only recently introduced (Mendeley blog, 2014), and it is possible that many users are not yet using this method of detecting profile updates. Another reason could be related to the availability of this site as a document sharing service and, as has been seen before, this function is not widely used. Therefore, if there are many profiles with no papers or with just a couple, it is logical to conclude that most of the users do not use it to follow other profiles in the network.

Mendeley also makes it compulsory to enter the academic status of its users. Although this information is not present in the profiles, it is indeed visible in the page results when profiles are searched. Following the procedure above, 1.6 million academic statuses were collected from the retrieved profiles. Figure 4.7 shows the distribution of profiles by academic status. The most frequent categories are Student (Bachelor) (21.1 per cent), Student (Master) (20.7 per cent) and Ph.D. Student (15.7 per cent). It is evident that Mendeley is mainly taken up by students (57.5 per cent) that are starting their academic careers. Many of them are involved in doctoral dissertations which demand bibliographic tools such as Mendeley to obtain in-depth knowledge of the current literature in their respective research areas. This distribution, biased towards young and starting scholars, could affect the ‘Readers’ indicator, as noted above. In this way, the likelihood that a paper is read by a student is more than 50 per cent, so it is not surprising that this type of scholar is reading the most documents in Mendeley (Mohammadi, Thelwall, Haustein, & Larivière, 2015b) and that this platform is preferred by students (Jordan, 2014).

From the sample, only 85,713 (8 per cent) users included their affiliation in their profiles, so the institutional representation could be biased and not representative of the entire population in Mendeley. Even so, the organizations with the most profiles are the Universidade de São Paulo (0.56 per cent), the Universidad Nacional Autónoma de México (0.45 per cent) and the University of Cambridge (0.41 per cent), that is large universities in terms of the number of academic staff (see Table 4.4). It is also interesting to note that among the first ten institutions there are four British universities so it is possible that this service, which is headquartered in London, is preferred by British academics. According to the number of publications, the French CNRS (0.44 per cent) and the British University of Oxford (0.21 per cent) and University of Cambridge (0.2 per cent) are the institutions with the most papers posted in Mendeley.

Exploring the distribution by country, 144,721 (14 per cent) users included location in their profiles, a proportion rather high in comparison with sites analysed previously (see Table 4.5). Nevertheless, this proportion casts serious doubt on the readers’ statistics at country level as they may not be representative enough (Haunschild, Bornmann, & Leydesdorff, 2015). The countries with the most profiles in the network are the United States (22.5 per cent), followed far behind by the United Kingdom (9.7 per cent) and Brazil (6.7 per cent). With regard to publications, the United States (9.4 per cent) and the United Kingdom (5.1 per cent) are still the countries with highest number of contributions, followed by Spain (3.8 per cent) and Germany (3.4 per cent). According to the degree of penetration, Mexico (3.3) and the two principal Portuguese speaking countries, Portugal (3.3) and Brazil (3.3), are the countries where Mendeley is most successful, followed by the United Kingdom (2.6), the country where this platform is located. The Country Spreading, the percentage of users among the first ten countries, is 63 per cent, the lowest percentage of the sites analysed and shows that this service has reached global diffusion.

4.1.3.3 Groups, the Readers’ Club

The third instrument of social participation in Mendeley is groups. These aim to gather together different users to compile specific bibliographies on determined issues or to gather working materials for projects or research groups. In addition to adding references, members’ groups can also discuss, rate and comment on these references or another issue.

Table 4.6 and Figure 4.8 show the distribution of groups and the number of papers included in these groups by research discipline. In April 2015, Mendeley counted 97,857 (138,387 by discipline) public groups that might be assigned to several disciplines. The private groups are only visible to their members, so the number of private groups is unknown. Since 2011, the growth of groups has been exponential, with an annual increase of CAGR =74 per cent (Jeng, He, Jiang, & Zhang, 2012). The thematic distribution of public groups is similar to the number of documents in the public catalogue. Thus the disciplines with the most groups are Biological Sciences (17.3 per cent), Medicine (12.9 per cent) and Computer and Information Science (12.7 per cent). According to the number of papers, 4.6 (5.7 by discipline) million documents were posted to these groups. This means that 31.3 per cent of the papers in the catalogue were shared in groups, which is an appreciable amount. No disciplinary differences exist when it comes to posting papers to the groups, so Biological Sciences (23.6 per cent) is the discipline with most posted papers, followed by Medicine (12.8 per cent) and Computer and Information Science (9.7 per cent). However, there are indeed differences in papers by group rate. The most active disciplines uploading papers are Environmental Sciences (60.1 papers by group), Linguistics (59.5 papers by group) and Biological Sciences (56.5 papers by group). Finally, the number of members that joined up to a group is 275,266, approximately 6.8 per cent of the global community of users (Mendeley blog, 2015), a proportion a little above the 5.7 per cent found by Oh and Jeng (2011). This proportion of users interested in groups is the lowest percentage observed and shows that this functionality is not attractive for most of the users.

If we look at the characteristics of the groups themselves, Table 4.7 shows the ten most active groups. An average of 73 papers is included in each set, a rate lower than that observed in CiteUlike (106.3) and BibSonomy (87.3). In addition, the number of members by group (3.1) is rather low, being more than half of the groups (52 per cent) formed by only one member. These values are lower than the figures found by Oh and Jeng (2011), which suggests that participation in groups could increase with time. All these indicators lead us to think that the use of public groups in order to share references is not very frequent and would not attract the attention of many users. As with the case of social bookmarking tools, the typology of groups in Mendeley is quite varied and among the most active groups can be found journal sites (i.e. Zootaxa, the European Journal of Entomology) that upload their publications to be discussed and read by their members, projects that attempt to create a comprehensive database of references on a specific theme (i.e. Ocean Acidification (OA-ICC), Amphibian Species of the World) or simply places where the most recent updates and news on a research line may be found (i.e. Translation Studies, POLYMER_BASED NANOCOMPOSITES).

Figure 4.9 allows us to observe the evolution of the activity of these groups over the years. The accumulated line describes an ascending linear trend while the bars reveal a pause in growth. Only 38 per cent of the groups have developed some activity (i.e. posting a paper, inserting a comment) since 2014. This growing pattern provides evidences that although the number of groups is small and the activity is not much higher, a considerable proportion of groups do have a current activity. However, this does not hide the fact that 62 per cent of the groups are losing activity and freshness.

4.2.1.1 Profiles, Just a Calling Card

Profiles in Zotero only contain basic information to identify each user, with data on affiliation, location, followers/followings and disciplines. It also includes links to access users’ libraries and make contact possible between them through internal messages. However, the access to these personal libraries is optional and only a small number of users keep their libraries open source (4 per cent). This makes it impossible to know how many documents are posted in the system. In total, it is estimated that Zotero has no more than 130,000 registered users in May 2015, a figure quite low in relation to other academic sites. This estimation was obtained by extracting the list of profiles from the main page in an iterative process until only duplicated profiles may be extracted.

Zotero is one of the sites that is the most harmful to spam profiles. Some social sites are utilised to insert advertisements or links to increase the visibility of certain websites, improving the ranking of those websites fraudulently when they are searched, for example, in Google. In the case of Zotero, 2,787 (2.1 per cent) profiles were identified as fake users, whose sole purpose was to include links to, for example, online pharmacy stores, real-estate advertisings and escort offers. These users do not carry out any action in the system, but simply make use of the high PageRank of the site to promote their businesses. This is a serious problem because it distorts the normal functioning of the network and affects the statistical analysis of this site.

Not all the profiles are assigned to a discipline and then only 88,455 (68 per cent) profiles were classified using Zotero’s own scheme. This system is based on the classification of academic disciplines published in Wikipedia (2015) to thematically organize a user’s profile. The scheme covers all the academic disciplines and is arranged in five main classes. However, this system has several weaknesses. The first is that there are complaints of constant and arbitrary modifications with the removing and renaming of disciplines and the allocation of new categories which produce serious instability. Another specific problem is that it divides academic disciplines and professions, splitting related categories into separate parts. In addition, some disciplines are misclassified such as Archeology and Anthropology which are included in Social Sciences. Perhaps due to this, Zotero only takes the main categories without any hierarchical relationship. In any event, this scheme contains five principal classes that make possible the grouping of Zotero’s categories. These classes have been re-adapted to make possible further comparison with other social sites.

Figure 4.10 shows the percentage of users by the six principal research areas. As we have been seen before, only 68 per cent of users fill in this information in their profiles. The figure shows that Zotero is unbalanced towards social scientists (33.8 per cent) and humanists (27.6 per cent), whereas physicians (6.1 per cent) and natural scientists (i.e. chemists, biologists) (11.4 per cent) present percentages that are way below their usual representation in other social sites and databases. Therefore, it can be assumed that this population is not thematically homogeneous, being biased towards Human and Social Sciences content.

According to the distribution of profiles by organization, Zotero describes a similar pattern to that observed with BibSonomy (see Table 4.8). The university with most profiles is the George Mason University (0.9 per cent), the place where this software was created. Walden University (0.7 per cent), a small university in Minnesota, comes second in terms of the number of profiles a surprising fact that could be explained by the strong support that this university’s library offers to Zotero, publishing an online user guide (Walden University, 2015). As a result of this the students and professors massively use this platform. The remaining universities are common organizations in these distributions, such as the Universidad Nacional Autónoma de México (0.58 per cent) and the University of Washington (0.5 per cent).

Grouping profiles in terms of countries, Table 4.9 confirms the massive presence of American users – to half of the users come from the United States (43.5 per cent), followed far behind by the United Kingdom (6.2 per cent) and Canada (3.8 per cent). In addition, the United States is the country with the highest penetration index (2.35 per cent), which confirms the local nature of this service. Australia (2.32 per cent) is the second country according the penetration index, suggesting that this utility has a mean spread across Anglo-Saxon countries. However, it is necessary to take into account that these countries are the most infected by spam users, so these results could be distorted by that malicious practice. With regard to the Country Spreading index, Zotero is the second web space with the highest concentration of population in the first ten countries (69.3 per cent). This confirms the local nature of the site and its weak diffusion into other parts of the world.

Zotero allows users to follow the posting activity of other profiles, promoting the interaction and sharing of references between them. In this way, it is possible to know the number of followers that a profile has and the amount of profiles that are followed. In addition, this number could be considered an indicator of the network’s degree of connection and could be a measure of the strength with which the users interact among them. Figure 4.11 shows the distribution of followings and followers by profile. Because the sample is reduced, data were not binned. Both describe a very similar trend, with a slight higher slope for followings (γ=2.13) than for followers (γ=2.16). However, the proportion of profiles that do not follow anybody is 93 per cent and 89 per cent who do not have any followers. The average of followers is 2.2 while the average of followings is 3.5. These figures show that the Zotero network is quite sparse and only a small fraction of profiles have connections between them. This unrelated network could be explained because users’ libraries are not open in all cases and it is hard to follow a profile that just contains identification data. Another cause, no less important, is the high presence of spammer profiles which show no interest in connecting with other users.

Figure 4.12 shows the evolution of the number of profiles in Zotero since 2009, the moment in which this reference management tool incorporated the creation of profiles by part of their users. These data from 2009 onwards have been obtained from Zotero’s main page using the WayBack Machine of Archive.org. Since that date, the number of profiles has been steadily increasing up to the present (CAGR=76 per cent). The accumulated trend describes a linear growth, with a pause in the increase. Despite the peak in 2011, the incorporation of new users has grown at a very slow rate.

4.2.1.2 Groups, the Only Way of Sharing

In the manner of Mendeley, Zotero allows any user to create groups that share his or her own library with other members. Groups can be private or public. The former are confidential and only visible to members, while the latter are open to any users and allow the exploration of their content. Zotero lists 31,784 public groups in May 2015, of which 64 per cent are open to new members while 35.6 per cent are closed to new admissions.

Table 4.10 presents the ten most important groups in Zotero according to activity. There are 106,597 (82 per cent) users enrolled in these groups, making this the social site in the study with the highest proportion of users registered in groups. However, this figure may not be as high because users can participate in several groups. Despite this, this high proportion would suggest that most of the users only feel attracted to Zotero for its networking possibilities and not so much for its individual functions. However, the users by group rate (3.3) is not that high, which could mean that many of these groups are small. In fact, 41 per cent of these groups have only one member, which could confirm that many individual users are creating groups to share their collections more easily. Observing the number of items posted, these groups in total share around 2.5 million documents, a considerable number in view of the number of users in the system. In line with this result, the average of uploaded references per group (105) also shows that the usage of these groups is high and they contain a substantial amount of documents. This perception is confirmed by the activity ratio (40.8), the second highest value of the sites analysed.

If we look at Table 4.10 in more detail, different groups for distinct purposes may be found. There are groups created for gathering references related to defined research issues such as the First World War (First World War Studies Bibliography) or Buddhism (H-Buddhism Bibliography Project). These groups arise with the objective of building exhaustive bibliographies that support the research in those fields. Other groups are created as bibliographic instruments for specific institutions in their scholarly activity (i.e. GEOgraphie urbaine, régionale et sociale-IGD, LCENVS). It is interesting to notice that half of the most active groups are devoted to Humanities, which confirms the strong presence of researchers from this field of study.

Figure 4.13 shows the increase in the number of new groups since the creation of this tool in 2009. The line shows a constant growth from that moment, but in a paused form. Thus, from 2009 to 2014, groups have grown 60 per cent each year in average. This increase is in relation to the previous results, confirming that this collaborative tool enjoys good health and suggesting that Zotero’s users are employing this instrument to interact with other users and the system in general.

Figure 4.14 illustrates another way to look at the positive working of this tool. It shows the number of groups by the year in which users posted their last item. As with the previous figure, the trend maintains a pattern of growth and many of the groups describe recent activity. Thus, 34.6 per cent of the groups posted their last items in the period 2014–15, a similar percentage to Mendeley.