GOOGLE SCHOLAR BIBLIOGRAPHY



2018 |2017 |2016 |2015 | 2014 | 2013 
| 2012 | 2011| 2010 | 2009 | 2008| 2007| 2006 | 2005

2018 

Delgado López-Cózar, E. , Orduna-Malea, E., Martín-Martín, A. (2018). Google Scholar as a data source for research assessment. In: Wolfgang Glaenzel, Henk Moed, Ulrich Schmoch, Michael Thelwall  (eds.). Springer Handbook of Science and Technology Indicators. Springers https://arxiv.org/abs/1806.04435   
The launch of Google Scholar (GS) marked the beginning of a revolution in the scientific information market. This search engine, unlike traditional databases, automatically indexes information from the academic web. Its ease of use, together with its wide coverage and fast indexing speed, have made it the first tool most scientists currently turn to when they need to carry out a literature search. Additionally, the fact that its search results were accompanied from the beginning by citation counts, as well as the later development of secondary products which leverage this citation data (such as Google Scholar Metrics and Google Scholar Citations), made many scientists wonder about its potential as a source of data for bibliometric analyses. The goal of this chapter is to lay the foundations for the use of GS as a supplementary source (and in some disciplines, arguably the best alternative) for scientific evaluation. First, we present a general overview of how GS works. Second, we present empirical evidences about its main characteristics (size, coverage, and growth rate). Third, we carry out a systematic analysis of the main limitations this search engine presents as a tool for the evaluation of scientific performance. Lastly, we discuss the main differences between GS and other more traditional bibliographic databases in light of the correlations found between their citation data. We conclude that Google Scholar presents a broader view of the academic world because it has brought to light a great amount of sources that were not previously visible.

Martín-Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. (2018). Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison. https://doi.org/10.1007/s11192-018-2820-9. https://arxiv.org/abs/1804.09479 
 This study explores the extent to which bibliometric indicators based on counts of highly-cited documents could be affected by the choice of data source. The initial hypothesis is that databases that rely on journal selection criteria for their document coverage may not necessarily provide an accurate representation of highly-cited documents across all subject areas, while inclusive databases, which give each document the chance to stand on its own merits, might be better suited to identify highly-cited documents. To test this hypothesis, an analysis of 2515 highly-cited documents published in 2006 that Google Scholar displays in its Classic Papers product is carried out at the level of broad subject categories, checking whether these documents are also covered in Web of Science and Scopus, and whether the citation counts offered by the different sources are similar. The results show that a large fraction of highly-cited documents in the Social Sciences and Humanities (8.6–28.2%) are invisible to Web of Science and Scopus. In the Natural, Life, and Health Sciences the proportion of missing highly-cited documents in Web of Science and Scopus is much lower. Furthermore, in all areas, Spearman correlation coefficients of citation counts in Google Scholar, as compared to Web of Science and Scopus citation counts, are remarkably strong (.83–.99). The main conclusion is that the data about highly-cited documents available in the inclusive database Google Scholar does indeed reveal significant coverage deficiencies in Web of Science and Scopus in several areas of research. Therefore, using these selective databases to compute bibliometric indicators based on counts of highly-cited documents might produce biased assessments in poorly covered areas.  

Martín-Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. (2018). Author-level metrics in the new academic profile platforms: The online behaviour of the Bibliometrics community. Journal of Informetrics, 12(2), 494-509. https://doi.org/10.1016/j.joi.2018.04.001 https://arxiv.org/abs/1804.05599 
The new web-based academic communication platforms do not only enable researchers to better advertise their academic outputs, making them more visible than ever before, but they also provide a wide supply of metrics to help authors better understand the impact their work is making. This study has three objectives: a) to analyse the uptake of some of the most popular platforms (Google Scholar Citations, ResearcherID, ResearchGate, Mendeley and Twitter) by a specific scientific community (bibliometrics, scientometrics, informetrics, webometrics, and altmetrics); b) to compare the metrics available from each platform; and c) to determine the meaning of all these new metrics. To do this, the data available in these platforms about a sample of 811 authors (researchers in bibliometrics for whom a public profile Google Scholar Citations was found) were extracted. A total of 31 metrics were analysed. The results show that a high number of the analysed researchers only had a profile in Google Scholar Citations (159), or only in Google Scholar Citations and ResearchGate (142). Lastly, we find two kinds of metrics of online impact. First, metrics related to connectivity (followers), and second, all metrics associated to academic impact. This second group can further be divided into usage metrics (reads, views), and citation metrics. The results suggest that Google Scholar Citations is the source that provides more comprehensive citation-related data, whereas Twitter stands out in connectivity-related metrics.

Martín-Martín, A., Delgado López-Cózar, E. (2018). Google Scholar’s citation graph: comprehensive, global… and inaccessible. Open citations seminar. Uppsala: Uppsala Universitet.  http://hdl.handle.net/10481/51153 
Google Scholar contains a vast wealth of citation data, but this data is not easily accessible. This slides present some of the main strengths and weaknesses of Google Scholar citation data, as compared to the citation databases that are most frequently used for bibliometric analyses (Web of Science and Scopus). It also describes our own workflow to extract and  work with Google Scholar data

Bramer, W. M., Rethlefsen, M. L., Kleijnen, J., & Franco, O. H. (2017). Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Systematic reviews, 6(1), 245. https://doi.org/10.1186/s13643-017-0644-y 
Background
Within systematic reviews, when searching for relevant references, it is advisable to use multiple databases. However, searching databases is laborious and time-consuming, as syntax of search strategies are database specific. We aimed to determine the optimal combination of databases needed to conduct efficient searches in systematic reviews and whether the current practice in published reviews is appropriate. While previous studies determined the coverage of databases, we analyzed the actual retrieval from the original searches for systematic reviews.
Methods
Since May 2013, the first author prospectively recorded results from systematic review searches that he performed at his institution. PubMed was used to identify systematic reviews published using our search strategy results. For each published systematic review, we extracted the references of the included studies. Using the prospectively recorded results and the studies included in the publications, we calculated recall, precision, and number needed to read for single databases and databases in combination. We assessed the frequency at which databases and combinations would achieve varying levels of recall (i.e., 95%). For a sample of 200 recently published systematic reviews, we calculated how many had used enough databases to ensure 95% recall.
Results
A total of 58 published systematic reviews were included, totaling 1746 relevant references identified by our database searches, while 84 included references had been retrieved by other search methods. Sixteen percent of the included references (291 articles) were only found in a single database; Embase produced the most unique references (n = 132). The combination of Embase, MEDLINE, Web of Science Core Collection, and Google Scholar performed best, achieving an overall recall of 98.3 and 100% recall in 72% of systematic reviews. We estimate that 60% of published systematic reviews do not retrieve 95% of all available relevant references as many fail to search important databases. Other specialized databases, such as CINAHL or PsycINFO, add unique references to some reviews where the topic of the review is related to the focus of the database.
Conclusions
Optimal searches in systematic reviews should search at least Embase, MEDLINE, Web of Science, and Google Scholar as a minimum requirement to guarantee adequate and efficient coverage.

Mikki, S., Ruwehy, H. A. A., Gjesdal, Ø. L., & Zygmuntowska, M. (2018). Filter bubbles in interdisciplinary research: a case study on climate and society. Library Hi Tech, in press. https://doi.org/10.1108/LHT-03-2017-0052
Introduction
The purpose of this paper is to compare the content of Web of Science (WoS) and Google Scholar (GS) by searching the interdisciplinary field of climate and ancient societies. The authors aim at analyzing the retrieved documents by open availability, received citations, co-authors and type of publication.
Methodology
The authors searched the services by a defined set of keyword. Data were retrieved and analyzed using a variety of bibliometric tools such as Publish or Perish, Sci2Tool and Gephi. In order to determine the proportion of open full texts based on the WoS result, the authors relocated the records in GS, using an off-campus internet connection.
Results
The authors found that the top 1,000 downloadable and analyzable GS items matched poorly with the items retrieved by WoS. Based on this approach (subject searching), the services appeared complementary rather than similar. Even though the first search results differ considerably by service, almost each single WoS title could be located in GS. Based on GS’s full text recognition, the authors found 74 percent of WoS items openly available and the citation median of these was twice as high as for documents behind paywalls.

Rovira, C., Guerrero-Solé, F.,  Codina, L. (2018). Received citations as a main SEO factor of Google Scholar results ranking. El profesional de la información, 27(3), 1699-2407.. https://doi.org/10.3145/epi.2018.may.09 
The aim of this article is to analyze the web positioning factors that can influence the order, by relevance, in Google Scholar and the subsequent evaluation of the importance of received citations in this ordering process. The methodology of reverse engineering was applied, in which a comparison was made between the Google Scholar ranking and another ranking consisting of only the number of citations received by documents. This investigation was conducted employing four types of searches without the use of keywords: by publication, year, author, and “cited by”. The results were matched in the four samples with correlation coefficients between the two highest rankings, which exceeded 0.9. The present study demonstrates more clearly than in previous research how citations are the most relevant off-page feature in the ranking of search results on Google Scholar. The other features have minimal influence. This information provides a solid basis for the academic search engine optimization (ASEO) discipline. We also developed a new analysis procedure for isolating off-page features that might be of practical use in forthcoming investigations.



2017 [Go back]

Bornmann, L., Butz, A., & Wohlrabe, K. (2017). What are the Top Five Journals in Economics? A New Meta–ranking. 
https://mpra.ub.uni-muenchen.de/id/eprint/79176 

We construct a meta–ranking of 277 economics journals based on 22 different rankings. The ranking incorporates bibliometric measures from four different databases (Web of Science, Scopus, Google Scholar and RePEc). We account for the different scaling of all bibliometric measures by standardizing each ranking score. We run a principal component analysis to assign weights to each ranking. In our meta–ranking the top five journals are given by: Quarterly Journal of Economics, Journal of Financial Economics, Journal of Economic Literature (JEL), Journal of Finance, and Econometrica. Additionally, leaving out the JEL as a survey journal and the finance journals in our top 10 we confirm the perceived top-5 journals in the economics profession.

Chen, Y., Ding, C., Hu, J., Chen, R., Hui, P., Fu, X. (2017). Building and Analyzing a Global Co-Authorship Network Using Google Scholar Data.WWW’17 Companion, April 3–7, 2017, Perth, Australia. DOI:http://dx.doi.org/10.1145/3041021.3053056
By publishing papers together, academic authors can form a co-authorship network, modeling the collaboration among them. This paper presents a data-driven study by crawling and analyzing the vast majority of author profiles of Google Scholar. We make the following major contributions: (1) We present a demographic analysis and get an informative overview of the authors from different aspects, such as the distribution of countries, scientific labels, and academic titles. (2) Based on the publication lists of crawled authors, we build a global co-authorship network with 402.39K authors to study the collaboration among authors. With the aid of social network analysis (SNA), we observe several unique features of this network. (3) We explore the relationship between the co-authorship network and citation metrics. We find a strong correlation between PageRank and h-index
Delgado López-Cózar, E. (2017). La investigación en Biblioteconomía y su impacto en la profesión ¿Puede ser Google Scholar una fuente de información adecuada para medirlo? Mi biblioteca: La revista del mundo bibliotecario 48, 10-11

Delgado López-Cózar, E. Orduña-Malea, E.,  Martín-Martín, A.,  Ayllón, J. M.(2017). 
Google Scholar: The big data bibliographic tool. En: Cantu-Ortiz, FJ. (ed.). Research Analytics: Boosting University Productivity and Competitiveness through Scientometrics. Boca Raton: CRC Press (Taylor & Francis Group), 2017 p.59-80 goo.gl/vvPf7g 
he launch of Google Scholar back in 2004 meant a revolution not only in the scientific information search market but also in research evaluation processes. Its dynamism, unparalleled coverage, and uncontrolled indexing make Google Scholar an unusual product, especially when compared to traditional bibliographic databases. Conceived primarily as a discovery tool for academic information, it presents a number of limitations as a bibliometric tool.  The main objective of this chapter is to show how Google Scholar operates and how its core database may be used for bibliometric purposes. To do this, the general features of the search engine (in terms of document typologies, disciplines, and coverage) are analyzed. Lastly, several bibliometric tools based on Google Scholar data, both official (Google Scholar Metrics, Google Scholar Citations) and some developed by third parties (H Index Scholar, Publishers Scholar Metrics, Proceedings Scholar Metrics, Journal Scholar Metrics, Scholar Mirrors), as well as software to collect and process data from this source (Publish or Perish, Scholarometer), are introduced, aiming to illustrate the potential bibliometric uses of this source.

Delgado López-Cózar, E.; Martín-Martín, A.; Orduña-Malea, E. (2017). Classic papers: déja vu, a step further in the bibliometric exploitation of Google Scholar. Granada: EC3 Working Papers, 24. https://arxiv.org/abs/1706.09258
After giving a brief overview of Eugene Garfield’s contributions to the issue of identifying and studying the most cited scientific articles, manifested in the creation of his Citation Classics, the main characteristics and features of Google Scholar’s new service -Classic Papers-, as well as its main strengths and weaknesses, are addressed. This product currently displays the most cited English-language original research articles by fields and published in 2006

Grigas, V., Juzéniené, S. (2017). 'Just Google it'-the scope of freely available information sources for doctoral thesis writing. Information Research, 22(1).
Introduction -  Recent developments in the field of scientific information resource provision lead us to the key research question, namely,what is the coverage of freely available information sources when writing doctoral theses, and whether the academic library can assume the leading role as a direct intermediator for information users. 
Method - Citation analysis of doctoral theses was conducted in the summer of 2015. A total of thirty-nine theses (with 6,998 references) defended at Vilnius University at the end of 2014 was selected (30 per cent of all defended theses). Theses were randomly chosen from different research fields: the humanities, social sciences, biomedical sciences, technological sciences, and physical sciences. 
Analysis - The research team was tasked with identifying whether certain resources could be found in the eCatalogue of an academic library, its subscribed databases, freely available online (through Google or Google Scholar), or whether the resources from the library's subscribed databases are identical to those which are freely available. The data gathering process included such resource categories as journal papers, printed and electronic books or book chapters, and other documents (legal reports, conference papers, newspaper articles, Websites, theses, etc.). 
Conclusions - Library collections and subscribed databases could cover up to 80 per cent of all information resources used in doctoral theses. Among the most significant findings to emerge from this study is the fact that on average more than half (57 per cent) of all utilised information resources were freely available or were accessed without library support. We may presume that the library as a direct intermediator for information users is potentially important and irreplaceable only in four out of ten attempts of PhD students to seek information.

Fagan, J.C. (2017). An evidence-based review of academic web search engines, 2014-2016: Implications for librarians’ practice and research agenda.  Information Technology and Libraries, 36, 2
Academic web search engines have become central to scholarly research. While the fitness of Google  Scholar for research purposes has been examined repeatedly, Microsoft Academic and Google Books  have not received much attention. Recent studies have much to tell us about Google Scholar’s  coverage of the sciences and its utility for evaluating researcher impact. But other aspects have been understudied, such as coverage of the arts and humanities, books, and non-Western, non-English  publications. User research has also tapered off. A small number of articles hint at the opportunity for  librarians to become expert advisors concerning scholarly communication made possible or enhanced by these platforms. This article seeks to summarize research concerning Google Scholar, Google Books, and Microsoft Academic from the past three years with a mind to informing practice and setting a research agenda. Selected literature from earlier time periods is included to illuminate key findings and to help shape the proposed research agenda, especially in understudied areas


Halevi, G., Moed, H., & Bar-Ilan, J. (2017). Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation. Review of the Literature. Journal of Informetrics, 11(3), 823-834
As Google Scholar (GS) gains more ground as free scholarly literature retrieval source it’s becoming important to understand its quality and reliability in terms of scope and content. Studies comparing GS to controlled databases such as Scopus, Web of Science (WOS) and others have been published almost since GS inception. These studies focus on its coverage, quality and ability to replace controlled databases as a source of reliable scientific literature. In addition, GS introduction of citations tracking and journal metrics have spurred a body of literature focusing on its ability to produce reliable metrics. In this article we aimed to review some studies in these areas in an effort to provide insights into GS ability to replace controlled databases in various subject areas. We reviewed 91 comparative articles from 2005 until 2016 which compared GS to various databases and especially Web of Science (WOS) and Scopus in an effort to determine whether GS can be used as a suitable source of scientific information and as a source of data for scientific evaluation. Our results show that GS has significantly expanded its coverage through the years which makes it a powerful database of scholarly literature. However, the quality of resources indexed and overall policy still remains known. Caution should be exercised when relying on GS for citations and metrics mainly because it can be easily manipulated and its indexing quality still remains a challenge

Gu, X., Blackmore, K. (2017). Characterisation of academic journals in the digital age. Scientometrics, 110(3),1333–1350 . http://dx.doi.org/10.1007/s11192-016-2219-4
Innovations in scholarly publishing have led to new possibilities for academic journals (e.g., open access), and provided scholars with a range of indicators that can be used to evaluate their characteristics and their impact. This study identifies and evaluates the journal characteristics reported in five databases: Ulrich’s Periodicals Directory (Ulrichs), Journal Citation Reports (JCR), SCImago Journal & Country Rank (SJR), Google Scholar Metrics (GS), and Cabell’s Periodical Directory (Cabells). It describes the 13 indicators (variables) that are available through these databases—scholarly impact, subject category, age, total articles, distribution medium, open access, peer review, acceptance rate, pricing, language, country, status, and issue frequency—and highlights the similarities and differences in the ways these indicators are defined and reported. The study also addresses the ways in which this kind of information can be used to better understand particular journals as well as the scholarly publishing system.




Marina, P., Fotini, P., Nikolaos, T. (2017). Evaluation of 50 Greek Science and Engineering University Departments using Google Scholar. arXiv preprint arXiv:1703.04478.
In this paper, the scientometric evaluation of faculty members of 50 Greek Science and Engineering University Departments is presented. 1978 academics were examined in total. The number of papers, citations, h-index and i10-index have been collected for each academic, department, school and university using Google Scholar and the citations analysis program Publish or Perish. Analysis of the collected data showed that departments of the same academic discipline are characterized by significant differences on the scientific outcome. In addition, in the majority of the evaluated departments a significant difference in h-index between academics who report scientific activity on the departments website and those who do not, was observed. Moreover, academics who earned their PhD title in the USA demonstrate higher indices in comparison to scholars who obtained their PhD title in Europe or in Greece. Finally, the correlation between the academic rank and the scholars h-index (or the number of their citations) is quite low in some departments, which, under specific circumstances, could be an indication of the lack of meritocracy.


Martín-Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. A novel method for depicting academic disciplines through Google Scholar Citations: The case of Bibliometrics. Scientometrics, in press. https://doi.org/10.1007/s11192-017-2587-4

This article describes a procedure to generate a snapshot of the structure of a specific scientific community and their outputs based on the information available in Google Scholar Citations (GSC). We call this method multifaceted analysis of disciplines through academic profiles (MADAP). The international community of researchers working in Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics was selected as a case study. The records of the top 1000 most cited documents by these authors according to GSC were manually processed to fill any missing information and deduplicate fields like the journal titles and book publishers. The results suggest that it is feasible to use GSC and the MADAP method to produce an accurate depiction of the community of researchers working in Bibliometrics (both specialists and occasional researchers) and their publication habits (main publication venues such as journals and book publishers). Additionally, the wide document coverage of Google Scholar (specially books and book chapters) enables more comprehensive analyses of the documents published in a specific discipline than were previously possible with other citation indexes, finally shedding light on what until now had been a blind spot in most citation analyses

Martín‐Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. (2017). Scholar Mirrors: Integrating evidence of impact from multiple sources into one platform to expedite researcher evaluation. STI 2017 Conference: Science, Technology and Innovation indicators. “Open indicators: innovation, participation and actor-based STI indicators”. Paris
This poster describes the creation of “Scholar Mirrors”, a prototype web application that aims to provide a quick but accurate representation of a scientific discipline by integrating data from multiple online platforms. We chose the discipline of Bibliometrics / Scientometrics as a case study. After carrying out a series of keywords searches in Google Scholar Citations (GSC) and Google Scholar (GS), 813 relevant researchers were identified. Researchers were further classified as core (those who work mainly on Scientometrics) or related (those who work in other disciplines, with occasional incursions into Scientometrics). Additional information about these researchers was collected from other platforms (ResearcherID, ResearchGate, Mendeley, and Twitter). Up to 28 author-level indicators were collected about each researcher, as well as data about up to 100 of the most cited documents displayed in their GSC profile. The document-level data from all GSC profiles, as well as the data extracted from the keyword searchers in GS, was aggregated to create a list of the top 1000 most cited documents in the discipline. This document collection was further processed to generate a list of the most influential journals and publishers in the discipline. The results are accessible from the “Scholar Mirrors” website, which presents the results in four sections: authors, documents, journals, and book publishers. Lastly, the poster presents the main features of the web application, and the main limitations and future challenges of the product.


Martín‐Martín, A., Orduna-Malea, E., Delgado López-Cózar, E. (2017). Journal Scholar Metrics: building an Arts, Humanities and Social Sciences journal ranking with Google Scholar data.  STI 2017 Conference: Science, Technology and Innovation indicators. “Open indicators: innovation, participation and actor-based STI indicators”. Paris
This paper describes the creation of “Journal Scholar Metrics” (JSM), a prototype web application that ranks journals in the areas of Arts, Humanities, and Social Sciences (AH&SS) on the basis of the citations their articles have received according to Google Scholar Metrics (GSM). To identify as many AH&SS journals as possible, a master list of 66,454 journals covered by various databases was developed. All AH&SS journals in that list were searched on GSM. Additionally, a series of keyword searches were carried out to identify journals covered by GSM which weren’t present in the master list. A total of 9,188 AH&SS journals with names written in Latin characters were found in the 2015 edition of GSM (which displays data about articles published between 2010 and 2014). Besides the journal-level indicators provided by GSM (H5-index and H5-median), several additional indicators were computed (H5-citations, H5-index and H5-citations without journal self-citations, and journal self-citation rate). Journals are displayed by subject categories and by country of publication. Quartiles were computed for each category, and journals in a category were further classified either as core (high affinity to the category) or related (partial affinity). A detail page for each journal is also available, displaying journal indicators, as well as a list of other databases were the journal is indexed.

Martin-Martin, A., Orduna-Malea, E, Harzing, A.W., Delgado López-Cózar, E. (2017). Can we use Google Scholar to identify highly-cited documents?. Journal of Informetrics, 11(1), 152-163. http://dx.doi.org/10.1016/j.joi.2016.11.008
The main objective of this paper is to empirically test whether the identification of highly-cited documents through Google Scholar is feasible and reliable. To this end, we carried out a longitudinal analysis (1950 to 2013), running a generic query (filtered only by year of publication) to minimise the effects of academic search engine optimisation. This gave us a final sample of 64,000 documents (1,000 per year). The strong correlation between a document’s citations and its position in the search results (r= -0.67) led us to conclude that Google Scholar is able to identify highly-cited papers effectively.
This, combined with Google Scholar’s unique coverage (no restrictions on document type and source), makes the academic search engine an invaluable tool for bibliometric research relating to the identification of the most influential scientific documents. We find evidence, however, that Google Scholar ranks those documents whose language (or geographical web domain) matches with the user’s interface language higher than could be expected based on citations. Nonetheless, this language effect and other factors related to the Google Scholar’s operation, i.e. the proper identification of versions and the date of publication, only have an incidental impact. They do not compromise the ability of Google Scholar to identify the highly-cited papers.


Mingers, John, and Martin Meyer. Normalizing Google Scholar data for use in research evaluation." Scientometrics (2017): in press DOI: 10.1007/s11192-017-2415-x
Using bibliometric data for the evaluation of the research of institutions and individuals is becoming increasingly common. Bibliometric evaluations across disciplines require that the data be normalized to the field because the fields are very different in their citation processes. Generally, the major bibliographic databases such as Web of Science (WoS) and Scopus are used for this but they have the disadvantage of limited coverage in the social science and humanities. Coverage in Google Scholar (GS) is much better but GS has less reliable data and fewer bibliometric tools. This paper tests a method for GS normalization developed by Bornmann et al. (J Assoc Inf Sci Technol 67:2778–2789, 2016) on an alternative set of data involving journal papers, book chapters and conference papers. The results show that GS normalization is possible although at the moment it requires extensive manual involvement in generating and validating the data. A comparison of the normalized results for journal papers with WoS data shows a high degree of convergent validity


Mitra, A., Awekar, A.  (2017). On Low Overlap Among Search Results of Academic Search Engines. arXiv preprint arXiv:1701.02617.
Number of published scholarly articles is growing exponentially. To tackle this information overload, researchers are increasingly depending on niche academic search engines. Recent works have shown that two major general web search engines: Google and Bing, have high level of agreement in their top search results. In contrast, we show that various academic search engines have low degree of agreement among themselves. We performed experiments using 2500 queries over four academic search engines. We observe that overlap in search result sets of any pair of academic search engines is significantly low and in most of the cases the search result sets are mutually exclusive. We also discuss implications of this low overlap.


Naude, F.  (2017). Comparing Downloads, Mendeley Readership and Google Scholar Citations as Indicators of Article Performance. The Electronic Journal of Information Systems in Developing Countries, 78(4).
This single journal article level indicator study investigated the relationship between download usage statistics, Mendeley readership scores and Google Scholar citation counts. The 378 articles published in the Electronic Journal of Information Systems in Developing Countries (EJISDC) in the 14-year period 2000 to 2013 were examined. Results showed that all 378 articles were downloaded and had Mendeley readers. Of the 378 articles, 359 (94.97%) articles received Google Scholar citations and 19 (5.03%) articles received no citations. For the 359 cited articles, the average Google Scholar citations per article were 28.82. The average EJISDC downloads were 7440.69, the average Mendeley readership was 19.30 and Google Scholar citations were 27.36. The most influential articles in the EJISDC were identified by ranking and by comparing the top 20 articles by downloads, Mendeley readership and Google Scholar citations. The leading authors were identified using the top 20 ranking and comparing it to the most productive authors. For this journal, the results seem to indicate that the highest correlations (Spearman correlation coefficient) were between Google Scholar citations and downloads, a slightly lower correlation between Google Scholar citations and Mendeley readership, and the lowest correlation was between downloads and Mendeley readership.
Orduna-Malea, E., Delgado López-Cózar, E. (2017). Performance behaviour patterns in author-level metrics: a disciplinary comparison of Google Scholar Citations, ResearchGate and ImpactStory. Frontiers in Research Metrics and Analytics, 2, 14. https://doi.org/10.3389/frma.2017.00014 
The main goal of this work is to verify the existence of diverse behavior patterns in academic production and impact, both among members of the same scientific community (inter-author variability) and for a single author (intra-author variability), as well as to find out whether this fact affects the correlation among author-level metrics (AutLMs) in disciplinary studies. To do this, two samples are examined: a general sample (members of a discipline, in this case Bibliometrics; n = 315 authors), and a specific sample (only one author; n = 119 publications). Four AutLMs (Total Citations, Recent Citations, Reads, and Online mentions) were extracted from three platforms (Google Scholar Citations, ResearchGate, and ImpactStory). The analysis of the general sample reveals the existence of different performance patterns, in the sense that there are groups of authors who perform prominently in some platforms, but exhibit a low impact in the others. The case study shows that the high performance in certain metrics and platforms is due to the coverage of document typologies, which is different in each platform (for example, Reads in working papers). It is concluded that the identification of the behavior pattern of each author (both at the inter-author and intra-author levels) is necessary to increase the precision and usefulness of disciplinary analyses that use AutLMs, and thus avoid masking effects

Orduna-Malea, E.; Martín-Martín, A.; Delgado López-Cózar, E. (2017). Google Scholar as a source for scholarly evaluation: a bibliographic review of database errors. Revista Española de Documentación Científica, 40 (4): e185.  http://dx.doi.org/10.3989/redc.2017.4.1500
Google Scholar (GS) is an academic search engine and discovery tool launched by Google (now Alphabet) in November 2004. The fact that GS provides the number of citations received by each article from all other indexed articles (regardless of their source) has led to its use in bibliometric analysis and academic assessment tasks, especially in social sciences and humanities. However, the existence of errors, sometimes of great magnitude, has provoked criticism from the academic community. The aim of this article is to carry out an exhaustive bibliographical review of all studies that provide either specific or incidental empirical evidence of the errors found in Google Scholar. The results indicate that the bibliographic corpus dedicated to errors in Google Scholar is still very limited (n= 49), excessively fragmented, and diffuse; the findings have not been based on any systematic methodology or on units that are comparable to each other, so they cannot be quantified, or their impact analysed, with any precision. Certain limitations of the search engine itself (time required for data cleaning, limit on citations per search result and hits per query) may be the cause of this absence of empirical studies

Orduña-Malea, E., Ayllón, J. M., Martín-Martín, A., Delgado López-Cózar, E. (2017). The lost academic home: institutional affiliation links in Google Scholar Citations. Online Information Review, 41 (6), 762-781.
Purpose
Google Scholar Citations (GSC) provides an institutional affiliation link which groups together authors who belong to the same institution. The purpose of this paper is to ascertain whether this feature is able to identify and normalize all the institutions entered by the authors, and whether it is able to assign all researchers to their own institution correctly.
Design/methodology/approach
Systematic queries to GSC’s internal search box were performed under two different forms (institution name and institutional e-mail web domain) in September 2015. The whole Spanish academic system (82 institutions) was used as a test. Additionally, specific searches to companies (Google) and world-class universities were performed to identify and classify potential errors in the functioning of the feature.
Findings
Although the affiliation tool works well for most institutions, it is unable to detect all existing institutions in the database, and it is not always able to create a unique standardized entry for each institution. Additionally, it also fails to group all the authors who belong to the same institution. A wide variety of errors have been identified and classified.
Research limitations/implications
Even though the analyzed sample is good enough to empirically answer the research questions initially proposed, a more comprehensive study should be performed to calibrate the real volume of the errors.
Practical implications
The discovered affiliation link errors prevent institutions from being able to access the profiles of all their respective authors using the institutions lists offered by GSC. Additionally, it introduces a shortcoming in the navigation features of Google Scholar which may impair web user experience.
Social implications
Some institutions (mainly universities) are under-represented in the affiliation feature provided by GSC. This fact might jeopardize the visibility of institutions as well as the use of this feature in bibliometric or webometric analyses.
Originality/value

This work proves inconsistencies in the affiliation feature provided by GSC. A whole national university system is systematically analyzed and several queries have been used to reveal errors in its functioning. The completeness of the errors identified and the empirical data examined are the most exhaustive to date regarding this topic. Finally, some recommendations about how to correctly fill in the affiliation data (both for authors and institutions) and how to improve this feature are provided as well.

Orduna-Malea, E.; Martín-Martín, A. & Delgado López-Cózar, E. (2017). Google Scholar and the gray literature: A reply to Bonato’s review. EC3 Working Papers, 27. 11 February 2017 .DOI:http://dx.doi.org/10.13140/RG.2.2.20703.87207 .

Recently, a review concluded that Google Scholar (GS) is not a suitable source of information “for identifying recent conference papers or other gray literature publications”. The goal of this letter is to demonstrate that GS can be an effective tool to search and find gray literature, as long as appropriate search strategies are used. To do this, we took as examples the same two case studies used by the original review, describing first how GS processes original’s search strategies, then proposing alternative search strategies, and finally generalizing each case study to compose a general search procedure aimed at finding gray literature in Google Scholar for two wide selected case studies: a) all contributions belonging to a congress (the ASCO Annual Meeting); and b) indexed guidelines as well as gray literature within medical institutions (National Institutes of Health) and governmental agencies (U.S. Department of Health & Human Services). The results confirm that original search strategies were undertrained offering misleading results and erroneous conclusions. Google Scholar lacks many of the advanced search features available in other bibliographic databases (such as Pubmed), however, it is one thing to have a friendly search experience, and quite another to find gray literature. We finally conclude that Google Scholar is a powerful tool for searching gray literature, as long as the users are familiar with all the possibilities it offers as a search engine. Poorly formulated searches will undoubtedly return misleading results.

Ortega, JL (2017). Toward a homogenization of academic social sites: A longitudinal study of profiles in Academia.edu, Google Scholar Citations and ResearchGate. Online Information Review, Vol. 41 Issue: 6, pp.812-25. 
https://doi.org/10.1108/OIR-01-2016-0012
Purpose
The purpose of this paper is to analyze the distribution of profiles from academic social networking sites according to disciplines, academic statuses and gender, and detect possible biases with regard to the real staff distribution. In this way, it intends to know whether these academic places tend to become specialized sites or, on the contrary, there is a homogenization process.
Design/methodology/approach
To this purpose, the evolution of profiles of one organization (Consejo Superior de Investigaciones Científicas) in three major academic social sites (Academia.edu, Google Scholar Citations and ResearchGate) through six quarterly samples since April 2014 to September 2015 are tracked.
Findings
Longitudinal results show important disciplinary biases but with strong increase of new profiles form different areas. They also suggest that these virtual spaces are gaining more stability and they tend toward a equilibrate environment.
Originality/value

This is the first longitudinal study of profiles from three major academic social networking sites and it allows to shed light on the future of these platforms’ populations.

Rochim, A.F., Sari, R.F. (2017). Evaluation of articles published in Mendeley and CrossRef in relation to the Google Scholar pages. ARPN Journal of Engineering and Applied Sciences , 12 ( 2 ), 330 - 335 .
This paper aims to show the performance of a researcher from their published articles. Our software crawled 10 (ten) most cited articles on the Google Scholar (GS), Mendeley and CrossRef with several of crawling methods. The method used in data retrieval is scrapping due to the limitations on the Application Programming Interface (API) provided by the Google search engine. To retrieve the Digital Object Identifier (DOI) data from Crossref, the API method has been used. In order to count the number of reader of paper on the Mendeley we used the API method. We used the R programming language, Python and Bash scripting shell. The operating system was based on Ubuntu 8.04 Linux and Mac OS. The Apache webserver were used to serve the website and we used the MySQL database to store the data. The database of MySQL is used for interfacing between R with the PHP language purposes. The Hypertext Preprocessor (PHP) is used for server-side scripting. Data was obtained by scrapping the best 10 articles from 100 Indonesia’s scientists indexed on the GS. Firstly, the data samples (S’) were obtained from the list of Indonesian scientists in Webometrics as the input of the GS scrapping. Secondly, the data resulted (S’’) were used as the input of the Crossref’s API query to obtain the DOI of each article. Finally, the DOIs were used as the input for the API query to get the number of the result to show the number of readers of each to article on Mendeley. The software produced can crawl the data from Google Scholar, Crossref and Mendeley reader count.
Tran, C. Y.,  Lyon, J. A. (2017). Faculty Use of Author Identifiers and Researcher Networking Tools. College & Research Libraries, 78(2): 171-172



This cross-sectional survey focused on faculty use and knowledge of author identifiers and researcher networking systems, and professional use of social media, at a large state university. Results from 296 completed faculty surveys representing all disciplines (9.3% response rate) show low levels of awareness and variable resource preferences. The most utilized author identifier was ORCID while ResearchGate, LinkedIn, and Google Scholar were the top profiling systems. Faculty also reported some professional use of social media platforms. The survey data will be utilized to improve library services and develop intra-institutional collaborations in scholarly communication, research networking, and research impact.
Thelwall, M., Kousha, K. (2017). ResearchGate versus Google Scholar: Which finds more early citations? Scientometrics ( Online version). DOI:http://dx.doi.org/10.1007/s11192-017-2400-4.
ResearchGate has launched its own citation index by extracting citations from documents uploaded to the site and reporting citation counts on article profile pages. Since authors may upload preprints to ResearchGate, it may use these to provide early impact evidence for new papers. This article assesses the whether the number of citations found for recent articles is comparable to other citation indexes using 2675 recently-published library and information science articles. The results show that in March 2017, ResearchGate found less citations than did Google Scholar but more than both Web of Science and Scopus. This held true for the dataset overall and for the six largest journals in it. ResearchGate correlated most strongly with Google Scholar citations, suggesting that ResearchGate is not predominantly tapping a fundamentally different source of data than Google Scholar. Nevertheless, preprint sharing in ResearchGate is substantial enough for authors to take seriously.

Tetsworth, K., Fraser, D., Glatt, V., Hohmann, E. (2017). Use of Google Scholar public profiles in orthopedics: Rate of growth and changing international patterns. Journal of Orthopaedic Surgery, 25(1).DOI:http://dx.doi.org/10.1177/2309499017690322 .
Introduction- The purpose of this study was to survey the growth of Google Scholar public profiles in orthopedics over a 12-month period and to investigate global patterns. 
Methods- Data was prospectively acquired from June 2013 to June 2014. Google Scholar queries specific to orthopedic surgery were performed at 90-day intervals. Demographic aspects of each user were also compiled, including gender, current location, and primary interests. To determine differences between the growth of Google Scholar public profile registrations and citation counts, as well as differences in growth in different regions, repeated measures of analysis of variance (RMANOVA) were used. 
Results- RMANOVA revealed statistically significant differences (p ¼ 0.0001) for regional growth. The largest growth was observed in the United Kingdom (p ¼ 0.009, 289%), followed by the Asia-Pacific region (p ¼ 0.004, 177%) and “Other” (p ¼ 0.006, 172%). The mean growth per 90-day interval is 19.9% (p ¼ 0.003) and the mean 12-month growth is 107% (p ¼ 0.05). Statistically significant differences between gender (male vs. female) and basic and clinical sciences (w2 ¼ 22.4, p ¼ 0.0001) were observed. 
Conclusion- This study suggests an exponential growth in the number of authors in the field of orthopedic surgery creating a Google Scholar public profile, and at the current rate participation doubles every 10.6 months.

2016 [Go back]


Bornmann, L., Thor, A., Marx, W., Schier, H. (2016): The application of bibliometrics to research evaluation in the humanities and social sciences: an exploratory study using normalized Google Scholar data for the publications of a research institute. Journal of the Association for Information Science and Technology, 67(11), 2778–2789 . DOI: http://dx.doi.org/10.1002/asi.23627. 
In the humanities and social sciences, bibliometric methods for the assessment of research performance are (so far) less common. This study uses a concrete example in an attempt to evaluate a research institute from the area of social sciences and humanities with the help of data from Google Scholar (GS). In order to use GS for a bibliometric study, we developed procedures for the normalization of citation impact, building on the procedures of classical bibliometrics. In order to test the convergent validity of the normalized citation impact scores, we calculated normalized scores for a subset of the publications based on data from the Web of Science (WoS) and Scopus. Even if scores calculated with the help of GS and the WoS/Scopus are not identical for the different publication types (considered here), they are so similar that they result in the same assessment of the institute investigated in this study: For example, the institute's papers whose journals are covered in the WoS are cited at about an average rate (compared with the other papers in the journals).
Bonato, S. (2016). Google Scholar and Scopus for finding gray literature publications. Journal of the Medical Library Association, 104(3), 252-254DOI:http://dx.doi.org/10.3163/1536-5050.104.3.021.

Bramer, W.M., Giustini, D., Kramer, B.M. (2016). Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study. Systematic Reviews, 5(1),1 . DOI:http://dx.doi.org/10.1186/s13643-016-0215-7. 

Background - Previously, we reported on the low recall of Google Scholar (GS) for systematic review (SR) searching. Here, we test our conclusions further in a prospective study by comparing the coverage, recall, and precision of SR search strategies previously performed in Embase, MEDLINE, and GS.

Methods - The original search results from Embase and MEDLINE and the first 1000 results of GS for librarian-mediated SR searches were recorded. Once the inclusion-exclusion process for the resulting SR was complete, search results from all three databases were screened for the SR’s included references. All three databases were then searched post hoc for included references not found in the original search results.
Results - We checked 4795 included references from 120 SRs against the original search results. Coverage of GS was high (97.2 %) but marginally lower than Embase and MEDLINE combined (97.5 %). MEDLINE on its own achieved 92.3 % coverage. Total recall of Embase/MEDLINE combined was 81.6 % for all included references, compared to GS at 72.8 % and MEDLINE alone at 72.6 %. However, only 46.4 % of the included references were among the downloadable first 1000 references in GS. When examining data for each SR, the traditional databases’ recall was better than GS, even when taking into account included references listed beyond the first 1000 search results. Finally, precision of the first 1000 references of GS is comparable to searches in Embase and MEDLINE combined.
Conclusions - Although overall coverage and recall of GS are high for many searches, the database does not achieve full coverage as some researchers found in previous research. Further, being able to view only the first 1000 records in GS severely reduces its recall percentages. If GS would enable the browsing of records beyond the first 1000, its recall would increase but not sufficiently to be used alone in SR searching. Time needed to screen results would also increase considerably. These results support our assertion that neither GS nor one of the other databases investigated, is on its own, an acceptable database to support systematic review searching.
Bramer, W.M. (2016). Variation in number of hits for complex searches in Google Scholar. Journal of the Medical Library Association: JMLA, 104(2), 143-145. DOI: http://dx.doi.org/10.3163/1536-5050.104.2.009.
Objective - Google Scholar is often used to search for medical literature. Numbers of results reported by Google Scholar outperform the numbers reported by traditional databases. How reliable are these numbers? Why are often not all available 1,000 references shown?
Methods - For several complex search strategies used in systematic review projects, the number of citations and the total number of versions were calculated. Several search strategies were followed over a two-year period, registering fluctuations in reported search results.
Results - Changes in numbers of reported search results varied enormously between search strategies and dates. Theories for calculations of the reported and shown number of hits were not proved.
Conclusions - The number of hits reported in Google Scholar is an unreliable measure. Therefore, its repeatability is problematic, at least when equal results are needed.
DeSanto, D., Nichols, A. (2016). Scholarly Metrics Baseline: A Survey of Faculty Knowledge, Use, and Opinion About Scholarly Metrics. College & Research Libraries, crl16-868, (Online Version).
This article presents the results of a faculty survey conducted at the University of Vermont during academic year 2014-2015. The survey asked faculty about: familiarity with scholarly metrics, metric seeking habits, help seeking habits, and the role of metrics in their department’s tenure and promotion process. The survey also gathered faculty opinions on how well scholarly metrics reflect the importance of scholarly work and how faculty feel about administrators gathering institutional scholarly metric information. Results point to the necessity of understanding the campus landscape of faculty knowledge, opinion, importance, and use of scholarly metrics before engaging faculty in further discussions about quantifying the impact of their scholarly work. 
 

Doğan, G., Şencan, I., Tonta, Y. (2016). Does dirty data affect google scholar citations?. In : Proceedings of the 79th ASIS&T Annual Meeting: Creating Knowledge, Enhancing Lives through Information & Technology (p. 98). American Society for Information Science.
Google Scholar (GS) is a database that enables researchers to create their scholarly profiles and keeps track of, among others, their citation counts, and h- and i10-index values. GS is now increasingly being used for research evaluation purposes. Although rich in bibliometric data, GS indexes some duplicate publications and citations, and therefore tends to inflate the citation counts to some extent. Based on a small sample of GS profiles of researchers, this paper aims to study the extent by which duplicates change the citation counts and metrics based thereupon. Findings show that duplicates in GS database somewhat inflates the citation metrics. The scale of the problem as well as the effect of dirty data on performance evaluations based on GS citations data need to be studied further using larger samples
Fatchur Rochim, A., Fitri Sari, R. (2016). An Exploration of Mendeley Reader and Google Scholar Citation for Analysing Indexed Article. In: Asea Uninet Scientific and Plenary Meeting 2016 , 78-85. 
This paper aims to analyze the number of readers from the published articles of 100 Indonesian researchers in Mendeley reference management software. The list of Indonesian scientists is obtained from the webometrics ranking of scientists. We used the Application Programming Interface (API) of Mendeley to count the number of readers for each article in Mendeley and combine it with Google Scholar citation using the scrap method. We processed ten mostly cited articles that are indexed in the first page of the Google Scholar for each designated scientist. Furthermore, we used the Pearson method to analyze the correlation of the Mendeley readers count and the Google Scholar citation. The results show that they are correlated with a value of 0.266 according to the method of Pearson with N = 1000. Furthermore we found that many online Indonesian journals have no Digital Object Identifier (DOI) yet. Our evaluation of the publication results of 100 Indonesian researchers shows that authors who upload their scientific work on Mendeley, have higher citation number in Google Scholar, because their papers are widely available on the Internet.
Gardner, T., Inger, S. (2016). How Readers Discover Content in Scholarly Publications. Abingdon, Renew Training. ISBN 978-0-9573920-4-5.

Gorraiz, J., Gumpenberger, C., Glade, T. (2016). On the bibliometric coordinates of four different research fields in Geography. Scientometrics, 107(2), 873–897. DOI: http://dx.doi.org/10.1007/s11192-016-1864-y. 
This study is a bibliometric analysis of the highly complex research discipline Geography. In order to identify the most popular and most cited publication channels, to reveal publication strategies, and to analyse the discipline’s coverage within publications, the three main data sources for citation analyses, namely Web of Science, Scopus and Google Scholar, have been utilized. This study is based on publication data collected for four individual evaluation exercises performed at the University of Vienna and related to four different subfields: Geoecology, Social and Economic Geography, Demography and Population Geography, and Economic Geography. The results show very heterogeneous and individual publication strategies, even in the same research fields. Monographs, journal articles and book chapters are the most cited document types. Differences between research fields more related to the natural sciences than to the social sciences are clearly visible, but less considerable when taking into account the higher number of co-authors. General publication strategies seem to be established for both natural science and social sciences, however, with significant differences. While in natural science mainly publications in international peer-reviewed scientific journals matter, the focus in social sciences is rather on book chapters, reports and monographs. Although an “iceberg citation model” is suggested, citation analyses for monographs, book chapters and reports should be conducted separately and should include complementary data sources, such as Google Scholar, in order to enhance the coverage and to improve the quality of the visibility and impact analyses. This is particularly important for social sciences related research within Geography.
 
Green, E.D. (2016). What are the most-cited publications in the social sciences (according to Google Scholar)?.  Impact of Social Sciences Blog (12 May 2016) Blog Entry.

Drawing on citation data that spans disciplines and time periods, Elliott Green has identified the most cited publications in the social sciences. Here he shares his findings on the 25 most cited books as well as the top ten journal articles. The sheer number of citations for these top cited publications is worth noting as is the fact that no one discipline dominates over the others in the top 20, with the top six books all from different disciplines.

Harzing, A.W., Alakangas, S.  (2016). Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics, 106(2), 787-804 DOI: http://dx.doi.org/10.1007/s11192-015-1798-9. 
This article aims to provide a systematic and comprehensive comparison of the coverage of the three major bibliometric databases: Google Scholar, Scopus and the Web of Science. Based on a sample of 146 senior academics in five broad disciplinary areas, we therefore provide both a longitudinal and a cross-disciplinary comparison of the three databases. Our longitudinal comparison of eight data points between 2013 and 2015 shows a consistent and reasonably stable quarterly growth for both publications and citations across the three databases. This suggests that all three databases provide sufficient stability of coverage to be used for more detailed cross-disciplinary comparisons. Our cross-disciplinary comparison of the three databases includes four key research metrics (publications, citations, h-index, and hI, annual, an annualised individual h-index) and five major disciplines (Humanities, Social Sciences, Engineering, Sciences and Life Sciences). We show that both the data source and the specific metrics used change the conclusions that can be drawn from cross-disciplinary comparisons.


Hassan, S.U., Gillani, U.A. (2016). Altmetrics of" altmetrics" using Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki. arXiv:1603.07992.
We measure the impact of "altmetrics" field by deploying altmetrics indicators using the data from Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki during 2010- 2014. To capture the social impact of scientific publications, we propose an index called alt-index, analogues to h-index. Across the deployed indices, our results have shown high correlation among the indicators that capture social impact. While we observe medium Pearson's correlation (\r{ho}= .247) among the alt-index and h-index, a relatively high correlation is observed between social citations and scholarly citations (\r{ho}= .646). Interestingly, we find high turnover of social citations in the field compared with the traditional scholarly citations, i.e. social citations are 42.2 % more than traditional citations. The social mediums such as Twitter and Mendeley appear to be the most effective channels of social impact followed by Facebook and Google-plus. Overall, altmetrics appears to be working well in the field of "altmetrics".

Jacobs, JA. (2016). Journal rankings in sociology: Using the H Index with Google Scholar. The American Sociologist, 47(2), 192–224. DOI: http://dx.doi.org/10.1007/s12108-015-9292-7. 
There is considerable interest in the ranking of journals, given the intense pressure to place articles in the “top” journals. In this article, a new index, h, and a new source of data—Google Scholar – are introduced, and a number of advantages of this methodology to assessing journals are noted. This approach is attractive because it provides a more robust account of the scholarly enterprise than do the standard Journal Citation Reports. Readily available software enables do-it-yourself assessments of journals, including those not otherwise covered, and enable the journal selection process to become a research endeavor that identifies particular articles of interest. While some critics are skeptical about the visibility and impact of sociological research, the evidence presented here indicates that most sociology journals produce a steady stream of papers that garner considerable attention. While the position of individual journals varies across measures, there is a high degree commonality across these measurement approaches. A clear hierarchy of journals remains no matter what assessment metric is used. Moreover, data over time indicate that the hierarchy of journals is highly stable and self-perpetuating. Yet highly visible articles do appear in journals outside the set of elite journals. In short, the h index provides a more comprehensive picture of the output and noteworthy consequences of sociology journals than do than standard impact scores, even though the overall ranking of journals does not markedly change.



Khalil, M.H., El-Gizawy, N.K., Ghanem, M.M. (2016). Impact of Benha University Scientific Research Fund on Building Research Capacities of Graduate Students. 6 th QS Middle East and Africa Professional Leaders in Education Conference, May 10-12, 2016, United Arab Emirates University (UAEU).
The activities of Scientific Research Fund (SRF) in Benha University (BU) were started at 2013 aiming to support the research capacity building of junior researchers and postgraduate students with financial fund. During the last three years, 83 research proposals were submitted to SRF and 35 projects were accepted. All the researchers received a training program of using the scientific research softwares such as Endnote, SPSS and the skillful use of digital library. All the researchers were mandated to upload their research papers on the Portal Website of BU along with the websites of Google Scholar and Research Gate to improve the World ranking of BU. Nowadays, 35 research groups were established in different faculties of BU. The major impacts of funding these groups were: 1) increasing significantly and considerably the number of citations in BU (32496 citation), 2) increasing the number of impacted publications (1 publication), 3) elevating significantly BU ranking globally to be 1238 in the last webometrics ranking issued in January 2016 and to reach the 3 rd position in the Egyptian Universities ranking. In practice, we are looking to have an advance ranking in QS ranking particularly in Arab region.

Lande, D.V., Andrushchenko, V.B. (2016). Formation of subject area and the co-authors network by sounding of Google Scholar Citations service. arXiv preprint arXiv:1605.02215. 
The suggested methodic is the way of formatting the subject areas models and co-authors networks by sounding the content networks. The paper represents the notion networks which match tags and authors of Google Scholar Citations service. Models depicted in the work were built for the physical optics area, and it can be applied for other domains. The proposed ways of defining connections between science areas and authors depicts the collaborations opportunities and versatility of interdisciplinary.



Levay, P., Ainsworth, N., Kettle, R., Morgan, A. (2016). Identifying evidence for public health guidance: a comparison of citation searching with Web of Science and Google Scholar. Research Synthesis Methods, 7(1), 34–45. DOI: http://dx.doi.org/10.1002/jrsm.1158 . 
Aim - to examine how effectively forwards citation searching with Web of Science (WOS) or Google Scholar (GS) identified evidence to support public health guidance published by the National Institute for Health and Care Excellence.
Method - forwards citation searching was performed using GS on a base set of 46 publications and replicated using WOS.
Outcomes WOS and GS were compared in terms of recall; precision; number needed to read (NNR); administrative time and costs; and screening time and costs. Outcomes for all publications were compared with those for a subset of highly important publications.
Results - the searches identified 43 relevant publications. The WOS process had 86.05% recall and 1.58% precision. The GS process had 90.7% recall and 1.62% precision. The NNR to identify one relevant publication was 63.3 with WOS and 61.72 with GS. There were nine highly important publications. WOS had 100% recall, 0.38% precision and NNR of 260.22. GS had 88.89% recall, 0.33% precision and NNR of 300.88. Administering the WOS results took 4 h and cost £88–£136, compared with 75 h and £1650–£2550 with GS.
Conclusion - WOS is recommended over GS, as citation searching was more effective, while the administrative and screening times and costs were lower. 


Martín-Martín, A.Orduna-Malea, E.Ayllón, J.M. , Delgado López-Cózar, E. (2016). Back to the past: on the shoulders of an academic search engine giant. Scientometrics, 107(3), 1477-1487. DOI: http://dx.doi.org/10.1007/s11192-016-1917-2.
A study released by the Google Scholar team found an apparently increasing fraction of citations to old articles from studies published in the last 24 years (1990–2013). To demonstrate this finding we conducted a complementary study using a different data source (Journal Citation Reports), metric (aggregate cited half-life), time spam (2003–2013), and set of categories (53 Social Science subject categories and 167 Science subject categories). Although the results obtained confirm and reinforce the previous findings, the possible causes of this phenomenon keep unclear. We finally hypothesize that “first page results syndrome” in conjunction with the fact that Google Scholar favours the most cited documents are suggesting the growing trend of citing old documents is partly caused by Google Scholar.
Martín-Martín, A., Orduna-Malea, E., Ayllón, J.M. , Delgado López-Cózar, E. (2016). A two-sided academic landscape: snapshot of highly-cited documents in Google Scholar (1950-2013) Revista Española de Documentación Científica ,39(4),  e149. DOI: http://dx.doi.org/10.3989/redc.2016.4.1405.
The main objective of this paper is to identify and define the core characteristics of the set of highly-cited documents in Google Scholar (document types, language, free availability, sources, and number of versions), on the hypothesis that the wide coverage of this search engine may provide a different portrait of these documents with respect to that offered by traditional bibliographic databases. To do this, a query per year was carried out from 1950 to 2013 identifying the top 1,000 documents retrieved from Google Scholar and obtaining a final sample of 64,000 documents, of which 40% provided a free link to full-text. The results obtained show that the average highly-cited document is a journal or book article (62% of the top 1% most cited documents of the sample), written in English (92.5% of all documents) and available online in PDF format (86.0% of all documents). Yet, the existence of errors should be noted, especially when detecting duplicates and linking citations properly. Nonetheless, the fact that the study focused on highly cited papers minimizes the effects of these limitations. Given the high presence of books and, to a lesser extent, of other document types (such as proceedings or reports), the present research concludes that the Google Scholar data offer an original and different vision of the most influential academic documents (measured from the perspective of their citation count), a
set composed not only of strictly scientific material (journal articles) but also of academic material in its broadest sense. 




Martín-Martín, A., Orduna-Malea, E., Ayllón, J.M. , Delgado López-Cózar, E. (2016). The counting house: measuring those who count. Presence of Bibliometrics, Scientometrics, Informetrics, Webometrics and Altmetrics in the Google Scholar Citations, ResearcherID, ResearchGate, Mendeley & Twitter”. EC3 Working Papers, 21. 19th of January 2016. DOI: http://dx.doi.org/10.13140/RG.2.1.4814.4402. 
Following in the footsteps of the model of scientific communication, which has recently gone through a metamorphosis (from the Gutenberg galaxy to the Web galaxy), a change in the model and methods of scientific evaluation is also taking place. A set of new scientific tools are now providing a variety of indicators which measure all actions and interactions among scientists in the digital space, making new aspects of scientific communication emerge. In this work we present a method for capturing the structure of an entire scientific community (the Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics community) and the main agents that are part of it (scientists, documents, and sources) through the lens of Google Scholar Citations. 
Additionally, we compare these author portraits to the ones offered by other profile or social platforms currently used by academics (ResearcherID, ResearchGate, Mendeley, and Twitter), in order to test their degree of use, completeness, reliability, and the validity of the information they provide. A sample of 814 authors (researchers in Bibliometrics with a public profile created in Google Scholar Citations was subsequently searched in the other platforms, collecting the main indicators computed by each of them. The data collection was carried out on September, 2015. The Spearman correlation was applied to these indicators (a total of 31) , and a Principal Component Analysis was carried out in order to reveal the relationships among metrics and platforms as well as the possible existence of metric clusters
 
Moed, H.F., Bar-Ilan, J., Halevi, G.  (2016). A new methodology for comparing Google Scholar and Scopus. Journal of Informetrics, 10(2), 533-551. DOI: http://dx.doi.org/10.1016/j.joi.2016.04.017.
A new methodology is proposed for comparing Google Scholar (GS) with other citation indexes. It focuses on the coverage and citation impact of sources, indexing speed, and data quality, including the effect of duplicate citation counts. The method compares GS with Elsevier’s Scopus, and is applied to a limited set of articles published in 12 journals from six subject fields, so that its findings cannot be generalized to all journals or fields. The study is exploratory, and hypothesis generating rather than hypothesis-testing. It confirms findings on source coverage and citation impact obtained in earlier studies. The ratio of GS over Scopus citation varies across subject fields between 1.0 and 4.0, while Open Access journals in the sample show higher ratios than their non-OA counterparts. The linear correlation between GS and Scopus citation counts at the article level is high: Pearson’s R is in the range of 0.8–0.9. A median Scopus indexing delay of two months compared to GS is largely though not exclusively due to missing cited references in articles in press in Scopus. The effect of double citation counts in GS due to multiple citations with identical or substantially similar meta-data occurs in less than 2% of cases. Pros and cons of article-based and what is termed as concept-based citation indexes are discussed.
Novaković, Ana (2016). Primerjava iskanja virov v DiKUL in Google Scholar : magistrsko delo . Univerza v Ljubljani, Filozofska fakulteta . 
We compared the simple searches in DiKUL, an academic search engine developed by University in Ljubljana for its own use that searches its sources and Google search, an academic search engine developed by one of the biggest players on the World Wide Web. We conducted simple thematic inquiries with both search engines on various scientific fields and analysed the types of documents in the search results, their age, availability of full text, their relevance and the most important bibliometric data within the first forty search results. The differences in the search results between the two search engines are due to their different algorithms for sorting results by relevance. There are also differences between different scientific fields - with our inquiries Google Scholar was comparatively worse in the fields of social and natural sciences, but better in the fields of medicine and engineering. However, comparatively poorer achievements with other criteria are not necessarily reflected in relevance. There are more different types of documents and more full text documents available in DiKUL's search results. Both search engines give good results, among which there is little overlap. DiKUL is more suitable for less experienced users, while Google Scholar is more suitable for more experienced users, who will know where and how to check the additional availability of the search results and will be able to critically approach the search results which are too old or of inappropriate type.


Orduna-Malea, E.Martín-Martín, A. Ayllón, J.M. , Delgado López-Cózar, E. (2016). La revolución Google Scholar: Destapando la caja de Pandora académica. Unión de Editoriales Universitarias Españolas (UNE), Universidad de Granada . ISBN 978-84-338-5985-3
Este libro es fruto de la intensa y extensa labor investigadora desplegada por los autores durante casi una década en el seno de la Universidad de Granada y la Universidad Politécnica de Valencia donde trabajan como profesores e investigadores. Resultado de todo este que hacer ha sido la publicación de más de 50 artículos, informes y documentos de trabajo así como el diseño de varios productos de evaluación bibliométrica (H Index Scholar, Journal Scholar Metrics, Scholar Mirrors, Publishers Scholar Metrics, Proceedings Scholar Metrics, Classics Scholar’ Profiles, Indice H revistas científicas españolas según Google Scholar Metrics, La Biblioteconomía y Documentación española según Google Scholar Citations), que han explorado terrenos ignotos para la evaluación científica y abierto por primera vez nuevas vías para captar la producción e impacto de los científicos y sus obras.
Google Scholar ha supuesto una auténtica revolución en la forma de buscar y de encontrar información científica y de acceder a ella de manera fácil y rápida. Hoy día su simple caja de búsqueda, convertida ya en un estándar de facto en la búsqueda de información, se encarga de mostrar numerosos y pertinentes documentos escritos en cualquier lengua, producidos en cualquier país, bajo cualquier formato y tipología documental. Y todo ello ofrecido de forma gratuita. Pero lo que fue creado para buscar información se ha transformado, sin quererlo, en una valiosísima fuente de datos para la evaluación científica con múltiples usos bibliométricos, es decir, en una auténtica “caja de pandora académica”. 
El objetivo de este libro es, ante todo, describir minuciosamente las características y prestaciones de Google Scholar. Se traza su origen y evolución, se desmenuza su funcionamiento general, se examina su tamaño, cobertura y crecimiento, se pormenorizan las prestaciones y servicios que proporciona como buscador y se apuntan sus fortalezas, debilidades y peligros. Se realiza un repaso exhaustivo de toda la literatura científica que ha indagado empíricamente sobre el comportamiento de Google Scholar hasta hoy. 
Pero también, esta obra pretende explicar los principales productos derivados de Google Scholar, aquellos que conforman lo que denominamos la familia Google Scholar: Google Scholar Citations, nacido como un servicio a los autores para que generen un perfil donde se muestren los documentos publicados y recogidos en Google Scholar, así como el número de citas que cada uno de ellos ha recibido, generando una serie de indicadores bibliométricos. Y Google Scholar Metrics nacido como ranking de publicaciones científicas que permite a partir de los datos de citaciones de Google Scholar y por medio de índice h identificar las más influyentes revistas, congresos y repositorios. Por otra parte, se ha incluido un somero análisis de aquellos productos independientes a la compañía y que se han gestado para ofrecer distintas herramientas bibliométricas: Publish or Perish, Scholarometer, H Index Scholar, Journal Scholar Metrics, Publishers Scholar Metrics, Proceedings Scholar Metrics y Scholar Mirrors. 
En definitiva estamos ante un libro que ofrece una revisión omnicomprensiva a la par que minuciosa sobre lo que es a día de hoy Google Scholar y sus derivaciones.

 
Piotr, S., Przemysław, Ś., Barbara, J.  (2016). Cytowania członków komitetów naukowych Polskiej Akademii Nauk według Google Scholar.  Zagadnienia Naukoznawstwa, 52(4 (210)), 529-560.
The article presents a comprehensive analysis of citations of Polish scientists who are members of the scientifi c committees of the Polish Academy of Sciences. In particular, they dealt with comparing different fields of science, but also showing that it is not always such comparisons are legitimate. Based on Google Scholar (Publish or Perish 3.0) in 2013 information about the 1.1 million citations for 3033 people, 95 are members of the
scientific committees were gathered. The analysis is presented in three aspects: the general population researchers, various scientific committees and the six main fields of science (on the basis of the OECD division of science). The results confirm the existence of differences between the fields of science, under both the specifics of domain, as well as different cultures and traditions citations researchers.
Prins, A.A.M., Costas, R., Van Leeuwen, T.N., Wouters, P.F.  (2016). Using Google Scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data. Research Evaluation, 25(3), 264-270DOI: http://dx.doi.org/10.1093/reseval/rvv049
In this paper, we report on the application of Google Scholar (GS)-based metrics in the formal assessment of research programs. Involved were programs in the fields of Education, Pedagogical Sciences, and Anthropology in The Netherlands. Also, a comparative analysis has been conducted of the results based on GS and Web of Science (WoS). Studies critical of GS point at its reliability of data. We show how the reliability of the GS data for the bibliometric analysis of the assessment can be improved by excluding non-verifiable citing sources from the full second-order GS citing data. The study of the background of these second-order sources demonstrates a broadening of the citing sources. The comparison of GS with WoS citations for the publications of the programs shows that it is promising to use GS for fields with lower degrees of coverage in WoS, in particular for fields that produce more diverse types of output than just research articles. Restrictions to the use of GS are the intensive manual data handling and cleaning, necessary for a feasible and proper data collection. We discuss wider implications of the findings for bibliometric analysis and for the practices and policies in research evaluation.


Spiroski, M.  (2016). Current Scientific Impact of Macedonian Biomedical Journals (2016) in Google Scholar Database Analysed with the Software Publish or Perish.Maced Med Electr J. 2016 May 30; 50022  . DOI: http://dx.doi.org/10.3889/mmej.2016.50022 .
Aim: The aims of this paper are: to analyze Macedonian biomedical journals in the Google Scholar database with the software Publish or Perish, to present their current scientific impact, to rank the journals, and to advice the authors about the possibilities where to publish their papers.
Material and Methods: Biomedical journals in the Republic of Macedonia included in Macedonian Association of Medical Editors (MAME) are analyzed. The results are obtained with the software Publish or Perish which analyze publicly available scholarly papers in the Google Scholar database (May 11, 2016).
Results: From 38 journals only 25 has indexed one or more papers in the Google Scholar database. The rest of 13 journals are without any paper indexed in this base and are not included in this investigation. The biggest number of citations in the Google Scholar database have the journals Prilozi - MANU, 0351-3254 (1622); Macedonian Journal of Medical Sciences, 1857-5773 (838) and Macedonian Journal of Chemistry and Chemical Engineering, 1857-5552 (705).
Conclusions: Macedonian biomedical journals have limited scientific influence, although the last years several journals are significantly improved and are included in the more important international scientific databases. Further coordinated efforts through MZMU are necessary in order to increase the scientific contribution of Macedonian biomedical journals.


Sun,Y., Xia, BS. (2016).The scholarly communication of economic knowledge: a citation analysis of Google Scholar.  Scientometrics, (Online Version) DOI: http://dx.doi.org/10.1007/s11192-016-2140-x. 
Citation counts can be used as a proxy to study the scholarly communication of knowledge and the impact of research in academia. Previous research has addressed several important factors of citation counts. In this study, we aim to investigate whether there exist quantitative patterns behind citations, and thus provide a detailed analysis of the factors behind successful research. The study involves conducting quantitative analyses on how various features, such as the author’s quality, the journal’s impact factor, and the publishing year, of a published scientific article affect the number of citations. We carried out full-text searches in Google Scholar to obtain our data set on citation counts. The data set is then set up into panels and used to conduct the proposed analyses by employing a negative binomial regression. Our results show that attributes such as the author’s quality and the journal’s impact factor do have important contributions to its citations. In addition, an article’s citation count does not only depend on its own properties as mentioned above but also depends on the quality, as measured by the number of citations, of its cited articles. That is, the number of citations of a paper seems to be affected by the number of citations of articles that the particular paper cites. This study provides statistical characteristics of how different features of an article affect the number of citations. In addition, it provides statistical evidence that the number of citations of a scientific article depends on the number of citations of the articles it cites.



Tost, C.M., Rindermann, H. (2016). Evaluation psychologischer und pädagogischer forschungsleistungen mithilfe der bibliometrischen datenbanken scopus und google Scholar/Evaluation of psychological and educational research using the bibliometric databases scopus and google scholar. Zeitschrift Für Evaluation, 15(2), 241-267 .  
To test the usability of bibliometric databases for the evaluation of psychological and educational research, the number of publications and the h-Index of 453 professors of psychology and 358 professors of educational science from Bavaria, Berlin, North Rhine-Westphalia, Saxony and from Austria were acquired from the databases Scopus and Google Scholar. Web of Science did not allow to identify the majority of professors and was thus excluded. In Scopus, the professors of educational science were identified less frequently than the professors of psychology; in Google Scholar such differences could not be detected. The mean number of publications and the mean h-Index for psychology were higher than those for educational science; these differences appeared much more obvious in Scopus. In spite of large mean differences between the databases, high correlations could be found (r^sub number^=.54, r^sub h^=.69). These results document the essential differences in publishing between the two academic fields and the usability of Google Scholar (after checking author assignment) for bibliometric analyses. Comparisons across databases have to consider their mean differences.




Trapp, J. (2016). Web of Science, Scopus, and Google Scholar citation rates: a case study of medical physics and biomedical engineering: what gets cited and what doesn’t?. Australasian Physical & Engineering Sciences in Medicine, (Online Version) DOI: http://dx.doi.org/10.1007/s13246-016-0478-2. 
There are often differences in a publication’s citation count, depending on the database accessed. Here, aspects of citation counts for medical physics and biomedical engineering papers are studied using papers published in the journal Australasian physical and engineering sciences in medicine. Comparison is made between the Web of Science, Scopus, and Google Scholar. Papers are categorised into subject matter, and citation trends are examined. It is shown that review papers as a group tend to receive more citations on average; however the highest cited individual papers are more likely to be research papers.



Tsou, A., Bowman, T.D., Sugimoto, T., Lariviere, V., Sugimoto, C.R. (2016). Self-presentation in scholarly profiles: Characteristics of images and perceptions of professionalism and attractiveness on academic social networking sites. First Monday, 21(4) DOI: http://dx.doi.org/10.5210/fm.v21i4.6381. 
Online self-presentation is of increasing importance in modern life, from establishing and maintaining personal relationships to forging professional identities. Academic scholars are no exception, and a host of social networking platforms designed specifically for scholars abound. This study used Amazon’s Mechanical Turk service to code 10,500 profile pictures used by scholars on three platforms — Mendeley, Microsoft Academic Search, and Google Scholar — in order to determine how academics are presenting themselves to their colleagues and to the public at large and how they are perceived — particularly in relation to professionalism and attractiveness. The majority of the individuals on Mendeley, Microsoft Academic Search, and Google Scholar were Caucasian, male, and perceived to be over the age of 35. Females and younger individuals were perceived as less professional than male and older individuals, while women were more likely to be perceived as “attractive.” In addition, the Mechanical Turk coders were susceptible to framing; the individuals in the profile pictures were considered more “professional” if they were identified as “scholars” rather than merely as “individuals.” The results have far-reaching implications for self-presentation and framing, both for scholars and for other professionals. In the academic realm, there are serious implications for hiring and the allocation of resources and rewards.


Van Bevern, R., Komusiewicz, C., Niedermeier, R., Sorge, M., & Walsh, T. (2016). H-index manipulation by merging articles: Models, theory, and experiments. Artificial Intelligence 2016, 240, 9–35. DOI:http://doi.org/10.1016/j.artint.2016.08.001. 
An author's profile on Google Scholar consists of indexed articles and associated data, such as the number of citations and the H-index. The author is allowed to merge articles; this may affect the H-index. We analyze the (parameterized) computational complexity of maximizing the H-index using article merges. Herein, to model realistic manipulation scenarios, we define a compatibility graph whose edges correspond to plausible merges. Moreover, we consider several different measures for computing the citation count of a merged article. For the measure used by Google Scholar, we give an algorithm that maximizes the H-index in linear time if the compatibility graph has constant-size connected components. In contrast, if we allow to merge arbitrary articles (that is, for compatibility graphs that are cliques), then already increasing the H-index by one is NP-hard. Experiments on Google Scholar profiles of AI researchers show that the H-index can be manipulated substantially only if one merges articles with highly dissimilar titles.
Yu, K., Mustapha, N., Oozeer, N. (2016). Google Scholar’s Filter Bubble: An Inflated Actuality? . In: Research 2.0 and the Impact of Digital Technologies on Scholarly Inquiry, 211. DOI: http://dx.doi.org/10.4018/978-1-5225-0830-4.ch011.
This chapter investigates the allegation that popular online search engine Google applies algorithms to personalise search results therefore yielding different results for the exact same search terms. It specifically examines whether the same alleged filter bubble applies to Google's academic product: Google Scholar. It reports the results from an exploratory experiment of nine keywords carried out for this purpose, varying variables such as disciplines (Natural Science, Social Science and Humanities), geographic locations (north/south), and levels (senior/junior researchers). It also reports a short survey on academic search behaviour. The finding suggests that while Google Scholar, together with Google, has emerged as THE dominant search engine among the participants of this study, the alleged filter bubble is only mildly observable. The Jaccard similarity of search results for all nine keywords is strikingly high, with only one keyword that exhibits a localized bubble at 95% level. This chapter therefore concludes that the filter bubble phenomenon does not warrant concern. 


2015 [Go back]

Bartol, T., Mackiewicz-Talarczyk, M. (2015).  Bibliometric Analysis of Publishing Trends in Fiber Crops in Google Scholar, Scopus, and Web of Science. Journal of Natural Fibers, 12(6), 531-541. DOI: http://dx.doi.org/10.1080/15440478.2014.972000. 
The aim was to evaluate natural fibers (fiber crops or fiber plants) in Scopus, Web of Science (WOS), and Google Scholar with regard to growth trends and leading countries by authors/coauthors of documents. Basic functionalities of information systems were assessed. Search syntax was based on article titles, abstracts, and keywords (topics). The three information systems can only be consistently compared on the basis of article titles. Different ranks of individual fiber crops can be observed among the systems. The cumulative data for the entire period show different characteristics than the more recent trends. Specific crops show more intensive growth in the recent period. Retrieval with topics in Scopus and WOS also shows differences, probably on account of indexing method (KeyWords Plus in WOS and thesauri descriptors in Scopus). Several countries are ranked much higher in WOS than Scopus, and vice versa, indicating differences in coverage of journals. The principal contributing countries are China, India, and USA. China returns similar total counts as USA in 1994–2013 but is producing twice as many records in the most recent period. Interpretation of results depends on the query (selection of search terms, fields, and search syntax) and database or information system under analysis.
Bornmann, L., Thor, A., Marx, W., Schier, H. (2015): The application of bibliometrics to research evaluation in the humanities and social sciences: an exploratory study using normalized Google Scholar data for the publications of a research institute. figshare. DOI: http://dx.doi.org/10.6084/m9.figshare.1293588. 
In the classical core areas of natural and life sciences, bibliometric methods have become an integral part of research evaluation. In the humanities and social sciences, these methods for the assessment of research performance are (so far) less common. The current study takes a concrete example in an attempt to evaluate a research institute from the area of social sciences and humanities with the help of data from Google Scholar (GS). In order to use GS for a bibliometric study, we have developed procedures for the normalisation of citation impact, building on the procedures of classical bibliometrics. In order to test the convergent validity of the normalized citation impact scores, we have calculated normalized scores for a subset of the publications based on data from the WoS or Scopus. Even if scores calculated with the help of GS and WoS/Scopus are not identical for the different publication types (considered here), they are so similar that they result in the same assessment of the institute investigated in this study: The institute’s papers whose journals are covered in WoS are cited at about an average rate (compared with the other papers in the journals). Whereas the papers whose journals are not covered in WoS, and the book chapters, are cited about 20 to 40% above the average, the conference papers are cited twice as often as one would expect for the papers from the same conference.
Cavacini, A. (2015). What is the best database for computer science journal articles?. Scientometrics, 102(3), 2059-2071.DOI: http://dx.doi.org/10.1007/s11192-014-1506-1.
We compared general and specialized databases, by searching bibliographic information regarding journal articles in the computer science field, and by evaluating their bibliographic coverage and the quality of the bibliographic records retrieved. We selected a sample of computer science articles from an Italian university repository (AIR) to carry out our comparison. The databases selected were INSPEC, Scopus, Web of Science (WoS), and DBLP. We found that DBLP and Scopus indexed the highest number of unique articles (4.14 and 4.05 % respectively), that each of the four databases indexed a set of unique articles, that 12.95 % of the articles sampled were not indexed in any of the databases selected, that Scopus was better than WoS for identifying computer science publications, and that DBLP had a greater number of unique articles indexed (19.03 %), when compared to INSPEC (11.28 %). We also measured the quality of a set of bibliographic records, by comparing five databases: Scopus, WoS, INSPEC, DBLP and Google Scholar (GS). We found that WoS, INSPEC and Scopus provided better quality indexing and better bibliographic records in terms of accuracy, control and granularity of information, when compared to GS and DBLP. WoS and Scopus also provided more sophisticated tools for measuring trends of scholarly publications.

Ciccone, K., Vickery, J. (2015). Summon, EBSCO Discovery Service, and Google Scholar: A Comparison of Search Performance Using User Queries. Evidence Based Library and Information Practice, 10(1), 34-49
Objectives - To evaluate and compare the results produced by Summon and EBSCO Discovery Service (EDS) for the types of searches typically performed by library users at North Carolina State University. Also, to compare the performance of these products to Google Scholar for the same types of searches.
Methods - A study was conducted to compare the search performance of two web-scale discovery services: ProQuest’s Summon and EBSCO Discovery Service (EDS). The performance of these services was also compared to Google Scholar. A sample of 183 actual user searches, randomly selected from the NCSU Libraries’ 2013 Summon search logs, was used for the study. For each query, searches were performed in Summon, EDS, and Google Scholar. The results of known-item searches were compared for retrieval of the known item, and the top ten results of topical searches were compared for the number of relevant results.
Results - There was no significant difference in the results between Summon and EDS for either known-item or topical searches. There was also no significant difference between the performance of the two discovery services and Google Scholar for known-item searches. However, Google Scholar outperformed both discovery services for topical searches.
Conclusions - There was no significant difference in the relevance of search results between Summon and EDS. Thus, any decision to purchase one of those products over the other should be based upon other considerations (e.g., technical issues, cost, customer service, or user interface)..

González Alonso, J. , Peréz González, Y.  (2015). Google Scholar`s and WEB presence for the Cuban Journal of Medicinal Plants. Revista Cubana de Plantas Medicinales, 20(1)
Introduction - indicators and references related to a better positioning of the cuban scientific production in Google Scholar and in particular for the Cuban Journal of Medicinal Plants were conceptualized 
Objective - to analyze the Google Scholar`s and WEB´s presence of the Cuban Journal of Medicinal Plants and on the basis of these findings make recommendations to improve the visibility of the journal in Google Scholar 
Methods - to analyze the presence in Google Scholar Publish & Perish was used. A SEO analysis was conducted with Seo Quake. In order to perform specific searches on the entire analyzed collection a Google specialized search engine was prepared 
Results - the presence in Google Scholar of the Cuban Journal of Medicinal Plants throughout its history was characterized. The magazine has maintained an average of 93.5 citations per year and 2.79 citations per article. The data indicate that from 2010 a tendency to increase the number of items that are not mentioned is observed. The magazine is well positioned both in Google, Alexa and Bing although the number of external links is low. 
Conclusions - the results demonstrate the importance of establishing a SEO strategy to allows a better visibility that can increase the number of citations per article: Based on the results it can be concluded that no completion of the three standards relating to electronic journals identified by Latindex is not evident.

Haddaway, N.R., Collins, A.M., Coughlin, D., Kirk, S. (2015). The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching. PloS one, 10(9), e0138237DOI: http://dx.doi.org/10.1371/journal.pone.0138237. 
Google Scholar (GS), a commonly used web-based academic search engine, catalogues between 2 and 100 million records of both academic and grey literature (articles not formally published by commercial academic publishers). Google Scholar collates results from across the internet and is free to use. As a result it has received considerable attention as a method for searching for literature, particularly in searches for grey literature, as required by systematic reviews. The reliance on GS as a standalone resource has been greatly debated, however, and its efficacy in grey literature searching has not yet been investigated. Using systematic review case studies from environmental science, we investigated the utility of GS in systematic reviews and in searches for grey literature. Our findings show that GS results contain moderate amounts of grey literature, with the majority found on average at page 80. We also found that, when searched for specifically, the majority of literature identified using Web of Science was also found using GS. However, our findings showed moderate/poor overlap in results when similar search strings were used in Web of Science and GS (10–67%), and that GS missed some important literature in five of six case studies. Furthermore, a general GS search failed to find any grey literature from a case study that involved manual searching of organisations’ websites. If used in systematic reviews for grey literature, we recommend that searches of article titles focus on the first 200 to 300 results. We conclude that whilst Google Scholar can find much grey literature and specific, known studies, it should not be used alone for systematic review searches. Rather, it forms a powerful addition to other traditional search methods. In addition, we advocate the use of tools to transparently document and catalogue GS search results to maintain high levels of transparency and the ability to be updated, critical to systematic reviews.


Harzing, A.W., Mijnhardt, Wilfred (2015). Proof over promise: Towards a more inclusive ranking of Dutch academics in Economics & Business. Scientometrics, 102(1), 727-749 DOI: http://dx.doi.org/10.1007/s11192-014-1370-z. T
he Dutch Economics top-40, based on publications in ISI listed journals, is—to the best of our knowledge—the oldest ranking of individual academics in Economics and is well accepted in the Dutch academic community. However, this ranking is based on publication volume, rather than on the actual impact of the publications in question. This paper therefore uses two relatively new metrics, the citations per author per year (CAY) metric and the individual annual h-index (hIa) to provide two alternative, citation-based, rankings of Dutch academics in Economics & Business. As a data source, we use Google Scholar instead of ISI to provide a more comprehensive measure of impact, including citations to and from publications in non-ISI listed journals, books, working and conference papers. The resulting rankings are shown to be substantially different from the original ranking based on publications. Just like other research metrics, the CAY or hIa-index should never be used as the sole criterion to evaluate academics. However, we do argue that the hIa-index and the related CAY metric provide an important additional perspective over and above a ranking based on publications in high impact journals alone. Citation-based rankings are also shown to inject a higher level of diversity in terms of age, gender, discipline and academic affiliation and thus appear to be more inclusive of a wider range of scholarship.




Jamali, HR., Nabavi, M
. (2015). Open access and sources of full-text articles in Google Scholar in different subject fields. Scientometrics, 105(3), 1635-1651 DOI: http://dx.doi.org/10.1007/s11192-015-1642-2 . 
Google Scholar, a widely used academic search engine, plays a major role in finding free full-text versions of articles. But little is known about the sources of full-text files in Google Scholar. The aim of the study was to find out about the sources of full-text items and to look at subject differences in terms of number of versions, times cited, rate of open access availability and sources of full-text files. Three queries were created for each of 277 minor subject categories of Scopus. The queries were searched in Google Scholar and the first ten hits for each query were analyzed. Citations and patents were excluded from the results and the time frame was limited to 2004–2014. Results showed that 61.1 % of articles were accessible in full-text in Google Scholar; 80.8 % of full-text articles were publisher versions and 69.2 % of full-text articles were PDF. There was a significant difference between the means of times cited of full text items and non-full-text items. The highest rate of full text availability for articles belonged to life science (66.9 %). Publishers’ websites were the main source of bibliographic information for non-full-text articles. For full-text articles, educational (edu, ac.xx etc.) and org domains were top two sources of full text files. ResearchGate was the top single website providing full-text files (10.5 % of full-text articles). 

Maia, J.L., Di Serio, L.C., Alves Filho, A.G. (2015). Bibliometric research on strategy as practice: exploratory results and source comparison. Sistemas & Gestão, 10(4), 654-669. DOI: http://dx.doi.org/10.20985/1980-5160.2015.v10n4.662
O objetivo deste trabalho é esboçar um panorama geral da produção científica no campo da Estratégia como Prática (ECP), recuperando e expandindo a pesquisa bibliométrica, trazendo novos aspectos e replicando tal trabalho utilizando o Google Scholar. Sobre a ECP, os resultados sinalizam que: (1) ela é um campo de pesquisa jovem, com maior parte de publicações após 2007; (2) Paula Jarzabkowski e Richard Whittington são os autores mais profícuos, embora a pesquisa do Google Scholar tenha sugerido uma grande dispersão; (3) “Estratégia” e “Prática” são os principais termos na pesquisa do Web of Science, enquanto o Google Scholar indica grande densidade de termos relacionados; (4) ambos os estudos indicam que a produção da ECP não tem sido publicada em periódicos clássicos, e há produção relevante em língua não inglesa e congressos. Sobre as fontes de pesquisa: (1) as limitações de quantidade do Google Scholar não permitem calcular indicadores clássicos de bibliometria; (2) o Google Scholar gerou base com mais de 30 vezes mais resultados que o Web of Science; (3) o Google Scholar gerou base muito mais dispersa e diversa e; (4) há preocupações em usar o Scholar como fonte de informações: 15% dos documentos não possuem data de publicação e 27% não possuem fonte.
Mikki, S., Zygmuntowska, M., Gjesdal, ØL., Al Ruwehy, HA. (2015). Digital Presence of Norwegian Scholars on Academic Network Sites—Where and Who Are They?. PLoS ONE 10(11): e0142709. DOI: http://dx.doi.org/10.1371/journal.pone.0142709 . 
The use of academic profiling sites is becoming more common, and emerging technologies boost researchers’ visibility and exchange of ideas. In our study we compared profiles at five different profiling sites. These five sites are ResearchGate, Academia.edu, Google Scholar Citations, ResearcherID and ORCID. The data set is enriched by demographic information including age, gender, position and affiliation, which are provided by the national CRIS-system in Norway. We find that approximately 37% of researchers at the University of Bergen have at least one profile, the prevalence being highest (> 40%) for members at the Faculty of Psychology and the Faculty of Social Sciences. Across all disciplines, ResearchGate is the most widely used platform. However, within Faculty of Humanities, Academia.edu is the preferred one. Researchers are reluctant to maintain multiple profiles, and there is little overlap between different services. Age turns out to be a poor indicator for presence in the investigated profiling sites, women are underrepresented and professors together with PhD students are the most likely profile holders. We next investigated the correlation between bibliometric measures, such as publications and citations, and user activities, such as downloads and followers. We find different bibliometric indicators to correlate strongly within individual platforms and across platforms. There is however less agreement between the traditional bibliometric and social activity indicators.

Moed, HF.,  Bar-Ilan, J., Halevi, G. (2015). Comparing source coverage, citation counts and speed of indexing in Google Scholar and Scopus. arXiv:1512.05741.
An analysis of 1,200 target articles in 12 journals in 6 subject fields and of 7,000 citations to 36 top cited articles found in virology and chemistry a ratio of Google Scholar (GS) over Scopus citation counts between 0.8 and 1.5, in Chinese studies between 1.8 and 2.8, in computational linguistics between 2 and 4, and in political science journals between 3 and 4. Open access journals show higher ratios than their non-OA counterparts. Unique GS sources come from Google Books and/or from large book publishers, and from large disciplinary and institutional repositories. Unique Scopus sources are mainly books and Chinese journals. There is a huge dispersion in GS source titles and web links. The citation impact of documents in surplus sources covered in GS but not in Scopus and vice versa is some 80 per cent lower than that of documents in sources indexed in both. Pearson R between GS and Scopus citation counts at the article level are in all 12 target journals above 0.8, and for 8 journals above 0.9. Effect of double citation counts due to multiple citations with identical or substantially similar meta data occurs in less than 2 per cent of cases. In GS, the trade-off between data quality and indexing speed seems to be in favor of the latter. A median Scopus indexing delay of two months compared to GS is largely though not exclusively due to missing cited references in articles in press. Pros and cons of article-based and concept-based citation indexes are discussed.
Orduña-Malea, E., Delgado López-Cózar, E. (2015). The dark side of Open Access in Google and Google Scholar: the case of Latin-American repositories. Scientometrics, 102(1), 829-846. DOI: http://dx.doi.org/10.1007/s11192-014-1369-5 . 
Since repositories are a key tool in making scholarly knowledge open access (OA), determining their web presence and visibility on the Web (both are proxies of web impact) is essential, particularly in Google (search engine par excellence) and Google Scholar (a tool increasingly used by researchers to search for academic information). The few studies conducted so far have been limited to very specific geographic areas (USA), which makes it necessary to find out what is happening in other regions that are not part of mainstream academia, and where repositories play a decisive role in the visibility of scholarly production. The main objective of this study is to ascertain the web presence and visibility of Latin American repositories in Google and Google Scholar through the application of page count and web mention indicators respectively. For a sample of 137 repositories, the results indicate that the indexing ratio is low in Google, and virtually nonexistent in Google Scholar; they also indicate a complete lack of correspondence between the repository records and the data produced by these two search tools. These results are mainly attributable to limitations arising from the use of description schemas that are incompatible with Google Scholar (repository design) and the reliability of web mention indicators (search engines). We conclude that neither Google nor Google Scholar accurately represent the actual size of OA content published by Latin American repositories; this may indicate a non-indexed, hidden side to OA, which could be limiting the dissemination and consumption of OA scholarly literature.

Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., Delgado López-Cózar, E. (2015). Methods for estimating the size of Google Scholar. Scientometrics, 104(3), 931-949 . DOI: http://dx.doi.org/10.1007/s11192-015-1614-6. 
The emergence of academic search engines (mainly Google Scholar and Microsoft Academic Search) that aspire to index the entirety of current academic knowledge has revived and increased interest in the size of the academic web. The main objective of this paper is to propose various methods to estimate the current size (number of indexed documents) of Google Scholar (May 2014) and to determine its validity, precision and reliability. To do this, we present, apply and discuss three empirical methods: an external estimate based on empirical studies of Google Scholar coverage, and two internal estimate methods based on direct, empty and absurd queries, respectively. The results, despite providing disparate values, place the estimated size of Google Scholar at around 160–165 million documents. However, all the methods show considerable limitations and uncertainties due to inconsistencies in the Google Scholar search functionalities.
Ortega, J.L. (2015). Diferencias y evolución del impacto académico en los perfiles de Google Scholar Citations: Una aplicación de árboles de decisión. Revista Española de Documentación Científica, 38 (4): e102DOI: http://dx.doi.org/10.3989/redc.2015.4.1225. 
El propósito de este artículo es analizar la producción e impacto de más de 3000 perfiles tomados de Google Scholar Citations con el fin de identificar qué segmentos (por género, puestos académicos y disciplinas) son más exitosos en términos de impacto científico. Este análisis se afrontó tanto desde una perspectiva estática como longitudinal. Los árboles de decisión fueron usados para detectar las variables más importantes para agrupar perfiles con un mayor número de citas por artículo e índice h. Resultados muestran que la carrera académica es el factor más importante para conseguir citas y mejorar el índice h. Los investigadores más veteranos son así los que ocupan las primeras posiciones, mientras que los jóvenes investigadores describen curriculums en ciernes. Por el contrario, estos resultados cambian cuando el crecimiento de los perfiles es observado. Así los curriculums más jóvenes son los que experimentan un crecimiento más fuerte, mientras que los más veteranos muestran signos de estabilización y estancamiento. Se concluye que los investigadores con una carrera estable pertenecientes a las ciencias de la vida tienen mejor impacto que los jóvenes investigadores de humanidades y ciencias sociales, a pesar de que estos últimos son los que más rápido crecen en número de citas por documento
Ortega, J.L. (2015). Disciplinary differences in the use of academic social networking site.  Online Information Review, 39(4), 520-536DOI: http://dx.doi.org/10.1108/OIR-03-2015-0093 . 
Purpose – The purpose of this paper is to detect and describe disciplinary differences in the users and use of several social networking sites by scientists.
Design/methodology/approach – Consejo Superior de Investigaciones Científicas (CSIC) (Spanish National Research Council) researchers registered in the most currently relevant academic social network sites (Google Scholar Citations, Academia.edu, ResearchGate (RG) and Mendeley) were analysed. In total, 6,132 profiles were classified according the eight research areas of the CSIC.
Findings – Results show that Academia.edu is massively populated by humanists and social scientists, while RG is popular among biologists. Disciplinary differences are observed across every platform. Thus, scientists from the humanities and social sciences and natural resources show a significant activity contacting other members. On the contrary, biologists are more passive using social tools.
Originality/value – This is the first study that analyses the disciplinary performance of a same sample of researchers on a varied number of academic social sites, comparing their numbers across web sites.
Ortega, J.L. (2015). How is an academic social site populated? A demographic study of Google Scholar Citations population, Scientometrics, 104(1), 1-18DOI: http://dx.doi.org/10.1007/s11192-015-1593-7 . 
This paper intends to describe the population evolution of a scientific information web service during 2011–2012. Quarterly samples from December 2011 to December 2012 were extracted from Google Scholar Citations to analyse the number of members, distribution of their bibliometric indicators, positions, institutional and country affiliations and the labels to describe their scientific activity. Results show that most of the users are young researchers, with a starting scientific career and mainly from disciplines related to information sciences and technologies. Another important result is that this service is settled by waves emanating from specific institutions and countries. This work concludes that this academic social network presents some biases in the population distribution that does not make it representative of the real scientific population .

Ortega, J.L. (2015).  Relationship between altmetric and bibliometric indicators across academic social sites: The case of CSIC's members. Journal of Informetrics, 9(1), 39-49. DOI: http://dx.doi.org/10.1016/j.joi.2014.11.004. 
This study explores the connections between social and usage metrics (altmetrics) and bibliometric indicators at the author level. It studies to what extent these indicators, gained from academic sites, can provide a proxy for research impact. Close to 10,000 author profiles belonging to the Spanish National Research Council were extracted from the principal scholarly social sites: ResearchGate, Academia.edu and Mendeley and academic search engines: Microsoft Academic Search and Google Scholar Citations. Results describe little overlapping between sites because most of the researchers only manage one profile (72%). Correlations point out that there is scant relationship between altmetric and bibliometric indicators at author level. This is due to the almetric ones are site-dependent, while the bibliometric ones are more stable across web sites. It is concluded that altmetrics could reflect an alternative dimension of the research performance, close, perhaps, to science popularization and networking abilities, but far from citation impact.
Serenko, A., Dumay, J. (2015). Citation classics published in knowledge management journals. Part I: articles and their characteristics. Journal of Knowledge Management, 19(2), 401-431. DOI: http://dx.doi.org/10.1108/JKM-06-2014-0220 . 
Purpose – The purpose of this study is to develop a list of citation classics published in knowledge management (KM) journals and to analyze the key attributes and characteristics of the selected articles to understand the development of the KM discipline.
Design/methodology/approach – This study identifies 100 citation classics from seven KM-centric journals based on their citation impact reported by Google Scholar and analyzes their attributes.
Findings – The KM discipline is at the pre-science stage because of the influence of normative studies espousing KM practice. However, KM is progressing toward normal science and academic maturity. While the discipline does not exhibit the signs of the superstar effect, scholars from the USA and UK have made the most significant impact on the development of the KM school of thought. KM scholars should be more engaged in international collaboration.
Practical implications – Practitioners played a key role in the development of the KM discipline and thus there is an opportunity to develop more scientific research approaches based on critical and performative research agenda.
Originality/value – The study is novel and a must read for KM scholars because it is the first to comprehensively analyze the ideas that are the origins of the KM discipline.
 
Serenko, A., Dumay, J. (2015). Citation Classics Published in Knowledge Management Journals. Part II: Studying Research Trends and Discovering the Google Scholar Effect. Journal of Knowledge Management, 19(6). DOI: http://dx.doi.org/10.1108/JKM-02-2015-0086 . 
Purpose – The purpose of this study was to discover growing, stable, and declining knowledge management (KM) research trends.
Methodology – The purpose of this study was to discover growing, stable, and declining knowledge management (KM) research trends. 
Findings – This research has two findings that were not theoretically expected. First, a majority of KM citation classics exhibit a bimodal citation distribution peak. Second, there is a growing number of citations for all research topics. These unexpected findings warranted further theoretical elaboration and empirical investigation. The analysis of erroneous citations and a five-year citation trend (2009–2013) reveals that the continuously growing volume of citations may result from what the authors call the Google Scholar Effect
Research implications – The results from this study open up two significant research opportunities. First, more research is needed to understand the impact Google Scholar is having on domains beyond KM. Second, more comprehensive research on the impact of erroneous citations is required because these have the most potential for damaging academic discourse and reputation.
Practical implications – Researchers need to be aware of how technology is changing their profession and their citation behavior because of the pressure from the contemporary “publish or perish” environment, which prevents research from being state-of-the-art. Similarly, KM reviewers and editors need to be more aware of the pressure and prevalence of miscitations and take action to raise awareness and to prevent miscitations from occurring. 
Originality – This study is important from a scientometric research perspective as part of a growing research field using Google Scholar to measure the impact and power it has in influencing what gets cited and by whom. 

Sittig, D.F., McCoy, A.B., Wright, A., Lin, J. (2015). Developing an Open-Source Bibliometric Ranking Website Using Google Scholar Citation Profiles for Researchers in the Field of Biomedical Informatics. Studies in health technology and informatics, 216, 1004-1004.. DOI: http://dx.doi.org/10.3233/978-1-61499-564-7-1004 . 
We developed the Biomedical Informatics Researchers ranking website (rank.informatics-review.com) to overcome many of the limitations of previous scientific productivity ranking strategies. The website is composed of four key components that work together to create an automatically updating ranking website: (1) list of biomedical informatics researchers, (2) Google Scholar scraper, (3) display page, and (4) updater. The site has been useful to other groups in evaluating researchers, such as tenure and promotions committees in interpreting the various citation statistics reported by candidates. Creation of the Biomedical Informatics Researchers ranking website highlights the vast differences in scholarly productivity among members of the biomedical informatics research community.  
Știrbu, S., Thirion, P., Schmitz, S., Haesbroeck, G., Greco, N. (2015). The utility of Google Scholar when searching geographical literature: comparison with three commercial bibliographic databases. The Journal of Academic Librarianship, 41(3), 322-329. DOI: http://dx.doi.org/10.1016/j.acalib.2015.02.013 . 
This study aims to highlight what benefits, if any, Google Scholar (GS) has for academic literature searches in the field of geography, compared to three commercial bibliographic databases: Web of Science (WoS), FRANCIS (multidisciplinary databases) and GeoRef (specialized in geosciences). This study focuses exclusively on evaluating the results, and not the features, of GS and the databases under examination. To ensure a valid comparison, identical bibliographic searches were applied using each of the four bibliographic tools. To exclude automatic variations of the ten keywords tested, they were placed between quotation marks and searched only in the “title” field of the respective search tools' interfaces. The results were limited to bibliographic references published from 2005 to 2009. In order to assess the repeatability of the results, the exact same process was repeated monthly between November 2010 and July 2011. Initially the whole set of results was analyzed, after which the search results for two keywords (selected since they yielded a manageable number of results) were studied in more detail.
The results indicate that GS search results show a large degree of overlap with those of the other databases but, moreover, yield numerous unique hits, which should be useful to researchers in both the fields of human and physical geography. GS leads the other tools widely on number of results, independently of keyword, subfield, year of publication, or time of search.   
 
Symonenk, T. (2015). Bibliometric systems Scopus and Google Scholar: Areas of use. Bibliotechnyi visnyk, 2 (226),10-13.  
The comparative analysis of bibliometric platforms such as Scopus and Google Scholar was conducted. The correlation of bibliometric indicators of scientists, according to these systems, was demonstrated and the areas of their rational use were defined. Also, the expediency of combining qualitative and quantitative methods of evaluating research effectiveness was ascertained. The system «Bibliometryka Ukrayinskoi Nauky» (the source base for expertise of research effectiveness of scientists and scientific groups) was considered.


Wildgaard, L. (2015). A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar. Scientometrics, 104(3), 873-906. DOI: http://dx.doi.org/10.1007/s11192-015-1608-4 . 
Author-level bibliometric indicators are becoming a standard tool in research assessment. It is important to investigate what these indicators actually measure to assess their appropriateness in scholar ranking and benchmarking average individual levels of performance. 17 author-level indicators were calculated for 512 researchers in Astronomy, Environmental Science, Philosophy and Public Health. Indicator scores and scholar rankings calculated in Web of Science (WoS) and Google Scholar (GS) were analyzed. The indexing policies of WoS and GS were found to have a direct effect on the amount of available bibliometric data, thus indicator scores and rankings in WoS and GS were different, correlations between 0.24 and 0.99. High correlation could be caused by scholars in bottom rank positions with a low number of publications and citations in both databases. The hg indicator produced scholar rankings with the highest level of agreement between WoS and GS and rankings with the least amount of variance. Expected average performance benchmarks were influenced by how the mean indicator value was calculated. Empirical validation of the aggregate mean h-index values compared to previous studies resulted in a very poor fit of predicted average scores. Rankings based on author-level indicators are influenced by (1) the coverage of papers and citations in the database, (2) how the indicators are calculated and, (3) the assessed discipline and seniority. Indicator rankings display the visibility of the scholar in the database not their impact in the academic community compared to their peers. Extreme caution is advised when choosing indicators and benchmarks in scholar rankings.   
 
Zarifmahmoudi, L., Jamali, J., Sadeghi, R. (2015). Google Scholar journal metrics: Comparison with impact factor and SCImago journal rank indicator for nuclear medicine journals. Iranian Journal of Nuclear Medicine, 23(1), 8-14.  
Introduction: In the current study, we compared h5-index provided by Google Scholar (GS), impact factor (IF) provided by web of sciences (WOS), and SCImago journal rank indicator (SJR) provided by SCOPUS for quality assessment of nuclear medicine  journals.
Methods: 2013 h5-index, 2012 IF, and 2011 SJR of nuclear medicine journals were extracted from their publishers namely GS, WOS, and SCOPUS. Rank of each journal according to each index was provided. Spearman correlation was used for evaluation of  the correlation between metrics.
Results: Overall 22 journals were identified. Spearman correlation coefficients between h5-index and other journal metrics were 0.907 for 2012 IF, 0.979 for 2011 JCR, and 0.978 for 2011 SCOPUS h-index (all p-values<0.00001). Wilcoxon signed ranks test  showed no statistically meaningful difference between rankings according to h5-index and other journal metrics (p values of 0.589, 0.565, and 0.542 for 2012 IF, 2011 SJR, and 2011 SCOPUS h-index respectively).
Conclusion: The new GS journal metrics are reliable tools for quality assessment of the nuclear medicine journals. In our opinion, h5-index, IF, and SJR should be used in a combination as their combination would give a more holistic view of journal quality. Development of new journal metrics in addition to SJR and IF by GS should be welcomed by the scientific community.

2014 [Go back]

Abdullah, A., Thelwall, M. (2014). Can the Impact of non-Western Academic Books be Measured? An investigation of Google Books and Google Scholar for Malaysia. Journal of the Association for Information Science and Technology, 65(12), 2498–2508. DOI: http://dx.doi.org/10.1002/asi.23145 . 
Citation indicators are increasingly used in book-based disciplines to support peer-review in the evaluation of authors and to gauge the prestige of publishers. However, since global citation databases seem to offer weak coverage of books outside the West, it is not clear whether the influence of non-Western books can be assessed with citations. To investigate this, citations were extracted from Google Books and Google Scholar to 1357 Arts, Humanities and Social Sciences (AHSS) books published by five university presses during 1961-2012 in one non-Western nation, Malaysia. A significant minority of the books (23% in Google Books and 37% in Google Scholar, 45% in total) had been cited, with a higher proportion cited if they were older or in English. The combination of Google Books and Google Scholar is therefore recommended, with some provisos, for non-Western countries seeking to differentiate between books with some impact and books with no impact, to identify the highly cited works or to develop an indicator of academic publisher prestige.   

Ale Ebrahim, N., Salehi, H., Embi, M. A., Bakhtiyari, K., Danaee, M., Mohammadjafari, M., Zavvari, A., Shakiba, M., Shahbazi-Moghadam, M. (2014). Equality of Google Scholar with Web of Science Citations: Case of Malaysian Engineering Highly Cited Papers. Modern Applied Science, 8(5), 63-69. DOI: http://dx.doi.org/105539/mas.v8n5p63 . 
This study uses citation analysis from two citation tracking databases, Google Scholar (GS) and ISI Web of Science, in order to test the correlation between them and examine the effect of the number of paper versions on citations. The data were retrieved from the Essential Science Indicators and Google Scholar for 101 highly cited papers from Malaysia in the field of engineering. An equation for estimating the citation in ISI based on Google scholar is offered. The results show a significant and positive relationship between both citation in Google Scholar and ISI Web of Science with the number of versions. This relationship is higher between versions and ISI citations (r = 0.395, p<0.01) than between versions and Google Scholar citations (r = 0.315, p<0.01). Free access to data provided by Google Scholar and the correlation to get ISI citation which is costly, allow more transparency in tenure reviews, funding agency and other science policy, to count citations and analyze scholars’ performance more precisely.
Acharya, A., Verstak, A., Suzuki, H., Henderson, S., Iakhiaev, M., Chiung Yu Lin, C., Shetty, N. (2014). Rise of the Rest: The Growing Impact of Non-Elite Journals. arXiv preprint arXiv:1410.2217v1. 
In this paper, we examine the evolution of the impact of non-elite journals. We attempt to answer two questions. First, what fraction of the top-cited articles are published in non-elite journals and how has this changed over time. Second, what fraction of the total citations are to non-elite journals and how has this changed over time.  We studied citations to articles published in 1995-2013. We computed the 10 most-cited journals and the 1000 most-cited articles each year for all 261 subject categories in Scholar Metrics. We marked the 10 most-cited journals in a category as the elite journals for the category and the rest as non-elite. There are two conclusions from our study. First, the fraction of top-cited articles published in non-elite journals increased steadily over 1995-2013. While the elite journals still publish a substantial fraction of high-impact articles, many more authors of well-regarded papers in diverse research fields are choosing other venues. The number of top-1000 papers published in non-elite journals for the representative subject category went from 149 in 1995 to 245 in 2013, a growth of 64%. Looking at broad research areas, 4 out of 9 areas saw at least one-third of the top-cited articles published in non-elite journals in 2013. For 6 out of 9 areas, the fraction of top-cited papers published in non-elite journals for the representative subject category grew by 45% or more. Second, now that finding and reading relevant articles in non-elite journals is about as easy as finding and reading articles in elite journals, researchers are increasingly building on and citing work published everywhere. Considering citations to all articles, the percentage of citations to articles in non-elite journals went from 27% in 1995 to 47% in 2013. Six out of nine broad areas had at least 50% of citations going to articles published in non-elite journals in 2013. 
Bensman, S.J., Daugherty, A., Smolinsky, L.J., Sage, D.S., Katz, J.S. (2014). Power-law distributions, the h-index, and Google Scholar (GS) citations: a test of their relationship with economics Nobelists. arXiv preprint arXiv:1411.0928. 
This paper presents proof that Google Scholar (GS) can construct documentary sets relevant for evaluating researchers' works. Nobelists in economics were the researchers under analysis, and two types of tests of the GS cites to their works were performed: distributional and semantic. Distributional tests found that the GS cites to the laureates' works conformed to the power-law model with an asymptote or "tail" conterminous with their h-index demarcating their core oeuvre, validating both GS and the h-index. Semantic tests revealed that their works highest in GS cites were on topics for which they were awarded the prize.

Bensman, S.J., Smolinsky, L.J., Sage, D.S. (2014). Comparison of the Research Effectiveness of Chemistry Nobelists and Fields Medalist Mathematicians with Google Scholar: the Yule-Simon Model. arXiv preprint arXiv:1404.4904. 
This paper uses the Yule-Simon model to estimate to what extent the work of chemistry Nobelists and Fields medalist mathematicians is incorporated into the knowledge corpus of their disciplines as measured by Google Scholar inlinks. Due to differences in the disciplines and prizes, it finds that the work of chemistry Nobelists is better incorporated than that of Fields medalists.
Bodlaender, H.L., van Kreveld, M. (2014). Google Scholar makes it Hard-the complexity of organizing one's publications. arXiv preprint arXiv:1410.3820. 
With Google Scholar, scientists can maintain their publications on personal profile pages, while the citations to these works are automatically collected and counted. Maintenance of publications is done manually by the researcher herself, and involves deleting erroneous ones, merging ones that are the same but which were not recognized as the same, adding forgotten co-authors, and correcting titles of papers and venues. The publications are presented on pages with 20 or 100 papers in the web page interface from 2012--2014. The interface does not allow a scientist to merge two version of a paper if they appear on different pages. This not only implies that a scientist who wants to merge certain subsets of publications will sometimes be unable to do so, but also, we show in this note that the decision problem to determine if it is possible to merge given subsets of papers is NP-complete.
Breuer, P.T., Bowen, J.P. (2014). Empirical Patterns in Google Scholar Citation Counts. arXiv preprint arXiv:1401.1861. 
Scholarly impact may be metricized using an au- thor’s total number of citations as a stand-in for real worth, but this measure varies in applicability between disciplines. The detail of the number of citations per publication is nowadays mapped in much more detail on the Web, exposing certain empirical patterns. This paper explores those patterns, using the citation data from Google Scholar for a number of authors. 

Chan, J.Y., Chan, K.C., Tong, J. Y., Zhang, F. F. (2014). Using Google Scholar citations to rank accounting programs: a global perspective. Review of Quantitative Finance and Accounting, 1-27. DOI: http://dx.doi.org/10.1007/s11156-014-0493-x 
We conduct an assessment on accounting program research performance based on Google Scholar citations for all articles from a set of 23 quality accounting journals during 1991–2010. Our work is a new approach in accounting by directly measuring the impact of the faculty research in accounting programs. We find that the top-5 accounting programs are the University of Pennsylvania, the University of Chicago, Stanford University, the University of Michigan, and Harvard University. These top programs produce a large number of high impact articles. In addition, using the mean citations from all articles in a journal, we find that the Review of Accounting Studies (RAST) is a top-5 journal, replacing Contemporary Accounting Research (CAR).
Danesh, F., Fattahi, R., Dayani, M. (2014). Scientific and Professional Performance, Web Presence of Iranian Knowledge and Information Science Academics and Their Publications in Google Scholar, ISCI & Google. Journal of Information Processing and Management, 30(1), 41-60.  
Web presence is one of the areas of research in the field of webometrics. Due to lack of precise information on the presence of the Knowledge and Information Science (KIS) faculty members in Iran, this study aims to address this problem. It studies the level and quality of the presence of the LIS faculty and its relation to their professional effectiveness, focusing on their academic and professional performance. This study is an applied research, taking two approaches: descriptive survey and webometrics. The study participants are the KIS faculty members in Iran who hold a PhD degree, have senior lecturer (assistant professor) position and positions above that, and have scientific publications and web citations. A checklist was used to collect the webrelated data on their presence, and a questionnaire was designed to collect the data on their academic and professional activities. Three databases were used to collect the web-related data, namely Google, Google Scholar and the Iran Science Citation Index. The data was analyzed using statistical analysis and tests of descriptive and interpretive types. From the web presence perspective, the findings revealed that besides experienced academics, younger academic members have remarkable web presence. The findings showed that there is a meaningful relationship between the level of presence of the studyparticipants and their publications. KIS academics of Ferdowsi University of Mshhad and Shahid Chamran University of Ahvaz have gained highest positions according to publication web presence in Google. Also, scientific databases, self citations and university websites have got the first to third ranks in order to publication web presence themes. In addition, results showed us publishing Farsi and English articles in credible refereed journals is the most important theme in scientific and professional performance.
Delgado López-Cózar, E., Orduña-Malea, E., Jiménez-Contreras, E., Ruiz-Pérez, R. (2014). H Index Scholar: el índice h de los profesores de las universidades públicas españolas en humanidades y ciencias sociales. El Profesional de la Información, 23(1), 87-94. DOI: http://dx.doi.org/10.3145/epi.2014.ene.11 . 
The H-Index Scholar is a bibliometric index that measures the productivity and scientific impact of the academic production in humanities and social sciences by professors and researchers at public Spanish universities. The methodology consisted of counting their publications and citations received in Google Scholar. The main features and characteristics of the index are explained. Despite technical and methodological problems that Google Scholar might have as a source of information, the authors estimate that they do not affect substantially the calculated h and g indexes, probably being the error lower than 10%. The total population analyzed was 40,993 researchers, but data are displayed only for 13,518 researchers, the ones located in the first tertile of their respective areas.
Delgado López-Cózar, E., Robinson-García, N., Torres-Salinas, D. (2014). The Google Scholar Experiment: how to index false papers and manipulate bibliometric indicators. Journal of the American Society for Information Science and Technology,  65(3), 446–454. DOI: http://dx.doi.org/10.1002/asi.23056  
Google Scholar has been well received by the research community. Its promises of free, universal, and easy access to scientific literature coupled with the perception that it covers the social sciences and the humanities better than other traditional multidisciplinary databases have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: 2 new bibliometric products that offer citation data at the individual level and at journal level. In this article, we show the results of an experiment undertaken to analyze Google Scholar's capacity to detect citation-counting manipulation. For this, we uploaded 6 documents to an institutional web domain that were authored by a fictitious researcher and referenced all the publications of the members of the EC3 research group at the University of Granada. The detection by Google Scholar of these papers caused an outburst in the number of citations included in the Google Scholar Citations profiles of the authors. We discuss the effects of such an outburst and how it could affect the future development of such products, at both the individual level and the journal level, especially if Google Scholar persists with its lack of transparency.
Doemeland, D., Trevino, J. (2014). Which World Bank reports are widely read ?. World Bank Policy Research Working Paper, (6851). 
Knowledge is central to development. The World Bank invests about one-quarter of its budget for country services in knowledge products. Still, there is little research about the demand for these knowledge products and how internal knowledge flows affect their demand. About 49 percent of the World Bank’s policy reports, which are published Economic and Sector Work or Technical Assistance reports, have the stated objective of informing the public debate or influencing the development community. This study uses information on downloads and citations to assesses whether policy reports meet this objective. About 13 percent of policy reports were downloaded at least 250 times while more than 31 percent of policy reports are never downloaded. Almost 87 percent of policy reports were never cited. More expensive, complex, multi-sector, core diagnostics reports on middle-income countries with larger populations tend to be downloaded more frequently. Multi-sector reports also tend to be cited more frequently. Internal knowledge sharing matters as cross support provided by the World Bank’s Research Department consistently increases downloads and citations

Fausto, S., Murakami, T.R.M. (2014).  Extracting and sharing data citations from Google Scholar for collaborative exploitation. STI 2014 - STI 2014 Leiden - 19th International Conference on Science & Tecnhnology Indicators. 

Franco Pérez, Á.M., Sanz Valero, J., Wanden-Berghe Lozano, C., Melian Fleitas, L. (2014). The Iberoamerican scientific production in nutritional sciences: The indexation in PubMed and Google Scholar. Nutrición Hospitalaria, 30(5). DOI: http://dx.doi.org/10.3305%2Fnutr+hosp.v30in05.7723 . 
Objective: Analyze by bibliometric technique, the Iberoamerican scientific literature related to the nutritional sciences and retrieved on main search engines with free access through Internet (PubMed and Google Scholar). 
Method: Bibliometric analysis of scientific production recovered in the different selected search tools. The data were obtained by applying to each of them, a composed search equation according to the scheme: Population (neoplasms), Intervention (nutritional status), Outcome (quality of life). 
Results: 789 references were reviewed, 604 of those were papers published in 277 journals, presenting 20 or more references: Supportive Care in Cancer 27 (4,47%; CI95% 2,82-6,12) and Clinical Nutrition 20 (3,31%; CI95% 1,88-4,74). Mean age of documents: 8,08 ± 6,40 (CI95% 7,63-8,53), median 6 years (Burton Kleber Index), maximum 34 years and Price Index of 43.90%. The predominant geographical distribution among the authors was American, while the articles were written primarily in English. Dispersion of literature (Bradford Law): core (1st tertile), 22 journals (7,94%;  CI95% 4,76-11,13) with 202 articles published (33,44%; CI95% 29,68-37,21). Statistics related to the impact factor of the core: mean 4,033 ± 4,022 and maximum 18,038 (Journal of Clinical Oncology). 
Conclusions: The studied thematic continues in force according to the current indicators, with a dominance of English as language of publication and United States filiation. The most referenced journals matching with high Impact publications on nutritional sciences and oncology. Highlighting the presence of an Iberoamerican journal (Nutrición Hospitalaria) with a clear international vocation.
Haley, M.R. (2014). Ranking top economics and finance journals using Microsoft academic search versus Google scholar: How does the new publish or perish option compare?. Journal of the Association for Information Science and Technology, 65, 1079–1084. DOI: http://dx.doi.org/10.1002/asi.23080 
Recently, Harzing's Publish or Perish software was updated to include Microsoft Academic Search as a second citation database search option for computing various citation-based metrics. This article explores the new search option by scoring 50 top economics and finance journals and comparing them with the results obtained using the original Google Scholar-based search option. The new database delivers significantly smaller scores for all metrics, but the rank correlations across the two databases for the h-index, g-index, AWCR, and e-index are significantly correlated, especially when the time frame is restricted to more recent years. Comparisons are also made to the Article Influence score from eigenfactor.org and to the RePEc h-index, both of which adjust for journal-level self-citations.
Harzing, A.W. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013. Scientometrics, 98(1), 565–575. DOI: http://dx.doi.org/10.1007/s11192-013-0975-y
Harzing (Scientometrics, 2013) showed that between April 2011 and January 2012, Google Scholar has very significantly expanded its coverage in Chemistry and Physics, with a more modest expansion for Medicine and a natural increase in citations only for Economics. However, we do not yet know whether this expansion of coverage was temporary or permanent, nor whether a further expansion of coverage has occurred. It is these questions we set out to respond in this research note. We use a sample of 20 Nobelists in Chemistry, Economics, Medicine and Physics and track their h-index, g-index and total citations in Google Scholar on a monthly basis. Our data suggest that—after a period of significant expansion for Chemistry and Physics—Google Scholar coverage is now increasing at a stable rate. Google Scholar also appears to provide comprehensive coverage for the four disciplines we studied. The increased stability and coverage might make Google Scholar much more suitable for research evaluation and bibliometric research purposes than it has been in the past.


Karlsson, N. (2014). The crossroads of academic electronic availability: How well does Google Scholar measure up against a university-based metadata system in 2014? Current Science, 107(10), 1661–1665
Electronic availability of information resources has increasingly become an important part of everyday vocation of academic libraries. This puts impetus on the libraries to know more about the way in which electronic information is being dispersed and handled. The present article aims to comparatively evaluate Uppsala University library's own metadata system Summon with the free, publicly available equivalent Google Scholar (GS). The evaluation is based on Péter Jacsó's theories on database evaluation which puts focus on Summon and GS via the use and application of ten different criteria. The uses of precision and relevance criteria were also implemented as additional evaluation tools. The results indicate that at present GS has to be seen as a necessary complement in retrieving electronic information due to the fact that Summon is not yet fully functioning on all levels and that GS has a wider intake of information sources. The use of web-based academic search tools is now vital. Will the open access movement evolve with Google as the main actor and take over the scene leaving costly databases and search tools behind? This article deals with the economic implications of comparing the practical functions of a costly in-house information system with a public equivalent. It reveals the complex situation that a world-class university is in as regards to information resources and the digitization and economic issues that follow.
Khabsa, M., Giles, C.L. (2014). The Number of Scholarly Documents on the Public Web. PLOS ONE, 9(5). DOI: http://dx.doi.org/10.1371/journal.pone.0093949.
The number of scholarly documents available on the web is estimated using capture/recapture methods by studying the coverage of two major academic search engines: Google Scholar and Microsoft Academic Search. Our estimates show that at least 114 million English-language scholarly documents are accessible on the web, of which Google Scholar has nearly 100 million. Of these, we estimate that at least 27 million (24%) are freely available since they do not require a subscription or payment of any kind. In addition, at a finer scale, we also estimate the number of scholarly documents on the web for fifteen fields: Agricultural Science, Arts and Humanities, Biology, Chemistry, Computer Science, Economics and Business, Engineering, Environmental Sciences, Geosciences, Material Science, Mathematics, Medicine, Physics, Social Sciences, and Multidisciplinary, as defined by Microsoft Academic Search. In addition, we show that among these fields the percentage of documents defined as freely available varies significantly, i.e., from 12 to 50%.

Khurshid, Z. (2014). Measuring the Quality of Contributions of Saudi Authors to LIS Journals Using Journal Impact Factor (JIF), SCImago Journal Rank (SJR), and Google Scholar Metrics (GSM). The Serials Librarian: From the Printed Page to the Digital Age, 67(1), 81-98. DOI: http://dx.doi.org/10.1080/0361526X.2014.909755 .
This research study evaluates the quality of articles published by Saudi and expatriate authors in foreign Library and Information Science (LIS) journals using three popular metrics for ranking journals—Journal Impact Factor (JIF), SCImago Journal Rank (SJR), and Google Scholar Metrics (GSM). The reason for using multiple metrics is to see how closely or differently journals are ranked by the three different methods of citation analysis. However, the 2012 JIF list of journals is too small, almost half the size of the SJR and GSM lists, which inhibited one-to-one comparison among the impact factors of the thirty-six journals selected by Saudi authors for publishing articles. Only seventeen journals were found common to all the three lists, limiting the usefulness of the data. A basic problem is that Saudi LIS authors generally lack the level of competency in the English language required to achieve publication in the most prominent LIS journals. The study will have implications for authors, directors, and deans of all types of academic libraries; chairmen and deans of library schools; and the Saudi Library Association. Hopefully these entities will take necessary steps to prepare and motivate both academics and practicing librarians to improve the quality of their research and publications and thus get published in higher ranked journals.

Kim, H. (2014). An Investigation of Information Usefulness of Google Scholar in Comparison with Web of Science. 한국비블리아학회지, 25(3), 215-234. 
The purpose of this study is to investigate whether Google Scholar (GS) can substitute Web of Science (WoS) for those who don’t have access to the subscription-based indexing service and if users feel GS is useful for scholarly information. To achieve the research purpose, the study evaluates both quantitative and qualitative aspects of the two databases. The major results through statistical analysis show that GS indexes much more records and citations for LIS journals than WoS(p < .01), but users’ feedback about GS is not better than those about WoS.


Martín-Martín, A., Orduña-Malea, E., Ayllón, J.M.,  Delgado López-Cózar, E. (2014). Does Google  Scholar contain all highly cited documents (1950-2013)? Granada: EC3 Working Papers, 19: October 29, 2014.arXiv preprint arXiv:1410.8464.  
The study of highly cited documents on Google Scholar (GS) has never been addressed to date in a comprehensive manner. The objective of this work is to identify the set of highly cited documents in Google Scholar and define their core characteristics: their languages, their file format, or how many of them can be accessed free of charge. We will also try to answer some additional questions that hopefully shed some light about the use of GS as a tool for assessing scientific impact through citations. The decalogue of research questions is shown below: 

1. Which are the most cited documents in GS? 
2. Which are the most cited document types in GS? 
3. What languages are the most cited documents written in GS? 
4. How many highly cited documents are freely accessible? 
4.1 What file types are the most commonly used to store these highly cited documents? 
4.2 Which are the main providers of these documents? 
5. How many of the highly cited documents indexed by GS are also indexed by WoS? 
6. Is there a correlation between the number of citations that these highly cited documents have received in GS and the number of citations they have received in WoS? 
7. How many versions of these highly cited documents has GS detected? 
8. Is there a correlation between the number of versions GS has detected for these documents, and the number citations they have received? 
9. Is there a correlation between the number of versions GS has detected for these documents, and their position in the search engine result pages? 
10. Is there some relation between the positions these documents occupy in the search engine result pages, and the number of citations they have received?




Martín-Martín, A., Ayllón, J.M., Orduña-Malea, E., Delgado López-Cózar, E. (2014). Google Scholar Metrics 2014: a low cost bibliometric tool. Granada: EC3 Working Papers, 17: 11 July 2014. arXiv preprint arXiv:1407.2827.  
We analyse the main features of the third edition of Google Scholar Metrics (GSM), released in June 2014, focusing on its more important changes, strengths, and weaknesses. Additionally, we present some figures that outline the dimensions of this new edition, and we compare them to those of previous editions. Principal among these figures are the number of visualized publications, publication types, languages, and the maximum and minimum h5-index and h5-median values by language, subject area, and subcategory. This new edition is marked by continuity. There is nothing new other than the updating of the time frame (2009-2013) and the removal of some redundant subcategories (from 268 to 261) for English written publications. Google has just updated the data, which means that some of the errors discussed in previous studies still persist. To sum up, GSM is a minimalist information product with few features, closed (it cannot be customized by the user), and simple (navigating it only takes a few clicks). For these reasons, we consider it a 'low cost' bibliometric tool, and propose a list of features it should incorporate in order to stop being labeled as such. Notwithstanding the above, this product presents a stability in its bibliometric indicators that supports its ability to measure and track the impact of scientific publications.




Murakami, T.R.M., Fausto, S., de Araujo, R. F. (2014). Exploração colaborativa através do compartilhamento de dados de citações do Google Scholar. Liinc em Revista, 10(2).
The lack of indexing for titles of scientific journals in the Social Sciences and Humanities in commercial databases makes it difficult to carry out an investigation on their impact. Open Access and tools such as Google Scholar (GS) and software for data processing allow search and the recovery of article citations, which can be regarded as an alternative for the studies on the impact of scientific production published in these areas. This study presents a pilot project for sharing citation data from Brazilian journals for further collaborative research by the national scientometrics community with the aim of encouraging greater use of GS for bibliometric purposes.
Norman, R., Couto, F.M. (2014). Using Google Scholar to predict self citation: A case study in Health Economics. arXiv preprint arXiv:1406.5241. 
Metrics designed to quantify the influence of academics are increasingly used and easily estimable, and perhaps the most popular is the h index. Metrics such as this are however potentially impacted through excessive self citation. This work explores the issue using a group of researchers working in a well defined sub field of economics, namely Health Economics. It then employs self citation identification software, and identifies the characteristics that best predict self citation. This provides evidence regarding the scale of self citation in the field, and the degree to which self citation impacts on inferences about the relative influence of individual Health Economists. Using data from 545 Health Economists, it suggests self citation to be associated with the geographical region and longevity of the Health Economist, with early career researchers and researchers from mainland Europe and Australasia self citing most frequently.
Orduña-Malea, E., Delgado López-Cózar, E. (2014). Google Scholar Metrics evolution: an analysis according to languages. Scientometrics, 98(3), 2353–2367. DOI: http://dx.doi.org/10.1007/s11192-013-1164-8 . 
November 2012 the Google Scholar Metrics (GSM) journal rankings were updated, making it possible to compare bibliometric indicators in the ten languages indexed—and their stability—with the April 2012 version. The h-index and h-5 median of 1,000 journals were analysed, comparing their averages, maximum and minimum values and the correlation coefficient within rankings. The bibliometric figures grew significantly. In just seven and a half months the h-index of the journals increased by 15 % and the median h-index by 17 %. This growth was observed for all the bibliometric indicators analysed and for practically every journal. However, we found significant differences in growth rates depending on the language in which the journal is published. Moreover, the journal rankings seem to be stable between April and November, reinforcing the credibility of the data held by Google Scholar and the reliability of the GSM journal rankings, despite the uncontrolled growth of Google Scholar. Based on the findings of this study we suggest, firstly, that Google should upgrade its rankings at least semi-annually and, secondly, that the results should be displayed in each ranking proportionally to the number of journals indexed by language.




Orduña-Malea, E., Martín-Martín, A., Ayllón, J.M., Delgado López-Cózar, E. (2014). The silent fading of an academic search engine: the case of Microsoft Academic Search. Online Information Review, 38(7), 936-953. DOI: http://dx.doi.org/10.1108/OIR-07-2014-0169
Purpose – The main objective of this paper is to describe the obsolescence process of Microsoft Academic Search (MAS) as well as the effects of this decline on the coverage of fields and journals, and their influence on the representation of organisations.
Design/methodology/approach – The total number of records and those belonging to the most reputable journals (1,762) and organisations (346), according to the Field Rating indicator in each of the 15 fields and 204 sub-fields of MAS, were collected and statistically analysed in March 2014, by means of an automated querying process via http, covering academic publications from 1700 to the present.
Findings – Microsoft Academic Search has not been updated since 2013, although this phenomenon began to be glimpsed in 2011, when its coverage plummeted. Throughout 2014, indexing of new records is still ongoing, but at a minimal rate, without following any apparent pattern.
Research limitations/implications – There are also retrospective records being indexed at present. In this sense this research provides a picture of what MAS offered during March 2014 when queried directly via http.
Practical implications – The unnoticed obsolescence of MAS affects the quality of the service offered to its users (both those who engage in scientific information seeking and also those who use it for quantitative purposes).
Social implications – The predominance of Google Scholar as a monopoly in the academic search engines market as well as the prevalence of an open construction model versus a closed model (MAS).
Originality/value – A complete longitudinal analysis of fields, journals and organisations on MAS has been performed for the first time, identifying an unnoticed obsolescence. There has  not been any public explanation or disclaimer note announced by the company responsible,  which is incomprehensible given its implications for the reliability and validity of the bibliometric data provided on fields, journals, authors and conferences as well as their fair  representation by the search engine.

Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., Delgado López-Cózar, E. (2014). Empirical Evidences in Citation-Based Search Engines: Is Microsoft Academic Search dead? Granada: EC3 Working Papers, 16: 28 April 2014. arXiv preprint arXiv:1404.7045. 
The goal of this working paper is to summarize the main empirical evidences provided by the scientific community as regards the comparison between the two main citation based academic search engines: Google Scholar and Microsoft Academic Search, paying special attention to the following issues: coverage, correlations between journal rankings, and usage of these academic search engines. Additionally, selfelaborated data is offered, which are intended to provide current evidence about the popularity of these tools on the Web, by measuring the number of rich files PDF, PPT and DOC in which these tools are mentioned, the amount of external links that both products receive, and the search queries frequency from Google Trends. The poor results obtained by MAS led us to an unexpected and unnoticed discovery: Microsoft Academic Search is outdated since 2013. Therefore, the second part of the working paper aims at advancing some data demonstrating this lack of update. For this purpose we gathered the number of total records indexed by Microsoft Academic Search since 2000. The data shows an abrupt drop in the number of documents indexed from 2,346,228 in 2010 to 8,147 in 2013 and 802 in 2014. This decrease is offered according to 15 thematic areas as well. In view of these problems it seems logical not only that Microsoft Academic Searchwas poorly used to search for articles by academics and students, who mostly use Google or Google Scholar, but virtually ignored by bibliometricians.





Orduña-Malea, E., Ayllón, J.M., Martín-Martín, A., Delgado López-Cózar, E. (2014). About the size of Google Scholar: playing the numbers. Granada: EC3 Working Papers, 18: 24 July 2014. arXiv preprint arXiv:1407.6239.  
The emergence of academic search engines (Google Scholar and Microsoft Academic Search essentially) has revived and increased the interest in the size of the academic web, since their aspiration is to index the entirety of current academic knowledge. The search engine functionality and human search patterns lead us to believe, sometimes, that what you see in the search engine's results page is all that really exists. And, even when this is not true, we wonder which information is missing and why. The main objective of this working paper is to calculate the size of Google Scholar at present (May 2014). To do this, we present, apply and discuss up to 4 empirical methods: Khabsa & Giles's method, an estimate based on empirical data, and estimates based on direct queries and absurd queries. The results, despite providing disparate values, place the estimated size of Google Scholar in about 160 million documents. However, the fact that all methods show great inconsistencies, limitations and uncertainties, makes us wonder why Google does not simply provide this information to the scientific community if the company really knows this figure.


Ortega, J.L., Aguillo, I.F. (2014). Microsoft academic search and Google scholar citations: Comparative analysis of author profiles. Journal of the Association for Information Journal of the Association for Information Science and Technology, 65, 1149–1156. DOI: http://dx.doi.org/10.1002/asi.23036 .  
This article offers a comparative analysis of the personal profiling capabilities of the two most important free citation-based academic search engines, namely, Microsoft Academic Search (MAS) and Google Scholar Citations (GSC). Author profiles can be useful for evaluation purposes once the advantages and the shortcomings of these services are described and taken into consideration. In total, 771 personal profiles appearing in both the MAS and the GSC databases were analyzed. Results show that the GSC profiles include more documents and citations than those in MAS but with a strong bias toward the information and computing sciences, whereas the MAS profiles are disciplinarily better balanced. MAS shows technical problems such as a higher number of duplicated profiles and a lower updating rate than GSC. It is concluded that both services could be used for evaluation proposes only if they are applied along with other citation indices as a way to supplement that information.


Ortega, J.L. (2014). Academic Search Engines: A quantitative outlook. Amsterdam : Chandos Publishing , 2014 .  ISBN : 9781843347910 .
 Academic Search Engines intends to run through the current panorama of the academic search engines through a quantitative approach that analyses the reliability and consistence of these services.


Patra, S.K. (2014). Google Scholar-based citation analysis of Indian library and information science journals. Annals of Library and Information Studies (ALIS), 61(3), 227-234. 
Indian library and information science (LIS) journals are not indexed in Web of Science (WoS) database and lately Scopus® database of Elsevier B.V. has indexed three Indian LIS journals. Hence, Google Scholar (GS) is the only available global database for the citation analysis of Indian LIS journals. Based on GS, this study has traced the citation and authorship patterns of selected LIS journals. Although, GS covers wide spectrum of scholarly literature worldwide, this study found that Indian LIS journals have low visibility even in GS database. In terms of citations, multiple-authored articles generally got more citations than the single-authored articles. This study suggests LIS researchers to increase collaborations for better visibility of their research. 
Pitol, S.P., De Groote, S.L. (2014). Google Scholar versions: do more versions of an article mean greater impact?. Library Hi Tech, 32(4), 594-611. DOI: http://dx.doi.org/10.1108/LHT-05-2014-0039 . 
Purpose ─ The growing dominance of Google Scholar (GS) as a first-stop resource for scholars  and researchers demands investigation of its influence on citation patterns, freedom of  information, and scholarly communication. This study breaks new ground in understanding the various versions GS indexes, correlations between the number of GS versions and citation counts, and the value of institutional repositories (IRs) for increasing scholarly impact. 
Design/methodology/approach ─ GS listings for 982 articles in several academic subjects from three universities were analyzed for (a) GS version types, including any institutional repository versions, (b) citation rates, and (c) availability of free full-text. 
Findings ─ (a) Open Access articles were cited more than articles that were not available in free full-text. While journal publisher websites were indexed most often, only a small number of those articles were available as free full-text. (b) There is no correlation between the number of versions of an article and the number of times an article has been cited. (c) Viewing the “versions” of an article may be useful when publisher access is restricted, as over 70% of articles had at least one free full-text version available through Google Scholar. 
Originality/value ─This paper investigates Google Scholar versions as an alternative source for a scholarly article. While other articles have looked at Google Scholar through various lenses,  the authors believe this specific aspect of the topic has not been previously explored.

Prins, A., Costas, R., Van Leeuwen, T., Wouters, P. (2014). Using Google Scholar in research evaluation of social science programs, with a comparison with Web of Science data. STI 2014 - STI 2014 Leiden - 19th International Conference on Science & Tecnhnology Indicators. 



Rodríguez Morales, A.J., Ochoa Orozco, S.A., Tristán, P.M. (2014). Impacto de las revistas de salud colombianas: comparación de Publindex versus Google Scholar Metrics, SciELO y SCOPUS. Revista Cubana de Información en Ciencias de la Salud, 25(1), 24-35. 
Los indicadores basados en la citación son reconocidos por la comunidad científica para evaluar la calidad de las revistas científicas. Colombia tiene un sistema de clasificación de revistas denominado Indice Bibliográfico Nacional (IBN)/Publindex. El objetivo del estudio es evaluar el impacto de las revistas de salud colombianas según Google Scholar Metrics (GSM), SciELO y SCOPUS, comparado con la clasificación del IBN para el período 2007-2011. Al analizar las revistas por el índice H de GSM, encontramos que entre las revistas peor clasificadas por Publindex, "C", existen publicaciones con mayor index H5 y mediana de H5 que revistas mejor clasificadas por el IBN, "B" y "A2". Existen revistas como la colombiana de Anestesiología, que sin estar en el IBN tiene más factor de impacto de SciELO que varias revistas "A1" del IBN. Existen revistas indizadas en Scopus que a pesar de ser del cuartil 3 (Q3) son clasificadas como "A2" por el IBN, pero son revistas del Q4 que el IBN clasifica como "A1". Esto muestra que la clasificación de Publindex no es consistente con los indicadores de impacto de una revista en tres sistemas de evaluación: GSM, Scopus y SciELO, tal como ha sido previamente sugerido por otros autores. Se requiere mejorar la clasificación de Publindex y que esta tome en cuenta parámetros de citación e impacto para que la calidad reflejada en indicadores internacionales sea consistente con la clasificación nacional. 


Stirbu, S. (2014). What about Google Scholar when searching information in Human and Physical Geography?. Association of American Geographers. Annual Meeting Abstracts. Tampa, 8-12 April , 2014. [http://hdl.handle.net/2268/166814]. 
This study focuses on four bibliographic tools and aims to highlight what the use of the Google Scholar search engine can yield, in comparison with three commercial bibliographical databases: the well known multidisciplinary database Web of Science (WoS); GeoRef, a specialized data base in geosciences; and FRANCIS, which includes international geographical bibliography. To identify the bibliographic references published yearly between the years 2005 to 2009, ten keywords placed between quotation marks have been searched through the title field of the search tool interfaces. On a monthly basis, from November 2010 to July 2011, an identical process was repeated to get some information on the repeatability of the searches. Initially the whole set of results was analysed, and then the search results of two keywords were studied in more details. The results indicate that GS provides a high overlap with the search results of the other databases, but also yields numerous unique hits. Moreover, it seems that it is also able to find diverse types of literature, while the others are more specialized.


Stone, S.M., Lowe, M.S. (2014). Who is Citing Undergraduate Theses in Institutional Digital Repositories? Implications for Scholarship and Information Literacy. College & Undergraduate Libraries, 21(3-4), 345-359. 
Undergraduate theses are available through open access institutional repositories. Is undergraduate work being integrated into the larger body of academic research, and, if so, how? Institutional repositories containing undergraduate theses were selected and titles were searched using the forward citation feature in Google Scholar to determine if and where undergraduate scholarship is being cited. Results show that 24 percent of citations to senior theses were in peer-reviewed or refereed journals and 33 percent in dissertations and theses. This article addresses citation source and the potential value of undergraduate scholarship as well as the implications for information literacy instruction to senior thesis students.
 
Túñez López, M., Martínez Solana , M.Y., Valarezo González, K.P. (2014). Analysis of the productivity, impact, and collective h-index of the communication research carried out in Spain based on the information shared by researchers in their individual Google Scholar profiles . Revista Latina de Comunicación Social, 69, 684-709. DOI: http://dx.doi.org/10.4185/RLCS-2014-1030 . 
Introduction. This article examines the productivity, collective and individual h and h5 indexes, dissemination platforms (mainly books and journals), and visibility of the communication research carried out by Spanish scholars in the last four decades, based on the information shared by the 683 members of Spain‟s largest communication research association (AE-IC) in Google Scholar Metrics (GSM). 
Method. The study is based on the analysis of the information shared in GSM by 683 researchers, of which 142 have a public profiles and together have more than 7,000 publications. We analysed the dissemination platform, year of publication, number of citations, and title of the nearly 2,300 works that had received at least one citation.
Results. The visibility of the area of communication is low. The average h-index is 4, while the global h-index for the community of communication researchers is 56, with an h5-index of 34: h-index of 44 and h5-index of 34 for articles, and h-index of 34 and h5-index of 13 for books. Four of every ten researchers had h-index of 0 while two of every three published works did not manage to receive a single citation. Individually, the highest h-index is 26 and the highest h5-index is 18. Meanwhile, the importance and impact of books and journals as dissemination platforms has been inverted. Before 1980, of each ten citations six were given to books and three to articles, but after 2010, of each ten citations three are given to books and seven to articles. In terms of differences across gender, female researchers have lower impact values than their male counterparts.
Túñez López, M. (2014). Perfiles de Comunicación en Google Scholar Metrics, índice h y nuevas estrategias de difusión de la investigación. Historia y Comunicación Social, 19, 15-25. 
Google Scholar Metrics permite, desde 2012, que los investigadores creen un perfil con su producción científica y académica enlazada en la red y evaluada con indicadores bibliométricos ih, i10 e ih5 personalizados, que referencian la correlación que hay entre la productividad de un investigador y el éxito que aprecian sus pares. Este artículo refleja los resultados de una investigación sobre el uso del perfil, medias de cita, indicadores h referenciales y soportes de citación habituales de investigadores del área de Comunicación en España. También se aproxima a las nuevas estrategias de difusión de la investigación que se derivan del uso del índice h como indicador frente al tradicional factor de impacto.
Turbanti, S. (2014). Navigare nel mare di Scopus, Web of science e Google Scholar: l’avvio di una ricerca sulla vitalità delle discipline archivistiche e biblioteconomiche italiane. AIBstudi, 54(2/3) . DOI: http://dx.doi.org/10.2426/aibstudi-10266 . 
UIl contributo nasce parallelamente all’avvio di una ricerca all’interno del 29. ciclo di dottorato in Scienze documentarie, linguistiche e letterarie dell’Università La Sapienza di Roma, che si propone di analizzare il livello di salute degli studi del settore M-STO/08, inteso come capacità di uscire fuori dalla propria nicchia ed essere presenti in aree disciplinari e/o linguistiche esterne alla propria.
Si descrive la ricerca – effettuata nei due grandi database citazionali, Web of science e Scopus, e in Google Scholar – dei lavori scientifici dei docenti e ricercatori del settore M-STO/08, illustrando il metodo seguito, le principali differenze d'uso nonché i limiti dei database interrogati. Si analizzano in particolare i criteri adottati per l’organizzazione del lavoro in Scholar, il problema della corretta identificazione dell'autore e la successiva pulizia delle informazioni ottenute. Vengono quindi esposti i risultati raggiunti nell’interrogazione delle banche dati di Thomson Reuters e Elsevier e le difficoltà incontrate nell’ambiente Google.
Il quadro che emerge, oltre a confermare la complessità di uso degli strumenti citazionali, mostra come sia difficile ricavare risultati quantitativi rilevanti in contesti come quelli delle discipline LIS italiane, poco rappresentati nel panorama citazionale. La priorità emersa al termine di questa prima fase della ricerca è dunque quella della collocazione dei dati quantitativi all’interno di un corretta cornice di riferimento; solo in tal modo, infatti, si potrà cercare di rappresentare nel modo più oggettivo possibile i fenomeni di presenza e impatto – in altre parole, di internazionalizzazione e vitalità – delle discipline LIS italiane.
 

Van Noorden, R. (2014). Online collaboration: Scientists and the social network. Nature, 512, 126–129. DOI: http://dx.doi.org/10.1038/512126a . 

Verstak, A., Acharya, A., Suzuki, H., Henderson, S., Iakhiaev, M., Chiung Yu Lin, C., Shetty, N. (2014). On the Shoulders of Giants: The Growing Impact of Older Articles. arXiv preprint arXiv:1411.0275. 
In this paper, we examine the evolution of the impact of older scholarly articles. We attempt to answer four questions. First, how often are older articles cited and how has this changed over time. Second, how does the impact of older articles vary across different research fields. Third, is the change in the impact of older articles accelerating or slowing down. Fourth, are these trends different for much older articles. 
To answer these questions, we studied citations from articles published in 1990-2013. We computed the fraction of citations to older articles from articles published each year as the measure of impact. We considered articles that were published at least 10 years before the citing article as older articles. We computed these numbers for 261 subject categories and 9 broad areas of research. Finally, we repeated the computation for two other definitions of older articles, 15 years and older and 20 years and older. 
There are three conclusions from our study. First, the impact of older articles has grown substantially over 1990-2013. In 2013, 36% of citations were to articles that are at least 10 years old; this fraction has grown 28% since 1990. The fraction of older citations increased over 1990-2013 for 7 out of 9 broad areas and 231 out of 261 subject categories. Second, the increase over the second half (2002-2013) was double the increase in the first half (1990-2001). 
Third, the trend of a growing impact of older articles also holds for even older articles. In 2013, 21% of citations were to articles >= 15 years old with an increase of 30% since 1990 and 13% of citations were to articles >= 20 years old with an increase of 36%. 
Now that finding and reading relevant older articles is about as easy as finding and reading recently published articles, significant advances aren't getting lost on the shelves and are influencing work worldwide for years after.
Wildgaard, L.E., Larsen, B., Schneider, J. (2014). ACUMEN D5. 4b–Consequences of Indicators: using indicators on data from Google Scholar. 
We investigate if Publish or Perish ready-to-use bibliometric indicators can be used by individual scholars to enrich their curriculum vitae. Selected indicators were tested in four different fields and across 5 different academic seniorities. The results show performance in bibliometric evaluation is highly individual and using indicators as “benchmarks” unwise. Further the simple calculation of cites per publication per years-since-first-publication is a more informative indicator than the ready-to-use ones and can also be used to estimate if it is at all worth the scholar’s time to apply indicators to their CV.

Wildgaard, L.E., Larsen, B., Schneider, J. (2014). Supplement to ACUMEN deliverable 5.4 a: Description and comparison of indicators in Google Scholar and Web of Science. 
We collected publication and citation data in two databases to investigate the extent performance of author-level indicators are effected by choice of database, the stability of indicators across databases and ultimately to illustrate how differences in the computed indicators change our perception of individual researchers. In this report we begin by comparing database coverage, coverage at seniority and gender-level and then the performance of four basic indicators computed in both databases. In the main deliverable 5.4a, we investigate in a cluster analysis the performance of our previously identified 108 indicators of author-level impact. Understanding the effect of the database used to source the data and the demographics of the researchers in our sample, will enable us to put the results of our cluster analysis in perspective and direct future studies.
Winter, J.C.F., Zadpoor, A., Dodou, D. (2014). The expansion of Google Scholar versus Web of Science: a longitudinal study. Scientometrics, 1547–1565. DOI: http://dx.doi.org/10.1007/s11192-013-1089-2 
Web of Science (WoS) and Google Scholar (GS) are prominent citation services with distinct indexing mechanisms. Comprehensive knowledge about the growth patterns of these two citation services is lacking. We analyzed the development of citation counts in WoS and GS for two classic articles and 56 articles from diverse research fields, making a distinction between retroactive growth (i.e., the relative difference between citation counts up to mid- 2005 measured in mid-2005 and citation counts up to mid-2005 measured in April 2013) and actual growth (i.e., the relative difference between citation counts up to mid-2005 measured in April 2013 and citation counts up to April 2013 measured in April 2013). One of the classic articles was used for a citation-by-citation analysis. Results showed that GS has substantially grown in a retroactive manner (median of 170 % across articles), especially for articles that initially had low citations counts in GS as compared to WoS. Retroactive growth of WoS was small, with a median of 2 % across articles. Actual growth percentages were moderately higher for GS than for WoS (medians of 54 vs. 41 %). The citation-by-citation analysis showed that the percentage of citations being unique in WoS was lower for more recent citations (6.8 % for citations from 1995 and later vs. 41 % for citations from before 1995), whereas the opposite was noted for GS (57 vs. 33 %). It is concluded that, since its inception, GS has shown substantial expansion, and that the majority of recent works indexed in WoS are now also retrievable via GS. A discussion is provided on quantity versus quality of citations, threats for WoS, weaknesses of GS, and implications for literature research and research evaluation.

Wu, M.D., Chen S.C. (2014). Graduate students appreciate Google Scholar, but still find use for libraries. Electronic Library, The, 32(3), 375-389. DOI: http://dx.doi.org/10.1108/EL-08-2012-0102 .  
Purpose - Google Scholar has provided a convenient alternative for finding scholarly documents since its inception in 2004 and has become a favoured tool for numerous academics. Knowledge of patrons’ usage patterns and attitudes toward Google Scholar will assist librarians in designing appropriate instruction programs to improve students’ research abilities. This study examines how graduate students perceive and use Google Scholar. Design/methodology/approach - This study interviews 32 graduate students from National Taiwan University whose fields of study are the humanities (10 students), social sciences (11 students), and science and technology (11 students). 
Findings - Students prefer the usability of Google Scholar over library databases. However, they appreciate the quality of documents retrieved from library databases and regard these databases as crucial tools for finding scholarly documents. Science and technology students favoured Google Scholar more than those who study the humanities and social sciences. Research limitations/implications - This study only examines the perceptions and behaviour of graduate students. Future studies should include undergraduate students to investigate their use of Google Scholar, thereby obtaining a comprehensive understanding of various patrons of university libraries. 
Practical implications - This study shows that graduate students appreciate and use Google Scholar to find scholarly documents, although some students experience difficulties. The findings of this study may assist university libraries in improving their instruction programs. Originality/value - The majority of previous studies have focused on coverage, quality, and retrieval performance of Google Scholar. However, this study evaluates Google Scholar from a user’s perspective.
2013 [Go back]


Adriaanse, L.S., Rensleigh, C. (2013). Web of Science, Scopus and Google Scholar: A content comprehensiveness comparison. Electronic Library, The, 31(6), 727–744. DOI: http://dx.doi.org/10.1108/EL-12-2011-0174 Purpose - The research aim for this study was to compare three citation resources with one another to identify the citation resource with the most representative South African scholarly environmental sciences citation coverage. This paper focuses on the results of the content verification process which measured amongst others the citation counts, multiple copies and inconsistencies encountered across the three citation resources ISI Web of Science, Scopus and Google Scholar.Design/methodology/approach - The research, the first phase of a longitudinal study, used a comparative research design method with a purposive, non-probability sample. Data from the South African scholarly environmental sciences journals for the year range 2004-2008 (first phase) were extracted from the three citation resources and compared. Findings - It became evident during the verification process that the citation resources retrieved varied results. The total citation counts indicated that ISI Web of Science (WOS) retrieved the most citation results, followed by Google Scholar (GS) and then Scopus. WOS performed the best with total coverage of the journal sample population and also retrieved the most unique items. The investigation into multiple copies indicated that WOS and Scopus retrieved no duplicates, while GS retrieved multiples copies. Scopus delivered the least inconsistencies regarding content verification and content quality compared to the other two citation resources. Additionally, GS also retrieved the most inconsistencies, with WOS retrieving more inconsistencies than Scopus. Examples of these inconsistencies include, author spelling and –sequence, volume- and issue number. Originality/value - The findings of the study contribute to the understanding of the completeness of citation results retrieved from different citation resources. In addition it will raise awareness amongst academics to check citations of their work. 


Asher, A.D., Duke, L.M. Wilson, S. (2013). Paths of Discovery: Comparing the Search effectiveness of EBSCO Discovery Service, Summon, Google Scholar, and Conventional Library Resources. College Research Libraries, 74(5), 464–488. In 2011, researchers at Bucknell University and Illinois Wesleyan University compared the search efficacy of Serial Solutions Summon, EBSCO Discovery Service, Google Scholar, and conventional library databases. Using a mixed-methods approach, qualitative and quantitative data were gathered on students’ usage of these tools. Regardless of the search system, students exhibited a marked inability to effectively evaluate sources and a heavy reliance on default search settings. This article describes these results and makes recommendations for libraries considering these tools.
Boeker, M., Vach, W., Motschall, E. (2013). Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough. BMC Medical Research Methodology, 13(131). DOI: http://dx.doi.org/10.1186/1471-2288-13-13 Background-Recent research indicates a high recall in Google Scholar searches for systematic reviews. These reports raised high expectations of Google Scholar as a unified and easy to use search interface. However, studies on the coverage of Google Scholar rarely used the search interface in a realistic approach but instead merely checked for the existence of gold standard references. In addition, the severe limitations of the Google Search interface must be taken into consideration when comparing with professional literature retrieval tools. The objectives of this work are to measure the relative recall and precision of searches with Google Scholar under conditions which are derived from structured search procedures conventional in scientific literature retrieval; and to provide an overview of current advantages and disadvantages of the Google Scholar search interface in scientific literature retrieval. Methods-General and MEDLINE-specific search strategies were retrieved from 14 Cochrane systematic reviews. Cochrane systematic review search strategies were translated to Google Scholar search expression as good as possible under consideration of the original search semantics. The references of the included studies from the Cochrane reviews were checked for their inclusion in the result sets of the Google Scholar searches. Relative recall and precision were calculated. Results-We investigated Cochrane reviews with a number of included references between 11 and 70 with a total of 396 references. The Google Scholar searches resulted in sets between 4,320 and 67,800 and a total of 291,190 hits. The relative recall of the Google Scholar searches had a minimum of 76.2% and a maximum of 100% (7 searches). The precision of the Google Scholar searches had a minimum of 0.05% and a maximum of 0.92%. The overall relative recall for all searches was 92.9%, the overall precision was 0.13%. Conclusion-The reported relative recall must be interpreted with care. It is a quality indicator of Google Scholar confined to an experimental setting which is unavailable in systematic retrieval due to the severe limitations of the Google Scholar search interface. Currently, Google Scholar does not provide necessary elements for systematic scientific literature retrieval such as tools for incremental query optimization, export of a large number of references, a visual search builder or a history function. Google Scholar is not ready as a professional searching tool for tasks where structured retrieval methodology is necessary.
Bramer, W.M., Giustini, D., Kramer, B.M.R., Anderson,  P.F. (2013). The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. Systematic reviews, 2(1), 115. DOI: http://dx.doi.org/10.1186/2046-4053-2-115 Background-The usefulness of Google Scholar (GS) as a bibliographic database for biomedical systematic review (SR) searching is a subject of current interest and debate in research circles. Recent research has suggested GS might even be used alone in SR searching. This assertion is challenged here by testing whether GS can locate all studies included in 21 previously published SRs. Second, it examines the recall of GS, taking into account the maximum number of items that can be viewed, and tests whether more complete searches created by an information specialist will improve recall compared to the searches used in the 21 published SRs. Methods-The authors identified 21 biomedical SRs that had used GS and PubMed as information sources and reported their use of identical, reproducible search strategies in both databases. These search strategies were rerun in GS and PubMed, and analyzed as to their coverage and recall. Efforts were made to improve searches that underperformed in each database. Results-GS’ overall coverage was higher than PubMed (98% versus 91%) and overall recall is higher in GS: 80% of the references included in the 21 SRs were returned by the original searches in GS versus 68% in PubMed. Only 72% of the included references could be used as they were listed among the first 1,000 hits (the maximum number shown). Practical precision (the number of included references retrieved in the first 1,000, divided by 1,000) was on average 1.9%, which is only slightly lower than in other published SRs. Improving searches with the lowest recall resulted in an increase in recall from 48% to 66% in GS and, in PubMed, from 60% to 85%. Conclusions-Although its coverage and precision are acceptable, GS, because of its incomplete recall, should not be used as a single source in SR searching. A specialized, curated medical database such as PubMed provides experienced searchers with tools and functionality that help improve recall, and numerous options in order to optimize precision. Searches for SRs should be performed by experienced searchers creating searches that maximize recall for as many databases as deemed necessary by the search expert.


Cabezas-Clavijo, Á., Delgado López-Cózar, E. (2013). Google Scholar and the h-index in biomedicine: the popularization of bibliometric assessment. Medicina Intensiva, 37(5), 343–354. DOI: http://dx.doi.org/10.1016/j.medin.2013.01.008 . The aim of this study is to review the features, benefits and limitations of the new scientific evaluation products derived from Google Scholar, such as Google Scholar Metrics and Google Scholar Citations, as well as the h-index, which is the standard bibliometric indicator adopted by these services. The study also outlines the potential of this new database as a source for studies in Biomedicine, and compares the h-index obtained by the most relevant journals and researchers in the field of intensive care medicine, based on data extracted from the Web of Science, Scopus and Google Scholar. Results show that although the average h-index values  in Google Scholar are almost 30% higher than those obtained in Web of Science , and about 15% higher than those collected by Scopus, there are no substantial changes in the rankings generated from one data source or the other. Despite some technical problems, it is concluded that Google Scholar is a valid tool for researchers in Health Sciences, both for purposes of information retrieval and for the computation of bibliometric indicators.  

Chan, K.C., Chang, C.H., Chang, Y. (2013). Ranking of finance journals: Some Google Scholar citation perspectives. Journal of Empirical Finance, 21, 241-250. DOI: http://dx.doi.org/10.1016/j.jempfin.2013.02.001 . We conduct rankings on finance journals based on a rich database of citations for all articles from a set of 23 finance journals during 1990–2010. Our study is a major improvement in the literature by directly measuring the impact of each article within a set of finance journals. Our findings in journal citations generally echo the concern in Smith (2004) that some articles in premier journals have no/low impact while some articles in non-premier journals have high impact. In addition, we document that premier (non-premier) journals exhibit a linear (convex) curve of cumulative normalized citations across zero citation to less than or equal to eight citation buckets. We also show that author concentration index and editorial board members' citations represent alternative methods to evaluate finance journals.
Cardenas, J., Udo, G.J. (2013). Knowledge Management Literature Trends: an ISI Web of Science and Google Scholar comparison. Proceedings of the 44th Annual Meeting of the Decision Sciences Institute. Baltimore, November 16-19, 2013.  Knowledge management (KM) literature has been growing continuously in the past years, but it is now showing a sign of deceleration as the concept is integrated into the general knowledge. This study compares the published literature about KM from two outlets: ISI Web of Science and Google Scholar with the aim of determining whether or not KM is a fade. Both outlets concur that KM is not a fad as the concept has gradually increased, reach peak and now is showing signs of decrease. Peer-reviewed articles from ISI show that literature has reached the highest generation peak, while Google Scholar shows that both journals and publications are still growing but slowing down. This review compares the 12,434 KM articles published between 1993 and 2012 in ISI Web of Science with the 33,600 KM articles listed between 1992 and 2012 in Google Scholar. We used Bass Diffusion Model to compare statistics and correlate prognosis from both of these databases. The paper analyzes and compares trends clustering the papers by originating country and popular KM concepts. The results show that KM is not a fad, ISI is leading Google on literature generation, both of these databases follow the diffusion s-curve, and we are reaching the late majority stage of the Bass’ diffusion model.

Chunli, L., Qincheng, H. (2013). Study on Correlation of Different Altmetrics Indicators for Paper Evaluation Based on Three Academic Social Networking Tools: Mendeley, F1000 and Google Scholar. Journal of the China Society for Scientific and Technical Information, 32(2), 206–212. The paper explored a new theory of information science called altmetrics. Then discussed in detail the connotation、characteristic and relevant research at home and abroad. Three kinds of altmetrics indicators in scholarly social media tools ({Mendeley、F1000} and Google Scholar) were chosen to test the correlation in paper evaluation respect. The results have shown that Mendeley's readers indicator correlated with Google Scholar's cited number significantly. 

Cusker, J. (2013). Elsevier Compendex and Google Scholar: A Quantitative Comparison of Two Resources for Engineering Research and an Update to Prior Comparisons. The Journal of Academic Librarianship, 39(3), 241–243. DOI: http://dx.doi.org/10.1016/j.acalib.2013.02.001 . Elsevier's Compendex product (and its ancestor, Engineering Index) has been a de rigeur indexing tool for searching the primary literature in engineering for many years. However, the price of Compendex continues to rise while at the same time, broader, lower-cost or even free alternatives to such expensive indexing tools have proliferated. This paper seeks to quantitatively and, to a lesser extent, qualitatively compare Compendex to Google Scholar. In this paper, the author used a combination of methodologies in prior comparisons of both Google Scholar and Compendex to other indexing services and to each other. We undertook a quantitative comparison of the retrieval capabilities of the two indexing tools with the specific aim of examining Google's suitability to be a primary indexing tool for engineering literature. The author also considered additional factors regarding the ease-of-use and ‘added value’ features of the two interfaces.
Davies, M. (2013). Google Scholar and COCA-Academic: Two very different approaches to examining academic English. Journal of English for Academic Purposes, 12(3), 155–165. DOI: http://dx.doi.org/10.1016/j.jeap.2013.01.003 . In a recent article in the Journal of English for Academic Purposes, Brezina (2012) compares Google Scholar to the 91 million word academic component of the Corpus of Contemporary American English (COCA). In this article, I examine this comparison and show that – with the searches done correctly – COCA offers much more data than Brezina suggests. More importantly, I discuss at some length the many types of searches related to academic English which are possible with COCA but not Google Scholar, including searching for constructions (using part of speech and lemmas), comparisons between academic and non-academic genres or between different sub-genres of academic, creating frequency lists, finding collocates (to examine word meaning and usage), and carrying out semantically-oriented searches with synonyms and customized lists. Finally, I show how the new WordAndPhrase.info site provides even more user-friendly access to COCA data, including the ability to browse through large frequency lists of academic English and input and analyze entire texts. All of these COCA-based searches provide a wealth of information for teachers and learners of academic English, and while they can be done quickly and easily with COCA, all of them would be difficult or impossible in Google Scholar.


Delgado-López-Cózar, E., Cabezas-Clavijo, A. (2013). Ranking journals: could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank?. Learned Publishing, 26(2), 101-113. DOI: http://dx.doi.org/10.1087/20130206 . The launch of Google Scholar Metrics as a tool for assessing scientific journals may be serious competition for Thomson Reuters' Journal Citation Reports, and for the Scopus-powered Scimago Journal Rank. A review of these bibliometric journal evaluation products is performed. We compare their main characteristics from different approaches: coverage, indexing policies, search and visualization, bibliometric indicators, results analysis options, economic cost, and differences in their ranking of journals. Despite its shortcomings, Google Scholar Metrics is a helpful tool for authors and editors in identifying core journals. As an increasingly useful tool for ranking scientific journals, it may also challenge established journals products. 



Delgado López-Cózar, E., Repiso, R. (2013). The Impact of Scientific Journals of Communication: Comparing Google Scholar Metrics, Web of Science and Scopus. Comunicar, 41, 45-52. DOI: http://dx.doi.org/10.3916/C41-2013-04 Google Scholar Metrics' launch in April 2012, a new bibliometric tool for the evaluation of scientific journals by means of citation counting, has ended with the duopoly exerted by the Web of Science and Scopus databases. This paper aims at comparing the coverage of these three databases and the similarity their journal rankings may have. We selected a sample of journals from the field of Communication Studies indexed in the three databases. Data was recollected on 1720 November, 2012. 277 journals were identified to which we calculated their hindex and ranked them according to such indicator. Then, we analyzed the correlation between the rankings generated. Google Scholar Metrics dobles the coverage of the other databases, reducing the bias toward English language both; web of Science and Scopus have. Google Scholar Metrics shows higher hindex values (an average 47% higher than Scopus and 40% higher than Web of Science), allowing to better rank journals. We conclude that Google Scholar Metrics is a tool capable of identifying the main journals in Communication Studies offering results as reliable and valid as the ones Web of Science and Scopus show. 



Dilger, A., Müller, H., (2013). A citation-based ranking of German-speaking researchers in business administration with data of Google Scholar. European Journal of Higher Education, 13(2), 140-150. DOI: http://dx.doi.org/10.1080/21568235.2013.779464 .  Rankings of academics can be constructed in two different ways, either based on journal rankings or based on citations. Although citation-based rankings promise some fundamental advantages they are still not common in German-speaking business administration. However, the choice of the underlying database is crucial. This article argues that for German-speaking researchers in business administration (as an example for a non-English speaking scientific community in the social sciences) Google Scholar is an appropriate database. Unfortunately, it contains some structural errors that require diligent corrections. With that in mind, all 1572 members of the German Academic Association for Business Research (VHB) are ranked according to the citations of their recent publications (2005–2009). The results are compared to those of the Handelsblatt-BWL-Ranking which is the most prominent journal-based ranking of German-speaking academics in this discipline. It becomes clear that differences in method lead to different results.
Gehanno, J.F., Rollin, L., Darmoni, S. (2013). Is the coverage of Google Scholar enough to be used alone for systematic reviews. BMC Medical Informatics and Decision Making, 13(7). DOI: http://dx.doi.org/10.1186/1472-6947-13-7 . Background-In searches for clinical trials and systematic reviews, it is said that Google Scholar (GS) should never be used in isolation, but in addition to PubMed, Cochrane, and other trusted sources of information. We therefore performed a study to assess the coverage of GS specifically for the studies included in systematic reviews and evaluate if GS was sensitive enough to be used alone for systematic reviews. Methods-All the original studies included in 29 systematic reviews published in the Cochrane Database Syst Rev or in the JAMA in 2009 were gathered in a gold standard database. GS was searched for all these studies one by one to assess the percentage of studies which could have been identified by searching only GS. Results-All the 738 original studies included in the gold standard database were retrieved in GS (100%). Conclusion-The coverage of GS for the studies included in the systematic reviews is 100%. If the authors of the 29 systematic reviews had used only GS, no reference would have been missed. With some improvement in the research options, to increase its precision, GS could become the leading bibliographic database in medicine and could be used alone for systematic reviews. 
Giustini, D., Boulos, M.N.K., (2013). Google Scholar is not enough to be used alone for systematic reviews. Online journal of public health informatics, 5(2), 214. DOI: http://dx.doi.org/10.5210/ojphi.v5i2.4623 . Background: Google Scholar (GS) has been noted for its ability to search broadly for important references in the literature. Gehanno et al. recently examined GS in their study: ‘Is Google scholar enough to be used alone for systematic reviews?’ In this paper, we revisit this important question, and some of Gehanno et al.’s other findings in evaluating the academic search engine. Methods: The authors searched for a recent systematic review (SR) of comparable size to run search tests similar to those in Gehanno et al. We selected Chou et al. (2013) contacting the authors for a list of publications they found in their SR on social media in health. We queried GS for each of those 506 titles (in quotes ""), one by one. When GS failed to retrieve a paper, or produced too many results, we used the allintitle: command to find papers with the same title. Results: Google Scholar produced records for ~95% of the papers cited by Chou et al. (n=476/506). A few of the 30 papers that were not in GS were later retrieved via PubMed and even regular Google Search. But due to its different structure, we could not run searches in GS that were originally performed by Chou et al. in PubMed, Web of Science, Scopus and PsycINFO®. Identifying 506 papers in GS was an inefficient process, especially for papers using similar search terms. Conclusions: Has Google Scholar improved enough to be used alone in searching for systematic reviews? No. GS’ constantly-changing content, algorithms and database structure make it a poor choice for systematic reviews. Looking for papers when you know their titles is a far different issue from discovering them initially. Further research is needed to determine when and how (and for what purposes) GS can be used alone. Google should provide details about GS’ database coverage and improve its interface (e.g., with semantic search filters, stored searching, etc.). Perhaps then it will be an appropriate choice for systematic reviews.

Harzing, A.W. (2013). A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners. Scientometrics, 94(3), 1057–1075. DOI: http://dx.doi.org/10.1007/s11192-012-0777-7 Most governmental research assessment exercises do not use citation data for the Social Sciences and Humanities as Web of Science or Scopus coverage in these disciplines is considered to be insufficient. We therefore assess to what extent Google Scholar can be used as an alternative source of citation data. In order to provide a credible alternative, Google Scholar needs to be stable over time, display comprehensive coverage, and provide non-biased comparisons across disciplines. This article assesses these conditions through a longitudinal study of 20 Nobel Prize winners in Chemistry, Economics, Medicine and Physics. Our results indicate that Google Scholar displays considerable stability over time. However, coverage for disciplines that have traditionally been poorly represented in Google Scholar (Chemistry and Physics) is increasing rapidly. Google Scholar's coverage is also comprehensive; all of the 800 most cited publications by our Nobelists can be located in Google Scholar, although in four cases there are some problems with the results. Finally, we argue that Google Scholar might provide a less biased comparison across disciplines than the Web of Science. The use of Google Scholar might therefore redress the traditionally disadvantaged position of the Social Sciences in citation analysis. 
Huh, S. (2013). Citation Analysis of the Korean Journal of Urology From Web of Science, Scopus, Korean Medical Citation Index, KoreaMed Synapse, and Google Scholar.Korean journal of urology, 54(4), 220–228. DOI: http://dx.doi.org/10.4111/kju.2013.54.4.220 The Korean Journal of Urology began to be published exclusively in English in 2010 and is indexed in PubMed Central/PubMed. This study analyzed a variety of citation indicators of the Korean Journal of Urology before and after 2010 to clarify the present position of the journal among the urology category journals. The impact factor, SCImago Journal Rank (SJR), impact index, Z-impact factor (ZIF, impact factor excluding self-citation), and Hirsch Index (H-index) were referenced or calculated from Web of Science, Scopus, SCImago Journal & Country Ranking, Korean Medical Citation Index (KoMCI), KoreaMed Synapse, and Google Scholar. Both the impact factor and the total citations rose rapidly beginning in 2011. The 2012 impact factor corresponded to the upper 84.9% in the nephrology-urology category, whereas the 2011 SJR was in the upper 58.5%. The ZIF in KoMCI was one fifth of the impact factor because there are only two other urology journals in KoMCI. Up to 2009, more than half of the citations in the Web of Science were from Korean researchers, but from 2010 to 2012, more than 85% of the citations were from international researchers. The H-indexes from Web of Science, Scopus, KoMCI, KoreaMed Synapse, and Google Scholar were 8, 10, 12, 9, and 18, respectively. The strategy of the language change in 2010 was successful from the perspective of citation indicators. The values of the citation indicators will continue to increase rapidly and consistently as the research achievement of authors of the Korean Journal of Urology increases. 
Kaluza, H. (2013). Google Scholar versus EBSCO Discovery Service : ein vergleichender Retrieval-Test. MALIS Praxisprojekte 2013 : Projektberichte aus dem berufsbegleitenden Masterstudiengang Bibliotheks- und Informationswissenschaft der Fachhochschule Köln, 59-79. A retrieval test focused on searching 500 interdisciplinary journal articles in both Google Scholar and the implementation of the EBSCO discovery service at the University Library of Cologne. The objective was to find out if the established and free scientific search engine Google scholar, which can be seen as an example for Web Discovery Services, is more than an alternative to these new services. The results showed that Google Scholar is inferior in all aspects (proof and availability of full text, availability of German articles, temporal coverage) to a service liable to pay costs. It is however highlighted that Google scholar is adequate for a first access into a scientific research because of its free access and the overall satisfying results.
Kavitha, M., Venkatesan, M.N. (2013). Analysis of Google Scholar Top Rated Journals with Special Reference to the Journal “Nature”. International Journal of Advanced Library and Information Science, 01(01). This paper attempts to analyze the citation analysis, growth and development of the Journal “Nature“, which is top ranked in Google Scholar Metrics. In which the journal nature has the total of 291 articles, were ranked by the impact factor with the growth analysis by the parameter of period. In which the year 2007 has the greater citation (120 articles) than other years. This study also analyses the impact factor of this journal over a period.

Minasny, B., Hartemink, A.E., McBratney, A., Jang, H. (2013) Citations and the h index of soil researchers and journals in the Web of Science, Scopus, and Google Scholar. PeerJ . DOI: http://dx.doi.org/10.7717/peerj.183  Citation metrics and h indices differ using different bibliometric databases. We compiled the number of publications, number of citations, h index and year since the first publication from 340 soil researchers from all over the world. On average, Google Scholar has the highest h index, number of publications and citations per researcher, and the Web of Science the lowest. The number of papers in Google Scholar is on average 2.3 times higher and the number of citations is 1.9 times higher compared to the data in the Web of Science. Scopus metrics are slightly higher than that of the Web of Science. The h index in Google Scholar is on average 1.4 times larger than Web of Science, and the h index in Scopus is on average 1.1 times larger than Web of Science. Over time, the metrics increase in all three databases but fastest in Google Scholar. The h index of an individual soil scientist is about 0.7 times the number of years since his/her first publication. There is a large difference between the number of citations, number of publications and the h index using the three databases. From this analysis it can be concluded that the choice of the database affects widely-used citation and evaluation metrics but that bibliometric transfer functions exist to relate the metrics from these three databases. We also investigated the relationship between journal's impact factor and Google Scholar's h5-index. The h5-index is a better measure of a journal's citation than the 2 or 5 year window impact factor.
Moskovkin, V.M., (2013). The construction of academic publishing and terminological structures using the Google Scholar search engine: An example of environmental terms in publications at the classical universities of Kharkiv and Skopje. Scientific and Technical Information Processing, 40(1), 11–16. DOI: http://dx.doi.org/10.3103/S0147688213010024 .  A methodology for constructing academic terminological and publishing structures using the Google Scholar search engine is presented. These structures are formed for the classical universities of Kharkiv and Skopje based on the example of basic environmental terms of a general nature that are distinguished in English-language publications. The environmental theme is more actively studied at Kharkiv National University. The first publications produced by the researchers of this university in the considered area of science were written in the early 1980s. An analysis of the most-frequently cited publications that contain the selected environmental terms shows that such publications are often a result of the work of an international team of authors. This is of great importance for the development of journal strategies and policies. By using Google Scholar, one can find significantly more publications on the website of Kharkiv University compared to that of the University of Skopje, due to the electronic open access archive system of publications at Kharkiv University. The frequency of occurrence for publications that contain selected environmental terms is higher in the case of the University of Skopje due to the fact that the publications of this university are poorly represented on the web and also because of their predominantly English-language character.

Ortega, J.L., Aguillo, I.F. (2013). Institutional and country collaboration in an online service of scientific profiles: Google Scholar Citations. Journal of Informetrics, 7(2), 394–403. DOI: http://dx.doi.org/10.1016/j.joi.2012.12.007 .  The purpose of this paper is to analyse and describe the topological properties of the institutional and national collaboration network from the profiles extracted from Google Scholar Citations (GSC). 19,912 unique profiles with “co-authors” were obtained from a web crawl performed in March 2012. Several statistical and network analysis techniques were used to map and analyse these collaboration relationships at the country and institution level. Results show that The United States dominates the world scientific map and that every research institution is grouped by national, geographical and cultural criteria. A clustering phenomenon based on the self-similarity and fractal properties of scale-free networks is also observed. We conclude that GSC is a suitable tool for collaboration studies only at macro level between countries and institutions.


Shariff, S.Z., Bejaimal, S.A.D., Sontrop, J.M., Iansavichus, A.V., Haynes, RB, Weir, M.A., Garg, A.X. (2013). Retrieving Clinical Evidence: A Comparison of PubMed and Google Scholar for Quick Clinical Searches. Journal of Medical Internet Research, 15(8). DOI: http://dx.doi.org/10.2196/jmir.2624 . Background-Physicians frequently search PubMed for information to guide patient care. More recently, Google Scholar has gained popularity as another freely accessible bibliographic database. Objective To compare the performance of searches in PubMed and Google Scholar. Methods-We surveyed nephrologists (kidney specialists) and provided each with a unique clinical question derived from 100 renal therapy systematic reviews. Each physician provided the search terms they would type into a bibliographic database to locate evidence to answer the clinical question. We executed each of these searches in PubMed and Google Scholar and compared results for the first 40 records retrieved (equivalent to 2 default search pages in PubMed). We evaluated the recall (proportion of relevant articles found) and precision (ratio of relevant to nonrelevant articles) of the searches performed in PubMed and Google Scholar. Primary studies included in the systematic reviews served as the reference standard for relevant articles. We further documented whether relevant articles were available as free full-texts. Results-Compared with PubMed, the average search in Google Scholar retrieved twice as many relevant articles (PubMed: 11%; Google Scholar: 22%; P<.001). Precision was similar in both databases (PubMed: 6%; Google Scholar: 8%; P=.07). Google Scholar provided significantly greater access to free full-text publications (PubMed: 5%; Google Scholar: 14%; P<.001). Conclusions-For quick clinical searches, Google Scholar returns twice as many relevant articles as PubMed and provides greater access to free full-text articles.
Śleszyński, P. (2013). Citations and impact of the Polish geographical centers by Google Scholar. Przegląd Geograficzny, 85(4), 599–627.
Túñez-López, M. (2013). The ‘h-index’ in Communication research in Spain, Portugal and Latin America: Web of Knowledge (WoK), Scopus and Google Scholar Metrics. Communication&Society/Comunicación y Sociedad, 26(4), 53-75. The h-index is a research impact and productivity indicator proposed by Hirsch (USA, 2005) and adopted by WoK and Scopus. However, it was not recognised in Communication until Google entered into the field of bibliometry in 2012 by introducing h-impact lists of scientific journals and h-index on scholars’ profiles. This article compares the h-index and the impact factor, identifies the context indicators (g, m, h5, i10), tries to define a reference hR index for the Communication field and analyses the h-impact of journals in this field in the Ibero-American region.

Wildgaard, L.E., Larsen, B., Schneider, J. (2013). ACUMEN deliverable 5.3: Selection of Samples, Part 1 & 2. Based on the samples from the four research fields used in the other WPs we have identified 793 researchers with online publication lists. Publication data from these researchers were gathered and combined with demographic data from the survey. Bibliometric analyses of these publications were undertaken in WoS and Google Scholar using a set of indicators designed for assessment at the individual level. The sample of 64 indicators were previously identified in the review of 114 bibliometric indicators, as presented in Madrid in January 2013. The set of 64 indicators has been reduced to 40 using a number of selection criteria.


Zarifmahmoudi, L., Kianifar, H.R., Sadeghi, R. (2013). Citation analysis of Iranian Journal of Basic Medical Sciences in ISI web of knowledge, Scopus, and Google Scholar. Iranian Journal of Basic Medical Sciences, 16(10), 1027–1030. Objective(s): Citation tracking is an important method to analyze the scientific impact of journal articles and can be done through Scopus (SC), Google Scholar (GS), or ISI web of knowledge (WOS). In the current study, we analyzed the citations to 2011-2012 articles of Iranian Journal of Basic Medical Sciences (IJBMS) in these three resources. Material and Methods: The relevant data from SC, GS, and WOS official websites. Total number of citations, their overlap and unique citations of these three recourses were evaluated. Results: WOS and SC covered 100% and GS covered 97% of the IJBMS items. Totally, 37 articles were cited at least once in one of the studied resources. Total number of citations were 20, 30, and 59 in WOS, SC, and GS respectively. Forty citations of GS, 6 citation of SC, and 2 citations of WOS were unique. Conclusion: Every scientific resource has its own inaccuracies in providing citation analysis information. Citation analysis studies are better to be done each year to correct any inaccuracy as soon as possible. IJBMS has gained considerable scientific attention from wide range of high impact journals and through citation tracking method; this visibility can be traced more thoroughly.
Zhang, T. (2013). User-centered evaluation of a discovery layer system with google scholar. In Design, User Experience, and Usability. Web, Mobile, and Product Design Lecture Notes in Computer Science Volume 8015, 2013, 313-322. DOI: http://dx.doi.org/10.1007/978-3-642-39253-5_34 Discovery layer systems allow library users to obtain search results from multiple library resources and view results in a consistent format. The implementation of a discovery layer is expected to simplify users’ workflow of searching for scholarly information. Previous studies on discovery layer systems focused on functionality and content, but not quality of search results from the user’s perspective. The objective of this study was to obtain users’ assessment of search results of a discovery layer system (Ex Libris Primo®) and compare that with a widely used scholarly search tool (Google Scholar). Results showed that Primo’s search results relevancy is comparable to Google Scholar, but it received significantly lower usability and preference ratings. A number of usability issues of Primo were also identified from the study. Results of the study are used to improve the interface of Primo and adjust relevancy ranking options. The empirical method of search results assessment and feedback collection used in this study can be extended to similar user-centered system implementation and evaluation efforts.
Zmigrodzki, P. (2013). What can you learn about Jȩzyk Polski from Google Scholar. Jezyk Polski, 93(1), 19–25.  The paper provides a citation analysis for Jȩzyk Polski (Polish Language) on the basis of Google Scholar data, retrieved using Publish or Perish software (Harzing 2007). The author points out the most frequently cited papers published in Jȩzyk Polski during all the 100 years and most influential scholars. The conclusion is, that Jȩzyk Polski ranks among the most influential and popular periodicals in the area of linguistics in Poland.


2012 [Go back]




Aguillo, I.F. (2012). Is Google Scholar useful for bibliometrics? A webometric analysis. Scientometrics, 91(2), 343–351. DOI: http://dx.doi.org/10.1007/s11192-011-0582-8 Google Scholar, the academic bibliographic database provided free-of-charge by the search engine giant Google, has been suggested as an alternative or complementary resource to the commercial citation databases like Web of Knowledge (ISI/Thomson) or Scopus (Elsevier). In order to check the usefulness of this database for bibliometric analysis, and especially research evaluation, a novel approach is introduced. Instead of names of authors or institutions, a webometric analysis of academic web domains is performed. The bibliographic records for 225 top level web domains (TLD), 19,240 university and 6,380 research centres institutional web domains have been collected from the Google Scholar database. About 63.8% of the records are hosted in generic domains like .com or .org, confirming that most of the Scholar data come from large commercial or non-profit sources. Considering only institutions with at least one record, one-third of the other items (10.6% from the global) are hosted by the 10,442 universities, while 3,901 research centres amount for an additional 7.9% from the total. The individual analysis show that universities from China, Brazil, Spain, Taiwan or Indonesia are far better ranked than expected. In some cases, large international or national databases, or repositories are responsible for the high numbers found. However, in many others, the local contents, including papers in low impact journals, popular scientific literature, and unpublished reports or teaching supporting materials are clearly overrepresented. Google Scholar lacks the quality control needed for its use as a bibliometric tool; the larger coverage it provides consists in some cases of items not comparable with those provided by other similar databases.


Altanopoulou, P., Dontsidou, M., Tselios, N. (2012). Evaluation of ninety-three major Greek university departments using Google Scholar. Quality in Higher Education, 18(1), 111-137. DOI: http://dx.doi.org/10.1080/13538322.2012.670918 . In this article, 93 Greek university departments were evaluated according to their academics’ h-index. A representative sample from the fields of social sciences and humanities, sciences, engineering, pharmacy and economics was adopted. In the reported study, 3354 (approximately 1 out of 3) academics serving in Greek universities were evaluated. The number of papers, citations and h-index have been collected for each academic, department, school and university using the Google Scholar scientific database and the citations analysis software program Publish or Perish. Analysis revealed that departments of the same academic discipline but located in different universities are characterised by strong differences on the scientific outcome. In addition, in the majority of the evaluated departments, a significant difference in h-index was observed between academics who report scientific activity on the departments’ website and those who do not. The viability of the adopted method for measuring and ranking the scientific performance of higher education departments proved to be quite high.

Amara, N., Landry, R. (2012). Counting citations in the field of business and management: why use Google Scholar rather than the Web of Science. Scientometrics, 93(3), 553–581. DOI: http://dx.doi.org/10.1007/s11192-012-0729-2 Research assessment carries important implications both at the individual and institutional levels. This paper examines the research outputs of scholars in business schools and shows how their performance assessment is significantly affected when using data extracted either from the Thomson ISI Web of Science (WoS) or from Google Scholar (GS). The statistical analyses of this paper are based on a large survey data of scholars of Canadian business schools, used jointly with data extracted from the WoS and GS databases. Firstly, the findings of this study reveal that the average performance of B scholars regarding the number of contributions, citations, and the h-index is much higher when performances are assessed using GS rather than WoS. Moreover, the results also show that the scholars who exhibit the highest performances when assessed in reference to articles published in ISI-listed journals also exhibit the highest performances in Google Scholar. Secondly, the absence of association between the strength of ties forged with companies, as well as between the customization of the knowledge transferred to companies and research performances of B scholars such as measured by indicators extracted from WoS and GS, provides some evidence suggesting that mode 1 and 2 knowledge productions might be compatible. Thirdly, the results also indicate that senior B scholars did not differ in a statistically significant manner from their junior colleagues with regard to the proportion of contributions compiled in WoS and GS. However, the results show that assistant professors have a higher proportion of citations in WoS than associate and full professors have. Fourthly, the results of this study suggest that B scholars in accounting tend to publish a smaller proportion of their work in GS than their colleagues in information management, finance and economics. Fifthly, the results of this study show that there is no significant difference between the contributions record of scholars located in English language and French language B schools when their performances are assessed with Google Scholar. However, scholars in English language B schools exhibit higher citation performances and higher h-indices both in WoS and GS. Overall, B scholars might not be confronted by having to choose between two incompatible knowledge production modes, but with the requirement of the evidence-based management approach. As a consequence, the various assessment exercises undertaken by university administrators, government agencies and associations of business schools should complement the data provided in WoS with those provided in GS
Arlitsch, K.,  O'Brien, P.S. (2012). Invisible institutional repositories: Addressing the low indexing ratios of IRs in Google Scholar. Library Hi Tech, 30(1), 60-81. DOI: http://dx.doi.org/10.1108/07378831211213210 . Google Scholar has difficulty indexing the contents of institutional repositories, and the authors hypothesize the reason is that most repositories use Dublin Core, which cannot express bibliographic citation information adequately for academic papers. Google Scholar makes specific recommendations for repositories, including the use of publishing industry metadata schemas over Dublin Core. This paper aims to test a theory that transforming metadata schemas in institutional repositories will lead to increased indexing by Google Scholar.
Beckmann, M., von Wehrden, H. (2012). Where you search is what you get: literature mining – Google Scholar versus Web of Science using a data set from a literature search in vegetation science. Journal of Vegetation Science, 23, 1197–1199. DOI: http://dx.doi.org/10.1111/j.1654-1103.2012.01454.x . Question: Is Google Scholar superior in literature search compared to the Web of Science? Location: The Internet. Methods: The maximum number of papers dealing with specific subjects was derived from a published review and compared with Google Scholar and Web of Science search results using GLM and a post-hoc test. Results: Search results acquired through Google Scholar were not significantly different from the maximum number of papers found by manual search, while the Web of Science search delivered significantly less. Conclusion: Researchers should give more prominent recognition to Google Scholar as a search tool, especially when conducting quantative reviews and meta-analysis. We compared the performance of Google Scholar and the Web of Science using a dataset from a quantitative review. Search results acquired through Google Scholar contained significantly more relevant results than those delivered by the Web of Science. Due to its full text search capabilities, Google Scholar should be recognized more as a useful search tool by the scientific community.


Brezina, V. (2012). Use of Google Scholar in corpus-driven EAP research. Journal of English for Academic Purposes, 11(4), 319–331. DOI: http://dx.doi.org/10.1016/j.jeap.2012.08.001 . This primarily methodological article makes a proposition for linguistic exploration of textual resources available through the Google Scholar search engine. These resources (Google Scholar virtual corpus) are significantly larger than any existing corpus of academic writing. Google Scholar, however, was not designed for linguistic searches and special attention therefore needs to be paid to maximising its effectiveness in corpus linguistics research. The article discusses the search capacity of Google Scholar and compares the Google Scholar virtual corpus with the largest traditional corpus of written academic English, COCA - academic. Finally, the article offers a case study on the as-author-reporting verb structure (and its modifications). The study demonstrates that Google Scholar can be employed effectively in EAP research offering us new insights into reporting practices in two disciplines, Applied Linguistics and Physics, which were chosen for comparison. The benefits of using Google Scholar virtual corpus are the following: 1) wide representativeness of written academic language, 2) possibility of capturing subtle variation in academic patterns, and 3) possibility of comparing linguistic patterns across different academic fields.
De Groote, S.L., Raszewski, R. (2012). Coverage of Google Scholar, Scopus, and Web of Science: A case study of the h-index in nursing. Nursing Outlook, 60(6), 391-400. DOI: http://dx.doi.org/10.1016/j.outlook.2012.04.007 . Purpose: This study compares the articles cited in CINAHL, Scopus, Web of Science (WOS), and Google Scholar and the h-index ratings provided by Scopus, WOS, and Google Scholar. Methods: The publications of 30 College of Nursing faculty at a large urban university were examined. Searches by author name were executed in Scopus, WOS, and POP (Publish or Perish, which searches Google Scholar), and the h-index for each author from each database was recorded. In addition, the citing articles of their published articles were imported into a bibliographic management program. This data was used to determine an aggregated h-index for each author. Results: Scopus, WOS, and Google Scholar provided different h-index ratings for authors and each database found unique and duplicate citing references. Conclusions: More than one tool should be used to calculate the h-index for nursing faculty because one tool alone cannot be relied on to provide a thorough assessment of a researcher's impact. If researchers are interested in a comprehensive h-index, they should aggregate the citing references located by WOS and Scopus. Because h-index rankings differ among databases, comparisons between researchers should be done only within a specified database.


Delgado López-Cózar, E., Robinson-García, N., (2012). Repositories in Google Scholar Metrics or what is this document type doing in a place as such?. Cybermetrics, 16(1), paper 4. The present paper analyzes GS Metrics, Google's newest product aiming at ranking journals according to their H-Index. Specifically, we analyze GS Metrics' decision of considering journals and repositories as equal and therefore, including them in the product. In this sense, the authors position themselves against this decision and provide several arguments of different nature warning against the shortcomings this product has. The first one is of a conceptual nature and is related to the definition of journal and repository. Secondly, they refer at the methodological issues mixing repositories and journals can bring out. Then, they deepen on many other flaws GS Metrics presents. Finally, GS Metrics and its possible use as an evaluation tool are discussed and possible solutions to its shortcomings are provided.

Gil Roales-Nieto, J., O’Neill, B. (2012). A Comparative Study of Journals Quality based on Web of Science , Scopus and Google Scholar : A Case Study with IJP & PT. International Journal of Psychology and Psychological Therapy, 12(3), 453–479. The purpose of this article is to analyze the evolution of the International Journal of Psychology & Psychological Therapy (IJP&PT) throughout its first decade of publication (from years 2001 to 2010), comparing the quality measures that result from applying the three most important social science databases: Thomson-Reuters Web of Science, Elsevier-Scopus, and Google Scholar. We compared the three databases, using the citations recorded for IJP&PT as a “case study” applied to a journal. As quality indicators we used IJP&PT data available in the three databases, as well as other indicators related to the internationality of the journal. The results shows a increasing tend in all quality criteria during the time period evaluated as a first-level journal among psychology journals edited in Spain. Also, the results shows serious discrepancies when the data of the three databases are compared. We discuss the need to improve the criteria used by the databases, as well as the convenience to use several quality indicators for journals’ evaluation.


Jacso, P. (2012). Google Scholar Author Citation Tracker: is it too little, too late?. Online Information Review, 36(1), 126-141. DOI: http://dx.doi.org/10.1108/14684521211209581   Purpose – Seven years after the release of Google Scholar in 2004, it was enhanced by a new module, the Google Scholar Author Citation Tracker (GSACT), currently a small subset of the complete Google Scholar (GS) database. The aim of this paper is to focus on this enhancement. Design/methodology/approach – The paper discusses the Google Scholar Author Citation Tracker, its features, potential benefits and problems. Findings – GSACT allows registered users to create and edit their scientific profiles and some bibliometric indicators, such as the h-index, total citation counts, and the i10 index. These metrics are provided for the entire academic career of authors and for the most recent five-year period. The new module also offers some long overdue essential options, such as sorting result lists of the documents by their publication year, title, and the citations received. Originality/value – The paper shows that, at present, GSACT may be too little, too late. However, with an extension of the current clean-up project it could possibly become a really scholarly resource in the long run.

Jacso, P. (2012). Google Scholar Metrics for Publications: The software and content features of a new open access bibliometric service. Online Information Review, 36(4), 604-619. DOI: http://dx.doi.org/10.1108/14684521211254121  Purpose – The purpose of this paper is to review the software and content features of the Google Scholar Metrics (GSM) service launched in April 2012. Design/methodology/approach – The paper reviews GSM, examining the software, browsing, searching and sorting functions, citation matching and content. Findings – The paper reveals that the service can offer a better alternative than the traditional Google Scholar service to discover and judge the standing of journals through the prism of their citedness. GSM could become a potentially useful complementary resource primarily by virtue of its brand recognition, and the convenience of not requiring the installation of additional software, but currently its bibliometric indicators are often inappropriate for decision making in matters of tenure, promotion, grants and accreditation.



Jacso, P. (2012). Using Google Scholar for journal impact factors and the h-index in nationwide publishing assessments in academia – siren songs and air-raid sirens. Online Information Review, 36(3), 462-478. DOI: http://dx.doi.org/10.1108/14684521211241503  Purpose – Google Scholar has been increasingly used in the past six to seven years as a highly efficient information source and service by librarians and other information professionals. The problem is when Google Scholar is touted and used as a bibliometric/scientometric tool and resource in the assessment of the quantity (productivity) and quality (impact) of research publications, in formal and informal ways, for decisions related to tenure, promotion and grant applications of individual researchers and research groups, as well as in journal subscriptions and cancellations. This paper aims to examine this issue. Design/methodology/approach – The paper discusses the use of Google Scholar for journal impact factors and the h-index in nationwide publishing assessments in academia. It focuses on the issues of access and excess in Google Scholar: the innate limits of Google Scholar and those imposed by its developers on the users. Findings – The paper reveals that issues of access and excess in Google Scholar prevent the researchers from doing appropriate content analysis that the best librarians and other information professionals do systematically to discover the pros and cons of databases. The excess content grossly dilutes the originally worthy collection of scholarly publications. The accuracy, reliability and reproducibility are essential for realistic research assessment through the prism of the quantity (publication counts) and quality (citation counts) of scholarly works. Unfortunately the metadata created by Google Scholar is substandard, neither reliable nor reproducible and it distorts the metric indicators at the individual, corporate and journal levels. Originality/value – The paper provides useful information on the use of Google Scholar for journal impact factors and the h-index in academic publishing.


Lasda Bergman, E.M., (2012). Finding Citations to Social Work Literature: The Relative Benefits of Using Web of Science, Scopus, or Google Scholar. Journal of Academic Librarianship, 38(6), 370–379. DOI: http://dx.doi.org/10.1016/j.acalib.2012.08.002 Past studies of citation coverage of Web of Science, Scopus, and Google Scholar do not demonstrate a consistent pattern that can be applied to the interdisciplinary mix of resources used in social work research. To determine the utility of these tools to social work researchers, an analysis of citing references to well-known social work journals was conducted. Web of Science had the fewest citing references and almost no variety in source format. Scopus provided higher citation counts, but the pattern of coverage was similar to Web of Science. Google Scholar provided substantially more citing references, but only a relatively small percentage of them were unique scholarly journal articles. The patterns of database coverage were replicated when the citations were broken out for each journal separately. The results of this analysis demonstrate the need to determine what resources constitute scholarly research and reflect the need for future researchers to consider the merits of each database before undertaking their research. This study will be of interest to scholars in library and information science as well as social work, as it facilitates a greater understanding of the strengths and limitations of each database and brings to light important considerations for conducting future research.



McFadden, P., Taylor, B.J., Campbell, A., McQuilkin, J. (2012). Systematically Identifying Relevant Research: Case Study on Child Protection Social Workers’ Resilience. Research on Social Work Practice, 22(6), 626-636. DOI: http://dx.doi.org/10.1177/1049731512453209  .  Context: The development of a consolidated knowledge base for social work requires rigorous approaches to identifying relevant research. Method: The quality of 10 databases and a web search engine were appraised by systematically searching for research articles on resilience and burnout in child protection social workers. Results: Applied Social Sciences Index and Abstracts, Social Services Abstracts and Social Sciences Citation Index (SSCI) had greatest sensitivity, each retrieving more than double than any other database. PsycINFO and Cumulative Index to Nursing and Allied Health (CINAHL) had highest precision. Google Scholar had modest sensitivity and good precision in relation to the first 100 items. SSCI, Google Scholar, Medline, and CINAHL retrieved the highest number of hits not retrieved by any other database. Conclusion: A range of databases is required for even modestly comprehensive searching. Advanced database searching methods are being developed but the profession requires greater standardization of terminology to assist in information retrieval. 



Miri, S.M., Raoofi, A., Heidari, Z. (2012). Citation Analysis of Hepatitis Monthly by Journal Citation Report (ISI), Google Scholar, and Scopus. Hepatitis Monthly, 12(9). DOI: http://dx.doi.org/10.5812/hepatmon.7441 . Background: Citation analysis as one of the most widely used methods of bibliometrics can be used for computing the various impact measures for scholars based on data from citation databases. Journal Citation Reports (JCR) from Thomson Reuters provides an- nual report in the form of impact factor (IF) for each journal. Objectives: We aimed to evaluate the citation parameters of Hepatitis Monthly by JCR in 2010 and compare them with GS and Sc. Materials and Methods: All articles of Hepat Mon published in 2009 and 2008 which had been cited in 2010 in three databases including WoS, Sc and GS gathered in a spread- sheet. The IFs were manually calculated. Results: Among the 104 total published articles the accuracy rates of GS and Sc in record- ing the total number of articles was 96% and 87.5%. There was a difference between IFs among the three databases (0.793 in ISI [Institute for Scientific Information], 0.945 in Sc and 0.85 GS). The missing rate of citations in ISI was 4% totally. Original articles were the main cited types, whereas, guidelines and clinical challenges were the least ones. Conclusions: None of the three databases succeed to record all articles published in the journal. Despite high sensitivity of GS comparing to Sc, it cannot be a reliable source for indexing since GS has lack of screening in the data collection and low specificity. Using an average of three IFs is suggested to find the correct IF. Editors should be more aware on the role of original articles in increasing IF and the potential efficacy of review articles in long term impact factor.

Moskovkin, V.M., Delux, T., Moskovkina, M.V. (2012). Comparative analysis of university publication activity by Google scholar (on example of leading Czech and Germany universities). Cybermetrics, 16(1), paper 2. With the help of the Google Scholar search engine, we have studied in detail the aggregated publication structure of the leading universities in the Czech Republic and Germany. We have also classified these structures and identified structural changes in them for German universities. These shifts have been observed in the Free University of Berlin and Humboldt University, and they all occurred within 5 years in the first decade of the 21st century when the major university publication activity moved from the sphere of medical research to the area of social sciences and humanities. Prospects for further research are in the comparative analysis of university publication activities with the help of Web of Science, Scopus and Google Scholar facilities.
Noe, D.E. (2012). Replicating Top Users’ Searches in Summon and Google Scholar. In M. Popp, & D. Dallis (Eds.) Planning and Implementing Resource Discovery Tools in Academic Libraries, 225-249. DOI: http://dx.doi.org/10.4018/978-1-4666-1821-3.ch013 . This chapter discusses the results of a review of the first 25 results for some of the most common searches in one college’s instance of Summon™1 and the results for the same searches in Google Scholar™2. The results of the searches were provided to a panel of three librarians who did not know from which discovery service the results came. The chapter treats each search and its results as case studies and discusses both quantitative and qualitative evaluations. The study finds that neither search tool can provide reliable results for a simple search without further refinement of the search.

Nourbakhsh, E., Nugent, R., Wang, H., Cevik, C., Nugent, K. (2012). Medical literature searches: a comparison of PubMed and Google Scholar. Health Information & Libraries Journal, 29: 214–222. DOI: http://dx.doi.org/10.1111/j.1471-1842.2012.00992.x . Background: Medical literature searches provide critical information for clinicians. However, the best strat- egy for identifying relevant high-quality literature is unknown. Objectives: We compared search results using PubMed and Google Scholar on four clinical questions and analysed these results with respect to article relevance and quality. Methods: Abstracts from the first 20 citations for each search were classified into three relevance catego- ries. We used the weighted kappa statistic to analyse reviewer agreement and nonparametric rank tests to compare the number of citations for each article and the corresponding journals’ impact factors. Results: Reviewers ranked 67.6% of PubMed articles and 80% of Google Scholar articles as at least possi- bly relevant (P = 0.116) with high agreement (all kappa P-values < 0.01). Google Scholar articles had a higher median number of citations (34 vs. 1.5, P < 0.0001) and came from higher impact factor journals (5.17 vs. 3.55, P = 0.036). Conclusions: PubMed searches and Google Scholar searches often identify different articles. In this study, Google Scholar articles were more likely to be classified as relevant, had higher numbers of citations and were published in higher impact factor journals. The identification of frequently cited articles using Goo- gle Scholar for searches probably has value for initial literature searches.
Ortega, J.L., Aguillo, I.F. (2012). Science is all in the eye of the beholder: Keyword maps in Google scholar citations. DOI: http://dx.doi.org/10.1002/asi.22761 . Journal of the American Society for Information Science and Technology, 63(12), 2370–2377. This paper introduces a keyword map of the labels used by the scientists registered in the Google Scholar Citations (GSC) database from December 2011. In all, 15,000 random queries were formulated to GSC to obtain a list of 26,682 registered users. From this list a network graph of 6,660 labels was built and classified according to the Scopus Subject Area classes. Results display a detailed label map of the most used (>15 times) tags. The structural analysis shows that the core of the network is occupied by computer science–related disciplines that account for the most used and shared labels. This core is surrounded by clusters of disciplines related or close to computing such as Information Sciences, Mathematics, or Bioinformatics. Classical areas such as Chemistry and Physics are marginalized in the graph. It is suggested that GSC would in the future be an accurate source to map Science because it is based on the labels that scientists themselves use to describe their own research activity.

Varshney, L.R. (2012). The Google effect in doctoral theses. Scientometrics, 92(3), 785-793. DOI: http://dx.doi.org/10.1007/s11192-012-0654-4. It is often said that successive generations of researchers face an increasing educational burden due to knowledge accumulation. On the other hand, technological advancement over time can improve the productivity of researchers and even change their cognitive processes. This paper presents a longitudinal study (2004–2011) of citation behavior in doctoral theses at the Massachusetts Institute of Technology’s Department of Electrical Engineering and Computer Science. It is found that the number of references cited has increased over the years. At the same time, there has been a decrease in the length of time in the doctoral program and a relative constancy in the culture of the department. This suggests that students are more productive in facing an increased knowledge burden, and indeed seem to encode prior literature as transactive memory to a greater extent, as evidenced by the greater use of older literature.

Wang, Y., Howard, P. (2012). Google Scholar Usage: An Academic Library’s Experience. Journal of Web Librarianship, 6(2), 94–108. Google Scholar is a free service that provides a simple way to broadly search for scholarly works and to connect patrons with the resources libraries provide. The researchers in this study analyzed Google Scholar usage data from 2006 for three library tools at San Francisco State University: SFX link resolver, Web Access Management proxy server, and ILLiad interlibrary loan server. Overall, the data suggested that Google Scholar had become a very useful resource in the library and was a significant addition to the library's collection of research databases. SFX data revealed requests from Google Scholar grew ten-fold from 2006 to 2011, and that Google Scholar became the top-ranked SFX source for requests in 2011. Library patrons favored Google Scholar over San Francisco State University's federated search tool, MetaLib, and it has become an important source for interlibrary loan requests. Analysis of San Francisco State University usage data will assist other libraries in their decisions about the implementation of Google Scholar.
Zarifmahmoudi, L., Sadeghi, R. (2012). Citation analysis of Iranian journal of nuclear medicine: Comparison of SCOPUS and Google scholar. Iranian Journal of Nuclear Medicine, 20(2), 1–7. Introduction: Citation tracking is a bibliometrics method to analyze the scientific impact of journal articles which can be done through Scopus (SC), Google Scholar (GS), or ISI web of knowledge (WOS). In the current study, we analyzed the citations to 2006-2012 articles of Iranian Journal of Nuclear Medicine (IJNM) in the SC and GS. Methods: We retrieved the relevant data from SC and GS official websites. The search was done on 10/2012. Total number of citations, their overlap and unique citations of SC and GS were evaluated in detail. Results: SC and GS covered 100% and 99% of articles and identified 53 and 62 citations to IJNM articles respectively with the overlap of 44 citations. Original articles were the main types of cited articles followed by review articles. Conclusion: Despite considerable overlap between GS and SC, they provide important unique citations to IJNM articles. Due to differences between citation analysis information in each database, authors should consider all the indexing databases when evaluating the scientific impact of the individual journal. Editors should consider original and review articles to increase long term visibility and hopefully impact factor of IJNM in the future.
Zarifmahmoudi, L., Sadeghi, R. (2012). Comparison of ISI web of knowledge, SCOPUS, and Google scholar h-indices of Iranian nuclear medicine scientists. Iranian Journal of Nuclear Medicine, 20(1), 1–4. Introduction: In the current study, we compared the h-indices of Web of Science (WOS), SCOPUS, and GS of the Iranian nuclear medicine scientists Methods: Full time members of two major nuclear medicine research centers of Iran with more than 5 year of experience (Nuclear Medicine Research Center of Mashhad University of Medical Sciences, and Research Institute for Nuclear Medicine of Tehran University of Medical Sciences) were included for h-index evaluation. H-indices of SCOPUS, WOS and GS were retrieved using their specific websites. Correlations of h-indices with each other were evaluated using spearman correlation coefficient. Results: Overall 11 researchers were included in the study. SCOPUS, WOS, and GS provided somehow different h-indices for each researcher. Spearman's correlation coefficients between different h-indices were high: 0.834, 0.817, 0.857 between SCOPUS and WOS, SCOPUS and GS, and GS and WOS respectively. Rankings of researchers according to different database however, were acceptably identical. Conclusion: H-indices provided by SCOPUS, Web of Science WOS, and Google Scholar (GS) for Iranian nuclear medicine researchers can be used interchangeably. However these h-indices can be different according to which database is used. Setting up “ReasercherID” in WOS and “User profile” in GS, as well as giving regular feedback to SCOPUS managers can increase the accuracy of h-indices calculation.
 

2011 [Go back]

Adriaanse, L.S., Rensleigh, C. (2011). Comparing Web of Science, Scopus and Google Scholar from an Environmental Sciences perspective. South African Journal of Library Information Science, 77(2), 169–178.  This paper presents a macro- and micro-level comparison of the citation resources Web of Science (WOS), Scopus and Google Scholar (GS) for the environmental sciences scholarly journals in South Africa during 2004-2008. The macro-level measuring instruments consisted of 26 evaluation criteria with the following broad categories: content, access, services, interface, searching, search results, cost, citation and analytical tools, and linking abilities. The micro-level measuring instrument's evaluation criteria represented the data fields of the journal records to establish comprehensivity. The macro-level evaluation results indicated that Scopus surpassed both WOS and GS whereas the micro-level evaluation results indicated that WOS surpassed both Scopus and GS. Based on the macro- and micro-level evaluation results the study was able to establish that GS is not yet a substitute but rather a supplementary citation resource for the fee-based WOS and / or Scopus for the South African international accredited scholarly environmental sciences journals during the period 2004-2008.

Cothran, T. (2011). Google Scholar acceptance and use among graduate students: A quantitative study. Library & Information Science Research, 33(4), 293–301. DOI: http://dx.doi.org/10.1016/j.lisr.2011.02.001 .  Adding the external variables of satisfaction and loyalty to Fred Davis' technology acceptance model (TAM), this study examined the extent to which graduate students perceived Google Scholar to be a resource that is useful and easy to use. A survey of 1141 graduate students at the University of Minnesota asked questions exploring their perceptions of Google Scholar as part of their research process. Seventy-five percent of survey participants had used Google Scholar at least once before, and a statistical analysis of the responses found that perceived usefulness, loyalty, and, to a lesser extent, perceived ease of use, were positively and significantly related to the graduate students' intended use of the information resource. This research showed that TAM is an applicable model for predicting graduate student use of Google Scholar, which can help academic librarians seeking to understand graduate student acceptance of new information sources. Additionally, this study provides information about how librarians might best promote Google Scholar and other library resources to graduate students.
Dagienė, E. (2011). Comparison of Scientometric indicators used by Web of Science and Google Scholar. Evolution of Science and Technology, 3(2), 162–178. DOI: http://dx.doi.org/10.3846/est.2011.13 . More than a decade ago, the academic community has started the continuously escalating debate trying to ascertain whether commercial databases sufficiently reflect researcher activities undertaken in various fields of science and whether publications outside such databases were worthless. These long discussions gave rise to rapidly developing and improving open access databases, such as Google Scholar, Scirus, Microsoft Academic Search and etc. The article presents a research that aims to compare scientometric indicators on researchers of different scientific fields derived from free access databases with those supplied by the globally recognised Web of Science, and to ascertain the differences between the two. With consent of well-known Lithuanian professors, habilitated doctors, their indicators were collected and presented in this article. The aforementioned professors represent four scientific fields: educology, physics, engineering and economics. As the research revealed that some Lithuanian physicists have same family names as well as first names, indicators of a young doctor of physics were supplied for comparison. The article compares the researcher indicators collected from the Web of Science with those retrieved with the help of Publish or Perish software. This free access and easy to install software draws on Google Scholar data to calculate and produce a wealth of indicators, such as: the number of papers; variety of citations: average per year, per author and etc.; h-index and its variations, and etc. According to creator of Publish or Perish software Anne-Wil Harzing, all these indicators are useful for researchers as they help choosing the best way to present dissemination of their papers and ideas as well as input into development of global science, required while submitting various applications and etc. Results of the research confirmed the broadly used argument that more papers in the field of social sciences can be found with the help of Google Scholar rather than Web of Science, consequently, indicators produced by Publish or Perish are better.

Gabbidon, S.L., Collins, R. (2011). Using Google Scholar to Determine the Most Cited Criminology and Criminal Justice-Related Books. American Journal of Criminal Justice, 37(1), 33–45. DOI: http://dx.doi.org/10.1007/s12103-011-9130-6 . Building on recent research investigating the role of books in the discipline of criminology and criminal justice (C/CJ), this paper uses Google Scholar to identify the most cited books in the field. In particular, the researchers examined the most cited books in four different eras. Prior to1900, the most cited C/CJ-related book was On the Origin of Species. Merton’s Social Theory and Social Structure was the most cited book in the second era (1900–1949). The third era (1950–1999) produced the most cited work in the study, Foucault’s Discipline & Punish. And in the final era (2000 to present), Garland’s Culture of Control was the most cited work. The researchers also sought to determine the most cited books by women and African Americans/Blacks. The most cited book by a female author was Judith Herman’s Trauma and Recovery, and the most cited book by an African American/Black scholar was William Julius Wilson’s The Truly Disadvantaged. The authors conclude by arguing for the continued emphasis on demarcating the “great books” in the discipline.

Hodge, D.R., Lacasse, J.R. (2011). Ranking disciplinary journals with the Google Scholar h-index: A new tool for constructing cases for tenure, promotion, and other professional decisions. Journal of Social Work Education, 47(3), 579-596.  Given the importance of journal rankings to tenure, promotion, and other professional decisions, this study examines a new method for ranking social work journals. The Google Scholar h-index correlated highly with the current gold standard for measuring journal quality, Thomson Institute for Scientific Information (ISI) impact factors, but provided data for more than 4 times as many disciplinary journals. Eighty disciplinary periodicals are identified and ranked using the Google Scholar h-index. The vast majority of these were ranked higher than the lowest ranked social work journal indexed by Thomson ISI. Although the results hold salience for many professional stakeholders, they may be of particular interest to faculty who publish in disciplinary journals not indexed by Thomson ISI. The Google Scholar h-index provides faculty with an additional tool to document the quality of the venues in which they publish.


Kirkwood, H.P.J., Kirkwood, M.C. (2011). EconLit and Google Scholar Go Head-to-Head. Online Exploring Technology Resources for Information Professionals, 35(2), 38–41. In academic libraries, there are increased reliance on one-stop searching resources such as Google and Google Scholar. Both students and faculty members are gravitating away from focused databases, such as EconLit, and toward Google Scholar as an acceptable, if not better, alternative to subscription databases. EconLit is considered the primary economics literature database. Google Scholar is a search engine for scholarly literature using tools and algorithms that are similar to the main Google search engine. Interestingly, the EconLit and Google Scholar results were completely different, with not a single matching citation between the two search sets. The authors discovered that comparing results was more problematic than they expected, due to the vagaries of each resource. It was apparent, early on, that comparing a focused database such as EconLit to a mammoth, more general resource such as Google Scholar would be ineffective if the authors just looked at the number of results returned.
Kousha, K., Thelwall, M., Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science, 62(11), 2147–2164. DOI: http://dx.doi.org/10.1002/asi.21608 . Citation indictors are increasingly used in some subject areas to support peer review in the evaluation of researchers and departments. Nevertheless, traditional journal-based citation indexes may be inadequate for the citation impact assessment of book-based disciplines. This article examines whether online citations from Google Books and Google Scholar can provide alternative sources of citation evidence. To investigate this, we compared the citation counts to 1,000 books submitted to the 2008 U.K. Research Assessment Exercise (RAE) from Google Books and Google Scholar with Scopus citations across seven book-based disciplines (archaeology; law; politics and international studies; philosophy; sociology; history; and communication, cultural, and media studies). Google Books and Google Scholar citations to books were 1.4 and 3.2 times more common than were Scopus citations, and their medians were more than twice and three times as high as were Scopus median citations, respectively. This large number of citations is evidence that in book-oriented disciplines in the social sciences, arts, and humanities, online book citations may be sufficiently numerous to support peer review for research evaluation, at least in the United Kingdom.
 

Sanni, S.A., Zainab, A.N. (2011). Evaluating the influence of a medical journal using Google Scholar. Learned Publishing, 24(2), 145–154. DOI: http://dx.doi.org/10.1087/20110210 . This study shows how a journal's influence can be calculated by using citations obtained from Google Scholar and other methods even though the journal is not covered by any citation databases. Influence is measured in terms of foreign contributions, 'equivalent' immediacy scores of recent articles, and the calculation of citations and 'equivalent' impact factor. A total of 580 articles published in the Medical Journal of Malaysia (MJM) between 2004 and 2008 served as the sample. Very few foreign authors contributed to MJM (12.5%), implying its low regional acceptance as a channel for research communication. Immediacy scores for each year indicate citations were received by recently published articles. A total of 1,164 citations were received by 446 of the 580 articles and the main citing sources were journals (1,083) with reasonable h index and impact factor. Yearly impact scores ranged between 0.367 and 0.616. Higher impact factor scores were obtained by older articles. ABSTRACT FROM AUTHOR; Copyright of Learned Publishing is the property of Association of Learned & Professional Society Publishers (ALPSP) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.

Thor, A., Bornmann, L. (2011). The calculation of the single publication h-index and related performance measures: A web application based on Google Scholar data. Online Information Review, 35(2), 291–300. DOI: http://dx.doi.org/10.1108/14684521111128050 . Purpose - The single publication h index has been introduced by Schubert as the h index calculated from the list of citing publications of one single publication. This paper aims to look at the calculation of the single publication h index and related performance measures. Design/methodology/ approach - In this paper a web application is presented where the single publication h index and related performance measures (the single publication m index, h 2 lower, h 2 centre, and h 2 upper) can be automatically calculated for any publication indexed by Google Scholar. Findings - The use of the application is demonstrated by means of the citation performance of two publications. Originality/value - To the authors' knowledge this web application is the first instrument to automatically calculate the single publication h index and related performance measures based on Google Scholar data. This is a new service especially from the perspective of the related performance measures.
 
Tober, M. (2011). PubMed, ScienceDirect, Scopus or Google Scholar – Which is the best search engine for an effective literature research in laser medicine?. Medical Laser Application, 26(3), 139–144.  DOI: http://dx.doi.org/10.1016/j.mla.2011.05.006. The four most popular search engines PubMed/MEDLINE, ScienceDirect, Scopus and Google Scholar are investigated to assess which search engine is most effective for literature research in laser medicine. Their search features are described and the results of a performance test are compared according to the criteria (1) recall, (2) precision, and (3) importance. As expected, the search features provided by PubMed/MEDLINE with a comprehensive investigation of medical documents are found to be exceptional compared to the other search engines. However the most effective search engine for an overview of a topic is Scopus, followed by ScienceDirect and Google Scholar. With regard to the criterion “importance” Scopus and Google Scholar are clearly more successful than their competitors.

Walters, W.H. (2011). Comparative recall and precision of simple and expert searches in google scholar and eight other databases. portal: Libraries and the Academy, 11(4), 972–1006. This study evaluates the effectiveness of simple and expert searches in Google Scholar (GS), EconLit, GEOBASE, PAIS, POPLINE, PubMed, Social Sciences Citation Index, Social Sciences Full Text, and Sociological Abstracts. It assesses the recall and precision of 32 searches in the field of later-life migration: nine simple keyword searches and 23 expert searches constructed by demography librarians at three top universities. For simple searches, Google Scholar's recall and precision are well above average. For expert searches, the relative effectiveness of GS depends on the number of results users are willing to examine. Although Google Scholar's expert-search performance is just average within the first fifty search results, GS is one of the few databases that retrieves relevant results with reasonably high precision after the fiftieth hit. The results also show that simple searches in GS, GEOBASE, PubMed, and Sociological Abstracts have consistently higher recall and precision than expert searches. This can be attributed not to differences in expert-search effectiveness, but to the unusually strong performance of simple searches in those four databases.

 

2010  [Go back]


Anders, M.E., Evans, D.P. (2010). Comparison of PubMed and Google Scholar literature searches. Respiratory care, 55(5), 578–583. Background: Literature searches are essential to evidence-based respiratory care. To conduct literature searches, respiratory therapists rely on search engines to retrieve information, but there is a dearth of literature on the comparative efficiencies of search engines for researching clinical questions in respiratory care.Objective: To compare PubMed and Google Scholar search results for clinical topics in respiratory care to that of a benchmark.Methods: We performed literature searches with PubMed and Google Scholar, on 3 clinical topics. In PubMed we used the Clinical Queries search filter. In Google Scholar we used the search filters in the Advanced Scholar Search option. We used the reference list of a related Cochrane Collaboration evidence-based systematic review as the benchmark for each of the search results. We calculated recall (sensitivity) and precision (positive predictive value) with 2 × 2 contingency tables. We compared the results with the chi-square test of independence and Fisher's exact test.Results: PubMed and Google Scholar had similar recall for both overall search results (71% vs 69%) and full-text results (43% vs 51%). PubMed had better precision than Google Scholar for both overall search results (13% vs 0.07%, P < .001) and full-text results (8% vs 0.05%, P < .001).Conclusions: Our results suggest that PubMed searches with the Clinical Queries filter are more precise than with the Advanced Scholar Search in Google Scholar for respiratory care topics. PubMed appears to be more practical to conduct efficient, valid searches for informing evidence-based patient-care protocols, for guiding the care of individual patients, and for educational purposes.

Bar-Ilan, J. (2010). Citations to the “Introduction to informetrics” indexed by WOS, Scopus and Google Scholar. Scientometrics, 82(3), 495–506. DOI: http://dx.doi.org/10.1007/s11192-010-0185-9 . Google Scholar and Scopus are recent rivals to Web of Science. In this paper we examined these three citation databases through the citations of the book "Introduction to informetrics" by Leo Egghe and Ronald Rousseau. Scopus citations are comparable to Web of Science citations when limiting the citation period to 1996 and onwards (the citation coverage of Scopus)-each database covered about 90% of the citations located by the other. Google Scholar missed about 30% of the citations covered by Scopus and Web of Science (90 citations), but another 108 citations located by Google Scholar were not covered either by Scopus or by Web of Science. Google Scholar performed considerably better than reported in previous studies, however Google Scholar is not very "user-friendly" as a bibliometric data collection tool at this point in time. Such "microscopic" analysis of the citing documents retrieved by each of the citation databases allows us a deeper understanding of the similarities and the differences between the databases.
Beel, J., Gipp, B., Wilde , E. (2010). Academic Search Engine Optimization (ASEO): Optimizing Scholarly Literature for Google Scholar and Co. Journal of Scholarly Publishing, 41 (2): 176–190. DOI: http://dx.doi.org/10.3138/jsp.41.2.176 This article introduces and discusses the concept of academic search engine optimization (ASEO). Based on three recently conducted studies, guidelines are provided on how to optimize scholarly literature for academic search engines in general and for Google Scholar in particular. In addition, we briefly discuss the risk of researchers’ illegitimately ‘over-optimizing’ their articles.

Beel, J., Gipp, B. (2010). Academic Search Engine Spam and Google Scholar’s Resilience Against it. Journal of electronic publishing,the(JEP), 13(3). DOI: http://dx.doi.org/10.3998/3336451.0013.305 In a previous paper we provided guidelines for scholars on optimizing research articles for academic search engines such as Google Scholar. Feedback in the academic community to these guidelines was diverse. Some were concerned researchers could use our guidelines to manipulate rankings of scientific articles and promote what we call ‘academic search engine spam’. To find out whether these concerns are justified, we conducted several tests on Google Scholar. The results show that academic search engine spam is indeed—and with little effort—possible: We increased rankings of academic articles on Google Scholar by manipulating their citation counts; Google Scholar indexed invisible text we added to some articles, making papers appear for keyword searches the articles were not relevant for; Google Scholar indexed some nonsensical articles we randomly created with the paper generator SciGen; and Google Scholar linked to manipulated versions of research papers that contained a Viagra advertisement. At the end of this paper, we discuss whether academic search engine spam could become a serious threat to Web-based academic search engines.


Beira, E. (2010). Innovation and competition in scholar information services: from Eugene Garfield to google Scholar. Encontros Bibli: Revista Eletrônica de Biblioteconomia e Ciência da Informação, (2010), 132–163. During last forty years scholarly information services developed from near non existence to a significant business with deep implications in the management of science and academic institutions. We discuss the innovative business struggle of Eugene Garfield and ISI Institute for Scientific Information until mid 90's, and the changes after the Google Scholar entered the market with an highly disruptive business model. The processes are discussed in the context of social frameworks for business innovation. Changes in the market during last decade are analysed and the implications for future are explored. Empirical data is presented comparing search results from ISI/WoS and Google Scholar that show the legacy effect of ISI / WoS data architecture. Results also show limitations from both services. The real number of citations is underestimated by both sources, and unique citations from each source are the majority for the cases discussed. The actual “citations wars” are discussed in the context of the previous “science wars”, and the permanent search for the meaning and justification of science and academic activities. We argue that the open and dynamic model of Google Scholar is much more coherent with the reality, allowing a much more complete view of all the burgeoning and varied processes involved in the continuous struggle of science and academic work – something with important policy implications. 



Chen, X. (2010). Google Scholar’s Dramatic Coverage Improvement Five Years after Debut. Serials Review, 36(4), 221–226. DOI: http://dx.doi.org/10.1080/00987913.2010.10765321 .  This article reports a 2010 empirical study using a 2005 study as a base to compare Google Scholar's coverage of scholarly journals with commercial services. Through random samples of eight databases, the author finds that, as of 2010, Google Scholar covers 98 to 100 percent of scholarly journals from both publicly accessible Web contents and from subscription-based databases that Google Scholar partners with. In 2005 the coverage of the same databases ranged from 30 to 88 percent. The author explores de-duplication of search results by Google Scholar and discusses its impacts on searches and library resources. With the dramatic improvement of Google Scholar, the uniqueness and effectiveness of subscription-based abstracts and indexes have dramatically changed. 
 

Cutler, D. (2010). ETDEWEB versus the World-Wide-Web: A Specific Database/Web Comparison (No. ETDE/OA--237). Energy Technology Data Exchange (ETDE Operating Agent), USDOE/OSTI (Office of Scientific and Technical Information), Oak Ridge, TN (United States). A study was performed comparing user search results from the specialized scientific database on energy-related information, ETDEWEB, with search results from the internet search engines Google and Google Scholar. The primary objective of the study was to determine if ETDEWEB (the Energy Technology Data Exchange – World Energy Base) continues to bring the user search results that are not being found by Google and Google Scholar. As a multilateral information exchange initiative, ETDE’s member countries and partners contribute cost- and task-sharing resources to build the largest database of energy-related information in the world. As of early 2010, the ETDEWEB database has 4.3 million citations to world-wide energy literature. One of ETDEWEB’s strengths is its focused scientific content and direct access to full text for its grey literature (over 300,000 documents in PDF available for viewing from the ETDE site and over a million additional links to where the documents can be found at research organizations and major publishers globally). Google and Google Scholar are well-known for the wide breadth of the information they search, with Google bringing in news, factual and opinion-related information, and Google Scholar also emphasizing scientific content across many disciplines. The analysis compared the results of 15 energy-related queries performed on all three systems using identical words/phrases. A variety of subjects was chosen, although the topics were mostly in renewable energy areas due to broad international interest. Over 40,000 search resultrecords from the three sources were evaluated. The study concluded that ETDEWEB is a significant resource to energy experts for discovering relevant energy information. For the 15 topics in this study, ETDEWEB was shown to bring the user unique results not shown by Google or Google Scholar 86.7% of the time. Much was learned from the study beyond just metric comparisons. Observations about the strengths of each system and factors impacting the search results are also shared along with background information and summary tables of the results. If a user knows a very specific title of a document, all three systems are helpful in finding the user a source for the document. But if the user is looking to discover relevant documents on a specific topic, each of the three systems will bring back a considerable volume of data, but quite different in focus. Google is certainly a highly-used and valuable tool to find significant ‘non-specialist’ information, and Google Scholar does help the user focus on scientific disciplines. But if a user’s interest is scientific and energy-specific, ETDEWEB continues to hold a strong position in the energy research, technology and development (RTD) information field and adds considerable value in knowledge discovery.

Dixon, L., Duncan, C., Fagan, J.C., Mandernach, M., Warlick, S.E. (2010). Finding Articles and Journals via Google Scholar, Journal Portals, and Link Resolvers: Usability Study Results. Reference & User Services Quarterly, 50(2), 170–181 . Finding journal titles and journal articles are two of the toughest tasks on academic library webpages. Challenges include choosing the best tools, using terms that make sense, and guiding the user through the process. In addition, the continued development of Google Scholar raises the question of whether it could become a better tool for finding a full-text article than link resolver software or journal portals. To study these issues, researchers at James Madison University analyzed results from two usability tests. One usability test focused on the library homepage navigation and had two tasks related to finding articles by citation and journals by title. The other test asked participants to find citations in three web interfaces: the library’s journal portal, Google Scholar, and the library’s link resolver form. Both usability studies revealed challenges with finding journal titles and journal articles. The latter study showed Google Scholar provided more effective user performance and user satisfaction than either the journal portal or the link resolver form. Based on the findings from the two usability studies, specific changes were made to the library webpages and to several library systems, including the catalog and link resolver form.

Etxebarria, G., Gomez-Uranga, M. (2010). Use of Scopus and Google Scholar to measure social sciences production in four major Spanish universities. Scientometrics, 82(2), 333-349. DOI: http://dx.doi.org/10.1007/s11192-009-0043-9 . A large part of Social Sciences and the Humanities do not adapt to international proceedings used in English for scientific output on databases such as the Web of Science and Scopus. The aim of this paper is to show the different results obtained in scientific work by comparing Social Sciences researchers with those of other sciences in four Spanish universities. The first finding is that some Social Sciences researchers are somewhat internationalised. However, the majority of individuals who are prestigious in their local academic-scientific community do not even appear on the information sources mentioned above.

García-Pérez, M.A. (2010). Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h indices in Psychology. Journal of the American Society for Information Science and Technology, 61(10), 2070–2085. DOI: http://dx.doi.org/10.1002/asi.21372 . Hirsch's h index is becoming the standard measure of an individual's research accomplishments. The aggregation of individuals' measures is also the basis for global measures at institutional or national levels. To investigate whether the h index can be reliably computed through alternative sources of citation records, the Web of Science (WoS), PsycINFO and Google Scholar (GS) were used to collect citation records for known publications of four Spanish psychologists. Compared with WoS, PsycINFO included a larger percentage of publication records, whereas GS outperformed WoS and PsycINFO in this respect. Compared with WoS, PsycINFO retrieved a larger number of citations in unique areas of psychology, but it retrieved a smaller number of citations in areas that are close to statistics or the neurosciences, whereas GS retrieved the largest numbers of citations in all cases. Incorrect citations were scarce in Wos (0.3%), more prevalent in PsycINFO (1.1%), and overwhelming in GS (16.5%). All platforms retrieved unique citations, the largest set coming from GS. WoS and PsycINFO cover distinct areas of psychology unevenly, thus applying different penalties on the h index of researches working in different fields. Obtaining fair and accurate h indices required the union of citations retrieved by all three platforms.
Hightower, C., Caldwell, C. (2010). Shifting Sands: Science researchers on Google Scholar, Web of Science, and PubMed, with implications for library collections budgets. Issues in Science and Technology Librarianship. DOI: http://dx.doi.org/10.5062/F4V40S4J . Science researchers at the University of California Santa Cruz were surveyed about their article database use and preferences in order to inform collection budget choices. Web of Science was the single most used database, selected by 41.6%. Statistically there was no difference between PubMed (21.5%) and Google Scholar (18.7%) as the second most popular database. 83% of those surveyed had used Google Scholar and an additional 13% had not used it but would like to try it. Very few databases account for the most use, and subject-specific databases are used less than big multidisciplinary databases (PubMed is the exception). While Google Scholar is favored for its ease of use and speed, those who prefer Web of Science feel more confident about the quality of their results than do those who prefer Google Scholar. When asked to choose between paying for article database access or paying for journal subscriptions, 66% of researchers chose to keep journal subscriptions, while 34% chose to keep article databases.

Jaćimović, J., Petrović, R., Živković, S. (2010). A citation analysis of Serbian Dental Journal using Web of Science, Scopus and Google Scholar. Stomatoloski glasnik Srbije, 57(4), 201-211. DOI: http://dx.doi.org/10.2298/SGS1004201J . Introduction. For a long time, The Institute for Scientific Information (ISI, now Thomson Scientific, Philadelphia, US) citation databases, available online through the Web of Science (WoS), had an unique position among bibliographic databases. The emergence of new citation databases, such as Scopus and Google Scholar (GS), call in question the dominance of WoS and the accuracy of bibliometric and citation studies exclusively based on WoS data. The aim of this study was to determine whether there were significant differences in the received citation counts for Serbian Dental Journal (SDJ) found in WoS and Scopus databases, or whether GS results differed significantly from those obtained by WoS and Scopus, and whether GS could be an adequate qualitative alternative for commercial databases in the impact assessment of this journal. Material and Methods. The data regarding SDJ citation was collected in September 2010 by searching WoS, Scopus and GS databases. For further analysis, all relevant data of both, cited and citing articles, were imported into Microsoft Access® database. Results. One hundred and fifty-eight cited papers from SDJ and 249 received citations were found in the three analyzed databases. 74% of cited articles were found in GS, 46% in Scopus and 44% in WoS. The greatest number of citations (189) was derived from GS, while only 15% of the citations, were found in all three databases. There was a significant difference in the percentage of unique citations found in the databases. 58% originated from GS, while Scopus and WoS gave 6% and 4% unique citations, respectively. The highest percentage of databases overlap was found between WoS and Scopus (70%), while the overlap between Scopus and GS was 18% only. In case of WoS and GS the overlap was 17%. Most of the SDJ citations came from original scientific articles. Conclusion. WoS, Scopus and GS produce quantitatively and qualitatively different citation counts for SDJ articles. None of the examined databases can provide a comprehensive picture and it is necessary to take into account all three available sources.


Jacso, P. (2010). Calculating the h-index and other bibliometric and scientometric indicators from Google Scholar with the Publish or Perish software. Online Information Review, 33(6), 1189-1200. DOI: http://dx.doi.org/10.1108/14684520911011070  Purpose – The purpose of this paper is to is to discuss the results of recent experiments in calculating the h-index and other bibliometric and scientometric indicators from Google Scholar with the Publish or Perish software. Design/methodology/approach – The paper discusses the Publish or Perish (PoP) software and finds that is a swift and elegant tool to provide the essential output features that Google Scholar does not offer. Findings – It is found that PoP allows the user to edit the result lists presented in a compact, efficient grid-format. It facilitates the identification and removal of duplicate entries by offering dynamic sorting of the set by eight metadata elements, un-checking items and instant recalculation of the indicators. Originality/value – Some changes are recommended to enhance this useful utility by allowing users to clean and edit the erroneous entries in the result set, and then back-load it to PoP for the recalculation of the indicators. It is also suggested that the option to upload into PoP the result lists produced in CSV format from Web of Science and Scopus (which have much more reliable and reproducible data than Google Scholar) should also be offered.



Jacso, P. (2010). Metadata mega mess in Google Scholar. Online Information Review, 34(1), 175-191Purpose – Google Scholar (GS) has shed the beta label on the fifth anniversary of launching its service. This paper aims to address this issue. Design/methodology/approach – As good as GS is – through its keyword search option – to find information about tens of millions of documents, many of them in open access full text format, it is as bad for metadata-based searching when, beyond keywords in the title, abstract, descriptor and/or full text, the searcher also has to use author name, journal title and/or publication year in specifying the query. This paper provides a review of recent developments in Google Scholar. Findings – GS is especially inappropriate for bibliometric searches, for evaluating the publishing performance and impact of researchers and journals. Originality/value – Even if the clean up of Google Scholar accelerates it should not be forgotten that those evaluations of individuals and journals that have been done based on Google Scholar in the past few years have grossly handicapped many authors and journals whose name was replaced by phantom entries.The paper highlights the very time-consuming process of corroborating data, tracing and counting valid citations and points out GS's unscholarly and irresponsible handling of data.


Jacso, P. (2010). Pragmatic issues in calculating and comparing the quantity and quality of research through rating and ranking of researchers based on peer reviews and bibliometric indicators from Web of Science, Scopus and Google Scholar. Online Information Review, 34(6), 972-982. DOI: http://dx.doi.org/10.1108/14684521011099432 Purpose – The purpose of this paper is to analyse the findings of two recently published papers (Norris and Oppenheim, 2003; and Li et al., 2010). Design/methodology/approach – The findings were analysed from the practitioner's perspective about the procedures involved in calculating the indicator values and the ranks and ratings. This was done with the purpose of playing the devil's advocate, contemplating the reservations and arguments of those who do not want to use metrics based on database searches. Findings – One advantage of this project is that its results can be compared at least partially with the findings of the three earlier RAEs (although its grade classes have changed), as well as with some of the other ranking lists in library and information management areas. Originality/value – Very importantly, the authors concluded that “it would be premature in the extreme to suggest that citation-based indicators could be used as a cost-effective alternative to expert judgments”. This is a strong, very realistic and fair statement. Even this recent project's results are very valuable in spite of the problems mentioned.



Law, R. (2010). An analysis of the impact of tourism journals on Google Scholar. In Information and communication technologies in tourism 2010 , 333-343. DOI: http://dx.doi.org/10.1007/978-3-211-99407-8_28 . In spite of the increasing emphasis on the quality of publications in academia, there exists no standard list of ranked journals that is accepted by all universities and researchers. Such an absence of ranked journals is particularly true in the context of tourism, an emerging academic discipline. This study introduced a novel approach that evaluates the impact of tourism journals, which is operationalized as the average number of citations for each published article in the included tourism journals that are found by Google Scholar [GS]. Utilized the data collected from GS in three different time periods, findings showed the mostly cited journals generally matched prior studies. In particular, Tourism Management [TM] ranked first in 2009 among all included journals. In addition, the Journal of Information Technology & Tourism [JITT], which ranked fifth among all included journals, appears as the specialized journal that received the largest average citations.
Lewandowski, D. (2010). Google Scholar as a tool for discovering journal articles in library and information science. Online Information Review, 34(2), 250–262. DOI: http://dx.doi.org/10.1108/14684521011036972 . Purpose The purpose of this paper is to measure the coverage of Google Scholar for Library and Information Science (LIS) journal literature as identified by a list of core LIS journals from a study by Schlo gl and Petschnig. Design/methodology/approach The paper checked every article from 35 major LIS journals from the years 2004 to 2006 for availability in Google Scholar. It also collected information on the type of availability whether a certain article was available as a PDF for a fee, as a free PDF or as a preprint. Findings The paper found that only some journals are completely indexed by Google Scholar, that the ratio of versions available depends on the type of publisher, and that availability varies a lot from journal to journal. Google Scholar cannot substitute for abstracting and indexing services in that it does not cover the complete literature of the field. However, it can be used in many cases to easily find available full texts of articles already identified using another tool. Originality/value The study differs from other Google Scholar coverage studies in that it takes into account not only whether an article is indexed in Google Scholar at all, but also the type of availability.

Li, J., Burnham, J.F., Lemley, T., Britton, R.M. (2010). Citation Analysis: Comparison of Web of Science®, Scopus™, SciFinder®, and Google Scholar. Journal of electronic resources in medical libraries, 7(3), 196-217. DOI: http://dx.doi.org/10.1080/15424065.2010.505518 In recent years, numerous articles have compared the coverage, features, and citation analysis capabilities of Scopus™ and Google Scholar with Web of Science®, a Web-based version of Science Citation Index. This article goes a step further and compares the citation analysis potential of four databases: Web of Science, Scopus, SciFinder, and Google Scholar. Each database presents its own strengths and weaknesses, including methods of analysis, differences in coverage, and means of linking references. As an illustration, Web of Science provides coverage back to 1900. In contrast, Scopus only has completed citation information from 1996 onward, yet Scopus provides better coverage of clinical medicine and nursing than Web of Science. SciFinder has the strongest coverage of chemistry and the natural sciences, while Google Scholar has the capability to link citation information to individual references. Although Scopus and Web of Science provide comprehensive citation reports, all databases miss linking to some references included in other databases.
Li, J., Sanderson, M., Willett, P., Norris, M., & Oppenheim, C. (2010). Ranking of library and information science researchers: Comparison of data sources for correlating citation data, and expert judgments. Journal of Informetrics, 4(4), 554-563 DOI: http://dx.doi.org/10.1080/15424065.2010.505518 This paper studies the correlations between peer review and citation indicators when evaluating research quality in library and information science (LIS). Forty-two LIS experts provided judgments on a 5-point scale of the quality of research published by 101 scholars; the median rankings resulting from these judgments were then correlated with h-, g- and H-index values computed using three different sources of citation data: Web of Science (WoS), Scopus and Google Scholar (GS). The two variants of the basic h-index correlated more strongly with peer judgment than did the h-index itself; citation data from Scopus was more strongly correlated with the expert judgments than was data from GS, which in turn was more strongly correlated than data from WoS; correlations from a carefully cleaned version of GS data were little different from those obtained using swiftly gathered GS data; the indices from the citation databases resulted in broadly similar rankings of the LIS academics; GS disadvantaged researchers in bibliometrics compared to the other two citation database while WoS disadvantaged researchers in the more technical aspects of information retrieval; and experts from the UK and other European countries rated UK academics with higher scores than did experts from the USA.



Mastrangelo, G., Fadda, E., Rossi, C.R., Zamprogno, E., Buja, A., Cegolon, L. (2010). Literature search on risk factors for sarcoma: PubMed and Google Scholar may be complementary sources. BMC research notes, 3(1), 131. DOI: http://dx.doi.org/10.1186/1756-0500-3-131 Background: Within the context of a European network dedicated to the study of sarcoma the relevant literature on sarcoma risk factors was collected by searching PubMed and Google Scholar, the two information storage and retrieval databases which can be accessed without charge. The present study aims to appraise the relative proficiency of PubMed and Google Scholar. Findings: Unlike PubMed, Google Scholar does not allow a choice between "Human" and "Animal" studies, nor between "Classical" and other types of studies. As a result, searches with Google Scholar produced high numbers of citations that have to be filtered. Google Scholar resulted in a higher sensitivity (proportion of relevant articles, meeting the search criteria), while PubMed in a higher specificity (proportion of lower quality articles not meeting the criteria, that are not retrieved). Concordance between Google Scholar and PubMed was as low as 8%. Conclusions: This study focused just on one topic. Although further studies are warranted, PM and GS appear to be complementary and their integration could greatly improve the search of references in medical research.


Miguel, S., Herrero-Solana, V. (2010). Visibility of Latin American journals of Library and Information Science from Google Scholar. Ciencia da Informação, 39(2), 54–67. DOI: http://dx.doi.org/10.1590/S0100-19652010000200004 .  The academic-scientific journals are one of the outstanding means of communication and diffusion of the investigation results. Nevertheless they don’t have all the same prestige and degree of infl uence in the scientific community. The possibilities that a work is well-known, read and cited depend on its quality and visibility. The aim of this investigation is to determine the visibility in Google Scholar of a set of Latin-American Library and Information Science journals, and to classify them according to its degree of visibility. Using the Publish or Perish application to recover journal cites, indicators based on citation were estimated. Techniques of multivaried analysis of data were combined to make groups of journals according to their degree of visibility. Finally, graphical representations that facilitate the interpretation of the results and the identifi cation of the groups of journals with high, mean and low visibility were elaborated.

Mingers, J., Lipitakis, E.A.E.C.G.  (2010). Counting the citations: a comparison of Web of Science and Google Scholar in the field of business and management. Scientometrics, 85(2), 613–625. DOI: http://dx.doi.org/10.1007/s11192-010-0270-0 . Assessing the quality of the knowledge produced by business and management academics is increasingly being metricated. Moreover, emphasis is being placed on the impact of the research rather than simply where it is published. The main metric for impact is the number of citations a paper receives. Traditionally this data has come from the ISI Web of Science but research has shown that this has poor coverage in the social sciences. A newer and different source for citations is Google Scholar. In this paper we compare the two on a dataset of over 4,600 publications from three UK Business Schools. The results show that Web of Science is indeed poor in the area of management and that Google Scholar, whilst somewhat unreliable, has a much better coverage. The conclusion is that Web of Science should not be used for measuring research impact in management.
Moussa, S., Touzani, M. (2010). Ranking marketing journals using the Google Scholar-based hg-index. Journal of Informetrics, 4(1), 107–117. DOI: http://dx.doi.org/10.1016/j.joi.2009.10.001 . This paper provides a ranking of 69 marketing journals using a new Hirsch-type index, the hg-index which is the geometric mean of hg. The applicability of this index is tested on data retrieved from Google Scholar on marketing journal articles published between 2003 and 2007. The authors investigate the relationship between the hg-ranking, ranking implied by Thomson Reuters’ Journal Impact Factor for 2008, and rankings in previous citation-based studies of marketing journals. They also test two models of consumption of marketing journals that take into account measures of citing (based on the hg-index), prestige, and reading preference.
Sanni, S.A., Zainab, A.N. (2010). Google Scholar as a source for citation and impact analysis for a non-ISI indexed medical journal. Malaysian Journal of Library and Information Science, 15(3), 35–51. It is difficult to determine the influence and impact of journals which are not covered by the ISI databases and Journal Citation Record. However, with the availability of databases such as MyAIS (Malaysian Abstracting and Indexing System), which offers sufficient information to support bibliometric analysis as well as being indexed by Google Scholar which provides citation information, it has become possible to obtain productivity, citation and impact information for non-ISI indexed journals. The bibliometric tool Harzing’s Publish and Perish was used to collate citation information from Google scholar. The study examines article productivity, the citations obtained by articles and calculates the impact factor of Medical Journal of Malaysia (MJM) published between 2004 and 2008. MJM is the oldest medical journal in Malaysia and the unit of analysis is 580 articles. The results indicate that once a journal is covered by MyAIS it becomes visible and accessible on the Web because Google Scholarindexes MyAIS. The results show that contributors to MJM were mainly Malaysian (91%) and the number of Malaysian-Foreign collaborated papers were very small (28 articles, 4.8%). However, citation information from Google scholar indicates that out of the 580 articles, 76.8% (446) have been cited over the 5-year period. The citations were received from both mainstrean foreign as well as Malaysian journals and the top three citors were from China, Malaysia and the United States. In general more citations were received from East Asian countries, Europe, and Southeast Asia.The 2-yearly impact factor calculated for MJM is 0.378 in 2009, 0.367 in 2008, 0.616 in 2007 and 0.456 in 2006.The 5-year impact factor is calculated as 0.577.The results show that although MJM is a Malaysian journal and not ISI indexed its contents have some international significance based on the citations and impact score it receives, indicating the importance of being visible especially in Google Scholar.
Saracevic, T., Garfield, E. (2010) On measuring the publication productivity and citation impact of a scholar: A case study.  The Janus Faced Scholar: A Festschrift in Honour of Peter Ingwersen, 6, 185-199. Det Informationsvidenskabelige Akademi (Royal School of Library and Information Science, Copenhagen); ISSI.
The purpose is to provide quantitative evidence of scholarly productivity and impact of Peter Ingwersen, a preeminent information science scholar, and at the same time illustrate and discuss problems and disparities in measuring scholarly contribution in general. Data is derived from searching Dialog, Web of Science, Scopus, and Google Scholar (using Publish or Perish software). In addition, a HistCite profile for Peter Ingwersen publications and citations was generated.
Šember, M., Utrobičić, A., Petrak, J. (2010). Croatian Medical Journal Citation Score in Web of Science, Scopus, and Google Scholar. Croatian Medical Journal, 51(2), 99–103. DOI: http://dx.doi.org/10.3325/cmj.2010.51.99 . Aim To analyze the 2007 citation count of articles published by the Croatian Medical Journal in 2005-2006 based on data from the Web of Science, Scopus, and Google Scholar. Methods Web of Science and Scopus were searched for the articles published in 2005-2006. As all articles returned by Scopus were included in Web of Science, the latter list was the sample for further analysis. Total citation counts for each article on the list were retrieved from Web of Science, Scopus, and Google Scholar. The overlap and unique citations were compared and analyzed. Proportions were compared using chi(2)-test. Results Google Scholar returned the greatest proportion of articles with citations (45%), followed by Scopus (42%), and Web of Science (38%). Almost a half (49%) of articles had no citations and 11% had an equal number of identical citations in all 3 databases. The greatest overlap was found between Web of Science and Scopus (54%), followed by Scopus and Google Scholar (51%), and Web of Science and Google Scholar (44%). The greatest number of unique citations was found by Google Scholar (n = 86). The majority of these citations (64%) came from journals, followed by books and {PhD} theses. Approximately 55% of all citing documents were full-text resources in open access. The language of citing documents was mostly English, but as many as 25 citing documents (29%) were in Chinese. Conclusion Google Scholar shares a total of 42% citations returned by two others, more influential, bibliographic resources. The list of unique citations in Google Scholar is predominantly journal based, but these journals are mainly of local character. Citations received by internationally recognized medical journals are crucial for increasing the visibility of small medical journals but Google Scholar may serve as an alternative bibliometric tool for an orientational citation insight.

Van Aalst, J. (2010). Using Google Scholar to Estimate the Impact of Journal Articles in Education. Educational Researcher, 39(5), 387–400. DOI: http://dx.doi.org/10.3102/0013189X10371120 .  This article discusses the potential of Google Scholar as an alternative or complement to the Web of Science and Scopus for measuring the impact of journal articles in education. Three handbooks on research in science education, language education, and educational technology were used to identify a sample of 112 accomplished scholars. Google Scholar, Web of Science, and Scopus citations for 401 journal articles published by these authors during the 5-year period from 2003 to 2007 were then analyzed. The findings illustrate the promise and pitfalls of using Google Scholar for characterizing the influence of research output, particularly in terms of differences between the three subfields in publication practices. A calibration of the growth of Google Scholar citations is also provided.

2009 [Go back]

Auffhammer, M. (2009). The State of Environmental and Resource Economics: A Google Scholar Perspective. Review of Environmental Economics and Policy, 3(2), 251–269. DOI: http://dx.doi.org/10.1093/reep/rep001 .  Until recently, ISI Thompson's Web of Science/Social Sciences Citation Index was the only rigorous tool for tracking citation counts of academic research papers. The recent emergence of Google Scholar provides an alternative measure for tracking citation counts for refereed journal articles, conference proceedings, working papers, and government reports. This article provides an overview of the state of environmental and resource economics using the Google Scholar measure of citations. It ranks and compares the major field journals, and the most cited papers in these journals, the most cited papers in the field that have been published in general economics journals, and the most cited technical books and textbooks, as well as the most cited researchers in the field.

Baldwin, V.A. (2009). Using google scholar to search for online availability of a cited article in engineering disciplines. Issues in Science and Technology Librarianship, 56. DOI: http://dx.doi.org/10.5062/F4WM1BBC . Many published studies examine the effectiveness of Google Scholar (Scholar) as an index for scholarly articles. This paper analyzes the value of Scholar in finding and labeling online full text of articles using titles from the citations of engineering faculty publications. For the fields of engineering and the engineering colleges in the study, Scholar identified online access for 25% of the chemical engineering and 13% of the mechanical engineering citations. During the study the format that Scholar (which is in beta version) used to present the result set changed. This change now makes discovery of online access to full text of an article readily apparent when it occurs.
Beel, J., Gipp, B. (2009). Google Scholar’s ranking algorithm: an introductory overview. In Proceedings of the 12th International Conference on Scientometrics and Informetrics (ISSI’09) , 1, 230-241. Google Scholar is one of the major academic search engines but its ranking algorithm for academic articles is unknown. We performed the first steps to reverse-engineering Google Scholars ranking algorithm and present the results in this research-in-progress paper. The results are: Citation counts is the highest weighed factor in Google Scholars ranking algorithm. Therefore, highly cited articles are found significantly more often in higher positions than articles that have been cited less often. As a consequence, Google Scholar seems to be more suitable for finding standard literature than gems or articles by authors advancing a new or different view from the mainstream. However, interesting exceptions for some search queries occurred. Moreover, the occurrence of a search term in an articles title seems to have a strong impact on the articles ranking. The impact of search term frequencies in an articles full text is weak. That means it makes no difference in an articles ranking if the article contains the query terms only once or multiple times. It was further researched whether the name of an author or journal has an impact on the ranking and whether differences exist between the ranking algorithms of different search modes that Google Scholar offers. The answer in both of these cases was "yes". The results of our research may help authors to optimize their articles for Google Scholar and enable researchers to estimate the usefulness of Google Scholar with respect to their search intention and hence the need to use further academic search engines or databases.

Bornmann, L., Marx, W., Schier, H., Rahm, E., Thor, A., & Daniel, H. D. (2009). Convergent validity of bibliometric Google Scholar data in the field of chemistry—Citation counts for papers thatwere accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Sc. Journal of Informetrics, 3(1), 27–35. DOI: http://dx.doi.org/10.1016/j.joi.2008.11.001 .  Examining a comprehensive set of papers (n=1837) that were accepted for publication by the journal Angewandte Chemie International Edition (one of the prime chemistry journals in the world) or rejected by the journal but then published elsewhere, this study tested the extent to which the use of the freely available database Google Scholar (GS) can be expected to yield valid citation counts in the field of chemistry. Analyses of citations for the set of papers returned by three fee-based databases – Science Citation Index, Scopus, and Chemical Abstracts – were compared to the analysis of citations found using GS data. Whereas the analyses using citations returned by the three fee-based databases showvery similar results, the results of the analysis using GS citation data differed greatly from the findings using citations fromthe fee-based databases. Our study therefore supports, on the one hand, the convergent validity of citation analyses based on data from the fee-based databases and, on the other hand, the lack of convergent validity of the citation analysis based on the GS data.
Chiroque-Solano, R., Padilla-Santoyo, P. (2009). Análisis de coautoría en la revista Biblios: Una aproximación desde Google Scholar. Biblios, (34), 1-11. Analyzes the social networks among the authors who published in the Biblios journal, thereby presenting the measures of centrality, and closeness in the brokerage network. Presents the importance of using Google Scholar in open access journals to understand more of their use and impact in the scientific community. Finally it is suggested to continue with studies to identify the themes that are addressed through the analysis of descriptors.



Couto, F.M., Grego, T., Pesquita, C., Verissimo, P. (2009). Handling self-citations using Google Scholar. Cybermetrics, 13(1), paper 2. The increasing use of citation impact indexes for evaluation and comparison not only of individual researchers but also of institutions, universities and even countries has prompted the development of new citation metrics. Currently, the number of publications and citations is widely accepted as an easy and balanced way to compare scientists. Calculation of such statistics depends on the availability of a comprehensive database of publications and their citations. Google Scholar aims at providing such a service and is currently the most widely used freely available search engine for scientific and academic literature. However, the citations generally used to calculate citation statistics include self-citations, which deviates from the intention of using citations as a reflection of research impact. To the best of our knowledge, there are no available tools for calculating citation statistics that account for self-citations. We present a web-based service CIDS (Citation Impact Discerning Self-citations), that takes into account self-citations. An assessment of CIDS in a research team has shown that both the number of citations and the h-index is sensitive to self-citations at the individual level, the h-index increasing 24% on average when considering them. However, self-citation is highly variable among individuals and its contribution highly variable. We conclude that at the individual and research unit level, self-citations are not dismissible when calculating citation statistics. Even the h-index is influenced by self-citation and comparing individuals without taking them in account can produce misleading results.


Franceschet, M. (2009). A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar. Scientometrics, 83(1), 243–258. DOI: http://dx.doi.org/10.1007/s11192-009-0021-2 . Given the current availability of different bibliometric indicators and of production and citation data sources, the following two questions immediately arise: do the indicators' scores differ when computed on different data sources? More importantly, do the indicator-based rankings significantly change when computed on different data sources? We provide a case study for computer science scholars and journals evaluated on Web of Science and Google Scholar databases. The study concludes that Google scholar computes significantly higher indicators' scores than Web of Science. Nevertheless, citation-based rankings of both scholars and journals do not significantly change when compiled on the two data sources, while rankings based on the h index show a moderate degree of variation.

Freeman, M.K., Lauderdale, S.A., Kendrach, M.G., Woolley, T.W. (2009). Google Scholar versus PubMed in locating primary literature to answer drug-related questions. Annals of Pharmacotherapy, 43(3), 478-484. DOI: http://dx.doi.org/10.1345/aph.1L223 . Background: Google Scholar linked more visitors to biomedical journal Web sites than did PubMed after the database's initial release; however, its usefulness in locating primary literature articles is unknown. Objective: To assess in both databases the availability of primary literature target articles; total number of citations; availability of free, full-text journal articles; and number of primary literature target articles retrieved by year within the first 100 citations of the search results. Methods: Drug information question reviews published in The Annals of Pharmacotherapy Drug Information Rounds column served as targets to determine the retrieval ability of Google Scholar and PubMed searches. Reviews printed in this column from January 2006 to June 2007 were eligible for study inclusion. Articles were chosen if at least 2 key words of the printed article were included in the PubMed Medical Subject Heading (MeSH) database, and these terms were searched in both databases. Results: Twenty-two of 33 (67%) eligible Drug Information Rounds articles met the inclusion criteria. The median number of primary literature articles used in each of these articles was 6.5 (IQR 4.8, 8.3; mean SD 8 5.4). No significant differences were found for the mean number of target primary literature articles located within the first 100 citations in Google Scholar and PubMed searches (5.1 3.9 vs 5.3 3.3; p = 0.868). Google Scholar searches located more total results than PubMed (2211.6 3999.5 vs 44.2 47.4; p = 0.019). The availability of free, full-text journal articles per Drug Information Rounds article was similar between the databases (1.8 1.7 vs 2.3 1.7; p = 0.325). More primary literature articles published prior to 2000 were located with Google Scholar searches compared with PubMed (62.8% vs 34.9%; p = 0.017); however, no statistically significant differences between the databases were observed for articles published after 2000 (66.4 vs 77.1; p = 0.074). Conclusions: No significant differences were identified in the number of target primary literature articles located between databases. PubMed searches yielded fewer total citations than Google Scholar results; however, PubMed appears to be more specific than Google Scholar for locating relevant primary literature articles.

Harzing, A.W., Van Der Wal, R. (2009). A Google Scholar h‐index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41-46. DOI: http://dx.doi.org/10.1002/asi.20953 . We propose a new data source (Google Scholar) and metric (Hirsch's h-index) to assess journal impact in the field of economics and business. A systematic comparison between the Google Scholar h-index and the ISI Journal Impact Factor for a sample of 838 journals in economics and business shows that the former provides a more accurate and comprehensive measure of journal impact.
Howland, J.L., Howell, S., Wright, T.C., Dickson, C. (2009). Google Scholar and the Continuing Education Literature. Journal of Continuing Higher Education, 57(1), 35–39. DOI: http://dx.doi.org/10.1080/07377360902806890 The recent introduction of Google Scholar has renewed hope that someday a powerful research tool will bring continuing education literature more quickly, freely, and completely to one's computer. The authors suggest that using Google Scholar with other traditional search methods will narrow the research gap between what is discoverable and available. They present results of an investigation in which the names of two scholars were submitted to research queries using traditional library databases and Google Scholar. While not all of the scholars' academic publications were identified in the search, more were identified by Google Scholar than the other databases. However, other databases identified some that Google Scholar did not. It was evident from this informal analysis that utilizing Google Scholar with other traditional research methods adds value and discoverability in the search for relevant continuing education literature.

Howland, J.L., Wright, T.C., Boughan, R.A., Roberts, B.C. (2009). How scholarly is Google Scholar? A comparison to library databases. College & Research Libraries, 70(3), 227-234. Google Scholar was released as a beta product in November of 2004. Since then, Google Scholar has been scrutinized and questioned by many in academia and the library field. Our objectives in undertaking this study were to determine how scholarly Google Scholar is in comparison with traditional library resources and to determine if the scholarliness of materials found in Google Scholar varies across disciplines. We found that Google Scholar is, on average, 17.6 percent more scholarly than materials found only in library databases and that there is no statistically significant difference between the scholarliness of materials found in Google Scholar across disciplines.
Jacobs, J.A. (2009). Where Credit Is Due: Assessing the Visibility of Articles Published in Gender & Society with Google Scholar. Gender & Society, 23(6), 817–832. DOI: http://dx.doi.org/10.1177/0891243209351029 . Gender & Society is the leading specialty journal in the sociology of gender, as indicated by its high ranking in the ISI Web of Knowledge Journal Citation Reports. The ISI system, however, does not track citations appearing in books, and thus a significant potential source of references for Gender & Society is missed. This article reports the results of an analysis of highly cited articles that compares their visibility in Google Scholar to that documented in the ISI data system. Google Scholar captures more than twice as many references to these Gender & Society articles than does the ISI Web of Knowledge. The analysis shows that the incremental coverage is greater for Gender & Society than for several other prominent sociology journals. The absolute and relative standing of Gender & Society would improve if a more comprehensive system of tracking citations were employed.



Kulkarni, A.V., Aziz, B., Shams, I., Busse, J.W. (2009). Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. JAMA, 302(10), 1092-1096. DOI: http://dx.doi.org/10.1001/jama.2009.1307 . Context: Until recently, Web of Science was the only database available to track citation counts for published articles. Other databases are now available, but their relative performance has not been established. Objective: To compare the citation count profiles of articles published in general medical journals among the citation databases of Web of Science, Scopus, and Google Scholar. Design: Cohort study of 328 articles published in JAMA, Lancet, or the New England Journal of Medicine between October 1, 1999, and March 31, 2000. Total citation counts for each article up to June 2008 were retrieved from Web of Science, Scopus, and Google Scholar. Article characteristics were analyzed in linear regression models to determine interaction with the databases. Main Outcome Measures: Number of citations received by an article since publication and article characteristics associated with citation in databases. Results: Google Scholar and Scopus retrieved more citations per article with a median of 160 (interquartile range IQR, 83 to 324) and 149 (IQR, 78 to 289), respectively, than Web of Science (median, 122; IQR, 66 to 241) (P < .001 for both comparisons). Compared with Web of Science, Scopus retrieved more citations from non-English-language sources (median, 10.2% vs 4.1%) and reviews (30.8% vs 18.2%), and fewer citations from articles (57.2% vs 70.5%), editorials (2.1% vs 5.9%), and letters (0.8% vs 2.6%) (all P < .001). On a log(10)-transformed scale, fewer citations were found in Google Scholar to articles with declared industry funding (nonstandardized regression coefficient, -0.09; 95% confidence interval CI, -0.15 to -0.03), reporting a study of a drug or medical device (-0.05; 95% CI, -0.11 to 0.01), or with group authorship (-0.29; 95% CI, -0.35 to -0.23). In multivariable analysis, group authorship was the only characteristic that differed among the databases; Google Scholar had significantly fewer citations to group-authored articles (-0.30; 95% CI, -0.36 to -0.23) compared with Web of Science. Conclusion: Web of Science, Scopus, and Google Scholar produced quantitatively and qualitatively different citation counts for articles published in 3 general medical journals.

Law, R., Ye, Q., Chen, W., Leung, R. (2009). An analysis of the most influential articles published in tourism journals from 2000 to 2007: A Google Scholar approach. Journal of Travel & Tourism Marketing, 26(7), 735-746. DOI: http://dx.doi.org/10.1080/10548400903284628 . This research note reports a study that analyzed the 100 most influential articles, which is operationalized as the most cited publications published in tourism journals from 2000 to 2007. A Google Scholar-based software system was developed in Java to retrieve the citation information. The empirical findings show that 10.16% of the citations were from Institute for Scientific Information-listed (ISI) journals, and that 71.64% of them were from neither ISI nor tourism journals. The most popular topics covered by these articles were psychology and tourist behavior, followed by destination image and marketing. This article contributes to the literature by providing an alternative means of assessing the impact of research into tourism
Ma, R., Dai, Q., Ni, C., Li, X. (2009). An author co-citation analysis of information science in China with Chinese Google Scholar search engine, 2004–2006. Scientometrics, 81(1), 33–46. DOI: http://dx.doi.org/10.1007/s11192-009-2063-x . Author co-citation analysis (ACA) is an important method for discovering the intellectual structure of a given scientific field. Since traditional ACA was confined to ISI Web of Knowledge (WoK), the co-citation counts of pairs of authors mainly depended on the data indexed in WoK. Fortunately, Google Scholar has integrated different academic databases from different publishers, providing an opportunity of conducting ACA in wider a range. In this paper, we conduct ACA of information science in China with the Chinese Google Scholar. Firstly, a brief introduction of Chinese Google Scholar is made, including retrieval principles and data formats. Secondly, the methods used in our paper are given. Thirdly, 31 most important authors of information science in China are selected as research objects. In the part of empirical study, factor analysis is used to find the main research directions of information science in China. Pajek, a powerful tool in social network analysis, is employed to visualize the author co-citation matrix as well. Finally, the resemblances and the differences between China and other countries in information science are pointed out.
Martell, C. (2009). A Citation Analysis of College & Research Libraries Comparing Yahoo, Google, Google Scholar, and ISI Web of Knowledge with Implications for Promotion and Tenure. College Research Libraries, 70(5), 460–472. Two hundred and seventeen articles in College & Research Libraries from 2000 to 2006 were searched by title on Yahoo, Google, Google Scholar, and ISI Web of Knowledge to determine the frequency with which articles in the journal are cited, the effectiveness of the four search services, and the relevance and applicability of findings to promotion and tenure. Yahoo, Google, and ISI Web of Knowledge averaged between 2.8 and 3.5 citations per title for the period covered and Google Scholar averaged 6.4. The value of citations counts in the promotion and tenure process and the importance of publications in the evaluation of librarians are discussed.

Mikki, S. (2009). Comparing Google Scholar and ISI Web of Science for Earth Sciences. Scientometrics, 82(2), 321–331. DOI: http://dx.doi.org/10.1007/s11192-009-0038-6 .  In order to measure the degree to which Google Scholar can compete with bibliographical databases, search results from this database is compared with Thomson's ISI WoS (Institute for Scientific Information, Web of Science). For earth science literature 85% of documents indexed by ISI WoS were recalled by Google Scholar. The rank of records displayed in Google Scholar and ISI WoS, is compared by means of Spearman's footrule. For impact measures the h-index is investigated. Similarities in measures were significant for the two sources.


Moskovkin, V.M. (2009). The potential of using the Google Scholar search engine for estimating the publication activities of universities. Scientific and Technical Information Processing, 36(4), 198–202. DOI: http://dx.doi.org/10.3103/S0147688209040029. This paper studies the potential of using the Google Scholar search engine for estimating the publication activities of universities and considers a procedure for such estimation with the help of queries for the English names of universities. The publication structures for 2008 have been built for ten selected universities of the world, including MSU. The publication activities of the universities under consideration in 2007, has been compared based on the citation database of the US Institute for Scientific Information (Web of Knowledge) and Google Scholar search engine (GS-publications).

Mukherjee, B. (2009), Do open-access journals in library and information science have any scholarly impact? A bibliometric study of selected open-access journals using Google Scholar. Journal of the American Society for Information Science and Technology, 60(3), 581–594. DOI: http://dx.doi.org/10.1002/asi.21003 . Using 17 fully open-access journals published uninterruptedly during 2000 to 2004 in the field of library and information science, the present study investigates the impact of these open-access journals in terms of quantity of articles published, subject distribution of the articles, synchronous and diachronous impact factor, immediacy index, and journals' and authors' self-citation. The results indicate that during this 5-year publication period, there are as many as 1,636 articles published by these journals. At the same time, the articles have received a total of 8,591 Web citations during a 7-year citation period. Eight of 17 journals have received more than 100 citations. First Monday received the highest number of citations; however, the average number of citations per article was the highest in D-Lib Magazine. The value of the synchronous impact factor varies from 0.6989 to 1.0014 during 2002 to 2005, and the diachronous impact factor varies from 1.472 to 2.487 during 2000 to 2004. The range of the immediacy index varies between 0.0714 and 1.395. D-Lib Magazine has an immediacy index value above 0.5 in all the years whereas the immediacy index value varies from year to year for the other journals. When the citations of sample articles were analyzed according to source, it was found that 40.32% of the citations came from full-text articles, followed by 33.35% from journal articles. The percentage of journals' self-citation was only 6.04%.


Onyancha, O.B., Ocholla, D.N. (2009). Assessing researchers’ performance in developing countries: is Google scholar an alternative?. Mousaion, 27(1), 43–64. [http://hdl.handle.net/10500/5269]. This article compares the representation of 10 purposefully selected LIS researchers in South Africa in Google Scholar (GS), Thomson Scientific’s (herein referred to as ISI – Institute for Scientific Information) citation indexes, and Elsevier’s Scopus, in order to determine whether or not Google Scholar is an alternative tool for evaluating research in developing countries, particularly those situated in Sub-Saharan Africa. Three indicators, namely the number of publications, the number of citations and the h-index, were used to measure the similarity or dissimilarity between the three databases/ services in the coverage of South Africa’s LIS documents. The data was also subjected to a Pearson correlation analysis to examine the relationship between GS and ISI, GS and Scopus and ISI and Scopus. Results show that GS covers more publications and citations than ISI and Scopus. There is a stronger correlation between GS and Scopus than there is between GS and ISI. We conclude that GS is an alternative service, but should be cautiously used when evaluating research in developing countries. Areas for further research are also recommended.

Onyancha, O.B. (2009). A Citation Analysis of Sub-Saharan African Library and Information Science Journals using Google Scholar. African Journal of Library Archives Information Science, 19(2), 101–116. In bibliometrics, the numbers of research articles and citations constitute the main measurement indicators of research output and impact respectively. This study evaluates the library and information science/studies (LIS) journals published in sub-Saharan African countries in order to assess their performance. Drawing its data from Google Scholar, the paper compares the performance of 13 LIS journals using the following indicators: number of publications; average number of records; number of citations; citations per year; citations per article; citedness and uncitedness of the records published in each journal; h-index and g-index; and citation impact factor. The paper also identifies journals with the most cited works and ranks the journals according to the above measurement indicators. Results indicate that publication of LIS journals in Sub-Saharan Africa is a relatively recent practice; a number of journals have not published any issues for close to 5 years; some journals have ceased publication; there is irregular publication of journals; there are about five core LIS journals in the region; AJLAIS was the most highly cited journal, but the most influential journals in terms of the IF include SAJLIS, Innovation and Mousaion. The challenges faced by journal publishers and researchers in sub-Saharan Africa, as well as recommendations on improving the visibility and impact of journals in the region and internationally, are outlined.
Orduña-Malea, E., Serrano-Cobos, J., Lloret-Romero, N. (2009). Spanish public universities in Google Scholar: presence, evolution and coverage of their scientific output. El Profesional de la Informacion, 18(5), 493–500. DOI: http://dx.doi.org/10.3145/epi.2009.sep.02 . The validity of Google Scholar as a service that reflects properly the scientific output of a university is analyzed, comparing its coverage of Spanish public universities with that of Scopus. The presence and evolution of scientific documentation on the websites of the Spanish public universities from January to July (both months inclusive) 2009 is also studied. The results show that, despite finding some interrelationship between Scholar and Scopus concerning the productivity of institutions, there are large differences in the total results that override the latter as a valid reflection of university productivity. Finally, Scholar shows positive, albeit modest, growth during the studied period for most university websites analyzed.

Rosenstreich, D., Wooliscroft, B. (2009). Measuring the impact of accounting journals using Google Scholar and the g-index. The British Accounting Review, 41(4), 227–239. DOI: http://dx.doi.org/10.1016/j.bar.2009.10.002 . The UK's proposed Research Excellence Framework promotes a move towards citation analysis for assessing research performance. However, for business disciplines, journal rankings are likely to remain an important aid in evaluating research quality. The accounting literature includes many journal rankings and citation studies, however there has been little coverage of recent advances in these areas. This study explores approaches to assessing the impact of accounting journals with a focus on quantitative measures as a complement to peer-review-based evaluation. New data sources and techniques for citation studies are reviewed, and the g-index is selected for further analysis. The g-index was developed by Professor Leo Egghe in 2006 as an improvement on the h-index. Like the h-index, the g-index represents a relationship between papers published and the level of citations they receive, but the g-index is more sensitive to highly cited paper. To apply the g-index to accounting journals, the study first combines eight published journals rankings to produce a list of 34 highly-regarded titles. Citation data are then gathered from Google Scholar and used to calculate g-index scores as the basis of a new ranking. Google Scholar is found to have broader coverage of accounting citations than Scopus or the Web of Science databases, but requires cleaning to remove duplicate entries. The use of the g-index for ranking journals is found to be a useful innovation in citation analysis, allowing a more robust assessment of the impact of journals.



Ślesemzyński, P. (2009). The position of Polish geographical journals and series as seen in the Google Scholar databases. Przeglad Geograficzny, 81(4), 551–578.


Walters, W.H. (2009). Google Scholar Search Performance: Comparative Recall and Precision. portal: Libraries and the Academy, 9(1), 5–24. DOI: http://dx.doi.org/10.1353/pla.0.0034 This paper presents a comparative evaluation of Google Scholar and 11 other bibliographic databases (Academic Search Elite, AgeLine, ArticleFirst, EconLit, GEOBASE, MEDLINE, PAIS International, POPLINE, Social Sciences Abstracts, Social Sciences Citation Index, and SocINDEX), focusing on search performance within the multidisciplinary field of later-life migration. The results of simple keyword searches are evaluated with reference to a set of 155 relevant articles identified in advance. In terms of both recall and precision, Google Scholar performs better than most of the subscription databases. This finding, based on a rigorous evaluation procedure, is contrary to the impressions of many early reviewers. The paper concludes with a discussion of a new approach to document relevance in educational settings-an approach that accounts for the instructors' goals as well as the students' assessments of relevance.



Yang, Y., Yanning, Z. (2009). The Effect of Open Access Journals on Citation Impact: A Citation Analysis of Open Access Journals Using Google Scholar. In Cooperation and Promotion of Information Resources in Science and Technology, 2009. COINFO'09. Fourth International Conference on , 278-280. DOI: http://dx.doi.org/10.1109/COINFO.2009.37 . Authors use four OA journals in the field of information and library science during 2001 to 2005. Our research concentrates on the citation analysis of these OA journals' articles. The four journals contained 455 articles and were cited 4338 times. The average rate of citations per OA articles was 9.49. We found that the citation can be seen as a trend of decrease during 2001 to 2004. We also found that most of the articles (32.03%) have received citations under the stratum of "1-5". 29.85% of these articles have not received any citation. During the research on citation life of articles, we found the speed of getting citations is not accelerating with the electronic technologies which accelerate the publication speed.

2008 [Go back]

Cooke, R. Donlan, R. (2008). Thinking Inside the Box: Comparing Federated Search Results from Google Scholar, Live Search Academic, and Central Search. Journal of Library Administration, 46(3), 31–42. DOI: http://dx.doi.org/10.1300/J111v46n03_03 In a comparison of Google Scholar, Windows Live Search Academic, and Serials Solutions' Central Search, relevant retrieval increases in direct relation to the complexity of the search interface. Central Search, as customized for Florida Gulf Coast University Library, permits far more complex searching than Google Scholar or Windows Academic and performs better than its simpler competitors. However, a close review of sample searches in these three databases reveals that the simpler, more streamlined interfaces may be equally useful, depending on the user's preference, and the information need.


Ford, L., O’Hara, L.H. (2008). It’s all academic: Google Scholar, Scirus, and Windows Live Academic search. Journal of Library Administration, 46(3), 43–52. DOI: http://dx.doi.org/10.1300/J111v46n03_04 . In April 2006 and again in November 2006, Google Scholar, Scirus, and Windows Live Academic Search were searched for a set of 39 citations found in databases covering science, computer science, electrical engineering, and physics, along with the British Library OPAC and WorldCAT. The functionality and coverage of each engine are compared. Changes in results and functionality between April and November 2006 are also noted. Google Scholar retrieved the most citations, followed by Scirus, and then by Microsoft's engine. Libraries should link to these engines and enable OpenURL and other link resolving systems to ease retrieval for their users.

Hartman, K.A., Mullen, L.B. (2008). Google Scholar and academic libraries: an update. New Library World, 109(5/6), 211–222. DOI: http://dx.doi.org/10.1108/03074800810873560 . Purpose - This paper aims to update the authors' original 2005 study of Google Scholar's integration into ARL libraries' web sites. Have more ARL libraries added Google Scholar? Design/methodology/approach - The library homepages of the 113 ARL academic institutions were examined for paths or links to Google Scholar. The coding scheme focused on noting whether Google Scholar appeared on the library homepage, in the OPAC, and on various database lists and subject guides. Findings - The 2007 data indicate continued acceptance of Google Scholar and integration of this resource on the web pages of ARL libraries. The mean number of paths to Google Scholar more than doubled from 2005 to 2007. Partnering institutions were more likely to include paths to Google Scholar and the number of partnering institutions increased dramatically. Practical implications - This study is useful for those making decisions about integration of Google Scholar into library collections and services, particularly the web site. Originality/value - This paper illustrates future directions for integrating new categories of resources into the academic library web site.
Harzing, A.W., van der Wal, R. (2008). Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8(1), 61–73. DOI: http://dx.doi.org/10.3354/esep00076 . Traditionally, the most commonly used source of bibliometric data is Thomson ISI Web of Knowledge, in particular the Web of Science and the Journal Citation Reports (JCR), which provide the yearly Journal Impact Factors (JIF). This paper presents an alternative source of data (Google Scholar, GS) as well as 3 alternatives to the JIF to assess journal impact (h-index, g-index and the number of citations per paper). Because of its broader range of data sources, the use of GS generally results in more comprehensive citation coverage in the area of management and international business. The use of GS particularly benefits academics publishing in sources that are not (well) covered in ISI. Among these are books, conference papers, non-US journals, and in general journals in the field of strategy and international business. The 3 alternative GS-based metrics showed strong correlations with the traditional JIF. As such, they provide academics and universities committed to JIFs with a good alternative for journals that are not ISI-indexed. However, we argue that these metrics provide additional advantages over the JIF and that the free availability of GS allows for a democratization of citation analysis as it provides every academic access to citation data regardless of their institution's financial means.
Jacso, P. (2008). The pros and cons of computing the h-index using Google Scholar. Online Information Review, 32 (3), 437-452. DOI: http://dx.doi.org/10.1108/14684520810889718 Purpose – A previous paper by the present author described the pros and cons of using the three largest cited reference enhanced multidisciplinary databases and discussed and illustrated in general how the theoretically sound idea of the h-index may become distorted depending on the software and the content of the database(s) used, and the searchers' skill and knowledge of the database features. The aim of this paper is to focus on Google Scholar (GS), from the perspective of calculating the h-index for individuals and journals. Design/methodology/approach – A desk-based approach to data collection is used and critical commentary is added. Findings – The paper shows that effective corroboration of the h-index and its two component indicators can be done only on persons and journals with which a researcher is intimately familiar. Corroborative tests must be done in every database for important research. Originality/value – The paper highlights the very time-consuming process of corroborating data, tracing and counting valid citations and points out GS's unscholarly and irresponsible handling of data.

Jacso, P. (2008). Testing the Calculation of a Realistic h-index in Google Scholar, Scopus, and Web of Science for F. W. Lancaster. Library Trends, 56(4), 784–815. DOI: http://dx.doi.org/10.1353/lib.0.0011 This paper focuses on the practical limitations in the content and software of the databases that are used to calculate the h-index for assessing the publishing productivity and impact of researchers. To celebrate F. W. Lancaster’s biological age of seventy-five, and “scientific age” of forty-five, this paper discusses the related features of Google Scholar, Scopus, and Web of Science (WoS), and demonstrates in the latter how a much more realistic and fair h-index can be computed for F. W. Lancaster than the one produced automatically. Browsing and searching the cited reference index of the 1945–2007 edition of WoS, which in my estimate has over a hundred million “orphan references” that have no counterpart master records to be attached to, and “stray references” that cite papers which do have master records but cannot be identified by the matching algorithm because of errors of omission and commission in the references of the citing works, can bring up hundreds of additional cited references given to works of an accomplished author but are ignored in the automatic process of calculating the h-index. The partially manual process doubled the h-index value for F. W. Lancaster from 13 to 26, which is a much more realistic value for an information scientist and professor of his stature.



Jorge, R. (2008). Acimed in Scholar Google a citation analysis of the Cuban Journal of Health Information and Communication Professionals. Acimed, 18(1), 1–14.  A citation analysis of Acimed, the Cuban Journal of Information and Communication Professionals in Health Sciences, during its first 15 years of existence was made in Scholar Google. To this end, the Publish or Perish software was used. The impact of Acimed was compared with that of other nine journals on Library and Information Science published in Spanish language. The most cited articles of the journal during the whole period were identified, as well as the topics with the highest visibility. The importance of Scholar Google as a tool for assessing research was analyzed, and the necessity to carry out comparative studies of Scholar Google with the Web of Science and Scopus in order to determine its advantages and limitations was exposed.

Kesen, S., Şenol, C., Yanar, Z. (2008). An Evaluation of Google Scholar and Scirus Search Engines Using Turkish Search Queries. Information World Bilgi Dunyasi, 9(1), 140–157.  Due to the rapid growth of information on the Internet, search engines are used to getting information included in documents created in different languages. This paper aims to find out if the two search engines providing access to Open Access information sources, Google Scholar and Scirus, display the search results appropriately for special Turkish characters (ç, ğ, l, ö, ş, ü) and to evaluate them on the basis of the total number of retrieved documents. Both search engines displayed the Turkish special characters correctly, but the search results differed. These differences create information retrieval problems for Turkish queries. Metadata of the Open Access Turkish information sources should be created using special Turkish characters and search engines should be developed to support them.

Kousha, K., Thelwall, M. (2008). Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines. Scientometrics, 74(2), 273–294. DOI: http://dx.doi.org/10.1007/s11192-008-0217-x For practical reasons, bibliographic databases can only contain a subset of the scientific literature. The ISI citation databases are designed to cover the highest impact scientific research journals as well as a few other sources chosen by the Institute for Scientific Information (ISI). Google Scholar also contains citation information, but includes a less quality controlled collection of publications from different types of web documents. We define Google Scholar unique citations as those retrieved by Google Scholar which are not in the ISI database. We took a sample of 882 articles from 39 open access ISI-indexed journals in 2001 from biology, chemistry, physics and computing and classified the type, language, publication year and accessibility of the Google Scholar unique citing sources. The majority of Google Scholar unique citations (70%) were from full-text sources and there were large disciplinary differences between types of citing documents, suggesting that a wide range of non-ISI citing sources, especially from non-journal documents, are accessible by Google Scholar. This might be considered to be an advantage of Google Scholar, since it could be useful for citation tracking in a wider range of open access scholarly documents and to give a broader type of citation impact. An important corollary from our study is that Google Scholar’s wider coverage of Open Access (OA) web documents is likely to give a boost to the impact of OA research and the OA movement.


Law, R., Van Der Veen, R. (2008). The popularity of prestigious hospitality journals: A Google Scholar approach. International Journal of Contemporary Hospitality Management, 20(2), 113–125.DOI: http://dx.doi.org/10.1108/09596110810852113 . Purpose - The purpose of this paper is to show that prior studies on rating hospitality journals primarily used two major assessment categories. The first category gauges experts' perceptions of a journal's quality in terms of prestige. The second category counts objective measures that reflect journal quality in terms of popularity. The purpose of this study is to introduce a new counting method that uses Google Scholar (GS) to evaluate the citation counts of the leading hospitality journals. Design/methodology/approach - The study began with the top hospitality journals in a recent study that rated journals based on perceived quality by hospitality experts. Next the paper examined the popularity, defined as the citations in Google Scholar (GS), of these leading hospitality journals. The collection of 1960 to 2006 GS citation counts was conducted from February to August 2007. Findings - The ranking of GS citation counts for the selected journals generally followed the perceived ratings. The International Journal of Contemporary Hospitality Management and Cornell Hotel and Restaurant Administration Quarterly performed the best in average citations per year. The International Journal of Contemporary Hospitality Management and International Journal of Hospitality Management received the largest number of average citations per published article. Research limitations/implications - The major limitations of this study are the mere eight hospitality journals included in the study and GS's proprietary indexing algorithm. Another limitation is the dynamic aspect of GS, which generates unequal findings over time. Still, the research findings help hospitality researchers, educators, practitioners, and students to understand the popularity of the most prestigious hospitality journals. Originality/value - This paper is a novel attempt that uses GS for ranking popularity of the most prestigious hospitality journals.



Levine-Clark, M., Gil, E.L. (2008). A Comparative Citation Analysis of Web of Science, Scopus, and Google Scholar. Journal of Business & Finance Librarianship, 14(1), 32–46. DOI: http://dx.doi.org/10.1080/08963560802176348 This article presents the results of a comparative study of Web of Science (WoS), Scopus, and Google Scholar (GS) for a set of 15 business and economics journals. Citations from the three sources were analyzed to determine whether one source is better than another, or whether a new database such as Scopus, or a free database such as GS could replace WoS. The authors concluded that scholars might want to use alternative tools collectively to get a more complete picture of the scholarly impact of an article.

Mayr, P., Walter, A.K. (2008). Studying Journal Coverage in Google Scholar. Journal of Library Administration, 47(1-2), 81–99. DOI: http://dx.doi.org/10.1080/01930820802110894 . The paper discusses and analyzes the coverage of scientific serials in Google Scholar (GS). The focus is on an exploratory study. The study shows deficiencies in the coverage and up-to-dateness of the GS index. Furthermore, the study points up which Web servers are the most important data providers for this search service and which information sources are highly represented. There is a relatively large gap in Google Scholar's coverage of German literature as well as weaknesses in the accessibility of Open Access content.

Meier, J.J., Conkling, T.W. (2008). Google Scholar’s Coverage of the Engineering Literature: An Empirical Study. The Journal of Academic Librarianship, 34(3), 196–201. DOI: http://dx.doi.org/10.1016/j.acalib.2008.03.002 . Google Scholar's coverage of the engineering literature is analyzed by comparing its contents with those of Compendex, the premier engineering database. Records retrieved from Compendex were searched in Google Scholar, and a decade by decade comparison was done from the 1950s through 2007. The results show that the percentage of records appearing in Google Scholar increased over time, approaching a 90 percent matching rate for materials published after 1990.
Neuhaus, C., Neuhaus, E., Asher, A. (2008). Google Scholar Goes to School: The Presence of Google Scholar on College and University Web Sites. The Journal of Academic Librarianship, 34(1), 39–51. DOI: http://dx.doi.org/10.1016/j.acalib.2007.11.009 This study measured the degree of Google Scholar adoption within academia by analyzing the frequency of Google Scholar appearances on 948 campus and library Web sites, and by ascertaining the establishment of link resolution between Google Scholar and library resources. Results indicate a positive correlation between the implementation of Google Scholar link resolution and the degree of Google Scholar adoption.
Noll, Hannah M.(2008). Where Google Stands on Art: An Evaluation of Content Coverage in Online Databases. A Master’s Paper for the M.S. in L.S degree. April, 2008. 43 pages. 
This study evaluates the content coverage of Google Scholar and three commercial databases (Arts & Humanities Citation Index, Bibliography of the History of Art and Art Full Text/Art Index Retrospective) on the subject of art history. Each database is tested using a bibliography method and evaluated based on Péter Jacsó's scope criteria for online databases. Of the 472 articles tested, Google Scholar indexed the smallest number of citations (35%), outshone by the Arts & Humanities Citation Index which covered 73% of the test set. This content evaluation also examines specific aspects of coverage to find that in comparison to the other databases, Google Scholar provides consistent coverage over the time range tested (1975-2008) and considerable access to article abstracts (56%). Google Scholar failed, however, to fully index the most frequently cited art periodical in the test set, the Artforum International. Finally, Google Scholar's total citation count is inflated by a significant percentage (23%) of articles which include duplicate, triplicate or multiple versions of the same record.
Norris, M., Oppenheim, C., Rowland, F. (2008). Finding open access articles using Google, Google Scholar, OAIster and OpenDOAR. Online Information Review, 32(6), 709–715. DOI: http://dx.doi.org/10.1108/14684520810923881 
Purpose - The purpose of this paper is to demonstrate the relative effectiveness of a range of search tools in finding open access (OA) versions of peer reviewed academic articles on the world wide web. 
Design/methodology/ approach - Some background is given on why and how academics may make their articles OA and how they may be found by others searching for them. Google, Google Scholar, OAIster and OpenDOAR were used to try to locate OA versions of peer reviewed journal articles drawn from three subjects (ecology, economics and sociology). 
Findings - Of the 2,519 articles, 967 were found to have OA versions on the world wide web. Google and Google Scholar found 76.84 per cent of them. The results from OpenDOAR and OAIster were disappointing, but some improvements are noted. Only in economics could OAIster and OpenDOAR be considered relative successes. 
Originality/value - The paper shows the relative effectiveness of the search tools in these three subjects. The results indicate that those wanting to find OA articles in these subjects, for the moment at least should use the general search engines Google and Google Scholar first rather than OpenDOAR or OAIster.
Sanderson, M. (2008). Revisiting h measured on UK LIS and IR academics. Journal of the American Society for Information Science and Technology, 59(7), 1184-1190. DOI: http://dx.doi.org/10.1002/asi.20771A brief communication appearing in this journal ranked UK-based LIS and (some) IR academics by their h-index using data derived from the Thomson ISI Web of Science™ (WoS). In this brief communication, the same academics were re-ranked, using other popular citation databases. It was found that for academics who publish more in computer science forums, their h was significantly different due to highly cited papers missed by WoS; consequently, their rank changed substantially. The study was widened to a broader set of UK-based LIS and IR academics in which results showed similar statistically significant differences. A variant of h, hmx, was introduced that allowed a ranking of the academics using all citation databases together.
Vanclay, J.K. (2008). Ranking forestry journals using the h-index. Journal of informetrics, 2(4), 326-334.
An expert ranking of forestry journals was compared with Journal Impact Factors and h-indices computed from the ISI Web of Science and internet-based data. Citations reported by Google Scholar offer an efficient way to rank all journals objectively, in a manner consistent with other indicators. This h-index exhibited a high correlation with the Journal Impact Factor (r = 0.92), but is not confined to journals selected by any particular commercial provider. A ranking of 180 forestry journals is presented, on the basis of this index.
Vaughan, L., Shaw, D. (2008). A new look at evidence of scholarly citation in citation indexes and from web sources. Scientometrics, 74(2), 317-330.DOI: http://dx.doi.org/10.1007/s11192-008-0220-2 
A sample of 1,483 publications, representative of the scholarly production of LIS faculty, was searched in Web of Science (WoS), Google, and Google Scholar. The median number of citations found through WoS was zero for all types of publications except book chapters; the median for Google Scholar ranged from 1 for print/subscription journal articles to 3 for books and book chapters. For Google the median number of citations ranged from 9 for conference papers to 41 for books. A sample of the web citations was examined and classified as representing intellectual or non-intellectual impact. Almost 92% of the citations identified through Google Scholar represented intellectual impact — primarily citations from journal articles. Bibliographic services (non-intellectual impact) were the largest single contributor of citations identified through Google. Open access journal articles attracted more web citations but the citations to print/subscription journal articles more often represented intellectual impact. In spite of problems with Google Scholar, it has the potential to provide useful data for research evaluation, especially in a field where rapid and fine-grained analysis is desirable.


2007 [Go back]

Bar-Ilan, J. (2007). Which h-index? — A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257–271. DOI: http://dx.doi.org/10.1007/s11192-008-0216-y This paper compares the h-indices of a list of highly-cited Israeli researchers based on citations counts retrieved from the Web of Science, Scopus and Google Scholar respectively. In several case the results obtained through Google Scholar are considerably different from the results based on the Web of Science and Scopus. Data cleansing is discussed extensively.
Christianson, M. (2007). Ecology Articles in Google Scholar : Levels of Access to Articles in Core Journals. Issues in Science and Technology Librarianship, (50), 1–17. DOI: http://dx.doi.org/10.5062/F4MS3QPD Eight-hundred forty articles from core ecology journals were searched in Google Scholar (GS) to determine level and completeness of indexing and access. Testing occurred both on campus and off, and within each venue searching was divided evenly into basic and advanced modes. Off campus, about nine percent and on campus, about thirty-eight percent of links led to text that could be opened directly, without barriers. Fifty-seven percent of test articles had full citations or better, and over seventy-seven percent had at least some type of completable citation. Older articles were less likely to be represented. Full-text articles were concentrated at author sites and at a small number of provider sites. The advanced search found somewhat more full text than did the basic search. Highly cited articles were more likely to be included in Google Scholar.

Haase, A., Follmann, M., Skipka, G., Kirchner, H. (2007). Developing search strategies for clinical practice guidelines in SUMSearch and Google Scholar and assessing their retrieval performance. BMC medical research methodology, 7(1), 28. DOI: http://dx.doi.org/10.1186/1471-2288-7-28 .  Background Information overload, increasing time constraints, and inappropriate search strategies complicate the detection of clinical practice guidelines (CPGs). The aim of this study was to provide clinicians with recommendations for search strategies to efficiently identify relevant CPGs in SUMSearch and Google Scholar. Methods We compared the retrieval efficiency (retrieval performance) of search strategies to identify CPGs in SUMSearch and Google Scholar. For this purpose, a two-term GLAD (GuideLine And Disease) strategy was developed, combining a defined CPG term with a specific disease term (MeSH term). We used three different CPG terms and nine MeSH terms for nine selected diseases to identify the most efficient GLAD strategy for each search engine. The retrievals for the nine diseases were pooled. To compare GLAD strategies, we used a manual review of all retrievals as a reference standard. The CPGs detected had to fulfil predefined criteria, e.g., the inclusion of therapeutic recommendations. Retrieval performance was evaluated by calculating so-called diagnostic parameters (sensitivity, specificity, and "Number Needed to Read" [NNR]) for search strategies. Results The search yielded a total of 2830 retrievals; 987 (34.9%) in Google Scholar and 1843 (65.1%) in SUMSearch. Altogether, we found 119 unique and relevant guidelines for nine diseases (reference standard). Overall, the GLAD strategies showed a better retrieval performance in SUMSearch than in Google Scholar. The performance pattern between search engines was similar: search strategies including the term "guideline" yielded the highest sensitivity (SUMSearch: 81.5%; Google Scholar: 31.9%), and search strategies including the term "practice guideline" yielded the highest specificity (SUMSearch: 89.5%; Google Scholar: 95.7%), and the lowest NNR (SUMSearch: 7.0; Google Scholar: 9.3). Conclusion SUMSearch is a useful tool to swiftly gain an overview of available CPGs. Its retrieval performance is superior to that of Google Scholar, where a search is more time consuming, as substantially more retrievals have to be reviewed to detect one relevant CPG. In both search engines, the CPG term "guideline" should be used to obtain a comprehensive overview of CPGs, and the term "practice guideline" should be used if a less time consuming approach for the detection of CPGs is desired.


Haya, G., Nygren, E., Widmark, W. (2007). Metalib and Google Scholar: a user study. Online Information Review, 31(3), 365–375. DOI: http://dx.doi.org/10.1108/14684520710764122 . Purpose - This paper aims to understand how students experience the search tools Google Scholar and Metalib and the role of prior instruction. Design/methodology/approach - A total of 32 undergraduate students searched academic articles for their thesis work. Searches were recorded using Morae software and were analysed along with the number of articles saved and responses to a questionnaire. All searched with both tools. Half of the students received training before searching. Findings - Google Scholar performed better in almost all measures. Training had a positive effect on the amount and quality of articles saved. Responses to Google Scholar were more positive than to Metalib. However, the students were not overwhelmingly enthusiastic about either of the tools. Research limitations/implications - Each Metalib implementation is to some extent unique, which limits the extent to which results can be generalised to other implementations. Practical implications - Training is valuable for both tools. The user interface to Metalib does not conform with students' expectations and needs further improvement. Both tools strive to be a first alternative search tool for academic literature but neither performed well enough in this study to recommend it to be used in that role in an academic library setting. Originality/value - These tools are important to academic libraries but few user studies have been published, particularly on Google Scholar. To one's knowledge no other user study on these tools has looked at the effects of instruction. © Emerald Group Publishing Limited.
Helms-Park, R., Radia, P., Stapleton, P. (2007). A preliminary assessment of Google Scholar as a source of EAP students' research materials. The Internet and higher education, 10(1), 65-76. DOI: http://dx.doi.org/10.1016/j.iheduc.2006.10.002 . While the use of a search engine to find secondary sources is now a commonplace practice among undergraduate writers, recent studies show that students' online searches often lead to materials that are wholly or partially unsuitable for academic purposes. Accordingly, this project set out to determine whether using a more specialized search engine, Google Scholar, would lead to qualitative differences in the sources selected by second-language (L2) students working on a research-based assignment in a first-year English for Academic Purposes (EAP) course. The participants in this study (N = 27) were required to submit an annotated bibliography consisting of ten sources, sought from print or electronic media, on their research topic. Students were required to indicate how these sources were located (e.g., Google, Google Scholar, the university library's catalogue of electronic resources, or a traditional search for print materials). Three independent raters, who were not given any information on the search mechanisms used, evaluated each electronic source (N = 72) using WATCH, an analytic website assessment scale, [Stapleton, P., & Helms-Park, R. (2006). Evaluating Web sources in an EAP course: Introducing a multi-trait instrument for feedback and assessment. English for specific Purposes, 25(4) 438–455.]. Mann–Whitney comparisons revealed no significant differences between sources obtained through Google Scholar and the university library's catalogue of electronic resources (p set at ≤ 0.05). On the other hand, there were significant differences between Google Scholar and Google sources, as well as between electronic sources obtained through the library and Google, in key areas such as academic rigor and objectivity.


Levine-Clark, M., Kraus, J. (2007). Finding Chemistry Information Using Google Scholar : A Comparison with Chemical Abstracts Service. Science Technology Libraries, 27(4), 3–17. DOI: http://dx.doi.org/10.1300/J122v27n04_02 . Since its introduction in November 2004, Google Scholar has been the subject of considerable discussion among librarians. Though there has been much concern about the lack of transparency of the product, there has been relatively little direct comparison between Google Scholar and traditional library resources. This study compares Google Scholar and Chemical Abstracts Service (CAS) as resources for finding chemistry information. Of the 702 records found in six different searches, 65.1% were in Google Scholar and 45.1% were in CAS. Of these, 55.0% were unique to Google Scholar, 34.9% were unique to CAS, and 10.1% overlapped. When each record found was searched by title in the two databases, the figures change, with 79.5% in Google Scholar, 85.6% in CAS, and 65.1% overlapping. Based on this, researchers are more likely to find known published information through CAS than in Google Scholar. Results vary by type of search, type of resource, and date. For many types of searching, CAS performs significantly better than Google Scholar. This is especially true for searches on compounds or a personal name, both of which take advantage of advanced search features in CAS. For simple keyword searches, Google Scholar tends to perform better, most probably because Google Scholar searches through the full text of journal articles, while a keyword search through CAS only finds abstract and index terms.
Meho, L.I., Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125. DOI: http://dx.doi.org/10.1002/asi.20677 . The Institute for Scientific Information's (ISI, now Thomson Scientific, Philadelphia, PA) citation databases have been used for decades as a starting point and often as the only tools for locating citations and/or conducting citation analyses. The ISI databases (or Web of Science [WoS]), however, may no longer be sufficient because new databases and tools that allow citation searching are now available. Using citations to the work of 25 library and information science (LIS) faculty members as a case study, the authors examine the effects of using Scopus and Google Scholar (GS) on the citation counts and rankings of scholars as measured by WoS. Overall, more than 10,000 citing and purportedly citing documents were examined. Results show that Scopus significantly alters the relative ranking of those scholars that appear in the middle of the rankings and that GS stands out in its coverage of conference proceedings as well as international, non-English language journals. The use of Scopus and GS, in addition to WoS, helps reveal a more accurate and comprehensive picture of the scholarly impact of authors. The WoS data took about 100 hours of collecting and processing time, Scopus consumed 200 hours, and GS a grueling 3,000 hours.

Robinson, M.L., Wusteman, J. (2007). Putting Google Scholar to the test: a preliminary study. Program: electronic library and information systems, 41(1), 71–80. DOI: http://dx.doi.org/0.1108/00330330710724908 .  Purpose - To describe a small-scale quantitative evaluation of the scholarly information search engine, Google Scholar. Design/methodology/approach - Google Scholar's ability to retrieve scholarly information was compared to that of three popular search engines: Ask.com, Google and Yahoo! Test queries were presented to all four search engines and the following measures were used to compare them: precision; Vaughan's Quality of Result Ranking; relative recall; and Vaughan's Ability to Retrieve Top Ranked Pages. Findings - Significant differences were found in the ability to retrieve top ranked pages between Ask.com and Google and between Ask.com and Google Scholar for scientific queries. No other significant differences were found between the search engines. This may be due to the relatively small sample size of eight queries. Results suggest that, for scientific queries, Google Scholar has the highest precision, relative recall and Ability to Retrieve Top Ranked Pages. However, it achieved the lowest score for these three measures for non-scientific queries. The best overall score for all four measures was achieved by Google. Vaughan's Quality of Result Ranking found a significant correlation between Google and scientific queries. Research limitations/implications - As with any search engine evaluation, the results pertain only to performance at the time of the study and must be considered in light of any subsequent changes in the search engine's configuration or functioning. Also, the relatively small sample size limits the scope of the study's findings. Practical implications - These results suggest that, although Google Scholar may prove useful to those in scientific disciplines, further development is necessary if it is to be useful to the scholarly community in general. Originality/value - This is a preliminary study in applying the accepted performance measures of precision and recall to Google Scholar. It provides information specialists and users with an objective evaluation of Google Scholar's abilities across both scientific and non-scientific disciplines and paves the way for a larger study.


Shultz, M. (2007). Comparing test searches in PubMed and Google Scholar. Journal of the Medical Library Association : JMLA, 95(4), 442–445. DOI: http://dx.doi.org/10.3163/1536-5050.95.4.442 .

Smith, A.G. (2007). Benchmarking Google Scholar with the New Zealand PBRF research assessment exercise. Scientometrics, 74(2), 309–316. DOI: http://dx.doi.org/10.1007/s11192-008-0219-8 . Google Scholar was used to generate citation counts to the web-based research output of New Zealand Universities. Total citations and hits from Google Scholar correlated with the research output as measured by the official New Zealand Performance-Based Research Fund (PBRF) exercise. The article discusses the use of Google Scholar as a cybermetric tool and methodology issues in obtaining citation counts for institutions. Google Scholar is compared with other tools that provide web citation data: Web of Science, SCOPUS, and the Wolverhampton Cybermetric Crawler.

Walters, W.H. (2007). Google Scholar coverage of a multidisciplinary field. Information Processing & Management, 43(4), 1121–1132. DOI: http://dx.doi.org/10.1016/j.ipm.2006.08.006 This paper evaluates the content of Google Scholar and seven other databases (Academic Search Elite, AgeLine, ArticleFirst, GEOBASE, POPLINE, Social Sciences Abstracts, and Social Sciences Citation Index) within the multidisciplinary subject area of later-life migration. Each database is evaluated with reference to a set of 155 core articles selected in advance-the most important studies of later-life migration published from 1990 to 2000. Of the eight databases, Google Scholar indexes the greatest number of core articles (93%) and provides the most uniform publisher and date coverage. It covers 27% more core articles than the second-ranked database (SSCI) and 2.4 times as many as the lowest-ranked database (GEOBASE). At the same time, a substantial proportion of the citations provided by Google Scholar are incomplete (32%) or presented without abstracts (33%).
 

2006 [Go back]

Bakkalbasi, N., Bauer, K., Glover, J., Wang, L. (2006). Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical digital libraries, 3(1), 7. DOI: http://dx.doi.org/10.1186/1742-5581-3-7 Background Researchers turn to citation tracking to find the most influential articles for a particular topic and to see how often their own published papers are cited. For years researchers looking for this type of information had only one resource to consult: the Web of Science from Thomson Scientific. In 2004 two competitors emerged – Scopus from Elsevier and Google Scholar from Google. The research reported here uses citation analysis in an observational study examining these three databases; comparing citation counts for articles from two disciplines (oncology and condensed matter physics) and two years (1993 and 2003) to test the hypothesis that the different scholarly publication coverage provided by the three search tools will lead to different citation counts from each. Methods Eleven journal titles with varying impact factors were selected from each discipline (oncology and condensed matter physics) using the Journal Citation Reports (JCR). All articles published in the selected titles were retrieved for the years 1993 and 2003, and a stratified random sample of articles was chosen, resulting in four sets of articles. During the week of November 7–12, 2005, the citation counts for each research article were extracted from the three sources. The actual citing references for a subset of the articles published in 2003 were also gathered from each of the three sources. Results For oncology 1993 Web of Science returned the highest average number of citations, 45.3. Scopus returned the highest average number of citations (8.9) for oncology 2003. Web of Science returned the highest number of citations for condensed matter physics 1993 and 2003 (22.5 and 3.9 respectively). The data showed a significant difference in the mean citation rates between all pairs of resources except between Google Scholar and Scopus for condensed matter physics 2003. For articles published in 2003 Google Scholar returned the largest amount of unique citing material for oncology and Web of Science returned the most for condensed matter physics. Conclusion This study did not identify any one of these three resources as the answer to all citation tracking needs. Scopus showed strength in providing citing literature for current (2003) oncology articles, while Web of Science produced more citing material for 2003 and 1993 condensed matter physics, and 1993 oncology articles. All three tools returned some unique material. Our data indicate that the question of which tool provides the most complete set of citing literature may depend on the subject and publication year of a given article.

Bar-Ilan, J. (2006). An ego-centric citation analysis of the works of Michael O. Rabin based on multiple citation indexes. Information Processing & Management, 42(6), 1553-1566The primary goal of this study was to carry out an ego-centric citation and reference analysis of the works of the mathematician and computer scientist, Michael O. Rabin. Until recently only a single citation database was available for such research – the ISI Citation Indexes. In this study we utilized and compared three major sources that provide citation data: the Web of Science, Google Scholar and Citeseer. Most cited works, citation identity, citation image makers and coauthors were identified. The citation image makers acquired through these sources differ considerably. Advantages and shortcomings of each of the tools are discussed in the context of computer science. A major issue in computer science is multiple manifestations of a work, i.e., its publication in several venues (technical reports, proceedings, journals, collections). The implications of multiple manifestations for citation analysis are discussed.
 
Bosman, J., Mourik, I. V., Rasch, M., Sieverts, E., Verhoeff, H. (2006). Scopus reviewed and compared: The coverage and functionality of the citation database Scopus, including comparisons with Web of Science and Google Scholar. Utrecht University Library (Report)
Scopus is a new entrant in the market for multidisciplinary citation databases. This report analyses the coverage and functionality of Scopus and compares it to ISI's Web of Science and Google Scholar. Scopus comes out as a user-friendly product with an overall broader coverage of life sciences and physical sciences, compared to Web of Science. In social sciences coverage is not yet fully convincing. There are some volume/issue gaps in Scopus coverage as well.
Mayr, P., Walter, A.K. (2006). Coverage and up-to-dateness of the Google Scholar index). Information Wissenschaft und Praxis, 57(3). [http://hdl.handle.net/10760/7526]. 
The paper discusses the new Google search service Google Scholar. This search engine, which is intended for searching exclusively scholarly documents, will be described with its most important functionality and then empirically tested. The study is based on queries against different journal lists: journals from Thomson Scientific, Open Access journals (DOAJ) and journals of the German social sciences literature database SOLIS as well as the analysis of result data from Google Scholar. The study shows deficiencies in the coverage and up-to-dateness of the Google Scholar index. Furthermore, the study points up which web servers are the most important data providers for this search service and which information sources are represented. We conclude that Google Scholar has some interesting potentials (e.g. citation analysis) but can not be seen as a substitute for the use of special literature databases due to a couple of weaknesses (e.g. transparency)

Neuhaus, C., Neuhaus, E., Asher, A., Wrede, C. (2006). The depth and breadth of Google Scholar: An empirical study. portal: Libraries and the Academy, 6(2), 127-141. DOI: http://dx.doi.org/10.1353/pla.2006.0026 . The introduction of Google Scholar in November 2004 was accompanied by fanfare, skepticism, and numerous questions about the scope and coverage of this database. Nearly one year after its inception, many of these questions remain unanswered. This study compares the contents of 47 different databases with that of Google Scholar. Included in this investigation are tests for Google Scholar publication date and publication language bias, as well as a study of upload frequency. Tests show Google Scholar's current strengths to be coverage of science and medical databases, open access databases, and single publisher databases. Current weaknesses include lack of coverage of social science and humanities databases and an English language bias.

Salisbury, L., Tekawade, A. (2006). Where is agricultural economics and agribusiness research information published and indexed? A comparison of coverage in Web of Knowledge, CAB Abstracts, Econlit, and Google Scholar. Journal of agricultural & food information, 7(2-3), 125-143. DOI: http://dx.doi.org/10.1300/J108v07n02_10. Identifying the best source of information to satisfy the needs of local clientele has always been the challenge facing collection development and instruction librarians. In order to provide users with the best possible access tool and source for comprehensive information, it is important that librarians be aware of the most productive sources of information in a field. This paper identifies where the bulk of agricultural economics research is published and indexed. It also ascertains whether Google Scholar is as productive in covering this information as CAB Abstracts and EconLit. The cited reference counts for the top 50 cited sources in Web of Knowledge and Google Scholar are also compared. The scatter of the journal literature in this field, based on the Library of Congress Subject Headings, is also provided. The study identified a broad range of article scattering in areas where agricultural economics and agribusiness materials are published. On the cited reference count, Google Scholar was as productive as the Web of Knowledge. Google Scholar is a free source of very useful information for cited references and other subject searches in the area of agricultural economics and could be used to complement traditional databases.
Tarantino, E. (2006). Troppo o troppo poco? Web of science, Scopus, Google scholar: tre database a confronto (un caso di studio). Bollettino AIB, 46(1/2),   23–34. Google Scholar (http://scholar.google.com) provides a new method of locating potentially relevant articles on a given subject by identifying subsequent articles that cite a previously published article. An important feature of Google Scholar is that researchers can use it to trace interconnections among authors citing articles on the same topic and to determine the frequency with which others cite a specific article, as it has a “cited by” feature. This study begins with an overview of how to use Google Scholar for citation analysis and identifies advanced search techniques not well documented by Google Scholar. This study also compares the citation counts provided by Web of Science and Google Scholar for articles in the field of “Webometrics.” It makes several suggestions for improving Google Scholar. Finally, it concludes that Google Scholar provides a free alternative or complement to other citation indexes.



Yang, K., Meho, L.I. (2006). Citation Analysis: A Comparison of Google Scholar, Scopus, and Web of Science. Proceedings of the American Society for Information Science and Technology, 43(1), 1–15. DOI: http://dx.doi.org/10.1002/meet.14504301185When faculty members are evaluated, they are judged in part by the impact and quality of their scholarly publications. While all academic institutions look to publication counts and venues as well as the subjective opinions of peers, many hiring, tenure, and promotion committees also rely on citation analysis to obtain a more objective assessment of an author's work. Consequently, faculty members try to identify as many citations to their published works as possible to provide a comprehensive assessment of their publication impact on the scholarly and professional communities. The Institute for Scientific Information's (ISI) citation databases, which are widely used as a starting point if not the only source for locating citations, have several limitations that may leave gaps in the coverage of citations to an author's work. This paper presents a case study comparing citations found in Scopus and Google Scholar with those found in Web of Science (the portal used to search the three ISI citation databases) for items published by two Library and Information Science full-time faculty members. In addition, the paper presents a brief overview of a prototype system called CiteSearch, which analyzes combined data from multiple citation databases to produce citation-based quality evaluation measures.

2005 [Go back]

Belew, R. K. (2005). Scientific impact quantity and quality: Analysis of two sources of bibliographic data. arXiv preprint arXiv:cs/0504036
Attempts to understand the consequence of any individual scientist's activity within the long-term trajectory of science is one of the most difficult questions within the philosophy of science. Because scientific publications play such as central role in the modern enterprise of science, bibliometric techniques which measure the ``impact'' of an individual publication as a function of the number of citations it receives from subsequent authors have provided some of the most useful empirical data on this question. Until recently, Thompson/ISI has provided the only source of large-scale ``inverted'' bibliographic data of the sort required for impact analysis. In the end of 2004, Google introduced a new service, GoogleScholar, making much of this same data available. Here we analyze 203 publications, collectively cited by more than 4000 other publications. We show surprisingly good agreement between data citation counts provided by the two services. Data quality across the systems is analyzed, and potentially useful complementarities between are considered. The additional robustness offered by multiple sources of such data promises to increase the utility of these measurements as open citation protocols and open access increase their impact on electronic scientific publication practices.

Callicott, B., Vaughn, D. (2005). Google Scholar vs. Library Scholar: Testing the Performance of Schoogle. Internet Reference Services Quarterly, 10(3/4), 71-88. DOI: http://dx.doi.org/10.1300/J136v10n03_08How does the content of Google Scholar, a.k.a. “Schoogle,” compare to that of subscription databases and the library catalog? Five sample research topics indigenous to undergraduate libraries were searched in Google Scholar, the College of Charleston online catalog, EBSCO's Academic Search Premier database, and a subject-specific subscription database. Points of consideration included document type, availability of full-text materials, local availability of materials (either in print or online), and relevance of materials to the research topics. Results showed that Google Scholar, while a substantive supplementary research tool, does not provide the samequality in terms of relevance for many research topics.

Gardner, S., Eng, S. (2005). Gaga over Google? Scholar in the Social Sciences. Library Hi Tech News, 22(8), 42-45. DOI: http://dx.doi.org/10.1108/07419050510633952Purpose – To provide a summary of the main features of Google Scholar.Design/methodology/approach – Reviews, contextualizes and provides a summary of Google Scholar. Findings – This article compares the results of a sample search on “homeschooling in Google Scholar against the results in three fee-based article index databases in the social sciences: PsycINFO, Social Science Citation Index, and ERIC. Comparisons are done in the areas of content, currency, relevancy, and overlap. Google Scholar yields more results and a greater variety in its types of sources along with a higher rate of relevancy, but less currency. Ultimately, Scholar’s lack of quality control and inability to let the user manipulate data make it less effective than the fee-based databases at finding scholarly material in the social sciences. Originality/value – Provides a useful summary for information professionals.

Jacso, P. (2005). As we may search–comparison of major features of the Web of Science, Scopus, and Google Scholar. Current Science, 89(9), 1537-1547.
Jacso, P. (2005). Comparison and analysis of the citedness scores in Web of Science and Google Scholar. In Digital libraries: Implementing strategies and sharing experiences, 360-369. DOI: http://dx.doi.org/10.1007/11599517_41. An increasing number of online information services calculate and report the citedness score of the source documents and provide a link to the group of records of the citing documents. The citedness score depends on the breadth of source coverage, and the ability of the software to identify the cited documents correctly. The citedness score may be a good indicator of the influence of the documents retrieved. Google Scholar gives the most prominence to the citedness score by using it in ranking the search results. Tests have been conducted to compare the individual and aggregate citedness scores of items in the results list of various known-item and subject searches in Web of Science (WoS) and Google Scholar (GS). This paper presents the findings of the comparison and analysis of the individual and aggregate citation scores calculated by WoS and GS for the papers published in 22 volumes of the Asian Pacific Journal of Allergy and Immunology (APJAI). The aggregate citedness score was 1,355 for the 675 papers retrieved by WoS, and 595 for 680 papers found in GS. The findings of the analysis and comparison of tests, and the reasons for the significant limitations of Google Scholar in calculating and reporting the citedness scores are presented.



Noruzi, A. (2005). Google Scholar: The New Generation of Citation Indexes. Libri, 55(4), 170-180. 
Google Scholar (http://scholar.google.com) provides a new method of locating potentially relevant articles on a given subject by identifying subsequent articles that cite a previously published article. An important feature of Google Scholar is that researchers can use it to trace interconnections among authors citing articles on the same topic and to determine the frequency with which others cite a specific article, as it has a “cited by” feature. This study begins with an overview of how to use Google Scholar for citation analysis and identifies advanced search techniques not well documented by Google Scholar. This study also compares the citation counts provided by Web of Science and Google Scholar for articles in the field of “Webometrics.” It makes several suggestions for improving Google Scholar. Finally, it concludes that Google Scholar provides a free alternative or complement to other citation indexes.