640 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo Javno zdravstvoPregledni znanstveni članek Centralna medicinska knjžnica, Medicinska fakulteta, Univerza v Ljubljani Korespondenca/ Correspondence: nana turk, e: nana.turk@mf.uni-lj.si Ključne besede: odprti dostop; vpliv odprtega dostopa na citiranje; citiranost; vpogledi; omembe v socialnih medijih; altmetrics Key words: open access; citation impact; citation advantage; citation count; article download; article usage; article level metrics; altmetrics; social media attention Citirajte kot/Cite as: Zdrav Vestn. 2016; 85:640–63. Prispelo: 28. 6. 2016 sprejeto: 15. 12. 2016 The impact of open access on the medical literature: a review of current literature Vpliv odprtega dostopa na znanstvene objave v medicini: pregled tekoče literature nana turk Izvleček Izhodišča: Namen članka je podati pregled vpliva odprtega dostopa na znanstvene objave v medicini glede na tri kategorije: citiranost, vidnost in omembe v socialnih medijih. Metode: Podatke smo pridobili z iskalno strategijo, ki je vsebovala ključne besede, kot so “open access”, “citation impact”, “citation advantage”, “citations count”, “article download”, “article usage”, “so- cial media attention” in “altmetrics”. Rezultati: Z iskalno strategijo, ki smo jo uporabili v treh zbirkah podatkov, smo pridobili stose- dem člankov, primernih za analizo. V sedeminšestdesetih člankih izmed teh se uporablja enostavna metodologija štetja citatov odprtodostopnih člankov in člankov iz naročniških revij, v devetnajst člankih se primerja število citatov in vpogledov, v enaindvajsetih člankih se raziskujejo omembe v socialnih medijih. Od tega je petindvajset člankov medicinskih, v katerih se obravnava vpliv vseh navedenih kategorij. Zaključki: V sedeminšestdesetih člankih izmed teh se uporablja enostavna metodologija štetja cita- tov odprtodostopnih člankov in člankov iz naročniških revij, v devetnajst člankih se primerja število citatov in vpogledov, v enaindvajsetih člankih se raziskujejo omembe v socialnih medijih. Od tega je petindvajset člankov medicinskih, v katerih se obravnava vpliv vseh navedenih kategorij. Abstract Background: The aim of the article is to conduct an overview of the impact of OA on the medical articles based on 3-part categorization. Methods: Data were identified by a search strategy with eight combinations of keywords (open ac- cess, citation impact, citation advantage, citation count, article download, article usage, social media attention, altmetrics) and searched in three different databases. Results: The analysis was conducted on 107 studies dealing with citations, downloads and social impact. Sixty-seven of them simply employed the counting citations to OA and non-OA articles; nineteen articles compared the downloads and citations counts; and twenty-one articles investigated the social impact of OA articles. Twenty-five articles investigated the citations, download counts, and social impact of medical articles. Conclusions: The studies investigating the citation impact mostly showed citation advantages. Those that employed citation and download counts of medical articles using randomized controlled trials showed that OA articles were downloaded significantly more frequently, but found no evidence of a citation advantage for open access articles. The citation advantage from open access maight be caused by other factors. Results of the studies comparing the social media attention and citations/ downloads of the medical articles are often diametrically opposed. Zdrav Vestn | november – december 2016 | Letnik the impact of open access on the medical literature: a review of current literature 641 PregLedni ZnanstVeni čLanek 1. Introduction The advent of the Internet and its endless possibilities for information processing and distribution has enabled an open access (OA) to scientific liter- ature. OA has been feasible since the World Wide Web was launched, but the term itself only gained currency in 2002, when it was first defined. With OA, in- stead of relying on subscriptions to subsidize publishing costs, expenses are paid by sponsorships or author-side fees, which may be covered by institutions or grant-funding organizations  (1). There are numerous publishing efforts to make digital versions of hundreds of journals, often published by societies freely avail- able online. OA journal provides an im- mediate OA to all of its articles on the publisher’s website  (2). An alternative way for researchers to disseminate their work is by self-archiving which involves the authors’ placing versions of their ar- ticles that are published in a tradition- al, subscription journals (where only subscribers have immediate access), in freely accessible archives (i.e. on their personal and/or institutional Web page or repositories/ archives). Over the past 20 years the effect of OA on the visibility or impact of scien- tific publications has become one of the most important issues in the fields of bibliometrics and information science. There has been much discussion and analysis as to whether open access af- fects the number of times a publication is cited. Increased citation rates translate into increased impact, which is import- ant for OA as a publishing model and for the researchers, research funders and universities. However, there is a strong correlation between the impact of OA and citations. The intensity of impact also differs from discipline to discipline. In life sci- ences it is not uncommon for a research- er to produce several journal articles; but social scientists and humanities scholars concentrate on publishing monographs. This so-called disciplinary culture has an impact on publishing processes and the ways by which researchers in each dis- cipline communicate their findings  (3). The current paper explores the effect of OA on the field of medicine. A review of the literature was conducted to map the ways in which impact has been defined, measured and studied, and to identify areas for future research. 1.1 Scope and purpose of present paper Research into the impact of Open Access uses two different methods to ex- amine researchers’ opinion on the sub- ject and the application of metrics to measure the impact of OA: surveys and interviews. This review of the literature focuses on data from studies using bib- liometric methods. The paper is primar- ily focused on medical articles. The purpose of this review is to high- light the findings about the impact of open access starting with the early re- search. An analysis of core articles will provide evidence of critical issues that this literature has identified and reveal trends in OA research. Most of the ear- ly studies employed a traditional biblio- metric tool for measuring impact – cita- tion analysis–which counts the number of times a given article has been cited. This measure provides a quantitative proxy to the quality of the articles, au- thors and journals (4). 1.2 A brief history of Open Access One of the triggers for the develop- ment of Open Access publishing was the »serials crisis« at the end of the 20th cen- 642 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo tury. Put simply, for many years, subscrip- tion costs for publications rose much faster than inflation. The emergence of the Internet, however, made it possible for anyone to publish and share information on the web. This posed new challenges and opportunities for scientific publish- ing. 1991 saw the creation of the first free scientific online archive or repository for physicists, called arXiv.org. Publishing articles in arXiv, however, had no effect on journals subscriptions in physics be- cause all it did was to encourage scien- tists to self-archive their pre-publication articles into an online depository. In 2000, the National Institute of Health (NIH) released PubMed Central, a digital repository of biomedical and life sciences literature that has now grown to almost 6 million articles. The NIH, the largest funder of medical research in the world, requires that papers describing re- search funded by the NIH must be avail- able to the public for free through PubMed Central within 12 months of publication. Participating publishers may archive their entire journals in PMC, and thus all the articles published in those journals would appear in PMC. The publisher may also choose to deposit other, non-NIH-fund- ed articles. Non-NIH-funded articles can be deposited by their authors or by the journals as well. The Wellcome Trust, a charitable foundation, supports an un- restricted access to published outputs. Its open access policy demands that the recipients of its grants deposit a copy of their articles with PubMed Central (5,6). Another initiative, BioMed Central (BMC), launched in 2000, is a Unit- ed Kingdom-based, for-profit scientif- ic publisher specialising in open access journal publication. In 2002, processing fees were introduced to cover the costs of free online access (7). The specific features of free and un- restricted availability of scientific liter- ature are spelled out in various public statements, including the 2002 Buda- pest Open Access Initiative (8), the 2003 Bethesda Statement on Open Access Publishing (9), and the 2003 Berlin Dec- laration on Open Access (10). At a 2005 follow-up conference, the Berlin decla- ration was refined to two key principles– researchers should be required to: • deposit a copy of their work in an open access repository and • encourage the publication of work in open access journals when available. These two concepts have given rise to what is often called “Green OA” and “Gold OA”, respectively, and the two combined are referred to as an open-ac- cess mandate (10). In 2002, Lund University in Sweden launched another open access project »The Directory of Open Access Jour- nals« with 300 open access journals. To- day it contains more than 9,000 open access journals covering all areas of sci- ence, technology, medicine, social sci- ence and humanities (11). One final development worth noting is the Public Library of Science (PLOS). This nonprofit, open access scientific publishing project, aimed at creating a library of open access journals and other scientific literature under an open con- tent license, was established in 2003. In 2008 it introduced article-level metrics into its journal platform, which can help users determine the value of an article to themselves and to their scientific com- munity (12). 1.3 Measuring impact Since Eugene Garfield founded Sci- ence Citation Index in 1964, citation counts and citation analysis have been used in bibliometric methods to trace relationships amongst academic jour- the impact of open access on the medical literature: a review of current literature 643 PregLedni ZnanstVeni čLanek nal citations or extract measurable data through the statistical analysis of text and information about how the text is being used (13). The results of bibliomet- ric analysis provide the data used to mea- sure the impact of information resources. As noted earlier, the Internet has fun- damentally changed the way in which information is distributed and accessed. The web has enabled a new system for communicating research and given rise to the Open Access (OA) movement. The OA movement with its emphasis on sharing information freely and elec- tronically, together with advances in dig- ital publishing, are creating more oppor- tunities than ever for researchers to have an impact. In this new environment, many researchers and journal publish- ers have begun to seek alternative im- pact indicators for OA resources such as mentions, acknowledgments, endorse- ments, downloads, recommendations, blog posts, and tweets (14). Bibliometrics is now a fast-growing, multidisciplinary field that ranges from webometrics to scientometrics to altmet- rics. The term altmetrics was coined by Jason Priem in 2010 as a broad, rapid and transient impact of scientific publishing. Altmetrics seek to measure impact not solely in terms of the number of times a scientific article gets cited but also by monitoring, tracking and measuring oth- er aspects of scientific literature, such as article downloads, views, comments and tweets. Altmetrics provide a new way of detecting the use of scientific publishing beyond formal citation (4,15). 1.4 Early research into Open Access The purpose of scientific literature is to disseminate research findings and provide a permanent archive. Research- ers and academics are not paid for pub- lishing. Why do they write in the absence of financial incentives? There are various reasons such as career advancement and a desire to share knowledge and advance their discipline. As a starting point the early studies looked at the prerequisites and barriers for OA publishing. The three channels investigated were open access journals, subject repositories and institutional re- positories (16). OA is not just about en- suring the dissemination of information; it is also about increasing the impact of articles made available through OA. Re- search into OA impact seeks to demon- strate how much an article is used and how much it is valued. The techniques for analysing scientific literature have a long history. Once scientific literature became available online, there were opportuni- ties to carry out new types of research into the ways in which science advances. Traditional methods of determining the quality and impact of research activities have been supplemented by new meth- ods of investigation. In particular, the original aim was to test whether there was an overall rise in citations for articles which were available through open ac- cess. The expectation was that OA would result in increased usage of published articles since OA allows research find- ings to reach an audience which hitherto lacked access. The expectation was that there would be a boost in citations, vary- ing in magnitude by discipline (17). These expectations were turned into  research questions. The main issue was whether OA, by increasing visibility, findability and accessibility for research articles, would increase citations to those articles. 1.5 Debates and controversies about the impact of Open Access The current debate about the impact of OA started in 2001 with the publi- 644 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo cation of a paper by Steve Lawrence in the journal Nature which analysed con- ference proceedings in the field of com- puter science. The author’s conclusion was that open access maximises impact, minimises redundancy and speeds sci- entific progress  (18). The paper stimu- lated debate and led other researchers to consider which factors maximise im- pact and  the effect of OA on different fields of scholarly research. By and large the debate centred on the impact of OA on citation. One question was wheth- er early online access (before publica- tion in a journal) boosts citation. In the early stages of research it was assumed that OA articles would generate more citations than non-OA articles. These studies were beset with methodological problems. Determining the appropriate time after publication to measure the citation rate needs to be specified. Un- fortunately, in many studies, the date on which an article was published in an OA journal was not always clear. The major- ity of these studies were observational, meaning that they simply observed the citation performance of two sets of ar- ticles. The selection advantage hypothesis argues that self-archiving of original research papers increases citation rates, because authors tend to make their highest quality papers available in insti- tutional repositories. Similarly, the early access advantage hypothesis proposes that self-archiving preprint increases ci- tation because they are available before the publication (19). Another point of debate relates to the analysis of article downloads as a complementary method for studying the impact of OA articles in hybrid OA journals. These are subscription journals in which some of the articles are open access. This status typically requires the payment of a publication fee (20). Many OA journal providers offer download statistics for the articles which are proxies for article readership or ar- ticle usage. This raises the question as to whether higher numbers of down- loads are associated with higher citation rates. As noted previously, in recent years there has been a growing interest in the use of alternative metrics (altmetrics) to assess the impact of publications. This involves using data from social media platforms such as blogs, Twitter feeds, Mendeley, LinkedIn, Facebook, and Ci- teULike. Social media, especially Twitter, is increasingly used as a way of demon- strating impact in biomedicine. Haust- ein  (21) reported that 20 % articles in- dexed in PubMed in 2012 received at last one tweet. This finding raises the ques- tion as to whether an OA paper which scores high on altmetrics tools actually gets much higher download and citation rates compared to a paper with a low alt- metrics score. Impact assessment is one of the ma- jor drivers in scholarly communication. Metrics used as an aid to the evaluation of research might help promote open ac- cess. 1.6 Methodology One of the challenges of carrying out a review of current literature into the impact of open access is the growing body of empirical literature published in the recent years (22). There is also the fact that this is an interdisciplinary sub- ject attracting researchers from diverse fields publishing in many different jour- nals. The aim of the present study was to perform a search of peer-reviewed pub- lications and reports in the published literature. The first step was to examine the core literature to get a sense of the themes and the critical issues that this literature has highlighted. the impact of open access on the medical literature: a review of current literature 645 PregLedni ZnanstVeni čLanek A preliminary scan of the literature suggested that there are two emergent themes to explore: 1. bibliometrics and altmetrics methods used to measure the impact of OA; 2. conflicting interpretations of causal factors. It was decided at the outset to focus this investigation on the literature relat- ing to bibliometrics and altmetrics, with a particular emphasis on literature dealing with quantitative analysis of open access. This paper will summarise the literature describing the advantages and disad- vantages of open access with respect to citations, downloads and altmetrics indi- cators. The analysis will focus on differ- ent variables employed by the studies to test the association of OA with citations, downloads and altmetrics. It will also fo- cus on the question whether there is sim- ply a correlation between OA publication and the number of citations or the stud- ies establish a causal relationship. The review deals with studies con- ducted between 2001 and 2016. The key words are: open access, citation impact, download, article usage and social me- dia. The search strategy used similar words such as citation advantage, cita- tion rate, and altmetrics. Searches were conducted using • Google Scholar, a widely used aca- demic search engine, which plays a major role in finding free full-text versions of articles; • MEDLINE (PubMed interface), a bib- liographic database produced by the U.S. National Library of Medicine; • Emerald Insight, a database of journal articles from Emerald, a cross-disci- plinary publisher, which enables ac- cess to more than 300 journals from different scientific fields; • several bibliographies, including SPARC Eureope’s (23), Wagner’s (24), Hitchcock’s (25) and Swan’s (17). The search covered these  sources, using the key words and their Boolean combinations. The basic Boolean search operator AND was used to narrow down search results and OR to broader the search more similar concepts. The pri- mary search strategy was “open access” AND (“citation impact” OR “citation ad- vantage” OR »article level metrics« OR “citation count” OR “article download” OR “article usage” OR “social media at- tention” OR “altmetrics”). As the func- tions of different search engines and da- tabases varied, it was necessary to adapt the terms to suit each one. Existing bibli- ographies on the impact of OA were also consulted. While conducting the literature search, we determined which articles contained relevant data. The first step was searching by the titles of the records. Then we identified the key elements in them and filtered out any peripheral materials. 186 records of the articles are identified. All articles identified in the search were screened for relevance to the key question. Relevant data from the papers selected for inclusion were ex- tracted or copied from the publication to a database to create the set of studies used to investigate  the citation impact, downloads and social media usage of OA articles. The criteria for inclusion were: • articles where the abstracts contained one or more of the key search terms; • articles which were peer-reviewed and used quantitative methods, such as counting article downloads, cita- tions or different altmetrics indica- tors. 646 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo Exclusion criteria: • studies that did not directly link with the impact of open access; • review papers and bibliographies. However, these papers were checked later to find the missing articles; • studies regarding the production, publishing, or availability of informa- tion (e.g. papers on the economics of scientific publishing); • works based on personal beliefs and anecdotal evidence; • studies of online newspapers, maga- zines, and trade publications; • opinion pieces. Once the process of identifying the key articles had been completed, a re- view of the literature was undertaken. Themes related to the measurement of OA impact were put into three groups: • citation impact; • download analyses; • social impact of open access articles. 2 The literature review – Summary of findings Considering that these inclusion and exclusion criteria may be modified to better identify relevant studies on a top- ic of open access advantages, the litera- ture review of the scholarly literature by summarizing and analyzing published work on that topic yielded 107 research peer-reviewed articles which were divid- ed into 3 categories: • studies which examined citation per- formance of OA (no = 67); • studies of downloads of the OA arti- cles (no = 19); • studies which explored the social im- pact of OA articles (no = 21) Table 1: the impact of the oa articles Citation performance of OA No. of the articles Percent impact of medical articles 5 4,67 impact of oa and non-oa journals 11 10,28 Citation rates before and after journal articles are made openly available 6 5,61 impact factor between oa and non-oa journals 11 10,28 number of oa and non-oa articles 34 31,78 Σ 67 62,62 % Downloads of the OA articles downloads of the medical articles 10 9,35 others 9 8,40 Σ 19 17,75 % Social impact of OA articles social impact of the medical articles 11 10,28 others 10 9,35 Σ 21 19,63 % the impact of open access on the medical literature: a review of current literature 647 PregLedni ZnanstVeni čLanek 2.1 Studies on the investigations of the citation performance of OA articles (n = 67) The goal was to test whether there is an overall rise in citations for OA which derived from a set of assumptions: • some of the researchers don’t have ac- cess to the subscription journals that are relevant to their work; • OA would raise the level of reader- ship and provide a citation boost (17). Some studies aiming to test if there is an overall rise of citations of OA articles employed the analyses between access status and citations based on: • the investigation the impact of OA and non-OA journals (26-36); • the comparison of the citation rates before and after journal articles are made openly available (34,37-41); • the comparison of the journal’s im- pact factor between OA and subscrip- tion journals (16,34,42-50). Currently, our studies on citation counts employ three approaches: • comparing the impact of OA and non-OA articles in the same journals; • cross-disciplinary comparison of OA and non-OA articles; • investigating the relationship be- tween self-archiving and citations. 2.1.1 The citation counts of OA and non-OA articles The methodology used by these stud- ies in this literature review is based on the basic metric of the average citation count for the OA and non-OA articles. In our study we included 38 articles employing the counting citations of OA and non-OA articles. We analysed the articles in regard to the comparison of the main number of citations between • articles published OA (made freely available on the Internet) and non- OA in the same journals (green OA) and • OA and non-OA articles been made in the repository (gold OA) 2.1.1.1 Articles published OA and non-OA in the same journals (Green OA)(n = 10) Ten articles (written between 2004 and 2015) showed the comparison of the OA articles to articles published in the same journals that had not been made OA. Studies focused on the different disciplines as are natural sciences (ag- riculture, physics, civil engineering, life sciences), formal sciences (mathemat- ics) and social sciences (anthropology, economics, law, library and information sciences, political sciences). Most studies on the open access ci- tation advantage (OACA) were made in the field of social sciences represent- ed by law, economics, library and infor- mation sciences, political sciences and anthropology. The samples were very different and varied from 286 records in three international civil engineer- ing journals  (51) to 6,024 journal arti- cles from law  (52). In these studies the citations were gathered from different services as are Thomson Reuter’s Web of Science  (33,53-55), RePEc  (53), Goo- gle Scholar  (51,53,56,57), Elsevier’s Sco- pus (51,57) or Shepard’s Citation Services from LexisNexis (22,52). The investigation on the annual com- parison of OA and non-OA articles in the field of law employed the study of 566 articles (124 OA and 442 non-OA ar- ticles) from 3 hybrid journals which are published at the University of Georgia School of Law and appeared to account for almost half of the output of two law faculties (22). The study which employed 648 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo 6,042 articles (3,489 OA and 2,553 non- OA) in 30 journals, each published at another faculty, included only lead ar- ticles  (52). The comparison of the OA and non-OA articles on economics were made with 639 articles (508 OA articles and 131 non-OA articles) in 13 journals including different quality levels of jour- nals (53). The comparison between OA arti- cles and non–OA articles in the field of library and information sciences was made by the investigation of the citation rate of 875 articles (486 OA articles, 389 non-OA articles) from 20 high-impact journals that appeared in the Journal Citation Reports (JCR) and Ulrich’s Pe- riodicals Directory lists, both with high journal impact factors (54). Political sciences were examined in 727 articles (404 OA articles and 323 non-OA articles) from 8 top journals by impact factor (56). The data set includes journals allowing authors to self-archive any version of articles. Anthropology was represented by 667 articles (200 OA and 467 non-OA articles) from a list of ten top-ranked journals versus 10 bottom-ranked journals  (58). Natural sciences represented by the studies em- ployed the field of biology and civil engi- neering. Biology was represented by the studies of the articles published in the Proceedings of the National Academy of Sciences (PNAS) journal. Citation data was extracted from the Web of Science. One of the studies  (59,60) used a sam- ple of 4,388 articles (723 OA articles and 3.665 non-OA articles) published in the area of biological sciences representing approximately 90 % of the papers pub- lished in PNAS. The other study used a sample of 1,704 articles (212 OA articles and 1280 non-OA articles)(55). Civil en- gineering involved two studies. One of them includes 2.026 records (442 OA and 1,584 non-OA articles) in 14 journals (57) and the other 286 records (81 OA and 205 non-OA articles)  (51) from 3 jour- nals with impact factor. In both studies, the journals are categorized in the same subject category “engineering, civil” and have published the researchers from the University of Ljubljana, Faculty of Civil and Geodetic Engineering. The citation data were obtained from Scopus, Google Scholar and Web of Science (51,57). Results All  (10) studies which used different sample sizes show the positive OACA, but reported different ranges of citation advantages. Two studies show general OACA. The research done on economics journals of different quality levels  (53) showed that OA articles have on aver- age a 307,9 % higher citation count than non-OA articles. Research on the OACA on the political sciences from 8 top jour- nals by impact factor showed from 23 % to 74 % higher citation rates of OA arti- cles  (53). The comparison of OACA in the field of law  (22,52) shows that OA articles received on average from 49 % to 58 % more citations than non-OA ar- ticles. Citation advantage was lower for OA articles in the highest-ranked jour- nals because their contents routinely saturate their topical areas regardless of the access option by which they are available. The results of the study from library and information sciences  (33) showed the OACA for articles from low- er-ranked journals. Likewise, the results of the studies in civil engineering Kol- er-Povh  (51,57) confirmed the OACA for the highest-ranked journals. Biology represented by Gaule’s study (59) showed statistically insignificant OACA, while the other study showed statistically sig- nificant OACA from 10–16 months after publication (55). the impact of open access on the medical literature: a review of current literature 649 PregLedni ZnanstVeni čLanek 2.1.1.2 Cross-disciplinary comparison of OA and non- OA articles (n = 9) The literature in the different science disciplines reflects the activities of them. Traditionally, citation indexes provided a view within them. Some of the larger studies of OACA have taken broad sub- ject categories rather than individual subject areas. The results of the studies regarding OACA of articles from differ- ent disciplines were compared to find out the trends. There were nine studies that em- ployed the cross-disciplinary compar- ison of OA and non-OA articles and o Ingwersen (61) compared the number of citations to 20 OA working papers and non-OA journal articles from humanis- tic and social science, agriculture, envi- ronmental and geo-field published in the same year by the same institute (Danish Institute for International Studies, Co- penhagen) and predominately the same authors. The citation impact within 270 articles in 9 disciplines from hard (biolo- gy, economics, physics, mathematics and chemical engineering) and soft scienc- es (anthropology, geography, sociology and psychology) was examined by Ton- ta  (62). The goal of the research was to find out if the OACA for hard sciences is higher than that for soft sciences. An- telman (63)examined the comparison of 1,017 articles (802 OA and 215 non-OA articles) of the 10 leading journals in four disciplines which were selected as disci- plines with a tradition of active use of preprints—mathematics, electrical and electronic engineering, political science, and philosophy. Ten leading journals in each discipline were selected, as defined by Thomson Reuter’s Journal Citation Reports (JCR). High-impact journals were selected as indicators of leading journals from the fields, while making no assumptions about journal quality. The articles from high impact factor journals were also chosen in two studies of Nor- ris (30) which researched the OACA in the study of 4,633 articles in the journals of 4 disciplines (ecology, mathematics, sociology and economics). Sotudeh (64) investigated the OACA of the sample of 474,205 articles (22,549 OA articles and 475,205 non-OA articles) published in 276 hybrid journals between 2007 and 2011. The sample contained the articles from natural and health sciences, life and social sciences and humanities. Archam- bault  (65) analysed 1 million articles in Scopus from 1996 to 2013 for 12 science fields including clinical medicine and biomedical research. Hajjem  (66) researched the citation advantages of 1,307,038 articles in 10 dis- ciplines (biology, psychology, sociology, health, political science, economics, ed- ucation, law, business, management) and used a robotic search algorithm for dis- covering of the OA articles and for the citations count. Xu-li  (67) took sample of 12,354 original research articles (6,904 OA and 5,458 non-OA articles) which were published in 93 Oxford Open jour- nals in 2009. They validated the hypoth- eses that there is OACA for OA articles, that the OACA varies between disci- plines, and that there is some correlation between impact factor and OACA. Results Six studies pointed out the OACA (19,30,63-67) they evidenced the differences between OACA between the subject fields. Antelman  (63) found an increase in citations of OA articles by 45 % in philosophy, 51 % in electri- cal engineering, 86 % in political sci- ence and by 91 % in mathematics. Hajj- em  (66) found open access produces a citation increase between 36 % (biology) and 172 % (sociology). Health science included in his research showed 57 % 650 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo OACA. Norris  (19,30) included the ar- ticles from high impact factor journals, found sociology showed the greatest OA advantage (88 %) and ecology the low- est (44 %). Xia (33,58) found the OACA for 138,87 % higher over non-OA ones. He pointed out the different subjects have different OACA. OA articles from humanities journals have a negative OACA. OA with lower impact factors have stronger OACA. Sotudeh, (64) who confirmed OACA and pointed out the different OACA of different disciplines as well, explained the OACA as the au- thors publish their high-quality articles in OA. In his study the OACA ranged from 3,14 % in social sciences and hu- manities to 35,95 % in natural sciences. The OACA of the articles from 8,26 % in life sciences to 33,29 % in health sciences was 33,29 %. Archambault  (65), who produced a massive study on OACA, found an in- crease in OACA from 9 % in chemistry to 80 % in visual and performing art. The increase of OACA was 18 % in biomed- ical research and 37 % in clinical medi- cine. Two studies measured the OACA indirectly. Tonta’s study  (62) examined whether there is a relationship between OA citation impact and the character- istics of the subject field. OA articles in physics, math and chemical engineering did not have higher research impact than the articles from economics, biology, which received twice as many citations than those in first group. He conclud- ed that there is no relationship between the OACA and the characteristics of the subject fields (hard or soft sciences). Only one study showed the negative OACA. Ingwersen (61), which compared OA and non-papers in the same year and by the same institute found that OA working papers were far less cited than subscribed peer-reviewed journal articles. 2.1.1.3 Citation counts of self- archived articles (n = 15) About 15–20 % of the 2,5 million ar- ticles published annually worldwide are being self-archived by their authors today  (66). Since institutional reposito- ries at their initial stage of development in the early 2000s followed what sub- ject-based repositories had already prac- ticed for many years, self-archiving has become the primary way of aggregating digital collections  (33). The purpose of the self-archiving of research papers is to maximize their accessibility and citation impact (68). In our sample there are fifeteen stud- ies dealing with the citation impact of articles which are self-archived in the different type of repositories (subject or institutional; mandated or non-mandat- ed). Eight studies examined the OACA of articles posted in the repository arXiv (Physics, Mathematics, Computer Sci- ence, Quantitative Biology, Quantita- tive Finance and Statistics) compared to non-OA articles. Gentil-Beccot  (31) compared citations to three sets of 286,180 OA articles: preprints (pre-peer- review) or postprints posted in arXiv and subsequently published in journals and articles posted in arXiv and never published in journals. Moed  (77) com- pared citations to 74,521 articles articles posted to arXiv with those to articles in the same journals that were not made available through arXiv. There are other investigations deal- ing with articles from a limited num- ber of journals in order to compare the OACA of articles between subscribed articles and articles deposited in arX- iv. Schwarz  (69) compared 795 OA ar- ticles posted to arXiv and subscribed articles from the Astrophysical Journal. Kurtz  (38,70) conducted two studies using citations from 4,271 articles in Astrophysical journal and 2,592 articles the impact of open access on the medical literature: a review of current literature 651 PregLedni ZnanstVeni čLanek in the seven core astrophysics journals. Within the sample of 4,900 articles from 16 journals, Aman (71) compared the speed until the first citation and to- tal number of citations between articles published with an OA arXiv preprint and those without. Metcalfe  (72) com- pared 7,000 articles from 13 major as- tronomy journals in order to find out if OA articles were made OA either in the arXiv or in Montana State Univer- sity’s solar physics Open Access archive. Henneken  (73) traced the citations of the articles published in 2 astronomy and 2 physics journals. Metcalfe  (74) researched 341 articles from 3 physics journals. Three studies examined the OACA of articles posted to other subject reposi- tories. Kim’s (41) study, which employed a multi-disciplinary online repository of scholarly research called SSRN, re- searched citations to papers in the same journals not made freely available. The author’s sample employed 4,205 arti- cles (385 OA articles and 3,820 non- OA articles). Frandsen  (75) and Muel- ler-Langer (76) compared the OACA of papers in the repositories from the eco- nomics. Their study involved 1,329 arti- cles (208 OA articles and 1,121 non-OA articles) in 15 journals and gathered the pre-prints from SSRN (Social Science Research Network) and RePEc. RePEC is a collaborative effort of hundreds of volunteers to enhance the dissemina- tion of research in economics and re- lated sciences hat covers 1,800 archives from 87 countries. Frandsen (75) focused on the repositories in economic, name- ly, EconLit (published by the American Economic Association) and RePEc, and examined the impact during 10 years of open availability for about 2,000 work- ing papers and 13,000 articles. The other four studies examined the OACA of articles posted in institution repositories. Gargouri (68,77) examined the set of 27,197 articles in 1,984 jour- nals posted in the institution reposito- ry from the field of engineering, biolo- gy, biomedicine, chemistry, psychology, mathematics, clinical medicine, health, physics, social sciences, earth sciences. They compared the OACA between ar- ticles with 6,215 mandated and 20,982 non-mandated status. Ottaviani  (78) compared the OACA in 89,895 non- OA articles and 3,850 OA articles with embargoes posted in the University of Michigan’s institutional repository. All articles in this sample were embargoed during some or all of their prime cita- tion years. Kullman  (79) examined the set of 3,470 articles (899 in full text, 2,571 with bibliographic data only) archived in the Chalmers University of Technology (Göteborg, Sweden) university reposi- tory. Snijder  (80) took a sample of 400 books (300 OA books and 100 control books) in the Amsterdam University Press. Results Three studies out of 15 indicated no OACA. Snijder’s  (80) study of OACA of institution repository found no re- lation between OA status and citation rates. However, contrary to expectations, there was also no diminishing effect of OA status on sales. Frandsen (75) found no clear tendency towards an increase in impact of open availability. Conversely, articles in high-impact journals do show a clear tendency for citation impact to increase. Mueller-Langer’s  (76) study showed no OACA, but the researchers concluded there would be significantly more citations to OA than non-OA ar- ticles, if they controlled for the quality of journals, articles, institutions and pre- prints citations. Twelve studies confirmed OA CA. Metcalfe (74) reported that articles post- 652 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo ed to repository gained more citations as non-OA articles. Ottaviani (78), who researched the OACA of the papers with embargos, found that citation advantages increase by 19 % when articles are made OA. The other ten studies supported an OACA, but they confirmed the early advantage and the self-selection effect (quality bias). Early advantage means the high citations achieved by OA pub- lications may attribute to the speed with which they become available to a world- wide audience  (17). Quality bias means that authors post their best articles that are more likely to be cited freely on the web. Ten studies confirm the phenom- enon of early advantage  (31,38,41,68- 73,77,81). Some of them found an ear- ly advantage for higher-cited arti- cles  (38,68,72,77). Gargouri  (68,77) examined the difference in OACA be- tween mandated and non-mandated ar- ticles and found no difference between them. Three studies confirmed quality bias  (68,70,81). Kurtz  (70) and Gargou- ri  (65) distinguished two dimensions for quality bias found that prominent authors may more often deposit their papers in arXiv and authors— be prom- inent or not— may tend to deposit their better papers in arXiv. Moed (81) inter- preted the quality bias as effect of more productive authors in terms of numbers of published papers. 2.1.1.4 Citation count of medical articles (n = 5) Five studies dealt with the citations of medical articles (citation counts of OA and non-OA articles within the journals and citation counts of self-archived ar- ticles). Four studies compared the cita- tion impact between OA and non-OA articles in the same journals, and one study investigated the citation impact of self-archived articles. Davis’s randomized controlled trial study (82) involved the sample of 11,013 articles from 11 journals. A large propor- tion of the articles where in the journal PNAS, which contributed nearly one- third of all articles in the dataset. Lan- singh  (83) compared 480 OA and 415 non-OA articles. Four subject areas were chosen to search the ophthalmology lit- erature in the PubMed database using the terms “cataract,” “diabetic retinopa- thy,” “glaucoma,” and “refractive errors.” Riera  (84) employed the sample of 161 articles from 1 journal from intensive care medicine. Lin  (44) compared the sample of 11 OA and 13 non-OA articles from 1 hybrid journal. Research made by De Groote (85) focused on the 45,716 ar- ticles from 122 journals posted to PMC repository. Results Results of Lasingh’s study  (83) showed no OACA, but the advantage correlated with number of authors, country of publication, language, sub- ject area and funding though not with access model. Study of Riera  (84) and Davis  (82) and editorial of Lin  (44) showed an increase in article citations. Riera  (84) found no significant differ- ence in raw number of citations be- tween all open access and non-open access articles, but showed a significant difference in number of citations be- tween the most highly cited open access and non-open access articles. Davis (82) employed randomized controlled trials and reported about a small, but signif- icant, 21 % increase in article citations. He explained that much of this citation increase can be explained by the influ- ence of one journal, PNAS. When this journal is removed from the analysis, the citation difference reduces by 7 %. Of the 11 journals only 2 show positive and significant open access effects. De the impact of open access on the medical literature: a review of current literature 653 PregLedni ZnanstVeni čLanek Groote’s study  (85) looked at a sample of articles published between 2006 and 2009 in 122 journals, separated into categories of NIH funded or non-NIH funded and whether they were depos- ited in PubMed Central. An analysis of the number of times these articles were cited found that NIH-funded 2006 arti- cles in PMC were not cited significant- ly more than NIH-funded non-PMC articles. The 2008 Public Access Policy which required mandatory deposit of articles is likely to a factor responsible for an increase in citations. The conse- quences were that 2009 NIH funded articles in PMC were cited 26 % more than 2009 NIH funded articles not in PMC, 5 years after publication. 2.2 Studies on the investigations of downloads and citations of the articles (n = 9) A complementary method for study- ing the advantages of OA is an analysis of article downloads. Downloads is the proxy for articles readership and usage. Measuring downloads means measur- ing the abstract downloads, full text (HTML) downloads, PDF downloads, and measuring of unique visitors to an article (86). An important research ques- tion is whether there is a correlation between downloads and the number of citations of OA articles. Nine studies compared the OA ar- ticles downloads and citations. Three studies focused on multidisciplinary sciences. Three studies examined the ar- ticles from 1 journal only. Three studies dealt with citation and download impact of self-archived articles. The samples in- cluded different size of articles. Two randomized controlled stud- ies focused on the comparison between OA citation and download advantage of the articles from the multidisciplinary sciences, including medicine. A Davis study  (87) sample of 3,265 articles (712 articles randomly assigned OA articles and 2,553 subscription-access articles published in 36 journals) and compared article downloads and number of cita- tions between these OA articles and sub- scription-only controls in the same set of journals. Another Davis study (88) used a sample of 3,245 articles of which 712 are OA published in 36 journals, comparing article downloads 12 months after publi- cation and citations after 3 years between gold OA and non-OA/green OA articles. Three studies that looked at articles posted to various repositories compared the citation and download advantages of articles. The studies dealt with the arti- cles from arXiv, the subject repository of physics, mathematics, etc. and articles from RePec, the repository of the eco- nomics. Davis (86) examined 2,726 arti- cles in 4 math journals, 511 articles were deposited in arXiv, with the remaining 2,254 were not. Asif-ul Haque (89) exam- ined 23,165 articles from arXiv. Chu (90) studied the top 200 downloaded papers from RePEc. Four studies employed the compari- son between citation and download ad- vantages of specific journals: Tetrahedron Letters, Journal of vision, Decision Support System, Journalism and Mass Communi- cation Quarterly. Tetrahedron Letters is an organic chemistry journal available in Elsevier’s ScienceDirect platform. The sample included 1,190 papers  (81). Jour- nal of Vision is an OA biological jour- nal with 153 articles in the sample  (91). Decision Support System is a journal for contributions from decision theory, eco- nomics, econometrics, statistics, etc. The sample included 994 articles available in ScienceDirect  (92). The sample of Ha’s study (93) employed the articles from the journal, Journalism and Mass Communi- cation Quarterly, and included 99 articles. 654 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo Results Four studies compared citation and download advantages of the OA arti- cles from the multidisciplinary scienc- es and found no positive relationship between them  (87,88,94,95). Davis  (94) showed that articles deposited in the arXiv received significantly more cita- tions than non-deposited articles and a fewer downloads at the publisher’s web- site than non-deposited articles. Davis’s studies from 2010 and 2011 revealed that OA articles received significantly more downloads within the first year, yet were cited no more frequently, nor earlier, than non-OA articles within 3 years. Da- vis explains the articles downloads and citations measure two different dimen- sions of scientific knowledge transfer. Downloads measure the general interest in a particular new piece of knowledge while citations measure a different set of intentions from a group of partici- pants in order to generate new scientific knowledge (87,88,95). Five other studies showed the positive relation between citations and down- loads. Ha  (93) employed the sample of 99 articles from 1 journal and report- ed that downloaded articles influenced the boost of short-term citations only. Studies of the papers deposited in re- positories and for the papers from 1 dis- cipline indicated a stronger connection between citations and downloads. As- if-ul Haque (89) considers the positional effects on early download and correlate a variety of downloads with long-term citations. The results of O’Leary’s  (92) study show that a downloaded paper receives twice as many citations. Wat- son (91) found out citations and down- loads increased with article age in a char- acteristic way, but relative to downloads. Moed (81) discovered that more down- loads of citing documents led to more downloads of the cited article through the citation. An analysis of 1,190 papers in the journal during a time interval of 2 years after the publication date revealed that there is about one citation for every 100 downloads. 2.3 Medical articles (n = 10) Eleven studies compared the cita- tions and download advantages in the field of medical sciences. Four studies are longitudinal, and retrospectively examined citation and downloads and the effect of PubMed Central on arti- cles downloads. Their samples are very large. Randomized controlled trails were made for 13,223 articles (5,999 published into PMC, 7,224 control) from 14 bio- medical research journals in nutrition, experimental biology, physiology, and radiology  (96) and 3,499 articles (1,886 treatment articles, which were deposit- ed and made freely available 12 months after publication and 1,613 articles avail- able to subscribers) in PMC published in 12 physiology journals  (97). The comparison was made between arti- cles downloaded from the journals’ site and the PubMed Central archive from 12 months through 24. Two other ran- domized controlled studies examined a sample of 1,619 articles from 11 physiol- ogy journals and compared downloads and citations between articles made OA immediately on publication (randomly picked) and those made OA 12 months after publication in subscription-only journals  (86,95). One randomized con- trolled study examined a sample of 712 OA articles and 2,533 control articles that were accessible by subscription (87). The other six studies were observa- tional. Two studies compared the cita- tions and downloads within oncology journals. Nieder (98) made a correlation between article download and citation from 250 most viewed articles between 5 the impact of open access on the medical literature: a review of current literature 655 PregLedni ZnanstVeni čLanek BioMed Central (BMC) open access on- cology journals. Four studies examined the articles from ScienceDirect journals. Schlögl  (99) examined 4,164,432 papers from 50 oncology journals; one study examined journals from pharmacolo- gy (100); and one study looked at 1,636 articles (935 subscription and 701 OA ar- ticles) from one pediatric journal  (101). Nicholas  (102) investigated the Nucleic Acids Research, which employed au- thor-pay publishing model. Results Seven articles (60 %) showed no posi- tive relation between citations and down- loads  (86,87,95-98,101). Davis  (86-87,95) reported OA articles received significantly more downloads and were not more likely to be cited than subscription-only articles. The studies from 2012 and 2013 found the reduction in article downloads from the journals’ websites when U.S. National In- stitutes of Health-sponsored articles be- come freely available from the PubMed Central repository. There is evidence to suggest that the effect of PMC is growing over time (96,97). In generally these stud- ies found OA articles were downloaded significantly more frequently, but found no evidence of an OACA. Davis conclud- ed the citation advantage from open ac- cess reported widely in the literature may be an artefact of other causes. Nieder (98), whose analysis focused on highly accessed articles published in 5 arbitrarily selected open access oncology journals, concluded download is not a universal surrogate for citations. Anderson  (101) found out that OA articles received 7 times more cita- tions per article than print articles. Three other studies  (67,99-100) showed a close relation between ci- tation and download frequencies. Schlögl  (99,100) revealed a moderate correlation between full-text article re- quests and article citations. Most of the articles are downloaded immediate- ly after they were put online. In many cases they reach their download maxi- mum even before they appear in print. Xue  (67) also found a positive correla- tion between downloads and citations frequencies, i.e., higher downloads fre- quencies is linked to higher citations frequencies. The peak time of citations frequencies comes relatively late, in the seventh to eighth year after being pub- lished, while the downloads frequency peaks quickly in the second year after being published. 2.4 Studies on the investigations of correlations between traditional metrics and social impact (n = 10) For as long as research outputs have been evaluated, researchers have thought how to investigate their impact. Tradi- tionally, citations from peer-reviewed articles and publishing in journals with high impact factors are generally accept- ed measures of scientific impact. The Web provides the possibility to construct innovative article-level or journal-lev- el metrics to gauge impact and influ- ence (103). One way to do so is by main- taining an online presence and by active involvement in social media. Social me- dia-based metrics or altmetrics may face attempts at manipulation similar to what Google must deal with in web search ranking. Metrics based on the social media could inform broader and faster measures of impact and complementing traditional citation metrics (104). Social attention may be tracked by mentions and shares of scientific papers across tra- ditional and social media outlets, blogs, public policy documents, post-publica- tion peer-review forums and online ref- erence managers. 656 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo Ten studies focused on the measuring of the social impact and public attention by social media. Five of them compared the citations and social impact from multidisciplinary fields while the others covered specific fields: biology, social sci- ence and humanity, information science, life sciences and astronomy, physics and math. The social media tools they in- vestigated were Facebook, Twitter, Goo- gle +, Blogs, Rabbit, Mendeley. Five authors  (105-109) compared ci- tations and altmetrics activities with downloads. The study in the field of physics employed 4,606 scientific arti- cles submitted to the preprint database arXiv.org  (105). Mathelus  (106) em- ployed 296 press-released articles from 99 Wiley-Blackwell journals and check out if the blogs increased the number of citations and downloads. Wang (107) investigated 1,761 papers published in Nature Communications used Twitter and Facebook. Mohamedi’s study  (108) from the social science and humanity employed 77,287 records and measured the social attention in Mendeley. Pe- ters’s (109) study focused on astronomy and math, employed 4,045 records and measured social attention using big alt- metrics data providers such as Impact- Story, Altmetric.com, and PlumX. Five other authors (110-114) analysed the relationship between altmetrics and citations. Alhoori (110) randomly select- ed 23 non-OA and hybrid OA journals from the top 100 journals from all fields. Costas (111) and Zahedi (112) investigated the altmetrics presence of publications from different science fields. Mohame- di’s (108) (study from the social science and humanity employed 77,287 records and measured the social attention with Mendeley. Peters’s  (109) study focused on astronomy and math employed 4,045 records and measured social at- tention using big altmetrics data pro- viders such as ImpactStory, Altmetric. com, and PlumX. Bar-Ilan (113) sampled 57 presenters from the library and in- formation science from the 2010 Leiden STI Conference, checking the attention generated in LinkedIn, Mendeley, Twit- ter, Google Scholar. Lin (114) compared 1613 papers in Nature and Science used CiteULike and Mendeley. Results Four studies  (105-107,110) pointed the correlations of citations, downloads and altmetrics between OA and non- OA articles. They confirmed Open Ac- cess Altmetric Advantage which means that OA articles received more altmet- rics than non-OA articles. Wang  (107) confirmed the OACA and download advantages. Alhoori’s (110) study report- ed a significant correlation between ci- tations and altmetrics for NOA and OA articles. Mathelus  (106) confirmed the promotion of scholarly journal articles to journalists and bloggers of press re- lease increased the number of citations and downloads. Seven studies investigated the correla- tion between altmetrics and downloads/ citations claimed positive correlations but relatively weak, thus supporting the idea that altmetrics do not reflect the same concept of impact as cita- tions  (105,108,109,111-114). Peters  (109), Bar-Ilan  (113) and Xuemei  (114) found the Mendeley as the most powerful tool for articles sampling. Shuai  (105) found that volume of Twitter mention is sta- tistically correlated with downloads and early citations. Costas  (111) showed that Twitter has a stronger focus on general medicine, psychology and social sciences as these disciplines have a higher density of tweets per publication than other dis- ciplines. the impact of open access on the medical literature: a review of current literature 657 PregLedni ZnanstVeni čLanek 2.5 Medical articles (n = 10) Ten articles compared the down- loads and/or citations and altmetrics scores. Seven studies investigated the relationship between altmetrics and citations. Two of them  (104,115) exam- ined the articles PLOS. Lin’s (115) study of 33,128 records investigated the cor- relation between citations and altmet- rics score, and Priem’s  (104) sample contained 24,331 papers that compared Mendeley mentions and citations. Two large samples included 1,4 million doc- uments (21,116), which were covered in PubMed and Web of Science (WoS). One of them investigated the relation- ship between tweeting and citations (21) and the other analysed how often arti- cles are mentioned on Twitter or saved by users on Mendeley  (116). Eysen- bach (103) employed a sample of 55 ar- ticles from the journal called Journal of Medical Internet Research in order to investigate if tweets can predict highly cited articles. Thelwall  (117) examined 208,739 PubMed articles with altmetrics tools (Twitter, Facebook, Google + ) and citations. Barbic’s (118) sample included 200 most frequently cited articles pub- lished in emergency medicine journals, conducted a cross-sectional analysis of citation classics and altmetrics scores from emergency medicine journal. Three studies investigated the cor- relation between download, citations and altmetrics with blogs, Twitter and Facebook. Allen  (119) took the sample of 16 articles from the field of clinical pain sciences. Tonia  (120) employed randomized controlled trials for articles published in the journal International Journal of Public Health. One study em- ployed the randomized controlled trial to investigate the relationship between downloads and altmetrics only for 243 articles from journal Circulation (121). Results Studies investigated the correlation between citation and altmetrics and showed different results. The results of Liu’s  (115) and Thelwall’s studies  (117) showed that the altmetrics are related to citation counts. Priem (104) and Bar- bic  (118) found a moderate correlation between altmetrics and citations. Ey- senbach  (103) found tweets can predict highly cited articles within the first 3 days of article publication. Haustein (21) found the number of Mendeley readers and tweets are two distinct social media metrics. The results of Haustein’s study showed that tweeting behaviour varies between journals and specialties and correlations between tweets and cita- tions are low (116). The studies investigating the relation- ship between downloads, citations and altmetrics found the dissemination of research through social media increased the number of downloads and also found no relationship between citation count and altmetrics  (119). Social me- dia attention had a significant effect on downloads and citations  (120). A study that researched the relationship between downloads and altmetrics reported that a social media strategy did not increase the downloads of the articles (121). 3 Discussions By looking into articles about OA im- pact on citations, downloads and altmet- rics, which were written over a period of 16 years, we can conclude that their ex- planations of the OA impact don’t pro- vide a definitive answer. Early studies were observational and confirmed OACA; generally they affirm an overall OACA variable by discipline, from 23 % OACA in political disci- pline  (56) to 307 % OACA in econom- ics (53). Thirty-seven (29 %) of the obser- 658 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo vational studies used the method based on citation analyses, and 27 of them simply confirmed a positive correlation between open access and citation counts. Analysed articles were mostly taken from high quality journals. There were only five studies focused on medical articles which mostly con- firmed OACA, whereas some others were doubtful about it  (44,82,84). Da- vis  (82) investigated the OACA using randomized controlled trials and con- cluded the OACA was influenced by one journal only, which contributed near- ly one-third of all articles. Riera  (84) showed a significant difference in the number of citations between the most highly cited open access and non-open access articles. The studies on the citations count were observational studies, in which the researchers did not manipulate or con- trol independent variables, but only ob- served the differences. De Groote  (85) investigated the OACA of deposited PMC articles employing the sample of the OA articles with Public Access Pol- icy, which required mandatory deposit of articles. Some other studies employed the samples of OA self-archived and subscribed articles and confirmed con- founding variables such as the quality bias  (31,38,41,68-74,77,81). Confounding variables were a methodological prob- lem in observational studies. The other problem is also the different size of sam- ples. Some studies made the research with 99 articles only, but the other with more than 20,000 articles (68,77). On the basis of evidence based med- icine involving a hierarchical system of classifying evidence the experimental studies were introduced with a high- er level of evidence. In the research of OACA, randomized controlled trials were selected as a type of methodology used to isolate the effect of the treatment under investigation. The results of these studies showed little evidence that open access status has an independent effect on citation counts but is significantly as- sociated with more downloads and more unique visitors (86,87,95-98,101). Social metrics, the new concept based on the idea that scientists and the public leave digital traces on the Internet when searching for or using information, pro- vide the opportunity to gather novel metrics from other sources that provide data in a structured format (103). In our review there are 10 articles dealing with social impact of medical articles. Many studies employed different social media tools and measured social metrics atten- tion with one or two tools. The investiga- tions of social media attentions and cita- tions/downloads showed the correlation between them. 4 Conclusions Our review of the literature focused on quantitative studies: bibliometrics and webometrics. We posed two questions at the be- ginning of this article: “Do open access articles increase the citations, downloads and social impact?” and “Is there any causal relation between them?”. Most of the observational studies showed a correlation between the OA articles and higher citation counts. However, these results were interpreted as causal and implied causality without due consider- ation of potential confounding factors. A certain amount of methodological defi- ciencies means that findings about the relationship between OA and citation counts were incomplete and therefore suitable for further research. However, the group of articles using experimental analysis has not confirmed any causal re- lationship and has actually shown a more complex set of confounding factors such the impact of open access on the medical literature: a review of current literature 659 PregLedni ZnanstVeni čLanek as a quality, early access, or self-selection bias which are the contributors to the ef- fect itself. Due to the fact that researchers have increasingly moved online, the traditional metrics are not able to follow non-tradi- tional web-based services such as tweets, likes, shares, bookmarks, views, down- loads and mentions. These outputs have been measured by alternative metrics or altmetrics which reflect the different concept of impact. The conclusions of alt- metrics researches in the field of medical articles are often diametrically opposed: • The altmetrics are related to citation counts  (115,117,119); there is a mod- erate correlation or a week positive correlation between them  (104,118); and altmetrics and citations measure different things. • The altmetrics are related to down- loads  (119-121). The dissemination of research through social media increased the number of down- load (119), but a social media strategy did not increase the downloads of the articles (120,121). Altmetrics, an alternative means to assess the research impact, is still in its infancy. One of the challenges is to pro- vide a reliable, relevant and standard- ized way for measuring research impact. Within medicine, the social media us- age has been growing very rapidly (122). Future attention should be paid to the connections between citations activities and mentions on social media in order to better understand whether social me- dia may increase the impact of articles or articles may cause more social media attention. In view of current researches who are not able to explain these con- nections through the formal network, it is recommended to use additional meth- ods as are authors’ interviews examining researchers’ opinions and perceptions of the OA advantages. These methods may contribute to the understanding of the articles transmission through the in- formal network, i.e. outside the core re- search community. Acknowledgment I wish to express my special appreci- ation and sincere thanks to Mr. A. Ben Wagner from the University of Buffalo for his guidance and valuable advice on completing this work. References 1. What is OA? [cited 2016 29.3.]. Avail- able from: http://guides.zsr.wfu.edu/c. php?g=34486&p=220933. 2. Solomon DJ. Digital Distribution of Academic Journals and its Impact on Scholarly Communica- tion: Looking Back After 20Years. The Journal of Academic Librarianship. 2013; 39(1): 23–8. 3. Becher T, Trowler P. Academic tribes and territo- ries: Intellectual enquiry and the culture of disci- plines: McGraw-Hill Education (UK); 2001. 4. Donato H. Traditional and alternative metrics: The full story of impact. Revista Portuguesa de Cardiologia. 2014; 20(01): 1–2. 5. PMC–NCBI Bethesda: US National Library of Medicine; [cited 2016 15.07.]. Available from: https://www.ncbi.nlm.nih.gov/pmc/. 6. Europe PMC Hinxton: European Bioinformat- ics Institute [cited 2016 25.07.]. Available from: https://europepmc.org/. 7. BioMed Central: The Open Access Publisher Lon- don: BioMed Central; [cited 2016 07.05.]. Available from: https://www.biomedcentral.com/. 8. Chan L, Cuplinskas D, Eisen M, Friend F, Genova Y, Guédon J-C, et al. Budapest open access initia- tive. 2002. 9. Brown JK, Byers T, Doyle C, Courneya KS, De- mark-Wahnefried W, Kushi LH, et al. Nutrition and physical activity during and after cancer treatment: An American Cancer Society Guide for informed choices. Ca-A Cancer Journal for Clini- cians. 2003; 53(5): 268–91. 10. Berlin declaration on open access to knowledge in the sciences and humanities 2003 [Available from: http://www.fu-berlin.de/sites/open_access/ Veranstaltungen/oa_berlin/poster/Berlin-Decla- ration_Simone-Rieger_MPIWG.pdf. 11. Directory of Open Access Journals [cited 2016 13.07.2016]. Available from: https://doaj.org/. 660 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo 12. PLOS Public Library of Science [Available from: https://www.plos.org/. 13. Roemer RB BR. Keeping Up With... Altmetrics Chicago: Association of Colleg & Research Li- brary; 2010 [22.3.2016]. Available from: http:// www.ala.org/acrl/publications/keeping_up_with/ altmetrics. 14. Cat C. Altmetric Blog [Internet]2015. [cited 2016]. 15. Bidyarthi D. The journey from librametry to alt- metrics: a look back. Golden Jubilee Celebration of Department of Library and Information Science, Jadavpur University; Kolkata: Jadavpur Universi- ty; 2014. 16. Björk B-C, Solomon D. Open access versus sub- scription journals: a comparison of scientific im- pact. BMC Medicine. 2012; 10(1): 1–10. 17. Swan A. The Open Access citation advantage: Studies and results to date. 2010. 18. Lawrence S. Free online availability substantially increases a paper’s impact. Nature. 2001; 411(6837): 521. 19. Norris M, Oppenheim C, Rowland F. The citation advantage of open-access articles. Journal of the American Society for Information Science and Technology. 2008; 59(12): 1963–72. 20. Hibrid open access journals [ 21. Haustein S, Larivière V, Thelwall M, Amyot D, Pe- ters I. Tweets vs. Mendeley readers: How do these two social media metrics differ? IT-Information Technology. 2014; 56(5): 207–15. 22. Donovan JM, Watson CA. Citation advantage of open access legal scholarship. Law Libr J. 2011; 103: 553. 23. The Open Access Citation Advantage Service [cit- ed 2016 17.06]. Available from: http://sparceurope. org/oaca/. 24. Wagner B. Open access citation advantage: An an- notated bibliography. Issues in Science and Tech- nology Librarianship. 2010(60): 2. 25. Hitchcock S. The effect of open access and down- loads (‘hits’) on citation impact: a bibliography of studies. 2004. 26. Sotudeh H, Horri A. The citation performance of open access journals: A disciplinary investigation of citation distribution models. Journal of the American Society for Information Science and Technology. 2007; 58(13): 2145–56. 27. Zhang Y. The effect of open access on citation im- pact: a comparison study based on web citation analysis. Libri. 2006; 56(3): 145–56. 28. Cheng W, Ren S. Evolution of open access publish- ing in Chinese scientific journals. Learned Pub- lishing. 2008; 21(2): 140–52. 29. Evans JA, Reimer J. Open access and global partic- ipation in science. Science. 2009; 323(5917): 1025-. 30. M N. The citation advantage of open access arti- cles. PhD thesis. Leicestershire: Loughborough University; 2009. Available from: http://hdl.han- dle.net/2134/4089. 31. Gentil-Beccot A, Mele S, Brooks T. Citing and reading behaviours in high-energy physics: How a community stopped worrying about journals and learned to love repositories. Retrieved October 2009. 2009. 32. Zawacki-Richter O, Anderson T, Tuncay N. The growing impact of open access distance education journals: A bibliometric analysis. Internation- al Journal of E-Learning & Distance Education. 2010; 24(3). 33. Xia J, Nakanishi K. Self-selection and the citation advantage of open access articles. Online Informa- tion Review. 2012; 36(1): 40–51. 34. McCabe M, Snyder CM. Identifying the effect of open access on citations using a panel of science journals. Economic Inquiry. 2014; 52(4): 1284–300. 35. Frisch NK, Nathan R, Ahmed YK, Shidham VB. Authors attain comparable or slightly higher rates of citation publishing in an open access journal (CytoJournal) compared to traditional cytopa- thology journals-A five year (2007–2011) experi- ence. CytoJournal. 2014; 11(1): 10. 36. McCabe MJ, Snyder CM. Does online availabili- ty increase citations? Theory and evidence from a panel of economics and business journals. Review of Economics and Statistics. 2015; 97(1): 144–65. 37. Sahu DK, Gogtay NJ, Bavdekar SB. Effect of open access on citation rates for a small biomedical journal. 5th International Congress on Peer Re- view and Biomedical Publication; 16‐18 September 2005; Chicago, USA2005. 38. Kurtz MJ, Henneken EA. Open Access does not increase citations for research articles from The Astrophysical Journal. arXiv preprint arXiv: 07090896. 2007. 39. Soong SC. Measuring citation advantages of open accessibility. D-Lib Magazine. 2009; 15. 40. Wu LL, Huang MH, Chen CY. Citation patterns of the pre-web and web-prevalent environments: The moderating effects of domain knowledge. Journal of the American Society for Information Science and Technology. 2012; 63(11): 2182–94. 41. Kim HH. The effect of free access on the diffusion of scholarly ideas. 2012. 42. Shin E-J. Do Impact Factors change with a change of medium? A comparison of Impact Factors when publication is by paper and through parallel publishing. Journal of Information Science. 2003; 29(6): 527–33. 43. McVeigh ME. Open access journals: in the ISI citation databases: analysis of impact factors and citation patterns. 2005. 44. Lin S-K. Non-open access and its adverse impact on molecules. Molecules. 2007; 12(7): 1436–7. 45. Lin S-K. Full open access journals have increased impact factors. Molecules. 2009; 14(6): 2254–5. 46. Mahesh G. Open access and impact factors. Cur- rent Science. 2012; 103(6). 47. Gorraiz J, Gumpenberger C, Schloegl C, Wieland M, editors. On the temporal stability of Garfield’s impact factor and its suitability to identify hot pa- pers2012. 48. Laakso M, Björk BC. Delayed open access: An overlooked high-impact category of openly avail- able scientific literature. Journal of the American Society for Information Science and Technology. 2013; 64(7): 1323–9. 49. Gumpenberger C, Ovalle-Perandones M-A, Gor- raiz J. On the impact of Gold Open Access jour- nals. Scientometrics. 2013; 96(1): 221–38. 50. Sabharwal S, Patel N, Johal K. Open access pub- lishing: a study of current practices in orthopaedic the impact of open access on the medical literature: a review of current literature 661 PregLedni ZnanstVeni čLanek research. International orthopaedics. 2014; 38(6): 1297–302. 51. Koler-Povh T, Južnič P, Turk G. Impact of Open Access on citation of scholarly publications in the field of civil engineering a. Scientometrics, 98(2): 1033–1045. 52. Donovan JM, Watson CA, Osborne C. The open access advantage for American law reviews. Avail- able at SSRN 2506913. 2014. 53. Wohlrabe K, Birkmeier D. Do open access arti- cles in economics have a citation advantage? No. 56842.: University Library of Munich, Germany; 2014. 54. Xia J, Myers RL, Wilhoite SK. Multiple open access availability and citation impact. Journal of Infor- mation Science. 2010: 0165551510389358. 55. Eysenbach G. Citation advantage of open access articles. PLoS Biol. 2006; 4(5):e157. 56. Atchison A, Bull J. Will open access get me cited? An analysis of the efficacy of open access publish- ing in political science. PS: Political Science & Pol- itics. 2015; 48(01): 129–37. 57. Koler-Povh T, Turk G, Južnič P. Does the Open Access business model have a significant impact on the citation of publications? Case study in the field of civil engineering. Proceedings of the Fifth Belgrade International Open Access Conference; 2012 May 18–19; Belgrad, Serbia[cited 2016 25.07.]. Available from: http://boac.ceon.rs/index.php/ BOAC/12/paper/view/68/8. 58. Xia J. Positioning Open Access Journals in a LIS Journal Ranking. College & Research Libraries. 2012; 73(2): 134–45. 59. Gaule P, Maystre N. Getting cited: does open ac- cess help? Research Policy. 2011; 40(10): 1332–8. 60. Gaulé P, Maystre N. Getting cited: Does open ac- cess help? Retrieved October 2009. 2008. 61. Ingwersen P, Elleby AB, editors. Do open access working papers attract more citations compared to printed journal articles from the same research unit?2011. 62. Tonta Y, Ünal Y, Al U. The research impact of open access journal articles. 2007. 63. Antelman K. Do open-access articles have a great- er research impact? College & research libraries. 2004; 65(5): 372–82. 64. Sotudeh H, Ghasempour Z, Yaghtin M. The cita- tion advantage of author-pays model: the case of Springer and Elsevier OA journals. Scientomet- rics. 2015; 104(2): 581–608. 65. Archambault É, Amyot D, Deschamps P, Nicol A, Provencher F, Rebout L, et al. Proportion of open access papers published in peer-reviewed journals at the European and world levels—1996–2013. 2014. 66. Hajjem C, Harnad S, Gingras Y. Ten-year cross-dis- ciplinary comparison of the growth of open access and how it increases research citation impact. arX- iv preprint cs/0606079. 2006. 67. Xue-li L, Hong-ling F, Mei-ying W. Correlation between download and citation and download-ci- tation deviation phenomenon for some papers in Chinese medical journals. Serials Review. 2011; 37(3): 157–61. 68. Gargouri Y, Hajjem C, Larivière V, Gingras Y, Carr L, Brody T, et al. Self-selected or mandated, open access increases citation impact for higher quality research. PloS one. 2010; 5(10):e13636. 69. Schwarz GJ, Kennicutt Jr RC. Demographic and citation trends in astrophysical journal papers and preprints. arXiv preprint astro-ph/0411275. 2004. 70. Kurtz MJ, Eichhorn G, Accomazzi A, Grant C, Demleitner M, Henneken E, et al. The effect of use and access on citations. Information Processing & Management. 2005; 41(6): 1395–402. 71. Aman V. The potential of preprints to accelerate scholarly communication-A bibliometric analysis based on selected journals. arXiv preprint arXiv: 13064856. 2013. 72. Metcalfe TS. The rise and citation impact of astro-ph in major journals. arXiv preprint as- tro-ph/0503519. 2005. 73. Henneken EA, Kurtz MJ, Eichhorn G, Accomazzi A, Grant C, Thompson D, et al. Effect of e-printing on citation rates in astronomy and physics. arXiv preprint cs/0604061. 2006. 74. Metcalfe TS. The citation impact of digital pre- print archives for solar physics papers. Solar Phys- ics. 2006; 239(1–2): 549–53. 75. Frandsen TF. The integration of open access journals in the scholarly communication system: Three science fields. Information Processing & Management. 2009; 45(1): 131–41. 76. Mueller-Langer F, Watt R. The Hybrid Open Ac- cess Citation Advantage: How Many More Cites is a $3,000 Fee Buying You? Max Planck Institute for Innovation & Competition Research Paper. 2014(14–02). 77. Gargouri YHS. Logistic regression of potential explanatory variables on citation counts. Preprint 11/04/2009. 2009. 78. Ottaviani J. The post-embargo open access cita- tion advantage: it exists (probably), its modest (usually), and the rich get richer (of course). PLoS One. 2016; 11(8):e0159614. 79. Kullman L, editor The Effect of Open Access on Citation Rates of Self-archived Articles at Chalm- ers2014. 80. Snijder R. The profits of free books: an experiment to measure the impact of open access publishing. Learned Publishing. 2010; 23(4): 293–301. 81. Moed HF. The effect of “open access” on citation impact: An analysis of ArXiv’s condensed matter section. Journal of the American Society for In- formation Science and Technology. 2006; 58(13): 2047–54. 82. Davis PM. Author-choice open-access publishing in the biological and medical literature: A citation analysis. Journal of the American Society for In- formation Science and Technology. 2009; 60(1): 3–8. 83. Lansingh VC, Carter MJ. Does open access in oph- thalmology affect how articles are subsequently cited in research? Ophthalmology. 2009; 116(8): 1425–31. 84. Riera M, Aibar E. ¿ Favorece la publicación en abierto el impacto de los artículos científicos? Un estudio empírico en el ámbito de la medicina in- tensiva. Medicina intensiva. 2013; 37(4): 232–40. 85. De Groote SL, Shultz M, Smalheiser NR. Examin- ing the Impact of the National Institutes of Health 662 Zdrav Vestn | november – december 2016 | Letnik JaVno ZdraVstVo Public Access Policy on the Citation Rates of Jour- nal Articles. PloS one. 2015; 10(10):e0139951. 86. Davis PM, Lewenstein BV, Simon DH, Booth JG, Connolly MJ. Open access publishing, article downloads, and citations: randomised controlled trial. BMJ (Clinical research ed). 2008; 337:a568. 87. Davis P. Access, readership, citations: A random- ized controlled trial of scientific journal publish- ing. Ithaca: Cornell University Graduate School; 2010. 88. Davis PM. Open access, readership, citations: a randomized controlled trial of scientific jour- nal publishing. The FASEB Journal. 2011; 25(7): 2129–34. 89. Haque Au, Ginsparg P. Positional effects on ci- tation and readership in arXiv. Journal of the American Society for Information Science and Technology. 2009; 60(11): 2203–18. 90. Chu H, Krichel T. Downloads vs. citations: rela- tionships, contributing factors and beyond. 2007. 91. Watson AB. Comparing citations and downloads for individual articles at the Journal of Vision. Journal of Vision. 2009; 9(4):i-i. 92. O’Leary DE. The relationship between citations and number of downloads in Decision Support Systems. Decision Support Systems. 2008; 45(4): 972–80. 93. Ha L, Jiang W, Bi C, Zhang R, Zhang T, Wen X. How online usage of subscription-based journal- ism and mass communication research journal articles predicts citations. Learned Publishing. 2016; 29(3): 183–92. 94. Davis P, Fromerth M. Does the arXiv lead to high- er citations and reduced publisher downloads for mathematics articles? Scientometrics. 2007; 71(2): 203–15. 95. Davis PM. Does open access lead to increased readership and citations? A randomized con- trolled trial of articles published in APS journals. The Physiologist. 2010; 53(6). 96. Davis PM. Public accessibility of biomedical arti- cles from PubMed Central reduces journal read- ership—retrospective cohort analysis. The FASEB Journal. 2013:fj-13. 97. Davis PM. The effect of public deposit of scientif- ic articles on readership. The Physiologist. 2012; 55(5): 161–3. 98. Nieder C, Dalhaug A, Aandahl G. Correlation between article download and citation figures for highly accessed articles from five open access on- cology journals. SpringerPlus. 2013; 2(1): 1–5. 99. Schloegl C, Gorraiz J. Comparison of citation and usage indicators: the case of oncology journals. Scientometrics. 2010; 82(3): 567–80. 100. Schloegl C, Gorraiz J. Global usage versus glob- al citation metrics: The case of pharmacology journals. Journal of the American Society for Information Science and Technology. 2011; 62(1): 161–70. 101. Anderson K, Sack J, Krauss L, O’Keefe L. Publish- ing online-only peer-reviewed biomedical liter- ature: Three years of citation, author perception, and usage experience. Journal of Electronic Pub- lishing. 2001; 6(3). 102. Nicholas D, Huntington P, Jamali HR. The impact of open access publishing (and other access ini- tiatives) on use and users of digital scholarly jour- nals. Learned publishing. 2007; 20(1): 11–5. 103. Eysenbach G. Can tweets predict citations? Met- rics of social impact based on Twitter and cor- relation with traditional metrics of scientific im- pact. Journal of medical Internet research. 2011; 13(4):e123. 104. Priem J, Piwowar HA, Hemminger BM. Altmet- rics in the wild: Using social media to explore scholarly impact. arXiv preprint arXiv: 12034745. 2012. 105. Shuai X, Pepe A, Bollen J. How the scientific com- munity reacts to newly submitted preprints: Ar- ticle downloads, twitter mentions, and citations. PloS one. 2012; 7(11):e47523. 106. Mathelus S, Pittman G, Yablonski-Crepeau J. Pro- motion of research articles to the lay press: a sum- mary of a three-year project. Learned Publishing. 2012; 25(3): 207–12. 107. Wang X, Liu C, Mao W, Fang Z. The open access advantage considering citation, article usage and social media attention. scientometrics. 2015; 103(2): 555–64. 108. Mohammadi E, Thelwall M, Haustein S, Lariv- ière V. Who reads research articles? An altmet- rics analysis of Mendeley user categories. Journal of the Association for Information Science and Technology. 2015; 66(9): 1832–46. 109. Peters I, Kraker P, Lex E, Gumpenberger C, Gor- raiz J. Research data explored: an extended anal- ysis of citations and altmetrics. Scientometrics. 2016; 107(2): 723–44. 110. Alhoori H, Ray Choudhury S, Kanan T, Fox E, Furuta R, Giles CL. On the Relationship between Open Access and Altmetrics. iConference 2015 Proceedings. 2015. 111. Costas R, Zahedi Z, Wouters P. Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a mul- tidisciplinary perspective. Journal of the Associ- ation for Information Science and Technology. 2015; 66(10): 2003–19. 112. Zahedi Z, Costas R, Wouters P. How well devel- oped are altmetrics? A cross-disciplinary analy- sis of the presence of ‘alternative metrics’ in sci- entific publications. Scientometrics. 2014; 101(2): 1491–513. 113. Bar-Ilan J, Haustein S, Peters I, Priem J, Shema H, Terliesner J. Beyond citations: Scholars’ visibility on the social Web. arXiv preprint arXiv: 12055611. 2012. 114. Xuemei L, Thelwall M. F1000, Mendeley and tra- ditional bibliometric indicators. Proceedings of the 17th international conference on science and technology indicators. 2012; 2: 451–551. 115. Liu J, Adie E. Five challenges in altmetrics: A tool- maker’s perspective. Bulletin of the American Society for Information Science and Technology. 2013; 39(4): 31–4. 116. Haustein S, Peters I, Sugimoto CR, Thelwall M, Larivière V. Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Sci- ence and Technology. 2014; 65(4): 656–69. the impact of open access on the medical literature: a review of current literature 663 PregLedni ZnanstVeni čLanek 117. Thelwall M, Haustein S, Larivière V, Sugimoto CR. Do altmetrics work? Twitter and ten other social web services. PloS one. 2013; 8(5):e64841. 118. Barbic D, Tubman M, Lam H, Barbic S. An anal- ysis of altmetrics in Emergency Medicine. Aca- demic Emergency Medicine. 2016. 119. Allen HG, Stanton TR, Di Pietro F, Moseley GL. Social media release increases dissemination of original articles in the clinical pain sciences. PloS one. 2013; 8(7):e68914. 120. Tonia T, Van Oyen H, Berger A, Schindler C, Kün- zli N. If I tweet will you cite? The effect of social media exposure of articles on downloads and citations. International journal of public health. 2016: 1–8. 121. Fox CS, Bonaca MA, Ryan JJ, Massaro JM, Barry K, Loscalzo J. A randomized trial of social media from Circulation. Circulation. 2015; 131(1): 28–33. 122. George DR, Rovniak LS, Kraschnewski JL. Dan- gers and opportunities for social media in med- icine. Clinical obstetrics and gynecology. 2013; 56(3).