Publication Bias in medical research: issues and communities Edgar Schiebel and Maria Elisabeth Züger edgar.schiebel@ait.ac.at, maria-elisabeth.zueger@ait.ac.at AIT Austrian Institute of Technology GmbH, Donau City Str. 1, A-1220 Vienna (Austria) Paper presented at the 14th International Society of Scientometrics and Informetrics Conference, Vienna 2013, July 18th Results from: UNCOVER - Evaluation and development of measures to uncover and overcome bias due to non-publication of clinical trials; FP7-HEALTH-2011- European Commision
Introduction Context: UNCOVER project: Evaluation and development of measures to uncover and overcome bias due to non-publication of clinical trials Reporting the outcome of clinical trials: Authors tend not to publish negative results and we see an inflation of publishing positive results on the other side this effect is called publication bias We frame the problem in terms of evidence-based medicine combined with system s theory. Firstly identify and map issues on publication bias and relevant stakeholders and opinion leaders like researchers, editors of journals, funding bodies, regulators, industry and ethic commissions on an international and global level, and secondly perform participatory workshops to involve the community and discuss possible measures (law, regulations, policies, practices, guidelines, methods, tools) to overcome bias. 2
Research Questions This work discusses the research questions and the results of the first part of the UNCOVER project Do we have a scientific discussion about publication bias in science and what are the concerned disciplines Are scientists aware and do they mention and examine non-publication of negative results that also plays a role for the quality of published work and have a consequence in practical use of research results? What are the research communities? We present a literature survey performed by relational bibliometric approaches to identify research groups, issues on publication bias. 3
Method and Data Search strategy: publication bias, citation bias, language bias, location bias, reference bias, and reporting bias search in ISI Web of Knowledge. Result: 3,891 publications over the time span 1990 to 2012 (partly), The bibliometric analysis was conducted on co-authorships and bibliographic coupling of the publications. Relationships between authors, instiutions were mapped and examined Research issues were identified by applying heat maps of bibliographic coupled publications. 4
List of the first 21 sources sorted by descendant number of publications Source titles Number of Publ. % of 3,891 1. COCHRANE DATABASE OF SYSTEMATIC REVIEWS 101 2.60 2. BRITISH MEDICAL JOURNAL 83 2.13 3. JOURNAL OF CLINICAL EPIDEMIOLOGY 78 2.01 4. JAMA JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION 7 1.47. ANNALS OF INTERNAL MEDICINE 3 1.36 6. AMERICAN JOURNAL OF EPIDEMIOLOGY 41 1.0 7. LANCET 36 0.93 8. STATISTICS IN MEDICINE 3 0.90 9. PLOS ONE 34 0.87 10. INTERNATIONAL JOURNAL OF EPIDEMIOLOGY 24 0.62 11. EPIDEMIOLOGY 23 0.9 12. AMERICAN JOURNAL OF CLINICAL NUTRITION 20 0.1 13. PLOS MEDICINE 20 0.1 14. STROKE 20 0.1 1. AMERICAN JOURNAL OF GASTROENTEROLOGY 19 0.49 16. MOLECULAR BIOLOGY REPORTS 19 0.49 17. ARCHIVES OF INTERNAL MEDICINE 17 0.44 18. CANADIAN MEDICAL ASSOCIATION JOURNAL 17 0.44 19. CURRENT MEDICAL RESEARCH AND OPINION 17 0.44 20. EUROPEAN JOURNAL OF CANCER 16 0.41 21. JOURNAL OF CLINICAL ONCOLOGY 16 0.41
Publication bias over time Publication Year Number of Publ. % of 3,891 Trend line 1990 4 0.10 1991 26 0.67 1992 42 1.08 1993 43 1.11 1994 2 1.34 199 2 1.34 1996 6 1.67 1997 78 2.01 1998 93 2.39 1999 108 2.78 2000 98 2.2 2001 10 2.70 2002 119 3.06 2003 124 3.19 2004 163 4.19 200 218.60 2006 244 6.27 2007 276 7.09 2008 33 9.07 2009 400 10.28 2010 422 10.8 2011 31 13.6 2012 (January-May) 27 7.07 6
Research communities on publication bias, (network of co-authorships) Dominating Authors Dominating authors Author # Pub 7
Agglomerations of research issues on publication bias (local density of bibliographically coupled publications weighted by similarities) Two dominating research issues: Meta studies on clinical topics, citing guidelines and methods Research about publication bias: guidelines and methods, etc. 8
Meta studies about clinical topics - Authors Index Author Author network Author sub-networks Publications in the two clusters Meta studies about clinical Size of the sub-network Highest cited publication topics Times cited (diamete r) No. of papers 1 Bagos, PG 11 12 2 Nikolopoulo s, GK 14 9 Subnetwork Bonovas, S Bonovas, S 3 Qin, LQ 6 9 Qin, LQ 4 Wang, J 10 12 Sutton, AJ 48 33 6 Dong, JY 1 7 Qin, LQ 7 Xu, MQ 7 8 Mengoli, C 38 9 Cruciani, M 38 10 Abrams, KR 46 22 Cruciani, M Cruciani, M 8 8 2 2 Association between the plasminogen activator inhibitor-i 4G/G polymorphism and venous thrombosis - A meta-analysis Association between the plasminogen activator inhibitor-i 4G/G polymorphism and venous thrombosis - A meta-analysis Milk consumption is a risk factor for prostate cancer in Western countries: evidence from cohort studies UCHL1 is a Parkinson's disease susceptibility gene Empirical assessment of effect of publication bias on meta-analyses Erectile Dysfunction and Risk of Cardiovascular Disease Meta-Analysis of Prospective Cohort Studies Quantitative assessment of the effect of angiotensinogen gene Polymorphisms on the risk of coronary heart disease Use of PCR for diagnosis of invasive aspergillosis: systematic review and meta-analysis Use of PCR for diagnosis of invasive aspergillosis: systematic review and meta-analysis A systematic review of molecular and biological tumor markers in neuroblastoma 10 8 8 7 Methodologies and guidelines for clinical studies 7 1 Highest ranked authors by the number of publications in the cluster, including author network information and title of the highest cited publication in this cluster. Due to co-publications, different authors can have the same publication listed. Times cited as from June 2012. 7 6 9
Methodologies and guidelines for clinical studies - Authors Index Author Author network Author sub-networks Publications in the two clusters Times cited (diamet er) No. of papers 1 Moher, D 128 33 2 Dickersin, K 178 18 3 Decullier, E 44 4 Tannock, IF 30 8 78 47 6 Rennie, D 217 14 7 Smith, GD 41 1 8 Egger, M 344 19 9 Cook, DJ 361 9 10 Bero, LA 133 8 Subnetwork Tannock, IF size of the sub-network 2 Highest cited publication Does the inclusion of grey literature influence estimates of intervention effectiveness reported in metaanalyses? Systematic reviews - identifying relevant studies for systematic reviews Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias Factors associated with failure to publish large randomized trials presented at an oncology meeting Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias Publication bias in editorial decision making Sifting the evidence - what's wrong with significance tests? Meta studies about clinical topics Methodologi es and guidelines for clinical studies 9 8 3 Uses and abuses of meta-analysis 4 Users guides to the medical literature.6. how to use an overview Pharmaceutical industry sponsorship and research outcome and quality: systematic review 4 4 4 4 Highest ranked authors by the number of publications in the cluster, including author network information and title of the highest cited publication in this cluster. Due to co-publications, different authors can have the same publication listed. Times cited as from June 2012. 10
Summary and Discussion The aim of this work was to identify key researchers about publication bias and research activities in the field of publication bias, A small number of publications about publication bias dates back almost two decades, to the year 1990. remarkable increase in the number of publications in the last years Research issues are: outcome reporting, registration of trials, ethic issues, role of editors, guidelines for performing clinical trials increase of the number of systematic reviews on different medical topics. It reflects the growing research activities in evidence based medicine, awareness and methods for meta-analysis and systematic reviews. headed by the United States, where we have the highest publication activity, England is leading the statistics of European countries. 11
Summary and Discussion The author network showed a high level of co-publishing in the field of publication bias. network positions and additional information like affiliation, country and research interest, were used to optimize the selection of stakeholders for up-coming interviews and workshops in the UNCOVER project. The map of bibliographically coupled publications showed two clusters: The bigger one is formed by publications about performed systematic reviews and meta-analysis about different medical subjects like myocardial infarction, blood preasure or diabetis mellitus. We called it meta-studies about clinical topics. It includes systematic reviews and meta-analyzes. The second represents publications about guidelines for clinical trials, mathematic and statistical methods for meta-analysis, registration of studies, reporting and research about different issues related to publication bias. The assigned name was methodologies and guidelines for clinical studies. 12
Thank you for your attention edgar.schiebel@ait.ac.at 13