John P.A. Ioannidis

All Publications

• Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review. The Lancet. Digital health Han, R., Acosta, J. N., Shakeri, Z., Ioannidis, J. P., Topol, E. J., Rajpurkar, P. 2024; 6 (5): e367-e373

  Abstract

  This scoping review of randomised controlled trials on artificial intelligence (AI) in clinical practice reveals an expanding interest in AI across clinical specialties and locations. The USA and China are leading in the number of trials, with a focus on deep learning systems for medical imaging, particularly in gastroenterology and radiology. A majority of trials (70 [81%] of 86) report positive primary endpoints, primarily related to diagnostic yield or performance; however, the predominance of single-centre trials, little demographic reporting, and varying reports of operational efficiency raise concerns about the generalisability and practicality of these results. Despite the promising outcomes, considering the likelihood of publication bias and the need for more comprehensive research including multicentre trials, diverse outcome measures, and improved reporting standards is crucial. Future AI trials should prioritise patient-relevant outcomes to fully understand AI's true effects and limitations in health care.

  View details for DOI 10.1016/S2589-7500(24)00047-5

  View details for PubMedID 38670745

• Best Practices for Data Management and Sharing in Experimental Biomedical Research. Physiological reviews Cunha-Oliveira, T., Ioannidis, J. P., Oliveira, P. J. 2024

  Abstract

  Effective data management is crucial for scientific integrity and reproducibility, a cornerstone of scientific progress. Well-organized and well-documented data enable validation and building upon results. Data management encompasses activities including organization, documentation, storage, sharing, and preservation. Robust data management establishes credibility, fostering trust within the scientific community and benefiting researchers' careers. In experimental biomedicine, comprehensive data management is vital due to the typically intricate protocols, extensive metadata, and large datasets. Low-throughput experiments, in particular, require careful management to address variations and errors in protocols and raw data quality. Transparent and accountable research practices rely on accurate documentation of procedures, data collection, and analysis methods. Proper data management ensures long-term preservation and accessibility of valuable datasets. Well-managed data can be revisited, contributing to cumulative knowledge and potential new discoveries. Publicly funded research has an added responsibility for transparency, resource allocation, and avoiding redundancy. Meeting funding agency expectations increasingly requires rigorous methodologies, adherence to standards, comprehensive documentation, and widespread sharing of data, code, and other auxiliary resources. This review provides critical insights into raw and processed data, metadata, high-throughput versus low-throughput datasets, a common language for documentation, experimental and reporting guidelines, efficient data management systems, sharing practices, and relevant repositories. We systematically present available resources and optimal practices for wide use by experimental biomedical researchers.

  View details for DOI 10.1152/physrev.00043.2023

  View details for PubMedID 38451234

• Peer Review and Scientific Publication at a Crossroads: Call for Research for the 10th International Congress on Peer Review and Scientific Publication. JAMA Ioannidis, J. P., Berkwits, M., Flanagin, A., Bloom, T. 2023

  View details for DOI 10.1001/jama.2023.17607

  View details for PubMedID 37738041

• An umbrella review of systematic reviews on the impact of the COVID-19 pandemic on cancer prevention and management, and patient needs. eLife Muka, T., Li, J. J., Farahani, S. J., Ioannidis, J. P. 2023; 12

  Abstract

  The COVID-19 pandemic led to relocation and reconstruction of health care resources and systems, and to a decrease in healthcare utilization, and this may have affected the treatment, diagnosis, prognosis, and psychosocial well-being of patients with cancer. We aimed to summarize and quantify the evidence on the impact of the COVID-19 pandemic on the full spectrum of cancer care. An umbrella review was undertaken to summarize and quantify the findings from systematic reviews on impact of the COVID-19 pandemic on cancer treatment modification, delays, and cancellations; delays or cancellations in screening and diagnosis; psychosocial well-being, financial distress, and use of telemedicine as well as on other aspects of cancer care. PubMed and WHO COVID-19 Database was searched for relevant systematic reviews with or without meta-analysis published before November 29th, 2022. Abstract, full text screening and data extraction were performed by two independent reviewers. AMSTAR-2 was used for critical appraisal of included systematic reviews. 51 systematic reviews evaluating different aspects of cancer care were included in our analysis. Most reviews were based on observational studies judged to be at medium and high risk of bias. Only 2 of the included reviews had high or moderate scores based on AMSTAR-2. Findings suggest treatment modifications in cancer care during the pandemic versus the pre-pandemic period were based on low level of evidence. Different degrees of delays and cancellations in cancer treatment, screening and diagnosis were observed, with low-and-middle income countries and countries that implemented lockdowns being disproportionally affected. A shift from in-person appointments to telemedicine use was observed, but utility of telemedicine, challenges in implementation and cost-effectiveness in different areas of cancer care were little explored. Evidence was consistent in suggesting psychosocial well-being (e.g., depression, anxiety, and social activities) of patients with cancer deteriorated, and cancer patients experienced financial distress, albeit results were in general not compared to pre-pandemic levels. Impact of cancer care disruption during the pandemic on cancer prognosis was little explored. In conclusion, Substantial but heterogenous impact of COVID-19 pandemic on cancer care has been observed. Evidence gaps exist on this topic, with mid- and long-term impact on cancer care being most uncertain.

  View details for DOI 10.7554/eLife.85679

  View details for PubMedID 37014058

• The Rapid Growth of Mega-Journals: Threats and Opportunities. JAMA Ioannidis, J. P., Pezzullo, A. M., Boccia, S. 2023

  Abstract

  This Viewpoint examines the increase in mega-journals (prolific publishers of medical articles) and both the opportunities and threats to scientific research they present.

  View details for DOI 10.1001/jama.2023.3212

  View details for PubMedID 36939740

• Statistical Guidance to Authors at Top-Ranked Journals across Scientific Disciplines AMERICAN STATISTICIAN Hardwicke, T. E., Salholz-Hillel, M., Malicki, M., Szucs, D., Bendixen, T., Ioannidis, J. A. 2022

  View details for DOI 10.1080/00031305.2022.2143897

  View details for Web of Science ID 000892943900001

• Massive covidization of research citations and the citation elite. Proceedings of the National Academy of Sciences of the United States of America Ioannidis, J. P., Bendavid, E., Salholz-Hillel, M., Boyack, K. W., Baas, J. 2022; 119 (28): e2204074119

  Abstract

  Massive scientific productivity accompanied the COVID-19 pandemic. We evaluated the citation impact of COVID-19 publications relative to all scientific work published in 2020 to 2021 and assessed the impact on scientist citation profiles. Using Scopus data until August 1, 2021, COVID-19 items accounted for 4% of papers published, 20% of citations received to papers published in 2020 to 2021, and >30% of citations received in 36 of the 174 disciplines of science (up to 79.3% in general and internal medicine). Across science, 98 of the 100 most-cited papers published in 2020 to 2021 were related to COVID-19; 110 scientists received ≥10,000 citations for COVID-19 work, but none received ≥10,000 citations for non-COVID-19 work published in 2020 to 2021. For many scientists, citations to their COVID-19 work already accounted for more than half of their total career citation count. Overall, these data show a strong covidization of research citations across science, with major impact on shaping the citation elite.

  View details for DOI 10.1073/pnas.2204074119

  View details for PubMedID 35867747

• Comparison of pandemic excess mortality in 2020-2021 across different empirical calculations. Environmental research Levitt, M., Zonta, F., Ioannidis, J. P. 2022: 113754

  Abstract

  Different modeling approaches can be used to calculate excess deaths for the COVID-19 pandemic period. We compared 6 calculations of excess deaths (4 previously published [3 without age-adjustment] and two new ones that we performed with and without age-adjustment) for 2020-2021. With each approach, we calculated excess deaths metrics and the ratio R of excess deaths over recorded COVID-19 deaths. The main analysis focused on 33 high-income countries with weekly deaths in the Human Mortality Database (HMD at mortality.org) and reliable death registration. Secondary analyses compared calculations for other countries, whenever available. Across the 33 high-income countries, excess deaths were 2.0-2.8 million without age-adjustment, and 1.6-2.1 million with age-adjustment with large differences across countries. In our analyses after age-adjustment, 8 of 33 countries had no overall excess deaths; there was a death deficit in children; and 0.478 million (29.7%) of the excess deaths were in people <65 years old. In countries like France, Germany, Italy, and Spain excess death estimates differed 2 to 4-fold between highest and lowest figures. The R values' range exceeded 0.3 in all 33 countries. In 16 of 33 countries, the range of R exceeded 1. In 25 of 33 countries some calculations suggest R > 1 (excess deaths exceeding COVID-19 deaths) while others suggest R < 1 (excess deaths smaller than COVID-19 deaths). Inferred data from 4 evaluations for 42 countries and from 3 evaluations for another 98 countries are very tenuous. Estimates of excess deaths are analysis-dependent and age-adjustment is important to consider. Excess deaths may be lower than previously calculated.

  View details for DOI 10.1016/j.envres.2022.113754

  View details for PubMedID 35753371

• The end of the COVID-19 pandemic. European journal of clinical investigation Ioannidis, J. P. 2022: e13782

  Abstract

  There are no widely accepted, quantitative definitions for the end of a pandemic like COVID-19. The end of the pandemic due to a new virus and the transition to endemicity may be defined based on a high proportion of the global population having some immunity from natural infection or vaccination. Other considerations include diminished death toll, diminished pressure on health systems, reduced actual and perceived personal risk, removal of restrictive measures, and diminished public attention. A threshold of 70% of the global population having being vaccinated or infected was probably already reached in the second half of 2021. Endemicity may still show major spikes of infections and seasonality, but typically less clinical burden, although some locations are still hit more than others. Death toll and ICU occupancy figures are also consistent with a transition to endemicity by end 2021/early 2022. Personal risk for the vast majority of the global population was already very small by end 2021, but perceived risk may still be grossly over-estimated. Restrictive measures of high stringency have persisted in many countries by early 2022. The gargantuan attention in news media, social media, and even scientific circles should be tempered. Public health officials need to declare the end of the pandemic. Mid- and long-term consequences of epidemic waves and of adopted measures on health, society, economy, civilization, and democracy may perpetuate a pandemic legacy long after the pandemic itself has ended.

  View details for DOI 10.1111/eci.13782

  View details for PubMedID 35342941

• Infection fatality rate of COVID-19 in community-dwelling elderly populations. European journal of epidemiology Axfors, C., Ioannidis, J. P. 2022

  Abstract

  This mixed design synthesis aimed to estimate the infection fatality rate (IFR) of Coronavirus Disease 2019 (COVID-19) in community-dwelling elderly populations and other age groups from seroprevalence studies. Protocol: https://osf.io/47cgb . Eligible were seroprevalence studies done in 2020 and identified by any of four existing systematic reviews; with ≥500 participants aged ≥70years; presenting seroprevalence in elderly people; aimed to generate samples reflecting the general population; and whose location had available data on cumulative COVID-19 deaths in elderly (primary cutoff≥70years; ≥65 or ≥60 also eligible). We extracted the most fully adjusted (if unavailable, unadjusted) seroprevalence estimates; age- and residence-stratified cumulative COVID-19 deaths (until 1week after the seroprevalence sampling midpoint) from official reports; and population statistics, to calculate IFRs adjusted for test performance. Sample size-weighted IFRs were estimated for countries with multiple estimates. Thirteen seroprevalence surveys representing 11 high-income countries were included in the main analysis. Median IFR in community-dwelling elderly and elderly overall was 2.9% (range 1.8-9.7%) and 4.5% (range 2.5-16.7%) without accounting for seroreversion (2.2% and 4.0%, respectively, accounting for 5% monthly seroreversion). Multiple sensitivity analyses yielded similar results. IFR was higher with larger proportions of people >85years. The IFR of COVID-19 in community-dwelling elderly is lower than previously reported.

  View details for DOI 10.1007/s10654-022-00853-w

  View details for PubMedID 35306604

• Ninth International Congress on Peer Review and Scientific Publication: Call for Abstracts. JAMA Ioannidis, J. P., Berkwits, M., Flanagin, A., Godlee, F., Bloom, T. 2021

  View details for DOI 10.1001/jama.2021.16596

  View details for PubMedID 34542570

• Pearls on science, collaboration, and mentorship in health research: A masterclass conversation with Dr. John Ioannidis. Journal of clinical epidemiology Naji, L., Kay, J., Johansson, I., Rodrigues, M., Hu, Z. J., Akula, K. K., Thabane, L., Ioannidis, J. P. 2021

  Abstract

  Effective collaboration and mentorship are essential to success in a career of health research. We summarize our conversation with Dr. John Ioannidis, professor at Stanford University, author of the most accessed manuscript in the history of the Public Library of Science, and one of the most cited scientists in history. Dr. Ioannidis was invited for a question and answer session as part of a graduate-level course on biostatistical collaboration hosted at McMaster University in December 2020. This text provides insight into the experiences and pearls he shared, that we hope will inspire and guide other researchers early or junior in their careers. He emphasized the importance of passion, enthusiasm and a sincere pursuit for high quality research as being the cornerstones to success and continued productivity in this field.

  View details for DOI 10.1016/j.jclinepi.2021.08.009

  View details for PubMedID 34400256

• Assessment of transparency indicators across the biomedical literature: How open is open? PLoS biology Serghiou, S. n., Contopoulos-Ioannidis, D. G., Boyack, K. W., Riedel, N. n., Wallach, J. D., Ioannidis, J. P. 2021; 19 (3): e3001107

  Abstract

  Recent concerns about the reproducibility of science have led to several calls for more open and transparent research practices and for the monitoring of potential improvements over time. However, with tens of thousands of new biomedical articles published per week, manually mapping and monitoring changes in transparency is unrealistic. We present an open-source, automated approach to identify 5 indicators of transparency (data sharing, code sharing, conflicts of interest disclosures, funding disclosures, and protocol registration) and apply it across the entire open access biomedical literature of 2.75 million articles on PubMed Central (PMC). Our results indicate remarkable improvements in some (e.g., conflict of interest [COI] disclosures and funding disclosures), but not other (e.g., protocol registration and code sharing) areas of transparency over time, and map transparency across fields of science, countries, journals, and publishers. This work has enabled the creation of a large, integrated, and openly available database to expedite further efforts to monitor, understand, and promote transparency and reproducibility in science.

  View details for DOI 10.1371/journal.pbio.3001107

  View details for PubMedID 33647013

• Use of E-values for addressing confounding in observational studies-an empirical assessment of the literature. International journal of epidemiology Blum, M. R., Tan, Y. J., Ioannidis, J. P. 2020

  Abstract

  BACKGROUND: E-values are a recently introduced approach to evaluate confounding in observational studies. We aimed to empirically assess the current use of E-values in published literature.METHODS: We conducted a systematic literature search for all publications, published up till the end of 2018, which cited at least one of two inceptive E-value papers and presented E-values for original data. For these case publications we identified control publications, matched by journal and issue, where the authors had not calculated E-values.RESULTS: In total, 87 papers presented 516 E-values. Of the 87 papers, 14 concluded that residual confounding likely threatens at least some of the main conclusions. Seven of these 14 named potential uncontrolled confounders. 19 of 87 papers related E-value magnitudes to expected strengths of field-specific confounders. The median E-value was 1.88, 1.82, and 2.02 for the 43, 348, and 125 E-values where confounding was felt likely to affect the results, unlikely to affect the results, or not commented upon, respectively. The 69 case-control publication pairs dealt with effect sizes of similar magnitude. Of 69 control publications, 52 did not comment on unmeasured confounding and 44/69 case publications concluded that confounding was unlikely to affect study conclusions.CONCLUSIONS: Few papers using E-values conclude that confounding threatens their results, and their E-values overlap in magnitude with those of papers acknowledging susceptibility to confounding. Facile automation in calculating E-values may compound the already poor handling of confounding. E-values should not be a substitute for careful consideration of potential sources of unmeasured confounding. If used, they should be interpreted in the context of expected confounding in specific fields.

  View details for DOI 10.1093/ije/dyz261

  View details for PubMedID 31930286

• Online randomized controlled experiments at scale: lessons and extensions to medicine. Trials Kohavi, R. n., Tang, D. n., Xu, Y. n., Hemkens, L. G., Ioannidis, J. P. 2020; 21 (1): 150

  Abstract

  Many technology companies, including Airbnb, Amazon, Booking.com, eBay, Facebook, Google, LinkedIn, Lyft, Microsoft, Netflix, Twitter, Uber, and Yahoo!/Oath, run online randomized controlled experiments at scale, namely hundreds of concurrent controlled experiments on millions of users each, commonly referred to as A/B tests. Originally derived from the same statistical roots, randomized controlled trials (RCTs) in medicine are now criticized for being expensive and difficult, while in technology, the marginal cost of such experiments is approaching zero and the value for data-driven decision-making is broadly recognized.This is an overview of key scaling lessons learned in the technology field. They include (1) a focus on metrics, an overall evaluation criterion and thousands of metrics for insights and debugging, automatically computed for every experiment; (2) quick release cycles with automated ramp-up and shut-down that afford agile and safe experimentation, leading to consistent incremental progress over time; and (3) a culture of 'test everything' because most ideas fail and tiny changes sometimes show surprising outcomes worth millions of dollars annually. Technological advances, online interactions, and the availability of large-scale data allowed technology companies to take the science of RCTs and use them as online randomized controlled experiments at large scale with hundreds of such concurrent experiments running on any given day on a wide range of software products, be they web sites, mobile applications, or desktop applications. Rather than hindering innovation, these experiments enabled accelerated innovation with clear improvements to key metrics, including user experience and revenue. As healthcare increases interactions with patients utilizing these modern channels of web sites and digital health applications, many of the lessons apply. The most innovative technological field has recognized that systematic series of randomized trials with numerous failures of the most promising ideas leads to sustainable improvement.While there are many differences between technology and medicine, it is worth considering whether and how similar designs can be applied via simple RCTs that focus on healthcare decision-making or service delivery. Changes - small and large - should undergo continuous and repeated evaluations in randomized trials and learning from their results will enable accelerated healthcare improvements.

  View details for DOI 10.1186/s13063-020-4084-y

  View details for PubMedID 32033614

• Forecasting for COVID-19 has failed. International journal of forecasting Ioannidis, J. P., Cripps, S. n., Tanner, M. A. 2020

  Abstract

  Epidemic forecasting has a dubious track-record, and its failures became more prominent with COVID-19. Poor data input, wrong modeling assumptions, high sensitivity of estimates, lack of incorporation of epidemiological features, poor past evidence on effects of available interventions, lack of transparency, errors, lack of determinacy, looking at only one or a few dimensions of the problem at hand, lack of expertise in crucial disciplines, groupthink and bandwagon effects and selective reporting are some of the causes of these failures. Nevertheless, epidemic forecasting is unlikely to be abandoned. Some (but not all) of these problems can be fixed. Careful modeling of predictive distributions rather than focusing on point estimates, considering multiple dimensions of impact, and continuously reappraising models based on their validated performance may help. If extreme values are considered, extremes should be considered for the consequences of multiple dimensions of impact so as to continuously calibrate predictive insights and decision-making. When major decisions (e.g. draconian lockdowns) are based on forecasts, the harms (in terms of health, economy, and society at large) and the asymmetry of risks need to be approached in a holistic fashion, considering the totality of the evidence.

  View details for DOI 10.1016/j.ijforecast.2020.08.004

  View details for PubMedID 32863495

  View details for PubMedCentralID PMC7447267

• The Importance of Predefined Rules and Prespecified Statistical Analyses Do Not Abandon Significance JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Ioannidis, J. A. 2019; 321 (21): 2067–68

  View details for DOI 10.1001/jama.2019.4582

  View details for Web of Science ID 000470158700006

• A standardized citation metrics author database annotated for scientific field. PLoS biology Ioannidis, J. P., Baas, J. n., Klavans, R. n., Boyack, K. W. 2019; 17 (8): e3000384

  Abstract

  Citation metrics are widely used and misused. We have created a publicly available database of 100,000 top scientists that provides standardized information on citations, h-index, coauthorship-adjusted hm-index, citations to papers in different authorship positions, and a composite indicator. Separate data are shown for career-long and single-year impact. Metrics with and without self-citations and ratio of citations to citing papers are given. Scientists are classified into 22 scientific fields and 176 subfields. Field- and subfield-specific percentiles are also provided for all scientists who have published at least five papers. Career-long data are updated to end of 2017 and to end of 2018 for comparison.

  View details for DOI 10.1371/journal.pbio.3000384

  View details for PubMedID 31404057

• Reproducible research practices, transparency, and open access data in the biomedical literature, 2015-2017. PLoS biology Wallach, J. D., Boyack, K. W., Ioannidis, J. P. 2018; 16 (11): e2006930

  Abstract

  Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical journals articles published in 2000-2014, the biomedical literature largely lacked transparency in important dimensions. Here, we surveyed a random sample of 149 biomedical articles published between 2015 and 2017 and determined the proportion reporting sources of public and/or private funding and conflicts of interests, sharing protocols and raw data, and undergoing rigorous independent replication and reproducibility checks. We also investigated what can be learned about reproducibility and transparency indicators from open access data provided on PubMed. The majority of the 149 studies disclosed some information regarding funding (103, 69.1% [95% confidence interval, 61.0% to 76.3%]) or conflicts of interest (97, 65.1% [56.8% to 72.6%]). Among the 104 articles with empirical data in which protocols or data sharing would be pertinent, 19 (18.3% [11.6% to 27.3%]) discussed publicly available data; only one (1.0% [0.1% to 6.0%]) included a link to a full study protocol. Among the 97 articles in which replication in studies with different data would be pertinent, there were five replication efforts (5.2% [1.9% to 12.2%]). Although clinical trial identification numbers and funding details were often provided on PubMed, only two of the articles without a full text article in PubMed Central that discussed publicly available data at the full text level also contained information related to data sharing on PubMed; none had a conflicts of interest statement on PubMed. Our evaluation suggests that although there have been improvements over the last few years in certain key indicators of reproducibility and transparency, opportunities exist to improve reproducible research practices across the biomedical literature and to make features related to reproducibility more readily visible in PubMed.

  View details for PubMedID 30457984

• In the Era of Precision Medicine and Big Data, Who Is Normal? JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Manrai, A. K., Patel, C. J., Ioannidis, J. A. 2018; 319 (19): 1981–82

  View details for PubMedID 29710130

• All science should inform policy and regulation. PLoS medicine Ioannidis, J. P. 2018; 15 (5): e1002576

  Abstract

  In the context of a recent proposal to exclude research from consideration at the Environmental Protection Agency, John Ioannidis points out that "perceived perfection is not a characteristic of science, but of dogma" and envisions how governments can promote a standard of openness in science.

  View details for PubMedID 29723196

• The Proposal to Lower P Value Thresholds to .005 JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Loannidis, J. A. 2018; 319 (14): 1429–30

  View details for PubMedID 29566133

• Meta-research: Why research on research matters PLOS BIOLOGY Ioannidis, J. A. 2018; 16 (3): e2005468

  Abstract

  Meta-research is the study of research itself: its methods, reporting, reproducibility, evaluation, and incentives. Given that science is the key driver of human progress, improving the efficiency of scientific investigation and yielding more credible and more useful research results can translate to major benefits. The research enterprise grows very fast. Both new opportunities for knowledge and innovation and new threats to validity and scientific integrity emerge. Old biases abound, and new ones continuously appear as novel disciplines emerge with different standards and challenges. Meta-research uses an interdisciplinary approach to study, promote, and defend robust science. Major disruptions are likely to happen in the way we pursue scientific investigation, and it is important to ensure that these disruptions are evidence based.

  View details for PubMedID 29534060

• Meta-assessment of bias in science PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA Fanelli, D., Costas, R., Ioannidis, J. P. 2017; 114 (14): 3714-3719

  Abstract

  Numerous biases are believed to affect the scientific literature, but their actual prevalence across disciplines is unknown. To gain a comprehensive picture of the potential imprint of bias in science, we probed for the most commonly postulated bias-related patterns and risk factors, in a large random sample of meta-analyses taken from all disciplines. The magnitude of these biases varied widely across fields and was overall relatively small. However, we consistently observed a significant risk of small, early, and highly cited studies to overestimate effects and of studies not published in peer-reviewed journals to underestimate them. We also found at least partial confirmation of previous evidence suggesting that US studies and early studies might report more extreme effects, although these effects were smaller and more heterogeneously distributed across meta-analyses and disciplines. Authors publishing at high rates and receiving many citations were, overall, not at greater risk of bias. However, effect sizes were likely to be overestimated by early-career researchers, those working in small or long-distance collaborations, and those responsible for scientific misconduct, supporting hypotheses that connect bias to situational factors, lack of mutual control, and individual integrity. Some of these patterns and risk factors might have modestly increased in intensity over time, particularly in the social sciences. Our findings suggest that, besides one being routinely cautious that published small, highly-cited, and earlier studies may yield inflated results, the feasibility and costs of interventions to attenuate biases in the literature might need to be discussed on a discipline-specific and topic-specific basis.

  View details for DOI 10.1073/pnas.1618569114

  View details for PubMedID 28320937

• Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature PLOS BIOLOGY Szucs, D., Ioannidis, J. A. 2017; 15 (3): e2000797

  Abstract

  We have empirically assessed the distribution of published effect sizes and estimated power by analyzing 26,841 statistical records from 3,801 cognitive neuroscience and psychology papers published recently. The reported median effect size was D = 0.93 (interquartile range: 0.64-1.46) for nominally statistically significant results and D = 0.24 (0.11-0.42) for nonsignificant results. Median power to detect small, medium, and large effects was 0.12, 0.44, and 0.73, reflecting no improvement through the past half-century. This is so because sample sizes have remained small. Assuming similar true effect sizes in both disciplines, power was lower in cognitive neuroscience than in psychology. Journal impact factors negatively correlated with power. Assuming a realistic range of prior probabilities for null hypotheses, false report probability is likely to exceed 50% for the whole literature. In light of our findings, the recently reported low replication success in psychology is realistic, and worse performance may be expected for cognitive neuroscience.

  View details for PubMedID 28253258

• Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials. JAMA internal medicine Wallach, J. D., Sullivan, P. G., Trepanowski, J. F., Sainani, K. L., Steyerberg, E. W., Ioannidis, J. P. 2017

  Abstract

  Many published randomized clinical trials (RCTs) make claims for subgroup differences.To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P < .05 from an interaction test) and corroborated by subsequent RCTs and meta-analyses.This meta-epidemiological survey examines data sets of trials with at least 1 subgroup claim, including Subgroup Analysis of Trials Is Rarely Easy (SATIRE) articles and Discontinuation of Randomized Trials (DISCO) articles. We used Scopus (updated July 2016) to search for English-language articles citing each of the eligible index articles with at least 1 subgroup finding in the abstract.Articles with a subgroup claim in the abstract with or without evidence of statistical heterogeneity (P < .05 from an interaction test) in the text and articles attempting to corroborate the subgroup findings.Study characteristics of trials with at least 1 subgroup claim in the abstract were recorded. Two reviewers extracted the data necessary to calculate subgroup-level effect sizes, standard errors, and the P values for interaction. For individual RCTs and meta-analyses that attempted to corroborate the subgroup findings from the index articles, trial characteristics were extracted. Cochran Q test was used to reevaluate heterogeneity with the data from all available trials.The number of subgroup claims in the abstracts of RCTs, the number of subgroup claims in the abstracts of RCTs with statistical support (subgroup findings), and the number of subgroup findings corroborated by subsequent RCTs and meta-analyses.Sixty-four eligible RCTs made a total of 117 subgroup claims in their abstracts. Of these 117 claims, only 46 (39.3%) in 33 articles had evidence of statistically significant heterogeneity from a test for interaction. In addition, out of these 46 subgroup findings, only 16 (34.8%) ensured balance between randomization groups within the subgroups (eg, through stratified randomization), 13 (28.3%) entailed a prespecified subgroup analysis, and 1 (2.2%) was adjusted for multiple testing. Only 5 (10.9%) of the 46 subgroup findings had at least 1 subsequent pure corroboration attempt by a meta-analysis or an RCT. In all 5 cases, the corroboration attempts found no evidence of a statistically significant subgroup effect. In addition, all effect sizes from meta-analyses were attenuated toward the null.A minority of subgroup claims made in the abstracts of RCTs are supported by their own data (ie, a significant interaction effect). For those that have statistical support (P < .05 from an interaction test), most fail to meet other best practices for subgroup tests, including prespecification, stratified randomization, and adjustment for multiple testing. Attempts to corroborate statistically significant subgroup differences are rare; when done, the initially observed subgroup differences are not reproduced.

  View details for DOI 10.1001/jamainternmed.2016.9125

  View details for PubMedID 28192563

• A manifesto for reproducible science NATURE HUMAN BEHAVIOUR Munafo, M. R., Nosek, B. A., Bishop, D. M., Button, K. S., Chambers, C. D., du Sert, N., Simonsohn, U., Wagenmakers, E., Ware, J. J., Ioannidis, J. A. 2017; 1 (1)

  View details for DOI 10.1038/s41562-016-0021

  View details for Web of Science ID 000418775900021

• What does research reproducibility mean? SCIENCE TRANSLATIONAL MEDICINE Goodman, S. N., Fanelli, D., Ioannidis, J. P. 2016; 8 (341)

  Abstract

  The language and conceptual framework of "research reproducibility" are nonstandard and unsettled across the sciences. In this Perspective, we review an array of explicit and implicit definitions of reproducibility and related terminology, and discuss how to avoid potential misunderstandings when these terms are used as a surrogate for "truth."

  View details for DOI 10.1126/scitranslmed.aaf5027

  View details for PubMedID 27252173

• Evidence-based medicine has been hijacked: a report to David Sackett JOURNAL OF CLINICAL EPIDEMIOLOGY Ioannidis, J. P. 2016; 73: 82-86

  View details for DOI 10.1016/j.jclinepi.2016.02.012

  View details for PubMedID 26934549

• Evolution of Reporting P Values in the Biomedical Literature, 1990-2015 JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Chavalarias, D., Wallach, J. D., Li, A. H., Ioannidis, J. P. 2016; 315 (11): 1141-1148

  Abstract

  The use and misuse of P values has generated extensive debates.To evaluate in large scale the P values reported in the abstracts and full text of biomedical research articles over the past 25 years and determine how frequently statistical information is presented in ways other than P values.Automated text-mining analysis was performed to extract data on P values reported in 12,821,790 MEDLINE abstracts and in 843,884 abstracts and full-text articles in PubMed Central (PMC) from 1990 to 2015. Reporting of P values in 151 English-language core clinical journals and specific article types as classified by PubMed also was evaluated. A random sample of 1000 MEDLINE abstracts was manually assessed for reporting of P values and other types of statistical information; of those abstracts reporting empirical data, 100 articles were also assessed in full text.P values reported.Text mining identified 4,572,043 P values in 1,608,736 MEDLINE abstracts and 3,438,299 P values in 385,393 PMC full-text articles. Reporting of P values in abstracts increased from 7.3% in 1990 to 15.6% in 2014. In 2014, P values were reported in 33.0% of abstracts from the 151 core clinical journals (n = 29,725 abstracts), 35.7% of meta-analyses (n = 5620), 38.9% of clinical trials (n = 4624), 54.8% of randomized controlled trials (n = 13,544), and 2.4% of reviews (n = 71,529). The distribution of reported P values in abstracts and in full text showed strong clustering at P values of .05 and of .001 or smaller. Over time, the "best" (most statistically significant) reported P values were modestly smaller and the "worst" (least statistically significant) reported P values became modestly less significant. Among the MEDLINE abstracts and PMC full-text articles with P values, 96% reported at least 1 P value of .05 or lower, with the proportion remaining steady over time in PMC full-text articles. In 1000 abstracts that were manually reviewed, 796 were from articles reporting empirical data; P values were reported in 15.7% (125/796 [95% CI, 13.2%-18.4%]) of abstracts, confidence intervals in 2.3% (18/796 [95% CI, 1.3%-3.6%]), Bayes factors in 0% (0/796 [95% CI, 0%-0.5%]), effect sizes in 13.9% (111/796 [95% CI, 11.6%-16.5%]), other information that could lead to estimation of P values in 12.4% (99/796 [95% CI, 10.2%-14.9%]), and qualitative statements about significance in 18.1% (181/1000 [95% CI, 15.8%-20.6%]); only 1.8% (14/796 [95% CI, 1.0%-2.9%]) of abstracts reported at least 1 effect size and at least 1 confidence interval. Among 99 manually extracted full-text articles with data, 55 reported P values, 4 presented confidence intervals for all reported effect sizes, none used Bayesian methods, 1 used false-discovery rates, 3 used sample size/power calculations, and 5 specified the primary outcome.In this analysis of P values reported in MEDLINE abstracts and in PMC articles from 1990-2015, more MEDLINE abstracts and articles reported P values over time, almost all abstracts and articles with P values reported statistically significant results, and, in a subgroup analysis, few articles included confidence intervals, Bayes factors, or effect sizes. Rather than reporting isolated P values, articles should include effect sizes and uncertainty metrics.

  View details for DOI 10.1001/jama.2016.1952

  View details for Web of Science ID 000372159800019

• Reproducible Research Practices and Transparency across the Biomedical Literature. PLoS biology Iqbal, S. A., Wallach, J. D., Khoury, M. J., Schully, S. D., Ioannidis, J. P. 2016; 14 (1)

  Abstract

  There is a growing movement to encourage reproducibility and transparency practices in the scientific community, including public access to raw data and protocols, the conduct of replication studies, systematic integration of evidence in systematic reviews, and the documentation of funding and potential conflicts of interest. In this survey, we assessed the current status of reproducibility and transparency addressing these indicators in a random sample of 441 biomedical journal articles published in 2000-2014. Only one study provided a full protocol and none made all raw data directly available. Replication studies were rare (n = 4), and only 16 studies had their data included in a subsequent systematic review or meta-analysis. The majority of studies did not mention anything about funding or conflicts of interest. The percentage of articles with no statement of conflict decreased substantially between 2000 and 2014 (94.4% in 2000 to 34.6% in 2014); the percentage of articles reporting statements of conflicts (0% in 2000, 15.4% in 2014) or no conflicts (5.6% in 2000, 50.0% in 2014) increased. Articles published in journals in the clinical medicine category versus other fields were almost twice as likely to not include any information on funding and to have private funding. This study provides baseline data to compare future progress in improving these indicators in the scientific literature.

  View details for DOI 10.1371/journal.pbio.1002333

  View details for PubMedID 26726926

• Meta-research: Evaluation and Improvement of Research Methods and Practices PLOS BIOLOGY Ioannidis, J. P., Fanelli, D., Dunne, D. D., Goodman, S. N. 2015; 13 (10)

  Abstract

  As the scientific enterprise has grown in size and diversity, we need empirical evidence on the research process to test and apply interventions that make it more efficient and its results more reliable. Meta-research is an evolving scientific discipline that aims to evaluate and improve research practices. It includes thematic areas of methods, reporting, reproducibility, evaluation, and incentives (how to do, report, verify, correct, and reward science). Much work is already done in this growing field, but efforts to-date are fragmented. We provide a map of ongoing efforts and discuss plans for connecting the multiple meta-research efforts across science worldwide.

  View details for DOI 10.1371/journal.pbio.1002264

  View details for PubMedID 26431313

• Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations JOURNAL OF CLINICAL EPIDEMIOLOGY Patel, C. J., Burford, B., Ioannidis, J. P. 2015; 68 (9): 1046-1058

  Abstract

  Model specification-what adjusting variables are analytically modeled-may influence results of observational associations. We present a standardized approach to quantify the variability of results obtained with choices of adjustments called the "vibration of effects" (VoE).We estimated the VoE for 417 clinical, environmental, and physiological variables in association with all-cause mortality using National Health and Nutrition Examination Survey data. We selected 13 variables as adjustment covariates and computed 8,192 Cox models for each of 417 variables' associations with all-cause mortality.We present the VoE by assessing the variance of the effect size and in the -log10(P-value) obtained by different combinations of adjustments. We present whether there are multimodality patterns in effect sizes and P-values and the trajectory of results with increasing adjustments. For 31% of the 417 variables, we observed a Janus effect, with the effect being in opposite direction in the 99th versus the 1st percentile of analyses. For example, the vitamin E variant α-tocopherol had a VoE that indicated higher and lower risk for mortality.Estimating VoE offers empirical estimates of associations are under different model specifications. When VoE is large, claims for observational associations should be very cautious.

  View details for DOI 10.1016/j.jclinepi.2015.05.029

  View details for Web of Science ID 000360597300011

  View details for PubMedCentralID PMC4555355

• Evaluation of Wellness Determinants and Interventions by Citizen Scientists. JAMA Naci, H., Ioannidis, J. P. 2015; 314 (2): 121-122

  View details for DOI 10.1001/jama.2015.6160

  View details for PubMedID 26068643

• Does screening for disease save lives in asymptomatic adults? Systematic review of meta-analyses and randomized trials INTERNATIONAL JOURNAL OF EPIDEMIOLOGY Saquib, N., Saquib, J., Ioannidis, J. P. 2015; 44 (1): 264-277

  Abstract

  Several popular screening tests, such as mammography and prostate-specific antigen, have met with wide controversy and/or have lost their endorsement recently. We systematically evaluated evidence from randomized controlled trials (RCTs) as to whether screening decreases mortality from diseases where death is a common outcome.We searched three sources: United States Preventive Services Task Force (USPSTF), Cochrane Database of Systematic Reviews, and PubMed. We extracted recommendation status, category of evidence and RCT availability on mortality for screening tests for diseases on asymptomatic adults (excluding pregnant women and children) from USPSTF. We identified meta-analyses and individual RCTs on screening and mortality from Cochrane and PubMed.We selected 19 diseases (39 tests) out of 50 diseases/disorders for which USPSTF provides screening evaluation. Screening is recommended for 6 diseases (12 tests) out of the 19. We assessed 9 non-overlapping meta-analyses and 48 individual trials for these 19 diseases. Among the results of the meta-analyses, reductions where the 95% confidence intervals (CIs) excluded the null occurred for four disease-specific mortality estimates (ultrasound for abdominal aortic aneurysm in men; mammography for breast cancer; fecal occult blood test and flexible sigmoidoscopy for colorectal cancer) and for none of the all-cause mortality estimates. Among individual RCTs, reductions in disease-specific and all-cause mortality where the 95% CIs excluded the null occurred in 30% and 11% of the estimates, respectively.Among currently available screening tests for diseases where death is a common outcome, reductions in disease-specific mortality are uncommon and reductions in all-cause mortality are very rare or non-existent.

  View details for DOI 10.1093/ije/dyu140

  View details for PubMedID 25596211

• Call to improve transparency of trials of non-regulated interventions. BMJ (Clinical research ed.) Dal-Ré, R., Bracken, M. B., Ioannidis, J. P. 2015; 350: h1323-?

  View details for DOI 10.1136/bmj.h1323

  View details for PubMedID 25820265

• Agreement Between Mega-Trials and Smaller Trials: A Systematic Review and Meta-Research Analysis. JAMA network open Kastrati, L., Raeisi-Dehkordi, H., Llanaj, E., Quezada-Pinedo, H. G., Khatami, F., Ahanchi, N. S., Llane, A., Mecani, R., Muka, T., Ioannidis, J. P. 2024; 7 (9): e2432296

  Abstract

  Importance: Mega-trials can provide large-scale evidence on important questions.Objective: To explore how the results of mega-trials compare with the meta-analysis results of trials with smaller sample sizes.Data Sources: ClinicalTrials.gov was searched for mega-trials until January 2023. PubMed was searched until June 2023 for meta-analyses incorporating the results of the eligible mega-trials.Study Selection: Mega-trials were eligible if they were noncluster nonvaccine randomized clinical trials, had a sample size over 10 000, and had a peer-reviewed meta-analysis publication presenting results for the primary outcome of the mega-trials and/or all-cause mortality.Data Extraction and Synthesis: For each selected meta-analysis, we extracted results of smaller trials and mega-trials included in the summary effect estimate and combined them separately using random effects. These estimates were used to calculate the ratio of odds ratios (ROR) between mega-trials and smaller trials in each meta-analysis. Next, the RORs were combined using random effects. Risk of bias was extracted for each trial included in our analyses (or when not available, assessed only for mega-trials). Data analysis was conducted from January to June 2024.Main Outcomes and Measures: The main outcomes were the summary ROR for the primary outcome and all-cause mortality between mega-trials and smaller trials. Sensitivity analyses were performed with respect to the year of publication, masking, weight, type of intervention, and specialty.Results: Of 120 mega-trials identified, 41 showed a significant result for the primary outcome and 22 showed a significant result for all-cause mortality. In 35 comparisons of primary outcomes (including 85 point estimates from 69 unique mega-trials and 272 point estimates from smaller trials) and 26 comparisons of all-cause mortality (including 70 point estimates from 65 unique mega-trials and 267 point estimates from smaller trials), no difference existed between the outcomes of the mega-trials and smaller trials for primary outcome (ROR, 1.00; 95% CI, 0.97-1.04) nor for all-cause mortality (ROR, 1.00; 95% CI, 0.97-1.04). For the primary outcomes, smaller trials published before the mega-trials had more favorable results than the mega-trials (ROR, 1.05; 95% CI, 1.01-1.10) and subsequent smaller trials published after the mega-trials (ROR, 1.10; 95% CI, 1.04-1.18).Conclusions and Relevance: In this meta-research analysis, meta-analyses of smaller studies showed overall comparable results with mega-trials, but smaller trials published before the mega-trials gave more favorable results than mega-trials. These findings suggest that mega-trials need to be performed more often given the relative low number of mega-trials found, their low significant rates, and the fact that smaller trials published prior to mega-trial report more beneficial results than mega-trials and subsequent smaller trials.

  View details for DOI 10.1001/jamanetworkopen.2024.32296

  View details for PubMedID 39240561

• EU-US data transfers: an enduring challenge for health research collaborations. NPJ digital medicine Lalova-Spinks, T., Valcke, P., Ioannidis, J. P., Huys, I. 2024; 7 (1): 215

  Abstract

  EU-US data transfers for health research remain a particularly thorny issue in view of the stringent rules of the EU General Data Protection Regulation (GDPR) and the challenges related to US mass surveillance programs, particularly the manner in which US law enforcement and national security agencies can access personal data originating from the EU. Since the entry into force of the GDPR, evidence of impeded collaborations is increasing, particularly in the case of sharing data with US public institutions. The adoption of a new EU-US adequacy decision in July 2023 does not hold the promise for a long-lasting solution due to the risks of being challenged and invalidated - yet again - at the Court of Justice of the EU. As the research community is calling for answers, the new proposal for a European Health Data Space regulation may hold a key to solving some of the existing issues. In this paper, we critically discuss the current rules and outline a possible way forward for transfers between public bodies.

  View details for DOI 10.1038/s41746-024-01205-6

  View details for PubMedID 39152232

  View details for PubMedCentralID 5110051

• Evidence base for yearly respiratory virus vaccines: Current status and proposed improved strategies. European journal of clinical investigation Barosa, M., Ioannidis, J. P., Prasad, V. 2024: e14286

  Abstract

  Annual vaccination is widely recommended for influenza and SARS-CoV-2. In this essay, we analyse and question the prevailing policymaking approach to these respiratory virus vaccines, especially in the United States. Every year, licensed influenza vaccines are reformulated to include specific strains expected to dominate in the season ahead. Updated vaccines are rapidly manufactured and approved without further regulatory requirement of clinical data. Novel vaccines (i.e. new products) typically undergo clinical trials, though generally powered for clinically unimportant outcomes (e.g. lab-confirmed infections, regardless of symptomatology or antibody levels). Eventually, the current and future efficacy of influenza and COVID-19 vaccines against hospitalization or death carries considerable uncertainty. The emergence of highly transmissible SARS-CoV-2 variants and waning vaccine-induced immunity led to plummeting vaccine effectiveness, at least against symptomatic infection, and booster doses have since been widely recommended. No further randomized trials were performed for clinically important outcomes for licensed updated boosters. In both cases, annual vaccine effectiveness estimates are generated by observational research, but observational studies are particularly susceptible to confounding and bias. Well-conducted experimental studies, particularly randomized trials, are necessary to address persistent uncertainties about influenza and COVID-19 vaccines. We propose a new research framework which would render results relevant to the current or future respiratory viral seasons. We demonstrate that experimental studies are feasible by adopting a more pragmatic approach and provide strategies on how to do so. When it comes to implementing policies that seriously impact people's lives, require substantial public resources and/or rely on widespread public acceptance, high evidence standards are desirable.

  View details for DOI 10.1111/eci.14286

  View details for PubMedID 39078026

• Evolving patterns of extreme publishing behavior across science SCIENTOMETRICS Ioannidis, J. A., Collins, T. A., Baas, J. 2024

  View details for DOI 10.1007/s11192-024-05117-w

  View details for Web of Science ID 001278329800001

• Artificial intelligence in scientific medical writing: Legitimate and deceptive uses and ethical concerns. European journal of internal medicine Ramoni, D., Sgura, C., Liberale, L., Montecucco, F., Ioannidis, J. P., Carbone, F. 2024

  Abstract

  The debate surrounding the integration of artificial intelligence (AI) into scientific writing has already attracted significant interest in medical and life sciences. While AI can undoubtedly expedite the process of manuscript creation and correction, it raises several criticisms. The crossover between AI and health sciences is relatively recent, but the use of AI tools among physicians and other scientists who work in the life sciences is growing very fast. Within this whirlwind, it is becoming essential to realize where we are heading and what the limits are, including an ethical perspective. Modern conversational AIs exhibit a context awareness that enables them to understand and remember any conversation beyond any predefined script. Even more impressively, they can learn and adapt as they engage with a growing volume of human language input. They all share neural networks as background mathematical models and differ from old chatbots for their use of a specific network architecture called transformer model [1]. Some of them exceed 100 terabytes (TB) (e.g., Bloom, LaMDA) or even 500 TB (e.g., Megatron-Turing NLG) of text data, the 4.0 version of ChatGPT (GPT-4) was trained with nearly 45 TB, but stays updated by the internet connection and may integrate with different plugins that enhance its functionality, making it multimodal.

  View details for DOI 10.1016/j.ejim.2024.07.012

  View details for PubMedID 39048335

• What is the vibration of effects? BMJ evidence-based medicine Vinatier, C., Hoffmann, S., Patel, C., DeVito, N. J., Cristea, I. A., Tierney, B., Ioannidis, J. P., Naudet, F. 2024

  View details for DOI 10.1136/bmjebm-2023-112747

  View details for PubMedID 38997151

• Subjective evidence evaluation survey for many-analysts studies. Royal Society open science Sarafoglou, A., Hoogeveen, S., van den Bergh, D., Aczel, B., Albers, C. J., Althoff, T., Botvinik-Nezer, R., Busch, N. A., Cataldo, A. M., Devezer, B., van Dongen, N. N., Dreber, A., Fried, E. I., Hoekstra, R., Hoffman, S., Holzmeister, F., Huber, J., Huntington-Klein, N., Ioannidis, J., Johannesson, M., Kirchler, M., Loken, E., Mangin, J., Matzke, D., Menkveld, A. J., Nilsonne, G., van Ravenzwaaij, D., Schweinsberg, M., Schulz-Kuempel, H., Shanks, D. R., Simons, D. J., Spellman, B. A., Stoevenbelt, A. H., Szaszi, B., Trubutschek, D., Tuerlinckx, F., Uhlmann, E. L., Vanpaemel, W., Wicherts, J., Wagenmakers, E. 2024; 11 (7): 240125

  Abstract

  Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.

  View details for DOI 10.1098/rsos.240125

  View details for PubMedID 39050728

• A novel framework to assess haematology and oncology registration trials: The THEOREMM project. European journal of clinical investigation Olivier, T., Haslam, A., Burke, P., Boutron, I., Naudet, F., Ioannidis, J. P., Prasad, V. 2024: e14267

  Abstract

  Methodological limitations affect a significant number of oncology and haematology trials, raising concerns about the applicability of their results. For example, a suboptimal control arm or limited access to best care upon progression may skew the trial results toward a benefit in the experimental arm. Beyond the fact that such limitations do not prevent drugs reaching the market, other assessment tools, such as those developed by professional societies-ESMO-MCBS and ASCO Value Framework-do not integrate these important shortcomings.We propose creating a novel framework with the scope of assessing registration cancer clinical trials in haematology and oncology (randomized or single arm)-that is trials leading to a marketing authorization. The main steps of the methods are (1) assembling a scientific board; (2) defining the scope, goal and methods through pre-specified, pre-registered and protocolized methodology; (3) preregistration of the protocol; (4) conducting a scoping review of limitations and biases affecting oncology trials and assessing existing scores or methods; (5) developing a list of features to be included and assessed within the framework; (6) assessing each feature through a questionnaire sent to highly cited haematologists and oncologists involved in clinical trials; and (7) finalizing the first version of framework.Not applicable.Our proposal emerged in response to the lack of consideration for key limitations in current trial assessments. The goal is to create a framework specifically designed to assess single trials leading to marketing authorization in the field of oncology and haematogy.

  View details for DOI 10.1111/eci.14267

  View details for PubMedID 38934596

• Individual Patient Data Meta-Analysis Estimates the Minimal Detectable Change of the Geriatric Depression Scale-15. Journal of clinical epidemiology González-Domínguez, N. P., Wu, Y., Fan, S., Levis, B., Sun, Y., Gilbody, S., Ioannidis, J. P., Harel, D., Vigod, S. N., Markham, S., Ziegelstein, R. C., Cuijpers, P., Patten, S. B., Boruff, J. T., Thombs, B. D., Benedetti, A. 2024: 111443

  Abstract

  To use individual participant data meta-analysis (IPDMA) to estimate the minimal detectable change (MDC) of the Geriatric Depression Scale-15 (GDS-15) and to examine whether MDC may differ based on participant characteristics and study-level variables.This was a secondary analysis of data from an IPDMA on the depression screening accuracy of the GDS. Datasets from studies published in any language were eligible for the present study if they included GDS-15 scores for participants aged 60 or older. MDC of the GDS-15 was estimated via random-effects meta-analysis using 2.77 (MDC95) and 1.41 (MDC67) standard errors of measurement (SEM). Subgroup analyses were used to evaluate differences in MDC by participant age and sex. Meta-regression was conducted to assess for differences based on study-level variables, including mean age, proportion male, proportion with major depression, and recruitment setting.5,876 participants (mean age 76 years, 40% male, 11% with major depression) from 21 studies were included. The MDC95 was 3.81 points (95% confidence interval [CI] 3.59, 4.04), and MDC67 was 1.95 (95% CI 1.83, 2.03). The difference in MDC95 was 0.26 points (95% CI 0.04, 0.48) between ≥ 80-year-olds and < 80-year-olds; MDC95 was similar for females and males (0.05, 95% CI -0.12, 0.22). The MDC95 increased by 0.29 points (95% CI 0.17, 0.41) per 10% increase in proportion of participants with major depression; mean age had a small association (0.04 points, 95% CI 0.00 to 0.09) with MDC95, but sex and recruitment setting were not significantly associated.The MDC95 was 3.81 points and MDC67 was 1.95 points. MDC95 increased with the proportion of participants with major depression. Results can be used to evaluate individual changes in depression symptoms and as a threshold for assessing minimal clinical important difference estimates.

  View details for DOI 10.1016/j.jclinepi.2024.111443

  View details for PubMedID 38942179

• Analyses of academician cohorts generate biased pandemic excess death estimates. Journal of clinical epidemiology Ioannidis, J. P. 2024: 111437

  Abstract

  Death data from cohorts of academicians have been used to estimate pandemic excess deaths. We aimed to evaluate the validity of this approach.Data were analyzed from living and deceased member lists from Mainland China, UK and Greece academies; and Nobel laureates (and US subset thereof). Samples of early elected academicians were probed for unrecorded deaths; datasets overtly missing deaths were excluded from further analyses. Actuarial risks were compared against the general population in the same country in respective age strata. Relative incidence risk increases in death in active pandemic periods were compared to population-wide pandemic excess death estimates for the same country.Royal Society and Academy of Athens datasets overtly missed deaths. Pre-pandemic death rates were 4-12-fold lower in the Chinese Academy of Engineering (CAE) versus respective age strata of the Mainland China population. A +158% relative increase in death risk was seen in CAE data during the first 12-months of wide viral spread. Both increases (+34% in British Academy) and decreases (-27% in US Nobel laureates) in death rates occurred in pandemic (2020-22) versus pre-pandemic (2017-2019) years; point estimates were far from known excess deaths in the respective countries (+6% and +14%, respectively). Published excess death estimates for urban-dwelling Mainland China selectively analyzed CAE that had double the pandemic death rates than another Chinese academy (Chinese Academy of Sciences).Missingness, lack of representativeness, large uncertainty, and selective analysis reporting make data from academy rosters unreliable for estimating general population excess deaths.

  View details for DOI 10.1016/j.jclinepi.2024.111437

  View details for PubMedID 38925342

• Comparison of scores on Patient Health Questionnaire-9, Edinburgh Postnatal Depression Scale and Hospital Anxiety and Depression - Depression subscale scores by administration mode: An individual participant data differential item functioning meta-analysis. Journal of affective disorders Harel, D., Wu, Y., Levis, B., Fan, S., Sun, Y., Xu, M., Rice, D. B., Boruff, J., Markham, S., Ioannidis, J. P., Takwoingi, Y., Patten, S. B., Ziegelstein, R. C., Cuijpers, P., Gilbody, S., Vigod, S., Akena, D., Benedetti, A., Thombs, B. D. 2024

  Abstract

  Administration mode of patient-reported outcome measures (PROMs) may influence responses. We assessed if Patient Health Questionnaire-9 (PHQ-9), Edinburgh Postnatal Depression Scale (EPDS) and Hospital Anxiety and Depression Scale - Depression subscale (HADS-D) item responses and scores were associated with administration mode. We compared (1) self-administration versus interview-administration; within self-administration (2) research or medical setting versus private; and (3) pen-and-paper versus electronic; and within interview-administration (4) in-person versus phone. We analysed individual participant data meta-analysis datasets with item-level data for the PHQ-9 (N = 34,529), EPDS (N = 16,813), and HADS-D (N = 16,768). We used multiple indicator multiple cause models to assess differential item functioning (DIF) by administration mode. We found statistically significant DIF for most items on all measures due to large samples, but influence on total scores was negligible. In 10 comparisons conducted across the PHQ-9, EPDS, and HADS-D, Pearson's correlations and intraclass correlation coefficients between latent depression symptom scores from models that did or did not account for DIF were between 0.995 and 1.000. Total PHQ-9, EPDS, and HADS-D scores did not differ materially across administration modes. Researcher and clinicians who evaluate depression symptoms with these questionnaires can select administration methods based on patient preferences, feasibility, or cost.

  View details for DOI 10.1016/j.jad.2024.06.033

  View details for PubMedID 38908554

• Panel stacking is a threat to consensus statement validity. Journal of clinical epidemiology Kepp, K. P., Aavitsland, P., Ballin, M., Balloux, F., Baral, S., Bardosh, K., Bauchner, H., Bendavid, E., Bhopal, R., Blumstein, D. T., Boffetta, P., Bourgeois, F., Brufsky, A., Collignon, P. J., Cripps, S., Cristea, I. A., Curtis, N., Djulbegovic, B., Faude, O., Flacco, M. E., Guyatt, G. H., Hajishengallis, G., Hemkens, L. G., Hoffmann, T., Joffe, A. R., Klassen, T. P., Koletsi, D., Kontoyiannis, D. P., Kuhl, E., La Vecchia, C., Lallukka, T., Lambris, J., Levitt, M., Makridakis, S., Maltezou, H. C., Manzoli, L., Marusic, A., Mavragani, C., Moher, D., Mol, B. W., Muka, T., Naudet, F., Noble, P. W., Nordström, A., Nordström, P., Pandis, N., Papatheodorou, S., Patel, C. J., Petersen, I., Pilz, S., Plesnila, N., Ponsonby, A. L., Rivas, M. A., Saltelli, A., Schabus, M., Schippers, M. C., Schünemann, H., Solmi, M., Stang, A., Streeck, H., Sturmberg, J. P., Thabane, L., Thombs, B. D., Tsakris, A., Wood, S. N., Ioannidis, J. P. 2024: 111428

  Abstract

  Consensus statements can be very influential in medicine and public health. Some of these statements use systematic evidence synthesis but others fail on this front. Many consensus statements use panels of experts to deduce perceived consensus through Delphi processes. We argue that stacking of panel members towards one particular position or narrative is a major threat, especially in absence of systematic evidence review. Stacking may involve financial conflicts of interest, but non-financial conflicts of strong advocacy can also cause major bias. Given their emerging importance, we describe here how such consensus statements may be misleading, by analysing in depth a recent high-impact Delphi consensus statement on COVID-19 recommendations as a case example. We demonstrate that many of the selected panel members and at least 35% of the core panel members had advocated towards COVID-19 elimination (zero-COVID) during the pandemic and were leading members of aggressive advocacy groups. These advocacy conflicts were not declared in the Delphi consensus publication, with rare exceptions. Therefore, we propose that consensus statements should always require rigorous evidence synthesis and maximal transparency on potential biases towards advocacy or lobbyist groups to be valid. While advocacy can have many important functions, its biased impact on consensus panels should be carefully avoided.

  View details for DOI 10.1016/j.jclinepi.2024.111428

  View details for PubMedID 38897481

• Protocol for the development of a reporting guideline for umbrella reviews on epidemiological associations using cross-sectional, case-control and cohort studies: the Preferred Reporting Items for Umbrella Reviews of Cross-sectional, Case-control and Cohort studies (PRIUR-CCC). BMJ open Solmi, M., Cobey, K., Moher, D., Ebrahimzadeh, S., Dragioti, E., Shin, J. I., Radua, J., Cortese, S., Shea, B., Veronese, N., Hartling, L., Pollock, M., Egger, M., Papatheodorou, S., Ioannidis, J. P., Carvalho, A. F. 2024; 14 (6): e071136

  Abstract

  Observational studies are fraught with several biases including reverse causation and residual confounding. Overview of reviews of observational studies (ie, umbrella reviews) synthesise systematic reviews with or without meta-analyses of cross-sectional, case-control and cohort studies, and may also aid in the grading of the credibility of reported associations. The number of published umbrella reviews has been increasing. Recently, a reporting guideline for overviews of reviews of healthcare interventions (Preferred Reporting Items for Overviews of Reviews (PRIOR)) was published, but the field lacks reporting guidelines for umbrella reviews of observational studies. Our aim is to develop a reporting guideline for umbrella reviews on cross-sectional, case-control and cohort studies assessing epidemiological associations.We will adhere to established guidance and prepare a PRIOR extension for systematic reviews of cross-sectional, case-control and cohort studies testing epidemiological associations between an exposure and an outcome, namely Preferred Reporting Items for Umbrella Reviews of Cross-sectional, Case-control and Cohort studies (PRIUR-CCC). Step 1 will be the project launch to identify stakeholders. Step 2 will be a literature review of available guidance to conduct umbrella reviews. Step 3 will be an online Delphi study sampling 100 participants among authors and editors of umbrella reviews. Step 4 will encompass the finalisation of PRIUR-CCC statement, including a checklist, a flow diagram, explanation and elaboration document. Deliverables will be (i) identifying stakeholders to involve according to relevant expertise and end-user groups, with an equity, diversity and inclusion lens; (ii) completing a narrative review of methodological guidance on how to conduct umbrella reviews, a narrative review of methodology and reporting in published umbrella reviews and preparing an initial PRIUR-CCC checklist for Delphi study round 1; (iii) preparing a PRIUR-CCC checklist with guidance after Delphi study; (iv) publishing and disseminating PRIUR-CCC statement.PRIUR-CCC has been approved by The Ottawa Health Science Network Research Ethics Board and has obtained consent (20220639-01H). Participants to step 3 will give informed consent. PRIUR-CCC steps will be published in a peer-reviewed journal and will guide reporting of umbrella reviews on epidemiological associations.

  View details for DOI 10.1136/bmjopen-2022-071136

  View details for PubMedID 38889936

• Expanding the data Ark: an attempt to make the data from highly cited social science papers publicly available ROYAL SOCIETY OPEN SCIENCE Dulitzki, C., Crane, S., Hardwicke, T. E., Ioannidis, J. A. 2024; 11 (5)

  View details for DOI 10.1098/rsos.240016

  View details for Web of Science ID 001222392700005

• Expanding the data Ark: an attempt to make the data from highly cited social science papers publicly available. Royal Society open science Dulitzki, C., Crane, S. M., Hardwicke, T. E., Ioannidis, J. P. 2024; 11 (5): 240016

  Abstract

  Access to scientific data can enable independent reuse and verification; however, most data are not available and become increasingly irrecoverable over time. This study aimed to retrieve and preserve important datasets from 160 of the most highly-cited social science articles published between 2008-2013 and 2015-2018. We asked authors if they would share data in a public repository-the Data Ark-or provide reasons if data could not be shared. Of the 160 articles, data for 117 (73%, 95% CI [67%-80%]) were not available and data for 7 (4%, 95% CI [0%-12%]) were available with restrictions. Data for 36 (22%, 95% CI [16%-30%]) articles were available in unrestricted form: 29 of these datasets were already available and 7 datasets were made available in the Data Ark. Most authors did not respond to our data requests and a minority shared reasons for not sharing, such as legal or ethical constraints. These findings highlight an unresolved need to preserve important scientific datasets and increase their accessibility to the scientific community.

  View details for DOI 10.1098/rsos.240016

  View details for PubMedID 39076822

  View details for PubMedCentralID PMC11285638

• Heterogeneity in elevated glucose and A1C as predictors of the prediabetes to diabetes transition: Framingham Heart Study, Multi-Ethnic Study on Atherosclerosis, Jackson Heart Study, and Atherosclerosis Risk In Communities. medRxiv : the preprint server for health sciences Patel, C. J., Ioannidis, J. P., Gregg, E. W., Vasan, R. S., Manrai, A. K. 2024

  Abstract

  There are a number of glycemic definitions for prediabetes; however, the heterogeneity in diabetes transition rates from prediabetes across different glycemic definitions in major US cohorts has been unexplored. We estimate the variability in risk and relative risk of adiposity based on diagnostic criteria like fasting glucose and hemoglobin A1C% (HA1C%).We estimated transition rate from prediabetes, as defined by fasting glucose between 100-125 and/or 110-125 mg/dL, and HA1C% between 5.7-6.5% in participant data from the Framingham Heart Study, Multi-Ethnic Study on Atherosclerosis, Atherosclerosis Risk in Communities, and the Jackson Heart Study. We estimated the heterogeneity and prediction interval across cohorts, stratifying by age, sex, and body mass index. For individuals who were prediabetic, we estimated the relative risk for obesity, blood pressure, education, age, and sex for diabetes.There is substantial heterogeneity in diabetes transition rates across cohorts and prediabetes definitions with large prediction intervals. We observed the highest range of rates in individuals with fasting glucose of 110-125 mg/dL ranging from 2-18 per 100 person-years. Across different cohorts, the association obesity or hypertension in the progression to diabetes was consistent, yet it varied in magnitude. We provide a database of transition rates across subgroups and cohorts for comparison in future studies.The absolute transition rate from prediabetes to diabetes significantly depends on cohort and prediabetes definitions.

  View details for DOI 10.1101/2024.03.16.24304398

  View details for PubMedID 38562763

  View details for PubMedCentralID PMC10984063

• Consolidated guidance for behavioral intervention pilot and feasibility studies. Pilot and feasibility studies Pfledderer, C. D., von Klinggraeff, L., Burkart, S., da Silva Bandeira, A., Lubans, D. R., Jago, R., Okely, A. D., van Sluijs, E. M., Ioannidis, J. P., Thrasher, J. F., Li, X., Beets, M. W. 2024; 10 (1): 57

  Abstract

  In the behavioral sciences, conducting pilot and/or feasibility studies (PFS) is a key step that provides essential information used to inform the design, conduct, and implementation of a larger-scale trial. There are more than 160 published guidelines, reporting checklists, frameworks, and recommendations related to PFS. All of these publications offer some form of guidance on PFS, but many focus on one or a few topics. This makes it difficult for researchers wanting to gain a broader understanding of all the relevant and important aspects of PFS and requires them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS. The purpose of this study was to develop a consolidated set of considerations for the design, conduct, implementation, and reporting of PFS for interventions conducted in the behavioral sciences.To develop this consolidation, we undertook a review of the published guidance on PFS in combination with expert consensus (via a Delphi study) from the authors who wrote such guidance to inform the identified considerations. A total of 161 PFS-related guidelines, checklists, frameworks, and recommendations were identified via a review of recently published behavioral intervention PFS and backward/forward citation tracking of a well-known PFS literature (e.g., CONSORT Ext. for PFS). Authors of all 161 PFS publications were invited to complete a three-round Delphi survey, which was used to guide the creation of a consolidated list of considerations to guide the design, conduct, and reporting of PFS conducted by researchers in the behavioral sciences.A total of 496 authors were invited to take part in the three-round Delphi survey (round 1, N = 46; round 2, N = 24; round 3, N = 22). A set of twenty considerations, broadly categorized into six themes (intervention design, study design, conduct of trial, implementation of intervention, statistical analysis, and reporting) were generated from a review of the 161 PFS-related publications as well as a synthesis of feedback from the three-round Delphi process. These 20 considerations are presented alongside a supporting narrative for each consideration as well as a crosswalk of all 161 publications aligned with each consideration for further reading.We leveraged expert opinion from researchers who have published PFS-related guidelines, checklists, frameworks, and recommendations on a wide range of topics and distilled this knowledge into a valuable and universal resource for researchers conducting PFS. Researchers may use these considerations alongside the previously published literature to guide decisions about all aspects of PFS, with the hope of creating and disseminating interventions with broad public health impact.

  View details for DOI 10.1186/s40814-024-01485-5

  View details for PubMedID 38582840

  View details for PubMedCentralID PMC10998328

• Requesting conflicts of interest declarations from the European Medicines Agency: 3-year follow-up status. Epidemiology and psychiatric sciences Boesen, K., Gotzsche, P. C., Ioannidis, J. P. 2024; 33: e17

  Abstract

  AIMS: We have previously described the European Medicines Agency's (EMA) and the US Food and Drug Administration's guidelines, each for a specific psychiatric indication, on how to design pivotal drug trials used in new drug applications. Here, we report on our efforts over 3years to retrieve conflicts of interest declarations from EMA. We wanted to assess potential internal industry influence judged as the proportion of guideline committee members with industry conflicts of interest.METHODS: We submitted Freedom of Information requests in February 2020 to access EMA's lists of committee members (and their declared conflicts of interest) involved in drafting the 13 'Clinical efficacy and safety' guidelines available on EMA's website pertaining to psychiatric indications. In our request, we did not specify the exact EMA committees. Here, we describe the received documents and report the proportion of members with industry interests (i.e.defined as any financial industry relationship). It is a follow-up paper to our first report (http://doi.org/10.1017/S2045796021000147).RESULTS: After 2years and 9months (November 2022), the EMA sent us member lists and corresponding conflicts of interest declarations from the Committee for Medicinal Products for Human use (CHMP) from 2012, 2013 and 2017. These member lists pertained to 3 of the 13 requested guidelines (schizophrenia, depression and autism spectrum disorder). The 10 remaining guidelines were published before 2011 and EMA stated that they needed to require permission from their expert members (with unknown retrieval rate) and foresaw excessive workload and long wait. Therefore, we withdrew our request. The CHMPs from 2012, 2013 and 2017 had from 34 to 36 members; 39%-44% declared any interests and we judged 14%-18% as having industry interests. For the schizophrenia guideline, we identified two members with industry interests to companies who submitted feedback on the guideline. We did not receive declarations from the Central Nervous System (CNS) Working Party, the CHMP appointed expert group responsible for drafting and incorporating feedback into the guidelines.CONCLUSIONS: After almost 3years, we received information, which only partly addressed our request. We recommend EMA to improve transparency by publishing the author names and their corresponding conflicts of interest declarations directly in the 'Clinical efficacy and safety' guidelines and to not remove conflicts of interest declarations after 1year from their website to reduce the risk of stealth corporate influence during the development of these influential guidelines.

  View details for DOI 10.1017/S2045796024000179

  View details for PubMedID 38529624

• Is society caught up in a Death Spiral? Modeling societal demise and its reversal. Frontiers in sociology Schippers, M. C., Ioannidis, J. P., Luijks, M. W. 2024; 9: 1194597

  Abstract

  Just like an army of ants caught in an ant mill, individuals, groups and even whole societies are sometimes caught up in a Death Spiral, a vicious cycle of self-reinforcing dysfunctional behavior characterized by continuous flawed decision making, myopic single-minded focus on one (set of) solution(s), denial, distrust, micromanagement, dogmatic thinking and learned helplessness. We propose the term Death Spiral Effect to describe this difficult-to-break downward spiral of societal decline. Specifically, in the current theory-building review we aim to: (a) more clearly define and describe the Death Spiral Effect; (b) model the downward spiral of societal decline as well as an upward spiral; (c) describe how and why individuals, groups and even society at large might be caught up in a Death Spiral; and (d) offer a positive way forward in terms of evidence-based solutions to escape the Death Spiral Effect. Management theory hints on the occurrence of this phenomenon and offers turn-around leadership as solution. On a societal level strengthening of democracy may be important. Prior research indicates that historically, two key factors trigger this type of societal decline: rising inequalities creating an upper layer of elites and a lower layer of masses; and dwindling (access to) resources. Historical key markers of societal decline are a steep increase in inequalities, government overreach, over-integration (interdependencies in networks) and a rapidly decreasing trust in institutions and resulting collapse of legitimacy. Important issues that we aim to shed light on are the behavioral underpinnings of decline, as well as the question if and how societal decline can be reversed. We explore the extension of these theories from the company/organization level to the society level, and make use of insights from both micro-, meso-, and macro-level theories (e.g., Complex Adaptive Systems and collapsology, the study of the risks of collapse of industrial civilization) to explain this process of societal demise. Our review furthermore draws on theories such as Social Safety Theory, Conservation of Resources Theory, and management theories that describe the decline and fall of groups, companies and societies, as well as offer ways to reverse this trend.

  View details for DOI 10.3389/fsoc.2024.1194597

  View details for PubMedID 38533441

  View details for PubMedCentralID PMC10964949

• The Subjective Interpretation of the Medical Evidence. JAMA health forum Bauchner, H., Ioannidis, J. P. 2024; 5 (3): e240213

  View details for DOI 10.1001/jamahealthforum.2024.0213

  View details for PubMedID 38551587

• Smoking Cessation, or How to Avert Half a Billion Premature Deaths - Now. NEJM evidence Ioannidis, J. P., Prochaska, J. J. 2024; 3 (3): EVIDe2300322

  Abstract

  An estimated 1.1 billion people currently smoke cigarettes,1 and 50 to 70% likely will die from tobacco-related causes.2 This translates to 550 to 770 million expected tobacco deaths among those who currently smoke. Many additional deaths will accrue in successive generations if the status quo continues. Of interest is the reversibility of the excess mortality risk of smoking. The meta-analysis by Cho etal.3 of four large national cohorts of nearly 1.5 million adults followed on average 14.8years yielded 23.0 million person-years of observational data with over 120,000 deaths identified through linked death registries.

  View details for DOI 10.1056/EVIDe2300322

  View details for PubMedID 38411449

• Are the Risk of Generalizability Biases Generalizable? A Meta-Epidemiological Study. Research square von Klinggraeff, L., Pfledderer, C. D., Burkart, S., Ramey, K., Smith, M., McLain, A. C., Armstrong, B., Weaver, R. G., Okely, A., Lubans, D., Ioannidis, J. P., Jago, R., Turner-McGrievy, G., Thrasher, J., Li, X., Beets, M. W. 2024

  Abstract

  Preliminary studies (e.g., pilot/feasibility studies) can result in misleading evidence that an intervention is ready to be evaluated in a large-scale trial when it is not. Risk of Generalizability Biases (RGBs, a set of external validity biases) represent study features that influence estimates of effectiveness, often inflating estimates in preliminary studies which are not replicated in larger-scale trials. While RGBs have been empirically established in interventions targeting obesity, the extent to which RGBs generalize to other health areas is unknown. Understanding the relevance of RGBs across health behavior intervention research can inform organized efforts to reduce their prevalence.The purpose of our study was to examine whether RGBs generalize outside of obesity-related interventions.A systematic review identified health behavior interventions across four behaviors unrelated to obesity that follow a similar intervention development framework of preliminary studies informing larger-scale trials (i.e., tobacco use disorder, alcohol use disorder, interpersonal violence, and behaviors related to increased sexually transmitted infections). To be included, published interventions had to be tested in a preliminary study followed by testing in a larger trial (the two studies thus comprising a study pair). We extracted health-related outcomes and coded the presence/absence of RGBs. We used meta-regression models to estimate the impact of RGBs on the change in standardized mean difference (ΔSMD) between the preliminary study and larger trial.We identified sixty-nine study pairs, of which forty-seven were eligible for inclusion in the analysis (k = 156 effects), with RGBs identified for each behavior. For pairs where the RGB was present in the preliminary study but removed in the larger trial the treatment effect decreased by an average of ΔSMD=-0.38 (range - 0.69 to -0.21). This provides evidence of larger drop in effectiveness for studies containing RGBs relative to study pairs with no RGBs present (treatment effect decreased by an average of ΔSMD =-0.24, range - 0.19 to -0.27).RGBs may be associated with higher effect estimates across diverse areas of health intervention research. These findings suggest commonalities shared across health behavior intervention fields may facilitate introduction of RGBs within preliminary studies, rather than RGBs being isolated to a single health behavior field.

  View details for DOI 10.21203/rs.3.rs-3897976/v1

  View details for PubMedID 38464006

  View details for PubMedCentralID PMC10925410

• Large language models for science and medicine. European journal of clinical investigation Telenti, A., Auli, M., Hie, B. L., Maher, C., Saria, S., Ioannidis, J. P. 2024: e14183

  Abstract

  Large language models (LLMs) are a type of machine learning model that learn statistical patterns over text, such as predicting the next words in a sequence of text. Both general purpose and task-specific LLMs have demonstrated potential across diverse applications. Science and medicine have many data types that are highly suitable for LLMs, such as scientific texts (publications, patents and textbooks), electronic medical records, large databases of DNA and protein sequences and chemical compounds. Carefully validated systems that can understand and reason across all these modalities may maximize benefits. Despite the inevitable limitations and caveats of any new technology and some uncertainties specific to LLMs, LLMs have the potential to be transformative in science and medicine.

  View details for DOI 10.1111/eci.14183

  View details for PubMedID 38381530

• Therapeutic interventions increasing seizure risk in multiple sclerosis: resolving discordant meta-analyses. Journal of neurology, neurosurgery, and psychiatry Ioannidis, J. P. 2024

  View details for DOI 10.1136/jnnp-2024-333329

  View details for PubMedID 38383155

• Estimating the extent of selective reporting: An application to economics. Research synthesis methods Bruns, S. B., Deressa, T. K., Stanley, T. D., Doucouliagos, C., Ioannidis, J. P. 2024

  Abstract

  Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reported p-values to range between 57.7% and 71.9% of the significant p-values. The counterfactual p-value distribution also allows us to assess shifts of p-values along the entire distribution of published p-values, revealing that particularly very small p-values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.

  View details for DOI 10.1002/jrsm.1711

  View details for PubMedID 38379427

• Impact of trial attrition rates on treatment effect estimates in chronic inflammatory diseases: A meta-epidemiological study. Research synthesis methods Overgaard, S. H., Moos, C. M., Ioannidis, J. P., Luta, G., Berg, J. I., Nielsen, S. M., Andersen, V., Christensen, R. 2024

  Abstract

  The objective of this meta-epidemiological study was to explore the impact of attrition rates on treatment effect estimates in randomised trials of chronic inflammatory diseases (CID) treated with biological and targeted synthetic disease-modifying drugs. We sampled trials from Cochrane reviews. Attrition rates and primary endpoint results were retrieved from trial publications; Odds ratios (ORs) were calculated from the odds of withdrawing in the experimental intervention compared to the control comparison groups (i.e., differential attrition), as well as the odds of achieving a clinical response (i.e., the trial outcome). Trials were combined using random effects restricted maximum likelihood meta-regression models and associations between estimates of treatment effects and attrition rates were analysed. From 37 meta-analyses, 179 trials were included, and 163 were analysed (301 randomised comparisons; n=62,220 patients). Overall, the odds of withdrawal were lower in the experimental compared to control groups (random effects summary OR=0.45, 95% CI, 0.41-0.50). The corresponding overall treatment effects were large (random effects summary OR=4.43, 95% CI 3.92-4.99) with considerable heterogeneity across interventions and clinical specialties (I2 =85.7%). The ORs estimating treatment effect showed larger treatment benefits when the differential attrition was more prominent with more attrition in the control group (OR=0.73, 95% CI 0.55-0.96). Higher attrition rates from the control arm are associated with larger estimated benefits of treatments with biological or targeted synthetic disease-modifying drugs in CID trials; differential attrition may affect estimates of treatment benefit in randomised trials.

  View details for DOI 10.1002/jrsm.1708

  View details for PubMedID 38351627

• Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics. Research synthesis methods Bartoš, F., Maier, M., Wagenmakers, E. J., Nippold, F., Doucouliagos, H., Ioannidis, J. P., Otte, W. M., Sladekova, M., Deresssa, T. K., Bruns, S. B., Fanelli, D., Stanley, T. D. 2024

  Abstract

  Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

  View details for DOI 10.1002/jrsm.1703

  View details for PubMedID 38327122

• Author Correction: Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials. Nature communications Axfors, C., Schmitt, A. M., Janiaud, P., Van't Hooft, J., Abd-Elsalam, S., Abdo, E. F., Abella, B. S., Akram, J., Amaravadi, R. K., Angus, D. C., Arabi, Y. M., Azhar, S., Baden, L. R., Baker, A. W., Belkhir, L., Benfield, T., Berrevoets, M. A., Chen, C. P., Chen, T. C., Cheng, S. H., Cheng, C. Y., Chung, W. S., Cohen, Y. Z., Cowan, L. N., Dalgard, O., de Almeida E Val, F. F., de Lacerda, M. V., de Melo, G. C., Derde, L., Dubee, V., Elfakir, A., Gordon, A. C., Hernandez-Cardenas, C. M., Hills, T., Hoepelman, A. I., Huang, Y. W., Igau, B., Jin, R., Jurado-Camacho, F., Khan, K. S., Kremsner, P. G., Kreuels, B., Kuo, C. Y., Le, T., Lin, Y. C., Lin, W. P., Lin, T. H., Lyngbakken, M. N., McArthur, C., McVerry, B. J., Meza-Meneses, P., Monteiro, W. M., Morpeth, S. C., Mourad, A., Mulligan, M. J., Murthy, S., Naggie, S., Narayanasamy, S., Nichol, A., Novack, L. A., O'Brien, S. M., Okeke, N. L., Perez, L., Perez-Padilla, R., Perrin, L., Remigio-Luna, A., Rivera-Martinez, N. E., Rockhold, F. W., Rodriguez-Llamazares, S., Rolfe, R., Rosa, R., Røsjø, H., Sampaio, V. S., Seto, T. B., Shahzad, M., Soliman, S., Stout, J. E., Thirion-Romero, I., Troxel, A. B., Tseng, T. Y., Turner, N. A., Ulrich, R. J., Walsh, S. R., Webb, S. A., Weehuizen, J. M., Velinova, M., Wong, H. L., Wrenn, R., Zampieri, F. G., Zhong, W., Moher, D., Goodman, S. N., Ioannidis, J. P., Hemkens, L. G. 2024; 15 (1): 1075

  View details for DOI 10.1038/s41467-024-45360-6

  View details for PubMedID 38316844

• An empirical comparison of statistical methods for multiple cut-off diagnostic test accuracy meta-analysis of the Edinburgh postnatal depression scale (EPDS) depression screening tool using published results vs individual participant data. BMC medical research methodology Negeri, Z. F., Levis, B., Ioannidis, J. P., Thombs, B. D., Benedetti, A., DEPRESsion Screening Data (DEPRESSD) EPDS Group, Sun, Y., He, C., Krishnan, A., Wu, Y., Bhandari, P. M., Neupane, D., Imran, M., Rice, D. B., Azar, M., Chiovitti, M. J., Riehm, K. E., Boruff, J. T., Cuijpers, P., Gilbody, S., Kloda, L. A., Patten, S. B., Ziegelstein, R. C., Markham, S., Comeau, L., Mitchell, N. D., Vigod, S. N., Bakare, M. O., Beck, C. T., Bunevicius, A., Couto, T. C., Chorwe-Sungani, G., Favez, N., Field, S., Garcia-Esteve, L., Honikman, S., Khalifa, D. S., Kohlhoff, J., Kusminskas, L., Kozinszky, Z., Nakic Rados, S., Pawlby, S. J., Rochat, T. J., Sharp, D. J., Smith-Nielsen, J., Su, K., Tadinac, M., Tandon, S. D., Thiagayson, P., Toreki, A., Torres-Gimenez, A., van Heyningen, T., Vega-Dienstmaier, J. M. 2024; 24 (1): 28

  Abstract

  BACKGROUND: Selective reporting of results from only well-performing cut-offs leads to biased estimates of accuracy in primary studies of questionnaire-based screening tools and in meta-analyses that synthesize results. Individual participant data meta-analysis (IPDMA) of sensitivity and specificity at each cut-off via bivariate random-effects models (BREMs) can overcome this problem. However, IPDMA is laborious and depends on the ability to successfully obtain primary datasets, and BREMs ignore the correlation between cut-offs within primary studies.METHODS: We compared the performance of three recent multiple cut-off models developed by Steinhauser et al., Jones et al., and Hoyer and Kuss, that account for missing cut-offs when meta-analyzing diagnostic accuracy studies with multiple cut-offs, to BREMs fitted at each cut-off. We used data from 22 studies of the accuracy of the Edinburgh Postnatal Depression Scale (EPDS; 4475 participants, 758 major depression cases). We fitted each of the three multiple cut-off models and BREMs to a dataset with results from only published cut-offs from each study (published data) and an IPD dataset with results for all cut-offs (full IPD data). We estimated pooled sensitivity and specificity with 95% confidence intervals (CIs) for each cut-off and the area under the curve.RESULTS: Compared to the BREMs fitted to the full IPD data, the Steinhauser et al., Jones et al., and Hoyer and Kuss models fitted to the published data produced similar receiver operating characteristic curves; though, the Hoyer and Kuss model had lower area under the curve, mainly due to estimating slightly lower sensitivity at lower cut-offs. When fitting the three multiple cut-off models to the full IPD data, a similar pattern of results was observed. Importantly, all models had similar 95% CIs for sensitivity and specificity, and the CI width increased with cut-off levels for sensitivity and decreased with an increasing cut-off for specificity, even the BREMs which treat each cut-off separately.CONCLUSIONS: Multiple cut-off models appear to be the favorable methods when only published data are available. While collecting IPD is expensive and time consuming, IPD can facilitate subgroup analyses that cannot be conducted with published data only.

  View details for DOI 10.1186/s12874-023-02134-w

  View details for PubMedID 38302928

• Personalized and longitudinal electronic informed consent in clinical trials: How to move the needle? Digital health De Sutter, E., Barbier, L., Borry, P., Geerts, D., Ioannidis, J. P., Huys, I. 2024; 10: 20552076231222361

  Abstract

  Changes in the clinical trials landscape have been driven by advancements in digital technology. The use of electronic informed consent to inform research participants and to obtain their consent electronically has the potential to improve participant-researcher interactions over time, facilitate clinical trial participation, and increase efficiency in clinical trial conduct. A personalized electronic informed consent platform that enables long-term interactions with the research team could function as a tool to empower participant engagement in clinical trials. However, significant challenges persist impeding successful and widespread implementation. This Perspective provides insights into the opportunities and challenges for the implementation of electronic informed consent in clinical trials. It sets out key recommendations to promote the implementation of this innovative approach to the informed consent process, including the creation of uniform electronic informed consent platforms at regional and national level.

  View details for DOI 10.1177/20552076231222361

  View details for PubMedID 38269372

  View details for PubMedCentralID PMC10807334

• Response to "homeopathy: a null field or effective psychotherapy?". Journal of clinical epidemiology Sigurdson, M. K., Sainani, K. L., Ioannidis, J. P. 2024: 111266

  View details for DOI 10.1016/j.jclinepi.2024.111266

  View details for PubMedID 38266741

• Availability of evidence and comparative effectiveness for surgical versus drug interventions: an overview of systematic reviews and meta-analyses. BMJ open Zavalis, E. A., Rameau, A., Saraswathula, A., Vist, J., Schuit, E., Ioannidis, J. P. 2024; 14 (1): e076675

  Abstract

  This study aims to examine the prevalence of comparisons of surgery to drug regimens, the strength of evidence of such comparisons and whether surgery or the drug intervention was favoured.Systematic review of systematic reviews (umbrella review).Cochrane Database of Systematic Reviews.Systematic reviews attempt to compare surgical to drug interventions.We extracted whether the review found any randomised controlled trials (RCTs) for eligible comparisons. Individual trial results were extracted directly from the systematic review.The outcomes of each meta-analysis were resynthesised into random-effects meta-analyses. Egger's test and excess significance were assessed.Overall, 188 systematic reviews intended to compare surgery versus drugs. Only 41 included data from at least one RCT (total, 165 RCTs) and covered a total of 103 different outcomes of various comparisons of surgery versus drugs. A GRADE assessment was performed by the Cochrane reviewers for 87 (83%) outcomes in the reviews, indicating the strength of evidence was high in 4 outcomes (4%), moderate in 22 (21%), low in 27 (26%) and very low in 33 (32%). Based on 95% CIs, the surgical intervention was favoured in 38/103 (37%), and the drugs were favoured in 13/103 (13%) outcomes. Of the outcomes with high GRADE rating, only one showed conclusive superiority in our reanalysis (sphincterotomy was better than medical therapy for anal fissure). Of the 22 outcomes with moderate GRADE rating, 6 (27%) were inconclusive, 14 (64%) were in favour of surgery and 2 (9%) were in favour of drugs. There was no evidence of excess significance.Though the relative merits of surgical versus drug interventions are important to know for many diseases, high strength randomised evidence is rare. More randomised trials comparing surgery to drug interventions are needed.

  View details for DOI 10.1136/bmjopen-2023-076675

  View details for PubMedID 38195174

• Methods proposed for monitoring the implementation of evidence-based research: a cross-sectional study. Journal of clinical epidemiology Puljak, L., Bala, M. M., Zając, J., Meštrović, T., Buttiġieġ, S., Yanakoulia, M., Briel, M., Lunny, C., Lesniak, W., Poklepović Peričić, T., Alonso-Coello, P., Clarke, M., Djulbegovic, B., Gartlehner, G., Giannakou, K., Glenny, A. M., Glenton, C., Guyatt, G., Hemkens, L. G., Ioannidis, J. P., Jaeschke, R., Juhl Jørgensen, K., Martins-Pfeifer, C. C., Marušić, A., Mbuagbaw, L., Meneses Echavez, J. F., Moher, D., Nussbaumer-Streit, B., Page, M. J., Pérez-Gaxiola, G., Robinson, K. A., Salanti, G., Saldanha, I. J., Savović, J., Thomas, J., Tricco, A. C., Tugwell, P., van Hoof, J., Pieper, D. 2024: 111247

  Abstract

  Evidence-based research (EBR) is the systematic and transparent use of prior research to inform a new study so that it answers questions that matter in a valid, efficient, and accessible manner. This study surveyed experts about existing (e.g. citation analysis) and new methods for monitoring EBR and collected ideas about implementing these methods.We conducted a cross-sectional study via an online survey between November 2022 and March 2023. Participants were experts from the fields of evidence synthesis and research methodology in health research. Open-ended questions were coded by recurring themes; descriptive statistics were used for quantitative questions.Twenty-eight expert participants suggested that citation analysis should be supplemented with content evaluation (not just what is cited, but also in which context), content expert involvement, and assessment of the quality of cited systematic reviews. They also suggested that citation analysis could be facilitated with automation tools. They emphasized that EBR monitoring should be conducted by ethics committees and funding bodies before the research starts. Challenges identified for EBR implementation monitoring were resource constraints and clarity on responsibility for EBR monitoring.Ideas proposed in this study for monitoring the implementation of EBR can be used to refine methods and define responsibility but should be further explored in terms of feasibility and acceptability. Different methods may be needed to determine if the use of EBR is improving over time.

  View details for DOI 10.1016/j.jclinepi.2024.111247

  View details for PubMedID 38185190

• Assessment of transparency indicators in space medicine. PloS one Bellomo, R. K., Zavalis, E. A., Ioannidis, J. P. 2024; 19 (4): e0300701

  Abstract

  Space medicine is a vital discipline with often time-intensive and costly projects and constrained opportunities for studying various elements such as space missions, astronauts, and simulated environments. Moreover, private interests gain increasing influence in this discipline. In scientific disciplines with these features, transparent and rigorous methods are essential. Here, we undertook an evaluation of transparency indicators in publications within the field of space medicine. A meta-epidemiological assessment of PubMed Central Open Access (PMC OA) eligible articles within the field of space medicine was performed for prevalence of code sharing, data sharing, pre-registration, conflicts of interest, and funding. Text mining was performed with the rtransparent text mining algorithms with manual validation of 200 random articles to obtain corrected estimates. Across 1215 included articles, 39 (3%) shared code, 258 (21%) shared data, 10 (1%) were registered, 110 (90%) contained a conflict-of-interest statement, and 1141 (93%) included a funding statement. After manual validation, the corrected estimates for code sharing, data sharing, and registration were 5%, 27%, and 1%, respectively. Data sharing was 32% when limited to original articles and highest in space/parabolic flights (46%). Overall, across space medicine we observed modest rates of data sharing, rare sharing of code and almost non-existent protocol registration. Enhancing transparency in space medicine research is imperative for safeguarding its scientific rigor and reproducibility.

  View details for DOI 10.1371/journal.pone.0300701

  View details for PubMedID 38564591

• Lifting of Embargoes to Data Sharing in Clinical Trials Published in Top Medical Journals. JAMA Siebert, M., Ioannidis, J. P. 2023

  View details for DOI 10.1001/jama.2023.25394

  View details for PubMedID 38153703

• Expert Perspectives on Pilot and Feasibility Studies: A Delphi Study and Consolidation of Considerations for Behavioral Interventions. Research square Pfledderer, C. D., von Klinggraeff, L., Burkart, S., da Silva Bandeira, A., Lubans, D. R., Jago, R., Okely, A. D., van Sluijs, E. M., Ioannidis, J. P., Thrasher, J. F., Li, X., Beets, M. W. 2023

  Abstract

  In the behavioral sciences, conducting pilot and/or feasibility studies (PFS) is a key step that provides essential information used to inform the design, conduct, and implementation of a larger-scale trial. There are more than 160 published guidelines, reporting checklists, frameworks, and recommendations related to PFS. All of these publications offer some form of guidance on PFS, but many focus on one or a few topics. This makes it difficult for researchers wanting to gain a broader understanding of all the relevant and important aspects of PFS and requires them to seek out multiple sources of information, which increases the risk of missing key considerations to incorporate into their PFS. The purpose of this study was to develop a consolidated set of considerations for the design, conduct, implementation, and reporting of PFS for interventions conducted in the behavioral sciences.To develop this consolidation, we undertook a review of the published guidance on PFS in combination with expert consensus (via a Delphi study) from the authors who wrote such guidance to inform the identified considerations. A total of 161 PFS-related guidelines, checklists, frameworks, and recommendations were identified via a review of recently published behavioral intervention PFS and backward/forward citation tracking of well-know PFS literature (e.g., CONSORT Ext. for PFS). Authors of all 161 PFS publications were invited to complete a three-round Delphi survey, which was used to guide the creation of a consolidated list of considerations to guide the design, conduct, and reporting of PFS conducted by researchers in the behavioral sciences.A total of 496 authors were invited to take part in the Delphi survey, 50 (10.1%) of which completed all three rounds, representing 60 (37.3%) of the 161 identified PFS-related guidelines, checklists, frameworks, and recommendations. A set of twenty considerations, broadly categorized into six themes (Intervention Design, Study Design, Conduct of Trial, Implementation of Intervention, Statistical Analysis and Reporting) were generated from a review of the 161 PFS-related publications as well as a synthesis of feedback from the three-round Delphi process. These 20 considerations are presented alongside a supporting narrative for each consideration as well as a crosswalk of all 161 publications aligned with each consideration for further reading.We leveraged expert opinion from researchers who have published PFS-related guidelines, checklists, frameworks, and recommendations on a wide range of topics and distilled this knowledge into a valuable and universal resource for researchers conducting PFS. Researchers may use these considerations alongside the previously published literature to guide decisions about all aspects of PFS, with the hope of creating and disseminating interventions with broad public health impact.

  View details for DOI 10.21203/rs.3.rs-3370077/v1

  View details for PubMedID 38168263

  View details for PubMedCentralID PMC10760234

• Prognostic Biomarkers in Kidney Transplantation: a Systematic Review and Critical Appraisal. Journal of the American Society of Nephrology : JASN Raynaud, M., Al-Awadhi, S., Louis, K., Zhang, H., Su, X., Goutaudier, V., Wang, J., Demir, Z., Wei, Y., Truchot, A., Bouquegneau, A., Del Bello, A., Bailly, E., Lombardi, Y., Maanaoui, M., Giarraputo, A., Naser, S., Divard, G., Aubert, O., Murad, M. H., Wang, C., Liu, L., Bestard, O., Naesens, M., Friedewald, J. J., Lefaucheur, C., Riella, L., Collins, G., Ioannidis, J. P., Loupy, A. 2023

  Abstract

  BACKGROUND: Despite the increasing number of biomarker studies published in the transplant literature over the past 20 years, demonstrations of their clinical benefit and their implementation in routine clinical practice are lacking. We hypothesized that suboptimal design, data, methodology and reporting might contribute to this phenomenon.METHODS: A systematic literature search was performed in PubMed, Embase, Scopus, Web of Science, and Cochrane Library between 1 January 2005 and 12 November 2022 (PROSPERO ID: CRD42020154747). All English language, original studies investigating the association between a biomarker and kidney-allograft outcome were included. The final set of publications was assessed by expert reviewers. After data collection, two independent reviewers randomly evaluated the inconsistencies for 30% of the references for each reviewer. If more than 5% of inconsistencies were observed for one given reviewer, a re-evaluation was conducted for all the references of the reviewer. The biomarkers were categorized according to their type and the biological milieu from which they were measured. The study characteristics related to the design, methods, results, and their interpretation were assessed, as well as reproducible research practices and transparency indicators.RESULTS: A total of 7372 publications were screened and 804 studies met the inclusion criteria. A total of 1143 biomarkers were assessed among the included studies from blood (n=821, 71.8%), intragraft (n=169, 14.8%), or urine (n=81, 7.1%) compartments. The number of studies significantly increased, with a median, yearly number of 31.5 studies (IQR: 23.8-35.5) between 2005 and 2012, and 57.5 (IQR: 53.3-59.8) between 2013 and 2022 (p<0.001). A total of 655 studies (81.5%) were retrospective, while 595 (74.0%) used data from a single center. The median number of patients included was 232 (IQR: 96-629) with a median follow-up posttransplant of 4.8 years (IQR: 3.0-6.2). Only 4.7% of studies were externally validated. A total of 346 studies (43.0%) did not adjust their biomarker for key prognostic factors while only 3.1% of studies adjusted the biomarker for standard-of-care patient monitoring factors. Data sharing, code sharing, and registration occurred in 8.8%, 1.1%, and 4.6% of studies, respectively. A total of 158 studies (20.0%) emphasized the clinical relevance of the biomarker despite the reported nonsignificant association of the biomarker with the outcome measure. A total of 288 studies assessed rejection as an outcome. We showed that these rejection studies shared the same characteristics as other studies.CONCLUSIONS: and Relevance Biomarker studies in kidney transplantation lack validation, rigorous design, methods and interpretation, and transparency. Higher standards in biomarker research may improve the clinical utility and clinical use.

  View details for DOI 10.1681/ASN.0000000000000260

  View details for PubMedID 38053242

• Variability in excess deaths across countries with different vulnerability during 2020-2023. Proceedings of the National Academy of Sciences of the United States of America Ioannidis, J. P., Zonta, F., Levitt, M. 2023; 120 (49): e2309557120

  Abstract

  Excess deaths provide total impact estimates of major crises, such as the COVID-19 pandemic. We evaluated excess death trajectories across countries with accurate death registration and population age structure data and assessed relationships with vulnerability indicators. Using the Human Mortality Database on 34 countries, excess deaths were calculated for 2020-2023 (to week 29, 2023) using 2017-2019 as reference, with adjustment for 5 age strata. Countries were divided into less and more vulnerable; the latter had per capita nominal GDP < $30,000, Gini > 0.35 for income inequality and/or at least ≥2.5% of their population living in poverty. Excess deaths (as proportion of expected deaths, p%) were inversely correlated with per capita GDP (r = -0.60), correlated with proportion living in poverty (r = 0.66), and modestly correlated with income inequality (r = 0.45). Incidence rate ratio for deaths was 1.062 (95% CI, 1.038-1.087) in more versus less vulnerable countries. Excess deaths started deviating in the two groups after the first wave. Between-country heterogeneity diminished gradually within each group. Less vulnerable countries had mean p% = -0.8% and 0.4% in 0-64 and >65-y-old strata. More vulnerable countries had mean p% = 7.0% and 7.2%, respectively. Lower death rates were seen in children of age 0-14 y during 2020-2023 versus prepandemic years. While the pandemic hit some countries earlier than others, country vulnerability dominated eventually the cumulative impact. Half the analyzed countries witnessed no substantial excess deaths versus prepandemic levels, while the others suffered major death tolls.

  View details for DOI 10.1073/pnas.2309557120

  View details for PubMedID 38019858

• In defense of quantitative metrics in researcher assessments. PLoS biology Ioannidis, J. P., Maniadis, Z. 2023; 21 (12): e3002408

  Abstract

  Qualitative assessments of researchers are resource-intensive, untenable in nonmeritocratic settings, and error-prone. Although often derided, quantitative metrics could help improve research practices if they are rigorous, field-adjusted, and centralized.

  View details for DOI 10.1371/journal.pbio.3002408

  View details for PubMedID 38048328

• Effectiveness of a fourth SARS-CoV-2 vaccine dose in previously infected individuals from Austria. European journal of clinical investigation Chalupka, A., Richter, L., Chakeri, A., El-Khatib, Z., Theiler-Schwetz, V., Trummer, C., Krause, R., Willeit, P., Benka, B., Ioannidis, J. P., Pilz, S. 2023: e14136

  Abstract

  Evidence is limited on the effectiveness of a fourth vaccine dose against coronavirus disease 2019 (COVID-19) in populations with prior severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections. We estimated the risk of COVID-19 deaths and SARS-CoV-2 infections according to vaccination status in previously infected individuals in Austria.This is a nationwide retrospective observational study. We calculated age and gender adjusted Cox proportional hazard ratios (HRs) of COVID-19 deaths (primary outcome) and SARS-CoV-2 infections (secondary outcome) from 1 November to 31 December 2022, primarily comparing individuals with four versus three vaccine doses. Relative vaccine effectiveness (rVE) was calculated as (1-HR) X 100.Among 3,986,312 previously infected individuals, 281,291 (7,1%) had four and 1,545,242 (38.8%) had three vaccinations at baseline. We recorded 69 COVID-19 deaths and 89,056 SARS-CoV-2 infections. rVE for four versus three vaccine doses was -24% (95% CI: -120 to 30) against COVID-19 deaths, and 17% (95% CI: 14-19) against SARS-CoV-2 infections. This latter effect rapidly diminished over time and infection risk with four vaccinations was higher compared to less vaccinated individuals during extended follow-up until June 2023. Adjusted HR (95% CI) for all-cause mortality for four versus three vaccinations was 0.79 (0.74-0.85).In previously infected individuals, a fourth vaccination was not associated with COVID-19 death risk, but with transiently reduced risk of SARS-CoV-2 infections and reversal of this effect in longer follow-up. All-cause mortality data suggest healthy vaccinee bias.

  View details for DOI 10.1111/eci.14136

  View details for PubMedID 38032853

• Statistical significance and publication reporting bias in abstracts of reproductive medicine studies. Human reproduction (Oxford, England) Feng, Q., Mol, B. W., Ioannidis, J. P., Li, W. 2023

  Abstract

  What were the frequency and temporal trends of reporting P-values and effect measures in the abstracts of reproductive medicine studies in 1990-2022, how were reported P-values distributed, and what proportion of articles that present with statistical inference reported statistically significant results, i.e. 'positive' results?Around one in six abstracts reported P-values alone without effect measures, while the prevalence of effect measures, whether reported alone or accompanied by P-values, has been increasing, especially in meta-analyses and randomized controlled trials (RCTs); the reported P-values were frequently observed around certain cut-off values, notably at 0.001, 0.01, or 0.05, and among abstracts present with statistical inference (i.e. P-value, CIs, or significant terms), a large majority (77%) reported at least one statistically significant finding.Publishing or reporting only results that show a 'positive' finding causes bias in evaluating interventions and risk factors and may incur adverse health outcomes for patients.Despite efforts to minimize publication reporting bias in medical research, it remains unclear whether the magnitude and patterns of the bias have changed over time.We studied abstracts of reproductive medicine studies from 1990 to 2022. The reproductive medicine studies were published in 23 first-quartile journals under the category of Obstetrics and Gynaecology and Reproductive Biology in Journal Citation Reports and 5 high-impact general medical journals (The Journal of the American Medical Association, The Lancet, The BMJ, The New England Journal of Medicine, and PLoS Medicine). Articles without abstracts, animal studies, and non-research articles, such as case reports or guidelines, were excluded.Automated text-mining was used to extract three types of statistical significance reporting, including P-values, CIs, and text description. Meanwhile, abstracts were text-mined for the presence of effect size metrics and Bayes factors. Five hundred abstracts were randomly selected and manually checked for the accuracy of automatic text extraction. The extracted statistical significance information was then analysed for temporal trends and distribution in general as well as in subgroups of study designs and journals.A total of 24 907 eligible reproductive medicine articles were identified from 170 739 screened articles published in 28 journals. The proportion of abstracts not reporting any statistical significance inference halved from 81% (95% CI, 76-84%) in 1990 to 40% (95% CI, 38-44%) in 2021, while reporting P-values alone remained relatively stable, at 15% (95% CI, 12-18%) in 1990 and 19% (95% CI, 16-22%) in 2021. By contrast, the proportion of abstracts reporting effect measures alone increased considerably from 4.1% (95% CI, 2.6-6.3%) in 1990 to 26% (95% CI, 23-29%) in 2021. Similarly, the proportion of abstracts reporting effect measures together with P-values showed substantial growth from 0.8% (95% CI, 0.3-2.2%) to 14% (95% CI, 12-17%) during the same timeframe. Of 30 182 statistical significance inferences, 56% (n = 17 077) conveyed statistical inferences via P-values alone, 30% (n = 8945) via text description alone such as significant or non-significant, 9.3% (n = 2820) via CIs alone, and 4.7% (n = 1340) via both CI and P-values. The reported P-values (n = 18 417), including both a continuum of P-values and dichotomized P-values, were frequently observed around common cut-off values such as 0.001 (20%), 0.05 (16%), and 0.01 (10%). Of the 13 200 reproductive medicine abstracts containing at least one statistical inference, 77% of abstracts made at least one statistically significant statement. Among articles that reported statistical inference, a decline in the proportion of making at least one statistically significant inference was only seen in RCTs, dropping from 71% (95% CI, 48-88%) in 1990 to 59% (95% CI, 42-73%) in 2021, whereas the proportion in the rest of study types remained almost constant over the years. Of abstracts that reported P-value, 87% (95% CI, 86-88%) reported at least one statistically significant P-value; it was 92% (95% CI, 82-97%) in 1990 and reached its peak at 97% (95% CI, 93-99%) in 2001 before declining to 81% (95% CI, 76-85%) in 2021.First, our analysis focused solely on reporting patterns in abstracts but not full-text papers; however, in principle, abstracts should include condensed impartial information and avoid selective reporting. Second, while we attempted to identify all types of statistical significance reporting, our text mining was not flawless. However, the manual assessment showed that inaccuracies were not frequent.There is a welcome trend that effect measures are increasingly reported in the abstracts of reproductive medicine studies, specifically in RCTs and meta-analyses. Publication reporting bias remains a major concern. Inflated estimates of interventions and risk factors could harm decisions built upon biased evidence, including clinical recommendations and planning of future research.No funding was received for this study. B.W.M. is supported by an NHMRC Investigator grant (GNT1176437); B.W.M. reports research grants and travel support from Merck and consultancy from Merch and ObsEva. W.L. is supported by an NHMRC Investigator Grant (GNT2016729). Q.F. reports receiving a PhD scholarship from Merck. The other author has no conflict of interest to declare.N/A.

  View details for DOI 10.1093/humrep/dead248

  View details for PubMedID 38015794

• Quantitative research assessment: using metrics against gamed metrics. Internal and emergency medicine Ioannidis, J. P., Maniadis, Z. 2023

  Abstract

  Quantitative bibliometric indicators are widely used and widely misused for research assessments. Some metrics have acquired major importance in shaping and rewarding the careers of millions of scientists. Given their perceived prestige, they may be widely gamed in the current "publish or perish" or "get cited or perish" environment. This review examines several gaming practices, including authorship-based, citation-based, editorial-based, and journal-based gaming as well as gaming with outright fabrication. Different patterns are discussed, including massive authorship of papers without meriting credit (gift authorship), team work with over-attribution of authorship to too many people (salami slicing of credit), massive self-citations, citation farms, H-index gaming, journalistic (editorial) nepotism, journal impact factor gaming, paper mills and spurious content papers, and spurious massive publications for studies with demanding designs. For all of those gaming practices, quantitative metrics and analyses may be able to help in their detection and in placing them into perspective. A portfolio of quantitative metrics may also include indicators of best research practices (e.g., data sharing, code sharing, protocol registration, and replications) and poor research practices (e.g., signs of image manipulation). Rigorous, reproducible, transparent quantitative metrics that also inform about gaming may strengthen the legacy and practices of quantitative appraisals of scientific work.

  View details for DOI 10.1007/s11739-023-03447-w

  View details for PubMedID 37921985

  View details for PubMedCentralID 6541803

• Industry Involvement and Transparency in the Most Cited Clinical Trials, 2019-2022. JAMA network open Siena, L. M., Papamanolis, L., Siebert, M. J., Bellomo, R. K., Ioannidis, J. P. 2023; 6 (11): e2343425

  Abstract

  Importance: Industry involvement is prominent in influential clinical trials, and commitments to transparency of trials are highly variable.Objective: To evaluate the modes of industry involvement and the transparency features of the most cited recent clinical trials across medicine.Design, Setting, and Participants: This cross-sectional study was a meta-research assessment including randomized and nonrandomized clinical trials published in 2019 or later. The 600 trials of any type of disease or setting that attracted highest number of citations in Scopus as of December 2022 were selected for analysis. Data were analyzed from March to September 2023.Main Outcomes and Measures: Outcomes of interest were industry involvement (sponsor, author, and analyst) and transparency (protocols, statistical analysis plans, and data and code availability).Results: Among 600 trials with a median (IQR) sample size of 415 (124-1046) participants assessed, 409 (68.2%) had industry funding and 303 (50.5%) were exclusively industry-funded. A total of 354 trials (59.0%) had industry authors, with 280 trials (46.6%) involving industry analysts and 125 trials (20.8%) analyzed exclusively by industry analysts. Among industry-funded trials, 364 (89.0%) reached conclusions favoring the sponsor. Most trials (478 trials [79.7%]) provided a data availability statement, and most indicated intention to share the data, but only 16 trials (2.7%) had data already readily available to others. More than three-quarters of trials had full protocols (482 trials [82.0%]) or statistical analysis plans (446 trials [74.3%]) available, but only 27 trials (4.5%) explicitly mentioned sharing analysis code (8 readily available; 19 on request). Randomized trials were more likely than nonrandomized studies to involve only industry analysts (107 trials [22.9%] vs 18 trials [13.6%]; P=.02) and to have full protocols (405 studies [86.5%] vs 87 studies [65.9%]; P<.001) and statistical analysis plans (373 studies [79.7%] vs 73 studies [55.3%]; P<.001) available. Almost all nonrandomized industry-funded studies (90 of 92 studies [97.8%]) favored the sponsor. Among industry-funded trials, exclusive industry funding (odds ratio, 2.9; 95% CI, 1.5-5.4) and industry-affiliated authors (odds ratio, 2.9; 95% CI, 1.5-5.6) were associated with favorable conclusions for the sponsor.Conclusions and Relevance: This cross-sectional study illustrates how industry involvement in the most influential clinical trials was prominent not only for funding, but also authorship and provision of analysts and was associated with conclusions favoring the sponsor. While most influential trials reported that they planned to share data and make both protocols and statistical analysis plans available, raw data and code were rarely readily available.

  View details for DOI 10.1001/jamanetworkopen.2023.43425

  View details for PubMedID 37962883

• Gender imbalances among top-cited scientists across scientific disciplines over time through the analysis of nearly 5.8 million authors. PLoS biology Ioannidis, J. P., Boyack, K. W., Collins, T. A., Baas, J. 2023; 21 (11): e3002385

  Abstract

  We evaluated how the gender composition of top-cited authors within different subfields of research has evolved over time. We considered 9,071,122 authors with at least 5 full papers in Scopus as of September 1, 2022. Using a previously validated composite citation indicator, we identified the 2% top-cited authors for each of 174 science subfields (Science-Metrix classification) in 4 separate publication age cohorts (first publication pre-1992, 1992 to 2001, 2002 to 2011, and post-2011). Using NamSor, we assigned 3,784,507 authors as men and 2,011,616 as women (for 36.1% gender assignment uncertain). Men outnumbered women 1.88-fold among all authors, decreasing from 3.93-fold to 1.36-fold over time. Men outnumbered women 3.21-fold among top-cited authors, decreasing from 6.41-fold to 2.28-fold over time. In the youngest (post-2011) cohort, 32/174 (18%) subfields had > = 50% women, 97/174 (56%) subfields had > = 30% women, and 3 subfields had = <10% women among the top-cited authors. Gender imbalances in author numbers decreased sharply over time in both high-income countries (including the United States of America) and other countries, but the latter had little improvement in gender imbalances for top-cited authors. In random samples of 100 women and 100 men from the youngest (post-2011) cohort, in-depth assessment showed that most were currently (April 2023) working in academic environments. 32 women and 44 men had some faculty appointment, but only 2 women and 2 men were full professors. Our analysis shows large heterogeneity across scientific disciplines in the amelioration of gender imbalances with more prominent imbalances persisting among top-cited authors and slow promotion pathways even for the most-cited young scientists.

  View details for DOI 10.1371/journal.pbio.3002385

  View details for PubMedID 37988334

• Stakeholder endorsement advancing the implementation of a patient-reported domain for harms in rheumatology clinical trials: Outcome of the OMERACT Safety Working Group. Seminars in arthritis and rheumatism Berthelsen, D. B., Simon, L. S., Ioannidis, J. P., Voshaar, M., Richards, P., Goel, N., Strand, V., Nielsen, S. M., Shea, B. J., Tugwell, P., Bartlett, S. J., Hazlewood, G. S., March, L., Singh, J. A., Suarez-Almazor, M. E., Boers, M., Stevens, R. M., Furst, D. E., Woodworth, T., Leong, A., Brooks, P. M., Flurey, C., Christensen, R., OMERACT Safety Working Group 2023; 63: 152288

  Abstract

  OBJECTIVES: To develop an understanding of the concept of safety/harms experienced by patients involved in clinical trials for their rheumatic and musculoskeletal diseases (RMDs) and to seek input from the OMERACT community before moving forward to developing or selecting an outcome measurement instrument.METHODS: OMERACT 2023 presented and discussed interview results from 34 patients indicating that up to 171 items might be important for patients' harm-reporting.RESULTS: Domain was defined in detail and supported by qualitative work. Participants in the Special-Interest-Group endorsed (96%) that enough qualitative data are available to start Delphi survey(s).CONCLUSION: We present a definition of safety/harms that represents the patient voice (i.e., patients' perception of safety) evaluating the symptomatic treatment-related adverse events for people with RMDs enrolled in clinical trials.

  View details for DOI 10.1016/j.semarthrit.2023.152288

  View details for PubMedID 37918049

• Causes for Retraction in the Biomedical Literature: A Systematic Review of Studies of Retraction Notices. Journal of Korean medical science Hwang, S. Y., Yon, D. K., Lee, S. W., Kim, M. S., Kim, J. Y., Smith, L., Koyanagi, A., Solmi, M., Carvalho, A. F., Kim, E., Shin, J. I., Ioannidis, J. P. 2023; 38 (41): e333

  Abstract

  Many studies have evaluated the prevalence of different reasons for retraction in samples of retraction notices. We aimed to perform a systematic review of such empirical studies of retraction causes.The PubMed/MEDLINE database and the Embase database were searched in June 2023. Eligible studies were those containing sufficient data on the reasons for retraction across samples of examined retracted notices.A 11,181 potentially eligible items were identified, and 43 studies of retractions were included in this systematic review. Studies limited to retraction notices of a specific subspecialty or country, journal/publication type are emerging since 2015. We noticed that the reasons for retraction are becoming more specific and more diverse. In a meta-analysis of 17 studies focused on different subspecialties, misconduct was responsible for 60% (95% confidence interval [CI], 53-67%) of all retractions while error and publication issues contributed to 17% (95% CI, 12-22%) and 9% (95% CI, 6-13%), respectively. The end year of the retraction period in all included studies and the proportion of misconduct presented a weak positive association (coefficient = 1.3% per year, P = 0.002).Misconduct seems to be the most frequently recorded reason for retraction across empirical analyses of retraction notices, but other reasons are not negligible. Greater specificity of causes and standardization is needed in retraction notices.

  View details for DOI 10.3346/jkms.2023.38.e333

  View details for PubMedID 37873630

• Slow data public health. European journal of epidemiology Chiolero, A., Tancredi, S., Ioannidis, J. P. 2023

  Abstract

  Surveillance and research data, despite their massive production, often fail to inform evidence-based and rigorous data-driven health decision-making. In the age of infodemic, as revealed by the COVID-19 pandemic, providing useful information for decision-making requires more than getting more data. Data of dubious quality and reliability waste resources and create data-genic public health damages. We call therefore for a slow data public health, which means focusing, first, on the identification of specific information needs and, second, on the dissemination of information in a way that informs decision-making, rather than devoting massive resources to data collection and analysis. A slow data public health prioritizes better data, ideally population-based, over more data and aims to be timely rather than deceptively fast. Applied by independent institutions with expertise in epidemiology and surveillance methods, it allows a thoughtful and timely public health response, based on high-quality data fostering trustworthiness.

  View details for DOI 10.1007/s10654-023-01049-6

  View details for PubMedID 37789225

  View details for PubMedCentralID 7542265

• Reducing bias in secondary data analysis via an Explore and Confirm Analysis Workflow (ECAW): a proposal and survey of observational researchers. Royal Society open science Thibault, R. T., Kovacs, M., Hardwicke, T. E., Sarafoglou, A., Ioannidis, J. P., Munafò, M. R. 2023; 10 (10): 230568

  Abstract

  Background. Although preregistration can reduce researcher bias and increase transparency in primary research settings, it is less applicable to secondary data analysis. An alternative method that affords additional protection from researcher bias, which cannot be gained from conventional forms of preregistration alone, is an Explore and Confirm Analysis Workflow (ECAW). In this workflow, a data management organization initially provides access to only a subset of their dataset to researchers who request it. The researchers then prepare an analysis script based on the subset of data, upload the analysis script to a registry, and then receive access to the full dataset. ECAWs aim to achieve similar goals to preregistration, but make access to the full dataset contingent on compliance. The present survey aimed to garner information from the research community where ECAWs could be applied-employing the Avon Longitudinal Study of Parents and Children (ALSPAC) as a case example. Methods. We emailed a Web-based survey to researchers who had previously applied for access to ALSPAC's transgenerational observational dataset. Results. We received 103 responses, for a 9% response rate. The results suggest that-at least among our sample of respondents-ECAWs hold the potential to serve their intended purpose and appear relatively acceptable. For example, only 10% of respondents disagreed that ALSPAC should run a study on ECAWs (versus 55% who agreed). However, as many as 26% of respondents agreed that they would be less willing to use ALSPAC data if they were required to use an ECAW (versus 45% who disagreed). Conclusion. Our data and findings provide information for organizations and individuals interested in implementing ECAWs and related interventions. Preregistration. https://osf.io/g2fw5 Deviations from the preregistration are outlined in electronic supplementary material A.

  View details for DOI 10.1098/rsos.230568

  View details for PubMedID 37830032

  View details for PubMedCentralID PMC10565389

• Recommendations on data sharing in HIV drug resistance research. PLoS medicine Inzaule, S. C., Siedner, M. J., Little, S. J., Avila-Rios, S., Ayitewala, A., Bosch, R. J., Calvez, V., Ceccherini-Silberstein, F., Charpentier, C., Descamps, D., Eshleman, S. H., Fokam, J., Frenkel, L. M., Gupta, R. K., Ioannidis, J. P., Kaleebu, P., Kantor, R., Kassaye, S. G., Kosakovsky Pond, S. L., Kouamou, V., Kouyos, R. D., Kuritzkes, D. R., Lessells, R., Marcelin, A., Mbuagbaw, L., Minalga, B., Ndembi, N., Neher, R. A., Paredes, R., Pillay, D., Raizes, E. G., Rhee, S., Richman, D. D., Ruxrungtham, K., Sabeti, P. C., Schapiro, J. M., Sirivichayakul, S., Steegen, K., Sugiura, W., van Zyl, G. U., Vandamme, A., Wensing, A. M., Wertheim, J. O., Gunthard, H. F., Jordan, M. R., Shafer, R. W. 2023; 20 (9): e1004293

  Abstract

  Author summary Human immunodeficiency virus (HIV) drug resistance has implications for antiretroviral treatment strategies and for containing the HIV pandemic because the development of HIV drug resistance leads to the requirement for antiretroviral drugs that may be less effective, less well-tolerated, and more expensive than those used in first-line regimens. HIV drug resistance studies are designed to determine which HIV mutations are selected by antiretroviral drugs and, in turn, how these mutations affect antiretroviral drug susceptibility and response to future antiretroviral treatment regimens. Such studies collectively form a vital knowledge base essential for monitoring global HIV drug resistance trends, interpreting HIV genotypic tests, and updating HIV treatment guidelines. Although HIV drug resistance data are collected in many studies, such data are often not publicly shared, prompting the need to recommend best practices to encourage and standardize HIV drug resistance data sharing. In contrast to other viruses, sharing HIV sequences from phylogenetic studies of transmission dynamics requires additional precautions as HIV transmission is criminalized in many countries and regions. Our recommendations are designed to ensure that the data that contribute to HIV drug resistance knowledge will be available without undue hardship to those publishing HIV drug resistance studies and without risk to people living with HIV.

  View details for DOI 10.1371/journal.pmed.1004293

  View details for PubMedID 37738247

• Peer review and scientific publication at a crossroads. BMJ (Clinical research ed.) Ioannidis, J. P., Berkwits, M., Flanagin, A., Bloom, T. 2023; 382: p1992

  View details for DOI 10.1136/bmj.p1992

  View details for PubMedID 37739425

• Twelve years after the ARRIVE guidelines: Animal research has not yet arrived at high standards. Laboratory animals Song, J., Solmi, M., Carvalho, A. F., Shin, J. I., Ioannidis, J. P. 2023: 236772231181658

  Abstract

  The reproducibility crisis across animal studies jeopardizes the credibility of the main findings derived from animal research, even though these findings are critical for informing human studies. To clarify and improve transparency among animal studies, the ARRIVE reporting guidelines were first announced in 2010 and upgraded to version 2.0 in 2020. However, compliance with and awareness of those reporting guidelines has remained suboptimal. Journal editors should encourage the authors to adhere to those guidelines. Authors, editors, referees, and reviewers should be aware of the ARRIVE guideline 2.0 when assessing and evaluating the methodology and findings of animal studies. However, we should also question whether reporting guidelines alone can change a research culture and improve the reproducibility of animal investigations. Reported research may not reflect actual research. Large segments of animal research efforts are wasted because of poor design choices and because of non-publication rather than suboptimal reporting. Better training of the scientific workforce, interventions at improving animal research at the design stage, registration practices, and alignment of the reward system with the publication of rigorous animal research may achieve more than reporting guidelines alone.

  View details for DOI 10.1177/00236772231181658

  View details for PubMedID 37728936

• Machine learning augmentation reduces prediction error in collective forecasting: development and validation across prediction markets with application to COVID events. EBioMedicine Gruen, A., Mattingly, K. R., Morwitch, E., Bossaerts, F., Clifford, M., Nash, C., Ioannidis, J. P., Ponsonby, A. 2023; 96: 104783

  Abstract

  BACKGROUND: The recent COVID-19 pandemic highlighted the challenges for traditional forecasting. Prediction markets are a promising way to generate collective forecasts and could potentially be enhanced if high-quality crowdsourced inputs were identified and preferentially weighted for likely accuracy in real-time with machine learning.METHODS: We aim to leverage human prediction markets with real-time machine weighting of likely higher accuracy trades to improve performance. The crowd sourced Almanis prediction market longitudinal platform (n=1822) and Next Generation Social Science (NGS2) platform (n=103) were utilised.FINDINGS: A 43-feature model predicted accurate forecasters, those with top quintile relative Brier accuracy, with subsequent replication in two out-of-sample datasets (pboth <1*10-9). Trades graded by this model as having higher accuracy scores than others produced a greater AUC temporal gain in the overall market after vs before trade. Accuracy score-weighted forecasts had higher accuracy than market forecasts alone, particularly when the two systems disagreed by 5% or more for binary event prediction: the hybrid system demonstrating substantial % AUC gains of 13.2%, p=1.35*10-14 and 13.8%, p=0.003 in two out-of-sample datasets. When discordant, the hybrid model was correct for COVID-19 event occurrence 72.7% of the time vs 27.3% for market models, p=0.007. This net classification benefit was replicated in the separate Almanis B dataset, p=2.4*10-7.INTERPRETATION: Real-time machine classification followed by weighting human trades according to likely accuracy improves collective forecasting performance. This could provide improved anticipation of and thus response to emerging risks.FUNDING: This work was supported by an AusIndustry R and D tax incentive program from the Department of Industry, Science, Energy and Resources, Australia, to SlowVoice Pty Ltd. (IR 2101990) and Fellowship (GNT 1110200) and Investigator grant (GNT 1197234) to A-L Ponsonby by the National Health and Medical Research Council of Australia.

  View details for DOI 10.1016/j.ebiom.2023.104783

  View details for PubMedID 37708701

• Few randomised trials in preterm birth prevention meet predefined usefulness criteria. Journal of clinical epidemiology van 't Hooft, J., van Dijk, C. E., Axfors, C., Alfirevic, Z., Oudijk, M. A., Mol, B. W., Bossuyt, P. M., Ioannidis, J. P. 2023

  Abstract

  OBJECTIVE: We operationalized a research usefulness tool identified through literature searches and consensus and examined if randomised controlled trials (RCTs) addressing preterm birth prevention met predefined criteria for usefulness.STUDY DESIGN AND SETTING: The usefulness tool included eight criteria combining 13 items. RCTs were evaluated for compliance with each item by multiple assessors (reviewer agreement 95-98%). Proportions of compliances with 95% confidence interval (CI) were calculated and change over time was assessed using ≧ 2010 as a cut-off.RESULTS: Among 347 selected RCTs, published within 56 preterm birth Cochrane reviews, only 36 (10%, 95% CI 7-14%) met more than half of the usefulness criteria. Compared to trials before 2010, recent trials used composite or surrogate (less informative) outcomes more often (13% vs 25%, relative risk 1.91, 95% CI 1.21-3.00). Only 16 trials reflected real practice (pragmatism) in design (5%, 95% CI 3-7%), with no improvements over time. No trials reported involvement of mothers to reflect patients' research priorities and outcomes selection. Recent trials were more transparent.CONCLUSION: Few preterm birth prevention RCTs met more than half of the usefulness criteria but most of usefulness criteria are improving after 2010. Use of informative outcomes, patient centeredness, pragmatism and transparency should be key targets for future research planning.

  View details for DOI 10.1016/j.jclinepi.2023.08.016

  View details for PubMedID 37657614

• Balancing risks and benefits of cannabis use: umbrella review of meta-analyses of randomised controlled trials and observational studies. BMJ (Clinical research ed.) Solmi, M., De Toffol, M., Kim, J. Y., Choi, M. J., Stubbs, B., Thompson, T., Firth, J., Miola, A., Croatto, G., Baggio, F., Michelon, S., Ballan, L., Gerdle, B., Monaco, F., Simonato, P., Scocco, P., Ricca, V., Castellini, G., Fornaro, M., Murru, A., Vieta, E., Fusar-Poli, P., Barbui, C., Ioannidis, J. P., Carvalho, A. F., Radua, J., Correll, C. U., Cortese, S., Murray, R. M., Castle, D., Shin, J. I., Dragioti, E. 2023; 382: e072348

  Abstract

  To systematically assess credibility and certainty of associations between cannabis, cannabinoids, and cannabis based medicines and human health, from observational studies and randomised controlled trials (RCTs).Umbrella review.PubMed, PsychInfo, Embase, up to 9 February 2022.Systematic reviews with meta-analyses of observational studies and RCTs that have reported on the efficacy and safety of cannabis, cannabinoids, or cannabis based medicines were included. Credibility was graded according to convincing, highly suggestive, suggestive, weak, or not significant (observational evidence), and by GRADE (Grading of Recommendations, Assessment, Development and Evaluations) (RCTs). Quality was assessed with AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews 2). Sensitivity analyses were conducted.101 meta-analyses were included (observational=50, RCTs=51) (AMSTAR 2 high 33, moderate 31, low 32, or critically low 5). From RCTs supported by high to moderate certainty, cannabis based medicines increased adverse events related to the central nervous system (equivalent odds ratio 2.84 (95% confidence interval 2.16 to 3.73)), psychological effects (3.07 (1.79 to 5.26)), and vision (3.00 (1.79 to 5.03)) in people with mixed conditions (GRADE=high), improved nausea/vomit, pain, spasticity, but increased psychiatric, gastrointestinal adverse events, and somnolence among others (GRADE=moderate). Cannabidiol improved 50% reduction of seizures (0.59 (0.38 to 0.92)) and seizure events (0.59 (0.36 to 0.96)) (GRADE=high), but increased pneumonia, gastrointestinal adverse events, and somnolence (GRADE=moderate). For chronic pain, cannabis based medicines or cannabinoids reduced pain by 30% (0.59 (0.37 to 0.93), GRADE=high), across different conditions (n=7), but increased psychological distress. For epilepsy, cannabidiol increased risk of diarrhoea (2.25 (1.33 to 3.81)), had no effect on sleep disruption (GRADE=high), reduced seizures across different populations and measures (n=7), improved global impression (n=2), quality of life, and increased risk of somnolence (GRADE=moderate). In the general population, cannabis worsened positive psychotic symptoms (5.21 (3.36 to 8.01)) and total psychiatric symptoms (7.49 (5.31 to 10.42)) (GRADE=high), negative psychotic symptoms, and cognition (n=11) (GRADE=moderate). In healthy people, cannabinoids improved pain threshold (0.74 (0.59 to 0.91)), unpleasantness (0.60 (0.41 to 0.88)) (GRADE=high). For inflammatory bowel disease, cannabinoids improved quality of life (0.34 (0.22 to 0.53) (GRADE=high). For multiple sclerosis, cannabinoids improved spasticity, pain, but increased risk of dizziness, dry mouth, nausea, somnolence (GRADE=moderate). For cancer, cannabinoids improved sleep disruption, but had gastrointestinal adverse events (n=2) (GRADE=moderate). Cannabis based medicines, cannabis, and cannabinoids resulted in poor tolerability across various conditions (GRADE=moderate). Evidence was convincing from observational studies (main and sensitivity analyses) in pregnant women, small for gestational age (1.61 (1.41 to 1.83)), low birth weight (1.43 (1.27 to 1.62)); in drivers, car crash (1.27 (1.21 to 1.34)); and in the general population, psychosis (1.71 (1.47 to 2.00)). Harmful effects were noted for additional neonatal outcomes, outcomes related to car crash, outcomes in the general population including psychotic symptoms, suicide attempt, depression, and mania, and impaired cognition in healthy cannabis users (all suggestive to highly suggestive).Convincing or converging evidence supports avoidance of cannabis during adolescence and early adulthood, in people prone to or with mental health disorders, in pregnancy and before and while driving. Cannabidiol is effective in people with epilepsy. Cannabis based medicines are effective in people with multiple sclerosis, chronic pain, inflammatory bowel disease, and in palliative medicine but not without adverse events.PROSPERO CRD42018093045.None.

  View details for DOI 10.1136/bmj-2022-072348

  View details for PubMedID 37648266

• Variability in excess deaths across countries with different vulnerability during 2020-2023. medRxiv : the preprint server for health sciences Ioannidis, J. P., Zonta, F., Levitt, M. 2023

  Abstract

  Excess deaths provide total impact estimates of major crises, such as the COVID-19 pandemic. We evaluated excess death's trajectories during 2020-2023 across countries with accurate death registration and population age structure data; and assessed relationships with economic indicators of vulnerability. Using the Human Mortality Database on 34 countries, excess deaths were calculated for 2020-2023 (to week 29, 2023) using 2017-2019 as reference, with weekly expected death calculations and adjustment for 5 age strata. Countries were divided into less and more vulnerable; the latter had per capita nominal GDP<$30,000, Gini>0.35 for income inequality and/or at least 2.5% of their population living in poverty. Excess deaths (as proportion of expected deaths, p%) were inversely correlated with per capita GDP (r=-0.60), correlated with proportion living in poverty (r=0.66) and modestly correlated with income inequality (r=0.45). Incidence rate ratio for deaths was 1.06 (95% confidence interval, 1.04-1.08) in the more versus less vulnerable countries. Excess deaths started deviating in the two groups after the first wave. Between-country heterogeneity diminished over time within each of the two groups. Less vulnerable countries had mean p%=-0.8% and 0.4% in 0-64 and >65 year-old strata while more vulnerable countries had mean p%=7.0% and 7.2%, respectively. Usually lower death rates were seen in children 0-14 years old during 2020-2023 versus pre-pandemic years. While the pandemic hit some countries earlier than others, country vulnerability dominated eventually the cumulative impact. Half of the analyzed countries witnessed no substantial excess deaths versus pre-pandemic levels, while the other half suffered major death tolls.

  View details for DOI 10.1101/2023.04.24.23289066

  View details for PubMedID 37162934

  View details for PubMedCentralID PMC10168510

• Threats and Opportunities Associated With Rapid Growth of Mega-Journals Reply JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Ioannidis, J. A., Pezzullo, A., Boccia, S. 2023; 330 (7): 663

  View details for Web of Science ID 001061435200028

  View details for PubMedID 37581676

• Threats and Opportunities Associated With Rapid Growth of Mega-Journals-Reply. JAMA Ioannidis, J. P., Pezzullo, A. M., Boccia, S. 2023; 330 (7): 663

  View details for DOI 10.1001/jama.2023.10780

  View details for PubMedID 37581676

• Principles for good scholarship in systematic reviews. Developmental medicine and child neurology Kolaski, K., Romeiser Logan, L., Ioannidis, J. P. 2023

  Abstract

  Many sources document problems that jeopardize the trustworthiness of systematic reviews. This is a major concern given their potential to influence patient care and impact people's lives. Responsibility for producing trustworthy conclusions on the evidence in systematic reviews is borne primarily by authors who need the necessary training and resources to correctly report on the current knowledge base. Peer reviewers and editors are also accountable; they must ensure that systematic reviews are accurate by demonstrating proper methods. To support all these stakeholders, we attempt to distill the sprawling guidance that is currently available in our recent co-publication about best tools and practices for systematic reviews. We specifically address how to meet methodological conduct standards applicable to key components of systematic reviews. In this complementary invited review, we place these standards in the context of good scholarship principles for systematic review development. Our intention is to reach a broad audience and potentially improve the trustworthiness of evidence syntheses published in the developmental medicine literature and beyond.

  View details for DOI 10.1111/dmcn.15719

  View details for PubMedID 37528533

• Impact of major awards on the subsequent work of their recipients. Royal Society open science Nepomuceno, A., Bayer, H., Ioannidis, J. P. 2023; 10 (8): 230549

  Abstract

  To characterize the impact of major research awards on recipients' subsequent work, we studied Nobel Prize winners in Chemistry, Physiology or Medicine, and Physics and MacArthur Fellows working in scientific fields. Using a case-crossover design, we compared scientists' citations, publications and citations-per-publication from work published in a 3-year pre-award period to their work published in a 3-year post-award period. Nobel Laureates and MacArthur Fellows received fewer citations for post- than for pre-award work. This was driven mostly by Nobel Laureates. Median decrease was 80.5 citations among Nobel Laureates (p = 0.004) and 2 among MacArthur Fellows (p = 0.857). Mid-career (42-57 years) and senior (greater than 57 years) researchers tended to earn fewer citations for post-award work. Early career researchers (less than 42 years, typically MacArthur Fellows) tended to earn more, but the difference was non-significant. MacArthur Fellows (p = 0.001) but not Nobel Laureates (p = 0.180) had significantly more post-award publications. Both populations had significantly fewer post-award citations per paper (p = 0.043 for Nobel Laureates, 0.005 for MacArthur Fellows, and 0.0004 for combined population). If major research awards indeed fail to increase (and even decrease) recipients' impact, one may need to reassess the purposes, criteria, and impacts of awards to improve the scientific enterprise.

  View details for DOI 10.1098/rsos.230549

  View details for PubMedID 37564070

• Mapping and systematic appraisal of umbrella reviews in epidemiological research: a protocol for a meta-epidemiological study. Systematic reviews Belbasis, L., Brooker, R. D., Zavalis, E., Pezzullo, A. M., Axfors, C., Ioannidis, J. P. 2023; 12 (1): 123

  Abstract

  Umbrella review is one of the terms used to describe an overview of systematic reviews. During the last years, a rapid increase in the number of umbrella reviews on epidemiological studies has been observed, but there is no systematic assessment of their methodological and reporting characteristics. Our study aims to fill this gap by performing a systematic mapping of umbrella reviews in epidemiological research.We will perform a meta-epidemiological study including a systematic review in MEDLINE and EMBASE to identify all the umbrella reviews that focused on systematic reviews of epidemiological studies and were published from inception until December 31, 2022. We will consider eligible any research article which was designed as an umbrella review and summarized systematic reviews and meta-analyses of epidemiological studies. From each eligible article, we will extract information about the research topic, the methodological characteristics, and the reporting characteristics. We will examine whether the umbrella reviews assessed the strength of the available evidence and the rigor of the included systematic reviews. We will also examine whether these characteristics change across time.Our study will systematically appraise the methodological and reporting characteristics of published umbrella reviews in epidemiological literature. The findings of our study can be used to improve the design and conduct of future umbrella reviews, to derive a standardized set of reporting and methodological guidelines for umbrella reviews, and to allow further meta-epidemiological work.osf.io/sxzc6.

  View details for DOI 10.1186/s13643-023-02265-7

  View details for PubMedID 37452309

  View details for PubMedCentralID PMC10347720

• Paired associated SARS-CoV-2 spike variable positions: a network analysis approach to emerging variants. mSystems Manoussopoulos, Y., Anastassopoulou, C., Ioannidis, J. P., Tsakris, A. 2023: e0044023

  Abstract

  Amino acids in variable positions of proteins may be correlated, with potential structural and functional implications. Here, we apply exact tests of independence in R * C contingency tables to examine noise-free associations between variable positions of the SARS-CoV-2 spike protein, using as a paradigm sequences from Greece deposited in GISAID (N = 6,683/1,078 full length) for the period 29 February 2020 to 26 April 2021 that essentially covers the first three pandemic waves. We examine the fate and complexity of these associations by network analysis, using associated positions (exact P ≤ 0.001 and Average Product Correction ≥ 2) as links and the corresponding positions as nodes. We found a temporal linear increase of positional differences and a gradual expansion of the number of position associations over time, represented by a temporally evolving intricate web, resulting in a non-random complex network of 69 nodes and 252 links. Overconnected nodes corresponded to the most adapted variant positions in the population, suggesting a direct relation between network degree and position functional importance. Modular analysis revealed 25 k-cliques comprising 3 to 11 nodes. At different k-clique resolutions, one to four communities were formed, capturing epistatic associations of circulating variants (Alpha, Beta, B.1.1.318), but also Delta, which dominated the evolutionary landscape later in the pandemic. Cliques of aminoacidic positional associations tended to occur in single sequences, enabling the recognition of epistatic positions in real-world virus populations. Our findings provide a novel way of understanding epistatic relationships in viral proteins with potential applications in the design of virus control procedures. IMPORTANCE Paired positional associations of adapted amino acids in virus proteins may provide new insights for understanding virus evolution and variant formation. We investigated potential intramolecular relationships between variable SARS-CoV-2 spike positions by exact tests of independence in R * C contingency tables, having applied Average Product Correction (APC) to eliminate background noise. Associated positions (exact P ≤ 0.001 and APC ≥ 2) formed a non-random, epistatic network of 25 cliques and 1-4 communities at different clique resolutions, revealing evolutionary ties between variable positions of circulating variants and a predictive potential of previously unknown network positions. Cliques of different sizes represented theoretical combinations of changing residues in sequence space, allowing the identification of significant aminoacidic combinations in single sequences of real-world populations. Our analytic approach that links network structural aspects to mutational aminoacidic combinations in the spike sequence population offers a novel way to understand virus epidemiology and evolution.

  View details for DOI 10.1128/msystems.00440-23

  View details for PubMedID 37432011

• Epidemiological characteristics and prevalence rates of research reproducibility across disciplines: a scoping review of articles published in 2018-2019. eLife Cobey, K. D., Fehlmann, C. A., Christ Franco, M., Ayala, A. P., Sikora, L., Rice, D. B., Xu, C., Ioannidis, J. P., Lalu, M. M., Ménard, A., Neitzel, A., Nguyen, B., Tsertsvadze, N., Moher, D. 2023; 12

  Abstract

  Introduction: Reproducibility is a central tenant of research. We aimed to synthesize the literature on reproducibility and describe its epidemiological characteristics, including how reproducibility is defined and assessed. We also aimed to determine and compare estimates for reproducibility across different fields. Methods: We conducted a scoping review to identify English language replication studies published between 2018-2019 in economics, education, psychology, health sciences and biomedicine. We searched Medline, Embase, PsycINFO, Cumulative Index of Nursing and Allied Health Literature - CINAHL, Education Source via EBSCOHost, ERIC, EconPapers, International Bibliography of the Social Sciences (IBSS), and EconLit. Documents retrieved were screened in duplicate against our inclusion criteria. We extracted year of publication, number of authors, country of affiliation of the corresponding author, and whether the study was funded. For the individual replication studies, we recorded whether a registered protocol for the replication study was used, whether there was contact between the reproducing team and the original authors, what study design was used, and what the primary outcome was. Finally, we recorded how reproducibilty was defined by the authors, and whether the assessed study(ies) successfully reproduced based on this definition. Extraction was done by a single reviewer and quality controlled by a second reviewer. Results: Our search identified 11,224 unique documents, of which 47 were included in this review. Most studies were related to either psychology (48.6%) or health sciences (23.7%). Among these 47 documents, 36 described a single reproducibility study while the remaining 11 reported at least two reproducibility studies in the same paper. Less than the half of the studies referred to a registered protocol. There was variability in the definitions of reproduciblity success. In total, across the 47 documents 177 studies were reported. Based on the definition used by the author of each study, 95 of 177 (53.7%) studies reproduced. Conclusion: This study gives an overview of research across five disciplines that explicitly set out to reproduce previous research. Such reproducibility studies are extremely scarce, the definition of a successfully reproduced study is ambiguous, and the reproducibility rate is overall modest. Funding: No external funding was received for this work.

  View details for DOI 10.7554/eLife.78518

  View details for PubMedID 37341380

• Pre-pandemic cross-reactive humoral immunity to SARS-CoV-2 in Africa: systematic review and meta-analysis. International journal of infectious diseases : IJID : official publication of the International Society for Infectious Diseases Ioannidis, J. P., Contopoulos-Ioannidis, D. G. 2023

  Abstract

  To assess the evidence on the presence of antibodies cross-reactive with SARS-CoV-2 antigens in pre-pandemic samples from African populations.We performed a systematic review and meta-analysis of studies evaluating pre-pandemic African samples using pre-set assay-specific thresholds for SARS-CoV-2 seropositivity.26 articles with 156 datasets were eligible, including 3,437 positives among 29,923 measurements (11.5%) with large between-dataset heterogeneity. Positivity was similar for anti-N (14%) and anti-S antibodies (11%), higher for anti-S1 (23%) and lower for anti-RBD antibodies (7%). Positivity was similar, on average, for IgM and IgG. Positivity was seen prominently in countries where malaria transmission occurs throughout and in datasets enriched in malaria cases (14%, 95% CI, 12-15% versus 2%, 95% CI 1-2% in other datasets). Substantial SARS-CoV-2 reactivity was seen in high malaria burden with or without high dengue burden (14% and 12%, respectively), and not without high malaria burden (2% and 0%, respectively). Lower SARS-CoV-2 cross-reactivity was seen in settings of high HIV seroprevalence. More sparse individual-level data showed associations of higher SARS-CoV-2 cross-reactivity with Plasmodium parasitemia and lower SARS-CoV-2 cross-reactivity with HIV seropositivity.Pre-pandemic samples from Africa show high levels of anti-SARS-CoV-2 seropositivity. At country level, cross-reactivity tracks especially with malaria prevalence.

  View details for DOI 10.1016/j.ijid.2023.06.009

  View details for PubMedID 37327857

• Inverse publication reporting bias favouring null, negative results. BMJ evidence-based medicine Ioannidis, J. P. 2023

  View details for DOI 10.1136/bmjebm-2023-112292

  View details for PubMedID 37315987

• Guidance to best tools and practices for systematic reviews. Systematic reviews Kolaski, K., Logan, L. R., Ioannidis, J. P. 2023; 12 (1): 96

  Abstract

  Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

  View details for DOI 10.1186/s13643-023-02255-9

  View details for PubMedID 37291658

  View details for PubMedCentralID PMC10248995

• Guidance to best tools and practices for systematic reviews. BMC infectious diseases Kolaski, K., Logan, L. R., Ioannidis, J. P. 2023; 23 (1): 383

  Abstract

  Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

  View details for DOI 10.1186/s12879-023-08304-x

  View details for PubMedID 37286949

  View details for PubMedCentralID PMC10247272

• Guidance to best tools and practices for systematic reviews. Acta anaesthesiologica Scandinavica Kolaski, K., Logan, L. R., Ioannidis, J. P. 2023

  Abstract

  Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

  View details for DOI 10.1111/aas.14295

  View details for PubMedID 37288997

• Guidance to best tools and practices for systematic reviews. JBI evidence synthesis Kolaski, K., Logan, L. R., Ioannidis, J. P. 2023

  Abstract

  Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

  View details for DOI 10.11124/JBIES-23-00139

  View details for PubMedID 37282594

• Guidance to best tools and practices for systematic reviews. British journal of pharmacology Kolaski, K., Logan, L. R., Ioannidis, J. P. 2023

  Abstract

  Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

  View details for DOI 10.1111/bph.16100

  View details for PubMedID 37282770

• What did COVID-19 really teach us about science, evidence and society? Journal of evaluation in clinical practice Saltelli, A., Sturmberg, J. P., Sarewitz, D., Ioannidis, J. P. 2023

  View details for DOI 10.1111/jep.13876

  View details for PubMedID 37282738

• Peer review before trial conduct could increase research value and reduce waste. Journal of clinical epidemiology Siebert, M., Naudet, F., Ioannidis, J. P. 2023

  Abstract

  Traditional peer-review of clinical trials happens too late, after the trials are already done. However, lack of methodological rigor and presence of many biases can be detected and remedied in advance. Here, we examine several options for review and improvement of trials before their conduct: protocol review by peers, sponsors, regulatory authorities, and institutional ethical committees; registration in registry sites; deposition of protocol and/or the statistical analysis plan in a public repository; peer-review and publication of the protocol and/or the statistical analysis plan in a journal; and Registered Reports. Some practices are considered standard (e.g. registration in trial registry), while others are still uncommon but are becoming more frequent (e.g. publication of full trial protocols and statistical analysis plans). Ongoing challenges hinder a large-scale implementation of some promising practices such as Registered Reports. Innovative ideas are necessary to advance peer-review efficiency and rigor in clinical trials but also to lower the cumulative burden for peer-reviewers. We make several suggestions to enhance pre-conduct peer-review. Making all steps of research process public and open may reverse siloed environments. Pre-conduct peer-review may be improved by making routinely publicly available all protocols that have gone through review by institutional review boards and regulatory agencies.

  View details for DOI 10.1016/j.jclinepi.2023.05.024

  View details for PubMedID 37286150

• Guidance to Best Tools and Practices for Systematic Reviews. JBJS reviews Kolaski, K., Logan, L. R., Ioannidis, J. P. 2023; 11 (6)

  Abstract

  » Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.» A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.» Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

  View details for DOI 10.2106/JBJS.RVW.23.00077

  View details for PubMedID 37285444

• Published registry-based pharmacoepidemiologic associations show limited concordance with agnostic medication-wide analyses. Journal of clinical epidemiology Axfors, C., Patel, C. J., Ioannidis, J. P. 2023

  Abstract

  To assess how the results of published national registry-based pharmacoepidemiology studies (where select associations are of interest) compare with an agnostic medication-wide approach (where all possible drug associations are tested).We systematically searched for publications that reported drug associations with any, breast, colon/colorectal, or prostate cancer in the Swedish Prescribed Drug Registry. Results were compared against a previously performed agnostic medication-wide study on the same registry.https://osf.io/kqj8n RESULTS: Most published studies (25/32) investigated previously reported associations. 421/913 (46%) associations had statistically significant results. 134 of the 162 unique drug-cancer associations could be paired with 70 associations in the agnostic study (corresponding drug categories and cancer types). Published studies reported smaller effect sizes and absolute effect sizes than the agnostic study, and generally used more adjustments. Agnostic analyses were less likely to report statistically significant protective associations (based on a multiplicity-corrected threshold) than their paired associations in published studies (McNemar odds ratio 0.13, p=0.0022). Among 162 published associations, 36 (22%) showed increased risk signal and 25 (15%) protective signal at p<0.05, while for agnostic associations, 237 (11%) showed increased risk signal and 108 (5%) protective signal at a multiplicity-corrected threshold. Associations belonging to drug categories targeted by individual published studies vs non-targeted had smaller average effect sizes; smaller p-values; and more frequent risk signals.Published pharmacoepidemiology studies using a national registry addressed mostly previously proposed associations, were mostly "negative", and showed only modest concordance with their respective agnostic analyses in the same registry.

  View details for DOI 10.1016/j.jclinepi.2023.05.014

  View details for PubMedID 37224981

• A 7-Step Guideline for Qualitative Synthesis and Meta-Analysis of Observational Studies in Health Sciences. Public health reviews Glisic, M., Raguindin, P. F., Gemperli, A., Taneri, P. E., Salvador, D. J., Voortman, T., Marques Vidal, P., Papatheodorou, S. I., Kunutsor, S. K., Bano, A., Ioannidis, J. P., Muka, T. 2023; 44: 1605454

  Abstract

  Objectives: To provide a step-by-step, easy-to-understand, practical guide for systematic review and meta-analysis of observational studies. Methods: A multidisciplinary team of researchers with extensive experience in observational studies and systematic review and meta-analysis was established. Previous guidelines in evidence synthesis were considered. Results: There is inherent variability in observational study design, population, and analysis, making evidence synthesis challenging. We provided a framework and discussed basic meta-analysis concepts to assist reviewers in making informed decisions. We also explained several statistical tools for dealing with heterogeneity, probing for bias, and interpreting findings. Finally, we briefly discussed issues and caveats for translating results into clinical and public health recommendations. Our guideline complements "A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research" and addresses peculiarities for observational studies previously unexplored. Conclusion: We provided 7 steps to synthesize evidence from observational studies. We encourage medical and public health practitioners who answer important questions to systematically integrate evidence from observational studies and contribute evidence-based decision-making in health sciences.

  View details for DOI 10.3389/phrs.2023.1605454

  View details for PubMedID 37260612

  View details for PubMedCentralID PMC10227668

• What Really Happened During the Massive SARS-CoV-2 Omicron Wave in China? JAMA internal medicine Ioannidis, J. P., Zonta, F., Levitt, M. 2023

  Abstract

  This Viewpoint discusses reports from China after its zero COVID-19 policy ended in December 2022.

  View details for DOI 10.1001/jamainternmed.2023.1547

  View details for PubMedID 37184847

• Improving systematic reviews: guidance on guidance and other options and challenges. Journal of clinical epidemiology Kolaski, K., Logan, L. R., Ioannidis, J. P. 2023

  Abstract

  Multiple guideline tools are available for systematic reviews. These tools intend to standardize protocol development, require comprehensive reporting, and improve methodological rigor, including risk of bias assessment of primary studies and appraisal of the conduct of the reviews themselves. We recently published a guidance paper concerning these instruments and we hope that it will prove useful to producers, appraisers, and users of systematic reviews. There are still numerous open frontiers in improving systematic reviews. These include but are not limited to training of systematic reviewers; education of peer-reviewers, editors, and publishers; improving funder-based incentives, diminishing redundancy, increasing transparency, requiring protocol registration, confirming reporting and conduct standards, and establishing expectations of current meta-analysis methods. Each of these issues has caveats and challenges. Moreover, too many influential reviews continue to be non-systematic and expert opinion based. We need to understand why these reviews continue to be favored in the literature. Additional opportunities and need for research arise in the connection between primary evidence and systematic reviews. In some cases, the two may become indistinguishable. Living reviews become increasingly attractive in the currently evolving research circumstances but require additional safeguards. The connection between systematic reviews and guidelines or other implementation and decision-making tools is transitioning as well. Guidance efforts gain increasing attention, and may indeed help improve evidence synthesis but proper meta-research is needed to rigorously assess any improvements. We should maximize the contribution of systematic reviews, but also reduce the chances of producing checklist-heavy, ritual-burdened documents of questionable utility.

  View details for DOI 10.1016/j.jclinepi.2023.05.008

  View details for PubMedID 37196861

• Transparency in infectious disease research: meta-research survey of specialty journals. The Journal of infectious diseases Zavalis, E. A., Contopoulos-Ioannidis, D. G., Ioannidis, J. P. 2023

  Abstract

  BACKGROUND: Infectious diseases carry large global burdens and have implications for society at large. Therefore, reproducible, transparent research is extremely important.METHODS: We evaluated transparency indicators (code and data sharing, registration, conflict and funding disclosures) in the 5340 PubMed Central Open Access articles published in 2019 or 2021 in the 9 most-cited specialty journals in infectious disease using the text-mining R package, rtransparent.RESULTS: 5340 articles were evaluated (1860 published in 2019 and 3480 in 2021 (of which 1828 on COVID-19)). Text-mining identified code sharing in 98 (2%) articles, data sharing in 498 (9%), registration in 446 (8%), conflict of interest disclosures in 4209 (79%) and funding disclosures in 4866 (91%). There were substantial differences across the 9 journals: 1-9% for code sharing, 5-25% for data sharing, 1-31% for registration, 7-100% for conflicts of interest, and 65-100% for funding disclosures. Validation-corrected imputed estimates were 3%, 11%, 8%, 79% and 92%, respectively. There were no major differences between articles published in 2019 and non-COVID-19 articles in 2021. In 2021, non-COVID-19 articles had more data sharing (12%) than COVID-19 articles (4%).CONCLUSIONS: Data sharing, code sharing, and registration are very uncommon in infectious disease specialty journals. Increased transparency is required.

  View details for DOI 10.1093/infdis/jiad130

  View details for PubMedID 37132475

• CONSORT Harms 2022 statement, explanation, and elaboration: updated guideline for the reporting of harms in randomised trials. BMJ (Clinical research ed.) Junqueira, D. R., Zorzela, L., Golder, S., Loke, Y., Gagnier, J. J., Julious, S. A., Li, T., Mayo-Wilson, E., Pham, B., Phillips, R., Santaguida, P., Scherer, R. W., Gøtzsche, P. C., Moher, D., Ioannidis, J. P., Vohra, S. 2023; 381: e073725

  View details for DOI 10.1136/bmj-2022-073725

  View details for PubMedID 37094878

• CONSORT Harms 2022 statement, explanation, and elaboration: updated guideline for the reporting of harms in randomized trials. Journal of clinical epidemiology Junqueira, D. R., Zorzela, L., Golder, S., Loke, Y., Gagnier, J. J., Julious, S. A., Li, T., Mayo-Wilson, E., Pham, B., Phillips, R., Santaguida, P., Scherer, R. W., Gotzsche, P. C., Moher, D., Ioannidis, J. P., Vohra, S., CONSORT Harms Group, Evans, S., Fonfa, A., Lang, T. A., Loder, E., Weeks, L. 2023

  Abstract

  Randomized controlled trials remain the reference standard for healthcare research on effects of interventions, and the need to report both benefits and harms is essential. The Consolidated Standards of Reporting Trials (the main CONSORT) statement includes one item on reporting harms (i.e., all important harms or unintended effects in each group). In 2004, the CONSORT group developed the CONSORT Harms extension; however, it has not been consistently applied and needs to be updated. Here, we describe CONSORT Harms 2022, which replaces the CONSORT Harms 2004 checklist, and shows how CONSORT Harms 2022 items could be incorporated into the main CONSORT checklist. Thirteen items from the main CONSORT were modified to improve harms reporting. Three new items were added. In this article, we describe CONSORT Harms 2022 and how it was integrated into the main CONSORT checklist and elaborate on each item relevant to complete reporting of harms in randomized controlled trials. Until future work from the CONSORT group produces an updated checklist, authors, journal reviewers, and editors of randomized controlled trials should use the integrated checklist presented in this paper.

  View details for DOI 10.1016/j.jclinepi.2023.04.005

  View details for PubMedID 37100738

• Flaws and uncertainties in pandemic global excess death calculations. European journal of clinical investigation Ioannidis, J. P., Zonta, F., Levitt, M. 2023: e14008

  Abstract

  Several teams have been publishing global estimates of excess deaths during the COVID-19 pandemic. Here, we examine potential flaws and underappreciated sources of uncertainty in global excess death calculations. Adjusting for changing population age structure is essential. Otherwise, excess deaths are markedly overestimated in countries with increasingly aging populations. Adjusting for changes in other high-risk indicators, such as residence in long-term facilities, may also make a difference. Death registration is highly incomplete in most countries; completeness corrections should allow for substantial uncertainty and consider that completeness may have changed during pandemic years. Excess death estimates have high sensitivity to modeling choice. Therefore different options should be considered and the full range of results should be shown for different choices of pre-pandemic reference periods and imposed models. Any post-modeling corrections in specific countries should be guided by pre-specified rules. Modeling of all-cause mortality (ACM) in countries that have ACM data and extrapolating these models to other countries is precarious; models may lack transportability. Existing global excess death estimates underestimate the overall uncertainty that is multiplicative across diverse sources of uncertainty. Informative excess death estimates require risk stratification, including age groups and ethnic/racial strata. Data to-date suggest a death deficit among children during the pandemic and marked socioeconomic differences in deaths, widening inequalities. Finally, causal explanations require great caution in disentangling SARS-CoV-2 deaths, indirect pandemic effects, and effects from measures taken. We conclude that excess deaths have many uncertainties, but globally deaths from SARS-CoV-2 may be the minority of calculated excess deaths.

  View details for DOI 10.1111/eci.14008

  View details for PubMedID 37067255

• Excess death estimates from multiverse analysis in 2009-2021. European journal of epidemiology Levitt, M., Zonta, F., Ioannidis, J. P. 2023

  Abstract

  Excess death estimates have great value in public health, but they can be sensitive to analytical choices. Here we propose a multiverse analysis approach that considers all possible different time periods for defining the reference baseline and a range of 1 to 4 years for the projected time period for which excess deaths are calculated. We used data from the Human Mortality Database on 33 countries with detailed age-stratified death information on an annual basis during the period 2009-2021. The use of different time periods for reference baseline led to large variability in the absolute magnitude of the exact excess death estimates. However, the relative ranking of different countries compared to others for specific years remained largely unaltered. The relative ranking of different years for the specific country was also largely independent of baseline. Averaging across all possible analyses, distinct time patterns were discerned across different countries. Countries had declines between 2009 and 2019, but the steepness of the decline varied markedly. There were also large differences across countries on whether the COVID-19 pandemic years 2020-2021 resulted in an increase of excess deaths and by how much. Consideration of longer projected time windows resulted in substantial shrinking of the excess deaths in many, but not all countries. Multiverse analysis of excess deaths over long periods of interest can offer an approach that better accounts for the uncertainty in estimating expected mortality patterns, comparative mortality trends across different countries, and the nature of observed mortality peaks.

  View details for DOI 10.1007/s10654-023-00998-2

  View details for PubMedID 37043153

  View details for PubMedCentralID 9225924

• Analysis of fatality impact and seroprevalence surveys in a community sustaining a SARS-CoV-2 superspreading event. Scientific reports Richter, E., Liebl, D., Schulte, B., Lehmann, N., Fuhrmann, C., Jöckel, K. H., Ioannidis, J. P., Streeck, H. 2023; 13 (1): 5440

  Abstract

  There is an ongoing debate on the COVID-19 infection fatality rate (IFR) and the impact of COVID-19 on overall population mortality. Here, we addressed these issues in a community in Germany with a major superspreader event analyzing deaths over time and auditing death certificates in the community.18 deaths that occurred within the first six months of the pandemic had a positive test for SARS-CoV-2. Six out of 18 deaths had non-COVID-19 related causes of death (COD). Individuals with COVID-19 COD typically died of respiratory failure (75%) and tended to have fewer reported comorbidities (p = 0.029). Duration between first confirmed infection and death was negatively associated with COVID-19 being COD (p = 0.04). Repeated seroprevalence essays in a cross-sectional epidemiological study showed modest increases in seroprevalence over time, and substantial seroreversion (30%). IFR estimates accordingly varied depending on COVID-19 death attribution. Careful ascertainment of COVID-19 deaths is important in understanding the impact of the pandemic.

  View details for DOI 10.1038/s41598-023-32441-7

  View details for PubMedID 37012282

  View details for PubMedCentralID PMC10069345

• Feasibility indicators in obesity-related behavioral intervention preliminary studies: a historical scoping review. Pilot and feasibility studies Pfledderer, C. D., von Klinggraeff, L., Burkart, S., Wolfenden, L., Ioannidis, J. P., Beets, M. W. 2023; 9 (1): 46

  Abstract

  BACKGROUND: Behavioral interventions are often complex, operate at multiple levels, across settings, and employ a range of behavior change techniques. Collecting and reporting key indicators of initial trial and intervention feasibility is essential to decisions for progressing to larger-scale trials. The extent of reporting on feasibility indicators and how this may have changed over time is unknown. The aims of this study were to (1) conduct a historical scoping review of the reporting of feasibility indicators in behavioral pilot/feasibility studies related to obesity published through 2020, and (2) describe trends in the amount and type of feasibility indicators reported in studies published across three time periods: 1982-2006, 2011-2013, and 2018-2020.METHODS: A search of online databases (PubMed, Embase, EBSCOhost, Web of Science) for health behavior pilot/feasibility studies related to obesity published up to 12/31/2020 was conducted and a random sample of 600 studies, 200 from each of the three timepoints (1982-2006, 2011-2013, and 2018-2020), was included in this review. The presence/absence of feasibility indicators, including recruitment, retention, participant acceptability, attendance, compliance, and fidelity, were identified/coded for each study. Univariate logistic regression models were employed to assess changes in the reporting of feasibility indicators across time.RESULTS: A total of 16,365 unique articles were identified of which 6873 of these were reviewed to arrive at the final sample of 600 studies. For the total sample, 428 (71.3%) studies provided recruitment information, 595 (99.2%) provided retention information, 219 (36.5%) reported quantitative acceptability outcomes, 157 (26.2%) reported qualitative acceptability outcomes, 199 (33.2%) reported attendance, 187 (31.2%) reported participant compliance, 23 (3.8%) reported cost information, and 85 (14.2%) reported treatment fidelity outcomes. When compared to the Early Group (1982-2006), studies in the Late Group (2018-2020) were more likely to report recruitment information (OR=1.60, 95%CI 1.03-2.49), acceptability-related quantitative (OR=2.68, 95%CI 1.76-4.08) and qualitative (OR=2.32, 95%CI 1.48-3.65) outcomes, compliance outcomes (OR=2.29, 95%CI 1.49-3.52), and fidelity outcomes (OR=2.13, 95%CI 1.21, 3.77).CONCLUSION: The reporting of feasibility indicators within behavioral pilot/feasibility studies has improved across time, but key aspects of feasibility, such as fidelity, are still not reported in the majority of studies. Given the importance of behavioral intervention pilot/feasibility studies in the translational science spectrum, there is a need for improving the reporting of feasibility indicators.

  View details for DOI 10.1186/s40814-023-01270-w

  View details for PubMedID 36949541

• Individual participant data meta-analysis to compare EPDS accuracy to detect major depression with and without the self-harm item. Scientific reports Qiu, X., Wu, Y., Sun, Y., Levis, B., Tian, J., Boruff, J. T., Cuijpers, P., Ioannidis, J. P., Markham, S., Ziegelstein, R. C., Vigod, S. N., Benedetti, A., Thombs, B. D., DEPRESsion Screening Data (DEPRESSD) EPDS Group, He, C., Krishnan, A., Bhandari, P. M., Neupane, D., Negeri, Z., Imran, M., Rice, D. B., Azar, M., Chiovitti, M. J., Gilbody, S., Kloda, L. A., Patten, S. B., Mitchell, N. D., Alvarado, R., Barnes, J., Beck, C. T., Bindt, C., Correa, H., Couto, T. C., Chorwe-Sungani, G., Eapen, V., Favez, N., Felice, E., Fellmeth, G., Fernandes, M., Field, S., Figueiredo, B., Fisher, J. R., Green, E. P., Honikman, S., Howard, L. M., Kettunen, P. A., Kohlhoff, J., Kozinszky, Z., Leonardou, A. A., Maes, M., Martinez, P., Rados, S. N., Nishi, D., Pawlby, S. J., Rochat, T. J., Rowe, H. J., Sharp, D. J., Skalkidou, A., Smith-Nielsen, J., Stein, A., Su, K., Sundstrom-Poromaa, I., Tadinac, M., Tandon, S. D., Tendais, I., Toreki, A., Tran, T. D., Trevillion, K., Turner, K., Vaver, M. S., van Heyningen, T., Vega-Dienstmaier, J. M., Wynter, K., Yonkers, K. A. 2023; 13 (1): 4026

  Abstract

  Item 10 of the Edinburgh Postnatal Depression Scale (EPDS) is intended to assess thoughts of intentional self-harm but may also elicit concerns about accidental self-harm. It does not specifically address suicide ideation but, nonetheless, is sometimes used as an indicator of suicidality. The 9-item version of the EPDS (EPDS-9), which omits item 10, is sometimes used in research due to concern about positive endorsements of item 10 and necessary follow-up. We assessed the equivalence of total score correlations and screening accuracy to detect major depression using the EPDS-9 versus full EPDS among pregnant and postpartum women. We searched Medline, Medline In-Process and Other Non-Indexed Citations, PsycINFO, and Web of Science from database inception to October 3, 2018 for studies that administered the EPDS and conducted diagnostic classification for major depression based on a validated semi-structured or fully structured interview among women aged 18 or older during pregnancy or within 12months of giving birth. We conducted an individual participant data meta-analysis. We calculated Pearson correlations with 95% prediction interval (PI) between EPDS-9 and full EPDS total scores using a random effects model. Bivariate random-effects models were fitted to assess screening accuracy. Equivalence tests were done by comparing the confidence intervals (CIs) around the pooled sensitivity and specificity differences to the equivalence margin of delta=0.05. Individual participant data were obtained from 41 eligible studies (10,906 participants, 1407 major depression cases). The correlation between EPDS-9 and full EPDS scores was 0.998 (95% PI 0.991, 0.999). For sensitivity, the EPDS-9 and full EPDS were equivalent for cut-offs 7-12 (difference range -0.02, 0.01) and the equivalence was indeterminate for cut-offs 13-15 (all differences -0.04). For specificity, the EPDS-9 and full EPDS were equivalent for all cut-offs (difference range 0.00, 0.01). The EPDS-9 performs similarly to the full EPDS and can be used when there are concerns about the implications of administering EPDS item 10.Trial registration: The original IPDMA was registered in PROSPERO (CRD42015024785).

  View details for DOI 10.1038/s41598-023-29114-w

  View details for PubMedID 36899016

• Unrestricted weighted least squares represent medical research better than random effects in 67,308 Cochrane meta-analyses. Journal of clinical epidemiology Stanley, T. D., Ioannidis, J. P., Maier, M., Doucouliagos, H., Otte, W. M., Bartoš, F. 2023

  Abstract

  To evaluate how well meta-analysis mean estimators represent reported medical research and establish which meta-analysis method is better using widely accepted model selection measures: Akaike information criterion (AIC) and Bayesian information criterion (BIC).We compiled 67,308 meta-analyses from the Cochrane Database of Systematic Reviews (CDSR) published between 1997 and 2020, collectively encompassing nearly 600,000 medical findings. We compared unrestricted weighted least squares (UWLS) versus random effects (RE); fixed effect (FE) was also secondarily considered.The probability that a randomly selected systematic review from the CDSR would favor UWLS over RE is 79.4% (CI95%: 79.1; 79.7). The odds ratio that a Cochrane systematic review would substantially favor UWLS over RE is 9.33 (CI95%: 8.94; 9.73) using the conventional criterion that a difference in AIC (or BIC) of two or larger represents a 'substantial' improvement. UWLS's advantage over RE is most prominent in the presence of low heterogeneity. However, UWLS also has a notable advantage in high heterogeneity research, across different sizes of meta-analyses and types of outcomes.UWLS frequently dominates RE in medical research, often substantially. Thus, the unrestricted weighted least squares should be reported routinely in the meta-analysis of clinical trials.

  View details for DOI 10.1016/j.jclinepi.2023.03.004

  View details for PubMedID 36889450

• Ten (not so) simple rules for clinical trial data-sharing. PLoS computational biology Pellen, C., Le Louarn, A., Spurrier-Bernard, G., Decullier, E., Chretien, J., Rosenthal, E., Le Goff, G., Moher, D., Ioannidis, J. P., Naudet, F. 2023; 19 (3): e1010879

  Abstract

  Clinical trial data-sharing is seen as an imperative for research integrity and is becoming increasingly encouraged or even required by funders, journals, and other stakeholders. However, early experiences with data-sharing have been disappointing because they are not always conducted properly. Health data is indeed sensitive and not always easy to share in a responsible way. We propose 10 rules for researchers wishing to share their data. These rules cover the majority of elements to be considered in order to start the commendable process of clinical trial data-sharing: Rule 1: Abide by local legal and regulatory data protection requirementsRule 2: Anticipate the possibility of clinical trial data-sharing before obtaining fundingRule 3: Declare your intent to share data in the registration stepRule 4: Involve research participantsRule 5: Determine the method of data accessRule 6: Remember there are several other elements to shareRule 7: Do not proceed aloneRule 8: Deploy optimal data management to ensure that the data shared is usefulRule 9: Minimize risksRule 10: Strive for excellence.

  View details for DOI 10.1371/journal.pcbi.1010879

  View details for PubMedID 36893146

• How do we increase the trustworthiness of medical publications? Fertility and sterility Mol, B. W., Ioannidis, J. P. 2023

  Abstract

  Trustworthiness of medical publications can depend on either good faith or verifiable data. Most medical publications to-date have been advertisements, some form of scholarly boasting. The authors practically announce to the world that they did some research. In good faith, other scientists as well as practitioners of medicine, guideline developers, and patients are asked to take these advertisements seriously, buy into them, and make important (occasionally life-or-death) decisions based on what they say. However, the raw data are usually not made available. Other crucial parts that would allow to verify the research, including the code, detailed protocols and statistical analysis plans have also been uncommonly shared - or they may not exist. Under such circumstances, is faith misplaced when one accepts that the work presented is real?

  View details for DOI 10.1016/j.fertnstert.2023.02.023

  View details for PubMedID 36842709

• Medical advertisements and scientific journals: Time for editors and publishers to take a stance. Journal of evaluation in clinical practice Boesen, K., Ioannidis, J. P. 2023

  View details for DOI 10.1111/jep.13816

  View details for PubMedID 36808410

• Influence of pilot and small trials in meta-analyses of behavioral interventions: a meta-epidemiological study. Systematic reviews Beets, M. W., Weaver, R. G., Ioannidis, J. P., Pfledderer, C. D., Jones, A., von Klinggraeff, L., Armstrong, B. 2023; 12 (1): 21

  Abstract

  BACKGROUND: Pilot/feasibility or studies with small sample sizes may be associated with inflated effects. This study explores the vibration of effect sizes (VoE) in meta-analyses when considering different inclusion criteria based upon sample size or pilot/feasibility status.METHODS: Searches were to identify systematic reviews that conducted meta-analyses of behavioral interventions on topics related to the prevention/treatment of childhood obesity from January 2016 to October 2019. The computed summary effect sizes (ES) were extracted from each meta-analysis. Individual studies included in the meta-analyses were classified into one of the following four categories: self-identified pilot/feasibility studies or based upon sample size but not a pilot/feasibility study (N≤100, N>100, and N>370 the upper 75th of sample size). The VoE was defined as the absolute difference (ABS) between the re-estimations of summary ES restricted to study classifications compared to the originally reported summary ES. Concordance (kappa) of statistical significance of summary ES between the four categories of studies was assessed. Fixed and random effects models and meta-regressions were estimated. Three case studies are presented to illustrate the impact of including pilot/feasibility and N≤100 studies on the estimated summary ES.RESULTS: A total of 1602 effect sizes, representing 145 reported summary ES, were extracted from 48 meta-analyses containing 603 unique studies (avg. 22 studies per meta-analysis, range 2-108) and included 227,217 participants. Pilot/feasibility and N≤100 studies comprised 22% (0-58%) and 21% (0-83%) of studies included in the meta-analyses. Meta-regression indicated the ABS between the re-estimated and original summary ES where summary ES ranged from 0.20 to 0.46 depending on the proportion of studies comprising the original ES were either mostly small (e.g., N≤100) or mostly large (N>370). Concordance was low when removing both pilot/feasibility and N≤100 studies (kappa=0.53) and restricting analyses only to the largest studies (N>370, kappa=0.35), with 20% and 26% of the originally reported statistically significant ES rendered non-significant. Reanalysis of the three case study meta-analyses resulted in the re-estimated ES rendered either non-significant or half of the originally reported ES.CONCLUSIONS: When meta-analyses of behavioral interventions include a substantial proportion of both pilot/feasibility and N≤100 studies, summary ES can be affected markedly and should be interpreted with caution.

  View details for DOI 10.1186/s13643-023-02184-7

  View details for PubMedID 36803891

• Implementing clinical trial data sharing requires training a new generation of biomedical researchers. Nature medicine Mansmann, U., Locher, C., Prasser, F., Weissgerber, T., Sax, U., Posch, M., Decullier, E., Cristea, I. A., Debray, T. P., Held, L., Moher, D., Ioannidis, J. P., Ross, J. S., Ohmann, C., Naudet, F. 2023

  View details for DOI 10.1038/s41591-022-02080-y

  View details for PubMedID 36732626

• Incidence, Risk, and Severity of SARS-CoV-2 Reinfections in Children and Adolescents Between March 2020 and July 2022 in Serbia. JAMA network open Medic, S., Anastassopoulou, C., Lozanov-Crvenkovic, Z., Dragnic, N., Petrovic, V., Ristic, M., Pustahija, T., Tsakris, A., Ioannidis, J. P. 2023; 6 (2): e2255779

  Abstract

  During the COVID-19 pandemic, children and adolescents were massively infected worldwide. In 2022, reinfections became a main feature of the endemic phase of SARS-CoV-2, so it is important to understand the epidemiology and clinical impact of reinfections.To assess the incidence, risk, and severity of pediatric SARS-CoV-2 reinfection.This retrospective cohort study used epidemiologic data of documented SARS-CoV-2 infections from the surveillance database of the Institute for Public Health of Vojvodina. A total of 32 524 children and adolescents from Vojvodina, Serbia, with laboratory-confirmed SARS-CoV-2 infection between March 6, 2020, and April 30, 2022, were followed up for reinfection until July 31, 2022.Incidence rates of documented SARS-CoV-2 reinfection per 1000 person-months, estimated risk of documented reinfection 90 days or more after laboratory confirmation of primary infection, reinfection severity, hospitalizations, and deaths.The study cohort included 32 524 children and adolescents with COVID-19 (mean [SD] age, 11.2 [4.9] years; 15 953 [49.1%] male), including 964 children (3.0%) who experienced documented reinfection. The incidence rate of documented reinfections was 3.2 (95% CI, 3.0-3.4) cases per 1000 person-months and was highest in adolescents aged 12 to 17 years (3.4; 95% CI, 3.2-3.7). Most reinfections (905 [93.9%]) were recorded in 2022. The cumulative reinfection risk was 1.3% at 6 months, 1.9% at 9 months, 4.0% at 12 months, 6.7% at 15 months, 7.2% at 18 months, and 7.9% after 21 months. Pediatric COVID-19 cases were generally mild. The proportion of severe clinical forms decreased from 14 (1.4%) in initial episodes to 3 (0.3%) in reinfections. Reinfected children were approximately 5 times less likely to have severe disease during reinfection compared with initial infection (McNemar odds ratio, 0.2; 95% CI, 0.0-0.8). Pediatric reinfections rarely led to hospitalization (0.5% vs 1.3% during primary infections), and none resulted in death.This cohort study found that the SARS-CoV-2 reinfection risk remained substantially lower for children and adolescents compared with adults as of July 2022. Pediatric infections were mild, and reinfections were even milder than primary infections.

  View details for DOI 10.1001/jamanetworkopen.2022.55779

  View details for PubMedID 36780157

• Comparison of the accuracy of the 7-item HADS Depression subscale and 14-item total HADS for screening for major depression: A systematic review and individual participant data meta-analysis. Psychological assessment Wu, Y., Levis, B., Daray, F. M., Ioannidis, J. P., Patten, S. B., Cuijpers, P., Ziegelstein, R. C., Gilbody, S., Fischer, F. H., Fan, S., Sun, Y., He, C., Krishnan, A., Neupane, D., Bhandari, P. M., Negeri, Z., Riehm, K. E., Rice, D. B., Azar, M., Yan, X. W., Imran, M., Chiovitti, M. J., Boruff, J. T., McMillan, D., Kloda, L. A., Markham, S., Henry, M., Ismail, Z., Loiselle, C. G., Mitchell, N. D., Al-Adawi, S., Beck, K. R., Beraldi, A., Bernstein, C. N., Boye, B., Buel-Drabe, N., Bunevicius, A., Can, C., Carter, G., Chen, C., Cheung, G., Clover, K., Conroy, R. M., Costa-Requena, G., Cukor, D., Dabscheck, E., De Souza, J., Downing, M., Feinstein, A., Ferentinos, P. P., Flint, A. J., Gallagher, P., Gandy, M., Grassi, L., Harter, M., Hernando, A., Jackson, M. L., Jenewein, J., Jette, N., Juliao, M., Kjargaard, M., Kohler, S., Konig, H., Krishna, L. K., Lee, Y., Lobner, M., Loosman, W. L., Love, A. W., Lowe, B., Malt, U. F., Marrie, R. A., Massardo, L., Matsuoka, Y., Mehnert, A., Michopoulos, I., Misery, L., Nelson, C. J., Ng, C. G., O'Donnell, M. L., O'Rourke, S. J., Ozturk, A., Pabst, A., Pasco, J. A., Peceliuniene, J., Pintor, L., Ponsford, J. L., Pulido, F., Quinn, T. J., Reme, S. E., Reuter, K., Riedel-Heller, S. G., Rooney, A. G., Sanchez-Gonzalez, R., Saracino, R. M., Schellekens, M. P., Scherer, M., Benedetti, A., Thombs, B. D., Et Al 2023; 35 (2): 95-114

  Abstract

  The seven-item Hospital Anxiety and Depression Scale Depression subscale (HADS-D) and the total score of the 14-item HADS (HADS-T) are both used for major depression screening. Compared to the HADS-D, the HADS-T includes anxiety items and requires more time to complete. We compared the screening accuracy of the HADS-D and HADS-T for major depression detection. We conducted an individual participant data meta-analysis and fit bivariate random effects models to assess diagnostic accuracy among participants with both HADS-D and HADS-T scores. We identified optimal cutoffs, estimated sensitivity and specificity with 95% confidence intervals, and compared screening accuracy across paired cutoffs via two-stage and individual-level models. We used a 0.05 equivalence margin to assess equivalency in sensitivity and specificity. 20,700 participants (2,285 major depression cases) from 98 studies were included. Cutoffs of ≥7 for the HADS-D (sensitivity 0.79 [0.75, 0.83], specificity 0.78 [0.75, 0.80]) and ≥15 for the HADS-T (sensitivity 0.79 [0.76, 0.82], specificity 0.81 [0.78, 0.83]) minimized the distance to the top-left corner of the receiver operating characteristic curve. Across all sets of paired cutoffs evaluated, differences of sensitivity between HADS-T and HADS-D ranged from -0.05 to 0.01 (0.00 at paired optimal cutoffs), and differences of specificity were within 0.03 for all cutoffs (0.02-0.03). The pattern was similar among outpatients, although the HADS-T was slightly (not nonequivalently) more specific among inpatients. The accuracy of HADS-T was equivalent to the HADS-D for detecting major depression. In most settings, the shorter HADS-D would be preferred. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

  View details for DOI 10.1037/pas0001181

  View details for PubMedID 36689386

• Availability of evidence and comparative effectiveness for surgical versus drug interventions: an overview of systematic reviews. medRxiv : the preprint server for health sciences Zavalis, E. A., Rameau, A., Saraswathula, A., Vist, J., Schuit, E., Ioannidis, J. P. 2023

  Abstract

  To examine the prevalence of comparisons of surgery to drug regimens, the strength of evidence of such comparisons, and whether surgery or the drug intervention was favored.Systematic review of systematic reviews (umbrella review).Cochrane Database of Systematic Reviews (CDSR).Using the search term "surg*" in CDSR, we retrieved systematic reviews of surgical interventions. Abstracts were subsequently screened to find systematic reviews that aimed to compare surgical to drug interventions; and then, among them, those that included any randomized controlled trials (RCTs) for such comparisons. Trial results data were extracted manually and synthesized into random-effects meta-analyses.Overall, 188 systematic reviews intended to compare surgery versus drugs. Only 41 included data from at least one RCT (total, 165 RCTs with data) and covered a total of 103 different outcomes of various comparisons of surgery versus drugs. A GRADE assessment was performed by the Cochrane reviewers for 87 (83%) outcomes in the reviews, indicating the strength of evidence was high in 4 outcomes (4%), moderate in 22 (21%), low in 27 (26%) and very low in 33 (32%). Based on 95% confidence intervals, the surgical intervention was favored in 38/103 (37%), and the drugs were favored in 13/103 (13%) outcomes. Of the outcomes with high GRADE rating, only one showed conclusive superiority (sphincterotomy was better than medical therapy for anal fissure). Of the 22 outcomes with moderate GRADE rating, 6 (27%) were inconclusive, 14 (64%) were in favor of surgery, and 2 (9%) were in favor of drugs.Though the relative merits of surgical versus drug interventions are important to know for many diseases, high strength randomized evidence is rare. More randomized trials comparing surgery to drug interventions are needed.https://osf.io/p9x3j.

  View details for DOI 10.1101/2023.01.30.23285207

  View details for PubMedID 36778340

  View details for PubMedCentralID PMC9915830

• A critical assessment of NICE guidelines for treatment of depression. World psychiatry : official journal of the World Psychiatric Association (WPA) Leichsenring, F., Steinert, C., Rost, F., Abbass, A., Heim, N., Ioannidis, J. P. 2023; 22 (1): 43-45

  View details for DOI 10.1002/wps.21039

  View details for PubMedID 36640399

• Homeopathy can offer empirical insights on treatment effects in a null field. Journal of clinical epidemiology Sigurdson, M. K., Sainani, K. L., Ioannidis, J. P. 2023

  Abstract

  A "null field" is a scientific field where there is nothing to discover and where observed associations are thus expected to simply reflect the magnitude of bias. We aimed to characterize a null field using a known example, homeopathy (a pseudoscientific medical approach based on using highly diluted substances), as a prototype.We identified 50 randomized placebo-controlled trials of homeopathy interventions from highly-cited meta-analyses. The primary outcome variable was the observed effect size in the studies. Variables related to study quality or impact were also extracted.The mean effect size for homeopathy was 0.36 standard deviations (Hedges' g; 95% CI: 0.21, 0.51) better than placebo, which corresponds to an odds ratio of 1.94 (95% CI: 1.69, 2.23) in favor of homeopathy. 80% of studies had positive effect sizes (favoring homeopathy). Effect size was significantly correlated with citation counts from journals in the Directory of Open Access Journals and CiteWatch. We identified common statistical errors in 25 studies.A null field like homeopathy can exhibit large effect sizes, high rates of favorable results, and high citation impact in the published scientific literature. Null fields may represent a useful negative control for the scientific process.

  View details for DOI 10.1016/j.jclinepi.2023.01.010

  View details for PubMedID 36736709

• Assessing the usefulness of randomized trials in obstetrics and gynaecology. BJOG : an international journal of obstetrics and gynaecology van 't Hooft, J., van Dijk, C. E., Axfors, C., Alfirevic, Z., Oudijk, M. A., Khan, K. S., Mol, B. W., Bossuyt, P. M., Ioannidis, J. P. 2023

  View details for DOI 10.1111/1471-0528.17411

  View details for PubMedID 36696225

• Estimates of COVID-19 deaths in Mainland China after abandoning zero COVID policy. European journal of clinical investigation Ioannidis, J. P., Zonta, F., Levitt, M. 2023: e13956

  Abstract

  BACKGROUND: China witnessed a surge of Omicron infections after abandoning "zero COVID" strategies on December 7, 2022. The authorities report very sparse deaths based on very restricted criteria, but massive deaths are speculated.METHODS: We aimed to estimate the COVID-19 fatalities in Mainland China until summer 2023 using the experiences of Hong Kong and of South Korea in 2022 as prototypes. Both these locations experienced massive Omicron waves after having had very few SARS-CoV-2 infections during 2020-2021. We estimated age-stratified infection fatality rates (IFRs) in Hong Kong and South Korea during 2022 and extrapolated to the population age structure of Mainland China. We also accounted separately for deaths of residents in long-term care facilities in both Hong Kong and South Korea.RESULTS: IFR estimates in non-elderly strata were modestly higher in Hong Kong than South Korea and projected 987,455 and 619,549 maximal COVID-19 deaths, respectively, if the entire China population was infected. Expected COVID-19 deaths in Mainland China until summer 2023 ranged from 49,962 to 691,219 assuming 25-70% of the non-elderly population being infected and variable protection of elderly (from none to three-quarter reduction in fatalities). The main analysis (45% of non-elderly population infected and fatality impact among elderly reduced by half) estimated 152,886-249,094 COVID-19 deaths until summer 2023. Large uncertainties exist regarding potential changes in dominant variant, health system strain, and impact on non-COVID-19 deaths.CONCLUSIONS: The most critical factor that can affect total COVID-19 fatalities in China is the extent to which the elderly can be protected.

  View details for DOI 10.1111/eci.13956

  View details for PubMedID 36691703

• Differential COVID-19 infection rates in children, adults, and elderly: Systematic review and meta-analysis of 38 pre-vaccination national seroprevalence studies. Journal of global health Axfors, C., Pezzullo, A. M., Contopoulos-Ioannidis, D. G., Apostolatos, A., Ioannidis, J. P. 2023; 13: 06004

  Abstract

  Background: Debate exists about whether extra protection of elderly and other vulnerable individuals is feasible in COVID-19. We aimed to assess the relative infection rates in the elderly vs the non-elderly and, secondarily, in children vs adults.Methods: We performed a systematic review and meta-analysis of seroprevalence studies conducted in the pre-vaccination era. We identified representative national studies without high risk of bias through SeroTracker and PubMed searches (last updated May 17, 2022). We noted seroprevalence estimates for children, non-elderly adults, and elderly adults, using cut-offs of 20 and 60 years (or as close to these ages, if they were unavailable) and compared them between different age groups.Results: We included 38 national seroprevalence studies from 36 different countries comprising 826963 participants. Twenty-six of these studies also included pediatric populations and twenty-five were from high-income countries. The median ratio of seroprevalence in elderly vs non-elderly adults (or non-elderly in general, if pediatric and adult population data were not offered separately) was 0.90-0.95 in different analyses, with large variability across studies. In five studies (all in high-income countries), we observed significant protection of the elderly with a ratio of <0.40, with a median of 0.83 in high-income countries and 1.02 elsewhere. The median ratio of seroprevalence in children vs adults was 0.89 and only one study showed a significant ratio of <0.40. The main limitation of our study is the inaccuracies and biases in seroprevalence studies.Conclusions: Precision shielding of elderly community-dwelling populations before the availability of vaccines was indicated in some high-income countries, but most countries failed to achieve any substantial focused protection.Registration: Open Science Framework (available at: https://osf.io/xvupr).

  View details for DOI 10.7189/jogh.13.06004

  View details for PubMedID 36655924

• Estimates of COVID-19 deaths in Mainland China after abandoning zero COVID policy. medRxiv : the preprint server for health sciences Ioannidis, J. P., Zonta, F., Levitt, M. 2023

  Abstract

  Background: China witnessed a surge of Omicron infections after abandoning zero COVID strategies on December 7, 2022. The authorities report very sparse deaths based on very restricted criteria, but massive deaths are speculated.Methods: We aimed to estimate the COVID-19 fatalities in Mainland China until summer 2023 using the experiences of Hong Kong and of South Korea in 2022 as prototypes. Both these locations experienced massive Omicron waves after having had very few SARS-CoV-2 infections during 2020-2021. We estimated age-stratified infection fatality rates (IFRs) in Hong Kong and South Korea during 2022 and extrapolated to the population age structure of Mainland China. We also accounted separately for deaths of residents in long-term care facilities in both Hong Kong and South Korea.Results: IFR estimates in non-elderly strata were modestly higher in Hong Kong than South Korea and projected 987,455 and 619,549 maximal COVID-19 deaths, respectively, if the entire China population was infected. Expected COVID-19 deaths in Mainland China until summer 2023 ranged from 49,962 to 691,219 assuming 25-70% of the non-elderly population being infected and variable protection of elderly (from none to three-quarter reduction in fatalities). The main analysis (45% of non-elderly population infected and fatality impact among elderly reduced by half) estimated 152,886-249,094 COVID-19 deaths until summer 2023. Large uncertainties exist regarding potential changes in dominant variant, health system strain, and impact on non-COVID-19 deaths.Conclusions: The most critical factor that can affect total COVID-19 fatalities in China is the extent to which the elderly can be protected.

  View details for DOI 10.1101/2022.12.29.22284048

  View details for PubMedID 36597526

• Lessons learnt from registration of biomedical research. Nature human behaviour Serghiou, S., Axfors, C., Ioannidis, J. P. 2023

  View details for DOI 10.1038/s41562-022-01499-0

  View details for PubMedID 36604496

• Exact inference for disease prevalence based on a test with unknown specificity and sensitivity. Journal of applied statistics Cai, B., Ioannidis, J. P., Bendavid, E., Tian, L. 2023; 50 (11-12): 2599-2623

  Abstract

  To make informative public policy decisions in battling the ongoing COVID-19 pandemic, it is important to know the disease prevalence in a population. There are two intertwined difficulties in estimating this prevalence based on testing results from a group of subjects. First, the test is prone to measurement error with unknown sensitivity and specificity. Second, the prevalence tends to be low at the initial stage of the pandemic and we may not be able to determine if a positive test result is a false positive due to the imperfect test specificity. The statistical inference based on a large sample approximation or conventional bootstrap may not be valid in such cases. In this paper, we have proposed a set of confidence intervals, whose validity doesn't depend on the sample size in the unweighted setting. For the weighted setting, the proposed inference is equivalent to hybrid bootstrap methods, whose performance is also more robust than those based on asymptotic approximations. The methods are used to reanalyze data from a study investigating the antibody prevalence in Santa Clara County, California in addition to several other seroprevalence studies. Simulation studies have been conducted to examine the finite-sample performance of the proposed method.

  View details for DOI 10.1080/02664763.2021.2019687

  View details for PubMedID 37529562

  View details for PubMedCentralID PMC10388830

• Prolific non-research authors in high impact scientific journals: meta-research study. Scientometrics Ioannidis, J. P. 2023; 128 (5): 3171-3184

  Abstract

  Journalistic papers published in high impact scientificjournals can be very influential, especially in hot fields. This meta-research analysis aimed to evaluate the publication profiles, impact, and disclosures of conflicts of interest of non-research authors who had published>200 Scopus-indexed papers in Nature, Science, PNAS, Cell, BMJ, Lancet, JAMA or New England Journal of Medicine. 154 prolific authors were identified, 148 of whom had published 67,825 papers in their main affiliated journal in a non-researcher capacity. Nature, Science, and BMJ have the lion's share of such authors. Scopus characterized 35% of the journalistic publications as full articles and another 11% as short surveys. 264 papers had received more than 100 citations. 40/41 most-cited papers in 2020-2022 were on hot COVID-19 topics. Of 25 massively prolific authors with>700 publications in one of these journals, many were highly-cited (median citations 2273), almost all had published little or nothing in the Scopus-indexed literature other than in their main affiliated journal, and their influential writing covered diverse hot topics over the years. Of the 25, only 3 had a PhD degree in any subject matter, and 7 had a Master's degree in journalism. Only the BMJ offered conflicts of interest disclosures for prolific science writers in its website, but even then only 2 of the 25 massively prolific authors disclosed potential conflicts with some specificity. The practice of assigning so much power to non-researchers in shaping scientific discourse should be further debated and disclosures of potential conflicts of interest should be emphasized.

  View details for DOI 10.1007/s11192-023-04687-5

  View details for PubMedID 37101975

• Guidance to best tools and practices for systematic reviews1. Journal of pediatric rehabilitation medicine Kolaski, K., Romeiser Logan, L., Ioannidis, J. P. 2023; 16 (2): 241-273

  Abstract

  Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

  View details for DOI 10.3233/PRM-230019

  View details for PubMedID 37302044

• Quality, integrity and utility of COVID-19 science: opportunities for public health researchers. European journal of public health Pezzullo, A. M., Ioannidis, J. P., Boccia, S. 2022

  View details for DOI 10.1093/eurpub/ckac183

  View details for PubMedID 36508565

• 'Optimal' cutoff selection in studies of depression screening tool accuracy using the PHQ-9, EPDS, or HADS-D: A meta-research study. International journal of methods in psychiatric research Brehaut, E., Neupane, D., Levis, B., Wu, Y., Sun, Y., Ioannidis, J. P., Markham, S., Cuijpers, P., Patten, S. B., Benedetti, A., Thombs, B. D. 2022: e1956

  Abstract

  OBJECTIVES: Optimal cutoff thresholds are selected to separate 'positive' from 'negative' screening results. We evaluated how depression screening tool studies select optimal cutoffs.METHODS: We included studies from previously conducted meta-analyses of Patient Health Questionnaire-9, Edinburgh Postnatal Depression Scale, or Hospital Anxiety and Depression Scale-Depression accuracy. Outcomes included whether an optimal cutoff was selected, method used, recommendations made, and reporting guideline and protocol citation.RESULTS: Of 212 included studies, 172 (81%) attempted to identify an optimal cutoff, and 147 of these 172 (85%) reported one or more methods. Methods were heterogeneous with Youden's J (N=35, 23%) most common. Only 23 of 147 (16%) studies described a rationale for their method. Rationales focused on balancing sensitivity and specificity without describing why desirable. 131 of 172 studies (76%) identified an optimal cutoff other than the standard; most did not make use recommendations (N=56; 43%) or recommended using a non-standard cutoff (N=53; 40%). Only 4 studies cited a reporting guideline, and 4 described a protocol with optimal cutoff selection methods, but none used the protocol method in the published study.CONCLUSIONS: Research is needed to guide how selection of cutoffs for depression screening tools can be standardized and reflect clinical considerations.

  View details for DOI 10.1002/mpr.1956

  View details for PubMedID 36461893

• Federal Funding and Citation Metrics of US Biomedical Researchers, 1996 to 2022. JAMA network open Ioannidis, J. P., Hozo, I., Djulbegovic, B. 2022; 5 (12): e2245590

  Abstract

  Both citation and funding metrics converge in shaping current perceptions of academic success.To evaluate what proportion of the most-cited US-based scientists are funded by biomedical federal agencies and whether funded scientists are more cited than nonfunded ones.This survey study used linkage of a Scopus-based database on top-cited US researchers (according to a composite citation metric) and the National Institutes of Health RePORTER database of federal funding (33 biomedical federal agencies). Matching was based on name and institution. US-based top-cited scientists who were allocated to any of 69 scientific subfields highly related to biomedicine were considered in the main analysis. Data were downloaded on June 11, 2022.Proportion of US-based top-cited biomedical scientists who had any (1996-2022), recent (2015-2022), and current (2021-2022) funding. Comparisons of funded and nonfunded scientists assessed total citations and a composite citation index.There were 204 603 records in RePORTER (1996-2022) and 75 316 US-based top-cited scientists in the career-long citation database; 40 887 scientists were included in the main analysis. The proportion of US-based top-cited biomedical scientists (according to career-long citation impact) who had received any federal funding from biomedical research agencies was 62.7% (25 650 of 40 887) for any funding (1996-2022), 23.1% (9427 of 40 887) for recent funding (2015-2022), and 14.1% (5778 of 40 887) for current funding (2021-2022). Respective proportions were 64.8%, 31.4%, and 20.9%, for top-cited scientists according to recent single-year citation impact. There was large variability across scientific subfields (eg, current funding: 31% of career-long impact top-cited scientists in geriatrics, 30% in bioinformatics and 29% in developmental biology, but 0% in legal and forensic medicine, general psychology and cognitive sciences, and gender studies). Funded top-cited researchers were overall more cited than nonfunded top-cited scientists (median [IQR], 9594 [5650-1703] vs 5352 [3057-9890] citations; P < .001) and substantial difference remained after adjusting for subfield and years since first publication. Differences were more prominent in some specific biomedical subfields.In this survey study, biomedical federal funding had offered support to approximately two-thirds of the top-cited biomedical scientists at some point during the last quarter century, but only a small minority of top-cited scientists had current federal biomedical funding. The large unevenness across subfields needs to be addressed with ways that improve equity, efficiency, excellence, and translational potential.

  View details for DOI 10.1001/jamanetworkopen.2022.45590

  View details for PubMedID 36477476

• Published correlational effect sizes in social and developmental psychology. Royal Society open science Weinerova, J., Szucs, D., Ioannidis, J. P. 2022; 9 (12): 220311

  Abstract

  The distribution of effect sizes may offer insights about the research done and reported in a scientific field. We have evaluated 12 412 manually collected correlation effect sizes (Sample 1) and 31 157 computer-extracted correlation effect sizes (Sample 2) published in journals focused on social or developmental psychology. Sample 1 consisted of 243 studies from six journals published in 2010 and 2019. Sample 2 consisted of 5012 papers published in 10 journals between 2010 and 2019. The 25th, 50th and 75th effect size percentiles were 0.08, 0.17 and 0.33, and 0.17, 0.31 and 0.52 in Samples 1 and 2, respectively. Sample 2 percentiles were probably larger because Sample 2 only included effect sizes from the text but not from tables. In text authors may have emphasized larger correlations. Large sample sizes were associated with smaller reported correlations. In Sample 1 about 70% of studies specified a directional hypothesis. In 2010 no papers had power calculations, while in 2019 14% of papers had power calculations. These data offer empirical insights into the distribution of reported correlations and may inform the interpretation of effect sizes. They also demonstrate the importance of computation of statistical power and highlight potential reporting bias.

  View details for DOI 10.1098/rsos.220311

  View details for PubMedID 36569230

• Time-varying risk of death after SARS-CoV-2 infection in Swedish long-term care facility residents: a matched cohort study. BMJ open Ballin, M., Ioannidis, J. P., Bergman, J., Kivipelto, M., Nordström, A., Nordström, P. 2022; 12 (11): e066258

  Abstract

  To evaluate whether SARS-CoV-2 infection in residents of long-term care (LTC) facilities is associated with higher mortality after the acute phase of infection, and to estimate survival in uninfected residents.Extended follow-up of a previous, propensity score-matched, retrospective cohort study based on the Swedish Senior Alert register.LTC facilities in Sweden.n=3604 LTC residents with documented SARS-CoV-2 until 15 September 2020 matched to 3604 uninfected controls using time-dependent propensity scores on age, sex, health status, comorbidities, prescription medications, geographical region and Senior Alert registration time. In a secondary analysis (n=3731 in each group), geographical region and Senior Alert registration time were not matched for in order to increase the follow-up time in controls and allow for an estimation of median survival.All-cause mortality until 24 October 2020, tracked using the National Cause of Death Register.Median age was 87 years and 65% were women. Excess mortality peaked at 5 days after documented SARS-CoV-2-infection (HR 21.5, 95% CI 15.9 to 29.2), after which excess mortality decreased. From the second month onwards, mortality rate became lower in infected residents than controls. The HR for death during days 61-210 of follow-up was 0.76 (95% CI 0.62 to 0.93). The median survival of uninfected controls was 1.6 years, which was much lower than the national life expectancy in Sweden at age 87 (5.05 years in men, 6.07 years in women).The risk of death after SARS-CoV-2 infection in LTC residents peaked after 5 days and decreased after 2 months, probably because the frailest residents died during the acute phase, leaving healthier residents remaining. The limited life expectancy in this population suggests that LTC resident status should be accounted for when estimating years of life lost due to COVID-19.

  View details for DOI 10.1136/bmjopen-2022-066258

  View details for PubMedID 36424110

• Does natural and hybrid immunity obviate the need for frequent vaccine boosters against SARS-CoV-2 in the endemic phase? European journal of clinical investigation Pilz, S., Ioannidis, J. P. 2022: e13906

  Abstract

  The coronavirus disease 2019 (COVID-19) pandemic has entered its endemic phase and we observe significantly declining infection fatality rates due to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). On this background, it is crucial but challenging to define current and future vaccine policy in a population with a high immunity against SARS-CoV-2 conferred by previous infections and/or vaccinations. Vaccine policy must consider the magnitude of the risks conferred by new infection(s) with current and evolving SARS-CoV-2 variants, how these risks vary in different groups of individuals, how to balance these risks against the apparently small, but existent, risks of harms of vaccination, and the cost-benefit of different options. More evidence from randomized controlled trials and continuously accumulating national health data is required to inform shared decision-making with people who consider vaccination options. Vaccine policy makers should cautiously weight what vaccination schedules are needed, and refrain from urging frequent vaccine boosters unless supported by sufficient evidence.

  View details for DOI 10.1111/eci.13906

  View details for PubMedID 36366946

• Age-stratified infection fatality rate of COVID-19 in the non-elderly population. Environmental research Pezzullo, A. M., Axfors, C., Contopoulos-Ioannidis, D. G., Apostolatos, A., Ioannidis, J. P. 2022; 216 (Pt 3): 114655

  Abstract

  The largest burden of COVID-19 is carried by the elderly, and persons living in nursing homes are particularly vulnerable. However, 94% of the global population is younger than 70 years and 86% is younger than 60 years. The objective of this study was to accurately estimate the infection fatality rate (IFR) of COVID-19 among non-elderly people in the absence of vaccination or prior infection. In systematic searches in SeroTracker and PubMed (protocol: https://osf.io/xvupr), we identified 40 eligible national seroprevalence studies covering 38 countries with pre-vaccination seroprevalence data. For 29 countries (24 high-income, 5 others), publicly available age-stratified COVID-19 death data and age-stratified seroprevalence information were available and were included in the primary analysis. The IFRs had a median of 0.034% (interquartile range (IQR) 0.013-0.056%) for the 0-59 years old population, and 0.095% (IQR 0.036-0.119%) for the 0-69 years old. The median IFR was 0.0003% at 0-19 years, 0.002% at 20-29 years, 0.011% at 30-39 years, 0.035% at 40-49 years, 0.123% at 50-59 years, and 0.506% at 60-69 years. IFR increases approximately 4 times every 10 years. Including data from another 9 countries with imputed age distribution of COVID-19 deaths yielded median IFR of 0.025-0.032% for 0-59 years and 0.063-0.082% for 0-69 years. Meta-regression analyses also suggested global IFR of 0.03% and 0.07%, respectively in these age groups. The current analysis suggests a much lower pre-vaccination IFR in non-elderly populations than previously suggested. Large differences did exist between countries and may reflect differences in comorbidities and other factors. These estimates provide a baseline from which to fathom further IFR declines with the widespread use of vaccination, prior infections, and evolution of new variants.

  View details for DOI 10.1016/j.envres.2022.114655

  View details for PubMedID 36341800

• COVID-19 models and expectations - Learning from the pandemic. Advances in biological regulation Ioannidis, J. P., Powis, S. H. 2022: 100922

  View details for DOI 10.1016/j.jbior.2022.100922

  View details for PubMedID 36241518

• An acceptance speech. Journal of evaluation in clinical practice Ioannidis, J. P. 2022

  View details for DOI 10.1111/jep.13776

  View details for PubMedID 36193625

• Beyond Random Effects: When Small-Study Findings Are More Heterogeneous ADVANCES IN METHODS AND PRACTICES IN PSYCHOLOGICAL SCIENCE Stanley, T. D., Doucouliagos, H., Ioannidis, J. A. 2022; 5 (4)

  View details for DOI 10.1177/25152459221120427

  View details for Web of Science ID 000878612000001

• Estimating conditional vaccine effectiveness. European journal of epidemiology Ioannidis, J. P. 2022

  Abstract

  Vaccine effectiveness for COVID-19 is typically estimated for different outcomes that often are hierarchical in severity (e.g. any documented infection, symptomatic infection, hospitalization, death) and subsets of each other. Conditional effectiveness for a more severe outcome conditional on a less severe outcome is the protection offered against the severe outcome (e.g. death) among those who already sustained the less severe outcome (e.g. documented infection). The concept applies also to the protection offered by previous infection rather than vaccination. Formulas and a nomogram are provided here for calculating conditional effectiveness. Illustrative examples are presented from recent vaccine effectiveness studies, including situations where effectiveness for different outcomes changed at different pace over time. E(death | documented infection) is the percent decrease in the case fatality rate and E(death | infection) is the percent decrease in the infection fatality rate (IFR). Conditional effectiveness depends on many factors and should not be misinterpreted as a causal effect estimate. However, it may be used for better personalized communication of the benefits of vaccination, considering also IFR and epidemic activity in public health decision-making and communication.

  View details for DOI 10.1007/s10654-022-00911-3

  View details for PubMedID 36155868

• Robustness of reported postacute health outcomes in children with SARS-CoV-2 infection: a systematic review ARCHIVES OF DISEASE IN CHILDHOOD Hirt, J., Janiaud, P., Gloy, V., Schandelmaier, S., Pereira, T., Contopoulos-Ioannidis, D., Goodman, S. N., Ioannidis, J., Munkholm, K., Hemkens, L. G. 2022

  View details for DOI 10.1136/archdischild-2022-324455

  View details for Web of Science ID 000850224800001

• Robustness of reported postacute health outcomes in children with SARS-CoV-2 infection: a systematic review. Archives of disease in childhood Hirt, J., Janiaud, P., Gloy, V. L., Schandelmaier, S., Pereira, T. V., Contopoulos-Ioannidis, D., Goodman, S. N., Ioannidis, J., Munkholm, K., Hemkens, L. G. 2022

  Abstract

  To systematically assess the robustness of reported postacute SARS-CoV-2 infection health outcomes in children.A search on PubMed and Web of Science was conducted to identify studies published up to 22 January 2022 that reported on postacute SARS-CoV-2 infection health outcomes in children (<18 years) with follow-up of ≥2 months since detection of infection or ≥1 month since recovery from acute illness. We assessed the consideration of confounding bias and causality, as well as the risk of bias.21 studies including 81 896 children reported up to 97 symptoms with follow-up periods of 2.0-11.5 months. Fifteen studies had no control group. The reported proportion of children with post-COVID syndrome was between 0% and 66.5% in children with SARS-CoV-2 infection (n=16 986) and between 2.0% and 53.3% in children without SARS-CoV-2 infection (n=64 910). Only two studies made a clear causal interpretation of an association between SARS-CoV-2 infection and the main outcome of 'post-COVID syndrome' and provided recommendations regarding prevention measures. The robustness of all 21 studies was seriously limited due to an overall critical risk of bias.The robustness of reported postacute SARS-CoV-2 infection health outcomes in children is seriously limited, at least in all the published articles we could identify. None of the studies provided evidence with reasonable certainty on whether SARS-CoV-2 infection has an impact on postacute health outcomes, let alone to what extent. Children and their families urgently need much more reliable and methodologically robust evidence to address their concerns and improve care.

  View details for DOI 10.1136/archdischild-2022-324455

  View details for PubMedID 36719840

• Risk and severity of SARS-CoV-2 reinfections during 2020-2022 in Vojvodina, Serbia: A population-level observational study. The Lancet regional health. Europe Medic, S., Anastassopoulou, C., Lozanov-Crvenkovic, Z., Vukovic, V., Dragnic, N., Petrovic, V., Ristic, M., Pustahija, T., Gojkovic, Z., Tsakris, A., Ioannidis, J. P. 2022; 20: 100453

  Abstract

  Background: Data on the rate and severity of SARS-CoV-2 reinfections in real-world settings are scarce and the effects of vaccine boosters on reinfection risk are unknown.Methods: In a population-level observational study, registered SARS-CoV-2 laboratory-confirmed Vojvodina residents, between March 6, 2020 and October 31, 2021, were followed for reinfection ≥90 days after primary infection. Data were censored at the end of follow-up (January 31, 2022) or death. The reinfection risk was visualized with Kaplan-Meier plots. To examine the protective effect of vaccination, the subset of individuals with primary infection in 2020 (March 6-December 31) were matched (1:2) with controls without reinfection.Findings: Until January 31, 2022, 13,792 reinfections were recorded among 251,104 COVID-19 primary infections (5.49%). Most reinfections (86.77%, 11,967/13,792) were recorded in January 2022. Reinfections were mostly mild (99.17%, 13,678/13,792). Hospitalizations were uncommon [1.08% (149/13,792) vs. 3.66% (505/13,792) in primary infection] and COVID-19 deaths were very rare (20/13,792, case fatality rate 0.15%). The overall incidence rate of reinfections was 5.99 (95% CI 5.89-6.09) per 1000 person-months. The reinfection risk was estimated as 0.76% at six months, 1.36% at nine months, 4.96% at 12 months, 16.68% at 15 months, and 18.86% at 18 months. Unvaccinated (OR=1.23; 95%CI=1.14-1.33), incompletely (OR=1.33; 95%CI=1.08-1.64) or completely vaccinated (OR=1.50; 95%CI=1.37-1.63), were modestly more likely to be reinfected compared with recipients of a third (booster) vaccine dose.Interpretation: SARS-CoV-2 reinfections were uncommon until the end of 2021 but became common with the advent of Omicron. Very few reinfections were severe. Boosters may modestly reduce reinfection risk.Funding: No specific funding was obtained for this study.

  View details for DOI 10.1016/j.lanepe.2022.100453

  View details for PubMedID 35791336

• Systematic Reviews for Basic Scientists: A Different Beast. Physiological reviews Ioannidis, J. P. 2022

  View details for DOI 10.1152/physrev.00028.2022

  View details for PubMedID 36049113

• Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions NATURE COMMUNICATIONS Wang, S., Sreedhara, S., Schneeweiss, S., REPEAT Initiative 2022; 13 (1): 5126

  Abstract

  Studies that generate real-world evidence on the effects of medical products through analysis of digital data collected in clinical practice provide key insights for regulators, payers, and other healthcare decision-makers. Ensuring reproducibility of such findings is fundamental to effective evidence-based decision-making. We reproduce results for 150 studies published in peer-reviewed journals using the same healthcare databases as original investigators and evaluate the completeness of reporting for 250. Original and reproduction effect sizes were positively correlated (Pearson's correlation = 0.85), a strong relationship with some room for improvement. The median and interquartile range for the relative magnitude of effect (e.g., hazard ratiooriginal/hazard ratioreproduction) is 1.0 [0.9, 1.1], range [0.3, 2.1]. While the majority of results are closely reproduced, a subset are not. The latter can be explained by incomplete reporting and updated data. Greater methodological transparency aligned with new guidance may further improve reproducibility and validity assessment, thus facilitating evidence-based decision-making. Study registration number: EUPAS19636.

  View details for DOI 10.1038/s41467-022-32310-3

  View details for Web of Science ID 000849359800003

  View details for PubMedID 36045130

  View details for PubMedCentralID PMC9430007

• Post-publication critique at top-ranked journals across scientific disciplines: a cross-sectional assessment of policies and practice. Royal Society open science Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Tzavella, L., Bendixen, T., Handcock, S. A., Koneke, V. E., Ioannidis, J. P. 2022; 9 (8): 220139

  Abstract

  Journals exert considerable control over letters, commentaries and online comments that criticize prior research (post-publication critique). We assessed policies (Study One) and practice (Study Two) related to post-publication critique at 15 top-ranked journals in each of 22 scientific disciplines (N = 330 journals). Two-hundred and seven (63%) journals accepted post-publication critique and often imposed limits on length (median 1000, interquartile range (IQR) 500-1200 words) and time-to-submit (median 12, IQR 4-26 weeks). The most restrictive limits were 175 words and two weeks; some policies imposed no limits. Of 2066 randomly sampled research articles published in 2018 by journals accepting post-publication critique, 39 (1.9%, 95% confidence interval [1.4, 2.6]) were linked to at least one post-publication critique (there were 58 post-publication critiques in total). Of the 58 post-publication critiques, 44 received an author reply, of which 41 asserted that original conclusions were unchanged. Clinical Medicine had the most active culture of post-publication critique: all journals accepted post-publication critique and published the most post-publication critique overall, but also imposed the strictest limits on length (median 400, IQR 400-550 words) and time-to-submit (median 4, IQR 4-6 weeks). Our findings suggest that top-ranked academic journals often pose serious barriers to the cultivation, documentation and dissemination of post-publication critique.

  View details for DOI 10.1098/rsos.220139

  View details for PubMedID 36039285

• Constructive and Obsessive Criticism in Science. European journal of clinical investigation Prasad, V., Ioannidis, J. P. 2022: e13839

  Abstract

  Social media and new tools for engagement offer democratic platforms for enhancing constructive scientific criticism which had previously been limited. Constructive criticism can now be massive, timely, and open. However, new options have also enhanced obsessive criticism. Obsessive criticism tends to focus on one or a handful of individuals and their work, often includes ad hominem aspects, and the critics often lack field-specific skills and technical expertise. Typical behaviors include: repetitive and persistent comments (including sealioning), lengthy commentaries/tweetorials/responses often longer than the original work, strong degree of moralizing, distortion of the underlying work, argumentum ad populum, calls to suspend/censor/retract the work or the author, guilt by association, reputational tarnishing, large gains in followers specifically through attacks, finding and positing sensitive personal information, anonymity or pseudonymity, social media campaigning, and unusual ratio of criticism to pursuit of one's research agenda. These behaviors may last months or years. Prevention and treatment options may include awareness, identifying and working around aggravating factors, placing limits on the volume by editors, constructive pairing of commissioned editorials, incorporation of some hot debates from unregulated locations such as social media or PubPeer to the pages of scientific journals, preserving decency and focusing on evidence and arguments and avoiding personal statements, or (in some cases) ignoring. We need more research on the role of social media and obsessive criticism on an evolving cancel culture, the social media credibility, the use/misuse of anonymity and pseudonymity, and whether potential interventions from universities may improve or further weaponize scientific criticism.

  View details for DOI 10.1111/eci.13839

  View details for PubMedID 35869811

• Mapping evolving treatment effects for overall and progression-free survival shows patterns across 2109 randomized oncology trials. Journal of clinical epidemiology Parish, A. J., Alina-Cristea, I., Schuit, E., Ioannidis, J. P. 2022

  Abstract

  OBJECTIVE: To assess the patterns and time trends in overall survival and progression-free survival treatment effects across randomized controlled trials (RCTs) in oncology.STUDY DESIGN AND SETTING: A PUBMED search for oncology network meta-analyses (NMAs) was carried (to September 30th, 2021). Relevant hazard ratios were extracted for systemic treatments from RCTs in the NMAs. After removing duplicate results, relationships between treatment effects, year of publication, trial design and other features were explored.RESULTS: From 241 oncology NMAs, 2109 unique eligible RCTs provided analyzable data. On average, there was a 12-14% reduction in hazard for overall survival and 27-30% reduction for progression-free survival, with substantial heterogeneity across different malignancies. Correlation between overall survival and progression-free survival treatment effects was modest (r=0.60, 95% confidence interval, 0.56-0.64). Over time, there was a suggestive trend of increased progression-free survival treatment effect, while overall survival treatment effects remained steady. Only one in five trials met criteria for clinically meaningful improvements in overall survival. Among 300 randomly selected trials, mean absolute improvement was 1.6 months for median progression-free survival and 1.4 months for median overall survival.CONCLUSIONS: Broad patterns across the past 50 years of oncology research suggest continuous progress has been made, but few results meet clinically meaningful thresholds for overall survival improvement.

  View details for DOI 10.1016/j.jclinepi.2022.06.013

  View details for PubMedID 35777712

• Susceptibility of SARS-CoV-2 Omicron Variants to Therapeutic Monoclonal Antibodies: Systematic Review and Meta-analysis. Microbiology spectrum Tao, K., Tzou, P. L., Kosakovsky Pond, S. L., Ioannidis, J. P., Shafer, R. W. 2022: e0092622

  Abstract

  SARS-CoV-2 Omicron variants contain many mutations in its spike receptor-binding domain, the target of all authorized monoclonal antibodies (MAbs). Determining the extent to which Omicron variants reduced MAb susceptibility is critical to preventing and treating COVID-19. We systematically reviewed PubMed and three preprint servers, last updated 11 April 2022, for the in vitro activity of authorized MAbs against the Omicron variants. Fifty-one studies were eligible, including 50 containing Omicron BA.1 susceptibility data and 17 containing Omicron BA.2 susceptibility data. The first two authorized MAb combinations, bamlanivimab/etesevimab and casirivimab/imdevimab, were largely inactive against the Omicron BA.1 and BA.2 variants. In 34 studies, sotrovimab displayed a median 4.0-fold (interquartile range [IQR]: 2.6 to 6.9) reduction in activity against Omicron BA.1, and in 12 studies, it displayed a median 17-fold (IQR: 13 to 30) reduction in activity against Omicron BA.2. In 15 studies, the combination cilgavimab/tixagevimab displayed a median 86-fold (IQR: 27 to 151) reduction in activity against Omicron BA.1, and in six studies, it displayed a median 5.4-fold (IQR: 3.7 to 6.9) reduction in activity against Omicron BA.2. In eight studies against Omicron BA.1 and six studies against Omicron BA.2, bebtelovimab displayed no reduction in activity. Disparate results between assays were common. For authorized MAbs, 51/268 (19.0%) results for wild-type control variants and 78/348 (22.4%) results for Omicron BA.1 and BA.2 variants were more than 4-fold below or 4-fold above the median result for that MAb. Highly disparate results between published assays indicate a need for improved MAb susceptibility test standardization or interassay calibration. IMPORTANCE Monoclonal antibodies (MAbs) targeting the SARS-CoV-2 spike protein are among the most effective measures for preventing and treating COVID-19. However, SARS-CoV-2 Omicron variants contain many mutations in their spike receptor-binding domains, the target of all authorized MAbs. Therefore, determining the extent to which Omicron variants reduced MAb susceptibility is critical to preventing and treating COVID-19. We identified 51 studies that reported the in vitro susceptibility of the two main Omicron variants BA.1 and BA.2 to therapeutic MAbs in advanced clinical development, including eight authorized individual MAbs and three authorized MAb combinations. We estimated the degree to which different MAbs displayed reduced activity against Omicron variants. The marked loss of activity of many MAbs against Omicron variants underscores the importance of developing MAbs that target conserved regions of spike. Highly disparate results between assays indicate the need for improved MAb susceptibility test standardization.

  View details for DOI 10.1128/spectrum.00926-22

  View details for PubMedID 35700134

• Most healthcare interventions tested in Cochrane Reviews are not effective according to high quality evidence: a systematic review and meta-analysis. Journal of clinical epidemiology Howick, J., Koletsi, D., Ioannidis, J. P., Madigan, C., Pandis, N., Loef, M., Walach, H., Sauer, S., Kleijnen, J., Seehra, J., Johnson, T., Schmidt, S. 2022

  Abstract

  OBJECTIVE: To estimate the proportion of healthcare interventions tested within Cochrane Reviews that are effective according to high-quality evidence.STUDY DESIGN AND SETTING: We selected a random sample of 2428 (35%) of all Cochrane Reviews published between 1 January 2008 and 5 March 2021. We extracted data about interventions within these reviews that were compared with placebo, or no treatment, and whose outcome quality was rated using Grading of Recommendations Assessment, Development and Evaluation (GRADE). We calculated the proportion of interventions whose effectiveness was based on high-quality evidence according to GRADE, had statistically significant positive effects, and were judged as beneficial by the review authors. We also calculated the proportion of interventions that suggested harm.RESULTS: Of 1567 eligible interventions, 87 (5.6%) had high quality evidence on first-listed primary outcomes, positive, statistically significant results and were rated by review authors as beneficial. Harms were measured for 577 (36.8%) interventions, 127 of which (8.1%) had statistically significant evidence of harm. Our dependence on the reliability of Cochrane author assessments (including their GRADE assessments) was a potential limitation of our study.CONCLUSION: Most healthcare interventions studied within recent Cochrane Reviews are not supported by high quality evidence, and harms are under-reported.

  View details for DOI 10.1016/j.jclinepi.2022.04.017

  View details for PubMedID 35447356

• High-cited favorable studies for COVID-19 treatments ineffective in large trials. Journal of clinical epidemiology Ioannidis, J. P. 2022

  Abstract

  OBJECTIVE: To evaluate for COVID-19 treatments without benefits in subsequent large RCTs how many of their most-cited clinical studies had declared favorable results.STUDY DESIGN: Scopus searches (December 23, 2021) identified articles on lopinavir-ritonavir, hydroxycholoroquine, azithromycin, remdesivir, convalescent plasma, colchicine or interferon (index interventions) that represented clinical trials and had >150 citations. Their conclusions were correlated with study design features. The ten most recent citations for the most-cited article on each index intervention were examined on whether they were critical to the highly-cited study. Altmetric scores were also obtained.RESULTS: 40 eligible articles of clinical studies had received >150 citations. 20/40 (50%) had favorable conclusions, 4 were equivocal. Highly-cited articles with favorable conclusions were rarely RCTs (3/20) while those without favorable conclusions were mostly RCTs (15/20, p=0.0003). Only 1 RCT with favorable conclusions had >160 patients. Citation counts correlated strongly with Altmetric scores, especially news items. Only 9 (15%) of 60 recent citations to the most highly-cited studies with favorable or equivocal conclusions were critical.CONCLUSION: Many clinical studies with favorable conclusions for largely ineffective COVID-19 treatments are uncritically heavily cited and disseminated. Early observational studies and small randomized trials may cause spurious claims of effectiveness that get perpetuated.

  View details for DOI 10.1016/j.jclinepi.2022.04.001

  View details for PubMedID 35398190

• Factors influencing estimated effectiveness of COVID-19 vaccines in non-randomised studies. BMJ evidence-based medicine Ioannidis, J. P. 2022

  Abstract

  Non-randomised studies assessing COVID-19 vaccine effectiveness need to consider multiple factors that may generate spurious estimates due to bias or genuinely modify effectiveness. These include pre-existing immunity, vaccination misclassification, exposure differences, testing, disease risk factor confounding, hospital admission decision, treatment use differences, and death attribution. It is useful to separate whether the impact of each factor admission decision, treatment use differences, and death attribution. Steps and measures to consider for improving vaccine effectiveness estimation include registration of studies and of analysis plans; sharing of raw data and code; background collection of reliable information; blinded assessment of outcomes, e.g. death causes; using maximal/best information in properly-matched studies, multivariable analyses, propensity analyses, and other models; performing randomised trials, whenever possible, for suitable questions, e.g. booster doses or comparative effectiveness of different vaccination strategies; living meta-analyses of vaccine effectiveness; better communication with both relative and absolute metrics of risk reduction and presentation of uncertainty; and avoidance of exaggeration in communicating results to the general public.

  View details for DOI 10.1136/bmjebm-2021-111901

  View details for PubMedID 35338091

• Reproducibility: Has Cancer Biology Failed beyond Repair? Clinical chemistry Ioannidis, J. P. 2022

  View details for DOI 10.1093/clinchem/hvac030

  View details for PubMedID 35260905

• The role of outdoor and indoor air quality in the spread of SARS-CoV-2: Overview and recommendations by the research group on COVID-19 and particulate matter (RESCOP commission). Environmental research Piscitelli, P., Miani, A., Setti, L., De Gennaro, G., Rodo, X., Artinano, B., Vara, E., Rancan, L., Arias, J., Passarini, F., Barbieri, P., Pallavicini, A., Parente, A., D'Oro, E. C., De Maio, C., Saladino, F., Borelli, M., Colicino, E., Goncalves, L. M., Di Tanna, G., Colao, A., Leonardi, G. S., Baccarelli, A., Dominici, F., Ioannidis, J. P., Domingo, J. L., RESCOP Commission established by Environmental Research (Elsevier 2022: 113038

  Abstract

  There are important questions surrounding the potential contribution of outdoor and indoor air quality in the transmission of SARS-CoV-2 and perpetuation of COVID-19 epidemic waves. Environmental health may be a critical component of COVID-19 prevention. The public health community and health agencies should consider the evolving evidence in their recommendations and statements, and work to issue relational occupational guidelines. Evidence coming from the current epidemiological and experimental research is expected to add knowledge about virus diffusion, COVID-19 severity in most polluted areas, inter-personal distance requirements and need for wearing face masks in indoor or outdoor environments. The COVID-19 pandemic has highlighted the need for maintaining particulate matter concentrations at low levels for multiple health-related reasons, which may also include the spread of SARS-CoV-2. Indoor environments represent even a more crucial challenge to cope with, as it is easier for the SARS-COV2 to spread, remain vital and infect other subjects in closed spaces in the presence of already infected asymptomatic or mildly symptomatic people. The potential merits of preventive measures, such as CO2 monitoring associated with natural or controlled mechanical ventilation and air purification, for schools, indoor public places (restaurants, offices, hotels, museums, theatres/cinemas etc.) and transportations need to be carefully considered. Hospital settings and nursing/retirement homes as well as emergency rooms, infectious diseases divisions and ambulances represent higher risk indoor environments and may require additional monitoring and specific decontamination strategies based on mechanical ventilation or air purification.

  View details for DOI 10.1016/j.envres.2022.113038

  View details for PubMedID 35231456

• Science with or without statistics: Discover-generalize-replicate? Discover-replicate-generalize? The Behavioral and brain sciences Ioannidis, J. P. 2022; 45: e23

  Abstract

  Overstated generalizability (external validity) is common in research. It may coexist with inflation of the magnitude and statistical support for effects and dismissal of internal validity problems. Generalizability may be secured before attempting replication of proposed discoveries or replication may precede efforts to generalize. These opposite approaches may decrease or increase, respectively, the use of inferential statistics with advantages and disadvantages.

  View details for DOI 10.1017/S0140525X21000054

  View details for PubMedID 35139936

• Citation impact and social media visibility of Great Barrington and John Snow signatories for COVID-19 strategy. BMJ open Ioannidis, J. P. 2022; 12 (2): e052891

  Abstract

  OBJECTIVE: The Great Barrington Declaration (GBD) and the John Snow Memorandum (JSM), each signed by numerous scientists, have proposed hotly debated strategies for handling the COVID-19 pandemic. The current analysis aimed to examine whether the prevailing narrative that GBD is a minority view among experts is true.METHODS: The citation impact and social media presence of the key GBD and JSM signatories was assessed. Citation data were obtained from Scopus using a previously validated composite citation indicator that incorporated also coauthorship and author order and ranking was against all authors in the same Science-Metrix scientific field with at least five full papers. Random samples of scientists from the longer lists of signatories were also assessed. The number of Twitter followers for all key signatories was also tracked.RESULTS: Among the 47 key GBD signatories, 20, 19 and 21, respectively, were top-cited authors for career impact, recent single-year (2019) impact or either. For comparison, among the 34 key JSM signatories, 11, 14 and 15, respectively, were top cited. Key signatories represented 30 different scientific fields (9 represented in both documents, 17 only in GBD and 4 only in JSM). In a random sample of n=30 scientists among the longer lists of signatories, five in GBD and three in JSM were top cited. By April 2021, only 19/47 key GBD signatories had personal Twitter accounts versus 34/34 of key JSM signatories; 3 key GBD signatories versus 10 key JSM signatories had >50000 Twitter followers and extraordinary Kardashian K-indices (363-2569). By November 2021, four key GBD signatories versus 13 key JSM signatories had >50000 Twitter followers.CONCLUSIONS: Both GBD and JSM include many stellar scientists, but JSM has far more powerful social media presence and this may have shaped the impression that it is the dominant narrative.

  View details for DOI 10.1136/bmjopen-2021-052891

  View details for PubMedID 35140152

• The efficacy of psychotherapies and pharmacotherapies for mental disorders in adults: an umbrella review and meta-analytic evaluation of recent meta-analyses. World psychiatry : official journal of the World Psychiatric Association (WPA) Leichsenring, F., Steinert, C., Rabung, S., Ioannidis, J. P. 1800; 21 (1): 133-145

  Abstract

  Mental disorders represent a worldwide public health concern. Psychotherapies and pharmacotherapies are recommended as first line treatments. However, evidence has emerged that their efficacy may be overestimated, due to a variety of shortcomings in clinical trials (e.g., publication bias, weak control conditions such as waiting list). We performed an umbrella review of recent meta-analyses of randomized controlled trials (RCTs) of psychotherapies and pharmacotherapies for the main mental disorders in adults. We selected meta-analyses that formally assessed risk of bias or quality of studies, excluded weak comparators, and used effect sizes for target symptoms as primary outcome. We searched PubMed and PsycINFO and individual records of the Cochrane Library for meta-analyses published between January 2014 and March 2021 comparing psychotherapies or pharmacotherapies with placebo or treatment-as-usual (TAU), or psychotherapies vs. pharmacotherapies head-to-head, or the combination of psychotherapy with pharmacotherapy to either monotherapy. One hundred and two meta-analyses, encompassing 3,782 RCTs and 650,514 patients, were included, covering depressive disorders, anxiety disorders, post-traumatic stress disorder, obsessive-compulsive disorder, somatoform disorders, eating disorders, attention-deficit/hyperactivity disorder, substance use disorders, insomnia, schizophrenia spectrum disorders, and bipolar disorder. Across disorders and treatments, the majority of effect sizes for target symptoms were small. A random effect meta-analytic evaluation of the effect sizes reported by the largest meta-analyses per disorder yielded a standardized mean difference (SMD) of 0.34 (95% CI: 0.26-0.42) for psychotherapies and 0.36 (95% CI: 0.32-0.41) for pharmacotherapies compared with placebo or TAU. The SMD for head-to-head comparisons of psychotherapies vs. pharmacotherapies was 0.11 (95% CI: -0.05 to 0.26). The SMD for the combined treatment compared with either monotherapy was 0.31 (95% CI: 0.19-0.44). Risk of bias was often high. After more than half a century of research, thousands of RCTs and millions of invested funds, the effect sizes of psychotherapies and pharmacotherapies for mental disorders are limited, suggesting a ceiling effect for treatment research as presently conducted. A paradigm shift in research seems to be required to achieve further progress.

  View details for DOI 10.1002/wps.20941

  View details for PubMedID 35015359

• IDENTIFICATION OF THRESHOLD FOR LARGE (DRAMATIC) EFFECTS THAT WOULD OBVIATE RANDOMIZED TRIALS IS NOT POSSIBLE. Journal of clinical epidemiology Hozo, I., Djulbegovic, B., Parish, A. J., Ioannidis, J. P. 1800

  Abstract

  OBJECTIVE: To analyze distribution of "dramatic", large treatment effects.STUDY DESIGN & SETTING: Pareto distribution modeling of previously reported cohorts of 3,486 randomized trials (RCTs) that enrolled 1,532,459 patients and 730 non-randomized studies (NRS) enrolling 1,650,658 patients.RESULTS: We calculated the Pareto alpha parameter, which determines the tail of the distribution for various starting points of distribution [odds ratiomin (ORmin)]. In default analysis using all data at ORmin ≥1, Pareto distribution fit well to the treatment effects of RCTs favoring the new treatments (p=0.21, Kolmogorov-Smirnov test) with best alpha=2.32. For NRS, Pareto fit for ORmin ≥2 with best alpha=1.91. For RCTs, theoretical 99th percentile OR was 32.7. The actual 99th percentile OR was 25; which converted into relative risk (RR)=7.1. The maximum observed effect size was OR=121 (RR=11.45). For NRS, theoretical 99th percentile was OR=315. The actual 99th percentile OR was 294 (RR=13). The maximum observed effect size was OR=1473 (RR=66).CONCLUSIONS: The effects sizes observed in RCTs and NRS considerably overlap. Large effects are rare and there is no clear threshold for dramatic effects that would obviate future RCTs.

  View details for DOI 10.1016/j.jclinepi.2022.01.016

  View details for PubMedID 35091046

• Pre-registration of mathematical models. Mathematical biosciences Ioannidis, J. P. 1800: 108782

  Abstract

  Pre-registration is a research practice where a protocol is deposited in a repository before a scientific project is performed. The protocol may be publicly visible immediately upon deposition or it may remain hidden until the work is completed/published. It may include the analysis plan, outcomes, and/or information about how evaluation of performance (e.g. forecasting ability) will be made Pre-registration aims to enhance the trust one can put on scientific work. Deviations from the original plan, may still often be desirable, but pre-registration makes them transparent. While pre-registration has been advocated and used to variable extent in diverse types of research, there has been relatively little attention given to the possibility of pre-registration for mathematical modeling studies. Feasibility of pre-registration depends on the type of modeling and the ability to pre-specify processes and outcomes. In some types of modeling, in particular those that involve forecasting or other outcomes that can be appraised in the future, trust in model performance would be enhanced through pre-registration. Pre-registration can also be seen as a component of a largest suite of research practices that aim to improve documentation, transparency, and sharing - eventually allowing better reproducibility of the research work. The current commentary discusses the evolving landscape of the concept of pre-registration as it relates to different mathematical modeling activities, the potential advantages and disadvantages, feasibility issues, and realistic goals.

  View details for DOI 10.1016/j.mbs.2022.108782

  View details for PubMedID 35090877

• Exact inference for disease prevalence based on a test with unknown specificity and sensitivity JOURNAL OF APPLIED STATISTICS Cai, B., Ioannidis, J. A., Bendavid, E., Tian, L. 2022

  View details for DOI 10.1080/02664763.2021.2019687

  View details for Web of Science ID 000738502700001

• COVID-19: A Catalyst for Transforming Randomized Trials. Journal of neurosurgical anesthesiology Lee, J. J., Price, J. C., Jackson, W. M., Whittington, R. A., Ioannidis, J. P. 2022; 34 (1): 107-112

  Abstract

  The coronavirus disease 2019 (COVID-19) pandemic incited a global clinical trial research agenda of unprecedented speed and high volume. This expedited research activity in a time of crisis produced both successes and failures that offer valuable learning opportunities for the scientific community to consider. Successes include the implementation of large adaptive and pragmatic trials as well as burgeoning efforts toward rapid data synthesis and open science principles. Conversely, notable failures include: (1) inadequate study design and execution; (2) data reversal, fraud, and retraction; and (3) research duplication and waste. Other challenges that became highlighted were the need to find unbiased designs for investigating complex, nonpharmaceutical interventions and the use of routinely collected data for outcomes assessment. This article discusses these issues juxtaposing the COVID-19 trials experience against trials in anesthesiology and other fields. These lessons may serve as a positive catalyst for transforming future clinical trial research.

  View details for DOI 10.1097/ANA.0000000000000804

  View details for PubMedID 34870631

• Environmental risk factors for non-Hodgkin's lymphoma: umbrella review and comparison of meta-analyses of summary and individual participant data. BMJ medicine Shi, X., Zhuo, H., Du, Y., Nyhan, K., Ioannidis, J., Wallach, J. D. 2022; 1 (1): e000184

  Abstract

  Objectives: To summarise the range, strength, and validity of reported associations between environmental risk factors and non-Hodgkin's lymphoma, and to evaluate the concordance between associations reported in meta-analyses of summary level data and meta-analyses of individual participant data.Design: Umbrella review and comparison of meta-analyses of summary and individual participant level data.Data sources: Medline, Embase, Scopus, Web of Science Core Collection, Cochrane Library, and Epistemonikos, from inception to 23 July 2021.Eligibility criteria for selecting studies: English language meta-analyses of summary level data and of individual participant data evaluating associations between environmental risk factors and incident non-Hodgkin's lymphoma (overall and subtypes).Data extraction and synthesis: Summary effect estimates from meta-analyses of summary level data comparing ever versus never exposure that were adjusted for the largest number of potential confounders were re-estimated using a random effects model and classified as presenting evidence that was non-significant, weak (P<0.05), suggestive (P<0.001 and >1000 cases), highly suggestive (P<0.000001, >1000 cases, largest study reporting a significant association), or convincing (P<0.000001, >1000 cases, largest study reporting a significant association, I2 <50%, 95% prediction interval excluding the null value, and no evidence of small study effects and excess significance bias) evidence. When the same exposures, exposure contrast levels, and outcomes were evaluated in meta-analyses of summary level data and meta-analyses of individual participant data from the International Lymphoma Epidemiology (InterLymph) Consortium, concordance in terms of direction, level of significance, and overlap of 95% confidence intervals was examined. Methodological quality of the meta-analyses of summary level data was assessed by the AMSTAR 2 tool.Results: We identified 85 meta-analyses of summary level data reporting 257 associations for 134 unique environmental risk factors and 10 subtypes of non-Hodgkin's lymphoma nearly all (79, 93%) were classified as having critically low quality. Most associations (225, 88%) presented either non-significant or weak evidence. The 11 (4%) associations presenting highly suggestive evidence were primarily for autoimmune or infectious disease related risk factors. Only one association, between history of coeliac disease and risk of non-Hodgkin's lymphoma, presented convincing evidence. Of 40 associations reported in meta-analyses of summary level data that were also evaluated in InterLymph meta-analyses of individual participant data, 22 (55%) pairs were in the same direction, had the same level of statistical significance, and had overlapping 95% confidence intervals; 28 (70%) pairs had summary effect sizes from the meta-analyses of individual participant data that were more conservative.Conclusion: This umbrella review suggests evidence of many meta-analyses of summary level data reporting weak associations between environmental risk factors and non-Hodgkin's lymphoma. Improvements to primary studies as well as evidence synthesis in evaluations of evironmental risk factors and non-Hodgkin's lymphoma are needed.Review registration number: PROSPERO CRD42020178010.

  View details for DOI 10.1136/bmjmed-2022-000184

  View details for PubMedID 36936582

• Conducting umbrella reviews. BMJ medicine Belbasis, L., Bellou, V., Ioannidis, J. P. 2022; 1 (1): e000071

  Abstract

  In this article, Lazaros Belbasis and colleagues explain the rationale for umbrella reviews and the key steps involved in conducting an umbrella review, using a working example.

  View details for DOI 10.1136/bmjmed-2021-000071

  View details for PubMedID 36936579

• Aggressive measures, rising inequalities, and mass formation during the COVID-19 crisis: An overview and proposed way forward. Frontiers in public health Schippers, M. C., Ioannidis, J. P., Joffe, A. R. 2022; 10: 950965

  Abstract

  A series of aggressive restrictive measures were adopted around the world in 2020-2022 to attempt to prevent SARS-CoV-2 from spreading. However, it has become increasingly clear the most aggressive (lockdown) response strategies may involve negative side-effects such as a steep increase in poverty, hunger, and inequalities. Several economic, educational, and health repercussions have fallen disproportionately on children, students, young workers, and especially on groups with pre-existing inequalities such as low-income families, ethnic minorities, and women. This has led to a vicious cycle of rising inequalities and health issues. For example, educational and financial security decreased along with rising unemployment and loss of life purpose. Domestic violence surged due to dysfunctional families being forced to spend more time with each other. In the current narrative and scoping review, we describe macro-dynamics that are taking place because of aggressive public health policies and psychological tactics to influence public behavior, such as mass formation and crowd behavior. Coupled with the effect of inequalities, we describe how these factors can interact toward aggravating ripple effects. In light of evidence regarding the health, economic and social costs, that likely far outweigh potential benefits, the authors suggest that, first, where applicable, aggressive lockdown policies should be reversed and their re-adoption in the future should be avoided. If measures are needed, these should be non-disruptive. Second, it is important to assess dispassionately the damage done by aggressive measures and offer ways to alleviate the burden and long-term effects. Third, the structures in place that have led to counterproductive policies should be assessed and ways should be sought to optimize decision-making, such as counteracting groupthink and increasing the level of reflexivity. Finally, a package of scalable positive psychology interventions is suggested to counteract the damage done and improve humanity's prospects.

  View details for DOI 10.3389/fpubh.2022.950965

  View details for PubMedID 36159300

• A meta-epidemiological assessment of transparency indicators of infectious disease models. PloS one Zavalis, E. A., Ioannidis, J. P. 2022; 17 (10): e0275380

  Abstract

  Mathematical models have become very influential, especially during the COVID-19 pandemic. Data and code sharing are indispensable for reproducing them, protocol registration may be useful sometimes, and declarations of conflicts of interest (COIs) and of funding are quintessential for transparency. Here, we evaluated these features in publications of infectious disease-related models and assessed whether there were differences before and during the COVID-19 pandemic and for COVID-19 models versus models for other diseases. We analysed all PubMed Central open access publications of infectious disease models published in 2019 and 2021 using previously validated text mining algorithms of transparency indicators. We evaluated 1338 articles: 216 from 2019 and 1122 from 2021 (of which 818 were on COVID-19); almost a six-fold increase in publications within the field. 511 (39.2%) were compartmental models, 337 (25.2%) were time series, 279 (20.9%) were spatiotemporal, 186 (13.9%) were agent-based and 25 (1.9%) contained multiple model types. 288 (21.5%) articles shared code, 332 (24.8%) shared data, 6 (0.4%) were registered, and 1197 (89.5%) and 1109 (82.9%) contained COI and funding statements, respectively. There was no major changes in transparency indicators between 2019 and 2021. COVID-19 articles were less likely to have funding statements and more likely to share code. Further validation was performed by manual assessment of 10% of the articles identified by text mining as fulfilling transparency indicators and of 10% of the articles lacking them. Correcting estimates for validation performance, 26.0% of papers shared code and 41.1% shared data. On manual assessment, 5/6 articles identified as registered had indeed been registered. Of articles containing COI and funding statements, 95.8% disclosed no conflict and 11.7% reported no funding. Transparency in infectious disease modelling is relatively low, especially for data and code sharing. This is concerning, considering the nature of this research and the heightened influence it has acquired.

  View details for DOI 10.1371/journal.pone.0275380

  View details for PubMedID 36206207

• SARS-CoV-2 reinfections: Overview of efficacy and duration of natural and hybrid immunity. Environmental research Pilz, S., Theiler-Schwetz, V., Trummer, C., Krause, R., Ioannidis, J. P. 2022: 112911

  Abstract

  Seroprevalence surveys suggest that more than a third and possibly more than half of the global population has been infected with SARS-CoV-2 by early 2022. As large numbers of people continue to be infected, the efficacy and duration of natural immunity in terms of protection against SARS-CoV-2 reinfections and severe disease is of crucial significance for the future. This narrative review provides an overview on epidemiological studies addressing this issue. National surveys covering 2020-2021 documented that a previous SARS-CoV-2 infection is associated with a significantly reduced risk of reinfections with efficacy lasting for at least one year and only relatively moderate waning immunity. Importantly, natural immunity showed roughly similar effect sizes regarding protection against reinfection across different SARS-CoV-2 variants, with the exception of the Omicron variant for which data are just emerging before final conclusions can be drawn. Risk of hospitalizations and deaths was also reduced in SARS-CoV-2 reinfections versus primary infections. Observational studies indicate that natural immunity may offer equal or greater protection against SARS-CoV-2 infections compared to individuals receiving two doses of an mRNA vaccine, but data are not fully consistent. The combination of a previous SARS-CoV-2 infection and a respective vaccination, termed hybrid immunity, seems to confer the greatest protection against SARS-CoV-2 infections, but several knowledge gaps remain regarding this issue. Natural immunity should be considered for public health policy regarding SARS-CoV-2.

  View details for DOI 10.1016/j.envres.2022.112911

  View details for PubMedID 35149106

• Antenatal corticosteroids prior to planned caesarean at term for improving neonatal outcomes. The Cochrane database of systematic reviews Sotiriadis, A., McGoldrick, E., Makrydimas, G., Papatheodorou, S., Ioannidis, J. P., Stewart, F., Parker, R. 1800; 12: CD006614

  Abstract

  BACKGROUND: Infants born at term by elective caesarean section are more likely to develop respiratory morbidity than infants born vaginally. Prophylactic corticosteroids in singleton preterm pregnancies accelerate lung maturation and reduce the incidence of respiratory complications. It is unclear whether administration at term gestations, prior to caesarean section, improves the respiratory outcomes for these babies without causing any unnecessary morbidity to the mother or the infant.OBJECTIVES: The objective of this review was to assess the effect of prophylactic corticosteroid administration before elective caesarean section at term, as compared to usual care (which could be placebo or no treatment), on fetal, neonatal and maternal morbidity. We also assessed the impact of the treatment on the child in later life.SEARCH METHODS: For this update, we searched Cochrane Pregnancy and Childbirth's Trials Register, ClinicalTrials.gov (20 January 2021) and reference lists of retrieved studies.SELECTION CRITERIA: We included randomised controlled trials comparing prophylactic antenatal corticosteroid administration (betamethasone or dexamethasone) with placebo or with no treatment, given before elective caesarean section at term (at or after 37 weeks of gestation). Quasi-randomised and cluster-randomised controlled trials were also eligible for inclusion.DATA COLLECTION AND ANALYSIS: We used standard Cochrane Pregnancy and Childbirth methods for data collection and analysis. Two review authors independently assessed trials for inclusion, assessed risk of bias, evaluated trustworthiness (based on predefined criteria developed by Cochrane Pregnancy and Childbirth), extracted data and checked them for accuracy andassessed the certainty of the evidence using the GRADE approach. Our primary outcomes were respiratory distress syndrome (RDS), transient tachypnoea of the neonate (TTN), admission to neonatal special care for respiratory morbidity and need for mechanical ventilation. We planned to perform subgroup analyses for the primary outcomes according to gestational age at randomisation and type of corticosteroid (betamethasone or dexamethasone). We also planned to perform sensitivity analysis, including only studies at low risk of bias.MAIN RESULTS: We included one trial in which participants were randomised to receive either betamethasone or usual care. The trial included 942 women and 942 neonates recruited from 10 UK hospitals between 1995 and 2002. This review includes only trials that met predefined criteria for trustworthiness. We removed three trials from the analysis that were included in the previous version of this review. The risk of bias was low for random sequence generation, allocation concealment and incomplete outcome data. The risk of bias for selective outcome reporting was unclear because there was no published trial protocol, and therefore it is unclear whether all the planned outcomes were reported in full. Due to a lack of blinding we judged there to be high risk of performance bias and detection bias. We downgraded the certainty of the evidence because of concerns about risk of bias and because of imprecision due to low event rates and wide 95% confidence intervals (CIs), which are consistent with possible benefit and possible harm Compared with usual care, it is uncertain if antenatal corticosteroids reduce the risk of RDS (relative risk (RR) 0.34 95% CI 0.07 to 1.65; 1 study; 942 infants) or TTN (RR 0.52, 95% CI 0.25 to 1.11;1 study; 938 infants) because the certainty of evidence is low and the 95% CIs are consistent with possible benefit and possible harm. Antenatal corticosteroids probably reduce the risk of admission to neonatal special care for respiratory complications, compared with usual care (RR 0.45, 95% CI 0.22 to 0.90; 1 study; 942 infants; moderate-certainty evidence). The proportion of infants admitted to neonatal special care for respiratory morbidityafter treatment with antenatal corticosteroids was 2.3% compared with 5.1% in the usual care group. It is uncertain if antenatal steroids have any effect on the risk of needing mechanical ventilation, compared with usual care (RR 4.07, 95% CI 0.46 to 36.27;1 study; 942 infants; very low-certainty evidence). The effect of antenatal corticosteroids on the maternal development of postpartum infection/pyrexia in the first 72 hours is unclear due to the very low certainty of the evidence; one study (942 women) reported zero cases. The included studies did not report any data for neonatal hypoglycaemia or maternal mortality/severe mortality.AUTHORS' CONCLUSIONS: Evidence from one randomised controlled trial suggests that prophylactic corticosteroids before elective caesarean section at term probably reduces admission to the neonatal intensive care unit for respiratory morbidity. It is uncertain if administration of antenatal corticosteroids reduces the rates of respiratory distress syndrome (RDS) or transient tachypnoea of the neonate (TTN). The overall certainty of the evidence for the primary outcomes was found to be low or very low, apart from the outcome of admission to neonatal special care (all levels) for respiratory morbidity, for which the evidence was of moderate certainty. Therefore, there is currently insufficient data to draw any firm conclusions. More evidence is needed to investigate the effect of prophylactic antenatal corticosteroids on the incidence of recognised respiratory morbidity such as RDS. Any future trials should assess the balance between respiratory benefit and potential immediate adverse effects (e.g. hypoglycaemia) and long-term adverse effects (e.g. academic performance) for the infant. There is very limited information on maternal health outcomes to provide any assurances that corticosteroids do not pose any increased risk of harm to the mother. Further research should consider investigating the effectiveness of antenatal steroids at different gestational ages prior to caesarean section. There are nine potentially eligible studies that are currently ongoing and could be included in future updates of this review.

  View details for DOI 10.1002/14651858.CD006614.pub4

  View details for PubMedID 34935127

• Data-dredging bias. BMJ evidence-based medicine Erasmus, A., Holman, B., Ioannidis, J. P. 1800

  View details for DOI 10.1136/bmjebm-2020-111584

  View details for PubMedID 34930812

• Association between convalescent plasma treatment and mortality in COVID-19: a collaborative systematic review and meta-analysis of randomized clinical trials. BMC infectious diseases Axfors, C., Janiaud, P., Schmitt, A. M., Van't Hooft, J., Smith, E. R., Haber, N. A., Abayomi, A., Abduljalil, M., Abdulrahman, A., Acosta-Ampudia, Y., Aguilar-Guisado, M., Al-Beidh, F., Alejandria, M. M., Alfonso, R. N., Ali, M., AlQahtani, M., AlZamrooni, A., Anaya, J., Ang, M. A., Aomar, I. F., Argumanis, L. E., Averyanov, A., Baklaushev, V. P., Balionis, O., Benfield, T., Berry, S., Birocco, N., Bonifacio, L. B., Bowen, A. C., Bown, A., Cabello-Gutierrez, C., Camacho, B., Camacho-Ortiz, A., Campbell-Lee, S., Cao, D. H., Cardesa, A., Carnate, J. M., Castillo, G. J., Cavallo, R., Chowdhury, F. R., Chowdhury, F. U., Ciccone, G., Cingolani, A., Climacosa, F. M., Compernolle, V., Cortez, C. F., Costa Neto, A., D'Antico, S., Daly, J., Danielle, F., Davis, J. S., De Rosa, F. G., Denholm, J. T., Denkinger, C. M., Desmecht, D., Diaz-Coronado, J. C., Diaz Ponce-Medrano, J. A., Donneau, A., Dumagay, T. E., Dunachie, S., Dungog, C. C., Erinoso, O., Escasa, I. M., Estcourt, L. J., Evans, A., Evasan, A. L., Fareli, C. J., Fernandez-Sanchez, V., Galassi, C., Gallo, J. E., Garcia, P. J., Garcia, P. L., Garcia, J. A., Garigliany, M., Garza-Gonzalez, E., Gauiran, D. T., Gaviria Garcia, P. A., Giron-Gonzalez, J., Gomez-Almaguer, D., Gordon, A. C., Gothot, A., Grass Guaqueta, J. S., Green, C., Grimaldi, D., Hammond, N. E., Harvala, H., Heralde, F. M., Herrick, J., Higgins, A. M., Hills, T. E., Hines, J., Holm, K., Hoque, A., Hoste, E., Ignacio, J. M., Ivanov, A. V., Janssen, M., Jennings, J. H., Jha, V., King, R. A., Kjeldsen-Kragh, J., Klenerman, P., Kotecha, A., Krapp, F., Labanca, L., Laing, E., Landin-Olsson, M., Laterre, P., Lim, L., Lim, J., Ljungquist, O., Llaca-Diaz, J. M., Lopez-Robles, C., Lopez-Cardenas, S., Lopez-Plaza, I., Lucero, J. A., Lundgren, M., Macias, J., Maganito, S. C., Malundo, A. F., Manrique, R. D., Manzini, P. M., Marcos, M., Marquez, I., Martinez-Marcos, F. J., Mata, A. M., McArthur, C. J., McQuilten, Z. K., McVerry, B. J., Menon, D. K., Meyfroidt, G., Mirasol, M. A., Misset, B., Molton, J. S., Mondragon, A. V., Monsalve, D. M., Moradi Choghakabodi, P., Morpeth, S. C., Mouncey, P. R., Moutschen, M., Muller-Tidow, C., Murphy, E., Najdovski, T., Nichol, A. D., Nielsen, H., Novak, R. M., O'Sullivan, M. V., Olalla, J., Osibogun, A., Osikomaiya, B., Oyonarte, S., Pardo-Oviedo, J. M., Patel, M. C., Paterson, D. L., Pena-Perez, C. A., Perez-Calatayud, A. A., Perez-Alba, E., Perkina, A., Perry, N., Pouladzadeh, M., Poyato, I., Price, D. J., Quero, A. K., Rahman, M. M., Rahman, M. S., Ramesh, M., Ramirez-Santana, C., Rasmussen, M., Rees, M. A., Rego, E., Roberts, J. A., Roberts, D. J., Rodriguez, Y., Rodriguez-Bano, J., Rogers, B. A., Rojas, M., Romero, A., Rowan, K. M., Saccona, F., Safdarian, M., Santos, M. C., Sasadeusz, J., Scozzari, G., Shankar-Hari, M., Sharma, G., Snelling, T., Soto, A., Tagayuna, P. Y., Tang, A., Tatem, G., Teofili, L., Tong, S. Y., Turgeon, A. F., Veloso, J. D., Venkatesh, B., Ventura-Enriquez, Y., Webb, S. A., Wiese, L., Wiken, C., Wood, E. M., Yusubalieva, G. M., Zacharowski, K., Zarychanski, R., Khanna, N., Moher, D., Goodman, S. N., Ioannidis, J. P., Hemkens, L. G. 2021; 21 (1): 1170

  Abstract

  BACKGROUND: Convalescent plasma has been widely used to treat COVID-19 and is under investigation in numerous randomized clinical trials, but results are publicly available only for a small number of trials. The objective of this study was to assess the benefits of convalescent plasma treatment compared to placebo or no treatment and all-cause mortality in patients with COVID-19, using data from all available randomized clinical trials, including unpublished and ongoing trials (Open Science Framework, https://doi.org/10.17605/OSF.IO/GEHFX ).METHODS: In this collaborative systematic review and meta-analysis, clinical trial registries (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform), the Cochrane COVID-19 register, the LOVE database, and PubMed were searched until April 8, 2021. Investigators of trials registered by March 1, 2021, without published results were contacted via email. Eligible were ongoing, discontinued and completed randomized clinical trials that compared convalescent plasma with placebo or no treatment in COVID-19 patients, regardless of setting or treatment schedule. Aggregated mortality data were extracted from publications or provided by investigators of unpublished trials and combined using the Hartung-Knapp-Sidik-Jonkman random effects model. We investigated the contribution of unpublished trials to the overall evidence.RESULTS: A total of 16,477 patients were included in 33 trials (20 unpublished with 3190 patients, 13 published with 13,287 patients). 32 trials enrolled only hospitalized patients (including 3 with only intensive care unit patients). Risk of bias was low for 29/33 trials. Of 8495 patients who received convalescent plasma, 1997 died (23%), and of 7982 control patients, 1952 died (24%). The combined risk ratio for all-cause mortality was 0.97 (95% confidence interval: 0.92; 1.02) with between-study heterogeneity not beyond chance (I2=0%). The RECOVERY trial had 69.8% and the unpublished evidence 25.3% of the weight in the meta-analysis.CONCLUSIONS: Convalescent plasma treatment of patients with COVID-19 did not reduce all-cause mortality. These results provide strong evidence that convalescent plasma treatment for patients with COVID-19 should not be used outside of randomized trials. Evidence synthesis from collaborations among trial investigators can inform both evidence generation and evidence application in patient care.

  View details for DOI 10.1186/s12879-021-06829-7

  View details for PubMedID 34800996

• Systematic reviews: guidance relevant for studies of older people (vol 46, pg 722, 2017) AGE AND AGEING Shenkin, S. D., Harrison, J. K., Wilkinson, T., Dodds, R. M., Ioannidis, J. A. 2021; 50 (6): E15

  View details for DOI 10.1093/ageing/afx185

  View details for Web of Science ID 000743035600010

• Effect Sizes Reported in Highly Cited Emotion Research Compared With Larger Studies and Meta-Analyses Addressing the Same Questions CLINICAL PSYCHOLOGICAL SCIENCE Cristea, I. A., Georgescu, R., Ioannidis, J. A. 2021

  View details for DOI 10.1177/21677026211049366

  View details for Web of Science ID 000715474900001

• Overall and COVID-19-specific citation impact of highly visible COVID-19 media experts: bibliometric analysis. BMJ open Ioannidis, J. P., Tezel, A., Jagsi, R. 2021; 11 (10): e052856

  Abstract

  OBJECTIVE: To evaluate whether the COVID-19 experts who appear most frequently in media have high citation impact for their research overall, and for their COVID-19 peer-reviewed publications in particular and to examine the representation of women among such experts.DESIGN: Cross-linking of data sets of most highly visible COVID-19 media experts with citation data on the impact of their published work (career-long publication record and COVID-19-specific work).SETTING: Cable news appearance in prime-time programming or overall media appearances.PARTICIPANTS: Most highly visible COVID-19 media experts in the USA, Switzerland, Greece and Denmark.INTERVENTIONS: None.OUTCOME MEASURES: Citation data from Scopus along with discipline-specific ranks of overall career-long and COVID-19-specific impact based on a previously validated composite citation indicator.RESULTS: We assessed 76 COVID-19 experts who were highly visible in US prime-time cable news, and 50, 12 and 2 highly visible experts in media in Denmark, Greece and Switzerland, respectively. Of those, 23/76, 10/50, 2/12 and 0/2 were among the top 2% of overall citation impact among scientists in the same discipline worldwide. Moreover, 37/76, 15/50, 7/12 and 2/2 had published anything on COVID-19 that was indexed in Scopus as of 30 August 2021. Only 18/76, 6/50, 2/12 and 0/2 of the highly visible COVID-19 media experts were women. 55 scientists in the USA, 5 in Denmark, 64 in Greece and 56 in Switzerland had a higher citation impact for their COVID-19 work than any of the evaluated highly visible media COVID-19 experts in the respective country; 10/55, 2/5, 22/64 and 14/56 of them were women.CONCLUSIONS: Despite notable exceptions, there is a worrisome disconnect between COVID-19 claimed media expertise and scholarship. Highly cited women COVID-19 experts are rarely included among highly visible media experts.

  View details for DOI 10.1136/bmjopen-2021-052856

  View details for PubMedID 34706959

• Medical journal requirements for clinical trial data sharing: Ripe for improvement. PLoS medicine Naudet, F., Siebert, M., Pellen, C., Gaba, J., Axfors, C., Cristea, I., Danchev, V., Mansmann, U., Ohmann, C., Wallach, J. D., Moher, D., Ioannidis, J. P. 2021; 18 (10): e1003844

  Abstract

  Florian Naudet and co-authors discuss strengthening requirements for sharing clinical trial data.

  View details for DOI 10.1371/journal.pmed.1003844

  View details for PubMedID 34695113

• Retrospective Median Power, False Positive Meta-Analysis and Large-Scale Replication. Research synthesis methods Stanley, T. D., Doucouliagos, H., Ioannidis, J. P. 2021

  Abstract

  Recent, high-profile, large-scale, preregistered failures to replicate uncover that many highly-regarded experiments are 'false positives;' that is, statistically significant results of underlying null effects. Large surveys of research reveal that statistical power is often low and inadequate. When the research record includes selective reporting, publication bias and/or questionable research practices, conventional meta-analyses are also likely to be falsely positive. At the core of research credibility lies the relation of statistical power to the rate of false positives. This study finds that high (>50-60%) median retrospective power (MRP) is associated with credible meta-analysis and large-scale, preregistered, multi-lab 'successful' replications; that is, with replications that corroborate the effect in question. When median retrospective power is low (<50%), positive meta-analysis findings should be interpreted with great caution or discounted altogether. This article is protected by copyright. All rights reserved.

  View details for DOI 10.1002/jrsm.1529

  View details for PubMedID 34628722

• Reproducibility in the UK biobank of genome-wide significant signals discovered in earlier genome-wide association studies. Scientific reports O'Sullivan, J. W., Ioannidis, J. P. 2021; 11 (1): 18625

  Abstract

  With the establishment of large biobanks, discovery of single nucleotide variants (SNVs, also known as single nucleotide polymorphisms (SNVs)) associated with various phenotypes has accelerated. An open question is whether genome-wide significant SNVs identified in earlier genome-wide association studies (GWAS) are replicated in later GWAS conducted in biobanks. To address this, we examined a publicly available GWAS database and identified two, independent GWAS on the same phenotype (an earlier, "discovery" GWAS and a later, "replication" GWAS done in the UK biobank). The analysis evaluated 136,318,924 SNVs (of which 6289 reached P<5e-8 in the discovery GWAS) from 4,397,962 participants across nine phenotypes. The overall replication rate was 85.0%; although lower for binary than quantitative phenotypes (58.1% versus 94.8% respectively). There was a 18.0% decrease in SNV effect size for binary phenotypes, but a 12.0% increase for quantitative phenotypes. Using the discovery SNV effect size, phenotype trait (binary or quantitative), and discovery P value, we built and validated a model that predicted SNV replication with area under the Receiver Operator Curve=0.90. While non-replication may reflect lack of power rather than genuine false-positives, these results provide insights about which discovered associations are likely to be replicated across subsequent GWAS.

  View details for DOI 10.1038/s41598-021-97896-y

  View details for PubMedID 34545148

• COVID-19 Vaccination in Children and University Students. European journal of clinical investigation Ioannidis, J. P. 2021: e13678

  Abstract

  Strategies for the use of COVID-19 vaccines in children and young adults (in particular university students) are hotly debated and important to optimize. As of late August 2021, recommendations on the use of these vaccines in children vary across different countries. Recommendations are more uniform for vaccines in young adults, but vaccination uptake in this age group shows a large range across countries. Mandates for vaccination of university students are a particularly debated topic with many campuses endorsing mandates in the USA in contrast to European countries, at least as of August 2021. The commentary discusses the potential indirect impact of vaccination of youth on the COVID-19 burden of disease for other age groups and societal functioning at large, estimates of direct impact on reducing fatalities and non-lethal COVID-19-related events in youth, estimates of potential lethal and non-lethal adverse events from vaccines, and differential considerations that may exist in the USA, European countries, and non-high-income countries. Decision-making for deploying COVID-19 vaccines in young people is subject to residual uncertainty on the future course of the pandemic and potential evolution towards endemicity. Rational recommendations would also benefit from better understanding of the clinical and sociodemographic features of COVID-19 risk in young populations, and from dissecting the role of re-infections and durability of natural versus vaccine-induced immunity.

  View details for DOI 10.1111/eci.13678

  View details for PubMedID 34529274

• The rapid, massive growth of COVID-19 authors in the scientific literature ROYAL SOCIETY OPEN SCIENCE Ioannidis, J. A., Salholz-Hillel, M., Boyack, K. W., Baas, J. 2021; 8 (9): 210389

  Abstract

  We examined the extent to which the scientific workforce in different fields was engaged in publishing COVID-19-related papers. According to Scopus (data cut, 1 August 2021), 210 183 COVID-19-related publications included 720 801 unique authors, of which 360 005 authors had published at least five full papers in their career and 23 520 authors were at the top 2% of their scientific subfield based on a career-long composite citation indicator. The growth of COVID-19 authors was far more rapid and massive compared with cohorts of authors historically publishing on H1N1, Zika, Ebola, HIV/AIDS and tuberculosis. All 174 scientific subfields had some specialists who had published on COVID-19. In 109 of the 174 subfields of science, at least one in 10 active, influential (top 2% composite citation indicator) authors in the subfield had authored something on COVID-19. Fifty-three hyper-prolific authors had already at least 60 (and up to 227) COVID-19 publications each. Among the 300 authors with the highest composite citation indicator for their COVID-19 publications, most common countries were USA (n = 67), China (n = 52), UK (n = 32) and Italy (n = 18). The rapid and massive involvement of the scientific workforce in COVID-19-related work is unprecedented and creates opportunities and challenges. There is evidence for hyper-prolific productivity.

  View details for DOI 10.1098/rsos.210389

  View details for Web of Science ID 000693136700001

  View details for PubMedID 34527271

  View details for PubMedCentralID PMC8422596

• Risk of harm in synthetic and biological intervention trials in patients with inflammatory arthritis: protocol for a metaepidemiological study focusing on contextual factors. BMJ open Malm, E., Nielsen, S. M., Berg, J., Ioannidis, J. P., Furst, D., Smolen, J. S., Taylor, P. C., Kristensen, L. E., Tarp, S., Ellingsen, T., Christensen, R. 2021; 11 (9): e049850

  Abstract

  INTRODUCTION: Inflammatory arthritis (IA) conditions, including rheumatoid arthritis, psoriatic arthritis and axial spondyloarthritis, are characterised by inflammatory infiltration of the joints. Biological disease-modifying antirheumatic drugs (bDMARDs) and targeted synthetic disease-modifying antirheumatic drugs (tsDMARDs), respectively, reduce the effects of proinflammatory cytokines and immune cells to ameliorate disease. However, immunosuppression can be associated with high rates of serious adverse events (SAEs), including serious infections, and maybe an increased risk of malignancies and cardiovascular events. Currently, there is no empirical evidence on the extent to which contextual factors and risk of bias (RoB) domains may modify these harm signals in randomised trials.METHODS AND ANALYSIS: We will search MEDLINE (via PubMed) for systematic reviews published since April 2015 and all Cochrane reviews. From these reviews, randomised trials will be eligible if they include patients with an IA condition with at least one group randomly allocated to bDMARD and/or tsDMARD treatments. A predefined form will be used for extracting data on population characteristics (eg, baseline characteristics or eligibility criteria, such as medication background) and specific harm outcome measures, such as number of withdrawals, numbers of patients discontinuing due to adverse events and number of patients having SAEs. RoB in individual trials will be assessed using a modified Cochrane RoB tool. We will estimate the potentially causal harm effects related to the experimental intervention compared with control comparator as risk ratios, and heterogeneity across randomised comparisons will be assessed statistically and evaluated as inconsistency using the I2 Index. Our metaregression analyses will designate population and trial characteristics and each RoB domain as independent variables, whereas the three harm domains will serve as dependent variables.ETHICS AND DISSEMINATION: Ethics approval is not required for this study. Results will be disseminated through publication in international peer-reviewed journals.PROSPERO REGISTRATION NUMBER: CRD42020171124.

  View details for DOI 10.1136/bmjopen-2021-049850

  View details for PubMedID 34489286

• Large Pediatric Randomized Clinical Trials in ClinicalTrials.gov. Pediatrics Cho, S. M., Serghiou, S., Ioannidis, J. P., Klassen, T. P., Contopoulos-Ioannidis, D. G. 2021; 148 (3)

  Abstract

  BACKGROUND: Large, randomized controlled trials (RCTs) are essential in answering pivotal questions in child health.METHODS: We created a bird's eye view of all large, noncluster, nonvaccine pediatric RCTs with ≥1000 participants registered in ClinicalTrials.gov (last search January 9, 2020). We analyzed the funding sources, countries, outcomes, publication status, and correlation with the pediatric global burden of disease (GBD) for eligible trials.RESULTS: We identified 247 large, nonvaccine, noncluster pediatric RCTs. Only 17 mega-trials with ≥5000 participants existed. Industry funding was involved in only 52 (21%) and exclusively funded 47 (19%) trials. Participants were from high-income countries (HICs) in 100 (40%) trials, from lower-middle-income countries (LMICs) in 122 (49%) trials, and from both HICs and LMICs in 19 (8%) trials; 6 trials did not report participants' country location. Of trials conducted in LMIC, 43% of investigators were from HICs. Of non-LMIC participants trials (HIC or HIC and LMIC), 39% were multicountry trials versus 11% of exclusively LMIC participants trials. Few trials (18%; 44 of 247) targeted mortality as an outcome. 35% (58 of 164) of the trials completed ≥12 months were unpublished at the time of our assessment. The number of trials per disease category correlated well with pediatric GBD overall (rho = 0.76) and in LMICs (rho = 0.69), but not in HICs (rho = 0.29).CONCLUSIONS: Incentivization of investigator collaborations across diverse country settings, timely publication of results of large pediatric RCTs, and alignment with the pediatric GBD are of pivotal importance to ultimately improve child health globally.

  View details for DOI 10.1542/peds.2020-049771

  View details for PubMedID 34465592

• Evaluation of a suggested novel method to adjust BMI calculated from self-reported weight and height for measurement error. Obesity (Silver Spring, Md.) Flegal, K. M., Graubard, B. I., Ioannidis, J. P. 2021

  Abstract

  OBJECTIVE: In 2019, Ward et al. proposed a method to adjust BMI calculated from self-reported weight and height for bias relative to measured data. They did not evaluate the adjusted values relative to measured BMI values for the same individuals.METHODS: A large data set (n = 37,439) with both measured and self-reported weight and height was randomly divided into two groups. The proposed method was used to adjust the BMI values in one group to the measured data from the other group. The adjusted values were then compared with the measured values for the same individuals.RESULTS: Before adjustment, 24.9% were incorrectly classified relative to measured BMI categories, including 7.9% in too high a category; after adjustment, 24.3% were incorrectly classified, with 12.8% in too high a category. The variance of the difference was unchanged. The adjustments reduced some errors and introduced new errors. At an individual level, results were unpredictable.CONCLUSIONS: The suggested method has little effect on misclassification, can introduce new errors, and could magnify errors associated with factors, such as age, race, educational level, or other characteristics. State-level estimates and projections of obesity prevalence from values adjusted by this method may be incorrect.

  View details for DOI 10.1002/oby.23239

  View details for PubMedID 34448365

• Data Mining Approaches to Reference Interval Studies. Clinical chemistry Moderators, Obstfeld, A. E., Patel, K., Experts, Boyd, J. C., Drees, J., Holmes, D. T., Ioannidis, J. P., Manrai, A. K. 2021

  View details for DOI 10.1093/clinchem/hvab137

  View details for PubMedID 34402506

• Over- and under-estimation of COVID-19 deaths. European journal of epidemiology Ioannidis, J. P. 2021

  Abstract

  The ratio of COVID-19-attributable deaths versus "true" COVID-19 deaths depends on the synchronicity of the epidemic wave with population mortality; duration of test positivity, diagnostic time window, and testing practices close to and at death; infection prevalence; the extent of diagnosing without testing documentation; and the ratio of overall (all-cause) population mortality rate and infection fatality rate. A nomogram is offered to assess the potential extent of over- and under-counting in different situations. COVID-19 deaths were apparently under-counted early in the pandemic and continue to be under-counted in several countries, especially in Africa, while over-counting probably currently exists for several other countries, especially those with intensive testing and high sensitization and/or incentives for COVID-19 diagnoses. Death attribution in a syndemic like COVID-19 needs great caution. Finally, excess death estimates are subject to substantial annual variability and include also indirect effects of the pandemic and the effects of measures taken.

  View details for DOI 10.1007/s10654-021-00787-9

  View details for PubMedID 34322831

• Comprehensive mapping of local and diaspora scientists: A database and analysis of 63,951 Greek scientists QUANTITATIVE SCIENCE STUDIES Ioannidis, J. A., Koutsioumpa, C., Vakka, A., Agoranos, G., Mantsiou, C., Drekolia, M., Avramidis, N., Contopoulos-Ioannidis, D. G., Drosatos, K., Baas, J. 2021; 2 (2): 733-752

  View details for DOI 10.1162/qss_a_00136

  View details for Web of Science ID 000753939000016

• Validity of observational evidence on putative risk and protective factors: appraisal of 3744 meta-analyses on 57 topics. BMC medicine Janiaud, P., Agarwal, A., Tzoulaki, I., Theodoratou, E., Tsilidis, K. K., Evangelou, E., Ioannidis, J. P. 2021; 19 (1): 157

  Abstract

  BACKGROUND: The validity of observational studies and their meta-analyses is contested. Here, we aimed to appraise thousands of meta-analyses of observational studies using a pre-specified set of quantitative criteria that assess the significance, amount, consistency, and bias of the evidence. We also aimed to compare results from meta-analyses of observational studies against meta-analyses of randomized controlled trials (RCTs) and Mendelian randomization (MR) studies.METHODS: We retrieved from PubMed (last update, November 19, 2020) umbrella reviews including meta-analyses of observational studies assessing putative risk or protective factors, regardless of the nature of the exposure and health outcome. We extracted information on 7 quantitative criteria that reflect the level of statistical support, the amount of data, the consistency across different studies, and hints pointing to potential bias. These criteria were level of statistical significance (pre-categorized according to 10-6, 0.001, and 0.05 p-value thresholds), sample size, statistical significance for the largest study, 95% prediction intervals, between-study heterogeneity, and the results of tests for small study effects and for excess significance.RESULTS: 3744 associations (in 57 umbrella reviews) assessed by a median number of 7 (interquartile range 4 to 11) observational studies were eligible. Most associations were statistically significant at P < 0.05 (61.1%, 2289/3744). Only 2.6% of associations had P < 10-6, ≥1000 cases (or ≥20,000 participants for continuous factors), P < 0.05 in the largest study, 95% prediction interval excluding the null, and no large between-study heterogeneity, small study effects, or excess significance. Across the 57 topics, large heterogeneity was observed in the proportion of associations fulfilling various quantitative criteria. The quantitative criteria were mostly independent from one another. Across 62 associations assessed in both RCTs and in observational studies, 37.1% had effect estimates in opposite directions and 43.5% had effect estimates differing beyond chance in the two designs. Across 94 comparisons assessed in both MR and observational studies, such discrepancies occurred in 30.8% and 54.7%, respectively.CONCLUSIONS: Acknowledging that no gold-standard exists to judge whether an observational association is genuine, statistically significant results are common in observational studies, but they are rarely convincing or corroborated by randomized evidence.

  View details for DOI 10.1186/s12916-021-02020-6

  View details for PubMedID 34225716

• Detecting Publication Selection Bias Through Excess Statistical Significance. Research synthesis methods Stanley, T. D., Doucouliagos, H., Ioannidis, J. P., Carter, E. C. 2021

  Abstract

  We introduce and evaluate three tests for publication selection bias based on excess statistical significance. The proposed tests incorporate heterogeneity explicitly in the formulas for expected and excess statistical significance. We calculate the expected proportion of statistically significant findings in the absence of selective reporting or publication bias based on each study's standard error and meta-analysis estimates of the mean and variance of the true-effect distribution. Comparing the expected to the observed proportion of statistically significant results leads to a simple proportion of statistical significance test (PSST). Alternatively, we propose a direct test of excess statistical significance (TESS). We also combine these two tests of excess statistical significance (TESSPSST). Simulations show that these excess statistical significance tests often outperform the conventional Egger test for publication selection bias and the three-parameter selection model. This article is protected by copyright. All rights reserved.

  View details for DOI 10.1002/jrsm.1512

  View details for PubMedID 34196473

• Citation Patterns Following a Strongly Contradictory Replication Result: Four Case Studies From Psychology ADVANCES IN METHODS AND PRACTICES IN PSYCHOLOGICAL SCIENCE Hardwicke, T. E., Szucs, D., Thibault, R. T., Cruwell, S., van den Akker, O. R., Nuijten, M. B., Ioannidis, J. A. 2021; 4 (3)

  View details for DOI 10.1177/25152459211040837

  View details for Web of Science ID 000708944200001

• An Umbrella Review of Effect Size, Bias and Power Across Meta-Analyses in Emergency Medicine. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine Parish, A. J., Yuan, D. M., Raggi, J. R., Omotoso, O. O., West, J. R., Ioannidis, J. P. 2021

  Abstract

  OBJECTIVES: To conduct an umbrella review of therapeutic studies relevant to emergency medicine, analyzing patterns in effect size, power and signals of potential bias across an entire field of clinical research.METHODS: We combined topic and journal-driven searches of PUBMED and Google Scholar for published articles of systematic reviews and meta-analyses relevant to emergency medicine (last search in November 2020). Data were screened and extracted by 6 investigators. Redundant meta-analyses were removed. Whenever possible for each comparison we extracted one meta-analysis on mortality with the most events, and one meta-analysis on a non-mortality outcome with the most studies. From each meta-analysis we extracted all individual study effects; outcomes were converted to odds ratios and placed on a common scale where an odds ratio <1.0 represents a reduction in a harmful outcome with an experimental treatment versus control. Outcomes were analyzed at the level of individual studies and at the level of summary effects across meta-analyses.RESULTS: 332 articles contained 431 eligible meta-analyses with a total of 3129 individual study outcomes; of these, 2593 (83%) were from randomized controlled trials. The median odds ratio across all studies was 0.70. Within each meta-analysis, the earliest study effect on average demonstrated larger benefit compared to the overall summary effect. Only 57 of 431meta-analyses (13%) both favored the experimental intervention and did not show any signal of small study effects or excess significance, and of those only 12had at least one study with 80% or higher power to detect an odds ratio of 0.70. Of these, no interventions significantly decreased mortality in well-powered trials. Although the power of studies increased somewhat over time, the majority of studies were underpowered.CONCLUSIONS: Few interventions studied within systematic reviews and meta-analyses relevant to emergency medicine seem to have strong and unbiased evidence for improving outcomes. The field would benefit from more optimally powered trials.

  View details for DOI 10.1111/acem.14312

  View details for PubMedID 34133813

• Preventive psychiatry: a blueprint for improving the mental health of young people. World psychiatry : official journal of the World Psychiatric Association (WPA) Fusar-Poli, P., Correll, C. U., Arango, C., Berk, M., Patel, V., Ioannidis, J. P. 2021; 20 (2): 200-221

  Abstract

  Preventive approaches have latterly gained traction for improving mental health in young people. In this paper, we first appraise the conceptual foundations of preventive psychiatry, encompassing the public health, Gordon's, US Institute of Medicine, World Health Organization, and good mental health frameworks, and neurodevelopmentally-sensitive clinical staging models. We then review the evidence supporting primary prevention of psychotic, bipolar and common mental disorders and promotion of good mental health as potential transformative strategies to reduce the incidence of these disorders in young people. Within indicated approaches, the clinical high-risk for psychosis paradigm has received the most empirical validation, while clinical high-risk states for bipolar and common mental disorders are increasingly becoming a focus of attention. Selective approaches have mostly targeted familial vulnerability and non-genetic risk exposures. Selective screening and psychological/psychoeducational interventions in vulnerable subgroups may improve anxiety/depressive symptoms, but their efficacy in reducing the incidence of psychotic/bipolar/common mental disorders is unproven. Selective physical exercise may reduce the incidence of anxiety disorders. Universal psychological/psychoeducational interventions may improve anxiety symptoms but not prevent depressive/anxiety disorders, while universal physical exercise may reduce the incidence of anxiety disorders. Universal public health approaches targeting school climate or social determinants (demographic, economic, neighbourhood, environmental, social/cultural) of mental disorders hold the greatest potential for reducing the risk profile of the population as a whole. The approach to promotion of good mental health is currently fragmented. We leverage the knowledge gained from the review to develop a blueprint for future research and practice of preventive psychiatry in young people: integrating universal and targeted frameworks; advancing multivariable, transdiagnostic, multi-endpoint epidemiological knowledge; synergically preventing common and infrequent mental disorders; preventing physical and mental health burden together; implementing stratified/personalized prognosis; establishing evidence-based preventive interventions; developing an ethical framework, improving prevention through education/training; consolidating the cost-effectiveness of preventive psychiatry; and decreasing inequalities. These goals can only be achieved through an urgent individual, societal, and global level response, which promotes a vigorous collaboration across scientific, health care, societal and governmental sectors for implementing preventive psychiatry, as much is at stake for young people with or at risk for emerging mental disorders.

  View details for DOI 10.1002/wps.20869

  View details for PubMedID 34002494

• Challenges and lessons learned from Covid-19 trials - should we be doing clinical trials differently? The Canadian journal of cardiology Janiaud, P., Hemkens, L. G., Ioannidis, J. P. 2021

  Abstract

  The COVID-19 crisis led to a flurry of clinical trials activity. The COVID-Evidence database shows 2,814 COVID-19 randomized trials registered as of February 16, 2021. Most were small (only 18% have a planned sample size >500) and the rare completed ones have not provided published results promptly (only 283 trial publications as of 2/2021). Small randomized trials and observational, non-randomized analyses have not had a successful track record and have generated misleading expectations. Different large trials on the same intervention have generally been far more efficient in producing timely and consistent evidence. The rapid generation of evidence and accelerated dissemination of results have led to new challenges for systematic reviews and meta-analyses (e.g. rapid, living, and scoping reviews). Pressure to regulatory agencies has also mounted with massive emergency authorizations, but some of them have had to be revoked. Pandemic circumstances have disrupted the way trials are conducted; therefore, new methods have been developed and adopted more widely to facilitate recruitment, consent, and overall trial conduct. Based on the COVID-19 experience and its challenges, planning of several large, efficient trials, and wider use of adaptive designs may change the future of clinical research. Pragmatism, integration in clinical care, efficient administration, promotion of collaborative structures, and enhanced integration of existing data and facilities may be several of the legacies of COVID-19 on future randomized trials.

  View details for DOI 10.1016/j.cjca.2021.05.009

  View details for PubMedID 34077789

• Author Correction: Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials. Nature communications Axfors, C., Schmitt, A. M., Janiaud, P., Van't Hooft, J., Abd-Elsalam, S., Abdo, E. F., Abella, B. S., Akram, J., Amaravadi, R. K., Angus, D. C., Arabi, Y. M., Azhar, S., Baden, L. R., Baker, A. W., Belkhir, L., Benfield, T., Berrevoets, M. A., Chen, C., Chen, T., Cheng, S., Cheng, C., Chung, W., Cohen, Y. Z., Cowan, L. N., Dalgard, O., de Almeida E Val, F. F., de Lacerda, M. V., de Melo, G. C., Derde, L., Dubee, V., Elfakir, A., Gordon, A. C., Hernandez-Cardenas, C. M., Hills, T., Hoepelman, A. I., Huang, Y., Igau, B., Jin, R., Jurado-Camacho, F., Khan, K. S., Kremsner, P. G., Kreuels, B., Kuo, C., Le, T., Lin, Y., Lin, W., Lin, T., Lyngbakken, M. N., McArthur, C., McVerry, B. J., Meza-Meneses, P., Monteiro, W. M., Morpeth, S. C., Mourad, A., Mulligan, M. J., Murthy, S., Naggie, S., Narayanasamy, S., Nichol, A., Novack, L. A., O'Brien, S. M., Okeke, N. L., Perez, L., Perez-Padilla, R., Perrin, L., Remigio-Luna, A., Rivera-Martinez, N. E., Rockhold, F. W., Rodriguez-Llamazares, S., Rolfe, R., Rosa, R., Rosjo, H., Sampaio, V. S., Seto, T. B., Shahzad, M., Soliman, S., Stout, J. E., Thirion-Romero, I., Troxel, A. B., Tseng, T., Turner, N. A., Ulrich, R. J., Walsh, S. R., Webb, S. A., Weehuizen, J. M., Velinova, M., Wong, H., Wrenn, R., Zampieri, F. G., Zhong, W., Moher, D., Goodman, S. N., Ioannidis, J. P., Hemkens, L. G. 2021; 12 (1): 3001

  View details for DOI 10.1038/s41467-021-23559-1

  View details for PubMedID 33990619

• Commentary: Time to improve the reporting of harms in randomized controlled trials. Journal of clinical epidemiology Junqueira, D. R., Phillips, R., Zorzela, L., Golder, S., Loke, Y., Moher, D., Ioannidis John, P. A., Vohra, S. 2021

  View details for DOI 10.1016/j.jclinepi.2021.04.020

  View details for PubMedID 33984494

• EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations. Epidemiology and psychiatric sciences Boesen, K., Gotzsche, P. C., Ioannidis, J. P. 2021; 30: e35

  Abstract

  AIMS: The European Medicines Agency (EMA) and the US Food and Drug Administration (FDA) produce guidelines for the design of pivotal psychiatric drug trials used in new drug applications. It is unknown who are involved in the guideline development and what specific trial design recommendations they give.METHODS: Cross-sectional study of EMA Clinical Efficacy and Safety Guidelines and FDA Guidance Documents. Study outcomes: (1) guideline committee members and declared conflicts of interest; (2) guideline development and organisation of commenting phases; (3) categorisation of stakeholders who comment on draft and final guidelines according to conflicts of interest ('industry', 'not-industry but with industry-related conflicts', 'independent', 'unclear'); and (4) trial design recommendations (trial duration, psychiatric comorbidity, 'enriched design', efficacy outcomes, comparator choice). Protocol registration https://doi.org/10.1101/2020.01.22.20018499 (27 January 2020).RESULTS: We included 13 EMA and five FDA guidelines covering 15 psychiatric indications. Eleven months after submission, the EMA had not processed our request regarding committee member disclosures. FDA offices draft the Guidance Documents, but the Agency is not in possession of employee conflicts of interest declarations because FDA employees generally may not hold financial interests (although some employees may hold interests up to $15,000). The EMA and FDA guideline development phases are similar; drafts and final versions are publicly announced and everybody can submit comments. Seventy stakeholders commented on ten guidelines: 38 (54%) 'industry', 18 (26%) 'not-industry but with industry-related conflicts', six (9%) 'independent' and eight (11%) 'unclear'. They submitted 1014 comments: 640 (68%) 'industry', 243 (26%) 'not-industry but with industry-related conflicts', 44 (5%) 'independent' and 20 (2%) 'unclear' (67 could not be assigned to a specific stakeholder). The recommended designs were generally for trials of short duration; with restricted trial populations; allowing previous exposure to the drug; and often recommending rating scale efficacy outcomes. EMA mainly recommended three arm designs (both placebo and active comparators), whereas FDA mainly recommended placebo-controlled designs. There were also other important differences and FDA's recommendations regarding the exclusion of psychiatric comorbidity seemed less restrictive.CONCLUSIONS: The EMA and FDA clinical research guidelines for psychiatric pivotal trials recommend designs that tend to have limited generalisability. Independent and non-conflicted stakeholders are underrepresented in the guideline development. It seems warranted with more active involvement of scientists and independent organisations without conflicts of interest in the guideline development process.

  View details for DOI 10.1017/S2045796021000147

  View details for PubMedID 33926608

• Intent to share Annals of Internal Medicine's trial data was not associated with data re-use. Journal of clinical epidemiology Pellen, C., Caquelin, L., Bail, A. J., Gaba, J., Verin, M., Moher, D., Ioannidis, J. P., Naudet, F. 2021

  Abstract

  OBJECTIVE: To explore the impact of the Annals of Internal Medicine (AIM) data-sharing policy for randomized controlled trials (RCTs) in terms of output from data-sharing (i.e. publications re-using the data).STUDY DESIGN AND SETTING: Retrospective study. RCTs published in the AIM between 2007 and 2017 were retrieved on PubMed. Publications where the data had been re-used were identified on Web of Science. Searches were performed by two independent reviewers. The primary outcome was any published re-use of the data (re-analysis, secondary analysis, or meta-analysis of individual participant data [MIPD]), where the first, last and corresponding authors were not among the authors of the RCT. Analyses used Cox (primary analysis) models adjusting for RCTs characteristics (registration: https://osf.io/8pj5e/).RESULTS: 185 RCTs were identified. 106 (57%) mentioned willingness to share data and 79 (43%) did not. 208 secondary analyses, 67 MIPD and no re-analyses were identified. No significant association was found between intent to share and re-use where the first, last and corresponding authors were not among the authors of the primary RCT (adjusted hazard ratio = 1.04 [0.47-2.30]).CONCLUSION: Over ten years, RCTs published in AIM expressing an intention to share data were not associated with more extensive re-use of the data.

  View details for DOI 10.1016/j.jclinepi.2021.04.011

  View details for PubMedID 33915263

• Educating educators on research on research. Perspectives on medical education Ioannidis, J. P. 2021

  View details for DOI 10.1007/s40037-021-00662-z

  View details for PubMedID 33877586

• Reconciling estimates of global spread and infection fatality rates of COVID-19: an overview of systematic evaluations. European journal of clinical investigation Ioannidis, J. P. 2021: e13554

  Abstract

  BACKGROUND: Estimates of community spread and infection fatality rate (IFR) of COVID-19 have varied across studies. Efforts to synthesize the evidence reach seemingly discrepant conclusions.METHODS: Systematic evaluations of seroprevalence studies that had no restrictions based on country and which estimated either total number of people infected and/or aggregate IFRs were identified. Information was extracted and compared on eligibility criteria, searches, amount of evidence included, corrections/adjustments of seroprevalence and death counts, quantitative syntheses and handling of heterogeneity, main estimates, and global representativeness.RESULTS: Six systematic evaluations were eligible. Each combined data from 10-338 studies (9-50 countries), because of different eligibility criteria. Two evaluations had some overt flaws in data, violations of stated eligibility criteria, and biased eligibility criteria (e.g. excluding studies with few deaths) that consistently inflated IFR estimates. Perusal of quantitative synthesis methods also exhibited several challenges and biases. Global representativeness was low with 78-100% of the evidence coming from Europe or the Americas; the two most problematic evaluations considered only 1 study from other continents. Allowing for these caveats, 4 evaluations largely agreed in their main final estimates for global spread of the pandemic and the other two evaluations would also agree after correcting overt flaws and biases.CONCLUSIONS: All systematic evaluations of seroprevalence data converge that SARS-CoV-2 infection is widely spread globally. Acknowledging residual uncertainties, the available evidence suggests average global IFR of ~0.15% and ~1.5-2.0 billion infections by February 2021 with substantial differences in IFR and in infection spread across continents, countries, and locations.

  View details for DOI 10.1111/eci.13554

  View details for PubMedID 33768536

• Effect Estimates of COVID-19 Non-Pharmaceutical Interventions are Non-Robust and Highly Model-Dependent. Journal of clinical epidemiology Chin, V., Ioannidis, J. P., Tanner, M. A., Cripps, S. 2021

  Abstract

  OBJECTIVE: To compare the inference regarding the effectiveness of the various non-pharmaceutical interventions (NPIs) for COVID-19 obtained from different SIR models.STUDY DESIGN AND SETTING: We explored two models developed by Imperial College that considered only NPIs without accounting for mobility (model 1) or only mobility (model 2), and a model accounting for the combination of mobility and NPIs (model 3). Imperial College applied models 1 and 2 to 11 European countries and to the USA, respectively. We applied these models to 14 European countries (original 11 plus another 3), over two different time horizons.RESULTS: While model 1 found that lockdown was the most effective measure in the original 11 countries, model 2 showed that lockdown had little or no benefit as it was typically introduced at a point when the time-varying reproduction number was already very low. Model 3 found that the simple banning of public events was beneficial, while lockdown had no consistent impact. Based on Bayesian metrics, model 2 was better supported by the data than either model 1 or model 3 for both time horizons.CONCLUSIONS: Inferences on effects of NPIs are non-robust and highly sensitive to model specification. In the SIR modeling framework, the impacts of lockdown are uncertain and highly model dependent.

  View details for DOI 10.1016/j.jclinepi.2021.03.014

  View details for PubMedID 33781862

• Response to Letters Re: "Assessing mandatory stay- at- home and business closure effects on the spread of COVID- 19". European journal of clinical investigation Bendavid, E., Oh, C., Bhattacharya, J., Ioannidis, J. P. 2021: e13553

  Abstract

  We are pleased to see the active discussion around our study on the relationship between mandatory stay- at- home and business closures and COVID-19 spread.1 In this response, we address issues raised in three letters.2-4 The claim that the study had sample size of n=10 countries is incorrect.2 Each of the 16 regression models represented in Figure 4 included, on average, 1,362 data points (range 771-3,493) on 52 subnational units (range 27-129). Each panel regression is, in effect, a "mini-meta-analysis": the effect size is evaluated within each subnational unit, and the overall effect size is estimated from a pooling of these "within" effects. So while we aggregated the results to 10 countries, the sample size is not n=10.

  View details for DOI 10.1111/eci.13553

  View details for PubMedID 33756017

• Cohort Profile: WELL living laboratory in China (WELL-China). International journal of epidemiology Min, Y., Zhao, X., Hsing, A. W., Zhu, S. 2021

  View details for DOI 10.1093/ije/dyaa283

  View details for PubMedID 33712826

• Metformin and health outcomes: an umbrella review of systematic reviews with meta-analyses. European journal of clinical investigation Li, X., Celotto, S., Pizzol, D., Gasevic, D., Ji, M., Barnini, T., Solmi, M., Stubbs, B., Smith, L., Lopez Sanchez, G. F., Pesolillo, G., Yu, Z., Tzoulaki, I., Theodoratou, E., Ioannidis, J. P., Veronese, N., Demurtas, J. 2021: e13536

  Abstract

  BACKGROUND: The objective was to capture the breadth of outcomes that have been associated with metformin use and to systematically assess the quality, strength and credibility of these associations using the umbrella review methodology.METHODS: Four major databases were searched until 31 May 2020. Meta-analyses of observational studies and meta-analyses of randomised controlled trials (RCTs) (including active and placebo control arms) were included.RESULTS: From 175 eligible publications, we identified 427 different meta-analyses, including 167 meta-analyses of observational studies, 147 meta-analyses of RCTs for metformin vs. placebo/no treatment and 113 meta-analyses of RCTs for metformin vs. active medications. There was no association classified as convincing or highly suggestive from meta-analyses of observational studies, but some suggestive/weak associations of metformin use with a lower mortality risk of CVD and cancer. In meta-analyses of RCTs, metformin was associated with a lower incidence of diabetes in people with pre-diabetes or no diabetes at baseline; lower ovarian hyperstimulation syndrome incidence (in women in controlled ovarian stimulation); higher success for clinical pregnancy rate in Poly-Cystic Ovary Syndrome (PCOS); significant reduction in body mass index in people with type 1 diabetes mellitus, in women who have obesity/overweight with PCOS and in obese/overweight women. Of 175 publications, 166 scored as low or critically low quality per AMSTAR 2 criteria.CONCLUSIONS: Observational evidence on metformin seems largely unreliable. Randomized evidence shows benefits for preventing diabetes and in some gynecological and obstetrical settings. However, almost all meta-analyses are of low or critically low quality according to AMSTAR 2 criteria.

  View details for DOI 10.1111/eci.13536

  View details for PubMedID 33709434

• Association of Convalescent Plasma Treatment With Clinical Outcomes in Patients With COVID-19: A Systematic Review and Meta-analysis. JAMA Janiaud, P., Axfors, C., Schmitt, A. M., Gloy, V., Ebrahimi, F., Hepprich, M., Smith, E. R., Haber, N. A., Khanna, N., Moher, D., Goodman, S. N., Ioannidis, J. P., Hemkens, L. G. 2021

  Abstract

  Importance: Convalescent plasma is a proposed treatment for COVID-19.Objective: To assess clinical outcomes with convalescent plasma treatment vs placebo or standard of care in peer-reviewed and preprint publications or press releases of randomized clinical trials (RCTs).Data Sources: PubMed, the Cochrane COVID-19 trial registry, and the Living Overview of Evidence platform were searched until January 29, 2021.Study Selection: The RCTs selected compared any type of convalescent plasma vs placebo or standard of care for patients with confirmed or suspected COVID-19 in any treatment setting.Data Extraction and Synthesis: Two reviewers independently extracted data on relevant clinical outcomes, trial characteristics, and patient characteristics and used the Cochrane Risk of Bias Assessment Tool. The primary analysis included peer-reviewed publications of RCTs only, whereas the secondary analysis included all publicly available RCT data (peer-reviewed publications, preprints, and press releases). Inverse variance-weighted meta-analyses were conducted to summarize the treatment effects. The certainty of the evidence was assessed using the Grading of Recommendations Assessment, Development, and Evaluation.Main Outcomes and Measures: All-cause mortality, length of hospital stay, clinical improvement, clinical deterioration, mechanical ventilation use, and serious adverse events.Results: A total of 1060 patients from 4 peer-reviewed RCTs and 10 722 patients from 6 other publicly available RCTs were included. The summary risk ratio (RR) for all-cause mortality with convalescent plasma in the 4 peer-reviewed RCTs was 0.93 (95% CI, 0.63 to 1.38), the absolute risk difference was -1.21% (95% CI, -5.29% to 2.88%), and there was low certainty of the evidence due to imprecision. Across all 10 RCTs, the summary RR was 1.02 (95% CI, 0.92 to 1.12) and there was moderate certainty of the evidence due to inclusion of unpublished data. Among the peer-reviewed RCTs, the summary hazard ratio was 1.17 (95% CI, 0.07 to 20.34) for length of hospital stay, the summary RR was 0.76 (95% CI, 0.20 to 2.87) for mechanical ventilation use (the absolute risk difference for mechanical ventilation use was -2.56% [95% CI, -13.16% to 8.05%]), and there was low certainty of the evidence due to imprecision for both outcomes. Limited data on clinical improvement, clinical deterioration, and serious adverse events showed no significant differences.Conclusions and Relevance: Treatment with convalescent plasma compared with placebo or standard of care was not significantly associated with a decrease in all-cause mortality or with any benefit for other clinical outcomes. The certainty of the evidence was low to moderate for all-cause mortality and low for other outcomes.

  View details for DOI 10.1001/jama.2021.2747

  View details for PubMedID 33635310

• Gender-related variables for health research. Biology of sex differences Nielsen, M. W., Stefanick, M. L., Peragine, D., Neilands, T. B., Ioannidis, J. P., Pilote, L., Prochaska, J. J., Cullen, M. R., Einstein, G., Klinge, I., LeBlanc, H., Paik, H. Y., Schiebinger, L. 2021; 12 (1): 23

  Abstract

  BACKGROUND: In this paper, we argue for Gender as a Sociocultural Variable (GASV) as a complement to Sex as a Biological Variable (SABV). Sex (biology) and gender (sociocultural behaviors and attitudes) interact to influence health and disease processes across the lifespan-which is currently playing out in the COVID-19 pandemic. This study develops a gender assessment tool-the Stanford Gender-Related Variables for Health Research-for use in clinical and population research, including large-scale health surveys involving diverse Western populations. While analyzing sex as a biological variable is widely mandated, gender as a sociocultural variable is not, largely because the field lacks quantitative tools for analyzing the influence of gender on health outcomes.METHODS: We conducted a comprehensive review of English-language measures of gender from 1975 to 2015 to identify variables across three domains: gender norms, gender-related traits, and gender relations. This yielded 11 variables tested with 44 items in three US cross-sectional survey populations: two internet-based (N = 2051; N = 2135) and a patient-research registry (N = 489), conducted between May 2017 and January 2018.RESULTS: Exploratory and confirmatory factor analyses reduced 11 constructs to 7 gender-related variables: caregiver strain, work strain, independence, risk-taking, emotional intelligence, social support, and discrimination. Regression analyses, adjusted for age, ethnicity, income, education, sex assigned at birth, and self-reported gender identity, identified associations between these gender-related variables and self-rated general health, physical and mental health, and health-risk behaviors.CONCLUSION: Our new instrument represents an important step toward developing more comprehensive and precise survey-based measures of gender in relation to health. Our questionnaire is designed to shed light on how specific gender-related behaviors and attitudes contribute to health and disease processes, irrespective of-or in addition to-biological sex and self-reported gender identity. Use of these gender-related variables in experimental studies, such as clinical trials, may also help us understand if gender factors play an important role as treatment-effect modifiers and would thus need to be further considered in treatment decision-making.

  View details for DOI 10.1186/s13293-021-00366-3

  View details for PubMedID 33618769

• Comparison of different scoring methods based on latent variable models of the PHQ-9: an individual participant data meta-analysis. Psychological medicine Fischer, F., Levis, B., Falk, C., Sun, Y., Ioannidis, J. P., Cuijpers, P., Shrier, I., Benedetti, A., Thombs, B. D., Depression Screening Data (DEPRESSD) PHQ Collaboration, He, C., Krishnan, A., Wu, Y., Negeri, Z., Bhandari, P. M., Neupane, D., Rice, D. B., Riehm, K. E., Saadat, N., Azar, M., Imran, M., Boruff, J., Kloda, L. A., Patten, S. B., Ziegelstein, R. C., Markham, S., Amtmann, D., Ayalon, L., Baradaran, H. R., Beraldi, A., Bernstein, C. N., Bombardier, C. H., Carter, G., Chagas, M. H., Chibanda, D., Clover, K., Conwell, Y., Diez-Quevedo, C., Fann, J. R., Gibson, L. J., Green, E. P., Greeno, C. G., Jette, N., Khamseh, M. E., Kwan, Y., Lara, M. A., Loureiro, S. R., Lowe, B., Ann Marrie, R., Marsh, L., Marx, B. P., Navarrete, L., Osorio, F. L., Picardi, A., Pugh, S. L., Quinn, T. J., Rooney, A. G., Shinn, E. H., Sidebottom, A., Simning, A., Spangenberg, L., Tan, P. L., Taylor-Rowan, M., Turner, A., van Weert, H. C., Wagner, L. I., White, J. 2021: 1–12

  Abstract

  BACKGROUND: Previous research on the depression scale of the Patient Health Questionnaire (PHQ-9) has found that different latent factor models have maximized empirical measures of goodness-of-fit. The clinical relevance of these differences is unclear. We aimed to investigate whether depression screening accuracy may be improved by employing latent factor model-based scoring rather than sum scores.METHODS: We used an individual participant data meta-analysis (IPDMA) database compiled to assess the screening accuracy of the PHQ-9. We included studies that used the Structured Clinical Interview for DSM (SCID) as a reference standard and split those into calibration and validation datasets. In the calibration dataset, we estimated unidimensional, two-dimensional (separating cognitive/affective and somatic symptoms of depression), and bi-factor models, and the respective cut-offs to maximize combined sensitivity and specificity. In the validation dataset, we assessed the differences in (combined) sensitivity and specificity between the latent variable approaches and the optimal sum score (⩾10), using bootstrapping to estimate 95% confidence intervals for the differences.RESULTS: The calibration dataset included 24 studies (4378 participants, 652 major depression cases); the validation dataset 17 studies (4252 participants, 568 cases). In the validation dataset, optimal cut-offs of the unidimensional, two-dimensional, and bi-factor models had higher sensitivity (by 0.036, 0.050, 0.049 points, respectively) but lower specificity (0.017, 0.026, 0.019, respectively) compared to the sum score cut-off of ⩾10.CONCLUSIONS: In a comprehensive dataset of diagnostic studies, scoring using complex latent variable models do not improve screening accuracy of the PHQ-9 meaningfully as compared to the simple sum score approach.

  View details for DOI 10.1017/S0033291721000131

  View details for PubMedID 33612144

• COVID-19 antibody seroprevalence in Santa Clara County, California. International journal of epidemiology Bendavid, E., Mulaney, B., Sood, N., Shah, S., Bromley-Dulfano, R., Lai, C., Weissberg, Z., Saavedra-Walker, R., Tedrow, J., Bogan, A., Kupiec, T., Eichner, D., Gupta, R., Ioannidis, J. P., Bhattacharya, J. 2021

  Abstract

  BACKGROUND: Measuring the seroprevalence of antibodies to Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) is central to understanding infection risk and fatality rates. We studied Coronavirus Disease 2019 (COVID-19)-antibody seroprevalence in a community sample drawn from Santa Clara County.METHODS: On 3 and 4 April 2020, we tested 3328 county residents for immunoglobulin G (IgG) and immunoglobulin M (IgM) antibodies to SARS-CoV-2 using a rapid lateral-flow assay (Premier Biotech). Participants were recruited using advertisements that were targeted to reach county residents that matched the county population by gender, race/ethnicity and zip code of residence. We estimate weights to match our sample to the county by zip, age, sex and race/ethnicity. We report the weighted and unweighted prevalence of antibodies to SARS-CoV-2. We adjust for test-performance characteristics by combining data from 18 independent test-kit assessments: 14 for specificity and 4 for sensitivity.RESULTS: The raw prevalence of antibodies in our sample was 1.5% [exact binomial 95% confidence interval (CI) 1.1-2.0%]. Test-performance specificity in our data was 99.5% (95% CI 99.2-99.7%) and sensitivity was 82.8% (95% CI 76.0-88.4%). The unweighted prevalence adjusted for test-performance characteristics was 1.2% (95% CI 0.7-1.8%). After weighting for population demographics, the prevalence was 2.8% (95% CI 1.3-4.2%), using bootstrap to estimate confidence bounds. These prevalence point estimates imply that 53000 [95% CI 26000 to 82000 using weighted prevalence; 23000 (95% CI 14000-35000) using unweighted prevalence] people were infected in Santa Clara County by late March-many more than the 1200 confirmed cases at the time.CONCLUSION: The estimated prevalence of SARS-CoV-2 antibodies in Santa Clara County implies that COVID-19 was likely more widespread than indicated by the number of cases in late March, 2020. At the time, low-burden contexts such as Santa Clara County were far from herd-immunity thresholds.

  View details for DOI 10.1093/ije/dyab010

  View details for PubMedID 33615345

• Data Sharing Under the General Data Protection Regulation: Time to Harmonize Law and Research Ethics? Hypertension (Dallas, Tex. : 1979) Vlahou, A., Hallinan, D., Apweiler, R., Argiles, A., Beige, J., Benigni, A., Bischoff, R., Black, P. C., Boehm, F., Ceraline, J., Chrousos, G. P., Delles, C., Evenepoel, P., Fridolin, I., Glorieux, G., van Gool, A. J., Heidegger, I., Ioannidis, J. P., Jankowski, J., Jankowski, V., Jeronimo, C., Kamat, A. M., Masereeuw, R., Mayer, G., Mischak, H., Ortiz, A., Remuzzi, G., Rossing, P., Schanstra, J. P., Schmitz-Drager, B. J., Spasovski, G., Staessen, J. A., Stamatialis, D., Stenvinkel, P., Wanner, C., Williams, S. B., Zannad, F., Zoccali, C., Vanholder, R. 2021: HYPERTENSIONAHA12016340

  Abstract

  The General Data Protection Regulation (GDPR) became binding law in the European Union Member States in 2018, as a step toward harmonizing personal data protection legislation in the European Union. The Regulation governs almost all types of personal data processing, hence, also, those pertaining to biomedical research. The purpose of this article is to highlight the main practical issues related to data and biological sample sharing that biomedical researchers face regularly, and to specify how these are addressed in the context of GDPR, after consulting with ethics/legal experts. We identify areas in which clarifications of the GDPR are needed, particularly those related to consent requirements by study participants. Amendments should target the following: (1) restricting exceptions based on national laws and increasing harmonization, (2) confirming the concept of broad consent, and (3) defining a roadmap for secondary use of data. These changes will be achieved by acknowledged learned societies in the field taking the lead in preparing a document giving guidance for the optimal interpretation of the GDPR, which will be finalized following a period of commenting by a broad multistakeholder audience. In parallel, promoting engagement and education of the public in the relevant issues (such as different consent types or residual risk for re-identification), on both local/national and international levels, is considered critical for advancement. We hope that this article will open this broad discussion involving all major stakeholders, toward optimizing the GDPR and allowing a harmonized transnational research approach.

  View details for DOI 10.1161/HYPERTENSIONAHA.120.16340

  View details for PubMedID 33583200

• SARS-CoV-2 re-infection risk in Austria. European journal of clinical investigation Pilz, S., Chakeri, A., Ioannidis, J. P., Richter, L., Theiler-Schwetz, V., Trummer, C., Krause, R., Allerberger, F. 2021: e13520

  Abstract

  BACKGROUND: A key question concerning coronavirus disease 2019 (COVID-19) is how effective and long lasting immunity against this disease is in individuals who were previously infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). We aimed to evaluate the risk of SARS-CoV-2 re-infections in the general population in Austria.METHODS: This is a retrospective observational study using national SARS-CoV-2 infection data from the Austrian epidemiological reporting system. As the primary outcome, we aim to compare the odds of SARS-CoV-2 re-infections of COVID-19 survivors of the first wave (February to April 30, 2020) versus the odds of first infections in the remainder general population by tracking polymerase chain reaction (PCR)-confirmed infections of both groups during the second wave from September 1 to November 30, 2020. Re-infection counts are tentative, since it cannot be excluded that the positive PCR in the first and/or second wave might have been a false positive.RESULTS: We recorded 40 tentative re-infections in 14,840 COVID-19 survivors of the first wave (0.27%) and 253,581 infections in 8,885,640 individuals of the remaining general population (2.85%) translating into an odds ratio (95% confidence interval) of 0.09 (0.07 to 0.13).CONCLUSIONS: We observed a relatively low re-infection rate of SARS-CoV-2 in Austria. Protection against SARS-CoV-2 after natural infection is comparable to the highest available estimates on vaccine efficacies. Further well-designed research on this issue is urgently needed for improving evidence-based decisions on public health measures and vaccination strategies.

  View details for DOI 10.1111/eci.13520

  View details for PubMedID 33583018

• Second versus first wave of COVID-19 deaths: shifts in age distribution and in nursing home fatalities. Environmental research Ioannidis, J. P., Axfors, C., Contopoulos-Ioannidis, D. G. 2021: 110856

  Abstract

  OBJECTIVE: To examine whether the age distribution of COVID-19 deaths and the share of deaths in nursing homes changed in the second versus the first pandemic wave.ELIGIBLE DATA: We considered all countries that had at least 4000 COVID-19 deaths occurring as of January 14, 2020, at least 200 COVID-19 deaths occurring in each of the two epidemic wave periods; and which had sufficiently detailed information available on the age distribution of these deaths. We also considered countries with data available on COVID-19 deaths of nursing home residents for the two waves.MAIN OUTCOME MEASURES: Change in the second wave versus the first wave in the proportion of COVID-19 deaths occurring in people <50 years ("young deaths") among all COVID-19 deaths and among COVID-19 deaths in people <70 years old; and change in the proportion of COVID-19 deaths in nursing home residents among all COVID-19 deaths.RESULTS: Data on age distribution were available for 14 eligible countries. Individuals <50 years old had small absolute difference in their share of the total COVID-19 deaths in the two waves across 13 high-income countries (absolute differences 0.0-0.4%). Their proportion was higher in Ukraine, but it decreased markedly in the second wave. The odds of young deaths was lower in the second versus the first wave (summary prevalence ratio 0.81, 95% CI 0.71-0.92) with large between-country heterogeneity. The odds of young deaths among deaths <70 years did not differ significantly across the two waves (summary prevalence ratio 0.96, 95% CI 0.86-1.06). Eligible data on nursing home COVID-19 deaths were available for 11 countries. The share of COVID-19 deaths that were accounted by nursing home residents decreased in the second wave significantly and substantially in 8 countries (prevalence ratio estimates: 0.36 to 0.78), remained the same in Denmark and Norway and markedly increased in Australia.CONCLUSIONS: In the examined countries, age distribution of COVID-19 deaths has been fairly similar in the second versus the first wave, but the contribution of COVID-19 deaths in nursing home residents to total fatalities has decreased in most countries in the second wave.

  View details for DOI 10.1016/j.envres.2021.110856

  View details for PubMedID 33581086

• Recalibrating the Use of Race in Medical Research. JAMA Ioannidis, J. P., Powe, N. R., Yancy, C. 2021

  View details for DOI 10.1001/jama.2021.0003

  View details for PubMedID 33492329

• Academic criteria for promotion and tenure in faculties of medicine: a cross-sectional study of the Canadian U15 universities FACETS Rice, D. B., Raffoul, H., Ioannidis, J. A., Moher, D. 2021; 6: 58–70

  View details for DOI 10.1139/facets-2020-0044

  View details for Web of Science ID 000614060900001

• Assessing Mandatory Stay-at-Home and Business Closure Effects on the Spread of COVID-19. European journal of clinical investigation Bendavid, E., Oh, C., Bhattacharya, J., Ioannidis, J. P. 2021: e13484

  Abstract

  BACKGROUND AND AIMS: The most restrictive non-pharmaceutical interventions (NPIs) for controlling the spread of COVID-19 are mandatory stay-at-home and business closures. Given the consequences of these policies, it is important to assess their effects. We evaluate the effects on epidemic case growth of more restrictive NPIs (mrNPIs), above and beyond those of less restrictive NPIs (lrNPIs).METHODS: We first estimate COVID-19 case growth in relation to any NPI implementation in subnational regions of 10 countries: England, France, Germany, Iran, Italy, Netherlands, Spain, South Korea, Sweden, and the US. Using first-difference models with fixed effects, we isolate the effects of mrNPIs by subtracting the combined effects of lrNPIs and epidemic dynamics from all NPIs. We use case growth in Sweden and South Korea, two countries that did not implement mandatory stay-at-home and business closures, as comparison countries for the other 8 countries (16 total comparisons).RESULTS: Implementing any NPIs was associated with significant reductions in case growth in 9 out of 10 study countries, including South Korea and Sweden that implemented only lrNPIs (Spain had a non-significant effect). After subtracting the epidemic and lrNPI effects, we find no clear, significant beneficial effect of mrNPIs on case growth in any country. In France, e.g., the effect of mrNPIs was +7% (95CI -5%-19%) when compared with Sweden, and +13% (-12%-38%) when compared with South Korea (positive means pro-contagion). The 95% confidence intervals excluded 30% declines in all 16 comparisons and 15% declines in 11/16 comparisons.CONCLUSIONS: While small benefits cannot be excluded, we do not find significant benefits on case growth of more restrictive NPIs. Similar reductions in case growth may be achievable with less restrictive interventions.

  View details for DOI 10.1111/eci.13484

  View details for PubMedID 33400268

• Harms reported by patients in rheumatology drug trials: a systematic review of randomized trials in the cochrane library from an OMERACT working group. Seminars in arthritis and rheumatism Berthelsen, D. B., Woodworth, T. G., Goel, N. n., Ioannidis, J. P., Tugwell, P. n., Devoe, D. n., Williamson, P. n., Terwee, C. B., Suarez-Almazor, M. E., Strand, V. n., Leong, A. L., Conaghan, P. G., Boers, M. n., Shea, B. J., Brooks, P. M., Simon, L. S., Furst, D. E., Christensen, R. n. 2021

  Abstract

  Underreporting of harms in randomized controlled trials (RCTs) may lead to incomplete or erroneous assessments of the perceived benefit-to-harm profile of an intervention. To compare benefit with harm in clinical practice and future clinical studies, adverse event (AE) profiles including severity need to be understood. Even though patients report harm symptoms earlier and more frequently than clinicians, rheumatology RCTs currently do not provide a reporting framework from the patient's perspective regarding harms. Our objective for this meta-research project was to identify AEs in order to determine harm clusters and whether these could be self-reported by patients. Our other objective was to examine reported severity grading of the reported harms.We considered primary publications of RCTs eligible if they were published between 2008 and 2018 evaluating pharmacological interventions in patients with a rheumatic or musculoskeletal condition and if they were included in Cochrane reviews. We extracted data on harms such as reported AE terms together with severity (if described), and categorized AE- and severity-terms into overall groups. We deemed all AEs with felt components appropriate for patient self-reporting.The literature search identified 187 possible Cochrane reviews, of which 94 were eligible for evaluation, comprising 1,297 articles on individual RCTs. Of these RCTs, 93 pharmacological trials met our inclusion criteria (including 31,023 patients; representing 20,844 accumulated patient years), which reported a total of 21,498 AEs, corresponding to 693 unique reported terms for AEs. We further sub-categorized these terms into 280 harm clusters (i.e., themes). AEs appropriate for patient self-reporting accounted for 58% of the AEs reported. Among the reported AEs, we identified medical terms for all of the 117 harm clusters appropriate for patient reporting and lay language terms for 86%. We intended to include severity grades of the reported AEs, but there was no evidence for systematic reporting of clinician- or patient-reported severity in the primary articles of the 93 trials. However, we identified 33 terms suggesting severity, but severity grading was discernible in only 9%, precluding a breakdown by severity in this systematic review.Our results support the need for a standardized framework for patients' reporting of harms in rheumatology trials. Reporting of AEs with severity should be included in future reporting of harms, both from the patients' and investigators' perspectives.PROSPERO: CRD42018108393.

  View details for DOI 10.1016/j.semarthrit.2020.09.023

  View details for PubMedID 33483129

• Change in age distribution of COVID-19 deaths with the introduction of COVID-19 vaccination. Environmental research Pastorino, R., Pezzullo, A. M., Villani, L., Causio, F. A., Axfors, C., Contopoulos-Ioannidis, D. G., Boccia, S., Ioannidis, J. P. 2021: 112342

  Abstract

  Most countries initially deployed COVID-19 vaccines preferentially in elderly populations. We aimed to evaluate whether population-level vaccine effectiveness is heralded by an increase in the relative proportion of deaths among non-elderly populations that were less covered by vaccination programs.We collected data from 40 countries on age-stratified COVID-19 deaths during the vaccination period (1/14/2021-5/31/2021) and two control periods (entire pre-vaccination period and excluding the first wave).We meta-analyzed the proportion of deaths in different age groups in vaccination versus control periods in countries with low vaccination rates; (2) countries with age-independent vaccination policies; and (3) countries with standard age-dependent vaccination policies.Countries that prioritized vaccination among older people saw an increasing share of deaths among 0-69 year old people in the vaccination versus the two control periods (summary proportion ratio 1.32 [95 CI% 1.24-1.41] and 1.35 [95 CI% 1.26-1.44)]. No such change was seen on average in countries with age-independent vaccination policies (1.05 [95 CI% 0.78-1.41 and 0.97 [95 CI% 0.95-1.00], respectively) and limited vaccination (0.93 [95 CI% 0.85-1.01] and 0.95 [95 CI% 0.87-1.03], respectively). Proportion ratios were associated with the difference of vaccination rates in elderly versus non-elderly people. No significant changes occurred in the share of deaths in age 0-49 among all 0-69 deaths in the vaccination versus pre-vaccination periods.The substantial shift in the age distribution of COVID-19 deaths in countries that rapidly implemented vaccination predominantly among elderly provides evidence for the population level-effectiveness of COVID-19 vaccination and a favorable evolution of the pandemic towards endemicity with fewer elderly deaths.

  View details for DOI 10.1016/j.envres.2021.112342

  View details for PubMedID 34748775

• Impact of risk of generalizability biases in adult obesity interventions: A meta-epidemiological review and meta-analysis. Obesity reviews : an official journal of the International Association for the Study of Obesity Beets, M. W., von Klinggraeff, L., Burkart, S., Jones, A., Ioannidis, J. P., Weaver, R. G., Okely, A. D., Lubans, D., van Sluijs, E., Jago, R., Turner-McGrievy, G., Thrasher, J., Li, X. 2021: e13369

  Abstract

  Biases introduced in early-stage studies can lead to inflated early discoveries. The risk of generalizability biases (RGBs) identifies key features of feasibility studies that, when present, lead to reduced impact in a larger trial. This meta-study examined the influence of RGBs in adult obesity interventions. Behavioral interventions with a published feasibility study and a larger scale trial of the same intervention (e.g., pairs) were identified. Each pair was coded for the presence of RGBs. Quantitative outcomes were extracted. Multilevel meta-regression models were used to examine the impact of RGBs on the difference in the effect size (ES, standardized mean difference) from pilot to larger scale trial. A total of 114 pairs, representing 230 studies, were identified. Overall, 75% of the pairs had at least one RGB present. The four most prevalent RGBs were duration (33%), delivery agent (30%), implementation support (23%), and target audience (22%) bias. The largest reductions in the ES were observed in pairs where an RGB was present in the pilot and removed in the larger scale trial (average reduction ES -0.41, range -1.06 to 0.01), compared with pairs without an RGB (average reduction ES -0.15, range -0.18 to -0.14). Eliminating RGBs during early-stage testing may result in improved evidence.

  View details for DOI 10.1111/obr.13369

  View details for PubMedID 34779122

• External Validation of a Shortened Screening Tool Using Individual Participant Data Meta-Analysis: a Case Study of the Patient Questionnaire-Dep-4. Methods (San Diego, Calif.) Harel, D., Levis, B., Sun, Y., Fischer, F., Ioannidis, J. P., Cuijpers, P., Patten, S. B., Ziegelstein, R. C., Markham, S., Benedetti, A., Thombs, B. D. 2021

  Abstract

  Shortened versions of self-reported questionnaires may be used to reduce respondent burden. When shortened screening tools are used, it is desirable to maintain equivalent diagnostic accuracy to full-length forms. This manuscript presents a case study that illustrates how external data and individual participant data meta-analysis can be used to assess the equivalence in diagnostic accuracy between a shortened and full-length form. This case study compares the Patient Health Questionnaire-9 (PHQ-9) and a 4-item shortened version (PHQ-Dep-4) that was previously developed using optimal test assembly methods. Using a large database of 75 primary studies (34,698 participants, 3,392 major depression cases), we evaluated whether the PHQ-Dep-4 cutoff of ≥ 4 maintained equivalent diagnostic accuracy to a PHQ-9 cutoff of ≥ 10. Using this external validation dataset, a PHQ-Dep-4 cutoff of ≥ 4 maximized the sum of sensitivity and specificity, with a sensitivity of 0.88 (95% CI 0.81, 0.93), 0.68 (95% CI 0.56, 0.78), and 0.80 (95% CI 0.73, 0.85) for the semi-structured, fully structured, and MINI reference standard categories, respectively, and a specificity of 0.79 (95% CI 0.74, 0.83), 0.85 (95% CI 0.78, 0.90), and 0.83 (95% CI 0.80, 0.86) for the semi-structured, fully structured, and MINI reference standard categories, respectively. While equivalence with a PHQ-9 cutoff of ≥ 10 was not established, we found the sensitivity of the PHQ-Dep-4 to be non-inferior to that of the PHQ-9, and the specificity of the PHQ-Dep-4 to be marginally smaller than the PHQ-9.

  View details for DOI 10.1016/j.ymeth.2021.11.005

  View details for PubMedID 34780986

• Inverse correlates of COVID-19 mortality across European countries during the first versus subsequent waves. BMJ global health Villani, L., Pastorino, R., Ricciardi, W., Ioannidis, J., Boccia, S. 2021; 6 (8)

  Abstract

  The objectives of the study were to calculate the standardised mortality rates (SMRs) for COVID-19 in European Union/European Economic Area countries plus the UK and Switzerland and to evaluate the correlation between SMRs and selected indicators in the first versus the subsequent waves until 23 June 2021. We used indirect standardisation (using Italy as the reference) to compute SMRs and considered 16 indicators of health and social well-being, health system capacity and COVID-19 response. The highest SMRs were in Belgium, the UK and Spain in the first wave (1.20-1.84) and in Hungary, Czechia and Slovakia in the subsequent waves (2.50-2.69). Human Development Index (HDI), life expectancy, urbanisation and healthcare expenditure had positive correlations with SMR in the first wave (rho=0.30-0.46), but negative correlations (rho=-0.67 to -0.47) in the subsequent waves. Retail/recreation mobility and transit mobility were negatively correlated with SMR in the first wave, while transit mobility was inversely correlated with SMR in the subsequent waves. The first wave hit most hard countries with high HDI, high life expectancy, high urbanisation, high health expenditures and high tourism. This pattern may reflect higher early community seeding and circulation of the virus. Conversely, in the subsequent waves, this pattern was completely inversed: countries with more resources and better health status did better than eastern European countries. While major SMR differences existed across countries in the first wave, these differences largely dissipated by 23 June 2021, with few exceptions.

  View details for DOI 10.1136/bmjgh-2021-006422

  View details for PubMedID 34373260

• Evidence generation and reproducibility in cell and gene therapy research: A call to action. Molecular therapy. Methods & clinical development Abou-El-Enein, M., Angelis, A., Appelbaum, F. R., Andrews, N. C., Bates, S. E., Bierman, A. S., Brenner, M. K., Cavazzana, M., Caligiuri, M. A., Clevers, H., Cooke, E., Daley, G. Q., Dzau, V. J., Ellis, L. M., Fineberg, H. V., Goldstein, L. S., Gottschalk, S., Hamburg, M. A., Ingber, D. E., Kohn, D. B., Krainer, A. R., Maus, M. V., Marks, P., Mummery, C. L., Pettigrew, R. I., Rutter, J. L., Teichmann, S. A., Terzic, A., Urnov, F. D., Williams, D. A., Wolchok, J. D., Lawler, M., Turtle, C. J., Bauer, G., Ioannidis, J. P. 2021; 22: 11-14

  View details for DOI 10.1016/j.omtm.2021.06.012

  View details for PubMedID 34377737

  View details for PubMedCentralID PMC8322039

• Benefit of COVID-19 vaccination accounting for potential risk compensation. NPJ vaccines Ioannidis, J. P. 2021; 6 (1): 99

  Abstract

  People receiving COVID-19 vaccines may subsequently markedly increase their previously suppressed exposure risk. A simple model can evaluate the benefit of vaccination to the vaccinated (index) person and others exposed to that person; and calculate the amount of risk compensation required to eliminate all the benefits or to halve the benefit. As shown, 2.5-fold increase in exposure will eliminate the benefit of a vaccine of moderate efficacy (E = 0.6) unless the probability of infection in the population of interest is very high. With very high vaccine efficacy (E = 0.95), substantial benefit is maintained except in situations where there is a very low probability of infection in the population. If the vaccine efficacy decreases to 0.8, the benefit gets eroded easily with modest risk compensation. Risk compensation may markedly affect the benefit of COVID-19 vaccination, especially if vaccine efficacy in real-life or specific high-risk populations (e.g., nursing home residents) is not very high.

  View details for DOI 10.1038/s41541-021-00362-z

  View details for PubMedID 34381059

• Ninth international congress on peer review and scientific publication-call for abstracts. BMJ (Clinical research ed.) Ioannidis, J. P., Berkwits, M., Flanagin, A., Godlee, F., Bloom, T. 2021; 374: n2252

  View details for DOI 10.1136/bmj.n2252

  View details for PubMedID 34544728

• Mortality outcomes with hydroxychloroquine and chloroquine in COVID-19 from an international collaborative meta-analysis of randomized trials. Nature communications Axfors, C., Schmitt, A. M., Janiaud, P., Van't Hooft, J., Abd-Elsalam, S., Abdo, E. F., Abella, B. S., Akram, J., Amaravadi, R. K., Angus, D. C., Arabi, Y. M., Azhar, S., Baden, L. R., Baker, A. W., Belkhir, L., Benfield, T., Berrevoets, M. A., Chen, C., Chen, T., Cheng, S., Cheng, C., Chung, W., Cohen, Y. Z., Cowan, L. N., Dalgard, O., de Almeida E Val, F. F., de Lacerda, M. V., de Melo, G. C., Derde, L., Dubee, V., Elfakir, A., Gordon, A. C., Hernandez-Cardenas, C. M., Hills, T., Hoepelman, A. I., Huang, Y., Igau, B., Jin, R., Jurado-Camacho, F., Khan, K. S., Kremsner, P. G., Kreuels, B., Kuo, C., Le, T., Lin, Y., Lin, W., Lin, T., Lyngbakken, M. N., McArthur, C., McVerry, B. J., Meza-Meneses, P., Monteiro, W. M., Morpeth, S. C., Mourad, A., Mulligan, M. J., Murthy, S., Naggie, S., Narayanasamy, S., Nichol, A., Novack, L. A., O'Brien, S. M., Okeke, N. L., Perez, L., Perez-Padilla, R., Perrin, L., Remigio-Luna, A., Rivera-Martinez, N. E., Rockhold, F. W., Rodriguez-Llamazares, S., Rolfe, R., Rosa, R., Rosjo, H., Sampaio, V. S., Seto, T. B., Shehzad, M., Soliman, S., Stout, J. E., Thirion-Romero, I., Troxel, A. B., Tseng, T., Turner, N. A., Ulrich, R. J., Walsh, S. R., Webb, S. A., Weehuizen, J. M., Velinova, M., Wong, H., Wrenn, R., Zampieri, F. G., Zhong, W., Moher, D., Goodman, S. N., Ioannidis, J. P., Hemkens, L. G. 2021; 12 (1): 2349

  Abstract

  Substantial COVID-19 research investment has been allocated to randomized clinical trials (RCTs) on hydroxychloroquine/chloroquine, which currently face recruitment challenges or early discontinuation. We aim to estimate the effects of hydroxychloroquine and chloroquine on survival in COVID-19 from all currently available RCT evidence, published and unpublished. We present a rapid meta-analysis of ongoing, completed, or discontinued RCTs on hydroxychloroquine or chloroquine treatment for any COVID-19 patients (protocol: https://osf.io/QESV4/ ). We systematically identified unpublished RCTs (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform, Cochrane COVID-registry up to June 11, 2020), and published RCTs (PubMed, medRxiv and bioRxiv up to October 16, 2020). All-cause mortality has been extracted (publications/preprints) or requested from investigators and combined in random-effects meta-analyses, calculating odds ratios (ORs) with 95% confidence intervals (CIs), separately for hydroxychloroquine and chloroquine. Prespecified subgroup analyses include patient setting, diagnostic confirmation, control type, and publication status. Sixty-three trials were potentially eligible. We included 14 unpublished trials (1308 patients) and 14 publications/preprints (9011 patients). Results for hydroxychloroquine are dominated by RECOVERY and WHO SOLIDARITY, two highly pragmatic trials, which employed relatively high doses and included 4716 and 1853 patients, respectively (67% of the total sample size). The combined OR on all-cause mortality for hydroxychloroquine is1.11 (95% CI: 1.02, 1.20; I=0%; 26 trials; 10,012 patients) and for chloroquine 1.77 (95%CI: 0.15, 21.13, I=0%; 4 trials; 307 patients). We identified no subgroup effects. We found that treatment with hydroxychloroquine is associated with increased mortality in COVID-19 patients, and there is no benefit of chloroquine. Findings have unclear generalizability to outpatients, children, pregnant women, and people with comorbidities.

  View details for DOI 10.1038/s41467-021-22446-z

  View details for PubMedID 33859192

• Reporting only relative effect measures was potentially misleading: some good practices for improving the soundness of epidemiological results: Precision in reporting risk ratios. Journal of clinical epidemiology Novelli, M. n., Antognini, A. B., Boffetta, P. n., Ioannidis, J. P., Spatari, G. n., Violante, F. S. 2021

  Abstract

  In the medical and epidemiological literature there is a growing tendency to report an excessive number of decimal digits (often three, sometimes four), especially when measures of relative occurrence are small; this can be misleading.We combined mathematical and statistical reasoning about the precision of relative risks with the meaning of the decimal part of the same measures from biological and public health perspectives.We identified a general rule for minimizing the mathematical error due to rounding of relative risks, depending on the background absolute rate, which justifies the use of one or more decimal digits for estimates close to 1.We suggest that both relative and absolute risk measures (expressed as a rates) should be reported, and two decimal digits should be used for relative risk close to 1 only if the background rate is at least 1/1,000 py. The use of more than two decimal digits is justified only when the background rate is high (i.e., 1/10 py).

  View details for DOI 10.1016/j.jclinepi.2021.04.006

  View details for PubMedID 33894329

• Media and social media attention to retracted articles according to Altmetric. PloS one Serghiou, S. n., Marton, R. M., Ioannidis, J. P. 2021; 16 (5): e0248625

  Abstract

  The number of retracted articles has grown fast. However, the extent to which researchers and the public are made adequately aware of these retractions and how the media and social media respond to them remains unknown. Here, we aimed to evaluate the media and social media attention received by retracted articles and assess also the attention they receive post-retraction versus pre-retraction. We downloaded all records of retracted literature maintained by the Retraction Watch Database and originally published between January 1, 2010 to December 31, 2015. For all 3,008 retracted articles with a separate DOI for the original and its retraction, we downloaded the respective Altmetric Attention Score (AAS) (from Altmetric) and citation count (from Crossref), for the original article and its retraction notice on June 6, 2018. We also compared the AAS of a random sample of 572 retracted full journal articles available on PubMed to that of unretracted full articles matched from the same issue and journal. 1,687 (56.1%) of retracted research articles received some amount of Altmetric attention, and 165 (5.5%) were even considered popular (AAS>20). 31 (1.0%) of 2,953 with a record on Crossref received >100 citations by June 6, 2018. Popular articles received substantially more attention than their retraction, even after adjusting for attention received post-retraction (Median difference, 29; 95% CI, 17-61). Unreliable results were the most frequent reason for retraction of popular articles (32; 19%), while fake peer review was the most common reason (421; 15%) for the retraction of other articles. In comparison to matched articles, retracted articles tended to receive more Altmetric attention (23/31 matched groups; P-value, 0.01), even after adjusting for attention received post-retraction. Our findings reveal that retracted articles may receive high attention from media and social media and that for popular articles, pre-retraction attention far outweighs post-retraction attention.

  View details for DOI 10.1371/journal.pone.0248625

  View details for PubMedID 33979339

• Evaluating and Strengthening the Evidence for Nutritional Bone Research: Ready to Break New Ground? Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research Lewis, J. R., Voortman, T. n., Ioannidis, J. P. 2021

  Abstract

  A healthy diet is essential to attain genetically determined peak bone mass and maintain optimal skeletal health across the adult lifespan. Despite the importance of nutrition for bone health, many of the nutritional requirements of the skeleton across the lifespan remain underexplored, poorly understood, or controversial. With increasingly aging populations, combined with rapidly changing diets and lifestyles globally, one anticipates large increases in the prevalence of osteoporosis and incidence of osteoporotic fractures. Robust, transparent, and reproducible nutrition research is a cornerstone for developing reliable public health recommendations to prevent osteoporosis and osteoporotic fractures. However, nutrition research is often criticized or ignored by healthcare professionals due to the overemphasis of weak science, conflicting, confusing or implausible findings, industry interests, common misconceptions, and strong opinions. Conversely, spurious research findings are often overemphasized or misconstrued by the media or prominent figures especially via social media, potentially leading to confusion and a lack of trust by the general public. Recently, reforms of the broader discipline of nutrition science have been suggested and promoted, leading to new tools and recommendations to attempt to address these issues. In this perspective, we provide a brief overview of what has been achieved in the field on nutrition and bone health, focusing on osteoporosis and osteoporotic fractures. We discuss what we view as some of the challenges, including inherent difficulties in assessing diet and its change, disentangling complex interactions between dietary components and between diet and other factors, selection of bone-related outcomes for nutrition studies, obtaining evidence with more unbiased designs, and perhaps most importantly, ensuring the trust of the public and healthcare professionals. This perspective also provides specific recommendations and highlights new developments and future opportunities for scientists studying nutrition and bone health. © 2021 American Society for Bone and Mineral Research (ASBMR).

  View details for DOI 10.1002/jbmr.4236

  View details for PubMedID 33503301

• Evaluation of Data Sharing After Implementation of the International Committee of Medical Journal Editors Data Sharing Statement Requirement. JAMA network open Danchev, V. n., Min, Y. n., Borghi, J. n., Baiocchi, M. n., Ioannidis, J. P. 2021; 4 (1): e2033972

  Abstract

  The benefits of responsible sharing of individual-participant data (IPD) from clinical studies are well recognized, but stakeholders often disagree on how to align those benefits with privacy risks, costs, and incentives for clinical trialists and sponsors. The International Committee of Medical Journal Editors (ICMJE) required a data sharing statement (DSS) from submissions reporting clinical trials effective July 1, 2018. The required DSSs provide a window into current data sharing rates, practices, and norms among trialists and sponsors.To evaluate the implementation of the ICMJE DSS requirement in 3 leading medical journals: JAMA, Lancet, and New England Journal of Medicine (NEJM).This is a cross-sectional study of clinical trial reports published as articles in JAMA, Lancet, and NEJM between July 1, 2018, and April 4, 2020. Articles not eligible for DSS, including observational studies and letters or correspondence, were excluded. A MEDLINE/PubMed search identified 487 eligible clinical trials in JAMA (112 trials), Lancet (147 trials), and NEJM (228 trials). Two reviewers evaluated each of the 487 articles independently.Publication of clinical trial reports in an ICMJE medical journal requiring a DSS.The primary outcomes of the study were declared data availability and actual data availability in repositories. Other captured outcomes were data type, access, and conditions and reasons for data availability or unavailability. Associations with funding sources were examined.A total of 334 of 487 articles (68.6%; 95% CI, 64%-73%) declared data sharing, with nonindustry NIH-funded trials exhibiting the highest rates of declared data sharing (89%; 95% CI, 80%-98%) and industry-funded trials the lowest (61%; 95% CI, 54%-68%). However, only 2 IPD sets (0.6%; 95% CI, 0.0%-1.5%) were actually deidentified and publicly available as of April 10, 2020. The remaining were supposedly accessible via request to authors (143 of 334 articles [42.8%]), repository (89 of 334 articles [26.6%]), and company (78 of 334 articles [23.4%]). Among the 89 articles declaring that IPD would be stored in repositories, only 17 (19.1%) deposited data, mostly because of embargo and regulatory approval. Embargo was set in 47.3% of data-sharing articles (158 of 334), and in half of them the period exceeded 1 year or was unspecified.Most trials published in JAMA, Lancet, and NEJM after the implementation of the ICMJE policy declared their intent to make clinical data available. However, a wide gap between declared and actual data sharing exists. To improve transparency and data reuse, journals should promote the use of unique pointers to data set location and standardized choices for embargo periods and access requirements.

  View details for DOI 10.1001/jamanetworkopen.2020.33972

  View details for PubMedID 33507256

• Precision shielding for COVID-19: metrics of assessment and feasibility of deployment. BMJ global health Ioannidis, J. P. 2021; 6 (1)

  Abstract

  The ability to preferentially protect high-risk groups in COVID-19 is hotly debated. Here, the aim is to present simple metrics of such precision shielding of people at high risk of death after infection by SARS-CoV-2; demonstrate how they can estimated; and examine whether precision shielding was successfully achieved in the first COVID-19 wave. The shielding ratio, S, is defined as the ratio of prevalence of infection among people in a high-risk group versus among people in a low-risk group. The contrasted risk groups examined here are according to age (≥70 vs <70 years), and institutionalised (nursing home) setting. For age-related precision shielding, data were used from large seroprevalence studies with separate prevalence data for elderly versus non-elderly and with at least 1000 assessed people≥70 years old. For setting-related precision shielding, data were analysed from 10 countries where information was available on numbers of nursing home residents, proportion of nursing home residents among COVID-19 deaths and overall population infection fatality rate (IFR). Across 17 seroprevalence studies, the shielding ratio S for elderly versus non-elderly varied between 0.4 (substantial shielding) and 1.6 (substantial inverse protection, that is, low-risk people being protected more than high-risk people). Five studies in the USA all yielded S=0.4-0.8, consistent with some shielding being achieved, while two studies in China yielded S=1.5-1.6, consistent with inverse protection. Assuming 25% IFR among nursing home residents, S values for nursing home residents ranged from 0.07 to 3.1. The best shielding was seen in South Korea (S=0.07) and modest shielding was achieved in Israel, Slovenia, Germany and Denmark. No shielding was achieved in Hungary and Sweden. In Belgium (S=1.9), the UK (S=2.2) and Spain (S=3.1), nursing home residents were far more frequently infected than the rest of the population. In conclusion, the experience from the first wave of COVID-19 suggests that different locations and settings varied markedly in the extent to which they protected high-risk groups. Both effective precision shielding and detrimental inverse protection can happen in real-life circumstances. COVID-19 interventions should seek to achieve maximal precision shielding.

  View details for DOI 10.1136/bmjgh-2020-004614

  View details for PubMedID 33514595

• Nutrition and Health: Setting Realistic Expectations and Changing Research Targets. Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research Ioannidis, J. P. 2021

  View details for DOI 10.1002/jbmr.4237

  View details for PubMedID 33434294

• Recruitment and Results Reporting of COVID-19 Randomized Clinical Trials Registered in the First 100 Days of the Pandemic. JAMA network open Janiaud, P. n., Axfors, C. n., Ioannidis, J. P., Hemkens, L. G. 2021; 4 (3): e210330

  View details for DOI 10.1001/jamanetworkopen.2021.0330

  View details for PubMedID 33646310

• Association of 152 Biomarker Reference Intervals with All-Cause Mortality in Participants of a General United States Survey from 1999 to 2010. Clinical chemistry Pho, N. n., Manrai, A. K., Leppert, J. T., Chertow, G. M., Ioannidis, J. P., Patel, C. J. 2021; 67 (3): 500–507

  Abstract

  Physicians sometimes consider whether or not to perform diagnostic testing in healthy people, but it is unknown whether nonextreme values of diagnostic tests typically encountered in such populations have any predictive ability, in particular for risk of death. The goal of this study was to quantify the associations among population reference intervals of 152 common biomarkers with all-cause mortality in a representative, nondiseased sample of adults in the United States.The study used an observational cohort derived from the National Health and Nutrition Examination Survey (NHANES), a representative sample of the United States population consisting of 6 survey waves from 1999 to 2010 with linked mortality data (unweighted N = 30 651) and a median followup of 6.1 years. We deployed an X-wide association study (XWAS) approach to systematically perform association testing of 152 diagnostic tests with all-cause mortality.After controlling for multiple hypotheses, we found that the values within reference intervals (10-90th percentiles) of 20 common biomarkers used as diagnostic tests or clinical measures were associated with all-cause mortality, including serum albumin, red cell distribution width, serum alkaline phosphatase, and others after adjusting for age (linear and quadratic terms), sex, race, income, chronic illness, and prior-year healthcare utilization. All biomarkers combined, however, explained only an additional 0.8% of the variance of mortality risk. We found modest year-to-year changes, or changes in association from survey wave to survey wave from 1999 to 2010 in the association sizes of biomarkers.Reference and nonoutlying variation in common biomarkers are consistently associated with mortality risk in the US population, but their additive contribution in explaining mortality risk is minor.

  View details for DOI 10.1093/clinchem/hvaa271

  View details for PubMedID 33674838

• Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study. BMJ (Clinical research ed.) Mc Cord, K. A., Ewald, H. n., Agarwal, A. n., Glinz, D. n., Aghlmandi, S. n., Ioannidis, J. P., Hemkens, L. G. 2021; 372: n450

  Abstract

  To compare effect estimates of randomised clinical trials that use routinely collected data (RCD-RCT) for outcome ascertainment with traditional trials not using routinely collected data.Meta-research study.Studies included in the same meta-analysis in a Cochrane review.Randomised clinical trials using any type of routinely collected data for outcome ascertainment, including from registries, electronic health records, and administrative databases, that were included in a meta-analysis of a Cochrane review on any clinical question and any health outcome together with traditional trials not using routinely collected data for outcome measurement.Effect estimates from trials using or not using routinely collected data were summarised in random effects meta-analyses. Agreement of (summary) treatment effect estimates from trials using routinely collected data and those not using such data was expressed as the ratio of odds ratios. Subgroup analyses explored effects in trials based on different types of routinely collected data. Two investigators independently assessed the quality of each data source.84 RCD-RCTs and 463 traditional trials on 22 clinical questions were included. Trials using routinely collected data for outcome ascertainment showed 20% less favourable treatment effect estimates than traditional trials (ratio of odds ratios 0.80, 95% confidence interval 0.70 to 0.91, I2=14%). Results were similar across various types of outcomes (mortality outcomes: 0.92, 0.74 to 1.15, I2=12%; non-mortality outcomes: 0.71, 0.60 to 0.84, I2=8%), data sources (electronic health records: 0.81, 0.59 to 1.11, I2=28%; registries: 0.86, 0.75 to 0.99, I2=20%; administrative data: 0.84, 0.72 to 0.99, I2=0%), and data quality (high data quality: 0.82, 0.72 to 0.93, I2=0%).Randomised clinical trials using routinely collected data for outcome ascertainment show smaller treatment benefits than traditional trials not using routinely collected data. These differences could have implications for healthcare decision making and the application of real world evidence.

  View details for DOI 10.1136/bmj.n450

  View details for PubMedID 33658187

• Infection fatality rate of COVID-19 inferred from seroprevalence data. Bulletin of the World Health Organization Ioannidis, J. P. 2021; 99 (1): 19–33F

  Abstract

  Objective: To estimate the infection fatality rate of coronavirus disease 2019 (COVID-19) from seroprevalence data.Methods: I searched PubMed and preprint servers for COVID-19 seroprevalence studies with a sample size ≥500 as of 9 September 2020. I also retrieved additional results of national studies from preliminary press releases and reports. I assessed the studies for design features and seroprevalence estimates. I estimated the infection fatality rate for each study by dividing the cumulative number of COVID-19 deaths by the number of people estimated to be infected in each region. I corrected for the number of immunoglobin (Ig) types tested (IgG, IgM, IgA).Findings: I included 61 studies (74 estimates) and eight preliminary national estimates. Seroprevalence estimates ranged from 0.02% to 53.40%. Infection fatality rates ranged from 0.00% to 1.63%, corrected values from 0.00% to 1.54%. Across 51 locations, the median COVID-19 infection fatality rate was 0.27% (corrected 0.23%): the rate was 0.09% in locations with COVID-19 population mortality rates less than the global average (<118 deaths/million), 0.20% in locations with 118-500 COVID-19 deaths/million people and 0.57% in locations with >500 COVID-19 deaths/million people. In people younger than70 years, infection fatality rates ranged from 0.00% to 0.31% with crude and corrected medians of 0.05%.Conclusion: The infection fatality rate of COVID-19 can vary substantially across different locations and this may reflect differences in population age structure and case-mix of infected and deceased patients and other factors. The inferred infection fatality rates tended to be much lower than estimates made earlier in the pandemic.

  View details for DOI 10.2471/BLT.20.265892

  View details for PubMedID 33716331

• Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014-2017). Perspectives on psychological science : a journal of the Association for Psychological Science Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., Ioannidis, J. P. 2021: 1745691620979806

  Abstract

  Psychologists are navigating an unprecedented period of introspection about the credibility and utility of their discipline. Reform initiatives emphasize the benefits of transparency and reproducibility-related research practices; however, adoption across the psychology literature is unknown. Estimating the prevalence of such practices will help to gauge the collective impact of reform initiatives, track progress over time, and calibrate future efforts. To this end, we manually examined a random sample of 250 psychology articles published between 2014 and 2017. Over half of the articles were publicly available (154/237, 65%, 95% confidence interval [CI] = [59%, 71%]); however, sharing of research materials (26/183; 14%, 95% CI = [10%, 19%]), study protocols (0/188; 0%, 95% CI = [0%, 1%]), raw data (4/188; 2%, 95% CI = [1%, 4%]), and analysis scripts (1/188; 1%, 95% CI = [0%, 1%]) was rare. Preregistration was also uncommon (5/188; 3%, 95% CI = [1%, 5%]). Many articles included a funding disclosure statement (142/228; 62%, 95% CI = [56%, 69%]), but conflict-of-interest statements were less common (88/228; 39%, 95% CI = [32%, 45%]). Replication studies were rare (10/188; 5%, 95% CI = [3%, 8%]), and few studies were included in systematic reviews (21/183; 11%, 95% CI = [8%, 16%]) or meta-analyses (12/183; 7%, 95% CI = [4%, 10%]). Overall, the results suggest that transparency and reproducibility-related research practices were far from routine. These findings establish baseline prevalence estimates against which future progress toward increasing the credibility and utility of psychology research can be compared.

  View details for DOI 10.1177/1745691620979806

  View details for PubMedID 33682488

• Systematic examination of preprint platforms for use in the medical and biomedical sciences setting. BMJ open Kirkham, J. J., Penfold, N. C., Murphy, F., Boutron, I., Ioannidis, J. P., Polka, J., Moher, D. 2020; 10 (12): e041849

  Abstract

  OBJECTIVES: The objective of this review is to identify all preprint platforms with biomedical and medical scope and to compare and contrast the key characteristics and policies of these platforms.STUDY DESIGN AND SETTING: Preprint platforms that were launched up to 25 June 2019 and have a biomedical and medical scope according to MEDLINE's journal selection criteria were identified using existing lists, web-based searches and the expertise of both academic and non-academic publication scientists. A data extraction form was developed, pilot tested and used to collect data from each preprint platform's webpage(s).RESULTS: A total of 44 preprint platforms were identified as having biomedical and medical scope, 17 (39%) were hosted by the Open Science Framework preprint infrastructure, 6 (14%) were provided by F1000 Research (the Open Research Central infrastructure) and 21 (48%) were other independent preprint platforms. Preprint platforms were either owned by non-profit academic groups, scientific societies or funding organisations (n=28; 64%), owned/partly owned by for-profit publishers or companies (n=14; 32%) or owned by individuals/small communities (n=2; 5%). Twenty-four (55%) preprint platforms accepted content from all scientific fields although some of these had restrictions relating to funding source, geographical region or an affiliated journal's remit. Thirty-three (75%) preprint platforms provided details about article screening (basic checks) and 14 (32%) of these actively involved researchers with context expertise in the screening process. Almost all preprint platforms allow submission to any peer-reviewed journal following publication, have a preservation plan for read access and most have a policy regarding reasons for retraction and the sustainability of the service.CONCLUSION: A large number of preprint platforms exist for use in biomedical and medical sciences, all of which offer researchers an opportunity to rapidly disseminate their research findings onto an open-access public server, subject to scope and eligibility.

  View details for DOI 10.1136/bmjopen-2020-041849

  View details for PubMedID 33376175

• Visualizing the invisible: The effect of asymptomatic transmission on the outbreak dynamics of COVID-19 COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING Peirlinck, M., Linka, K., Costabal, F., Bhattacharya, J., Bendavid, E., Ioannidis, J. A., Kuhl, E. 2020; 372

  View details for DOI 10.1016/j.cma.2020.113410

  View details for Web of Science ID 000592535100007

• Visualizing the invisible: The effect of asymptomatic transmission on the outbreak dynamics of COVID-19. Computer methods in applied mechanics and engineering Peirlinck, M., Linka, K., Sahli Costabal, F., Bhattacharya, J., Bendavid, E., Ioannidis, J. P., Kuhl, E. 2020; 372: 113410

  Abstract

  Understanding the outbreak dynamics of the COVID-19 pandemic has important implications for successful containment and mitigation strategies. Recent studies suggest that the population prevalence of SARS-CoV-2 antibodies, a proxy for the number of asymptomatic cases, could be an order of magnitude larger than expected from the number of reported symptomatic cases. Knowing the precise prevalence and contagiousness of asymptomatic transmission is critical to estimate the overall dimension and pandemic potential of COVID-19. However, at this stage, the effect of the asymptomatic population, its size, and its outbreak dynamics remain largely unknown. Here we use reported symptomatic case data in conjunction with antibody seroprevalence studies, a mathematical epidemiology model, and a Bayesian framework to infer the epidemiological characteristics of COVID-19. Our model computes, in real time, the time-varying contact rate of the outbreak, and projects the temporal evolution and credible intervals of the effective reproduction number and the symptomatic, asymptomatic, and recovered populations. Our study quantifies the sensitivity of the outbreak dynamics of COVID-19 to three parameters: the effective reproduction number, the ratio between the symptomatic and asymptomatic populations, and the infectious periods of both groups. For nine distinct locations, our model estimates the fraction of the population that has been infected and recovered by Jun 15, 2020 to 24.15% (95% CI: 20.48%-28.14%) for Heinsberg (NRW, Germany), 2.40% (95% CI: 2.09%-2.76%) for Ada County (ID, USA), 46.19% (95% CI: 45.81%-46.60%) for New York City (NY, USA), 11.26% (95% CI: 7.21%-16.03%) for Santa Clara County (CA, USA), 3.09% (95% CI: 2.27%-4.03%) for Denmark, 12.35% (95% CI: 10.03%-15.18%) for Geneva Canton (Switzerland), 5.24% (95% CI: 4.84%-5.70%) for the Netherlands, 1.53% (95% CI: 0.76%-2.62%) for Rio Grande do Sul (Brazil), and 5.32% (95% CI: 4.77%-5.93%) for Belgium. Our method traces the initial outbreak date in Santa Clara County back to January 20, 2020 (95% CI: December 29, 2019-February 13, 2020). Our results could significantly change our understanding and management of the COVID-19 pandemic: A large asymptomatic population will make isolation, containment, and tracing of individual cases challenging. Instead, managing community transmission through increasing population awareness, promoting physical distancing, and encouraging behavioral changes could become more relevant.

  View details for DOI 10.1016/j.cma.2020.113410

  View details for PubMedID 33518823

  View details for PubMedCentralID PMC7831913

• Testing Clinical Prediction Models-Reply. JAMA Adibi, A., Sadatsafavi, M., Ioannidis, J. P. 2020; 324 (19): 2000

  View details for DOI 10.1001/jama.2020.19413

  View details for PubMedID 33201201

• Clinical Trial Evidence Supporting US Food and Drug Administration Approval of Novel Cancer Therapies Between 2000 and 2016. JAMA network open Ladanie, A., Schmitt, A. M., Speich, B., Naudet, F., Agarwal, A., Pereira, T. V., Sclafani, F., Herbrand, A. K., Briel, M., Martin-Liberal, J., Schmid, T., Ewald, H., Ioannidis, J. P., Bucher, H. C., Kasenda, B., Hemkens, L. G. 2020; 3 (11): e2024406

  Abstract

  Importance: Clinical trial evidence used to support drug approval is typically the only information on benefits and harms that patients and clinicians can use for decision-making when novel cancer therapies become available. Various evaluations have raised concern about the uncertainty surrounding these data, and a systematic investigation of the available information on treatment outcomes for cancer drugs approved by the US Food and Drug Administration (FDA) is warranted.Objective: To describe the clinical trial data available on treatment outcomes at the time of FDA approval of all novel cancer drugs approved for the first time between 2000 and 2016.Design, Setting, and Participants: This comparative effectiveness study analyzed randomized clinical trials and single-arm clinical trials of novel drugs approved for the first time to treat any type of cancer. Approval packages were obtained from drugs@FDA, a publicly available database containing information on drug and biologic products approved for human use in the US. Data from January 2000 to December 2016 were included in this study.Main Outcomes and Measures: Regulatory and clinical trial characteristics were described. For randomized clinical trials, summary treatment outcomes for overall survival, progression-free survival, and tumor response across all therapies were calculated, and median absolute survival increases were estimated. Tumor types and regulatory characteristics were assessed separately.Results: Between 2000 and 2016, 92 novel cancer drugs were approved by the FDA for 100 indications based on data from 127 clinical trials. The 127 clinical trials included a median of 191 participants (interquartile range [IQR], 106-448 participants). Overall, 65 clinical trials (51.2%) were randomized, and 95 clinical trials (74.8%) were open label. Of 100 indications, 44 indications underwent accelerated approval, 42 indications were for hematological cancers, and 58 indications were for solid tumors. Novel drugs had mean hazard ratios of 0.77 (95% CI, 0.73-0.81; I2=46%) for overall survival and 0.52 (95% CI, 0.47-0.57; I2=88%) for progression-free survival. The median tumor response, expressed as relative risk, was 2.37 (95% CI, 2.00-2.80; I2=91%). The median absolute survival benefit was 2.40 months (IQR, 1.25-3.89 months).Conclusions and Relevance: In this study, data available at the time of FDA drug approval indicated that novel cancer therapies were associated with substantial tumor responses but with prolonging median overall survival by only 2.40 months. Approval data from 17 years of clinical trials suggested that patients and clinicians typically had limited information available regarding the benefits of novel cancer treatments at market entry.

  View details for DOI 10.1001/jamanetworkopen.2020.24406

  View details for PubMedID 33170262

• Redundant meta-analyses are common in genetic epidemiology JOURNAL OF CLINICAL EPIDEMIOLOGY Sigurdson, M. K., Khoury, M. J., Ioannidis, J. A. 2020; 127: 40–48

  Abstract

  The massive growth in the publication of meta-analyses may cause redundancy and wasted efforts. We performed a metaepidemiologic study to evaluate the extent of potential redundancy in published meta-analyses in genetic epidemiology.Using a sample of 38 index meta-analyses of genetic associations published in 2010, we retrieved additional meta-analyses that evaluated identical associations (same genetic variant and phenotype) using the Human Genome Epidemiology (HuGE) Navigator and PubMed databases. We analyzed the frequency of potential duplication and examined whether subsequent meta-analyses cited previous meta-analyses on the exact same association.Based on 38 index meta-analyses, we retrieved a total of 99 duplicate meta-analyses. Only 12 (32%) of the index meta-analyses were unambiguously unique. We found a mean of 2.6 duplicates and a median of 2 duplicates per meta-analysis. In case studies, only 29-54% of previously published meta-analyses were cited by subsequent ones.These results suggest that duplication is common in meta-analyses of genetic associations.

  View details for DOI 10.1016/j.jclinepi.2020.05.035

  View details for Web of Science ID 000589799000009

• Hundreds of thousands of zombie randomised trials circulate among us. Anaesthesia Ioannidis, J. P. 2020

  View details for DOI 10.1111/anae.15297

  View details for PubMedID 33124075

• Overestimation of Postpartum Depression Prevalence Based on a 5-item Version of the EPDS: Systematic Review and Individual Participant Data Meta-analysis. Canadian journal of psychiatry. Revue canadienne de psychiatrie Thombs, B. D., Levis, B., Lyubenova, A., Neupane, D., Negeri, Z., Wu, Y., Sun, Y., He, C., Krishnan, A., Vigod, S. N., Bhandari, P. M., Imran, M., Rice, D. B., Azar, M., Chiovitti, M. J., Saadat, N., Riehm, K. E., Boruff, J. T., Cuijpers, P., Gilbody, S., Ioannidis, J. P., Kloda, L. A., Patten, S. B., Shrier, I., Ziegelstein, R. C., Comeau, L., Mitchell, N. D., Tonelli, M., Barnes, J., Beck, C. T., Bindt, C., Figueiredo, B., Helle, N., Howard, L. M., Kohlhoff, J., Kozinszky, Z., Leonardou, A. A., Rados, S. N., Quispel, C., Rochat, T. J., Stein, A., Stewart, R. C., Tadinac, M., Tandon, S. D., Tendais, I., Toreki, A., Tran, T. D., Trevillion, K., Turner, K., Vega-Dienstmaier, J. M., Benedetti, A. 2020: 706743720934959

  Abstract

  OBJECTIVE: The Maternal Mental Health in Canada, 2018/2019, survey reported that 18% of 7,085 mothers who recently gave birth reported "feelings consistent with postpartum depression" based on scores ≥7 on a 5-item version of the Edinburgh Postpartum Depression Scale (EPDS-5). The EPDS-5 was designed as a screening questionnaire, not to classify disorders or estimate prevalence; the extent to which EPDS-5 results reflect depression prevalence is unknown. We investigated EPDS-5 ≥7 performance relative to major depression prevalence based on a validated diagnostic interview, the Structured Clinical Interview for DSM (SCID).METHODS: We searched Medline, Medline In-Process & Other Non-Indexed Citations, PsycINFO, and the Web of Science Core Collection through June 2016 for studies with data sets with item response data to calculate EPDS-5 scores and that used the SCID to ascertain depression status. We conducted an individual participant data meta-analysis to estimate pooled percentage of EPDS-5 ≥7, pooled SCID major depression prevalence, and the pooled difference in prevalence.RESULTS: A total of 3,958 participants from 19 primary studies were included. Pooled prevalence of SCID major depression was 9.2% (95% confidence interval [CI] 6.0% to 13.7%), pooled percentage of participants with EPDS-5 ≥7 was 16.2% (95% CI 10.7% to 23.8%), and pooled difference was 8.0% (95% CI 2.9% to 13.2%). In the 19 included studies, mean and median ratios of EPDS-5 to SCID prevalence were 2.1 and 1.4 times.CONCLUSIONS: Prevalence estimated based on EPDS-5 ≥7 appears to be substantially higher than the prevalence of major depression. Validated diagnostic interviews should be used to establish prevalence.

  View details for DOI 10.1177/0706743720934959

  View details for PubMedID 33104415

• Depression prevalence based on the Edinburgh Postnatal Depression Scale compared to Structured Clinical Interview for DSM DIsorders classification: Systematic review and individual participant data meta-analysis. International journal of methods in psychiatric research Lyubenova, A., Neupane, D., Levis, B., Wu, Y., Sun, Y., He, C., Krishnan, A., Bhandari, P. M., Negeri, Z., Imran, M., Rice, D. B., Azar, M., Chiovitti, M. J., Saadat, N., Riehm, K. E., Boruff, J. T., Ioannidis, J. P., Cuijpers, P., Gilbody, S., Kloda, L. A., Patten, S. B., Shrier, I., Ziegelstein, R. C., Comeau, L., Mitchell, N. D., Tonelli, M., Vigod, S. N., Aceti, F., Barnes, J., Bavle, A. D., Beck, C. T., Bindt, C., Boyce, P. M., Bunevicius, A., Chaudron, L. H., Favez, N., Figueiredo, B., Garcia-Esteve, L., Giardinelli, L., Helle, N., Howard, L. M., Kohlhoff, J., Kusminskas, L., Kozinszky, Z., Lelli, L., Leonardou, A. A., Meuti, V., Rados, S. N., Garcia, P. N., Pawlby, S. J., Quispel, C., Robertson-Blackmore, E., Rochat, T. J., Sharp, D. J., Siu, B. W., Stein, A., Stewart, R. C., Tadinac, M., Tandon, S. D., Tendais, I., Toreki, A., Torres-Gimenez, A., Tran, T. D., Trevillion, K., Turner, K., Vega-Dienstmaier, J. M., Benedetti, A., Thombs, B. D. 2020: e1860

  Abstract

  OBJECTIVES: Estimates of depression prevalence in pregnancy and postpartum are based on the Edinburgh Postnatal Depression Scale (EPDS) more than on any other method. We aimed to determine if any EPDS cutoff can accurately and consistently estimate depression prevalence in individual studies.METHODS: We analyzed datasets that compared EPDS scores to Structured Clinical Interview for DSM (SCID) major depression status. Random-effects meta-analysis was used to compare prevalence with EPDS cutoffs versus the SCID.RESULTS: Seven thousand three hundred and fifteen participants (1017 SCID major depression) from 29 primary studies were included. For EPDS cutoffs used to estimate prevalence in recent studies (≥9 to ≥14), pooled prevalence estimates ranged from 27.8% (95% CI: 22.0%-34.5%) for EPDS ≥ 9 to 9.0% (95% CI: 6.8%-11.9%) for EPDS ≥ 14; pooled SCID major depression prevalence was 9.0% (95% CI: 6.5%-12.3%). EPDS ≥14 provided pooled prevalence closest to SCID-based prevalence but differed from SCID prevalence in individual studies by a mean absolute difference of 5.1% (95% prediction interval: -13.7%, 12.3%).CONCLUSION: EPDS ≥14 approximated SCID-based prevalence overall, but considerable heterogeneity in individual studies is a barrier to using it for prevalence estimation.

  View details for DOI 10.1002/mpr.1860

  View details for PubMedID 33089942

• Dental Research Waste in Design, Analysis, and Reporting: A Scoping Review. Journal of dental research Pandis, N., Fleming, P. S., Katsaros, C., Ioannidis, J. P. 2020: 22034520962751

  Abstract

  Research waste is highly prevalent across biomedical investigations. We aimed to assess the evidence on the extent of research waste in dental research. We performed a scoping review of empirical evaluations of dental studies assessing the prevalence and impact of limitations in design, conduct, analysis, and reporting of research. PubMed was searched using specific terms to retrieve studies dealing with design, conduct, analysis, and reporting of studies in dentistry, with no year or language restrictions. Of the 1,807 publications identified from the search and from manual searches, 71 were included in this review. The topic and article selection was based on the expert opinion of the authors. The existing evidence suggests that, although there are improvements over time, substantial deficiencies in all areas (design, conduct, analysis, reporting) were prevalent in dental research publications. Waste in research is a multifaceted problem without a simple solution. However, an appreciation of optimal research design and execution is a prerequisite and should be underpinned by policies that include appropriate training in research methods and properly aligned incentives.

  View details for DOI 10.1177/0022034520962751

  View details for PubMedID 33054504

• Access to data from clinical trials in the COVID-19 crisis: open, flexible, and time-sensitive. Journal of clinical epidemiology Ewers, M., Ioannidis, J. P., Plesnila, N. 2020

  View details for DOI 10.1016/j.jclinepi.2020.10.008

  View details for PubMedID 33068714

• Global perspective of COVID-19 epidemiology for a full-cycle pandemic. European journal of clinical investigation Ioannidis, J. P. 2020: e13421

  Abstract

  As of October 2020, there are >1 million documented deaths with COVID-19. Excess deaths can be caused by both COVID-19 and the measures taken. COVID-19 shows extremely strong risk stratification across age, socioeconomic factors, and clinical factors. Calculation of years-of-life-lost from COVID-19 is methodologically challenging that can yield misleading over-estimates. Many early deaths may have been due to suboptimal management, malfunctional health systems, hydroxychloroquine, sending COVID-19 patients to nursing homes, and nosocomial infections; such deaths are partially avoidable moving forward. About 10% of the global population may be infected by October 2020. Global infection fatality rate is 0.15-0.20% (0.03-0.04% in those <70 years), with large variability across locations with different age-structure, institutionalization rates, socioeconomic inequalities, population-level clinical risk profile, public health measures, and health care. There is debate on whether at least 60% of the global population must be infected for herd immunity, or, conversely, mixing heterogeneity and pre-existing cross-immunity may allow substantially lower thresholds. Simulations are presented with a total of 1.58-8.76 million COVID-19 deaths over 5-years (1/2000-12/2024) globally (0.5-2.9% of total global deaths). The most favorable figures in that range would be feasible if high risk groups can be preferentially protected with lower infection rates than the remaining population. Death toll may also be further affected by potential availability of effective vaccines and treatments, optimal management and measures taken, COVID-19 interplay with influenza and other health problems, reinfection potential, and any chronic COVID-19 consequences. Targeted, precise management of the pandemic and avoiding past mistakes would help minimize mortality.

  View details for DOI 10.1111/eci.13423

  View details for PubMedID 33026101

• Transparency and reproducibility in artificial intelligence. Nature Haibe-Kains, B., Adam, G. A., Hosny, A., Khodakarami, F., Massive Analysis Quality Control (MAQC) Society Board of Directors, Waldron, L., Wang, B., McIntosh, C., Goldenberg, A., Kundaje, A., Greene, C. S., Broderick, T., Hoffman, M. M., Leek, J. T., Korthauer, K., Huber, W., Brazma, A., Pineau, J., Tibshirani, R., Hastie, T., Ioannidis, J. P., Quackenbush, J., Aerts, H. J., Shraddha, T., Kusko, R., Sansone, S., Tong, W., Wolfinger, R. D., Mason, C. E., Jones, W., Dopazo, J., Furlanello, C. 2020; 586 (7829): E14–E16

  View details for DOI 10.1038/s41586-020-2766-y

  View details for PubMedID 33057217

• Updated science-wide author databases of standardized citation indicators. PLoS biology Ioannidis, J. P., Boyack, K. W., Baas, J. 2020; 18 (10): e3000918

  Abstract

  This Formal Comment presents an update to citation databases of top-cited scientists across all scientific fields, including more granular information on diverse indicators.

  View details for DOI 10.1371/journal.pbio.3000918

  View details for PubMedID 33064726

• A Genome-wide Association Study for Concussion Risk. Medicine and science in sports and exercise Kim, S. K., Roche, M. D., Fredericson, M., Dragoo, J. L., Horton, B. H., Avins, A. L., Belanger, H. G., Ioannidis, J. P., Abrams, G. D. 2020

  Abstract

  PURPOSE: To screen the entire genome for genetic markers associated with risk for concussion.METHODS: A genome-wide-association (GWA) analyses was performed utilizing data from the Kaiser Permanente Research Board (KPRB) and the United Kingdom (UK) Biobank. Concussion cases were identified based on electronic health records from KPRB and UK Biobank from individuals of European ancestry. Genome-wide association analyses from both cohorts were tested for concussion using a logistic regression model adjusting for sex, height, weight and race/ethnicity using allele counts for single nucleotide polymorphisms (SNPs). Previously identified genes within the literature were also tested for association with concussion.RESULTS: There was a total of 4,064 cases of concussion and 291,472 controls within the databases, with two SNPs demonstrating a genome-wide significant association with concussion. The first polymorphism, rs144663795 (p = 9.7x10; OR=2.91 per allele copy), is located within the intron of SPATA5. Strong, deleterious mutations in SPATA5 cause intellectual disablility, hearing loss and vision loss. The second polymorphism, rs117985931 (p = 3.97x10; OR= 3.59 per allele copy) is located within PLXNA4. PLXNA4 plays a key role is axon outgrowth during neural development, and DNA variants in PLXNA4 are associated with risk for Alzheimer's disease. Previous investigations have identified five candidate genes that may be associated with concussion, but none showed a significant association in the current model (p < 0.05).CONCLUSION: Two genetic markers were identified as potential risk factors for concussion and deserve further validation and investigation of molecular mechanisms.

  View details for DOI 10.1249/MSS.0000000000002529

  View details for PubMedID 33017352

• Global assessment of C-reactive protein and health-related outcomes: an umbrella review of evidence from observational studies and Mendelian randomization studies. European journal of epidemiology Markozannes, G., Koutsioumpa, C., Cividini, S., Monori, G., Tsilidis, K. K., Kretsavos, N., Theodoratou, E., Gill, D., Ioannidis, J. P., Tzoulaki, I. 2020

  Abstract

  C-reactive protein (CRP) has been studied extensively for association with a large number of non-infectious diseases and outcomes. We aimed to evaluate the breadth and validity of associations between CRP and non-infectious, chronic health outcomes and biomarkers. We conducted an umbrella review of systematic reviews and meta-analyses and a systematic review of Mendelian randomization (MR) studies. PubMed, Scopus, and Cochrane Database of Systematic Reviews were systematically searched from inception up to March 2019. Meta-analyses of observational studies and MR studies examining associations between CRP and health outcomes were identified, excluding studies on the diagnostic value of CRP for infections. We found 113 meta-analytic comparisons of observational studies and 196 MR analyses, covering a wide range of outcomes. The overwhelming majority of the meta-analyses of observational studies reported a nominally statistically significant result (95/113, 84.1%); however, the majority of the meta-analyses displayed substantial heterogeneity (47.8%), small study effects (39.8%) or excess significance (41.6%). Only two outcomes, cardiovascular mortality and venous thromboembolism, showed convincing evidence of association with CRP levels. When examining the MR literature, we found MR studies for 53/113 outcomes examined in the observational study meta-analyses but substantial support for a causal association with CRP was not observed for any phenotype. Despite the striking amount of research on CRP, convincing evidence for associations and causal effects is remarkably limited.

  View details for DOI 10.1007/s10654-020-00681-w

  View details for PubMedID 32978716

• Depression prevalence using the HADS-D compared to SCID major depression classification: An individual participant data meta-analysis. Journal of psychosomatic research Brehaut, E., Neupane, D., Levis, B., Wu, Y., Sun, Y., Krishnan, A., He, C., Bhandari, P. M., Negeri, Z., Riehm, K. E., Rice, D. B., Azar, M., Yan, X. W., Imran, M., Chiovitti, M. J., Saadat, N., Cuijpers, P., Ioannidis, J. P., Markham, S., Patten, S. B., Ziegelstein, R. C., Henry, M., Ismail, Z., Loiselle, C. G., Mitchell, N. D., Tonelli, M., Boruff, J. T., Kloda, L. A., Beraldi, A., Braeken, A. P., Carter, G., Clover, K., Conroy, R. M., Cukor, D., da Rocha E Silva, C. E., De Souza, J., Downing, M. G., Feinstein, A., Ferentinos, P. P., Fischer, F. H., Flint, A. J., Fujimori, M., Gallagher, P., Goebel, S., Jette, N., Juliao, M., Keller, M., Kjargaard, M., Love, A. W., Lowe, B., Martin-Santos, R., Michopoulos, I., Navines, R., O'Rourke, S. J., Ozturk, A., Pintor, L., Ponsford, J. L., Rooney, A. G., Sanchez-Gonzalez, R., Schwarzbold, M. L., Sharpe, M., Simard, S., Singer, S., Stone, J., Tung, K., Turner, A., Walker, J., Walterfang, M., White, J., Benedetti, A., Thombs, B. D. 2020; 139: 110256

  Abstract

  OBJECTIVES: Validated diagnostic interviews are required to classify depression status and estimate prevalence of disorder, but screening tools are often used instead. We used individual participant data meta-analysis to compare prevalence based on standard Hospital Anxiety and Depression Scale - depression subscale (HADS-D) cutoffs of ≥8 and ≥11 versus Structured Clinical Interview for DSM (SCID) major depression and determined if an alternative HADS-D cutoff could more accurately estimate prevalence.METHODS: We searched Medline, Medline In-Process & Other Non-Indexed Citations via Ovid, PsycINFO, and Web of Science (inception-July 11, 2016) for studies comparing HADS-D scores to SCID major depression status. Pooled prevalence and pooled differences in prevalence for HADS-D cutoffs versus SCID major depression were estimated.RESULTS: 6005 participants (689 SCID major depression cases) from 41 primary studies were included. Pooled prevalence was 24.5% (95% Confidence Interval (CI): 20.5%, 29.0%) for HADS-D≥8, 10.7% (95% CI: 8.3%, 13.8%) for HADS-D≥11, and 11.6% (95% CI: 9.2%, 14.6%) for SCID major depression. HADS-D≥11 was closest to SCID major depression prevalence, but the 95% prediction interval for the difference that could be expected for HADS-D≥11 versus SCID in a new study was -21.1% to 19.5%.CONCLUSIONS: HADS-D≥8 substantially overestimates depression prevalence. Of all possible cutoff thresholds, HADS-D≥11 was closest to the SCID, but there was substantial heterogeneity in the difference between HADS-D≥11 and SCID-based estimates. HADS-D should not be used as a substitute for a validated diagnostic interview.

  View details for DOI 10.1016/j.jpsychores.2020.110256

  View details for PubMedID 33069051

• An empirical comparison of three methods for multiple cut-off diagnostic test meta-analysis of the Patient Health Questionnaire-9 (PHQ-9) depression screening tool using published data versus individual level data. Research synthesis methods Andrea, B., Brooke, L., Gerta, R., Jones Hayley, E., Martin, S., Ioannidis John, P. A., Brett, T., DEPRESsion Screening Data (DEPRESSD) Collaboration, Riehm, K. E., Saadat, N., Levis, A. W., Azar, M., Rice, D. B., Sun, Y., Krishnan, A., He, C., Wu, Y., Bhandari, P. M., Imran, M., Boruff, J., Cuijpers, P., Gilbody, S., Kloda, L. A., McMillan, D., Shrier, I., Ziegelstein, R. C., Akena, D. H., Arroll, B., Baradaran, H. R., Bombardier, C. H., Butterworth, P., Carter, G., Chagas, M. H., Chan, J. C., Cholera, R., Chowdhary, N., Clover, K., Conwell, Y., de Man-van Ginkel, J. M., Delgadillo, J., Fischer, F. H., Fung, D., Gelaye, B., Goodyear-Smith, F., Harter, M., Hyphantis, T., Ismail, K., Jette, N., Khamseh, M. E., Kiely, K. M., Liu, S., Lotrakul, M., Loureiro, S. R., Lowe, B., Marsh, L., McGuire, A., Sidik, S. M., Munhoz, T. N., Patel, V., Pence, B. W., Reuter, K., Rooney, A. G., Santos, I. S., Shaaban, J., Sidebottom, A., Stafford, L., Turner, A., van der Feltz-Cornelis, C. M., van Weert, H. C., Vohringer, P. A., White, J., Whooley, M. A., Winkley, K., Yamada, M., Zhang, Y. 2020

  Abstract

  Selective cut-off reporting in primary diagnostic accuracy studies with continuous or ordinal data may result in biased estimates when meta-analyzing studies. Collecting individual participant data (IPD) and estimating accuracy across all relevant cut-offs for all studies can overcome such bias but is labour-intensive. We meta-analyzed the diagnostic accuracy of the Patient Health Questionnaire-9 (PHQ-9) depression screening tool. We compared results for two statistical methods proposed by Steinhauser and by Jones to account for missing cut-offs, with results from a series of bivariate random effects models (BRM) estimated separately at each cut-off. We applied the methods to a dataset that contained information only on cut-offs that were reported in the primary publications, and to the full IPD dataset that contained information for all cut-offs for every study. For each method, we estimated pooled sensitivity and specificity and associated 95% confidence intervals for each cut-off and area under the curve (AUC). The full IPD dataset comprised data from 45 studies, 15020 subjects and 1972 cases of major depression, and included information on every possible cut-off. When using data available in publications, using statistical approaches out-performed the BRM applied to the same data. AUC was similar for all approaches when using the full IPD dataset, though pooled estimates were slightly different. Overall, using statistical methods to fill in missing cut-off data recovered the receiver operating characteristic (ROC) curve from the full IPD dataset well when using only the published subset. All methods performed similarly when applied to the full IPD dataset. This article is protected by copyright. All rights reserved.

  View details for DOI 10.1002/jrsm.1443

  View details for PubMedID 32896096

• Population-level COVID-19 mortality risk for non-elderly individuals overall and for non-elderly individuals without underlying diseases in pandemic epicenters. Environmental research Ioannidis, J. P., Axfors, C., Contopoulos-Ioannidis, D. G. 2020; 188: 109890

  Abstract

  OBJECTIVE: To provide estimates of the relative rate of COVID-19 death in people <65 years old versus older individuals in the general population, the absolute risk of COVID-19 death at the population level during the first epidemic wave, and the proportion of COVID-19 deaths in non-elderly people without underlying diseases in epicenters of the pandemic.ELIGIBLE DATA: Cross-sectional survey of countries and US states with at least 800 COVID-19 deaths as of April 24, 2020 and with information on the number of deaths in people with age <65. Data were available for 14 countries (Belgium, Canada, France, Germany, India, Ireland, Italy, Mexico, Netherlands, Portugal, Spain, Sweden, Switzerland, UK) and 13 US states (California, Connecticut, Florida, Georgia, Illinois, Indiana, Louisiana, Maryland, Massachusetts, Michigan, New Jersey, New York, Pennsylvania). We also examined available data on COVID-19 deaths in people with age <65 and no underlying diseases.MAIN OUTCOME MEASURES: Proportion of COVID-19 deaths in people <65 years old; relative mortality rate of COVID-19 death in people <65 versus ≥65 years old; absolute risk of COVID-19 death in people <65 and in those ≥80 years old in the general population as of June 17, 2020; absolute COVID-19 mortality rate expressed as equivalent of mortality rate from driving a motor vehicle.RESULTS: Individuals with age <65 account for 4.5-11.2% of all COVID-19 deaths in European countries and Canada, 8.3-22.7% in the US locations, and were the majority in India and Mexico. People <65 years old had 30- to 100-fold lower risk of COVID-19 death than those ≥65 years old in 11 European countries and Canada, 16- to 52-fold lower risk in US locations, and less than 10-fold in India and Mexico. The absolute risk of COVID-19 death as of June 17, 2020 for people <65 years old in high-income countries ranged from 10 (Germany) to 349 per million (New Jersey) and it was 5 per million in India and 96 per million in Mexico. The absolute risk of COVID-19 death for people ≥80 years old ranged from 0.6 (Florida) to 17.5 per thousand (Connecticut). The COVID-19 mortality rate in people <65 years old during the period of fatalities from the epidemic was equivalent to the mortality rate from driving between 4 and 82 miles per day for 13 countries and 5 states, and was higher (equivalent to the mortality rate from driving 106-483 miles per day) for 8 other states and the UK. People <65 years old without underlying predisposing conditions accounted for only 0.7-3.6% of all COVID-19 deaths in France, Italy, Netherlands, Sweden, Georgia, and New York City and 17.7% in Mexico.CONCLUSIONS: People <65 years old have very small risks of COVID-19 death even in pandemic epicenters and deaths for people <65 years without underlying predisposing conditions are remarkably uncommon. Strategies focusing specifically on protecting high-risk elderly individuals should be considered in managing the pandemic.

  View details for DOI 10.1016/j.envres.2020.109890

  View details for PubMedID 32846654

• Prediction of RECRUITment In randomized clinical Trials (RECRUIT-IT)-rationale and design for an international collaborative study. Trials Kasenda, B., Liu, J., Jiang, Y., Gajewski, B., Wu, C., von Elm, E., Schandelmaier, S., Moffa, G., Trelle, S., Schmitt, A. M., Herbrand, A. K., Gloy, V., Speich, B., Hopewell, S., Hemkens, L. G., Sluka, C., McGill, K., Meade, M., Cook, D., Lamontagne, F., Treluyer, J., Haidich, A., Ioannidis, J. P., Treweek, S., Briel, M. 2020; 21 (1): 731

  Abstract

  BACKGROUND: Poor recruitment of patients is the predominant reason for early termination of randomized clinical trials (RCTs). Systematic empirical investigations and validation studies of existing recruitment models, however, are lacking. We aim to provide evidence-based guidance on how to predict and monitor recruitment of patients into RCTs. Our specific objectives are the following: (1) to establish a large sample of RCTs (target n=300) with individual patient recruitment data from a large variety of RCTs, (2) to investigate participant recruitment patterns and study site recruitment patterns and their association with the overall recruitment process, (3) to investigate the validity of a freely available recruitment model, and (4) to develop a user-friendly tool to assist trial investigators in the planning and monitoring of the recruitment process.METHODS: Eligible RCTs need to have completed the recruitment process, used a parallel group design, and investigated any healthcare intervention where participants had the free choice to participate. To establish the planned sample of RCTs, we will use our contacts to national and international RCT networks, clinical trial units, and individual trial investigators. From included RCTs, we will collect patient-level information (date of randomization), site-level information (date of trial site activation), and trial-level information (target sample size). We will examine recruitment patterns using recruitment trajectories and stratifications by RCT characteristics. We will investigate associations of early recruitment patterns with overall recruitment by correlation and multivariable regression. To examine the validity of a freely available Bayesian prediction model, we will compare model predictions to collected empirical data of included RCTs. Finally, we will user-test any promising tool using qualitative methods for further tool improvement.DISCUSSION: This research will contribute to a better understanding of participant recruitment to RCTs, which could enhance efficiency and reduce the waste of resources in clinical research with a comprehensive, concerted, international effort.

  View details for DOI 10.1186/s13063-020-04666-8

  View details for PubMedID 32825846

• A case study in model failure? COVID-19 daily deaths and ICU bed utilisation predictions in New York state. European journal of epidemiology Chin, V., Samia, N. I., Marchant, R., Rosen, O., Ioannidis, J. P., Tanner, M. A., Cripps, S. 2020

  Abstract

  Forecasting models have been influential in shaping decision-making in the COVID-19 pandemic. However, there is concern that their predictions may have been misleading. Here, we dissect the predictions made by four models for the daily COVID-19 death counts between March 25 and June 5 in New York state, as well as the predictions of ICU bed utilisation made by the influential IHME model. We evaluated the accuracy of the point estimates and the accuracy of the uncertainty estimates of the model predictions. First, we compared the "ground truth" data sources on daily deaths against which these models were trained. Three different data sources were used by these models, and these had substantial differences in recorded daily death counts. Two additional data sources that we examined also provided different death counts per day. For accuracy of prediction, all models fared very poorly. Only 10.2% of the predictions fell within 10% of their training ground truth, irrespective of distance into the future. For accurate assessment of uncertainty, only one model matched relatively well the nominal 95% coverage, but that model did not start predictions until April 16, thus had no impact on early, major decisions. For ICU bed utilisation, the IHME model was highly inaccurate; the point estimates only started to match ground truth after the pandemic wave had started to wane. We conclude that trustworthy models require trustworthy input data to be trained upon. Moreover, models need to be subjected to prespecified real time performance tests, before their results are provided to policy makers and public health officials.

  View details for DOI 10.1007/s10654-020-00669-6

  View details for PubMedID 32780189

• The quality of evidence for medical interventions does not improve or worsen: a Meta-Epidemiological Study of Cochrane Reviews. Journal of clinical epidemiology Howick, J., DiplDS, D. K., Pandis, N., Fleming, P. S., Loef, M., Walach, H., Schmidt, S., Ioannidis, J. P. 2020

  Abstract

  BACKGROUND: A previous analysis of Cochrane Reviews published between January 1st, 2013 and June 30th, 2014 found that only 13.5% reported high quality evidence for the intervention according the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system. 31.7% had low level, and 24% revealed very low level of evidence. Many of these reviews have been updated, and it is unknown whether the updated reviews report a change in the quality of evidence.OBJECTIVES: To determine the change in quality of evidence in updates of Cochrane reviews that were initially published between 1st January 2013 and 30th June 2014.METHODS: We searched the Cochrane Database of Systematic Reviews on March 20th, 2020 to identify which of the reviews from the initial (2013/14) sample have been updated. Using the same methods to determine the quality of evidence in the previous analysis, we assessed the quality of evidence for the first listed primary outcomes in the updated reviews.RESULTS: Of the 608 reviews in the original sample, 154 had been updated with 151 presenting available data for both original and updated SRs (24.8%). The updated reviews included: 15 (9.9%) with high quality evidence, 56 (37.1%) with moderate, 47 (31.1%) with low, and 33 (21.9%) with very low-quality evidence. No change in the GRADE quality of evidence was found for most (103, 68.2%) of the updated reviews. Of the 48 reviews with a change in GRADE rating (58.3%) were downgraded, mostly to low or very low. The quality of evidence rating improved in 20 (41.7%), although only 6 reviews were promoted to high quality.CONCLUSIONS: Updated systematic reviews continued to suggest that only a minority of outcomes for healthcare interventions are supported by high-quality evidence. The quality of the evidence did not consistently improve or worsen in updated reviews.

  View details for DOI 10.1016/j.jclinepi.2020.08.005

  View details for PubMedID 32890636

• Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne Schandelmaier, S., Briel, M., Varadhan, R., Schmid, C. H., Devasenapathy, N., Hayward, R. A., Gagnier, J., Borenstein, M., van der Heijden, G. J., Dahabreh, I. J., Sun, X., Sauerbrei, W., Walsh, M., Ioannidis, J. P., Thabane, L., Guyatt, G. H. 2020; 192 (32): E901–E906

  Abstract

  BACKGROUND: Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN).METHODS: To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification.RESULTS: The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument.INTERPRETATION: The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others.

  View details for DOI 10.1503/cmaj.200077

  View details for PubMedID 32778601

• Reproducible research practices and transparency in reproductive endocrinology and infertility articles. Fertility and sterility Kemper, J. M., Rolnik, D. L., Mol, B. W., Ioannidis, J. P. 2020

  Abstract

  OBJECTIVE: To analyse the published literature in reproductive endocrinology and infertility (REI) to examine the transparency and the use of reproducible research practices of the scientific literature and to identify possible avenues for improvement.DESIGN: Meta-epidemiologic study. We examined the first 20 consecutive full-text original articles presenting primary data from five REI-specific journals for 2013 and for 2018, and eligible REI articles published in 2013-2018 in five high-impact general journals. Eligible articles were required to be full-text original articles, presenting primary data.SETTING: Not applicable.PATIENT(S): Not applicable.INTERVENTION(S): Not applicable.MAIN OUTCOME MEASURE(S): Each article was assessed for study type, trial registration, protocol and raw data availability, funding and conflict of interest declarations, inclusion in subsequent systematic reviews and/or meta-analyses, sample size, and whether the work claimed to be novel or replication. Sample sizes and citation counts also were obtained.RESULT(S): A total of 222 articles were deemed eligible; 98 from REI journals published in 2013, 90 from REI journals published in 2018, and 34 from high-impact journals. There were 37 studies registered, 15 contained a protocol, and two stated actively that they were willing to share data. Most studies provided a statement about funding and conflicts of interest. Two articles explicitly described themselves as replications. All randomized controlled trial published in REI journals were registered prospectively; many meta-analyses were not registered. High-impact journal articles had a greater median sample size and more citations and were more likely to be registered, to have a protocol, and to claim novelty explicitly when compared with REI 2013 and 2018 articles.CONCLUSION(S): Research in REI can be improved in prospective registration, routine availability of protocols, wider sharing of raw data whenever feasible, and more emphasis on replication.

  View details for DOI 10.1016/j.fertnstert.2020.05.020

  View details for PubMedID 32771255

• Sample size evolution in neuroimaging research: an evaluation of highly-cited studies (1990-2012) and of latest practices (2017-2018) in high-impact journals. NeuroImage Szucs, D., Ioannidis, J. P. 2020: 117164

  Abstract

  We evaluated 1038 of the most cited structural and functional (fMRI) magnetic resonance brain imaging papers (1161 studies) published during 1990-2012 and 270 papers (300 studies) published in top neuroimaging journals in 2017 and 2018. 96% of highly cited experimental fMRI studies had a single group of participants and these studies had median sample size of 12, highly cited clinical fMRI studies (with patient participants) had median sample size of 14.5, and clinical structural MRI studies had median sample size of 50. The sample size of highly cited experimental fMRI studies increased at a rate of 0.74 participant/year and this rate of increase was commensurate with the median sample sizes of neuroimaging studies published in top neuroimaging journals in 2017 (23 participants) and 2018 (24 participants). Only 4 of 131 papers in 2017 and 5 of 142 papers in 2018 had pre-study power calculations, most for single t-tests and correlations. Only 14% of highly cited papers reported the number of excluded participants whereas 49% of papers with their own data in 2017 and 2018 reported excluded participants. Publishers and funders should require pre-study power calculations necessitating the specification of effect sizes. The field should agree on universally required reporting standards. Reporting formats should be standardized so that crucial study parameters could be identified unequivocally.

  View details for DOI 10.1016/j.neuroimage.2020.117164

  View details for PubMedID 32679253

• Spin, Bias, and Clinical Utility in Systematic Reviews of Diagnostic Studies. Clinical chemistry Ioannidis, J. P. 2020

  View details for DOI 10.1093/clinchem/hvaa114

  View details for PubMedID 32613243

• MINIMAR (MINimum Information for Medical AI Reporting): Developing reporting standards for artificial intelligence in health care. Journal of the American Medical Informatics Association : JAMIA Hernandez-Boussard, T., Bozkurt, S., Ioannidis, J. P., Shah, N. H. 2020

  Abstract

  The rise of digital data and computing power have contributed to significant advancements in artificial intelligence (AI), leading to the use of classification and prediction models in health care to enhance clinical decision-making for diagnosis, treatment and prognosis. However, such advances are limited by the lack of reporting standards for the data used to develop those models, the model architecture, and the model evaluation and validation processes. Here, we present MINIMAR (MINimum Information for Medical AI Reporting), a proposal describing the minimum information necessary to understand intended predictions, target populations, and hidden biases, and the ability to generalize these emerging technologies. We call for a standard to accurately and responsibly report on AI in health care. This will facilitate the design and implementation of these models and promote the development and use of associated clinical decision support tools, as well as manage concerns regarding accuracy and bias.

  View details for DOI 10.1093/jamia/ocaa088

  View details for PubMedID 32594179

• Risk factors and risk prediction models for colorectal cancer metastasis and recurrence: an umbrella review of systematic reviews and meta-analyses of observational studies. BMC medicine Xu, W., He, Y., Wang, Y., Li, X., Young, J., Ioannidis, J. P., Dunlop, M. G., Theodoratou, E. 2020; 18 (1): 172

  Abstract

  BACKGROUND: There is a clear need for systematic appraisal of models/factors predicting colorectal cancer (CRC) metastasis and recurrence because clinical decisions about adjuvant treatment are taken on the basis of such variables.METHODS: We conducted an umbrella review of all systematic reviews of observational studies (with/without meta-analysis) that evaluated risk factors of CRC metastasis and recurrence. We also generated an updated synthesis of risk prediction models for CRC metastasis and recurrence. We cross-assessed individual risk factors and risk prediction models.RESULTS: Thirty-four risk factors for CRC metastasis and 17 for recurrence were investigated. Twelve of 34 and 4/17 risk factors with p<0.05 were estimated to change the odds of the outcome at least 3-fold. Only one risk factor (vascular invasion for lymph node metastasis [LNM] in pT1 CRC) presented convincing evidence. We identified 24 CRC risk prediction models. Across 12 metastasis models, six out of 27 unique predictors were assessed in the umbrella review and four of them changed the odds of the outcome at least 3-fold. Across 12 recurrence models, five out of 25 unique predictors were assessed in the umbrella review and only one changed the odds of the outcome at least 3-fold.CONCLUSIONS: This study provides an in-depth evaluation and cross-assessment of 51 risk factors and 24 prediction models. Our findings suggest that a minority of influential risk factors are employed in prediction models, which indicates the need for a more rigorous and systematic model construction process following evidence-based methods.

  View details for DOI 10.1186/s12916-020-01618-6

  View details for PubMedID 32586325

• Academic criteria for promotion and tenure in biomedical sciences faculties: cross sectional analysis of international sample of universities. BMJ (Clinical research ed.) Rice, D. B., Raffoul, H., Ioannidis, J. P., Moher, D. 2020; 369: m2081

  Abstract

  OBJECTIVE: To determine the presence of a set of pre-specified traditional and non-traditional criteria used to assess scientists for promotion and tenure in faculties of biomedical sciences among universities worldwide.DESIGN: Cross sectional study.SETTING: International sample of universities.PARTICIPANTS: 170 randomly selected universities from the Leiden ranking of world universities list.MAIN OUTCOME MEASURE: Presence of five traditional (for example, number of publications) and seven non-traditional (for example, data sharing) criteria in guidelines for assessing assistant professors, associate professors, and professors and the granting of tenure in institutions with biomedical faculties.RESULTS: A total of 146 institutions had faculties of biomedical sciences, and 92 had eligible guidelines available for review. Traditional criteria of peer reviewed publications, authorship order, journal impact factor, grant funding, and national or international reputation were mentioned in 95% (n=87), 37% (34), 28% (26), 67% (62), and 48% (44) of the guidelines, respectively. Conversely, among non-traditional criteria, only citations (any mention in 26%; n=24) and accommodations for employment leave (37%; 34) were relatively commonly mentioned. Mention of alternative metrics for sharing research (3%; n=3) and data sharing (1%; 1) was rare, and three criteria (publishing in open access mediums, registering research, and adhering to reporting guidelines) were not found in any guidelines reviewed. Among guidelines for assessing promotion to full professor, traditional criteria were more commonly reported than non-traditional criteria (traditional criteria 54.2%, non-traditional items 9.5%; mean difference 44.8%, 95% confidence interval 39.6% to 50.0%; P=0.001). Notable differences were observed across continents in whether guidelines were accessible (Australia 100% (6/6), North America 97% (28/29), Europe 50% (27/54), Asia 58% (29/50), South America 17% (1/6)), with more subtle differences in the use of specific criteria.CONCLUSIONS: This study shows that the evaluation of scientists emphasises traditional criteria as opposed to non-traditional criteria. This may reinforce research practices that are known to be problematic while insufficiently supporting the conduct of better quality research and open science. Institutions should consider incentivising non-traditional criteria.STUDY REGISTRATION: Open Science Framework (https://osf.io/26ucp/?view_only=b80d2bc7416543639f577c1b8f756e44).

  View details for DOI 10.1136/bmj.m2081

  View details for PubMedID 32586791

• Efficacy and acceptability of pharmacological and non-pharmacological interventions for non-specific chronic low back pain: a protocol for a systematic review and network meta-analysis. Systematic reviews Thompson, T., Dias, S., Poulter, D., Weldon, S., Marsh, L., Rossato, C., Shin, J. I., Firth, J., Veronese, N., Dragioti, E., Stubbs, B., Solmi, M., Maher, C. G., Cipriani, A., Ioannidis, J. P. 2020; 9 (1): 130

  Abstract

  BACKGROUND: Despite the enormous financial and humanistic burden of chronic low back pain (CLBP), there is little consensus on what constitutes the best treatment options from a multitude of competing interventions. The objective of this network meta-analysis (NMA) is to determine the relative efficacy and acceptability of primary care treatments for non-specific CLBP, with the overarching aim of providing a comprehensive evidence base for informing treatment decisions.METHODS: We will perform a systematic search to identify randomised controlled trials of interventions endorsed in primary care guidelines for the treatment of non-specific CLBP in adults. Information sources searched will include major bibliographic databases (MEDLINE, Embase, CENTRAL, CINAHL, PsycINFO and LILACS) and clinical trial registries. Our primary outcomes will be patient-reported pain ratings and treatment acceptability (all-cause discontinuation), and secondary outcomes will be functional ability, quality of life and patient/physician ratings of overall improvement. A hierarchical Bayesian class-based NMA will be performed to determine the relative effects of different classes of pharmacological (NSAIDs, opioids, paracetamol, anti-depressants, muscle relaxants) and non-pharmacological (exercise, patient education, manual therapies, psychological therapy, multidisciplinary approaches, massage, acupuncture, mindfulness) interventions and individual treatments within a class (e.g. NSAIDs: diclofenac, ibuprofen, naproxen). We will conduct risk of bias assessments and threshold analysis to assess the robustness of the findings to potential bias. We will compute the effect of different interventions relative to placebo/no treatment for both short- and long-term efficacy and acceptability.DISCUSSION: While many factors are important in selecting an appropriate intervention for an individual patient, evidence for the analgesic effects and acceptability of a treatment are key factors in guiding this selection. Thus, this NMA will provide an important source of evidence to inform treatment decisions and future clinical guidelines.SYSTEMATIC REVIEW REGISTRATION: PROSPERO registry number: CRD42019138115.

  View details for DOI 10.1186/s13643-020-01398-3

  View details for PubMedID 32503666

• Effect of low-dose aspirin on health outcomes: An umbrella review of systematic reviews and meta-analyses. British journal of clinical pharmacology Veronese, N., Demurtas, J., Thompson, T., Solmi, M., Pesolillo, G., Celotto, S., Barnini, T., Stubbs, B., Maggi, S., Pilotto, A., Onder, G., Theodoratou, E., Vaona, A., Firth, J., Smith, L., Koyanagi, A., Ioannidis, J. P., Tzoulaki, I. 2020

  Abstract

  AIMS: This study aimed to use an umbrella review methodology to capture the range of outcomes that were associated with low-dose aspirin and to systematically assess the credibility of this evidence.METHODS: Aspirin is associated with several health outcomes, but the overall benefit/risk balance related to aspirin use is unclear. We searched three major databases up to 15 August 2019 for meta-analyses of observational studies and randomized controlled trials (RCTs) including low-dose aspirin compared to placebo or other treatments. Based on random-effects summary effect sizes, 95% prediction intervals, heterogeneity, small-study effects and excess significance, significant meta-analyses of observational studies were classified from convincing (class I) to weak (class IV). For meta-analyses of RCTs, outcomes with random effects P-value < .005 and a moderate/high GRADE assessment, were classified as strong evidence. From 6802 hits, 67 meta-analyses (156 outcomes) were eligible.RESULTS: Observational data showed highly suggestive evidence for aspirin use and increased risk of upper gastrointestinal bleeding (RR = 2.28, 95% CI: 1.97-2.64). In RCTs of low-dose aspirin, we observed strong evidence for lower risk of CVD in people without CVD (RR = 0.83; 95% CI: 0.79-0.87) and in general population (RR = 0.83; 95% CI: 0.79-0.89), higher risk of major gastrointestinal (RR = 1.47; 95% CI: 1.26-1.72) and intracranial bleeding (RR = 1.34; 95% CI: 1.18-1.53), and of major bleedings in people without CVD (RR = 1.62; 95% CI: 1.26-2.08).CONCLUSION: Compared to other active medications, low-dose aspirin had strong evidence for lower risk of bleeding, but also lower comparative efficacy. Low-dose aspirin significantly lowers CVD risk and increases risk of bleeding. Evidence for multiple other health outcomes is limited.

  View details for DOI 10.1111/bcp.14310

  View details for PubMedID 32488906

• PROTOCOL: When and how to replicate systematic reviews CAMPBELL SYSTEMATIC REVIEWS Karunananthan, S., Maxwell, L. J., Welch, V., Petkovic, J., Pardo, J., Rader, T., Avey, M. T., Baptiste-Ngobi, J., Batista, R., Curran, J. A., Ghogomu, E., Graham, I. D., Grimshaw, J. M., Ioannidis, J., Jordan, Z., Jull, J., Lyddiatt, A., Moher, D., Petticrew, M., Pottie, K., Rada, G., Shamseer, L., Shea, B., Siontis, K. C., Tschirhart, N., Vachon, B., Wells, G. A., White, H., Tugwell, P. 2020; 16 (2)

  View details for DOI 10.1002/cl2.1087

  View details for Web of Science ID 000632294200006

• Meta-research: bird's eye views of primary care research. Family practice Tatsioni, A., Ioannidis, J. P. 2020

  View details for DOI 10.1093/fampra/cmaa025

  View details for PubMedID 32424419

• Coronavirus disease 2019: the harms of exaggerated information and non-evidence-based measures. European journal of clinical investigation Ioannidis, J. P. 2020: e13223

  Abstract

  The evolving coronavirus disease 2019 (COVID-19) pandemic1 is certainly cause for concern. Proper communication and optimal decision-making is an ongoing challenge, as data evolve. The challenge is compounded, however, by exaggerated information. This can lead to inappropriate actions. It is important to differentiate promptly the true epidemic from an epidemic of false claims and potentially harmful actions.

  View details for DOI 10.1111/eci.13223

  View details for PubMedID 32202659

• Generating comparative evidence on new drugs and devices after approval. Lancet (London, England) Cipriani, A., Ioannidis, J. P., Rothwell, P. M., Glasziou, P., Li, T., Hernandez, A. F., Tomlinson, A., Simes, J., Naci, H. 2020; 395 (10228): 998–1010

  Abstract

  Certain limitations of evidence available on drugs and devices at the time of market approval often persist in the post-marketing period. Often, post-marketing research landscape is fragmented. When regulatory agencies require pharmaceutical and device manufacturers to conduct studies in the post-marketing period, these studies might remain incomplete many years after approval. Even when completed, many post-marketing studies lack meaningful active comparators, have observational designs, and might not collect patient-relevant outcomes. Regulators, in collaboration with the industry and patients, ought to ensure that the key questions unanswered at the time of drug and device approval are resolved in a timely fashion during the post-marketing phase. We propose a set of seven key guiding principles that we believe will provide the necessary incentives for pharmaceutical and device manufacturers to generate comparative data in the post-marketing period. First, regulators (for drugs and devices), notified bodies (for devices in Europe), health technology assessment organisations, and payers should develop customised evidence generation plans, ensuring that future post-approval studies address any limitations of the data available at the time of market entry impacting the benefit-risk profiles of drugs and devices. Second, post-marketing studies should be designed hierarchically: priority should be given to efforts aimed at evaluating a product's net clinical benefit in randomised trials compared with current known effective therapy, whenever possible, to address common decisional dilemmas. Third, post-marketing studies should incorporate active comparators as appropriate. Fourth, use of non-randomised studies for the evaluation of clinical benefit in the post-marketing period should be limited to instances when the magnitude of effect is deemed to be large or when it is possible to reasonably infer the comparative benefits or risks in settings, in which doing a randomised trial is not feasible. Fifth, efficiency of randomised trials should be improved by streamlining patient recruitment and data collection through innovative design elements. Sixth, governments should directly support and facilitate the production of comparative post-marketing data by investing in the development of collaborative research networks and data systems that reduce the complexity, cost, and waste of rigorous post-marketing research efforts. Last, financial incentives and penalties should be developed or more actively reinforced.

  View details for DOI 10.1016/S0140-6736(19)33177-0

  View details for PubMedID 32199487

• Generating comparative evidence on new drugs and devices after approval LANCET Cipriani, A., Ioannidis, J. A., Rothwell, P. M., Glasziou, P., Li, T., Hernandez, A. F., Tomlinson, A., Simes, J., Naci, H. 2020; 395 (10228): 998–1010

  View details for Web of Science ID 000521744600031

• Coronavirus disease 2019: the harms of exaggerated information and non-evidence-based measures. European journal of clinical investigation Ioannidis, J. P. 2020: e13222

  Abstract

  The evolving coronavirus disease 2019 (COVID-19) epidemic1 is certainly cause for concern. Proper communication and optimal decision-making is an ongoing challenge, as data evolve. The challenge is compounded, however, by exaggerated information. This can lead to inappropriate actions. It is important to differentiate promptly the true epidemic from an epidemic of false claims and potentially harmful actions.

  View details for DOI 10.1111/eci.13222

  View details for PubMedID 32191341

• Science, advocacy, and quackery in nutritional books: an analysis of conflicting advice and purported claims of nutritional best-sellers PALGRAVE COMMUNICATIONS Marton, R. M., Wang, X., Barabasi, A., Ioannidis, J. A. 2020; 6 (1)

  View details for DOI 10.1057/s41599-020-0415-6

  View details for Web of Science ID 000616197800004

• Dissenting Opinions in Nutrition Research-Reply. JAMA Ioannidis, J. P. 2020; 323 (10): 1000–1001

  View details for DOI 10.1001/jama.2020.0491

  View details for PubMedID 32154857

• Validation and Utility Testing of Clinical Prediction Models: Time to Change the Approach. JAMA Adibi, A., Sadatsafavi, M., Ioannidis, J. P. 2020

  View details for DOI 10.1001/jama.2020.1230

  View details for PubMedID 32134437

• Determinants of economic growth: Different time different answer? JOURNAL OF MACROECONOMICS Bruns, S. B., Ioannidis, J. A. 2020; 63

  View details for DOI 10.1016/j.jmacro.2019.103185

  View details for Web of Science ID 000528047300007

• An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017) ROYAL SOCIETY OPEN SCIENCE Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Cruewell, S., Ioannidis, J. A. 2020; 7 (2)

  View details for DOI 10.1098/rsos.190806

  View details for Web of Science ID 000517151500011

• An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017). Royal Society open science Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., Ioannidis, J. P. 2020; 7 (2): 190806

  Abstract

  Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.

  View details for DOI 10.1098/rsos.190806

  View details for PubMedID 32257301

  View details for PubMedCentralID PMC7062098

• Estimating the sample mean and standard deviation from commonly reported quantiles in meta-analysis STATISTICAL METHODS IN MEDICAL RESEARCH McGrath, S., Zhao, X., Steele, R., Thombs, B. D., Benedetti, A., Levis, B., Riehm, K. E., Saadat, N., Levis, A. W., Azar, M., Rice, D. B., Sun, Y., Krishnan, A., He, C., Wu, Y., Bhandari, P., Neupane, D., Imran, M., Boruff, J., Cuijpers, P., Gilbody, S., Ioannidis, J. A., Kloda, L. A., McMillan, D., Patten, S. B., Shrier, I., Ziegelstein, R. C., Akena, D. H., Arroll, B., Ayalon, L., Baradaran, H. R., Baron, M., Beraldi, A., Bombardier, C. H., Butterworth, P., Carter, G., Chagas, M. H., Chan, J. N., Cholera, R., Chowdhary, N., Clover, K., Conwell, Y., Ginkel, J., Delgadillo, J., Fann, J. R., Fischer, F. H., Fischler, B., Fung, D., Gelaye, B., Goodyear-Smith, F., Greeno, C. G., Hall, B. J., Harrison, P. A., Harter, M., Hegerl, U., Hides, L., Hobfoll, S. E., Hudson, M., Hyphantis, T., Inagaki, M., Ismail, K., Jette, N., Khamseh, M. E., Kiely, K. M., Kwan, Y., Lamers, F., Liu, S., Lotrakul, M., Loureiro, S. R., Loewe, B., Marsh, L., McGuire, A., Sidik, S., Munhoz, T. N., Muramatsu, K., Osorio, F. L., Patel, V., Pence, B. W., Persoons, P., Picardi, A., Reuter, K., Rooney, A. G., Santos, I. S., Shaaban, J., Sidebottom, A., Simning, A., Stafford, L., Sung, S. C., Tan, P., Turner, A., van der Feltz-Cornelis, C. M., van Weert, H. C., Vohringer, P. A., White, J., Whooley, M. A., Winkley, K., Yamada, M., Zhang, Y., DEPRESsion Screening Data DEPRESSD 2020

  View details for DOI 10.1177/0962280219889080

  View details for Web of Science ID 000510409300001

• Vibration of effects in epidemiologic studies of alcohol consumption and breast cancer risk. International journal of epidemiology Chu, L., Ioannidis, J. P., Egilman, A. C., Vasiliou, V., Ross, J. S., Wallach, J. D. 2020

  Abstract

  BACKGROUND: Different analytical approaches can influence the associations estimated in observational studies. We assessed the variability of effect estimates reported within and across observational studies evaluating the impact of alcohol on breast cancer.METHODS: We abstracted largest harmful, largest protective and smallest (closest to the null value of 1.0) relative risk estimates in studies included in a recent alcohol-breast cancer meta-analysis, and recorded how they differed based on five model specification characteristics, including exposure definition, exposure contrast levels, study populations, adjustment covariates and/or model approaches. For each study, we approximated vibration of effects by dividing the largest by the smallest effect estimate [i.e. ratio of odds ratio (ROR)].RESULTS: Among 97 eligible studies, 85 (87.6%) reported both harmful and protective relative effect estimates for an alcohol-breast cancer relationship, which ranged from 1.1 to 17.9 and 0.0 to 1.0, respectively. The RORs comparing the largest and smallest estimates in value ranged from 1.0 to 106.2, with a median of 3.0 [interquartile range (IQR) 2.0-5.2]. One-third (35, 36.1%) of the RORs were based on extreme effect estimates with at least three different model specification characteristics; the vast majority (87, 89.7%) had different exposure definitions or contrast levels. Similar vibrations of effect were observed when only extreme estimates with differences based on study populations and/or adjustment covariates were compared.CONCLUSIONS: Most observational studies evaluating the impact of alcohol on breast cancer report relative effect estimates for the same associations that diverge by >2-fold. Therefore, observational studies should estimate the vibration of effects to provide insight regarding the stability of findings.

  View details for DOI 10.1093/ije/dyz271

  View details for PubMedID 31967637

• The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement: Explanation and Elaboration ANNALS OF INTERNAL MEDICINE Kent, D. M., van Klaveren, D., Paulus, J. K., D'Agostino, R., Goodman, S., Hayward, R., Ioannidis, J. A., Patrick-Lake, B., Morton, S., Pencina, M., Raman, G., Ross, J. S., Selker, H. P., Varadhan, R., Vickers, A., Wong, J. B., Steyerberg, E. W. 2020; 172 (1): W1–W25

  Abstract

  The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed to promote the conduct of, and provide guidance for, predictive analyses of heterogeneity of treatment effects (HTE) in clinical trials. The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risk with versus without the intervention, taking into account all relevant patient attributes simultaneously, to support more personalized clinical decision making than can be made on the basis of only an overall average treatment effect. The authors distinguished 2 categories of predictive HTE approaches (a "risk-modeling" and an "effect-modeling" approach) and developed 4 sets of guidance statements: criteria to determine when risk-modeling approaches are likely to identify clinically meaningful HTE, methodological aspects of risk-modeling methods, considerations for translation to clinical practice, and considerations and caveats in the use of effect-modeling approaches. They discuss limitations of these methods and enumerate research priorities for advancing methods designed to generate more personalized evidence. This explanation and elaboration document describes the intent and rationale of each recommendation and discusses related analytic considerations, caveats, and reservations.

  View details for DOI 10.7326/M18-3668

  View details for Web of Science ID 000506650200002

  View details for PubMedID 31711094

• Reply to the Letter to the Editor: "Mixing Apples and Oranges in Assessing Outcomes of Repetitive Transcranial Stimulation Meta-Analyses". Psychotherapy and psychosomatics Amad, A. n., Jardri, R. n., Rousseau, C. n., Larochelle, Y. n., Ioannidis, J. P., Naudet, F. n. 2020: 1

  View details for DOI 10.1159/000505133

  View details for PubMedID 31940648

• Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ (Clinical research ed.) [Anonymous] 2020; 369: m1312

  View details for DOI 10.1136/bmj.m1312

  View details for PubMedID 32238345

• Empirical assessment of bias in machine learning diagnostic test accuracy studies. Journal of the American Medical Informatics Association : JAMIA Crowley, R. J., Tan, Y. J., Ioannidis, J. P. 2020

  Abstract

  Machine learning (ML) diagnostic tools have significant potential to improve health care. However, methodological pitfalls may affect diagnostic test accuracy studies used to appraise such tools. We aimed to evaluate the prevalence and reporting of design characteristics within the literature. Further, we sought to empirically assess whether design features may be associated with different estimates of diagnostic accuracy.We systematically retrieved 2 × 2 tables (n = 281) describing the performance of ML diagnostic tools, derived from 114 publications in 38 meta-analyses, from PubMed. Data extracted included test performance, sample sizes, and design features. A mixed-effects metaregression was run to quantify the association between design features and diagnostic accuracy.Participant ethnicity and blinding in test interpretation was unreported in 90% and 60% of studies, respectively. Reporting was occasionally lacking for rudimentary characteristics such as study design (28% unreported). Internal validation without appropriate safeguards was used in 44% of studies. Several design features were associated with larger estimates of accuracy, including having unreported (relative diagnostic odds ratio [RDOR], 2.11; 95% confidence interval [CI], 1.43-3.1) or case-control study designs (RDOR, 1.27; 95% CI, 0.97-1.66), and recruiting participants for the index test (RDOR, 1.67; 95% CI, 1.08-2.59).Significant underreporting of experimental details was present. Study design features may affect estimates of diagnostic performance in the ML diagnostic test accuracy literature.The present study identifies pitfalls that threaten the validity, generalizability, and clinical value of ML diagnostic tools and provides recommendations for improvement.

  View details for DOI 10.1093/jamia/ocaa075

  View details for PubMedID 32548642

• Using credibility ceilings to explore skepticism about observational evidence. Journal of clinical epidemiology Ioannidis, J. P. 2020

  View details for DOI 10.1016/j.jclinepi.2020.05.004

  View details for PubMedID 32438023

• Credibility ceilings da capo. Journal of clinical epidemiology Ioannidis, J. P. 2020

  View details for DOI 10.1016/j.jclinepi.2020.05.005

  View details for PubMedID 32438025

• Evaluation of confounding in epidemiologic studies assessing alcohol consumption on the risk of ischemic heart disease. BMC medical research methodology Wallach, J. D., Serghiou, S. n., Chu, L. n., Egilman, A. C., Vasiliou, V. n., Ross, J. S., Ioannidis, J. P. 2020; 20 (1): 64

  Abstract

  Among different investigators studying the same exposures and outcomes, there may be a lack of consensus about potential confounders that should be considered as matching, adjustment, or stratification variables in observational studies. Concerns have been raised that confounding factors may affect the results obtained for the alcohol-ischemic heart disease relationship, as well as their consistency and reproducibility across different studies. Therefore, we assessed how confounders are defined, operationalized, and discussed across individual studies evaluating the impact of alcohol on ischemic heart disease risk.For observational studies included in a recent alcohol-ischemic heart disease meta-analysis, we identified all variables adjusted, matched, or stratified for in the largest reported multivariate model (i.e. potential confounders). We recorded how the variables were measured and grouped them into higher-level confounder domains. Abstracts and Discussion sections were then assessed to determine whether authors considered confounding when interpreting their study findings.85 of 87 (97.7%) studies reported multivariate analyses for an alcohol-ischemic heart disease relationship. The most common higher-level confounder domains included were smoking (79, 92.9%), age (74, 87.1%), and BMI, height, and/or weight (57, 67.1%). However, no two models adjusted, matched, or stratified for the same higher-level confounder domains. Most (74/87, 85.1%) articles mentioned or alluded to "confounding" in their Abstract or Discussion sections, but only one stated that their main findings were likely to be affected by residual confounding. There were five (5/87, 5.7%) authors that explicitly asked for caution when interpreting results.There is large variation in the confounders considered across observational studies evaluating the impact of alcohol on ischemic heart disease risk and almost all studies spuriously ignore or eventually dismiss confounding in their conclusions. Given that study results and interpretations may be affected by the mix of potential confounders included within multivariate models, efforts are necessary to standardize approaches for selecting and accounting for confounders in observational studies.

  View details for DOI 10.1186/s12874-020-0914-6

  View details for PubMedID 32171256

• Evidence Relating Health Care Provider Burnout and Quality of Care. Annals of internal medicine Tawfik, D. S., Ioannidis, J. P. 2020; 172 (6): 438–39

  View details for DOI 10.7326/L19-0827

  View details for PubMedID 32176907

• Probability of Major Depression Classification Based on the SCID, CIDI, and MINI Diagnostic Interviews: A Synthesis of Three Individual Participant Data Meta-Analyses. Psychotherapy and psychosomatics Wu, Y. n., Levis, B. n., Ioannidis, J. P., Benedetti, A. n., Thombs, B. D. 2020: 1–13

  Abstract

  Three previous individual participant data meta-analyses (IPDMAs) reported that, compared to the Structured Clinical Interview for the DSM (SCID), alternative reference standards, primarily the Composite International Diagnostic Interview (CIDI) and the Mini International Neuropsychiatric Interview (MINI), tended to misclassify major depression status, when controlling for depression symptom severity. However, there was an important lack of precision in the results.To compare the odds of the major depression classification based on the SCID, CIDI, and MINI.We included and standardized data from 3 IPDMA databases. For each IPDMA, separately, we fitted binomial generalized linear mixed models to compare the adjusted odds ratios (aORs) of major depression classification, controlling for symptom severity and characteristics of participants, and the interaction between interview and symptom severity. Next, we synthesized results using a DerSimonian-Laird random-effects meta-analysis.In total, 69,405 participants (7,574 [11%] with major depression) from 212 studies were included. Controlling for symptom severity and participant characteristics, the MINI (74 studies; 25,749 participants) classified major depression more often than the SCID (108 studies; 21,953 participants; aOR 1.46; 95% confidence interval [CI] 1.11-1.92]). Classification odds for the CIDI (30 studies; 21,703 participants) and the SCID did not differ overall (aOR 1.19; 95% CI 0.79-1.75); however, as screening scores increased, the aOR increased less for the CIDI than the SCID (interaction aOR 0.64; 95% CI 0.52-0.80).Compared to the SCID, the MINI classified major depression more often. The odds of the depression classification with the CIDI increased less as symptom levels increased. Interpretation of research that uses diagnostic interviews to classify depression should consider the interview characteristics.

  View details for DOI 10.1159/000509283

  View details for PubMedID 32814337

• Visualizing the invisible: The effect of asymptomatic transmission on the outbreak dynamics of COVID-19. medRxiv : the preprint server for health sciences Peirlinck, M. n., Linka, K. n., Costabal, F. S., Bhattacharya, J. n., Bendavid, E. n., Ioannidis, J. P., Kuhl, E. n. 2020

  Abstract

  Understanding the outbreak dynamics of the COVID-19 pandemic has important implications for successful containment and mitigation strategies. Recent studies suggest that the population prevalence of SARS-CoV-2 antibodies, a proxy for the number of asymptomatic cases, could be an order of magnitude larger than expected from the number of reported symptomatic cases. Knowing the precise prevalence and contagiousness of asymptomatic transmission is critical to estimate the overall dimension and pandemic potential of COVID-19. However, at this stage, the effect of the asymptomatic population, its size, and its outbreak dynamics remain largely unknown. Here we use reported symptomatic case data in conjunction with antibody seroprevalence studies, a mathematical epidemiology model, and a Bayesian framework to infer the epidemiological characteristics of COVID-19. Our model computes, in real time, the time-varying contact rate of the outbreak, and projects the temporal evolution and credible intervals of the effective reproduction number and the symptomatic, asymptomatic, and recovered populations. Our study quantifies the sensitivity of the outbreak dynamics of COVID-19 to three parameters: the effective reproduction number, the ratio between the symptomatic and asymptomatic populations, and the infectious periods of both groups For nine distinct locations, our model estimates the fraction of the population that has been infected and recovered by Jun 15, 2020 to 24.15% (95% CI: 20.48%-28.14%) for Heinsberg (NRW, Germany), 2.40% (95% CI: 2.09%-2.76%) for Ada County (ID, USA), 46.19% (95% CI: 45.81%-46.60%) for New York City (NY, USA), 11.26% (95% CI: 7.21%-16.03%) for Santa Clara County (CA, USA), 3.09% (95% CI: 2.27%-4.03%) for Denmark, 12.35% (95% CI: 10.03%-15.18%) for Geneva Canton (Switzerland), 5.24% (95% CI: 4.84%-5.70%) for the Netherlands, 1.53% (95% CI: 0.76%-2.62%) for Rio Grande do Sul (Brazil), and 5.32% (95% CI: 4.77%-5.93%) for Belgium. Our method traces the initial outbreak date in Santa Clara County back to January 20, 2020 (95% CI: December 29, 2019 - February 13, 2020). Our results could significantly change our understanding and management of the COVID-19 pandemic: A large asymptomatic population will make isolation, containment, and tracing of individual cases challenging. Instead, managing community transmission through increasing population awareness, promoting physical distancing, and encouraging behavioral changes could become more relevant.

  View details for DOI 10.1101/2020.05.23.20111419

  View details for PubMedID 32869035

  View details for PubMedCentralID PMC7457606

• Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ (Clinical research ed.) Vollmer, S., Mateen, B. A., Bohner, G., Kiraly, F. J., Ghani, R., Jonsson, P., Cumbers, S., Jonas, A., McAllister, K. S., Myles, P., Granger, D., Birse, M., Branson, R., Moons, K. G., Collins, G. S., Ioannidis, J. P., Holmes, C., Hemingway, H. 2020; 368: l6927

  View details for DOI 10.1136/bmj.l6927

  View details for PubMedID 32198138

• Should governments continue lockdown to slow the spread of covid-19? BMJ (Clinical research ed.) Melnick, E. R., Ioannidis, J. P. 2020; 369: m1924

  View details for DOI 10.1136/bmj.m1924

  View details for PubMedID 32493767

• Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: a systematic review and meta-analysis. The international journal of behavioral nutrition and physical activity Beets, M. W., Weaver, R. G., Ioannidis, J. P., Geraci, M. n., Brazendale, K. n., Decker, L. n., Okely, A. D., Lubans, D. n., van Sluijs, E. n., Jago, R. n., Turner-McGrievy, G. n., Thrasher, J. n., Li, X. n., Milat, A. J. 2020; 17 (1): 19

  Abstract

  Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. "Risk of generalizability biases (RGB)" in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity.We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes.A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of - 0.325 (95CI - 0.556 to - 0.094), - 0.346 (- 0.640 to - 0.052), - 0.342 (- 0.498 to - 0.187), and - 0.360 (- 0.631 to - 0.089), respectively.Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.

  View details for DOI 10.1186/s12966-020-0918-y

  View details for PubMedID 32046735

• When to replicate systematic reviews of interventions: consensus checklist. BMJ (Clinical research ed.) Tugwell, P. n., Welch, V. A., Karunananthan, S. n., Maxwell, L. J., Akl, E. A., Avey, M. T., Bhutta, Z. A., Brouwers, M. C., Clark, J. P., Cook, S. n., Cuervo, L. G., Curran, J. A., Ghogomu, E. T., Graham, I. G., Grimshaw, J. M., Hutton, B. n., Ioannidis, J. P., Jordan, Z. n., Jull, J. E., Kristjansson, E. n., Langlois, E. V., Little, J. n., Lyddiatt, A. n., Martin, J. E., Marušić, A. n., Mbuagbaw, L. n., Moher, D. n., Morton, R. L., Nasser, M. n., Page, M. J., Pardo Pardo, J. n., Petkovic, J. n., Petticrew, M. n., Pigott, T. n., Pottie, K. n., Rada, G. n., Rader, T. n., Riddle, A. Y., Rothstein, H. n., Schüneman, H. J., Shamseer, L. n., Shea, B. J., Simeon, R. n., Siontis, K. C., Smith, M. n., Soares-Weiser, K. n., Thavorn, K. n., Tovey, D. n., Vachon, B. n., Valentine, J. n., Villemaire, R. n., Walker, P. n., Weeks, L. n., Wells, G. n., Wilson, D. B., White, H. n. 2020; 370: m2864

  View details for DOI 10.1136/bmj.m2864

  View details for PubMedID 32933948

• Reporting of demographic data and representativeness in machine learning models using electronic health records. Journal of the American Medical Informatics Association : JAMIA Bozkurt, S. n., Cahan, E. M., Seneviratne, M. G., Sun, R. n., Lossio-Ventura, J. A., Ioannidis, J. P., Hernandez-Boussard, T. n. 2020

  Abstract

  The development of machine learning (ML) algorithms to address a variety of issues faced in clinical practice has increased rapidly. However, questions have arisen regarding biases in their development that can affect their applicability in specific populations. We sought to evaluate whether studies developing ML models from electronic health record (EHR) data report sufficient demographic data on the study populations to demonstrate representativeness and reproducibility.We searched PubMed for articles applying ML models to improve clinical decision-making using EHR data. We limited our search to papers published between 2015 and 2019.Across the 164 studies reviewed, demographic variables were inconsistently reported and/or included as model inputs. Race/ethnicity was not reported in 64%; gender and age were not reported in 24% and 21% of studies, respectively. Socioeconomic status of the population was not reported in 92% of studies. Studies that mentioned these variables often did not report if they were included as model inputs. Few models (12%) were validated using external populations. Few studies (17%) open-sourced their code. Populations in the ML studies include higher proportions of White and Black yet fewer Hispanic subjects compared to the general US population.The demographic characteristics of study populations are poorly reported in the ML literature based on EHR data. Demographic representativeness in training data and model transparency is necessary to ensure that ML models are deployed in an equitable and reproducible manner. Wider adoption of reporting guidelines is warranted to improve representativeness and reproducibility.

  View details for DOI 10.1093/jamia/ocaa164

  View details for PubMedID 32935131

• Mortality and Paclitaxel-Coated Devices: An Individual Patient Data Meta-Analysis. Circulation Rocha-Singh, K. J., Duval, S. n., Jaff, M. R., Schneider, P. A., Ansel, G. M., Lyden, S. P., Mullin, C. M., Ioannidis, J. P., Misra, S. n., Tzafriri, A. R., Edelman, E. R., Granada, J. F., White, C. J., Beckman, J. A. 2020

  Abstract

  Background: Paclitaxel-containing devices (PTXD) significantly reduce reintervention in patients with symptomatic femoropopliteal peripheral artery disease (PAD). A recent aggregate-data meta-analysis reported increased late mortality in PAD patients treated with PTXD. We performed an individual patient data (IPD) meta-analysis to evaluate mortality. Methods: Manufacturers of FDA approved and commercially available devices in the United States provided de-identified IPD for independent analysis. Cox proportional hazards one-stage meta-analysis models using intention-to-treat (ITT) methods were used for the primary analysis. A secondary analysis of additionally recovered missing vital status data was performed. The impact of control crossover to PTXD, cause-specific mortality and drug dose-mortality were assessed. Results: 2,185 subjects and 386 deaths from eight PTXD trials with 4-year median follow-up were identified. The primary analysis indicated a 38% (95% confidence interval [CI], 6% to 80%) increased relative mortality risk, corresponding to 4.6% absolute increase, at 5 years associated with PTXD use. Control and treatment arm loss to follow-up and withdrawal were 24% and 23%, respectively. With inclusion of recovered vital status data the excess relative mortality risk was 27% (95% CI, 3% to 58%). This observation was consistent across various scenarios, including as-treated analyses, with no evidence of increased risk over time with PTXD. Mortality risk tended to be increased for all major causes of death. There were no subgroup differences. No drug dosemortality association was identified. Conclusions: This IPD meta-analysis, based on the most complete available data set of mortality events from PTXD randomized controlled trials, identified an absolute 4.6% increased mortality risk associated with PTXD use.

  View details for DOI 10.1161/CIRCULATIONAHA.119.044697

  View details for PubMedID 32370548

• Accuracy of Smartphone Camera Applications for Detecting Atrial Fibrillation: A Systematic Review and Meta-analysis. JAMA network open O'Sullivan, J. W., Grigg, S. n., Crawford, W. n., Turakhia, M. P., Perez, M. n., Ingelsson, E. n., Wheeler, M. T., Ioannidis, J. P., Ashley, E. A. 2020; 3 (4): e202064

  Abstract

  Atrial fibrillation (AF) affects more than 6 million people in the United States; however, much AF remains undiagnosed. Given that more than 265 million people in the United States own smartphones (>80% of the population), smartphone applications have been proposed for detecting AF, but the accuracy of these applications remains unclear.To determine the accuracy of smartphone camera applications that diagnose AF.MEDLINE and Embase were searched until January 2019 for studies that assessed the accuracy of any smartphone applications that use the smartphone's camera to measure the amplitude and frequency of the user's fingertip pulse to diagnose AF.Bivariate random-effects meta-analyses were constructed to synthesize data. The study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) of Diagnostic Test Accuracy Studies reporting guideline.Sensitivity and specificity were measured with bivariate random-effects meta-analysis. To simulate the use of these applications as a screening tool, the positive predictive value (PPV) and negative predictive value (NPV) for different population groups (ie, age ≥65 years and age ≥65 years with hypertension) were modeled. Lastly, the association of methodological limitations with outcomes were analyzed with sensitivity analyses and metaregressions.A total of 10 primary diagnostic accuracy studies, with 3852 participants and 4 applications, were included. The oldest studies were published in 2016 (2 studies [20.0%]), while most studies (4 [40.0%]) were published in 2018. The applications analyzed the pulsewave signal for a mean (range) of 2 (1-5) minutes. The meta-analyzed sensitivity and specificity for all applications combined were 94.2% (95% CI, 92.2%-95.7%) and 95.8% (95% CI, 92.4%-97.7%), respectively. The PPV for smartphone camera applications detecting AF in an asymptomatic population aged 65 years and older was between 19.3% (95% CI, 19.2%-19.4%) and 37.5% (95% CI, 37.4%-37.6%), and the NPV was between 99.8% (95% CI, 99.83%-99.84%) and 99.9% (95% CI, 99.94%-99.95%). The PPV and NPV increased for individuals aged 65 years and older with hypertension (PPV, 20.5% [95% CI, 20.4%-20.6%] to 39.2% [95% CI, 39.1%-39.3%]; NPV, 99.8% [95% CI, 99.8%-99.8%] to 99.9% [95% CI, 99.9%-99.9%]). There were methodological limitations in a number of studies that did not appear to be associated with diagnostic performance, but this could not be definitively excluded given the sparsity of the data.In this study, all smartphone camera applications had relatively high sensitivity and specificity. The modeled NPV was high for all analyses, but the PPV was modest, suggesting that using these applications in an asymptomatic population may generate a higher number of false-positive than true-positive results. Future research should address the accuracy of these applications when screening other high-risk population groups, their ability to help monitor chronic AF, and, ultimately, their associations with patient-important outcomes.

  View details for DOI 10.1001/jamanetworkopen.2020.2064

  View details for PubMedID 32242908

• Preserving equipoise and performing randomized trials for COVID-19 social distancing interventions. Epidemiology and psychiatric sciences Cristea, I. A., Naudet, F. n., Ioannidis, J. P. 2020: 1–27

  View details for DOI 10.1017/S2045796020000992

  View details for PubMedID 33109299

• Scientific petitions and open letters in the era of covid-19. BMJ (Clinical research ed.) Ioannidis, J. P. 2020; 371: m4048

  View details for DOI 10.1136/bmj.m4048

  View details for PubMedID 33106240

• Examining the robustness of observational associations to model, measurement and sampling uncertainty with the vibration of effects framework. International journal of epidemiology Klau, S. n., Hoffmann, S. n., Patel, C. J., Ioannidis, J. P., Boulesteix, A. L. 2020

  Abstract

  The results of studies on observational associations may vary depending on the study design and analysis choices as well as due to measurement error. It is important to understand the relative contribution of different factors towards generating variable results, including low sample sizes, researchers' flexibility in model choices, and measurement error in variables of interest and adjustment variables.We define sampling, model and measurement uncertainty, and extend the concept of vibration of effects in order to study these three types of uncertainty in a common framework. In a practical application, we examine these types of uncertainty in a Cox model using data from the National Health and Nutrition Examination Survey. In addition, we analyse the behaviour of sampling, model and measurement uncertainty for varying sample sizes in a simulation study.All types of uncertainty are associated with a potentially large variability in effect estimates. Measurement error in the variable of interest attenuates the true effect in most cases, but can occasionally lead to overestimation. When we consider measurement error in both the variable of interest and adjustment variables, the vibration of effects are even less predictable as both systematic under- and over-estimation of the true effect can be observed. The results on simulated data show that measurement and model vibration remain non-negligible even for large sample sizes.Sampling, model and measurement uncertainty can have important consequences for the stability of observational associations. We recommend systematically studying and reporting these types of uncertainty, and comparing them in a common framework.

  View details for DOI 10.1093/ije/dyaa164

  View details for PubMedID 33147614

• The worldwide clinical trial research response to the COVID-19 pandemic - the first 100 days. F1000Research Janiaud, P., Axfors, C., Van't Hooft, J., Saccilotto, R., Agarwal, A., Appenzeller-Herzog, C., Contopoulos-Ioannidis, D. G., Danchev, V., Dirnagl, U., Ewald, H., Gartlehner, G., Goodman, S. N., Haber, N. A., Ioannidis, A. D., Ioannidis, J. P., Lythgoe, M. P., Ma, W., Macleod, M., Malicki, M., Meerpohl, J. J., Min, Y., Moher, D., Nagavci, B., Naudet, F., Pauli-Magnus, C., O'Sullivan, J. W., Riedel, N., Roth, J. A., Sauermann, M., Schandelmaier, S., Schmitt, A. M., Speich, B., Williamson, P. R., Hemkens, L. G. 2020; 9: 1193

  Abstract

  Background: Never before have clinical trials drawn as much public attention as those testing interventions for COVID-19. We aimed to describe the worldwide COVID-19 clinical research response and its evolution over the first 100 days of the pandemic. Methods: Descriptive analysis of planned, ongoing or completed trials by April 9, 2020 testing any intervention to treat or prevent COVID-19, systematically identified in trial registries, preprint servers, and literature databases. A survey was conducted of all trials to assess their recruitment status up to July 6, 2020. Results: Most of the 689 trials (overall target sample size 396,366) were small (median sample size 120; interquartile range [IQR] 60-300) but randomized (75.8%; n=522) and were often conducted in China (51.1%; n=352) or the USA (11%; n=76). 525 trials (76.2%) planned to include 155,571 hospitalized patients, and 25 (3.6%) planned to include 96,821 health-care workers. Treatments were evaluated in 607 trials (88.1%), frequently antivirals (n=144) or antimalarials (n=112); 78 trials (11.3%) focused on prevention, including 14 vaccine trials. No trial investigated social distancing. Interventions tested in 11 trials with >5,000 participants were also tested in 169 smaller trials (median sample size 273; IQR 90-700). Hydroxychloroquine alone was investigated in 110 trials. While 414 trials (60.0%) expected completion in 2020, only 35 trials (4.1%; 3,071 participants) were completed by July 6. Of 112 trials with detailed recruitment information, 55 had recruited <20% of the targeted sample; 27 between 20-50%; and 30 over 50% (median 14.8% [IQR 2.0-62.0%]). Conclusions: The size and speed of the COVID-19 clinical trials agenda is unprecedented. However, most trials were small investigating a small fraction of treatment options. The feasibility of this research agenda is questionable, and many trials may end in futility, wasting research resources. Much better coordination is needed to respond to global health threats.

  View details for DOI 10.12688/f1000research.26707.1

  View details for PubMedID 33082937

• Does the COVID-19 pandemic provide an opportunity to eliminate the tobacco industry? The Lancet. Global health Ioannidis, J. P., Jha, P. n. 2020

  View details for DOI 10.1016/S2214-109X(20)30466-6

  View details for PubMedID 33120026

• Preprint Servers' Policies, Submission Requirements, and Transparency in Reporting and Research Integrity Recommendations. JAMA Malicki, M., Jeroncic, A., Ter Riet, G., Bouter, L. M., Ioannidis, J. P., Goodman, S. N., Aalbersberg, I. J. 2020; 324 (18): 1901–3

  View details for DOI 10.1001/jama.2020.17195

  View details for PubMedID 33170231

• Calibrating the Scientific Ecosystem Through Meta-Research Annual Review of Statistics and Its Application Tom, H. E., Stylianos, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., Ioannidis, J. P. 2020; 7

  View details for DOI 10.1146/annurev-statistics-031219-041104

• Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies in medical imaging. BMJ (Clinical research ed.) Nagendran, M. n., Chen, Y. n., Lovejoy, C. A., Gordon, A. C., Komorowski, M. n., Harvey, H. n., Topol, E. J., Ioannidis, J. P., Collins, G. S., Maruthappu, M. n. 2020; 368: m689

  Abstract

  To systematically examine the design, reporting standards, risk of bias, and claims of studies comparing the performance of diagnostic deep learning algorithms for medical imaging with that of expert clinicians.Systematic review.Medline, Embase, Cochrane Central Register of Controlled Trials, and the World Health Organization trial registry from 2010 to June 2019.Randomised trial registrations and non-randomised studies comparing the performance of a deep learning algorithm in medical imaging with a contemporary group of one or more expert clinicians. Medical imaging has seen a growing interest in deep learning research. The main distinguishing feature of convolutional neural networks (CNNs) in deep learning is that when CNNs are fed with raw data, they develop their own representations needed for pattern recognition. The algorithm learns for itself the features of an image that are important for classification rather than being told by humans which features to use. The selected studies aimed to use medical imaging for predicting absolute risk of existing disease or classification into diagnostic groups (eg, disease or non-disease). For example, raw chest radiographs tagged with a label such as pneumothorax or no pneumothorax and the CNN learning which pixel patterns suggest pneumothorax.Adherence to reporting standards was assessed by using CONSORT (consolidated standards of reporting trials) for randomised studies and TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) for non-randomised studies. Risk of bias was assessed by using the Cochrane risk of bias tool for randomised studies and PROBAST (prediction model risk of bias assessment tool) for non-randomised studies.Only 10 records were found for deep learning randomised clinical trials, two of which have been published (with low risk of bias, except for lack of blinding, and high adherence to reporting standards) and eight are ongoing. Of 81 non-randomised clinical trials identified, only nine were prospective and just six were tested in a real world clinical setting. The median number of experts in the comparator group was only four (interquartile range 2-9). Full access to all datasets and code was severely limited (unavailable in 95% and 93% of studies, respectively). The overall risk of bias was high in 58 of 81 studies and adherence to reporting standards was suboptimal (<50% adherence for 12 of 29 TRIPOD items). 61 of 81 studies stated in their abstract that performance of artificial intelligence was at least comparable to (or better than) that of clinicians. Only 31 of 81 studies (38%) stated that further prospective studies or trials were required.Few prospective deep learning studies and randomised trials exist in medical imaging. Most non-randomised trials are not prospective, are at high risk of bias, and deviate from existing reporting standards. Data and code availability are lacking in most studies, and human comparator groups are often small. Future studies should diminish risk of bias, enhance real world clinical relevance, improve reporting and transparency, and appropriately temper conclusions.PROSPERO CRD42019123605.

  View details for DOI 10.1136/bmj.m689

  View details for PubMedID 32213531

• Patient Health Questionnaire-9 scores do not accurately estimate depression prevalence: individual participant data meta-analysis. Journal of clinical epidemiology Levis, B. n., Benedetti, A. n., Ioannidis, J. P., Sun, Y. n., Negeri, Z. n., He, C. n., Wu, Y. n., Krishnan, A. n., Bhandari, P. M., Neupane, D. n., Imran, M. n., Rice, D. B., Riehm, K. E., Saadat, N. n., Azar, M. n., Boruff, J. n., Cuijpers, P. n., Gilbody, S. n., Kloda, L. A., McMillan, D. n., Patten, S. B., Shrier, I. n., Ziegelstein, R. C., Alamri, S. H., Amtmann, D. n., Ayalon, L. n., Baradaran, H. R., Beraldi, A. n., Bernstein, C. N., Bhana, A. n., Bombardier, C. H., Carter, G. n., Chagas, M. H., Chibanda, D. n., Clover, K. n., Conwell, Y. n., Diez-Quevedo, C. n., Fann, J. R., Fischer, F. H., Gholizadeh, L. n., Gibson, L. J., Green, E. P., Greeno, C. G., Hall, B. J., Haroz, E. E., Ismail, K. n., Jetté, N. n., Khamseh, M. E., Kwan, Y. n., Lara, M. A., Liu, S. I., Loureiro, S. R., Löwe, B. n., Marrie, R. A., Marsh, L. n., McGuire, A. n., Muramatsu, K. n., Navarrete, L. n., Osório, F. L., Petersen, I. n., Picardi, A. n., Pugh, S. L., Quinn, T. J., Rooney, A. G., Shinn, E. H., Sidebottom, A. n., Spangenberg, L. n., Lynnette Tan, P. L., Taylor-Rowan, M. n., Turner, A. n., van Weert, H. C., Vöhringer, P. A., Wagner, L. I., White, J. n., Winkley, K. n., Thombs, B. D. 2020

  Abstract

  Depression symptom questionnaires are not for diagnostic classification. Patient Health Questionnaire-9 (PHQ-9) scores ≥ 10 are nonetheless often used to estimate depression prevalence. We compared PHQ-9 ≥ 10 prevalence to Structured Clinical Interview for DSM (SCID) major depression prevalence and assessed whether an alternative PHQ-9 cutoff could more accurately estimate prevalence.Individual participant data meta-analysis of datasets comparing PHQ-9 scores to SCID major depression status.9,242 participants (1,389 SCID major depression cases) from 44 primary studies were included. Pooled PHQ-9 ≥ 10 prevalence was 24.6% (95% CI: 20.8%, 28.9%); pooled SCID major depression prevalence was 12.1% (95% CI: 9.6%, 15.2%); pooled difference was 11.9% (95% CI: 9.3%, 14.6%). Mean study-level PHQ-9 ≥ 10 to SCID-based prevalence ratio was 2.5 times. PHQ-9 ≥ 14 and the PHQ-9 diagnostic algorithm provided prevalence closest to SCID major depression prevalence, but study-level prevalence differed from SCID-based prevalence by an average absolute difference of 4.8% for PHQ-9 ≥ 14 (95% prediction interval: -13.6%, 14.5%) and 5.6 % for the PHQ-9 diagnostic algorithm (95% prediction interval: -16.4%, 15.0%).PHQ-9 ≥ 10 substantially overestimates depression prevalence. There is too much heterogeneity to correct statistically in individual studies.

  View details for DOI 10.1016/j.jclinepi.2020.02.002

  View details for PubMedID 32105798

• Work honored by Nobel prizes clusters heavily in a few scientific fields. PloS one Ioannidis, J. P., Cristea, I., Boyack, K. W. 2020; 15 (7): e0234612

  Abstract

  We aimed to assess whether Nobel prizes (widely considered the most prestigious award in science) are clustering in work done in a few specific disciplines. We mapped the key Nobel prize-related publication of each laureate awarded the Nobel Prize in Medicine, Physics, and Chemistry (1995-2017). These key papers mapped in only narrow sub-regions of a 91,726-cluster map of science created from 63 million Scopus-indexed published items. For each key Nobel paper, a median of 435 (range 0 to 88383) other Scopus-indexed items were published within one year and were more heavily cited than the Nobel paper. Of the 114 high-level domains that science can be divided into, only 36 have had a Nobel prize. Five of the 114 domains (particle physics [14%], cell biology [12.1%], atomic physics [10.9%], neuroscience [10.1%], molecular chemistry [5.3%]) have the lion's share, accounting in total for 52.4% of the Nobel prizes. Using a more granular classification with 849 sub-domains shows that only 71 of these sub-domains (8.3%) have at least one Nobel-related paper. Similar clustering was seen when we mapped all the 40,819 Scopus-indexed publications representing the career-long output of all the Nobel laureates. In conclusion, work resulting in Nobel prizes is concentrated in a small minority of scientific disciplines.

  View details for DOI 10.1371/journal.pone.0234612

  View details for PubMedID 32726312

• What Other Countries Can Learn From Italy During the COVID-19 Pandemic. JAMA internal medicine Boccia, S. n., Ricciardi, W. n., Ioannidis, J. P. 2020

  View details for DOI 10.1001/jamainternmed.2020.1447

  View details for PubMedID 32259190

• Achieving balance with power: lessons from the Balanced Anaesthesia Study. British journal of anaesthesia Spence, J. n., Ioannidis, J. P., Avidan, M. S. 2020

  View details for DOI 10.1016/j.bja.2019.12.027

  View details for PubMedID 31973826

• Citation metrics for appraising scientists: misuse, gaming and proper use. The Medical journal of Australia Ioannidis, J. P., Boyack, K. W. 2020

  View details for DOI 10.5694/mja2.50493

  View details for PubMedID 32017115

• A consensus-based transparency checklist. Nature human behaviour Aczel, B., Szaszi, B., Sarafoglou, A., Kekecs, Z., Kucharsky, S., Benjamin, D., Chambers, C. D., Fisher, A., Gelman, A., Gernsbacher, M. A., Ioannidis, J. P., Johnson, E., Jonas, K., Kousta, S., Lilienfeld, S. O., Lindsay, D. S., Morey, C. C., Monafo, M., Newell, B. R., Pashler, H., Shanks, D. R., Simons, D. J., Wicherts, J. M., Albarracin, D., Anderson, N. D., Antonakis, J., Arkes, H. R., Back, M. D., Banks, G. C., Beevers, C., Bennett, A. A., Bleidorn, W., Boyer, T. W., Cacciari, C., Carter, A. S., Cesario, J., Clifton, C., Conroy, R. M., Cortese, M., Cosci, F., Cowan, N., Crawford, J., Crone, E. A., Curtin, J., Engle, R., Farrell, S., Fearon, P., Fichman, M., Frankenhuis, W., Freund, A. M., Gaskell, M. G., Giner-Sorolla, R., Green, D. P., Greene, R. L., Harlow, L. L., de la Guardia, F. H., Isaacowitz, D., Kolodner, J., Lieberman, D., Logan, G. D., Mendes, W. B., Moersdorf, L., Nyhan, B., Pollack, J., Sullivan, C., Vazire, S., Wagenmakers, E. 2019

  View details for DOI 10.1038/s41562-019-0772-6

  View details for PubMedID 31792401

• Use and reporting of Bland-Altman analyses in studies of self-reported versus measured weight and height. International journal of obesity (2005) Flegal, K. M., Graubard, B., Ioannidis, J. P. 2019

  Abstract

  BACKGROUND/OBJECTIVES: Bland-Altman methods for assessing the agreement between two measures are highly cited. However, these methods may often not be used to assess agreement, and when used, they are not always presented or interpreted correctly. Our objective was to evaluate the use and the quality of reporting of Bland-Altman analyses in studies that compare self-reported with measured weight and height.METHODS: We evaluated the use of Bland-Altman methods in 394 published articles that compared self-reported and measured weight and height data for adolescents or adults. Six reporting criteria were developed: assessment of the normality of the distribution of differences, a complete and correctly labeled Bland-Altman plot displaying the mean difference and limits of agreement (LOA), numerical values and confidence intervals, standard errors, or standard deviations for mean difference, numerical values of LOA, confidence intervals for LOA, and prespecified criteria for acceptable LOA.RESULTS: Only 72/394 (18%) studies comparing self-reported with measured weight and height or BMI used some form of Bland-Altman analyses. No study using Bland-Altman analyses satisfied more than four of the six criteria. Of the 72 studies, 64 gave mean differences along with confidence intervals or standard deviations, 55 provided complete Bland-Altman plots that were appropriately labeled and described, 37 provided numerical values for LOA, 4 reported that they examined the normality of the distribution of differences, 3 provided confidence intervals for LOA, and 3 had prespecified criteria for agreement.CONCLUSIONS: Bland-Altman methods appear to be infrequently used in studies comparing measured with self-reported weight, height, or BMI, and key information is missing in many of those that do use Bland-Altman methods. Future directions would be defining acceptable LOA values and improving the reporting and application of Bland-Altman methods in studies of self-reported anthropometry.

  View details for DOI 10.1038/s41366-019-0499-5

  View details for PubMedID 31792334

• Risk factors for posttraumatic stress disorder: An umbrella review of systematic reviews and meta-analyses NEUROSCIENCE AND BIOBEHAVIORAL REVIEWS Tortella-Feliu, M., Fullana, M. A., Perez-Vigil, A., Torres, X., Chamorro, J., Littarelli, S. A., Solanes, A., Ramella-Cravaro, V., Vilar, A., Gonzalez-Parra, J. A., Andero, R., Reichenberg, A., Mataix-Cols, D., Vieta, E., Fusar-Poli, P., Ioannidis, J. A., Stein, M. B., Radua, J., Fernandez de la Cruz, L. 2019; 107: 154–65

  Abstract

  Approximately one third of individuals who experience a severe traumatic event will develop posttraumatic stress disorder (PTSD). It is crucial to identify what factors may be associated with increased or decreased risk for PTSD. We conducted an umbrella review of systematic reviews and meta-analyses of risk/protective factors for PTSD and assessed and graded the evidence of the association between each factor and PTSD. Thirty-three systematic reviews and meta-analyses were included and 130 potential risk factors were identified. Of those, 57 showed a significant association with PTSD. Being female or being indigenous people of the Americas, among the sociodemographic factors; history of physical disease and family history of psychiatric disorder, among the pretrauma factors; and cumulative exposure to potentially traumatic experiences, trauma severity, and being trapped during an earthquake, among the peritrauma factors, showed convincing or highly suggestive evidence of an association with PTSD. Data from prospective studies were less conclusive. Our results have the potential of helping refine PTSD prediction models and contributing to the design of prevention strategies.

  View details for DOI 10.1016/j.neubiorev.2019.09.013

  View details for Web of Science ID 000501388000015

  View details for PubMedID 31520677

• Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI). Journal of visualized experiments : JoVE Albajes-Eizagirre, A., Solanes, A., Fullana, M. A., Ioannidis, J. P., Fusar-Poli, P., Torrent, C., Solé, B., Bonnín, C. M., Vieta, E., Mataix-Cols, D., Radua, J. 2019

  Abstract

  Most methods for conducting meta-analysis of voxel-based neuroimaging studies do not assess whether effects are not null, but whether there is a convergence of peaks of statistical significance, and reduce the assessment of the evidence to a binary classification exclusively based on p-values (i.e., voxels can only be "statistically significant" or "non-statistically significant"). Here, we detail how to conduct a meta-analysis using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI), a novel method that uses a standard permutation test to assess whether effects are not null. We also show how to grade the strength of the evidence according to a set of criteria that considers a range of statistical significance levels (from more liberal to more conservative), the amount of data or the detection of potential biases (e.g., small-study effect and excess of significance). To exemplify the procedure, we detail the conduction of a meta-analysis of voxel-based morphometry studies in obsessive-compulsive disorder, and we provide all the data already extracted from the manuscripts to allow the reader to replicate the meta-analysis easily. SDM-PSI can also be used for meta-analyses of functional magnetic resonance imaging, diffusion tensor imaging, position emission tomography and surface-based morphometry studies.

  View details for DOI 10.3791/59841

  View details for PubMedID 31840658

• Most recommended medical interventions reach P<0.005 for their primary outcomes in meta-analyses. International journal of epidemiology Koletsi, D., Solmi, M., Pandis, N., Fleming, P. S., Correll, C. U., Ioannidis, J. P. 2019

  Abstract

  BACKGROUND: It has been proposed that the threshold of statistical significance should shift from P-value<0.05 to P-value<0.005, but there is concern that this move may dismiss effective, useful interventions. We aimed to assess how often medical interventions are recommended although their evidence in meta-analyses of randomized trials lies between P-value=0.05 and P-value=0.005.METHODS: We included Cochrane systematic reviews (SRs) published from 1 January 2013 to 30 June 2014 that had at least one meta-analysis with GRADE (Grading of Recommendations Assessment, Development and Evaluation) assessment and at least one primary outcome having favourable results for efficacy at P-value<0.05. Only comparisons of randomized trials between active versus no treatment/placebo were included. We then assessed the respective UpToDate recommendations for clinical practice from 22 May 2018 to 5 October 2018 and recorded how many treatments were recommended and what were the P-values in their meta-analysis evidence. The primary analysis was based on the first-listed outcomes.RESULTS: Of 608 screened SRs with GRADE assessment, 113 SRs were eligible, including 143 comparisons of which 128 comparisons had first-listed primary outcomes with UpToDate coverage. Altogether, 60% (58/97) of interventions with P-values<0.005 for their evidence were recommended versus 32% (10/31) of those with P-value 0.005-0.05. Therefore, most (58/68, 85.2%) of the recommended interventions had P-values<0.005 for the first-listed primary outcome. Of the 10 exceptions, 4 had other primary outcomes with P-values<0.005 and another 4 had additional extensive evidence for similar indications that would allow extrapolation for practice recommendations.CONCLUSIONS: Few interventions are recommended without their evidence from meta-analyses of randomized trials reaching P-value<0.005.

  View details for DOI 10.1093/ije/dyz241

  View details for PubMedID 31764988

• Prevalence and significance of race and ethnicity subgroup analyses in Cochrane intervention reviews. Clinical trials (London, England) Liu, P., Ross, J. S., Ioannidis, J. P., Dhruva, S. S., Vasiliou, V., Wallach, J. D. 2019: 1740774519887148

  View details for DOI 10.1177/1740774519887148

  View details for PubMedID 31709809

• Non-randomized studies using causal-modelling may give different answers than RCTs: a meta-epidemiological study. Journal of clinical epidemiology Ewald, H., Ioannidis, J. P., Ladanie, A., Cord, K. M., Bucher, H. C., Hemkens, L. G. 2019

  Abstract

  OBJECTIVES: To evaluate how estimated treatment effects agree between non-randomized studies using causal modelling with marginal structural models (MSM-studies) and randomized trials (RCTs).STUDY DESIGN: Meta-epidemiological study.SETTING: MSM-studies providing effect estimates on any healthcare outcome of any treatment were eligible. We systematically sought RCTs on the same clinical question and compared the direction of treatment effects, effect sizes, and confidence intervals.RESULTS: The main analysis included 19 MSM-studies (1039570 patients) and 141 RCTs (120669 patients). MSM-studies indicated effect estimates in the opposite direction from RCTs for 8 clinical questions (42%), and their 95% CI did not include the RCT estimate in 9 clinical questions (47%). The effect estimates deviated 1.58-fold between the study designs (median absolute deviation OR 1.58; IQR 1.37 to 2.16). Overall, we found no systematic disagreement regarding benefit or harm but confidence intervals were wide (summary ratio of odds ratios (sROR) 1.04; 95% CI 0.88 to 1.23). The subset of MSM-studies focusing on healthcare decision-making tended to overestimate experimental treatment benefits (sROR 1.44; 95% CI 0.99 to 2.09).CONCLUSION: Non-randomized studies using causal modelling with MSM may give different answers than RCTs. Caution is still required when non-randomized "real world" evidence is used for healthcare decisions.

  View details for DOI 10.1016/j.jclinepi.2019.10.012

  View details for PubMedID 31704350

• Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI) JOVE-JOURNAL OF VISUALIZED EXPERIMENTS Albajes-Eizagirre, A., Solanes, A., Angel Fullana, M., Ioannidis, J. A., Fusar-Poli, P., Torrent, C., Sole, B., Mar Bonnin, C., Vieta, E., Mataix-Cols, D., Radua, J. 2019

  View details for DOI 10.3791/59841

  View details for Web of Science ID 000500362600015

• Gene-environment interactions and colorectal cancer risk: An umbrella review of systematic reviews and meta-analyses of observational studies INTERNATIONAL JOURNAL OF CANCER Yang, T., Li, X., Montazeri, Z., Little, J., Farrington, S. M., Ioannidis, J. A., Dunlop, M. G., Campbell, H., Timofeeva, M., Theodoratou, E. 2019; 145 (9): 2315–29

  View details for DOI 10.1002/ijc.32057

  View details for Web of Science ID 000483585800001

• Age-treatment subgroup analyses in Cochrane intervention reviews: a meta-epidemiological study. BMC medicine Liu, P., Ioannidis, J. P., Ross, J. S., Dhruva, S. S., Luxkaranayagam, A. T., Vasiliou, V., Wallach, J. D. 2019; 17 (1): 188

  Abstract

  BACKGROUND: There is growing interest in evaluating differences in healthcare interventions across routinely collected demographic characteristics. However, individual subgroup analyses in randomized controlled trials are often not prespecified, adjusted for multiple testing, or conducted using the appropriate statistical test for interaction, and therefore frequently lack credibility. Meta-analyses can be used to examine the validity of potential subgroup differences by collating evidence across trials. Here, we characterize the conduct and clinical translation of age-treatment subgroup analyses in Cochrane reviews.METHODS: For a random sample of 928 Cochrane intervention reviews of randomized trials, we determined how often subgroup analyses of age are reported, how often these analyses have a P<0.05 from formal interaction testing, how frequently subgroup differences first observed in an individual trial are later corroborated by other trials in the same meta-analysis, and how often statistically significant results are included in commonly used clinical management resources (BMJ Best Practice, UpToDate, Cochrane Clinical Answers, Google Scholar, and Google search).RESULTS: Among 928 Cochrane intervention reviews, 189 (20.4%) included plans to conduct age-treatment subgroup analyses. The vast majority (162 of 189, 85.7%) of the planned analyses were not conducted, commonly because of insufficient trial data. There were 22 reviews that conducted their planned age-treatment subgroup analyses, and another 3 reviews appeared to perform unplanned age-treatment subgroup analyses. These 25 (25 of 928, 2.7%) reviews conducted a total of 97 age-treatment subgroup analyses, of which 65 analyses (in 20 reviews) had non-overlapping subgroup levels. Among the 65 age-treatment subgroup analyses, 14 (21.5%) did not report any formal interaction testing. Seven (10.8%) reported P<0.05 from formal age-treatment interaction testing; however, none of these seven analyses were in reviews that discussed the potential biological rationale or clinical significance of the subgroup findings or had results that were included in common clinical practice resources.CONCLUSION: Age-treatment subgroup analyses in Cochrane intervention reviews were frequently planned but rarely conducted, and implications of detected interactions were not discussed in the reviews or mentioned in common clinical resources. When subgroup analyses are performed, authors should report the findings, compare the results to previous studies, and outline any potential impact on clinical care.

  View details for DOI 10.1186/s12916-019-1420-8

  View details for PubMedID 31639007

• Neglecting Major Health Problems and Broadcasting Minor, Uncertain Issues in Lifestyle Science. JAMA Ioannidis, J. P. 2019: 1–2

  View details for DOI 10.1001/jama.2019.17576

  View details for PubMedID 31626274

• A new instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses Schandelmaier, S., Briel, M., Varadhan, R., Schmid, C. H., Devasenapathy, N., Hayward, R. A., Gagnier, J., Borenstein, M., van der Heijden, G., Dahabreh, I., Sun, X., Sauerbrei, W., Walsh, M., Ioannidis, J. A., Thabane, L., Guyatt, G. H. BMC. 2019

  View details for Web of Science ID 000491426300194

• Vibration of effects from diverse inclusion/exclusion criteria and analytical choices: 9216 different ways to perform an indirect comparison meta-analysis. BMC medicine Palpacuer, C., Hammas, K., Duprez, R., Laviolle, B., Ioannidis, J. P., Naudet, F. 2019; 17 (1): 174

  Abstract

  BACKGROUND: Different methodological choices such as inclusion/exclusion criteria and analytical models can yield different results and inferences when meta-analyses are performed. We explored the range of such differences, using several methodological choices for indirect comparison meta-analyses to compare nalmefene and naltrexone in the reduction of alcohol consumption as a case study.METHODS: All double-blind randomized controlled trials (RCTs) comparing nalmefene to naltrexone or one of these compounds to a placebo in the treatment of alcohol dependence or alcohol use disorders were considered. Two reviewers searched for published and unpublished studies in MEDLINE (August 2017), the Cochrane Library, Embase, and ClinicalTrials.gov and contacted pharmaceutical companies, the European Medicines Agency, and the Food and Drug Administration. The indirect comparison meta-analyses were performed according to different inclusion/exclusion criteria (based on medical condition, abstinence of patients before inclusion, gender, somatic and psychiatric comorbidity, psychological support, treatment administered and dose, treatment duration, outcome reported, publication status, and risk of bias) and different analytical models (fixed and random effects). The primary outcome was the vibration of effects (VoE), i.e. the range of different results of the indirect comparison between nalmefene and naltrexone. The presence of a "Janus effect" was investigated, i.e. whether the 1st and 99th percentiles in the distribution of effect sizes were in opposite directions.RESULTS: Nine nalmefene and 51 naltrexone RCTs were included. No study provided a direct comparison between the drugs. We performed 9216 meta-analyses for the indirect comparison with a median of 16 RCTs (interquartile range=12-21) included in each meta-analysis. The standardized effect size was negative at the 1st percentile (-0.29, favouring nalmefene) and positive at the 99th percentile (0.29, favouring naltrexone). A total of 7.1% (425/5961) of the meta-analyses with a negative effect size and 18.9% (616/3255) of those with a positive effect size were statistically significant (p<0.05).CONCLUSIONS: The choice of inclusion/exclusion criteria and analytical models for meta-analysis can result in entirely opposite results. VoE evaluations could be performed when overlapping meta-analyses on the same topic yield contradictory result.TRIAL REGISTRATION: This study was registered on October 19, 2016, in the Open Science Framework (OSF, protocol available at https://osf.io/7bq4y/ ).

  View details for DOI 10.1186/s12916-019-1409-3

  View details for PubMedID 31526369

• US Food and Drug Administration Approvals of Drugs and Devices Based on Nonrandomized Clinical Trials: A Systematic Review and Meta-analysis. JAMA network open Razavi, M., Glasziou, P., Klocksieben, F. A., Ioannidis, J. P., Chalmers, I., Djulbegovic, B. 2019; 2 (9): e1911111

  Abstract

  Importance: The size of estimated treatment effects on the basis of which the US Food and Drug Administration (FDA) has approved drugs and devices with data from nonrandomized clinical trials (non-RCTs) remains unknown.Objectives: To determine how often the FDA has authorized novel interventions based on non-RCTs and to assess whether there is an association of the magnitude of treatment effects with FDA requirements for additional testing in randomized clinical trials (RCTs).Data Sources: Overall, 606 drug applications for the Breakthrough Therapy designation from its inception in January 2012 were downloaded from the FDA website in January 2017 and August 2018, and 71 medical device applications for the Humanitarian Device Exemption from its inception in June 1996 were downloaded in August 2017.Study Selection: Approved applications based on non-RCTs were included; RCTs, studies with insufficient information, duplicates, and safety data were excluded.Data Extraction and Synthesis: Data were extracted by 2 independent investigators. A statistical association of the magnitude of estimated effect (expressed as an odds ratio) with FDA requests for RCTs was assessed. The data were also meta-analyzed to evaluate the differences in odds ratios between applications that required further testing and those that did not. The results are reported according to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.Main Outcomes and Measures: Disease, laboratory, and patient-related outcomes, including disease response or patient survival, were considered.Results: Among 677 drug and medical device applications, 68 (10.0%) were approved by the FDA based on non-RCTs. Estimates of effects were larger when no further RCTs were required (mean natural logarithm of the odds ratios, 2.18 vs 1.12; odds ratios, 8.85 vs 3.06; P=.03). The meta-analysis results confirmed these findings: estimated effects were approximately 2.5-fold higher for treatments or devices that were approved based on non-RCTs than for treatments or devices for which further testing in RCTs was required (6.30 [95% CI, 4.38-9.06] vs 2.46 [95% CI, 1.70-3.56]; P<.001). Overall, 9 of 677 total applications (1.3%) that were approved on the basis of non-RCTs had relative risks of 10 or greater and 12 (1.7%) had relative risks of 5 or greater. No clear threshold above which the FDA approved interventions based on the magnitude of estimated effect alone was detected.Conclusions and Relevance: In this study, estimated magnitudes of effect were larger among studies for which the FDA did not require RCTs compared with studies for which it did. There was no clear threshold of treatment effect above which no RCTs were requested.

  View details for DOI 10.1001/jamanetworkopen.2019.11111

  View details for PubMedID 31509209

• Toward a paradigm shift in treatment and research of mental disorders. Psychological medicine Leichsenring, F., Steinert, C., Ioannidis, J. P. 2019: 1–7

  View details for DOI 10.1017/S0033291719002265

  View details for PubMedID 31474241

• Non-inferiority versus superiority trial design for new antibiotics in an era of high antimicrobial resistance: the case for post-marketing, adaptive randomised controlled trials. The Lancet. Infectious diseases Lanini, S., Ioannidis, J. P., Vairo, F., Pletschette, M., Portella, G., Di Bari, V., Mammone, A., Pisapia, R., Merler, S., Nguhuni, B., Langer, M., Di Caro, A., Edwards, S. J., Petrosillo, N., Zumla, A., Ippolito, G. 2019

  Abstract

  Antimicrobial resistance is one of the most important threats to global health security. A range of Gram-negative bacteria associated with high morbidity and mortality are now resistant to almost all available antibiotics. In this context of urgency to develop novel drugs, new antibiotics for multidrug-resistant Gram-negative bacteria (namely, ceftazidime-avibactam, plazomicin, and meropenem-vaborbactam) have been approved by regulatory authorities based on non-inferiority trials that provided no direct evidence of their efficacy against multidrug-resistant bacteria such as Enterobacteriaceae spp, Pseudomonas aeruginosa, Stenotrophomonas maltophilia, Burkholderia cepacia, and Acinetobacter baumannii. The use of non-inferiority and superiority trials, and selection of appropriate and optimal study designs, remains a major challenge in the development, registration, and post-marketing implementation of new antibiotics. Using an example of the development process of ceftazidime-avibactam, we propose a strategy for a new research framework based on adaptive randomised clinical trials. The operational research strategy has the aim of assessing the efficacy of new antibiotics in special groups of patients, such as those infected with multidrug-resistant bacteria, who were not included in earlier phase studies, and for whom it is important to establish an appropriate standard of care.

  View details for DOI 10.1016/S1473-3099(19)30284-1

  View details for PubMedID 31451421

• Lost Evidence From Registered Large Long-Unpublished Randomized Controlled Trials: A Survey ANNALS OF INTERNAL MEDICINE Tatsioni, A., Karassa, F. B., Goodman, S. N., Zarin, D. A., Fanelli, D., Ioannidis, J. A. 2019; 171 (4): 300-+

  View details for DOI 10.7326/M19-0440

  View details for Web of Science ID 000481642800027

• Options for publishing research without any P-values. European heart journal Ioannidis, J. P. 2019; 40 (31): 2555–56

  View details for DOI 10.1093/eurheartj/ehz556

  View details for PubMedID 31411717

• Publishing research with P-values: Prescribe more stringent statistical significance or proscribe statistical significance? European heart journal Ioannidis, J. P. 2019; 40 (31): 2553–54

  View details for DOI 10.1093/eurheartj/ehz555

  View details for PubMedID 31411718

• Petitions in scientific argumentation: dissecting the request to retire statistical significance. European journal of clinical investigation Hardwicke, T. E., Ioannidis, J. P. 2019

  Abstract

  Petitions have a long history of being used for political, social, ethical, and injustice issues, however, it is unclear how/whether they should be implemented in scientific argumentation. Recently, an extremely influential commentary published in Nature (Amrhein et al., 2019) calling for the abandonment of "statistical significance" was signed by 854 scientists. We surveyed signatories and observed substantial heterogeneity in respondents' perceptions of the petition process, motivations for signing, and views on aspects of abandoning statistical significance. The top-cited signatories were strongly concentrated in a few scientific fields. In a random sample of 100 signatories, 62 published at least one paper in 2018 using statistical inference and most of them had used the phrase "statistical significance". When scientists sign petitions, they may have variable views on important aspects and it is useful to understand this diversity. This article is protected by copyright. All rights reserved.

  View details for DOI 10.1111/eci.13162

  View details for PubMedID 31380567

• Lethal news: The dexterous infiltration of news media by the tobacco industry agenda EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Ioannidis, J. A. 2019; 49 (7)

  View details for DOI 10.1111/eci.13125

  View details for Web of Science ID 000474478100010

• How does exercise treatment compare with antihypertensive medications? A network meta-analysis of 391 randomised controlled trials assessing exercise and medication effects on systolic blood pressure BRITISH JOURNAL OF SPORTS MEDICINE Naci, H., Salcher-Konrad, M., Dias, S., Blum, M. R., Sahoo, S., Nunan, D., Ioannidis, J. A. 2019; 53 (14): 859-+

  View details for DOI 10.1136/bjsports-2018-099921

  View details for Web of Science ID 000496277500009

• A user's guide to inflated and manipulated impact factors. European journal of clinical investigation Ioannidis, J. P., Thombs, B. D. 2019: e13151

  Abstract

  The journal impact factor (JIF)1 is without a doubt the most widely used, misused and abused bibliometric index in academic science. Journals are ranked within their field based on JIF, and JIF is seen as a reflection of the importance of a journal's publications. The contributions of individual scientists are also gauged based on the JIF of the journals where their work is published, and in academic settings funding and promotion decisions rely heavily on JIF. Not surprisingly, there is intense pressure on journal editors to game the system and increase their journals' JIF in ways that do not contribute to advancing science and that in many cases distort the scientific process. This article is protected by copyright. All rights reserved.

  View details for DOI 10.1111/eci.13151

  View details for PubMedID 31206647

• Solutions to Reduce Unnecessary Imaging-Reply. JAMA Oren, O., Kebebew, E., Ioannidis, J. P. 2019; 321 (22): 2243

  View details for DOI 10.1001/jama.2019.4021

  View details for PubMedID 31184737

• Risk and protective factors for anxiety and obsessive-compulsive disorders: an umbrella review of systematic reviews and meta-analyses. Psychological medicine Fullana, M. A., Tortella-Feliu, M., Fernandez de la Cruz, L., Chamorro, J., Perez-Vigil, A., Ioannidis, J. P., Solanes, A., Guardiola, M., Almodovar, C., Miranda-Olivos, R., Ramella-Cravaro, V., Vilar, A., Reichenberg, A., Mataix-Cols, D., Vieta, E., Fusar-Poli, P., Fatjo-Vilas, M., Radua, J. 2019: 1–16

  Abstract

  BACKGROUND: A multitude of risk/protective factors for anxiety and obsessive-compulsive disorders have been proposed. We conducted an umbrella review to summarize the evidence of the associations between risk/protective factors and each of the following disorders: specific phobia, social anxiety disorder, generalized anxiety disorder, panic disorder, and obsessive-compulsive disorder, and to assess the strength of this evidence whilst controlling for several biases.METHODS: Publication databases were searched for systematic reviews and meta-analyses examining associations between potential risk/protective factors and each of the disorders investigated. The evidence of the association between each factor and disorder was graded into convincing, highly suggestive, suggestive, weak, or non-significant according to a standardized classification based on: number of cases (>1000), random-effects p-values, 95% prediction intervals, confidence interval of the largest study, heterogeneity between studies, study effects, and excess of significance.RESULTS: Nineteen systematic reviews and meta-analyses were included, corresponding to 216 individual studies covering 427 potential risk/protective factors. Only one factor association (early physical trauma as a risk factor for social anxiety disorder, OR 2.59, 95% CI 2.17-3.1) met all the criteria for convincing evidence. When excluding the requirement for more than 1000 cases, five factor associations met the other criteria for convincing evidence and 22 met the remaining criteria for highly suggestive evidence.CONCLUSIONS: Although the amount and quality of the evidence for most risk/protective factors for anxiety and obsessive-compulsive disorders is limited, a number of factors significantly increase the risk for these disorders, may have potential prognostic ability and inform prevention.

  View details for DOI 10.1017/S0033291719001247

  View details for PubMedID 31172897

• Excess significance bias in repetitive transcranial magnetic stimulation literature for neuropsychiatric disorders Amad, A., Jardri, R., Rousseau, C., Larochelle, Y., Ioannidis, J. P., Naudet, F. WILEY. 2019: 12

  View details for Web of Science ID 000472935400026

• Author Reply to: The name of the game: Is preventive screening "cancer screening?" EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Adami, H., Kalager, M., Valdimarsdottir, U., Bretthauer, M., Ioannidis, J. A. 2019; 49 (6)

  View details for DOI 10.1111/eci.13097

  View details for Web of Science ID 000469356800002

• Meta-analyses identify differentially expressed micrornas in Parkinson's disease ANNALS OF NEUROLOGY Schulz, J., Takousis, P., Wohlers, I., Itua, M. G., Dobricic, V., Ruecker, G., Binder, H., Middleton, L., Loannidis, J. A., Perneczky, R., Bertram, L., Lill, C. M. 2019; 85 (6): 835–51

  View details for DOI 10.1002/ana.25490

  View details for Web of Science ID 000467860200006

• Vibration of effects and Janus phenomenon from diverse inclusion/exclusion criteria and analytical choices: 9216 different ways to perform an indirect comparison meta-analysis Palpacuer, C., Hammas, K., Duprez, R., Laviolle, B., Ioannidis, J., Naudet, F. WILEY. 2019: 9

  View details for Web of Science ID 000472935400017

• Validation protocols for blood pressure measuring devices: the impact of the European Society of Hypertension International Protocol and the development of a Universal Standard. Blood pressure monitoring O'Brien, E., Stergiou, G., Palatini, P., Asmar, R., Ioannidis, J. P., Kollias, A., Lacy, P., McManus, R. J., Myers, M. G., Shennan, A., Wang, J., Parati, G., European Society of Hypertension Working Group on Blood Pressure Monitoring 2019

  Abstract

  In the last three decades protocols for the validation of blood pressure measuring devices have been developed by the US Association for the Advancement of Medical Instrumentation, the British Hypertension Society, the German Hypertension League, the European Society of Hypertension Working Group on blood pressure Monitoring and the International Organization for Standardization. The European Society of Hypertension International Protocol required much smaller sample size than the other protocols, aiming to reduce the time, resources and cost of validation studies and thereby increase the number of validated devices. Given its specifications, the European Society of Hypertension International Protocol was adequate for 'high- and low-accuracy' devices, yet assessment of 'moderate accuracy' devices had high uncertainty with resultant high rate of device failure. Thus, devices validated using the European Society of Hypertension International Protocol should be considered to be as accurate as those validated with the previous Association for the Advancement of Medical Instrumentation or British Hypertension Society protocols. However, the European Society of Hypertension International Protocol did not allow subgroup evaluation (arm sizes, special populations, etc). The mission of the European Society of Hypertension International Protocol to promote the concept of validation has been well achieved, as almost double studies have been published using it than all the other protocols together. However, the maintenance of different validation protocols is confusing and therefore experts from the Association for the Advancement of Medical Instrumentation, European Society of Hypertension International Protocol and International Organization for Standardization have now developed the AAMI/ESH/ISO Universal Standard (ISO 81060-2:2018) as the recommended 21st-century procedure for worldwide application. The European Society of Hypertension Working Group has published a practical guide for using the Universal Standard. It is in the interests of all scientific bodies to propagate the Universal Standard and ensure its wide implementation.

  View details for DOI 10.1097/MBP.0000000000000391

  View details for PubMedID 31116156

• Lethal news: the dexterous infiltration of news media by the tobacco industry agenda. European journal of clinical investigation Ioannidis, J. P. 2019: e13125

  Abstract

  News media have an obligation to defend their readers' health from tobacco products, the most lethal public health danger. These media can be effective partners in the anti-tobacco campaign [1]. Unfortunately, news coverage of tobacco is disproportionately slim versus its huge disease burden [2], even if differences exist across countries [3] and across specific media outlets. Communications media must be mercilessly tough with the tobacco industry and its schemes. This article is protected by copyright. All rights reserved.

  View details for PubMedID 31058313

• Environmental risk factors and interventions for obesity EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Solmi, M., Ioannidis, J. A., Carvalho, A. F. 2019; 49 (5)

  View details for DOI 10.1111/eci.13080

  View details for Web of Science ID 000467982600003

• Signals Among Signals: Prioritizing Nongenetic Associations in Massive Data Sets AMERICAN JOURNAL OF EPIDEMIOLOGY Manrai, A. K., Ioannidis, J. A., Patel, C. J. 2019; 188 (5): 846–50

  View details for DOI 10.1093/aje/kwz031

  View details for Web of Science ID 000492993400007

• Reproducible pharmacokinetics. Journal of pharmacokinetics and pharmacodynamics Ioannidis, J. P. 2019

  Abstract

  Reproducibility is a highly desired feature of scientific investigation in general, and it has special connotations for research in pharmacokinetics, a vibrant field with over 500,000 publications to-date. It is important to be able to differentiate between genuine heterogeneity in pharmacokinetic parameters from heterogeneity that is due to errors and biases. This overview discusses efforts and opportunities to diminish the latter type of undesirable heterogeneity. Several reporting and research guidance documents and standards have been proposed for pharmacokinetic studies, but their adoption is still rather limited. Quality problems in the methods used and model evaluations have been examined in some empirical studies of the literature. Standardization of statistical and laboratory tools and procedures can be improved in the field. Only a small fraction of pharmacokinetic studies become pre-registered and only 9995 such studies have been registered in ClinicalTrials.gov as of August 2018. It is likely that most pharmacokinetic studies remain unpublished. Publication bias affecting the results and inferences has been documented in case studies, but its exact extent is unknown for the field at-large. The use of meta-analyses in the field is still limited. Availability of raw data, detailed protocols, software and codes is hopefully improving with multiple ongoing initiatives. Several research practices can contribute to greater transparency and reproducibility for pharmacokinetic investigations.

  View details for PubMedID 31004315

• Meta-analyses identify differentially expressed microRNAs in Parkinson's disease. Annals of neurology Schulz, J., Takousis, P., Wohlers, I., Itua, I. O., Dobricic, V., Rucker, G., Binder, H., Middleton, L., Ioannidis, J. P., Perneczky, R., Bertram, L., Lill, C. M. 2019

  Abstract

  OBJECTIVE: MicroRNA-mediated (dys)regulation of gene expression has been implicated in Parkinson's disease (PD), although results of microRNA expression studies remain inconclusive. We aimed to identify microRNAs that show consistent differential expression across all published expression studies in PD.METHODS: We performed a systematic literature search on microRNA expression studies in PD and extracted data from eligible publications. After stratification for brain, blood, and cerebrospinal fluid (CSF)-derived specimen we performed meta-analyses across microRNAs assessed in three or more independent datasets. Meta-analyses were performed using effect-size and p-value based methods, as applicable.RESULTS: After screening 599 publications we identified 47 datasets eligible for meta-analysis. On these, we performed 160 meta-analyses on microRNAs quantified in brain (n=125), blood (n=31), or CSF samples (n=4). Twenty-one meta-analyses were performed using effect sizes. We identified 13 significantly (Bonferroni-adjusted alpha=3.13x10-4 ) differentially expressed microRNAs in brain (n=3) and blood (n=10) with consistent effect directions across studies. The most compelling findings were with hsa-miR-132-3p (p=6.37x10-5 ), hsa-miR-497-5p (p=1.35x10-4 ), and hsa-miR-133b (p=1.90x10-4 ) in brain, and with hsa-miR-221-3p (p=4.49x10-35 ), hsa-miR-214-3p (p=2.00x10-34 ), and hsa-miR-29c-3p (p=3.00x10-12 ) in blood. No significant signals were found in CSF. Analyses of GWAS data for target genes of brain microRNAs showed significant association (alpha=9.40x10-5 ) of genetic variants in nine loci.INTERPRETATION: We identified several microRNAs that showed highly significant differential expression in PD. Future studies may assess the possible role of the identified brain miRNAs in pathogenesis and disease progression as well as the potential of the top blood microRNAs as biomarkers for diagnosis, progression or prediction of PD. This article is protected by copyright. All rights reserved.

  View details for PubMedID 30990912

• Unreformed nutritional epidemiology: a lamp post in the dark forest EUROPEAN JOURNAL OF EPIDEMIOLOGY Ioannidis, J. A. 2019; 34 (4): 327–31

  View details for DOI 10.1007/s10654-019-00487-5

  View details for Web of Science ID 000463671200002

• How often can meta-analyses of individual-level data individualize treatment? A meta-epidemiologic study INTERNATIONAL JOURNAL OF EPIDEMIOLOGY Schuit, E., Li, A. H., Ioannidis, J. A. 2019; 48 (2): 596–608

  View details for DOI 10.1093/ije/dyy239

  View details for Web of Science ID 000479285400035

• Hypnotic depth and postoperative death: a Bayesian perspective and an Independent Discussion of a clinical trial. British journal of anaesthesia Vlisides, P. E., Ioannidis, J. P., Avidan, M. S. 2019; 122 (4): 421–27

  View details for PubMedID 30857598

• Independent discussion sections for improving inferential reproducibility in published research. British journal of anaesthesia Avidan, M. S., Ioannidis, J. P., Mashour, G. A. 2019; 122 (4): 413–20

  Abstract

  There is a reproducibility crisis in science. There are many potential contributors to replication failure in research across the translational continuum. In this perspective piece, we focus on the narrow topic of inferential reproducibility. Although replication of methods and results is necessary to demonstrate reproducibility, it is not sufficient. Also fundamental is consistent interpretation in the Discussion section. Current deficiencies in the Discussion sections of manuscripts might limit the inferential reproducibility of scientific research. Lack of contextualisation using systematic reviews, overinterpretation and misinterpretation of results, and insufficient acknowledgement of limitations are common problems in Discussion sections; these deficiencies can harm the translational process. Proposed solutions include eliminating or not reading Discussions, writing accompanying editorials, and post-publication review and comments; however, none of these solutions works very well. A second Discussion written by an independent author with appropriate expertise in research methodology is a new testable solution that could help probe inferential reproducibility, and address some deficiencies in primary Discussion sections.

  View details for PubMedID 30857597

• Exploration, Inference, and Prediction in Neuroscience and Biomedicine TRENDS IN NEUROSCIENCES Bzdok, D., Ioannidis, J. A. 2019; 42 (4): 251–62

  View details for DOI 10.1016/j.tins.2019.02.001

  View details for Web of Science ID 000462476500004

• Hypnotic depth and postoperative death: a Bayesian perspective and an Independent Discussion of a clinical trial BRITISH JOURNAL OF ANAESTHESIA Vlisides, P. E., Ioannidis, J. A., Avidan, M. S. 2019; 122 (4): 421–27

  View details for DOI 10.1016/j.bja.2019.01.012

  View details for Web of Science ID 000460648300010

• Reproducible pharmacokinetics JOURNAL OF PHARMACOKINETICS AND PHARMACODYNAMICS Ioannidis, J. A. 2019; 46 (2): 111–16

  View details for DOI 10.1007/s10928-019-09621-y

  View details for Web of Science ID 000466490300003

• Stealth research: Lack of peer-reviewed evidence from healthcare unicorns EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Cristea, I. A., Cahan, E. M., Ioannidis, J. A. 2019; 49 (4)

  View details for DOI 10.1111/eci.13072

  View details for Web of Science ID 000462569100001

• A Comprehensive Analysis of Protocols for Deriving Dopaminergic Neurons from Human Pluripotent Stem Cells STEM CELLS TRANSLATIONAL MEDICINE Marton, R. M., Ioannidis, J. A. 2019; 8 (4): 366–74

  View details for DOI 10.1002/sctm.18-0088

  View details for Web of Science ID 000462158700008

• Independent discussion sections for improving inferential reproducibility in published research BRITISH JOURNAL OF ANAESTHESIA Avidan, M. S., Ioannidis, J. A., Mashour, G. A. 2019; 122 (4): 413–20

  View details for DOI 10.1016/j.bja.2018.12.010

  View details for Web of Science ID 000460648300009

• Bayes factors for superiority, non-inferiority, and equivalence designs BMC MEDICAL RESEARCH METHODOLOGY van Ravenzwaaij, D., Monden, R., Tendeiro, J. N., Ioannidis, J. A. 2019; 19

  View details for DOI 10.1186/s12874-019-0699-7

  View details for Web of Science ID 000462881500002

• Retiring significance: a free pass to bias NATURE Ioannidis, J. A. 2019; 567 (7749): 461

  View details for DOI 10.1038/d41586-019-00969-2

  View details for Web of Science ID 000462655800030

  View details for PubMedID 30903096

• Signals Among Signals: Prioritizing Non-genetic Associations in Massive Datasets. American journal of epidemiology Manrai, A. K., Ioannidis, J. P., Patel, C. J. 2019

  Abstract

  Massive datasets are often regarded as a panacea to the underpowered studies of the past. At the same time, it is becoming clear that in many of these datasets where thousands of variables are measured across hundreds of thousands or millions of individuals, almost any desired relationship can be inferred with a suitable combination of covariates or analytic choices. Inspired by the Genome-Wide Association Study (GWAS) analysis paradigm that has transformed human genetics, "X-Wide Association Studies" or "XWASs" have emerged as a popular approach to systematically analyze non-genetic datasets and guard against false positives. However, these studies often yield hundreds or thousands of associations characterized by modest effect sizes and miniscule p-values. Many of these associations will be spurious and emerge due to confounding and other biases. One way of characterizing confounding in the genomics paradigm is the genomic inflation factor. An analogous "X-Wide Inflation Factor," denoted lambdaX, can be defined and applied to published XWASs. Effects that arise in XWAS may be prioritized using replication, triangulation, quantification of measurement error, contextualization of each effect in the distribution of all effect sizes within a field, and pre-registration. Criteria like those of Bradford Hill need to be reconsidered in light of exposure-wide epidemiology to prioritize signals among signals.

  View details for PubMedID 30877292

• Author Reply to: The name of the game: Is preventive screening "cancer screening?" European journal of clinical investigation Adami, H., Kalager, M., Valdimarsdottir, U., Bretthauer, M., Ioannidis, J. P. 2019: e13097

  Abstract

  A clear distinction is justified between any screening intervention and primary prevention. In contrast to the former, the latter aims to reduce or eliminate exposure to any cause of cancer (such as smoking) without any knowledge whether any cell has yet undergone the early genetic changes that may lead to malignant transformation. We have no strong opinion about the semantic issue concerning the best terminology to describe population screening that aims to detect potential precursor lesions. This article is protected by copyright. All rights reserved.

  View details for PubMedID 30829396

• Recommendations and Practical Guidance for performing and reporting validation studies according to the Universal Standard for the validation of blood pressure measuring devices by the Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO). Journal of hypertension Stergiou, G. S., Palatini, P., Asmar, R., Ioannidis, J. P., Kollias, A., Lacy, P., McManus, R. J., Myers, M. G., Parati, G., Shennan, A., Wang, J., O'Brien, E., European Society of Hypertension Working Group on Blood Pressure Monitoring 2019; 37 (3): 459–66

  Abstract

  : In the past 30 years, several organizations have developed protocols for clinical validation of blood pressure measuring devices. An international initiative was recently launched by the US Association for the Advancement of Medical Instrumentation (AAMI), the European Society of Hypertension Working Group on Blood Pressure Monitoring (ESH) and the International Organization for Standardization (ISO), aiming to reach consensus on a universal AAMI/ESH/ISO validation standard. The purpose of this statement by the ESH Working Group on Blood Pressure Monitoring is to provide practical guidance for investigators performing validation studies according to the AAMI/ESH/ISO Universal Standard (ISO 81060-2:2018), to ensure that its stipulations are meticulously implemented and data are fully reported. Thus, this statement provides: a list of key recommendations for validation studies of intermittent non-invasive automated blood pressure measuring devices according to the AAMI/ESH/ISO Universal Standard; practical stepwise guidance for researchers performing these validation studies; a checklist for authors and reviewers of such studies; an example of a complete validation study report.

  View details for PubMedID 30702492

• Are all mental disorders related to all other medical diseases and vice versa? Journal of psychosomatic research Fiedorowicz, J. G., Ioannidis, J. P. 2019; 118: 71–72

  View details for PubMedID 30782358

• Marginal structural models and other analyses allow multiple estimates of treatment effects in randomized clinical trials: Meta-epidemiological analysis JOURNAL OF CLINICAL EPIDEMIOLOGY Ewald, H., Speich, B., Ladanie, A., Bucher, H. C., Ioannidis, J. A., Hemkens, L. G. 2019; 107: 12–26

  View details for DOI 10.1016/j.jclinepi.2018.11.001

  View details for Web of Science ID 000462696000003

• Cochrane crisis: Secrecy, intolerance and evidence-based values EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Ioannidis, J. A. 2019; 49 (3)

  View details for DOI 10.1111/eci.13058

  View details for Web of Science ID 000459627400001

• Time to abandon early detection cancer screening EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Adami, H., Kalager, M., Valdimarsdottir, U., Bretthauer, M., Ioannidis, J. A. 2019; 49 (3)

  View details for DOI 10.1111/eci.13062

  View details for Web of Science ID 000459627400008

• Are all mental disorders related to all other medical diseases and vice versa? JOURNAL OF PSYCHOSOMATIC RESEARCH Fiedorowicz, J. G., Ioannidis, J. A. 2019; 118: 71–72

  View details for DOI 10.1016/j.jpsychores.2019.01.018

  View details for Web of Science ID 000460190000013

• Recommendations and Practical Guidance for performing and reporting validation studies according to the Universal Standard for the validation of blood pressure measuring devices by the Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO) JOURNAL OF HYPERTENSION Stergiou, G. S., Palatini, P., Asmar, R., Ioannidis, J. P., Kollias, A., Lacy, P., McManus, R. J., Myers, M. G., Parati, G., Shennan, A., Wang, J., O'Brien, E., European Soc Hypertension Working 2019; 37 (3): 459–66

  View details for DOI 10.1097/HJH.0000000000002039

  View details for Web of Science ID 000467338000001

• Most UK scientists who publish extremely highly-cited papers do not secure funding from major public and charity funders: A descriptive analysis PLOS ONE Stavropoulou, C., Somai, M., Ioannidis, J. A. 2019; 14 (2)

  View details for DOI 10.1371/journal.pone.0211460

  View details for Web of Science ID 000459806400018

• Exploration, Inference, and Prediction in Neuroscience and Biomedicine. Trends in neurosciences Bzdok, D., Ioannidis, J. P. 2019

  Abstract

  Recent decades have seen dramatic progress in brain research. These advances were often buttressed by probing single variables to make circumscribed discoveries, typically through null hypothesis significance testing. New ways for generating massive data fueled tension between the traditional methodology that is used to infer statistically relevant effects in carefully chosen variables, and pattern-learning algorithms that are used to identify predictive signatures by searching through abundant information. In this article we detail the antagonistic philosophies behind two quantitative approaches: certifying robust effects in understandable variables, and evaluating how accurately a built model can forecast future outcomes. We discourage choosing analytical tools via categories such as 'statistics' or 'machine learning'. Instead, to establish reproducible knowledge about the brain, we advocate prioritizing tools in view of the core motivation of each quantitative analysis: aiming towards mechanistic insight or optimizing predictive accuracy.

  View details for PubMedID 30808574

• Hypothesis, analysis and synthesis, it's all Greek to me. eLife Iliopoulos, I., Ananiadou, S., Danchin, A., Ioannidis, J. P., Katsikis, P. D., Ouzounis, C. A., Promponas, V. J. 2019; 8

  Abstract

  The linguistic foundations of science and technology include many terms that have been borrowed from ancient languages. In the case of terms with origins in the Greek language, the modern meaning can often differ significantly from the original one. Here we use the PubMed database to demonstrate the prevalence of words of Greek origin in the language of modern science, and call for scientists to exercise care when coining new terms.

  View details for DOI 10.7554/eLife.43514

  View details for PubMedID 30782313

• Unreformed nutritional epidemiology: a lamp post in the dark forest. European journal of epidemiology Ioannidis, J. P. 2019

  View details for PubMedID 30746584

• PREDIMED trial of Mediterranean diet: retracted, republished, still trusted? BMJ (Clinical research ed.) Agarwal, A., Ioannidis, J. P. 2019; 364: l341

  View details for PubMedID 30733217

• Environmental risk factors and interventions for obesity. European journal of clinical investigation Solmi, M., Ioannidis, J. P., Carvalho, A. F. 2019: e13080

  Abstract

  We thank Cohen for her letter. There is some inconsistency in the used terminology for risk factors. Investigators with different interests use "environmental" with different connotations or different specificity thresholds. Our umbrella review assessed any non-genetic risk factors, in the broadest possible fashion. This included lifestyle, biobehavioral, sociodemograhic, mental health, and many other factors and it also included all the factors that Cohen uses the term "environmental" for in a very strict sense. Therefore, the statements of Cohen that our paper "does not, in any way, shape, or form, address or review environmental risk factors" is incorrect. This article is protected by copyright. All rights reserved.

  View details for PubMedID 30725492

• Flawed methods and inappropriate conclusions for health policy on overweight and obesity: the Global BMI Mortality Collaboration meta-analysis JOURNAL OF CACHEXIA SARCOPENIA AND MUSCLE Flegal, K. M., Ioannidis, J. A., Doehner, W. 2019; 10 (1): 9–13

  View details for DOI 10.1002/jcsm.12378

  View details for Web of Science ID 000462626100003

• Extremely large outlier treatment effects may be a footprint of bias in trials from less developed countries: randomized trials of gabapentinoids JOURNAL OF CLINICAL EPIDEMIOLOGY Desai, K., Carroll, I., Asch, S., Hernandez-Boussard, T., Ioannidis, J. A. 2019; 106: 80–87

  View details for DOI 10.1016/j.jclinepi.2018.10.012

  View details for Web of Science ID 000457818700010

• New clinical trial designs in the era of precision medicine: An overview of definitions, strengths, weaknesses, and current use in oncology CANCER TREATMENT REVIEWS Janiaud, P., Serghiou, S., Ioannidis, J. A. 2019; 73: 20–30

  View details for DOI 10.1016/j.ctrv.2018.12.003

  View details for Web of Science ID 000458711900003

• Stealth research: lack of peer-reviewed evidence from healthcare unicorns. European journal of clinical investigation Cristea, I. A., Cahan, E. M., Ioannidis, J. P. 2019: e13072

  Abstract

  In 2014, one of us (JPAI) wrote a viewpoint article coining the term "stealth research" for touted biomedical innovation happening outside the peer-reviewed literature in a confusing mix of "possibly brilliant ideas, aggressive corporate announcements, and mass media hype". These reflections were prompted by Theranos, a medical diagnosis start-up company; Theranos had not published any peer-reviewed papers [1] but made claims that its technology would "disrupt medicine." However, in contrast to the tech sector, in healthcare published peer-reviewed research is essential to ensure a minimum threshold of transparency, accountability, and credibility for the underlying work in the scientific community. This article is protected by copyright. All rights reserved.

  View details for PubMedID 30690709

• Toward unrestricted use of public genomic data. Science (New York, N.Y.) Amann, R. I., Baichoo, S., Blencowe, B. J., Bork, P., Borodovsky, M., Brooksbank, C., Chain, P. S., Colwell, R. R., Daffonchio, D. G., Danchin, A., de Lorenzo, V., Dorrestein, P. C., Finn, R. D., Fraser, C. M., Gilbert, J. A., Hallam, S. J., Hugenholtz, P., Ioannidis, J. P., Jansson, J. K., Kim, J. F., Klenk, H., Klotz, M. G., Knight, R., Konstantinidis, K. T., Kyrpides, N. C., Mason, C. E., McHardy, A. C., Meyer, F., Ouzounis, C. A., Patrinos, A. A., Podar, M., Pollard, K. S., Ravel, J., Munoz, A. R., Roberts, R. J., Rossello-Mora, R., Sansone, S., Schloss, P. D., Schriml, L. M., Setubal, J. C., Sorek, R., Stevens, R. L., Tiedje, J. M., Turjanski, A., Tyson, G. W., Ussery, D. W., Weinstock, G. M., White, O., Whitman, W. B., Xenarios, I. 2019; 363 (6425): 350–52

  View details for DOI 10.1126/science.aaw1280

  View details for PubMedID 30679363

• Curbing Unnecessary and Wasted Diagnostic Imaging JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Oren, O., Kebebew, E., Ioannidis, J. A. 2019; 321 (3): 245–46

  View details for DOI 10.1001/jama.2018.20295

  View details for Web of Science ID 000456347000012

• Reforming Nutritional Epidemiologic Research-Reply. JAMA Ioannidis, J. P. 2019; 321 (3): 310

  View details for PubMedID 30667499

• Flawed methods and inappropriate conclusions for health policy on overweight and obesity: the Global BMI Mortality Collaboration meta-analysis. Journal of cachexia, sarcopenia and muscle Flegal, K. M., Ioannidis, J. P., Doehner, W. 2019

  Abstract

  Guideline recommendations and health policy decisions rely on evidence from clinical and epidemiological studies. Adequate methodology and appropriate conclusions are essential to support healthcare and health policy decisions. An analysis of body mass index and mortality by the Global BMI Mortality Collaboration (GBMC) concluded that the association of excess body weight with higher mortality was similar worldwide and that overweight and obesity should be combated everywhere. To reach this conclusion, the GBMC used highly selected data, rather than a systematic approach. The GBMC initially chose individual participant data from 239 prospective studies with approximately 10.6 million participants. The GBMC then excluded over 60% of data and over 75% of fatal events by eliminating all cases with any reported disease at baseline or smoking history and all events within the first 5years of follow-up. After applying these restrictions, the association of overweight with lower mortality was reversed and the association of obesity with higher mortality was increased. Given the major flaws in the selection process, in the adequacy of the data, in the data analysis, and in the interpretation, the GBMC conclusions should be viewed sceptically as a guide to action, either for clinical decisions or for public health in general. The flawed conclusion that overweight is uniformly associated with substantially increased risk of death and thus should be combated in any circumstances may lead not only to unjustified treatment efforts and potential harm in a wide range of clinical conditions but also to a tremendous waste of resources.

  View details for PubMedID 30656860

• Limitations and Misinterpretations of E-Values for Sensitivity Analyses of Observational Studies ANNALS OF INTERNAL MEDICINE Ioannidis, J. A., Tan, Y., Blum, M. R. 2019; 170 (2): 108-+

  View details for DOI 10.7326/M18-2159

  View details for Web of Science ID 000455659100015

• Curbing Unnecessary and Wasted Diagnostic Imaging. JAMA Oren, O., Kebebew, E., Ioannidis, J. P. 2019

  View details for PubMedID 30615023

• Infographic. How does exercise treatment compare with antihypertensive medications? British journal of sports medicine Castillo-Garcia, A. n., Naci, H. n., Valenzuela, P. L., Salcher-Konrad, M. n., Dias, S. n., Blum, M. R., Sahoo, S. A., Nunan, D. n., Morales, J. S., Lucia, A. n., Ioannidis, J. P. 2019

  View details for DOI 10.1136/bjsports-2019-101522

  View details for PubMedID 31857338

• Comparison of major depression diagnostic classification probability using the SCID, CIDI, and MINI diagnostic interviews among women in pregnancy or postpartum: An individual participant data meta-analysis. International journal of methods in psychiatric research Levis, B. n., McMillan, D. n., Sun, Y. n., He, C. n., Rice, D. B., Krishnan, A. n., Wu, Y. n., Azar, M. n., Sanchez, T. A., Chiovitti, M. J., Bhandari, P. M., Neupane, D. n., Saadat, N. n., Riehm, K. E., Imran, M. n., Boruff, J. T., Cuijpers, P. n., Gilbody, S. n., Ioannidis, J. P., Kloda, L. A., Patten, S. B., Shrier, I. n., Ziegelstein, R. C., Comeau, L. n., Mitchell, N. D., Tonelli, M. n., Vigod, S. N., Aceti, F. n., Alvarado, R. n., Alvarado-Esquivel, C. n., Bakare, M. O., Barnes, J. n., Beck, C. T., Bindt, C. n., Boyce, P. M., Bunevicius, A. n., Couto, T. C., Chaudron, L. H., Correa, H. n., de Figueiredo, F. P., Eapen, V. n., Fernandes, M. n., Figueiredo, B. n., Fisher, J. R., Garcia-Esteve, L. n., Giardinelli, L. n., Helle, N. n., Howard, L. M., Khalifa, D. S., Kohlhoff, J. n., Kusminskas, L. n., Kozinszky, Z. n., Lelli, L. n., Leonardou, A. A., Lewis, B. A., Maes, M. n., Meuti, V. n., Nakić Radoš, S. n., Navarro García, P. n., Nishi, D. n., Okitundu Luwa E-Andjafono, D. n., Robertson-Blackmore, E. n., Rochat, T. J., Rowe, H. J., Siu, B. W., Skalkidou, A. n., Stein, A. n., Stewart, R. C., Su, K. P., Sundström-Poromaa, I. n., Tadinac, M. n., Tandon, S. D., Tendais, I. n., Thiagayson, P. n., Töreki, A. n., Torres-Giménez, A. n., Tran, T. D., Trevillion, K. n., Turner, K. n., Vega-Dienstmaier, J. M., Wynter, K. n., Yonkers, K. A., Benedetti, A. n., Thombs, B. D. 2019: e1803

  Abstract

  A previous individual participant data meta-analysis (IPDMA) identified differences in major depression classification rates between different diagnostic interviews, controlling for depressive symptoms on the basis of the Patient Health Questionnaire-9. We aimed to determine whether similar results would be seen in a different population, using studies that administered the Edinburgh Postnatal Depression Scale (EPDS) in pregnancy or postpartum.Data accrued for an EPDS diagnostic accuracy IPDMA were analysed. Binomial generalised linear mixed models were fit to compare depression classification odds for the Mini International Neuropsychiatric Interview (MINI), Composite International Diagnostic Interview (CIDI), and Structured Clinical Interview for DSM (SCID), controlling for EPDS scores and participant characteristics.Among fully structured interviews, the MINI (15 studies, 2,532 participants, 342 major depression cases) classified depression more often than the CIDI (3 studies, 2,948 participants, 194 major depression cases; adjusted odds ratio [aOR] = 3.72, 95% confidence interval [CI] [1.21, 11.43]). Compared with the semistructured SCID (28 studies, 7,403 participants, 1,027 major depression cases), odds with the CIDI (interaction aOR = 0.88, 95% CI [0.85, 0.92]) and MINI (interaction aOR = 0.95, 95% CI [0.92, 0.99]) increased less as EPDS scores increased.Different interviews may not classify major depression equivalently.

  View details for DOI 10.1002/mpr.1803

  View details for PubMedID 31568624

• Consideration of confounding was suboptimal in the reporting of observational studies in psychiatry: a meta-epidemiological study. Journal of clinical epidemiology Munkholm, K. n., Faurholt-Jepsen, M. n., Ioannidis, J. P., Hemkens, L. G. 2019

  Abstract

  When reporting observational studies, authors should explicitly discuss the potential for confounding and other biases but it is unclear to what extent this is done within the psychiatric field.We reviewed a random sample of 120 articles in the five psychiatric specialty journals with the highest 5-year impact factor in 2015-2018. We evaluated how confounding and bias was considered in the reporting of the Discussion and Abstract and assessed the relationship with yearly citations.The term "confounding" was explicitly mentioned in the Abstract or Discussion in 66 articles (55.0%; 95% confidence interval (CI): 46.1-63.6) and the term "bias" in 68 articles (56.7%; 95% CI: 47.7-65.2). The authors of 25 articles (20.8%; 95% CI: 14.5-28.9) acknowledged unadjusted confounders. With one exception (0.8%, 95% CI: 0.0-4.6), authors never expressed any caution, limitation or uncertainty in relation to confounding or other bias in their conclusions or in the Abstract. Articles acknowledging non-adjusted confounders were not less frequently cited than articles that did not (median 7.9 vs. 5.6 citations per year, P = 0.03).Confounding is overall inadequately addressed in the reporting and bias is often ignored in the interpretation of high-impact observational research in psychiatry.

  View details for DOI 10.1016/j.jclinepi.2019.12.002

  View details for PubMedID 31809848

• Probability of major depression diagnostic classification based on the SCID, CIDI and MINI diagnostic interviews controlling for Hospital Anxiety and Depression Scale - Depression subscale scores: An individual participant data meta-analysis of 73 primary studies. Journal of psychosomatic research Wu, Y. n., Levis, B. n., Sun, Y. n., Krishnan, A. n., He, C. n., Riehm, K. E., Rice, D. B., Azar, M. n., Yan, X. W., Neupane, D. n., Bhandari, P. M., Imran, M. n., Chiovitti, M. J., Saadat, N. n., Boruff, J. T., Cuijpers, P. n., Gilbody, S. n., McMillan, D. n., Ioannidis, J. P., Kloda, L. A., Patten, S. B., Shrier, I. n., Ziegelstein, R. C., Henry, M. n., Ismail, Z. n., Loiselle, C. G., Mitchell, N. D., Tonelli, M. n., Al-Adawi, S. n., Beraldi, A. n., Braeken, A. P., Büel-Drabe, N. n., Bunevicius, A. n., Carter, G. n., Chen, C. K., Cheung, G. n., Clover, K. n., Conroy, R. M., Cukor, D. n., da Rocha E Silva, C. E., Dabscheck, E. n., Daray, F. M., Douven, E. n., Downing, M. G., Feinstein, A. n., Ferentinos, P. P., Fischer, F. H., Flint, A. J., Fujimori, M. n., Gallagher, P. n., Gandy, M. n., Goebel, S. n., Grassi, L. n., Härter, M. n., Jenewein, J. n., Jetté, N. n., Julião, M. n., Kim, J. M., Kim, S. W., Kjærgaard, M. n., Köhler, S. n., Loosman, W. L., Löwe, B. n., Martin-Santos, R. n., Massardo, L. n., Matsuoka, Y. n., Mehnert, A. n., Michopoulos, I. n., Misery, L. n., Navines, R. n., O'Donnell, M. L., Öztürk, A. n., Peceliuniene, J. n., Pintor, L. n., Ponsford, J. L., Quinn, T. J., Reme, S. E., Reuter, K. n., Rooney, A. G., Sánchez-González, R. n., Schwarzbold, M. L., Senturk Cankorur, V. n., Shaaban, J. n., Sharpe, L. n., Sharpe, M. n., Simard, S. n., Singer, S. n., Stafford, L. n., Stone, J. n., Sultan, S. n., Teixeira, A. L., Tiringer, I. n., Turner, A. n., Walker, J. n., Walterfang, M. n., Wang, L. J., White, J. n., Wong, D. K., Benedetti, A. n., Thombs, B. D. 2019; 129: 109892

  Abstract

  Two previous individual participant data meta-analyses (IPDMAs) found that different diagnostic interviews classify different proportions of people as having major depression overall or by symptom levels. We compared the odds of major depression classification across diagnostic interviews among studies that administered the Depression subscale of the Hospital Anxiety and Depression Scale (HADS-D).Data accrued for an IPDMA on HADS-D diagnostic accuracy were analysed. We fit binomial generalized linear mixed models to compare odds of major depression classification for the Structured Clinical Interview for DSM (SCID), Composite International Diagnostic Interview (CIDI), and Mini International Neuropsychiatric Interview (MINI), controlling for HADS-D scores and participant characteristics with and without an interaction term between interview and HADS-D scores.There were 15,856 participants (1942 [12%] with major depression) from 73 studies, including 15,335 (97%) non-psychiatric medical patients, 164 (1%) partners of medical patients, and 357 (2%) healthy adults. The MINI (27 studies, 7345 participants, 1066 major depression cases) classified participants as having major depression more often than the CIDI (10 studies, 3023 participants, 269 cases) (adjusted odds ratio [aOR] = 1.70 (0.84, 3.43)) and the semi-structured SCID (36 studies, 5488 participants, 607 cases) (aOR = 1.52 (1.01, 2.30)). The odds ratio for major depression classification with the CIDI was less likely to increase as HADS-D scores increased than for the SCID (interaction aOR = 0.92 (0.88, 0.96)).Compared to the SCID, the MINI may diagnose more participants as having major depression, and the CIDI may be less responsive to symptom severity.

  View details for DOI 10.1016/j.jpsychores.2019.109892

  View details for PubMedID 31911325

• Air pollution as cause of mental disease: Appraisal of the evidence. PLoS biology Ioannidis, J. P. 2019; 17 (8): e3000370

  Abstract

  A causal association of air pollution with mental diseases is an intriguing possibility raised in a Short Report just published in PLOS Biology. Despite analyses involving large data sets, the available evidence has substantial shortcomings, and a long series of potential biases may invalidate the observed associations. Only bipolar disorder shows consistent results, with similar effects across United States and Denmark data sets, but the effect has modest magnitude, appropriate temporality is not fully secured, and biological gradient, plausibility, coherence, and analogy offer weak support. The signal seems to persist in some robustness analyses, but more analyses by multiple investigators, including contrarians, are necessary. Broader public sharing of data sets would also enhance transparency.

  View details for DOI 10.1371/journal.pbio.3000370

  View details for PubMedID 31430279

• The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement. Annals of internal medicine Kent, D. M., Paulus, J. K., van Klaveren, D. n., D'Agostino, R. n., Goodman, S. n., Hayward, R. n., Ioannidis, J. P., Patrick-Lake, B. n., Morton, S. n., Pencina, M. n., Raman, G. n., Ross, J. S., Selker, H. P., Varadhan, R. n., Vickers, A. n., Wong, J. B., Steyerberg, E. W. 2019

  Abstract

  Heterogeneity of treatment effect (HTE) refers to the nonrandom variation in the magnitude or direction of a treatment effect across levels of a covariate, as measured on a selected scale, against a clinical outcome. In randomized controlled trials (RCTs), HTE is typically examined through a subgroup analysis that contrasts effects in groups of patients defined "1 variable at a time" (for example, male vs. female or old vs. young). The authors of this statement present guidance on an alternative approach to HTE analysis, "predictive HTE analysis." The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risks with versus without the intervention, taking into account all relevant patient attributes simultaneously. The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed using a multidisciplinary technical expert panel, targeted literature reviews, simulations to characterize potential problems with predictive approaches, and a deliberative process engaging the expert panel. The authors distinguish 2 categories of predictive HTE approaches: a "risk-modeling" approach, wherein a multivariable model predicts the risk for an outcome and is applied to disaggregate patients within RCTs to define risk-based variation in benefit, and an "effect-modeling" approach, wherein a model is developed on RCT data by incorporating a term for treatment assignment and interactions between treatment and baseline covariates. Both approaches can be used to predict differential absolute treatment effects, the most relevant scale for clinical decision making. The authors developed 4 sets of guidance: criteria to determine when risk-modeling approaches are likely to identify clinically important HTE, methodological aspects of risk-modeling methods, considerations for translation to clinical practice, and considerations and caveats in the use of effect-modeling approaches. The PATH Statement, together with its explanation and elaboration document, may guide future analyses and reporting of RCTs.

  View details for DOI 10.7326/M18-3667

  View details for PubMedID 31711134

• The Ninth International Congress on Peer Review and Scientific Publication: A Call for Research. JAMA Ioannidis, J. P., Berkwits, M. n., Flanagin, A. n., Godlee, F. n., Bloom, T. n. 2019

  View details for DOI 10.1001/jama.2019.15516

  View details for PubMedID 31524942

• An empirical assessment of research practices across 163 clinical trials of tumor-bearing companion dogs. Scientific reports Tan, Y. J., Crowley, R. J., Ioannidis, J. P. 2019; 9 (1): 11877

  Abstract

  Comparative clinical trials of domestic dogs with spontaneously-occurring cancers are increasingly common. Canine cancers are likely more representative of human cancers than induced murine tumors. These trials could bridge murine models and human trials and better prioritize drug candidates. Such investigations also benefit veterinary patients. We aimed to evaluate the design and reporting practices of clinical trials containing ≥2 arms and involving tumor-bearing dogs. 163 trials containing 8552 animals were systematically retrieved from PubMed (searched 1/18/18). Data extracted included sample sizes, response criteria, study design, and outcome reporting. Low sample sizes were prevalent (median n = 33). The median detectable hazard ratio was 0.3 for overall survival and 0.06 for disease progression. Progressive disease thresholds for studies that did not adopt VCOG-RECIST guidelines varied in stringency. Additionally, there was significant underreporting across all Cochrane risk of bias categories. The proportion of studies with unclear reporting ranged from 44% (randomization) to 94% (selective reporting). 72% of studies also failed to define a primary outcome. The present study confirms previous findings that clinical trials in dogs need to be improved, particularly regarding low statistical power and underreporting of design and outcomes.

  View details for DOI 10.1038/s41598-019-48425-5

  View details for PubMedID 31417164

• 'Stealth' corporate innovation: an emerging threat for therapeutic drug development. Nature immunology Mastellos, D. C., Blom, A. M., Connolly, E. S., Daha, M. R., Geisbrecht, B. V., Ghebrehiwet, B. n., Gros, P. n., Hajishengallis, G. n., Holers, V. M., Huber-Lang, M. n., Kinoshita, T. n., Mollnes, T. E., Montgomery, R. A., Morgan, B. P., Nilsson, B. n., Pio, R. n., Ricklin, D. n., Risitano, A. M., Taylor, R. P., Mantovani, A. n., Ioannidis, J. P., Lambris, J. D. 2019

  View details for DOI 10.1038/s41590-019-0503-1

  View details for PubMedID 31562490

• Ninth international congress on peer review and scientific publication: call for research. BMJ (Clinical research ed.) Ioannidis, J. P., Berkwits, M., Flanagin, A., Godlee, F., Bloom, T. 2019; 366: l5475

  View details for DOI 10.1136/bmj.l5475

  View details for PubMedID 31527134

• Limitations and Misinterpretations of E-Values for Sensitivity Analyses of Observational Studies. Annals of internal medicine Ioannidis, J. P., Tan, Y. J., Blum, M. R. 2019

  Abstract

  The E-value was recently introduced on the basis of earlier work as "the minimum strength of associationthat an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates." E-values have been proposed for wide application in observational studies evaluating causality. However, they have limitations and are prone to misinterpretation. E-values have a monotonic, almost linear relationship with effect estimates and thus offer no additional information beyond what effect estimates can convey. Whereas effect estimates are based on real data, E-values may make unrealistic assumptions. No general rule can exist about what is a "small enough" E-value, and users of the biomedical literature are not familiar with how to interpret a range of E-values. Problems arise for any measure dependent on effect estimates and their CIs-for example, bias due to selective reporting and dependence on choice of exposure contrast and level of confidence. The automation of E-values may give an excuse not to think seriously about confounding. Moreover, biases other than confounding may still undermine results. Instead of misused or misinterpreted E-values, the authors recommend judicious use of existing methods for sensitivity analyses with careful assumptions; systematic assessments of whether and how known confounders have been handled, along with consideration of their prevalence and magnitude; thorough discussion of the potential for unknown confounders considering the study design and field of application; and explicit caution in making causal claims from observational studies.

  View details for PubMedID 30597486

• PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies ANNALS OF INTERNAL MEDICINE Wolff, R. F., Moons, K. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., Altman, D., Bossuyt, P., Cook, N. R., D'Amico, G., Debray, T. A., Deeks, J., de Groot, J., di Angelantonio, E., Fahey, T., Harrell, F., Hayden, J. A., Heymans, M. W., Hooft, L., Hyde, C., Ioannidis, J., Iorio, A., Kaptoge, S., Knottnerus, A., Leeflang, M., Nixon, F., Perel, P., Phillips, B., Raatz, H., Riemsma, R., Rovers, M., Rutjes, A. S., Sauerbrei, W., Sauerland, S., Scheibler, F., Scholten, R., Schuit, E., Steyerberg, E., Tan, T., ter Riet, G., van der Windt, D., Vergouwe, Y., Vickers, A., Wood, A. M., PROBAST Grp, PROBAST Steering Grp, PROBAST Delphi Grp 2019; 170 (1): 51-+

  View details for DOI 10.7326/M18-1376

  View details for Web of Science ID 000454685300011

• Therapy and prevention for mental health: What if mental diseases are mostly not brain disorders? The Behavioral and brain sciences Ioannidis, J. P. 2019; 42: e13

  Abstract

  Neurobiology-based interventions for mental diseases and searches for useful biomarkers of treatment response have largely failed. Clinical trials should assess interventions related to environmental and social stressors, with long-term follow-up; social rather than biological endpoints; personalized outcomes; and suitable cluster, adaptive, and n-of-1 designs. Labor, education, financial, and other social/political decisions should be evaluated for their impacts on mental disease.

  View details for PubMedID 30940221

• Precision medicine for individual patients should use population group averages and larger, not smaller, groups EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Djulbegovic, B., Ioannidis, J. A. 2019; 49 (1): e13031

  View details for PubMedID 30251305

• A systematic review of the genetic mechanisms of dolutegravir resistance. The Journal of antimicrobial chemotherapy Rhee, S. Y., Grant, P. M., Tzou, P. L., Barrow, G. n., Harrigan, P. R., Ioannidis, J. P., Shafer, R. W. 2019

  Abstract

  Characterizing the mutations selected by the integrase strand transfer inhibitor (INSTI) dolutegravir and their effects on susceptibility is essential for identifying viruses less likely to respond to dolutegravir therapy and for monitoring persons with virological failure (VF) on dolutegravir therapy.We systematically reviewed dolutegravir resistance studies to identify mutations emerging under dolutegravir selection pressure, the effect of INSTI resistance mutations on in vitro dolutegravir susceptibility, and the virological efficacy of dolutegravir in antiretroviral-experienced persons.We analysed 14 studies describing 84 in vitro passage experiments, 26 studies describing 63 persons developing VF plus INSTI resistance mutations on a dolutegravir-containing regimen, 41 studies describing dolutegravir susceptibility results, and 22 clinical trials and 16 cohort studies of dolutegravir-containing regimens. The most common INSTI resistance mutations in persons with VF on a dolutegravir-containing regimen were R263K, G118R, N155H and Q148H/R, with R263K and G118R predominating in previously INSTI-naive persons. R263K reduced dolutegravir susceptibility ∼2-fold. G118R generally reduced dolutegravir susceptibility >5-fold. The highest levels of reduced susceptibility occurred in viruses containing Q148 mutations in combination with G140 and/or E138 mutations. Dolutegravir two-drug regimens were highly effective for first-line therapy and for virologically suppressed persons provided dolutegravir's companion drug was fully active. Dolutegravir three-drug regimens were highly effective for salvage therapy in INSTI-naive persons provided one or more of dolutegravir's companion drugs was fully active. However, dolutegravir monotherapy in virologically suppressed persons and functional dolutegravir monotherapy in persons with active viral replication were associated with a non-trivial risk of VF plus INSTI resistance mutations.

  View details for DOI 10.1093/jac/dkz256

  View details for PubMedID 31280314

• What Have We (Not) Learnt from Millions of Scientific Papers with P Values? AMERICAN STATISTICIAN Ioannidis, J. A. 2019; 73: 20–25

  View details for DOI 10.1080/00031305.2018.1447512

  View details for Web of Science ID 000462083800002

• Lost Evidence From Registered Large Long-Unpublished Randomized Controlled Trials: A Survey. Annals of internal medicine Tatsioni, A. n., Karassa, F. B., Goodman, S. N., Zarin, D. A., Fanelli, D. n., Ioannidis, J. P. 2019

  View details for PubMedID 31060046

• Shortening self-report mental health symptom measures through optimal test assembly methods: Development and validation of the Patient Health Questionnaire-Depression-4 DEPRESSION AND ANXIETY Ishihara, M., Harel, D., Levis, B., Levis, A. W., Riehm, K. E., Saadat, N., Azar, M., Rice, D. B., Sanchez, T. A., Chiovitti, M. J., Cuijpers, P., Gilbody, S., Ioannidis, J. A., Kloda, L. A., McMillan, D., Patten, S. B., Shrier, I., Arroll, B., Bombardier, C. H., Butterworth, P., Carter, G., Clover, K., Conwell, Y., Goodyear-Smith, F., Greeno, C. G., Hambridge, J., Harrison, P. A., Hudson, M., Jette, N., Kiely, K. M., McGuire, A., Pence, B. W., Rooney, A. G., Sidebottom, A., Simning, A., Turner, A., White, J., Whooley, M. A., Winkley, K., Benedetti, A., Thombs, B. D. 2019; 36 (1): 82–92

  Abstract

  The objective of this study was to develop and validate a short form of the Patient Health Questionnaire-9 (PHQ-9), a self-report questionnaire for assessing depressive symptomatology, using objective criteria.Responses on the PHQ-9 were obtained from 7,850 English-speaking participants enrolled in 20 primary diagnostic test accuracy studies. PHQ unidimensionality was verified using confirmatory factor analysis, and an item response theory model was fit. Optimal test assembly (OTA) methods identified a maximally precise short form for each possible length between one and eight items, including and excluding the ninth item. The final short form was selected based on prespecified validity, reliability, and diagnostic accuracy criteria.A four-item short form of the PHQ (PHQ-Dep-4) was selected. The PHQ-Dep-4 had a Cronbach's alpha of 0.805. Sensitivity and specificity of the PHQ-Dep-4 were 0.788 and 0.837, respectively, and were statistically equivalent to the PHQ-9 (sensitivity = 0.761, specificity = 0.866). The correlation of total scores with the full PHQ-9 was high (r = 0.919).The PHQ-Dep-4 is a valid short form with minimal loss of information of scores when compared to the full-length PHQ-9. Although OTA methods have been used to shorten patient-reported outcome measures based on objective, prespecified criteria, further studies are required to validate this general procedure for broader use in health research. Furthermore, due to unexamined heterogeneity, there is a need to replicate the results of this study in different patient populations.

  View details for PubMedID 30238571

• Family History-Wide Association Study ("FamWAS") for Identifying Clinical and Environmental Risk Factors for Common Chronic Diseases. American journal of epidemiology Rasooly, D. n., Ioannidis, J. P., Khoury, M. J., Patel, C. J. 2019

  Abstract

  Family history is a strong risk factor for many common chronic diseases and summarizes shared environmental and genetic risk, but how this increased risk is mediated is unknown. We developed a "Family History-Wide Association Study" (FamWAS) to systematically and comprehensively test Clinical and Environmental Quantitative Traits (CEQTs) for their association with family history of disease. We implemented our method on 457 CEQTs for association with family history of diabetes, asthma, and coronary heart disease (CHD) in 42,940 adults spanning 8 waves of the 1999-2014 National Health and Nutrition Examination Survey (NHANES). We conducted pooled analyses of the 8 survey waves and analyzed trait associations using survey-weighted logistic regression. We identified 172 (37.6% of total), 32 (7.0%), and 78 (17.1%) CEQTs associated with family history of diabetes, asthma, and CHD, respectively, in sub-cohorts of individuals without the respective disease. 20 associated CEQTs were shared across family history of diabetes, asthma, and CHD, far more than expected by chance. FamWAS can examine traits not previously studied in association with family history and uncover trait overlap, highlighting a putative shared mechanism by which family history influences disease risk.

  View details for DOI 10.1093/aje/kwz125

  View details for PubMedID 31172187

• Interventions to improve cardiopulmonary resuscitation: a review of meta-analyses and future agenda. Critical care (London, England) Chalkias, A. n., Ioannidis, J. P. 2019; 23 (1): 210

  View details for DOI 10.1186/s13054-019-2495-5

  View details for PubMedID 31174581

• Excess Significance Bias in Repetitive Transcranial Magnetic Stimulation Literature for Neuropsychiatric Disorders. Psychotherapy and psychosomatics Amad, A. n., Jardri, R. n., Rousseau, C. n., Larochelle, Y. n., Ioannidis, J. P., Naudet, F. n. 2019: 1–8

  Abstract

  Repetitive transcranial magnetic stimulation (rTMS) has been widely tested and promoted for use in multiple neuropsychiatric conditions, but as for many other medical devices, some gaps may exist in the literature and the evidence base for the clinical efficacy of rTMS remains under debate.We aimed to test for an excess number of statistically significant results in the literature on the therapeutic efficacy of rTMS across a wide range of meta-analyses and to characterize the power of studies included in these meta-analyses.Based on power calculations, we computed the expected number of "positive" datasets for a medium effect size (standardized mean difference, SMD = 0.30) and compared it with the number of observed "positive" datasets. Sensitivity analyses considered small (SMD = 0.20), modest (SMD = 0.50), and large (SMD = 0.80) effect sizes.A total of 14 meta-analyses with 228 datasets (110 for neurological disorders and 118 for psychiatric disorders) were assessed. For SMD = 0.3, the number of observed "positive" studies (n = 94) was larger than expected (n = 35). We found evidence for an excess of significant findings overall (p < 0.0001) and in 8/14 meta-analyses. Evidence for an excess of significant findings was also observed for SMD = 0.5 for neurological disorders. Of the 228 datasets, 0 (0%), 0 (0%), 3 (1%), and 53 (23%) had a power >0.80, respectively, for SMDs of 0.30, 0.20, 0.50, and 0.80.Most studies in the rTMS literature are underpowered. This results in fragmentation and waste of research efforts. The somewhat high frequency of "positive" results seems spurious and may reflect bias. Caution is warranted in accepting rTMS as an established treatment for neuropsychiatric conditions.

  View details for DOI 10.1159/000502805

  View details for PubMedID 31590171

• The Accuracy of the Patient Health Questionnaire-9 Algorithm for Screening to Detect Major Depression: An Individual Participant Data Meta-Analysis. Psychotherapy and psychosomatics He, C. n., Levis, B. n., Riehm, K. E., Saadat, N. n., Levis, A. W., Azar, M. n., Rice, D. B., Krishnan, A. n., Wu, Y. n., Sun, Y. n., Imran, M. n., Boruff, J. n., Cuijpers, P. n., Gilbody, S. n., Ioannidis, J. P., Kloda, L. A., McMillan, D. n., Patten, S. B., Shrier, I. n., Ziegelstein, R. C., Akena, D. H., Arroll, B. n., Ayalon, L. n., Baradaran, H. R., Baron, M. n., Beraldi, A. n., Bombardier, C. H., Butterworth, P. n., Carter, G. n., Chagas, M. H., Chan, J. C., Cholera, R. n., Clover, K. n., Conwell, Y. n., de Man-van Ginkel, J. M., Fann, J. R., Fischer, F. H., Fung, D. n., Gelaye, B. n., Goodyear-Smith, F. n., Greeno, C. G., Hall, B. J., Harrison, P. A., Härter, M. n., Hegerl, U. n., Hides, L. n., Hobfoll, S. E., Hudson, M. n., Hyphantis, T. N., Inagaki, M. n., Ismail, K. n., Jetté, N. n., Khamseh, M. E., Kiely, K. M., Kwan, Y. n., Lamers, F. n., Liu, S. I., Lotrakul, M. n., Loureiro, S. R., Löwe, B. n., Marsh, L. n., McGuire, A. n., Mohd-Sidik, S. n., Munhoz, T. N., Muramatsu, K. n., Osório, F. L., Patel, V. n., Pence, B. W., Persoons, P. n., Picardi, A. n., Reuter, K. n., Rooney, A. G., da Silva Dos Santos, I. S., Shaaban, J. n., Sidebottom, A. n., Simning, A. n., Stafford, L. n., Sung, S. n., Tan, P. L., Turner, A. n., van Weert, H. C., White, J. n., Whooley, M. A., Winkley, K. n., Yamada, M. n., Thombs, B. D., Benedetti, A. n. 2019: 1–13

  Abstract

  Screening for major depression with the Patient Health Questionnaire-9 (PHQ-9) can be done using a cutoff or the PHQ-9 diagnostic algorithm. Many primary studies publish results for only one approach, and previous meta-analyses of the algorithm approach included only a subset of primary studies that collected data and could have published results.To use an individual participant data meta-analysis to evaluate the accuracy of two PHQ-9 diagnostic algorithms for detecting major depression and compare accuracy between the algorithms and the standard PHQ-9 cutoff score of ≥10.Medline, Medline In-Process and Other Non-Indexed Citations, PsycINFO, Web of Science (January 1, 2000, to February 7, 2015). Eligible studies that classified current major depression status using a validated diagnostic interview.Data were included for 54 of 72 identified eligible studies (n participants = 16,688, n cases = 2,091). Among studies that used a semi-structured interview, pooled sensitivity and specificity (95% confidence interval) were 0.57 (0.49, 0.64) and 0.95 (0.94, 0.97) for the original algorithm and 0.61 (0.54, 0.68) and 0.95 (0.93, 0.96) for a modified algorithm. Algorithm sensitivity was 0.22-0.24 lower compared to fully structured interviews and 0.06-0.07 lower compared to the Mini International Neuropsychiatric Interview. Specificity was similar across reference standards. For PHQ-9 cutoff of ≥10 compared to semi-structured interviews, sensitivity and specificity (95% confidence interval) were 0.88 (0.82-0.92) and 0.86 (0.82-0.88).The cutoff score approach appears to be a better option than a PHQ-9 algorithm for detecting major depression.

  View details for DOI 10.1159/000502294

  View details for PubMedID 31593971

• Therapy and prevention for mental health: What if mental diseases are mostly not brain disorders? BEHAVIORAL AND BRAIN SCIENCES Ioannidis, J. A. 2019; 42

  View details for DOI 10.1017/S0140525X1800105X

  View details for Web of Science ID 000486459400012

• Single pivotal trials with few corroborating characteristics were used for FDA approval of cancer therapies. Journal of clinical epidemiology Ladanie, A. n., Speich, B. n., Briel, M. n., Sclafani, F. n., Bucher, H. C., Agarwal, A. n., Ioannidis, J. P., Pereira, T. V., Kasenda, B. n., Hemkens, L. G. 2019

  Abstract

  Novel cancer therapies are often approved with evidence from a single pivotal trial alone. There are concerns about the credibility of this evidence. Higher validity may be indicated by five methodological and statistical characteristics of pivotal trial evidence that were described by the US Food and Drug Administration (FDA) which may corroborate the reliance on a single trial alone for approval decisions.We did a meta-epidemiologic evaluation of all single pivotal trials supporting FDA approval of novel drugs and therapeutic biologicals for cancers between 2000 and 2016. For each trial, we determined the presence of these five characteristics, which we operationalized as (1) large and multicenter trial (≥200 patients; more than one center); consistent treatment benefits across (2) multiple patient subgroups (in view of FDA reviewers), (3) multiple endpoints (including overall survival , progression-free survival, response rate, health related quality of life) and (4) multiple treatment comparisons (e.g. multi-arm studies); (5) "statistically very persuasive" results (p-values <0.00125).Thirty-five of 100 approvals were based on evidence from a single pivotal trial without any further supporting evidence on beneficial effects (20 randomized controlled trials and 15 single-arm trials). The number increased substantially from 1 approval before 2006 to 23 after 2011. Sixty-six percent (23/35) of the trials were large multicenter trials (median 301 patients and 63 centers). Consistent effects were demonstrated across subgroups in 66% (23/35), across endpoints in 43% (15/35), and across multiple comparisons in 3% (1/35). Very low p-values for the primary endpoint were seen in 34% (12/35). At least one of the corroborating characteristics was present in 94% (33/35) of all approvals, two or more were present in 54% (19/35) and none had all characteristics.and relevance: Single pivotal trials typically have some of the corroborating characteristics, but often only one or two. These characteristics need to be better operationalized, defined and reported and whether single trials with such characteristics provide similar evidence about benefits and harms of novel treatments as multiple trials would do, needs to be shown.

  View details for DOI 10.1016/j.jclinepi.2019.05.033

  View details for PubMedID 31158450

• Most UK scientists who publish extremely highly-cited papers do not secure funding from major public and charity funders: A descriptive analysis. PloS one Stavropoulou, C., Somai, M., Ioannidis, J. P. 2019; 14 (2): e0211460

  Abstract

  The UK is one of the largest funders of health research in the world, but little is known about how health funding is spent. Our study explores whether major UK public and charitable health research funders support the research of UK-based scientists producing the most highly-cited research. To address this question, we searched for UK-based authors of peer-reviewed papers that were published between January 2006 and February 2018 and received over 1000 citations in Scopus. We explored whether these authors have held a grant from the National Institute for Health Research (NIHR), the Medical Research Council (MRC) and the Wellcome Trust and compared the results with UK-based researchers who serve currently on the boards of these bodies. From the 1,370 papers relevant to medical, biomedical, life and health sciences with more than 1000 citations in the period examined, we identified 223 individuals from a UK institution at the time of publication who were either first/last or single authors. Of those, 164 are still in UK academic institutions, while 59 are not currently in UK academia (have left the country, are retired, or work in other sectors). Of the 164 individuals, only 59 (36%; 95% CI: 29-43%) currently hold an active grant from one of the three funders. Only 79 (48%; 95% CI: 41-56%) have held an active grant from any of the three funders between 2006-2017. Conversely, 457 of the 664 board members of MRC, Wellcome Trust, and NIHR (69%; 95% CI: 65-72%) have held an active grant in the same period by any of these funders. Only 7 out of 655 board members (1.1%) were first, last or single authors of an extremely highly-cited paper. There are many reasons why the majority of the most influential UK authors do not hold a grant from the country's major public and charitable funding bodies. Nevertheless, the results are worrisome and subscribe to similar patterns shown in the US. We discuss possible implications and suggest ways forward.

  View details for PubMedID 30811411

• True and false positive rates for different criteria of evaluating statistical evidence from clinical trials. BMC medical research methodology van Ravenzwaaij, D. n., Ioannidis, J. P. 2019; 19 (1): 218

  Abstract

  Until recently a typical rule that has often been used for the endorsement of new medications by the Food and Drug Administration has been the existence of at least two statistically significant clinical trials favoring the new medication. This rule has consequences for the true positive (endorsement of an effective treatment) and false positive rates (endorsement of an ineffective treatment).In this paper, we compare true positive and false positive rates for different evaluation criteria through simulations that rely on (1) conventional p-values; (2) confidence intervals based on meta-analyses assuming fixed or random effects; and (3) Bayes factors. We varied threshold levels for statistical evidence, thresholds for what constitutes a clinically meaningful treatment effect, and number of trials conducted.Our results show that Bayes factors, meta-analytic confidence intervals, and p-values often have similar performance. Bayes factors may perform better when the number of trials conducted is high and when trials have small sample sizes and clinically meaningful effects are not small, particularly in fields where the number of non-zero effects is relatively large.Thinking about realistic effect sizes in conjunction with desirable levels of statistical evidence, as well as quantifying statistical evidence with Bayes factors may help improve decision-making in some circumstances.

  View details for DOI 10.1186/s12874-019-0865-y

  View details for PubMedID 31775644

• A limited number of medicines pragmatic trials had potential for waived informed consent following the 2016 CIOMS ethical guidelines. Journal of clinical epidemiology Dal-Ré, R. n., Avendaño-Solà, C. n., de Boer, A. n., James, S. K., Rosendaal, F. R., Stephens, R. n., Ioannidis, J. P. 2019; 114: 60–71

  Abstract

  European regulations do not allow modification or waiver of informed consent for medicines randomized controlled trials (RCTs) where the three 2016 Council for International Organizations of Medical Sciences (CIOMS) provisions are met (consent would be impractical or unfeasible, yet the trial would have high social value and pose no or minimal risk to participants). We aimed to identify whether any such trials of medicines were being conducted in Europe.This is a survey of all phase 4 "ongoing" RCTs on the EU clinical trial register between July 1, 2016 and June 30, 2018, to identify those with potentially high levels of pragmatism. Trials that were excluded were as follows: those conducted on rare diseases; conducted on healthy volunteers (except those assessing vaccines); masked (single-, double-blind) trials; single-center trials; those where one could expect to lead patients to prefer one intervention over the other; and miscellaneous reasons. The degree of pragmatism of the RCTs was self-assessed by trials' investigators by means of the PRECIS-2 tool. Investigators of those trials considered to be highly pragmatic assessed the fulfillment of the three CIOMS provisions. Seven patients assessed the social value of the RCTs. Finally, 33 members of 11 research ethics committees (RECs) assessed the social value of the trials and whether they posed no more than minimal risk to participants. Investigators, patients, and REC members assessed the fulfillment of the CIOMS provisions as "yes," "not sure" or "no."Of the 638 phase 4 trials, 420 were RCTs, and 21 of these (5%) were candidates to be pragmatic. Investigators of 15 of these 21 RCTs self-assessed their trial's degree of pragmatism: 14 were highly pragmatic. Of these 14, eight fulfilled the three CIOMS provisions. Assessments by patients and RECs were inconsistent for several trials.We found few low-risk participant-level pragmatic RCTs that could be suitable for modified or waived participants' informed consent. European regulators should consider amending the current regulation and encouraging the conduct of such trials.

  View details for DOI 10.1016/j.jclinepi.2019.06.007

  View details for PubMedID 31212001

• Equivalency of the diagnostic accuracy of the PHQ-8 and PHQ-9: a systematic review and individual participant data meta-analysis. Psychological medicine Wu, Y. n., Levis, B. n., Riehm, K. E., Saadat, N. n., Levis, A. W., Azar, M. n., Rice, D. B., Boruff, J. n., Cuijpers, P. n., Gilbody, S. n., Ioannidis, J. P., Kloda, L. A., McMillan, D. n., Patten, S. B., Shrier, I. n., Ziegelstein, R. C., Akena, D. H., Arroll, B. n., Ayalon, L. n., Baradaran, H. R., Baron, M. n., Bombardier, C. H., Butterworth, P. n., Carter, G. n., Chagas, M. H., Chan, J. C., Cholera, R. n., Conwell, Y. n., de Man-van Ginkel, J. M., Fann, J. R., Fischer, F. H., Fung, D. n., Gelaye, B. n., Goodyear-Smith, F. n., Greeno, C. G., Hall, B. J., Harrison, P. A., Härter, M. n., Hegerl, U. n., Hides, L. n., Hobfoll, S. E., Hudson, M. n., Hyphantis, T. n., Inagaki, M. D., Jetté, N. n., Khamseh, M. E., Kiely, K. M., Kwan, Y. n., Lamers, F. n., Liu, S. I., Lotrakul, M. n., Loureiro, S. R., Löwe, B. n., McGuire, A. n., Mohd-Sidik, S. n., Munhoz, T. N., Muramatsu, K. n., Osório, F. L., Patel, V. n., Pence, B. W., Persoons, P. n., Picardi, A. n., Reuter, K. n., Rooney, A. G., Santos, I. S., Shaaban, J. n., Sidebottom, A. n., Simning, A. n., Stafford, M. D., Sung, S. n., Tan, P. L., Turner, A. n., van Weert, H. C., White, J. n., Whooley, M. A., Winkley, K. n., Yamada, M. n., Benedetti, A. n., Thombs, B. D. 2019: 1–13

  Abstract

  Item 9 of the Patient Health Questionnaire-9 (PHQ-9) queries about thoughts of death and self-harm, but not suicidality. Although it is sometimes used to assess suicide risk, most positive responses are not associated with suicidality. The PHQ-8, which omits Item 9, is thus increasingly used in research. We assessed equivalency of total score correlations and the diagnostic accuracy to detect major depression of the PHQ-8 and PHQ-9.We conducted an individual patient data meta-analysis. We fit bivariate random-effects models to assess diagnostic accuracy.16 742 participants (2097 major depression cases) from 54 studies were included. The correlation between PHQ-8 and PHQ-9 scores was 0.996 (95% confidence interval 0.996 to 0.996). The standard cutoff score of 10 for the PHQ-9 maximized sensitivity + specificity for the PHQ-8 among studies that used a semi-structured diagnostic interview reference standard (N = 27). At cutoff 10, the PHQ-8 was less sensitive by 0.02 (-0.06 to 0.00) and more specific by 0.01 (0.00 to 0.01) among those studies (N = 27), with similar results for studies that used other types of interviews (N = 27). For all 54 primary studies combined, across all cutoffs, the PHQ-8 was less sensitive than the PHQ-9 by 0.00 to 0.05 (0.03 at cutoff 10), and specificity was within 0.01 for all cutoffs (0.00 to 0.01).PHQ-8 and PHQ-9 total scores were similar. Sensitivity may be minimally reduced with the PHQ-8, but specificity is similar.

  View details for DOI 10.1017/S0033291719001314

  View details for PubMedID 31298180

• Author Correction: 'Stealth' corporate innovation: an emerging threat for therapeutic drug development. Nature immunology Mastellos, D. C., Blom, A. M., Connolly, E. S., Daha, M. R., Geisbrecht, B. V., Ghebrehiwet, B. n., Gros, P. n., Hajishengallis, G. n., Holers, V. M., Huber-Lang, M. n., Kinoshita, T. n., Mollnes, T. E., Montgomery, R. A., Morgan, B. P., Nilsson, B. n., Pio, R. n., Ricklin, D. n., Risitano, A. M., Taylor, R. P., Mantovani, A. n., Ioannidis, J. P., Lambris, J. D. 2019

  Abstract

  An amendment to this paper has been published and can be accessed via a link at the top of the paper.

  View details for DOI 10.1038/s41590-019-0531-x

  View details for PubMedID 31605100

• Evidence Relating Health Care Provider Burnout and Quality of Care: A Systematic Review and Meta-analysis. Annals of internal medicine Tawfik, D. S., Scheid, A. n., Profit, J. n., Shanafelt, T. n., Trockel, M. n., Adair, K. C., Sexton, J. B., Ioannidis, J. P. 2019

  Abstract

  Whether health care provider burnout contributes to lower quality of patient care is unclear.To estimate the overall relationship between burnout and quality of care and to evaluate whether published studies provide exaggerated estimates of this relationship.MEDLINE, PsycINFO, Health and Psychosocial Instruments (EBSCO), Mental Measurements Yearbook (EBSCO), EMBASE (Elsevier), and Web of Science (Clarivate Analytics), with no language restrictions, from inception through 28 May 2019.Peer-reviewed publications, in any language, quantifying health care provider burnout in relation to quality of patient care.2 reviewers independently selected studies, extracted measures of association of burnout and quality of care, and assessed potential bias by using the Ioannidis (excess significance) and Egger (small-study effect) tests.A total of 11 703 citations were identified, from which 123 publications with 142 study populations encompassing 241 553 health care providers were selected. Quality-of-care outcomes were grouped into 5 categories: best practices (n = 14), communication (n = 5), medical errors (n = 32), patient outcomes (n = 17), and quality and safety (n = 74). Relations between burnout and quality of care were highly heterogeneous (I2 = 93.4% to 98.8%). Of 114 unique burnout-quality combinations, 58 indicated burnout related to poor-quality care, 6 indicated burnout related to high-quality care, and 50 showed no significant effect. Excess significance was apparent (73% of studies observed vs. 62% predicted to have statistically significant results; P = 0.011). This indicator of potential bias was most prominent for the least-rigorous quality measures of best practices and quality and safety.Studies were primarily observational; neither causality nor directionality could be determined.Burnout in health care professionals frequently is associated with poor-quality care in the published literature. The true effect size may be smaller than reported. Future studies should prespecify outcomes to reduce the risk for exaggerated effect size estimates.Stanford Maternal and Child Health Research Institute.

  View details for DOI 10.7326/M19-1152

  View details for PubMedID 31590181

• Author Correction: A consensus-based transparency checklist. Nature human behaviour Aczel, B. n., Szaszi, B. n., Sarafoglou, A. n., Kekecs, Z. n., Kucharský, Š. n., Benjamin, D. n., Chambers, C. D., Fisher, A. n., Gelman, A. n., Gernsbacher, M. A., Ioannidis, J. P., Johnson, E. n., Jonas, K. n., Kousta, S. n., Lilienfeld, S. O., Lindsay, D. S., Morey, C. C., Munafò, M. n., Newell, B. R., Pashler, H. n., Shanks, D. R., Simons, D. J., Wicherts, J. M., Albarracin, D. n., Anderson, N. D., Antonakis, J. n., Arkes, H. R., Back, M. D., Banks, G. C., Beevers, C. n., Bennett, A. A., Bleidorn, W. n., Boyer, T. W., Cacciari, C. n., Carter, A. S., Cesario, J. n., Clifton, C. n., Conroy, R. M., Cortese, M. n., Cosci, F. n., Cowan, N. n., Crawford, J. n., Crone, E. A., Curtin, J. n., Engle, R. n., Farrell, S. n., Fearon, P. n., Fichman, M. n., Frankenhuis, W. n., Freund, A. M., Gaskell, M. G., Giner-Sorolla, R. n., Green, D. P., Greene, R. L., Harlow, L. L., de la Guardia, F. H., Isaacowitz, D. n., Kolodner, J. n., Lieberman, D. n., Logan, G. D., Mendes, W. B., Moersdorf, L. n., Nyhan, B. n., Pollack, J. n., Sullivan, C. n., Vazire, S. n., Wagenmakers, E. J. 2019

  Abstract

  An amendment to this paper has been published and can be accessed via a link at the top of the paper.

  View details for DOI 10.1038/s41562-019-0812-2

  View details for PubMedID 31873202

• Time to abandon early detection cancer screening. European journal of clinical investigation Adami, H., Kalager, M., Valdimarsdottir, U., Bretthauer, M., Ioannidis, J. P. 2018: e13062

  Abstract

  Ever since 1971, when the U.S. President signed the "War on Cancer" [1] National Cancer Act, screening has been a hallmark in cancer control. The fundamental idea was that more cancers would be cured if they were detected and treated before symptoms arise. During the following decades, astronomic amounts of money and great hopes were invested to implement population-based screening programs. This article is protected by copyright. All rights reserved.

  View details for PubMedID 30565674

• How does exercise treatment compare with antihypertensive medications? A network meta-analysis of 391 randomised controlled trials assessing exercise and medication effects on systolic blood pressure. British journal of sports medicine Naci, H., Salcher-Konrad, M., Dias, S., Blum, M. R., Sahoo, S. A., Nunan, D., Ioannidis, J. P. 2018

  Abstract

  OBJECTIVE: To compare the effect of exercise regimens and medications on systolic blood pressure (SBP).DATA SOURCES: Medline (via PubMed) and the Cochrane Library.ELIGIBILITY CRITERIA: Randomised controlled trials (RCTs) of angiotensin-converting enzyme inhibitors (ACE-I), angiotensin-2 receptor blockers (ARBs), beta-blockers, calcium channel blockers (CCBs) and diuretics were identified from existing Cochrane reviews. A previously published meta-analysis of exercise interventions was updated to identify recent RCTs that tested the SBP-lowering effects of endurance, dynamic resistance, isometric resistance, and combined endurance and resistance exercise interventions (up to September 2018).DESIGN: Random-effects network meta-analysis.OUTCOME: Difference in mean change from baseline SBP between comparator treatments (change from baseline in one group minus that in the other group) and its 95% credible interval (95% CrI), measured in mmHg.RESULTS: We included a total of 391 RCTs, 197 of which evaluated exercise interventions (10461 participants) and 194 evaluated antihypertensive medications (29281 participants). No RCTs compared directly exercise against medications. While all medication trials included hypertensive populations, only 56 exercise trials included hypertensive participants (≥140mmHg), corresponding to 3508 individuals. In a 10% random sample, risk of bias was higher in exercise RCTs, primarily due to lack of blinding and incomplete outcome data. In analyses that combined all populations, antihypertensive medications achieved higher reductions in baseline SBP compared with exercise interventions (mean difference -3.96mmHg, 95% CrI -5.02 to -2.91). Compared with control, all types of exercise (including combination of endurance and resistance) and all classes of antihypertensive medications were effective in lowering baseline SBP. Among hypertensive populations, there were no detectable differences in the SBP-lowering effects of ACE-I, ARB, beta-blocker and diuretic medications when compared with endurance or dynamic resistance exercise. There was no detectable inconsistency between direct and indirect comparisons. Although there was evidence of small-study effects, this affected both medication and exercise trials.CONCLUSIONS: The effect of exercise interventions on SBP remains under-studied, especially among hypertensive populations. Our findings confirm modest but consistent reductions in SBP in many studied exercise interventions across all populations but individuals receiving medications generally achieved greater reductions than those following structured exercise regimens. Assuming equally reliable estimates, the SBP-lowering effect of exercise among hypertensive populations appears similar to that of commonly used antihypertensive medications. Generalisability of these findings to real-world clinical settings should be further evaluated.

  View details for PubMedID 30563873

• Gene-environment interactions and colorectal cancer risk: an umbrella review of systematic reviews and meta-analyses of observational studies. International journal of cancer Yang, T., Li, X., Montazeri, Z., Little, J., Farrington, S. M., Ioannidis, J. P., Dunlop, M. G., Campbell, H., Timofeeva, M., Theodoratou, E. 2018

  Abstract

  The cause of colorectal cancer (CRC) is multifactorial, involving both genetic variants and environmental risk factors. We systematically searched the MEDLINE, EMBASE, China National Knowledge Infrastructure (CNKI) and Wanfang databases from inception to December 2016, to identify systematic reviews and meta-analyses of observational studies that investigated gene-environment (G*E) interactions in CRC risk. Then, we critically evaluated the cumulative evidence for the G*E interactions using an extension of the Human Genome Epidemiology Network's Venice criteria. Overall, 15 articles reporting systematic reviews of observational studies on 89 G*E interactions, 20 articles reporting meta-analyses of candidate gene- or single-nucleotide polymorphisms-based studies on 521 G*E interactions, and 8 articles reporting 33 genome-wide G*E interaction analyses were identified. On the basis of prior and observed scores, only the interaction between rs6983267 (8q24) and aspirin use was found to have a moderate overall credibility score as well as main genetic and environmental effects. Though 5 other interactions were also found to have moderate evidence, these interaction effects were tenuous due to the lack of main genetic effects and/or environmental effects. We did not find highly convincing evidence for any interactions, but several associations were found to have moderate or weak strength of evidence. Our conclusions are based on application of the Venice criteria which were designed to provide a conservative assessment of gene-environment interactions and thus do not include an evaluation of biological plausibility of an observed joint effect. This article is protected by copyright. All rights reserved.

  View details for PubMedID 30536881

• New clinical trial designs in the era of precision medicine: An overview of definitions, strengths, weaknesses, and current use in oncology. Cancer treatment reviews Janiaud, P., Serghiou, S., Ioannidis, J. P. 2018; 73: 20–30

  Abstract

  With expanding knowledge in tumor biology and biomarkers, oncology therapies are increasingly moving away from the "one-size-fits-all" rationale onto biomarker-driven therapies tailored according to patient-specific characteristics, most commonly the tumor's molecular profile. The advent of precision medicine in oncology has been accompanied by the introduction of novel clinical trial designs that aim to identify biomarker-matched subgroups of patients that will benefit the most from targeted therapies. This innovation comes with the promise of answering more treatment questions, more efficiently and in less time. In this article, we give an overview of the different biomarker-based designs, comparing the features of enrichment, randomize-all, umbrella, and basket trials, and highlighting their advantages and disadvantages. We focus more on the novel designs known as master protocols, which include umbrella and basket trials. We have also conducted a search in ClinicalTrials.gov for registered oncology-related protocols of ongoing or completed trials labeled as umbrella or basket trials for solid tumors; we also included additional relevant trials retrieved from other reviews. We present and discuss the key features of the 30 eligible basket trials and 27 eligible umbrella trials. Only a minority of them are randomized (2 and 9, respectively), including three trials with adaptive randomization. Five of these trials have been completed as of July 2018. Precision medicine trial designs fuel new hopes for identifying best treatments, but there is also the potential for hype. The benefits and challenges associated with their use will need continued monitoring.

  View details for PubMedID 30572165

• A Comprehensive Analysis of Protocols for Deriving Dopaminergic Neurons from Human Pluripotent Stem Cells. Stem cells translational medicine Marton, R. M., Ioannidis, J. P. 2018

  Abstract

  The potential applications of human embryonic and induced pluripotent stem cells has led to immense interest in developing new protocols to differentiate specific cell types or modifying existing protocols. To investigate to what extent and why new protocols for the same cell types are developed and adopted, we systematically evaluated 158 publications (2004-2017) that differentiated human stem cells into dopaminergic neurons. We categorized each article by degree of novelty and recorded motivations for protocol development. 74 novel or modified protocols were developed. Most (65%) were not used again in subsequent studies. Diverse motivations were recorded and performance of new methods was assessed with substantially different approaches across studies. There was improvement over time in yield of neuron production, but not in yield of dopaminergic neurons or time required for getting neurons. Standardized reporting of performance metrics may help rational choice of the best methods. Stem Cells Translational Medicine 2018.

  View details for PubMedID 30537442

• Cochrane crisis: secrecy, intolerance, and evidence-based values. European journal of clinical investigation Ioannidis, J. P. 2018: e13058

  Abstract

  The Cochrane Collaboration was launched in 1993 with great enthusiasm. It aimed to offer a volunteer-based, community-strong, independent, and critical effort for materializing the goals of evidence-based medicine worldwide through the production of high-quality, rigorous systematic reviews (1). In the next quarter of a century, the effort did accomplish an enormous amount and its members should be proud of their achievements. The quality, depth, and breadth of expertise of the people involved in this collaborative endeavor is unmatched. Cochrane systematic reviews gained a well-deserved reputation of excellence (2). This article is protected by copyright. All rights reserved.

  View details for PubMedID 30520025

• Diagnostic accuracy of the Geriatric Depression Scale-30, Geriatric Depression Scale-15, Geriatric Depression Scale-5 and Geriatric Depression Scale-4 for detecting major depression: protocol for a systematic review and individual participant data meta-analysis. BMJ open Benedetti, A., Wu, Y., Levis, B., Wilchesky, M., Boruff, J., Ioannidis, J. P., Patten, S. B., Cuijpers, P., Shrier, I., Gilbody, S., Ismail, Z., McMillan, D., Mitchell, N., Ziegelstein, R. C., Thombs, B. D. 2018; 8 (12): e026598

  Abstract

  INTRODUCTION: The 30-item Geriatric Depression Scale (GDS-30) and the shorter GDS-15, GDS-5 and GDS-4 are recommended as depression screening tools for elderly individuals. Existing meta-analyses on the diagnostic accuracy of the GDS have not been able to conduct subgroup analyses, have included patients already identified as depressed who would not be screened in practice and have not accounted for possible bias due to selective reporting of results from only better-performing cut-offs in primary studies. Individual participant data meta-analysis (IPDMA), which involves a standard systematic review, then a synthesis of individual participant data, rather than summary results, could address these limitations. The objective of our IPDMA is to generate accuracy estimates to detect major depression for all possible cut-offs of each version of the GDS among studies using different reference standards, separately and among participant subgroups based on age, sex, dementia diagnosis and care settings. In addition, we will use a modelling approach to generate individual participant probabilities for major depression based on GDS scores (rather than a dichotomous cut-off) and participant characteristics (eg, sex, age, dementia status, care setting).METHODS AND ANALYSIS: Individual participant data comparing GDS scores to a major depression diagnosis based on a validated structured or semistructured diagnostic interview will be sought via a systematic review. Data sources will include Medline, Medline In-Process & Other Non-Indexed Citations, PsycINFO and Web of Science. Bivariate random-effects models will be used to estimate diagnostic accuracy parameters for each cut-off of the different versions of the GDS. Prespecified subgroup analyses will be conducted. Risk of bias will be assessed with the Quality Assessment of Diagnostic Accuracy Studies-2 tool.ETHICS AND DISSEMINATION: The findings of this study will be of interest to stakeholders involved in research, clinical practice and policy.PROSPERO REGISTRATION NUMBER: CRD42018104329.

  View details for PubMedID 30518594

• Environmental risk factors and nonpharmacological and nonsurgical interventions for obesity: An umbrella review of meta-analyses of cohort studies and randomized controlled trials EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Solmi, M., Kohler, C. A., Stubbs, B., Koyanagi, A., Bortolato, B., Monaco, F., Vancampfort, D., Machado, M. O., Maes, M., Tzoulaki, I., Firth, J., Ioannidis, J. A., Carvalho, A. F. 2018; 48 (12): e12982

  Abstract

  Multiple environmental factors have been implicated in obesity, and multiple interventions, besides drugs and surgery, have been assessed in obese patients. Results are scattered across many studies and meta-analyses, and they often mix obese and overweight individuals.PubMed and Cochrane Database of Systematic Reviews were searched through 21 January 2017 for meta-analyses of cohort studies assessing environmental risk factors for obesity, and randomized controlled trials investigating nonpharmacological and nonsurgical therapeutic interventions for obesity. We excluded data on overweight participants. Evidence from observational studies was graded according to criteria that included the statistical significance of the random-effects summary estimate and of the largest study in a meta-analysis, the number of obesity cases, heterogeneity between studies, 95% prediction intervals, small-study effects and excess significance. The evidence of intervention studies for obesity was assessed with the GRADE framework.Fifty-four articles met eligibility criteria, including 26 meta-analyses of environmental risk factors (166 studies) and 46 meta-analyses of nondrug, nonsurgical interventions (206 trials). In adults, the only risk factor with convincing evidence was depression, and childhood obesity, adolescent obesity, childhood abuse and short sleep duration had highly suggestive evidence. Infancy weight gain during the first year of life, depression and low maternal education had convincing evidence for association with paediatric obesity. All interventions had low or very-low-quality evidence with one exception of moderate-quality evidence for one comparison (no differences in efficacy between brief lifestyle primary care interventions and other interventions for paediatric obesity). Summary effect sizes were mostly small across compared interventions (maximum 5.1 kg in adults and 1.78 kg in children) and even these estimates may be inflated.Depression, obesity in earlier age groups, short sleep duration, childhood abuse and low maternal education have the strongest support among proposed risk factors for obesity. Furthermore, there is no high-quality evidence to recommend treating obesity with a specific nonpharmacological and nonsurgical intervention among many available, and whatever benefits in terms of magnitude of weight loss appear small.

  View details for PubMedID 29923186

• The role of meta-analyses and umbrella reviews in assessing the harms of psychotropic medications: beyond qualitative synthesis EPIDEMIOLOGY AND PSYCHIATRIC SCIENCES Solmi, M., Correll, C. U., Carvalho, A. F., Ioannidis, J. A. 2018; 27 (6): 537–42

  Abstract

  ὠφελέειν, ἢ μὴ βλάπτειν (Primum non nocere) - Hιppocrates' principle should still guide daily medical prescribing. Therefore, assessing evidence of psychopharmacologic agents' safety and harms is essential. Randomised controlled trials (RCTs) and observational studies may provide complementary information about harms of psychopharmacologic medications from both experimental and real-world settings. It is considered that RCTs provide a better control of confounding variables, while observational studies provide evidence from larger samples, longer follow-ups, in more representative samples, which may be more reflective of real-life clinical scenarios. However, this may not always hold true. Moreover, in observational studies, safety data are poorly or inconsistently reported, precluding reliable quantitative synthesis in meta-analyses. Beyond individual studies, meta-analyses, which represent the highest level of 'evidence', can be misleading, redundant and of low methodological quality. Overlapping meta-analyses sometimes even reach different conclusions on the same topic. Meta-analyses should be assessed systematically. Descriptive reviews of reviews can be poorly informative. Conversely, 'umbrella reviews' can use a quantitative approach to grade evidence. In this editorial, we present the main factors involved in the assessment of psychopharmacologic agents' harms from individual studies, meta-analyses and umbrella reviews. Study design features, sample size, number of the events of interest, summary effect sizes, p-values, heterogeneity, 95% prediction intervals, confounding factor adjustment and tests of bias (e.g., small-study effects and excess significance) can be combined with other assessment tools, such as AMSTAR and GRADE to create a framework for assessing the credibility of evidence.

  View details for DOI 10.1017/S204579601800032X

  View details for Web of Science ID 000448298400003

  View details for PubMedID 30008278

• Disclosures Can Always Be Improved Reply JAMA PSYCHIATRY Cristea, I., Ioannidis, J. A. 2018; 75 (12): 1303–4

  View details for DOI 10.1001/jamapsychiatry.2018.2785

  View details for Web of Science ID 000452682200020

• Industry-funded versus non-profit-funded critical care research: a meta-epidemiological overview (vol 44, pg 1613, 2018) INTENSIVE CARE MEDICINE Janiaud, P., Cristea, I., Ioannidis, J. A. 2018; 44 (12): 2323

  View details for DOI 10.1007/s00134-018-5437-9

  View details for Web of Science ID 000452162900059

• Diagnostic accuracy of the Geriatric Depression Scale-30, Geriatric Depression Scale-15, Geriatric Depression Scale-5 and Geriatric Depression Scale-4 for detecting major depression: protocol for a systematic review and individual participant data meta-analysis BMJ OPEN Benedetti, A., Wu, Y., Levis, B., Wilchesky, M., Boruff, J., Ioannidis, J. A., Patten, S. B., Cuijpers, P., Shrier, I., Gilbody, S., Ismail, Z., McMillan, D., Mitchell, N., Ziegelstein, R. C., Thombs, B. D. 2018; 8 (12)

  View details for DOI 10.1136/bmjopen-2018-026598

  View details for Web of Science ID 000455309300179

• Replication, Duplication, and Waste in a Quarter Million Systematic Reviews and Meta-Analyses. Circulation. Cardiovascular quality and outcomes Siontis, K. C., Ioannidis, J. P. 2018; 11 (12): e005212

  View details for PubMedID 30562075

• How often can meta-analyses of individual-level data individualize treatment? A meta-epidemiologic study. International journal of epidemiology Schuit, E., Li, A. H., Ioannidis, J. P. 2018

  Abstract

  Background: One of the claimed main advantages of individual participant data meta-analysis (IPDMA) is that it allows assessment of subgroup effects based on individual-level participant characteristics, and eventually stratified medicine. In this study, we evaluated the conduct and results of subgroup analyses in IPDMA.Methods: We searched PubMed, EMBASE and the Cochrane Library from inception to 31 December 2014. We included papers if they described an IPDMA based on randomized clinical trials that investigated a therapeutic intervention on human subjects and in which the meta-analysis was preceded by a systematic literature search. We extracted data items related to subgroup analysis and subgroup differences (subgroup-treatment interaction p<0.05).Results: Overall, 327 IPDMAs were eligible. A statistically significant subgroup-treatment interaction for the primary outcome was reported in 102 (36.6%) of 279 IPDMAs that reported at least one subgroup analysis. This corresponded to 187 different statistically significant subgroup-treatment interactions: 124 for an individual-level subgrouping variable (in 76 IPDMAs) and 63 for a group-level subgrouping variable (in 36 IPDMAs). Of the 187, only 7 (3.7%; 6 individual and 1 group-level subgrouping variables) had a large difference between strata (standardized effect difference d≥0.8). Among the 124 individual-level statistically significant subgroup differences, the IPDMA authors claimed that 42 (in 21 IPDMAs) should lead to treating the subgroups differently. None of these 42 had d≥0.8.Conclusions: Availability of individual-level data provides statistically significant interactions for relative treatment effects in about a third of IPDMAs. A modest number of these interactions may offer opportunities for stratified medicine decisions.

  View details for PubMedID 30445577

• Marginal structural models and other analyses allow multiple estimates of treatment effects in randomized clinical trials: meta-epidemiological analysis. Journal of clinical epidemiology Ewald, H., Speich, B., Ladanie, A., Bucher, H. C., Ioannidis, J. P., Hemkens, L. G. 2018

  Abstract

  OBJECTIVE: To determine how marginal structural models (MSMs), which are increasingly used to estimate causal effects, are used in randomized clinical trials (RCTs) and compare their results with those from intention-to-treat (ITT) or other analyses.DESIGN: and Setting: We searched PubMed, Scopus, citations of key references, and Clinicaltrials.gov. Eligible RCTs reported clinical effects based on MSMs and at least one other analysis.RESULTS: We included 12 RCTs reporting 138 analyses for 24 clinical questions. In 19/24 (79%), MSM-based and other effect estimates were all in the same direction, 22/22 had overlapping 95%CIs, and in 19/22 (86%), the MSM-effect estimate lay within all 95%CIs of all other effects (in two cases no CIs were reported). For the same clinical question, the largest effect estimate from any analysis was 1.19-fold (median; IQR 1.13-1.34) larger than the smallest. All MSM and ITT-effect estimates were in the same direction and had overlapping 95% CIs. In 71% (12/17), they also agreed on the presence of statistical significance. MSM-based effect estimates deviated more from the null than those based on ITT (p=0.18). The effect estimates of both approaches differed 1.12-fold (median; IQR 1.02-1.22).CONCLUSIONS: MSMs provided largely similar effect estimates as other available analyses. Nevertheless, some of the differences in effect estimates or statistical significance may become important in clinical decision-making and the multiple estimates require utmost attention of possible selective reporting bias.

  View details for PubMedID 30423375

• Correction to: Industry-funded versus non-profit-funded critical care research: a meta-epidemiological overview. Intensive care medicine Janiaud, P., Cristea, I., Ioannidis, J. P. 2018

  Abstract

  The original article can be found online.

  View details for PubMedID 30406805

• Systematic Screening For Environmental And Behavioral Determinants Identifies Factors Detrimental to Skeletal Health Oei, L., Wu, J., Oei, E., Rivadeneira, F., Uitterlinden, A., Ioannidis, J., Snyder, M., Patel, C. WILEY. 2018: 279

  View details for Web of Science ID 000450475401405

• Mapping the universe of registered reports. Nature human behaviour Hardwicke, T. E., Ioannidis, J. P. 2018; 2 (11): 793-796

  View details for DOI 10.1038/s41562-018-0444-y

  View details for PubMedID 31558810

• Mapping the universe of registered reports NATURE HUMAN BEHAVIOUR Hardwicke, T. E., Ioannidis, J. A. 2018; 2 (11): 793–96

  View details for DOI 10.1038/s41562-018-0444-y

  View details for Web of Science ID 000449539900003

• How to design preclinical studies in nanomedicine and cell therapy to maximize the prospects of clinical translation NATURE BIOMEDICAL ENGINEERING Ioannidis, J. A., Kim, B. S., Trounson, A. 2018; 2 (11): 797–809

  View details for DOI 10.1038/s41551-018-0314-y

  View details for Web of Science ID 000449679300005

• The Complexities of Evaluating the Exposome in Psychiatry: A Data-Driven Illustration of Challenges and Some Propositions for Amendments SCHIZOPHRENIA BULLETIN Guloksuz, S., Rutten, B. F., Pries, L., ten Have, M., de Graaf, R., van Dorsselaer, S., Klingenberg, B., van Os, J., Ioannidis, J. A., European Network Natl Schizophr 2018; 44 (6): 1175–79

  Abstract

  Identifying modifiable factors through environmental research may improve mental health outcomes. However, several challenges need to be addressed to optimize the chances of success. By analyzing the Netherlands Mental Health Survey and Incidence Study-2 data, we provide a data-driven illustration of how closely connected the exposures and the mental health outcomes are and how model and variable specifications produce "vibration of effects" (variation of results under multiple different model specifications). Interdependence of exposures is the rule rather than the exception. Therefore, exposure-wide systematic approaches are needed to separate genuine strong signals from selective reporting and dissect sources of heterogeneity. Pre-registration of protocols and analytical plans is still uncommon in environmental research. Different studies often present very different models, including different variables, despite examining the same outcome, even if consistent sets of variables and definitions are available. For datasets that are already collected (and often already analyzed), the exploratory nature of the work should be disclosed. Exploratory analysis should be separated from prospective confirmatory research with truly pre-specified analysis plans. In the era of big-data, where very low P values for trivial effects are detected, several safeguards may be considered to improve inferences, eg, lowering P-value thresholds, prioritizing effect sizes over significance, analyzing pre-specified falsification endpoints, and embracing alternative approaches like false discovery rates and Bayesian methods. Any claims for causality should be cautious and preferably avoided, until intervention effects have been validated. We hope the propositions for amendments presented here may help with meeting these pressing challenges.

  View details for PubMedID 30169883

• How to design preclinical studies in nanomedicine and cell therapy to maximize the prospects of clinical translation. Nature biomedical engineering Ioannidis, J. P., Kim, B. Y., Trounson, A. 2018; 2 (11): 797-809

  Abstract

  The clinical translation of promising products, technologies and interventions from the disciplines of nanomedicine and cell therapy has been slow and inefficient. In part, translation has been hampered by suboptimal research practices that propagate biases and hinder reproducibility. These include the publication of small and underpowered preclinical studies, suboptimal study design (in particular, biased allocation of experimental groups, experimenter bias and lack of necessary controls), the use of uncharacterized or poorly characterized materials, poor understanding of the relevant biology and mechanisms, poor use of statistics, large between-model heterogeneity, absence of replication, lack of interdisciplinarity, poor scientific training in study design and methods, a culture that does not incentivize transparency and sharing, poor or selective reporting, misaligned incentives and rewards, high costs of materials and protocols, and complexity of the developed products, technologies and interventions. In this Perspective, we discuss special manifestations of these problems in nanomedicine and in cell therapy, and describe mitigating strategies. Progress on reducing bias and enhancing reproducibility early on ought to enhance the translational potential of biomedical findings and technologies.

  View details for DOI 10.1038/s41551-018-0314-y

  View details for PubMedID 30931172

  View details for PubMedCentralID PMC6436641

• Massive citations to misleading methods and research tools: Matthew effect, quotation error and citation copying EUROPEAN JOURNAL OF EPIDEMIOLOGY Ioannidis, J. A. 2018; 33 (11): 1021–23

  View details for DOI 10.1007/s10654-018-0449-x

  View details for Web of Science ID 000447899500001

• Reproducible research practices, transparency, and open access data in the biomedical literature, 2015-2017 PLOS BIOLOGY Wallach, O. D., Boyack, K. W., Ioannidis, J. A. 2018; 16 (11)

  View details for DOI 10.1371/journal.pbio.2006930

  View details for Web of Science ID 000452442600013

• Extremely large outlier treatment effects may be a footprint of bias in trials from less developed countries: randomized trials of gabapentinoids. Journal of clinical epidemiology Desai, K., Carroll, I., Asch, S., Hernandez-Boussard, T., Ioannidis, J. P. 2018

  Abstract

  OBJECTIVE: Court documents have proven that a manufacturer-orchestrated strategy tried to promote gabapentin by distorting evidence in randomized trials. Given this background, we aimed to assess whether implausibly large treatment effects for gabapentin and for a similar gabapentinoid, pregabalin may have been published.STUDY DESIGN AND SETTING: We identified meta-analyses on gabapentin or pregabalin on any outcome from Google Scholar, PubMed and EMBASE. We explored excess of significance in meta-analyses and whether outlier studies with extreme results (differing >0.8 standard deviations from the summary effect of the meta-analysis) were scrutinized.RESULTS: All 10 evaluated meta-analyses showed statistically significant favorable findings. Heterogeneity I2 estimates exceeding 90% were noted in 4 meta-analyses of post-operative pain. In these 4 meta-analyses, 77 studies had estimates differing >0.8 standard deviations from the summary estimate. 39/77 represented extremely favorable results and 33 of them came from less developed countries with no tradition of clinical research, 22 reported no information on funding and 20 reported no conflicts of interest. Conversely, 27/38 studies with unfavorable results came from more developed countries.CONCLUSION: Extremely favorable outlier studies in the meta-analyzed literature of gabapentin and pregabalin may be a footprint of bias in studies done in less developed countries.

  View details for PubMedID 30366063

• Statins and Multiple Noncardiovascular Outcomes Umbrella Review of Meta-analyses of Observational Studies and Randomized Controlled Trials ANNALS OF INTERNAL MEDICINE He, Y., Li, X., Gasevic, D., Brunt, E., McLachlan, F., Millenson, M., Timofeeva, M., Ioannidis, J. A., Campbell, H., Theodoratou, E. 2018; 169 (8): 543-+

  View details for DOI 10.7326/M18-0808

  View details for Web of Science ID 000447340800016

• Using Big Data to Determine Reference Values for Laboratory Tests Reply JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Manrai, A. K., Patel, C. J., Ioannidis, J. A. 2018; 320 (14): 1496

  View details for PubMedID 30304422

• Statins and Multiple Noncardiovascular Outcomes: Umbrella Review of Meta-analyses of Observational Studies and Randomized Controlled Trials. Annals of internal medicine He, Y., Li, X., Gasevic, D., Brunt, E., McLachlan, F., Millenson, M., Timofeeva, M., Ioannidis, J. P., Campbell, H., Theodoratou, E. 2018

  Abstract

  Background: Many effects of statins on non-cardiovascular disease (non-CVD) outcomes have been reported.Purpose: To evaluate the quantity, validity, and credibility of evidence regarding associations between statins and non-CVD outcomes and the effects of statins on these outcomes.Data Sources: MEDLINE and EMBASE (English terms only, inception to 28 May 2018).Study Selection: Meta-analyses (published in English) of observational studies and of randomized controlled trials (RCTs) that examined non-CVD outcomes of statin intake.Data Extraction: Two investigators extracted data from meta-analyses and individual studies. Credibility assessments based on summary effect sizes from a random-effects model, between-study heterogeneity, 95% prediction interval, small-study effect, excess significance, and credibility ceilings were devised to classify evidence.Data Synthesis: This review explored 278 unique non-CVD outcomes from 112 meta-analyses of observational studies and 144 meta-analyses of RCTs. For observational studies, no convincing (class I) evidence, 2 highly suggestive (class II) associations (decreased cancer mortality in patients with cancer and decreased exacerbation in patients with chronic obstructive pulmonary disease), 21 suggestive (class III) associations, and 42 weak (class IV) associations were identified. One outcome from the RCTs (decreased all-cause mortality in patients with chronic kidney disease) attained a sufficient amount of evidence with no hints of bias. For adverse events, observational studies showed suggestive evidence that statins increase the risk for diabetes and myopathy. Among the RCTs, no statistically significant effects were found on myopathy, myalgia, or rhabdomyolysis.Limitations: Studies with relevant data and outcomes not included in the meta-analyses may have been missed. Credibility assessments relied on several assumptions and arbitrary thresholds.Conclusion: The absence of convincing evidence of an association between statins and non-CVD outcomes supports leaving the current recommendations unchanged.Primary Funding Source: None.

  View details for PubMedID 30304368

• Massive citations to misleading methods and research tools: Matthew effect, quotation error and citation copying. European journal of epidemiology Ioannidis, J. P. 2018

  View details for PubMedID 30291530

• Disclosures Can Always Be Improved-Reply. JAMA psychiatry Cristea, I., Ioannidis, J. P. 2018

  View details for PubMedID 30285040

• Guidelines Do Not Entangle Practitioners With Unavoidable Conflicts as Authors, and When There Is No Evidence, Just Say So CIRCULATION-CARDIOVASCULAR QUALITY AND OUTCOMES Ioannidis, J. A. 2018; 11 (10): e005205

  View details for PubMedID 30354581

• Examining the readiness of best evidence in medical education guides for integration into educational practice: Ameta-synthesis PERSPECTIVES ON MEDICAL EDUCATION Maggio, L. A., Thomas, A., Chen, H., Ioannidis, J. A., Kanter, S. L., Norton, C., Tannery, N. H., Artino, A. R. 2018; 7 (5): 292–301

  Abstract

  To support evidence-informed education, health professions education (HPE) stakeholders encourage the creation and use of knowledge syntheses or reviews. However, it is unclear if these knowledge syntheses are ready for translation into educational practice. Without understanding the readiness, defined by three criteria-quality, accessibility and relevance-we risk translating weak evidence into practice and/or providing information that is not useful to educators.A librarian searched Web of Science for knowledge syntheses, specifically Best Evidence in Medical Education (BEME) Guides. This meta-synthesis focuses on BEME Guides because of their explicit goal to inform educational practice and policy. Two authors extracted data from all Guides, guided by the 25-item STructured apprOach to the Reporting In healthcare education of Evidence Synthesis (STORIES).Forty-two Guides published in Medical Teacher between 1999 and 2017 were analyzed. No Guide met all STORIES criteria, but all included structured summaries and most described their literature search (n = 39) and study inclusion/exclusion (n = 40) procedures. Eleven Guides reported the presence of theory and/or educational principles, and eight consulted with external subject matter experts. Accessibility to each Guide's full-text and supplemental materials was variable.For a subset of HPE knowledge syntheses, BEME Guides, this meta-synthesis identifies factors that support readiness and indicates potential areas of improvement, such as consistent access to Guides and inclusion of external subject matter experts on the review team. This analysis is useful for understanding the current readiness of HPE knowledge syntheses and informing future reviews to evolve so they can catalyze translation of evidence into educational practice.

  View details for PubMedID 30229529

• RETHINK FUNDING SCIENTIFIC AMERICAN Ioannidis, J. A. 2018; 319 (4): 52–55

  View details for DOI 10.1038/scientificamerican1018-52

  View details for Web of Science ID 000446011100020

  View details for PubMedID 30273303

• Industry-funded versus non-profit-funded critical care research: a meta-epidemiological overview INTENSIVE CARE MEDICINE Janiaud, P., Cristea, I., Ioannidis, J. A. 2018; 44 (10): 1613–27

  View details for DOI 10.1007/s00134-018-5325-3

  View details for Web of Science ID 000447967000002

• Systematic meta-analyses of small RNA profiling studies identify differentially expressed microRNAs in Parkinson's disease Schulz, J., Takousis, P., Wohlers, I., Itua, I. G., Binder, H., Ioannidis, J. P., Perneczky, R., Bertram, L., Lill, C. M. NATURE PUBLISHING GROUP. 2018: 409–10

  View details for Web of Science ID 000489312603168

• Professional Societies Should Abstain From Authorship of Guidelines and Disease Definition Statements CIRCULATION-CARDIOVASCULAR QUALITY AND OUTCOMES Ioannidis, J. A. 2018; 11 (10): e004889

  View details for PubMedID 30354582

• Network meta-analysis of antidepressants Reply LANCET Cipriani, A., Salanti, G., Furukawa, T. A., Turner, E., Ioannidis, J. A., Geddes, J. R. 2018; 392 (10152): 1012–13

  View details for PubMedID 30264703

• The Comparative Effectiveness of Innovative Treatments for Cancer (CEIT-Cancer) project: Rationale and design of the database and the collection of evidence available at approval of novel drugs TRIALS Ladanie, A., Speich, B., Naudet, F., Agarwal, A., Pereira, T. V., Sclafani, F., Martin-Liberal, J., Schmid, T., Ewald, H., Ioannidis, J. A., Bucher, H. C., Kasenda, B., Hemkens, L. G. 2018; 19: 505

  Abstract

  The available evidence on the benefits and harms of novel drugs and therapeutic biologics at the time of approval is reported in publicly available documents provided by the US Food and Drug Administration (FDA). We aimed to create a comprehensive database providing the relevant information required to systematically analyze and assess this early evidence in meta-epidemiological research.We designed a modular and flexible database of systematically collected data. We identified all novel cancer drugs and therapeutic biologics approved by the FDA between 2000 and 2016, recorded regulatory characteristics, acquired the corresponding FDA approval documents, identified all clinical trials reported therein, and extracted trial design characteristics and treatment effects. Herein, we describe the rationale and design of the data collection process, particularly the organization of the data capture, the identification and eligibility assessment of clinical trials, and the data extraction activities.We established a comprehensive database on the comparative effects of drugs and therapeutic biologics approved by the FDA over a time period of 17 years for the treatment of cancer (solid tumors and hematological malignancies). The database provides information on the clinical trial evidence available at the time of approval of novel cancer treatments. The modular nature and structure of the database and the data collection processes allow updates, expansions, and adaption for a continuous meta-epidemiological analysis of novel drugs. The database allows us to systematically evaluate benefits and harms of novel drugs and therapeutic biologics. It provides a useful basis for meta-epidemiological research on the comparative effects of innovative cancer treatments and continuous evaluations of regulatory developments.

  View details for PubMedID 30231912

• The scientists who publish a paper every five days NATURE Ioannidis, J. A., Klavans, R., Boyack, K. W. 2018; 561 (7722): 167–69

  View details for Web of Science ID 000444437900018

  View details for PubMedID 30209384

• The Challenge of Reforming Nutritional Epidemiologic Research JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Ioannidis, J. A. 2018; 320 (10): 969–70

  View details for DOI 10.1001/jama.2018.11025

  View details for Web of Science ID 000444341400005

• The Challenge of Reforming Nutritional Epidemiologic Research. JAMA Ioannidis, J. P. 2018; 320 (10): 969-970

  View details for DOI 10.1001/jama.2018.11025

  View details for PubMedID 30422271

• Lowering the P Value Threshold Reply JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Ioannidis, J. A. 2018; 320 (9): 937–38

  View details for PubMedID 30193273

• Assessment of Pragmatism in Recently Published Randomized Clinical Trials JAMA INTERNAL MEDICINE Janiaud, P., Dal-Re, R., Ioannidis, J. A. 2018; 178 (9): 1278-+

  View details for DOI 10.1001/jamainternmed.2018.3321

  View details for Web of Science ID 000443911200037

• New Principles for Assessing ISSUES IN SCIENCE AND TECHNOLOGY Moher, D., Naudet, F., Cristea, I. A., Miedema, F., Ioannidis, J. P., Goodman, S. N. 2018; 35 (1): 20–23

  View details for Web of Science ID 000447682000006

• Off-label prescription: experience is a gloomy lantern that does not even illuminate its bearer. Author response JOURNAL OF CLINICAL EPIDEMIOLOGY Ladanie, A., Ioannidis, J. A., Stafford, R. S., Ewald, H., Bucher, H. C., Hemkens, L. G., Hemkens, L. G. 2018; 101: 127–28

  View details for PubMedID 29800688

• Industry-funded versus non-profit-funded critical care research: a meta-epidemiological overview. Intensive care medicine Janiaud, P., Cristea, I., Ioannidis, J. P. 2018

  Abstract

  PURPOSE: To study the landscape of funding in intensive care research and assess whether the reported outcomes of industry-funded randomized controlled trials (RCTs) are more favorable.METHODS: We systematically assembled meta-analyses evaluating any type of intervention in the critical care setting and reporting the source of funding for each included RCT. Furthermore, when the intervention was a drug or biologic, we searched also the original RCT articles, when their funding information was unavailable in the meta-analysis. We then qualitatively summarized the sources of funding. For binary outcomes, separate summary odds ratios were calculated for trials with and without industry funding. We then calculated the ratio of odds ratios (RORs) and the summary ROR (sROR) across topics. ROR<1 implies that the experimental intervention is relatively more favorable in trials with industry funding compared with trials without industry funding. For RCTs included in the ROR analysis, we also examined the conclusions of their abstract.RESULTS: Across 67 topics with 568 RCTs, 88 were funded by industry and another 73 had both industry and non-profit funding. Across 33 topics with binary outcomes, the sROR was 1.10 [95% CI (0.96-1.26), I2=1%]. Conclusions were not significantly more commonly unfavorable for the experimental arm interventions in industry-funded trials (21.3%) compared with trials without industry funding (18.2%).CONCLUSION: Industry-funded RCTs are the minority in intensive care. We found no evidence that industry-funded trials in intensive care yield more favorable results or are less likely to reach unfavorable conclusions.

  View details for PubMedID 30151688

• Populating the Data ArK: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles PLOS ONE Hardwicke, T. E., Ioannidis, J. A. 2018; 13 (8)

  View details for DOI 10.1371/journal.pone.0201856

  View details for Web of Science ID 000440778600109

• Mapping risk factors for depression across the lifespan: An umbrella review of evidence from meta-analyses and Mendelian randomization studies JOURNAL OF PSYCHIATRIC RESEARCH Kohler, C. A., Evangelou, E., Stubbs, B., Solmi, M., Veronese, N., Belbasis, L., Bortolato, B., Melo, M. A., Coelho, C. A., Fernandes, B. S., Olfson, M., Ioannidis, J. A., Carvalho, A. F. 2018; 103: 189–207

  Abstract

  The development of depression may involve a complex interplay of environmental and genetic risk factors. PubMed and PsycInfo databases were searched from inception through August 3, 2017, to identify meta-analyses and Mendelian randomization (MR) studies of environmental risk factors associated with depression. For each eligible meta-analysis, we estimated the summary effect size and its 95% confidence interval (CI) by random-effects modeling, the 95% prediction interval, heterogeneity with I2, and evidence of small-study effects and excess significance bias. Seventy meta-analytic reviews met the eligibility criteria and provided 134 meta-analyses for associations from 1283 primary studies. While 109 associations were nominally significant (P < 0.05), only 8 met the criteria for convincing evidence and, when limited to prospective studies, convincing evidence was found in 6 (widowhood, physical abuse during childhood, obesity, having 4-5 metabolic risk factors, sexual dysfunction, job strain). In studies in which depression was assessed through a structured diagnostic interview, only associations with widowhood, job strain, and being a Gulf War veteran were supported by convincing evidence. Additionally, 8 MR studies were included and provided no consistent evidence for the causal effects of obesity, smoking, and alcohol consumption. The proportion of variance explained by genetic risk factors was extremely small (0.1-0.4%), which limited the evidence provided by the MR studies. Our findings suggest that despite the large number of putative risk factors investigated in the literature, few associations were supported by robust evidence. The current findings may have clinical and research implications for the early identification of individuals at risk for depression.

  View details for PubMedID 29886003

• Randomized controlled trials: Often flawed, mostly useless, clearly indispensable: A commentary on Deaton and Cartwright SOCIAL SCIENCE & MEDICINE Ioannidis, J. A. 2018; 210: 53–56

  View details for DOI 10.1016/j.socscimed.2018.04.029

  View details for Web of Science ID 000441652900012

• Why Cochrane should prioritise sharing data BMJ-BRITISH MEDICAL JOURNAL Shokraneh, F., Adams, C. E., Clarke, M., Amato, L., Bastian, H., Beller, E., Brassey, J., Buchbinder, R., Davoli, M., Del Mar, C., Glasziou, P., Gluud, C., Heneghan, C., Hoffmann, T., Ioannidis, J. A., Jayaram, M., Kwong, J., Moher, D., Ota, E., Sheriff, R., Vale, L., Ben Goldacre 2018; 362: k3229

  View details for PubMedID 30061322

• Assessment of Pragmatism in Recently Published Randomized Clinical Trials. JAMA internal medicine Janiaud, P., Dal-Re, R., Ioannidis, J. P. 2018

  View details for PubMedID 30039169

• The association of depression and all-cause and cause-specific mortality: an umbrella review of systematic reviews and meta-analyses BMC MEDICINE Machado, M. O., Veronese, N., Sanches, M., Stubbs, B., Koyanagi, A., Thompson, T., Tzoulaki, I., Solmi, M., Vancampfort, D., Schuch, F. B., Maes, M., Fava, G. A., Ioannidis, J. A., Carvalho, A. F. 2018; 16: 112

  Abstract

  Depression is a prevalent and disabling mental disorder that frequently co-occurs with a wide range of chronic conditions. Evidence has suggested that depression could be associated with excess all-cause mortality across different settings and populations, although the causality of these associations remains unclear.We conducted an umbrella review of systematic reviews and meta-analyses of observational studies. PubMed, PsycINFO, and Embase electronic databases were searched through January 20, 2018. Systematic reviews and meta-analyses that investigated associations of depression and all-cause and cause-specific mortality were selected for the review. The evidence was graded as convincing, highly suggestive, suggestive, or weak based on quantitative criteria that included an assessment of heterogeneity, 95% prediction intervals, small-study effects, and excess significance bias.A total of 26 references providing 2 systematic reviews and data for 17 meta-analytic estimates met inclusion criteria (19 of them on all-cause mortality); data from 246 unique studies (N = 3,825,380) were synthesized. All 17 associations had P < 0.05 per random effects summary effects, but none of them met criteria for convincing evidence. Associations of depression and all-cause mortality in patients after acute myocardial infarction, in individuals with heart failure, in cancer patients as well as in samples from mixed settings met criteria for highly suggestive evidence. However, none of the associations remained supported by highly suggestive evidence in sensitivity analyses that considered studies employing structured diagnostic interviews. In addition, associations of depression and all-cause mortality in cancer and post-acute myocardial infarction samples were supported only by suggestive evidence when studies that tried to adjust for potential confounders were considered.Even though associations between depression and mortality have nominally significant results in all assessed settings and populations, the evidence becomes weaker when focusing on studies that used structured interviews and those that tried to adjust for potential confounders. A causal effect of depression on all-cause and cause-specific mortality remains unproven, and thus interventions targeting depression are not expected to result in lower mortality rates at least based on current evidence from observational studies.

  View details for PubMedID 30025524

• Conflict of Interest in Nutrition Research-Reply. JAMA Ioannidis, J. P., Trepanowski, J. F. 2018; 320 (1): 94–95

  View details for PubMedID 29971394

• Conflict of Interest in Nutrition Research Reply JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Ioannidis, J. A., Trepanowski, J. F. 2018; 320 (1): 94–95

  View details for DOI 10.1001/jama.2018.5678

  View details for Web of Science ID 000437219400028

• Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How. Advances in nutrition (Bethesda, Md.) Trepanowski, J. F., Ioannidis, J. P. 2018; 9 (4): 367–77

  Abstract

  A large majority of human nutrition research uses nonrandomized observational designs, but this has led to little reliable progress. This is mostly due to many epistemologic problems, the most important of which are as follows: difficulty detecting small (or even tiny) effect sizes reliably for nutritional risk factors and nutrition-related interventions; difficulty properly accounting for massive confounding among many nutrients, clinical outcomes, and other variables; difficulty measuring diet accurately; and suboptimal research reporting. Tiny effect sizes and massive confounding are largely unfixable problems that narrowly confine the scenarios in which nonrandomized observational research is useful. Although nonrandomized studies and randomized trials have different priorities (assessment of long-term causality compared with assessment of treatment effects), the odds for obtaining reliable information with the former are limited. Randomized study designs should therefore largely replace nonrandomized studies in human nutrition research going forward. To achieve this, many of the limitations that have traditionally plagued most randomized trials in nutrition, such as small sample size, short length of follow-up, high cost, and selective reporting, among others, must be overcome. Pivotal megatrials with tens of thousands of participants and lifelong follow-up are possible in nutrition science with proper streamlining of operational costs. Fixable problems that have undermined observational research, such as dietary measurement error and selective reporting, need to be addressed in randomized trials. For focused questions in which dietary adherence is important to maximize, trials with direct observation of participants in experimental in-house settings may offer clean answers on short-term metabolic outcomes. Other study designs of randomized trials to consider in nutrition include registry-based designs and "N-of-1" designs. Mendelian randomization designs may also offer some more reliable leads for testing interventions in trials. Collectively, an improved randomized agenda may clarify many things in nutrition science that might never be answered credibly with nonrandomized observational designs.

  View details for PubMedID 30032218

• Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How ADVANCES IN NUTRITION Trepanowski, J. F., Ioannidis, J. A. 2018; 9 (4): 367–77

  View details for DOI 10.1093/advances/nmy014

  View details for Web of Science ID 000444712000001

• Physical activity and cancer: an umbrella review of the literature including 22 major anatomical sites and 770 000 cancer cases BRITISH JOURNAL OF SPORTS MEDICINE Machado de Rezende, L., de Sa, T., Markozannes, G., Rey-Lopez, J., Lee, I., Tsilidis, K. K., Ioannidis, J. A., Eluf-Neto, J. 2018; 52 (13): 826–33

  Abstract

  To provide an overview of the breadth and validity of claimed associations between physical activity and risk of developing or dying from cancer.Umbrella review.We searched Medline, Embase, Cochrane Database and Web of Science.Systematic reviews about physical activity and cancer incidence and cancer mortality in different body sites among general population.We included 19 reviews covering 22 cancer sites, 26 exposure-outcome pairs meta-analyses and 541 original studies. Physical activity was associated with lower risk of seven cancer sites (colon, breast, endometrial, lung, oesophageal, pancreas and meningioma). Only colon (a protective association with recreational physical activity) and breast cancer (a protective association with overall physical activity) were supported by strong evidence and highly suggestive evidence, respectively. Evidence from endometrial, lung, oesophageal, pancreas and meningioma presented hints of uncertainty and bias in the literature (eg, not reaching P values<10-6) showing large between-study heterogeneity and/or not demonstrating a definite direction for the effect when 95% prediction intervals were considered. Four of the 26 meta-analyses showed small study effects and 4 showed excess significance.Physical activity is associated with a lower risk of several cancers, but only colon and breast cancer associations were supported by strong or highly suggestive evidence, respectively. Evidence from other cancer sites was less consistent, presenting hints of uncertainty and/or bias.

  View details for PubMedID 29146752

• Prevalence and outcomes of incidental imaging findings: umbrella review BMJ-BRITISH MEDICAL JOURNAL O'Sullivan, J. W., Muntinga, T., Grigg, S., Ioannidis, J. A. 2018; 361: k2387

  Abstract

  To provide an overview of the evidence on prevalence and outcomes of incidental imaging findings.Umbrella review of systematic reviews.Searches of MEDLINE, EMBASE up to August 2017; screening of references in included papers.Criteria included systematic reviews and meta-analyses of observational studies that gave a prevalence of incidental abnormalities ("incidentalomas"). An incidental imaging finding was defined as an imaging abnormality in a healthy, asymptomatic patient or an imaging abnormality in a symptomatic patient, where the abnormality was not apparently related to the patient's symptoms. Primary studies that measured the prevalence of incidentalomas in patients with a history of malignancy were also considered in sensitivity analyses.20 systematic reviews (240 primary studies) were identified from 7098 references from the database search. Fifteen systematic reviews provided data to quantify the prevalence of incidentalomas, whereas 18 provided data to quantify the outcomes of incidentalomas (13 provided both). The prevalence of incidentalomas varied substantially between imaging tests; it was less than 5% for chest computed tomography for incidental pulmonary embolism in patients with and without cancer and whole body positron emission tomography (PET) or PET/computed tomography (for patients with and without cancer). Conversely, incidentalomas occurred in more than a third of images in cardiac magnetic resonance imaging (MRI), chest computed tomography (for incidentalomas of thorax, abdomen, spine, or heart), and computed tomography colonoscopy (for extra-colonic incidentalomas). Intermediate rates occurred with MRI of the spine (22%) and brain (22%). The rate of malignancy in incidentalomas varied substantially between organs; the prevalence of malignancy was less than 5% in incidentalomas of the brain, parotid, and adrenal gland. Extra-colonic, prostatic, and colonic incidentalomas were malignant between 10% and 20% of the time, whereas renal, thyroid, and ovarian incidentalomas were malignant around a quarter of the time. Breast incidentalomas had the highest percentage of malignancy (42%, 95% confidence interval 31% to 54%). Many assessments had high between-study heterogeneity (15 of 20 meta-analyses with I2 >50%).There is large variability across different imaging techniques both in the prevalence of incidentalomas and in the prevalence of malignancy for specific organs. This umbrella review will aid clinicians and patients weigh up the pros and cons of requesting imaging scans and will help with management decisions after an incidentaloma diagnosis. Our results can underpin the creation of guidelines to assist these decisions.PROSPERO: CRD42017075679.

  View details for PubMedID 29914908

• Larger effect sizes in nonrandomized studies are associated with higher rates of EMA licensing approval JOURNAL OF CLINICAL EPIDEMIOLOGY Djulbegovic, B., Glasziou, P., Klocksieben, F. A., Reljic, T., VanDenBergh, M., Mhaskar, R., Ioannidis, J. A., Chalmers, L. 2018; 98: 24–32

  Abstract

  The aim of this study was to evaluate how often the European Medicines Agency (EMA) has authorized drugs based on nonrandomized studies and whether there is an association between treatment effects and EMA preference for further testing in randomized clinical trials (RCTs).We reviewed all initial marketing authorizations in the EMA database on human medicines between 1995 and 2015 and included authorizations granted without randomized data. We extracted data on treatment effects and EMA preference for further testing in RCTs.Of 723 drugs, 51 were authorized based on nonrandomized data. These 51 drugs were licensed for 71 indications. In the 51 drug-indication pairs with no preference for further RCT testing, effect estimates were large [odds ratio (OR): 12.0 (95% confidence interval {CI}: 8.1-17.9)] compared to effect estimates in the 20 drug-indication pairs for which future RCTs were preferred [OR: 4.3 (95% CI 2.8-6.6)], with a significant difference between effects (P = 0.0005).Nonrandomized data were used for 7% of EMA drug approvals. Larger effect sizes were associated with greater likelihood of approval based on nonrandomized data alone. We did not find a clear treatment effect threshold for drug approval without RCT evidence.

  View details for PubMedID 29432860

• Meta-analyses in environmental and occupational health OCCUPATIONAL AND ENVIRONMENTAL MEDICINE Ioannidis, J. A. 2018; 75 (6): 443–45

  View details for DOI 10.1136/oemed-2016-104128

  View details for Web of Science ID 000433243000009

• Improving Disclosure of Financial Conflicts of Interest for Research on Psychosocial Interventions JAMA PSYCHIATRY Cristea, I., Ioannidis, J. A. 2018; 75 (6): 541–42

  View details for DOI 10.1001/jamapsychiatry.2018.0382

  View details for Web of Science ID 000434408700001

• Blood Pressure Measurement and Hypertension Diagnosis in the 2017 US Guidelines First Things First HYPERTENSION Stergiou, G., Palatini, P., Asmar, R., de la Sierra, A., Myers, M., Shennan, A., Wang, J., O'Brien, E., Parati, G. 2018; 71 (6): 963–65

  View details for DOI 10.1161/HYPERTENSIONAHA.118.10853

  View details for Web of Science ID 000441020300008

• Antidepressants might work for people with major depression: where do we go from here? LANCET PSYCHIATRY Cipriani, A., Salanti, G., Furukawa, T. A., Egger, M., Leucht, S., Ruhe, H. G., Turner, E. H., Atkinson, L. Z., Chaimani, A., Higgins, J. T., Ogawa, Y., Takeshima, N., Hayasaka, Y., Imai, H., Shinohara, K., Tajika, A., Ioannidis, J. A., Geddes, J. R. 2018; 5 (6): 461–63

  View details for PubMedID 29628364

• Probability of major depression diagnostic classification using semi-structured versus fully structured diagnostic interviews BRITISH JOURNAL OF PSYCHIATRY Levis, B., Benedetti, A., Riehm, K. E., Saadat, N., Levis, A. W., Azar, M., Rice, D. B., Chiovitti, M. J., Sanchez, T. A., Cuijpers, P., Gilbody, S., Ioannidis, J. A., Kloda, L. A., McMillan, D., Patten, S. B., Shrier, I., Steele, R. J., Ziegelstein, R. C., Akena, D. H., Arroll, B., Ayalon, L., Baradaran, H. R., Baron, M., Beraldi, A., Bombardier, C. H., Butterworth, P., Carter, G., Chagas, M. H., Chan, J. N., Cholera, R., Chowdhary, N., Clover, K., Conwell, Y., de Man-van Ginkel, J. M., Delgadillo, J., Fann, J. R., Fischer, F. H., Fischler, B., Fung, D., Gelaye, B., Goodyear-Smith, F., Greeno, C. G., Hall, B. J., Hambridge, J., Harrison, P. A., Hegerl, U., Hides, L., Hobfoll, S. E., Hudson, M., Hyphantis, T., Inagaki, M., Ismail, K., Jette, N., Khamseh, M. E., Kiely, K. M., Lamers, F., Liu, S., Lotrakul, M., Loureiro, S. R., Lowe, B., Marsh, L., McGuire, A., Sidik, S., Munhoz, T. N., Muramatsu, K., Osorio, F. L., Patel, V., Pence, B. W., Persoons, P., Picardi, A., Rooney, A. G., Santos, I. S., Shaaban, J., Sidebottom, A., Simning, A., Stafford, L., Sung, S., Tan, P., Turner, A., van der Feltz-Cornelis, C. M., van Weert, H. C., Vohringer, P. A., White, J., Whooley, M. A., Winkley, K., Yamada, M., Zhang, Y., Thombs, B. D. 2018; 212 (6): 377–85

  Abstract

  Different diagnostic interviews are used as reference standards for major depression classification in research. Semi-structured interviews involve clinical judgement, whereas fully structured interviews are completely scripted. The Mini International Neuropsychiatric Interview (MINI), a brief fully structured interview, is also sometimes used. It is not known whether interview method is associated with probability of major depression classification.AimsTo evaluate the association between interview method and odds of major depression classification, controlling for depressive symptom scores and participant characteristics.Data collected for an individual participant data meta-analysis of Patient Health Questionnaire-9 (PHQ-9) diagnostic accuracy were analysed and binomial generalised linear mixed models were fit.A total of 17 158 participants (2287 with major depression) from 57 primary studies were analysed. Among fully structured interviews, odds of major depression were higher for the MINI compared with the Composite International Diagnostic Interview (CIDI) (odds ratio (OR) = 2.10; 95% CI = 1.15-3.87). Compared with semi-structured interviews, fully structured interviews (MINI excluded) were non-significantly more likely to classify participants with low-level depressive symptoms (PHQ-9 scores ≤6) as having major depression (OR = 3.13; 95% CI = 0.98-10.00), similarly likely for moderate-level symptoms (PHQ-9 scores 7-15) (OR = 0.96; 95% CI = 0.56-1.66) and significantly less likely for high-level symptoms (PHQ-9 scores ≥16) (OR = 0.50; 95% CI = 0.26-0.97).The MINI may identify more people as depressed than the CIDI, and semi-structured and fully structured interviews may not be interchangeable methods, but these results should be replicated.Declaration of interestDrs Jetté and Patten declare that they received a grant, outside the submitted work, from the Hotchkiss Brain Institute, which was jointly funded by the Institute and Pfizer. Pfizer was the original sponsor of the development of the PHQ-9, which is now in the public domain. Dr Chan is a steering committee member or consultant of Astra Zeneca, Bayer, Lilly, MSD and Pfizer. She has received sponsorships and honorarium for giving lectures and providing consultancy and her affiliated institution has received research grants from these companies. Dr Hegerl declares that within the past 3 years, he was an advisory board member for Lundbeck, Servier and Otsuka Pharma; a consultant for Bayer Pharma; and a speaker for Medice Arzneimittel, Novartis, and Roche Pharma, all outside the submitted work. Dr Inagaki declares that he has received grants from Novartis Pharma, lecture fees from Pfizer, Mochida, Shionogi, Sumitomo Dainippon Pharma, Daiichi-Sankyo, Meiji Seika and Takeda, and royalties from Nippon Hyoron Sha, Nanzando, Seiwa Shoten, Igaku-shoin and Technomics, all outside of the submitted work. Dr Yamada reports personal fees from Meiji Seika Pharma Co., Ltd., MSD K.K., Asahi Kasei Pharma Corporation, Seishin Shobo, Seiwa Shoten Co., Ltd., Igaku-shoin Ltd., Chugai Igakusha and Sentan Igakusha, all outside the submitted work. All other authors declare no competing interests. No funder had any role in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review or approval of the manuscript; and decision to submit the manuscript for publication.

  View details for PubMedID 29717691

• Poor performance of clinical prediction models: the harm of commonly applied methods JOURNAL OF CLINICAL EPIDEMIOLOGY Steyerberg, E. W., Uno, H., Ioannidis, J. A., van Calster, B., Collaborators 2018; 98: 133–43

  Abstract

  To evaluate limitations of common statistical modeling approaches in deriving clinical prediction models and explore alternative strategies.A previously published model predicted the likelihood of having a mutation in germline DNA mismatch repair genes at the time of diagnosis of colorectal cancer. This model was based on a cohort where 38 mutations were found among 870 participants, with validation in an independent cohort with 35 mutations. The modeling strategy included stepwise selection of predictors from a pool of over 37 candidate predictors and dichotomization of continuous predictors. We simulated this strategy in small subsets of a large contemporary cohort (2,051 mutations among 19,866 participants) and made comparisons to other modeling approaches. All models were evaluated according to bias and discriminative ability (concordance index, c) in independent data.We found over 50% bias for five of six originally selected predictors, unstable model specification, and poor performance at validation (median c = 0.74). A small validation sample hampered stable assessment of performance. Model prespecification based on external knowledge and using continuous predictors led to better performance (c = 0.836 and c = 0.852 with 38 and 2,051 events respectively).Prediction models perform poorly if based on small numbers of events and developed with common but suboptimal statistical approaches. Alternative modeling strategies to best exploit available predictive information need wider implementation, with collaborative research to increase sample sizes.

  View details for PubMedID 29174118

• Lack of evidence to favor specific preventive interventions in psychosis: a network meta-analysis WORLD PSYCHIATRY Davies, C., Cipriani, A., Ioannidis, J. A., Radua, J., Stahl, D., Provenzani, U., McGuire, P., Fusar-Poli, P. 2018; 17 (2): 196–209

  Abstract

  Preventing psychosis in patients at clinical high risk may be a promising avenue for pre-emptively ameliorating outcomes of the most severe psychiatric disorder. However, information on how each preventive intervention fares against other currently available treatment options remains unavailable. The aim of the current study was to quantify the consistency and magnitude of effects of specific preventive interventions for psychosis, comparing different treatments in a network meta-analysis. PsycINFO, Web of Science, Cochrane Central Register of Controlled Trials, and unpublished/grey literature were searched up to July 18, 2017, to identify randomized controlled trials conducted in individuals at clinical high risk for psychosis, comparing different types of intervention and reporting transition to psychosis. Two reviewers independently extracted data. Data were synthesized using network meta-analyses. The primary outcome was transition to psychosis at different time points and the secondary outcome was treatment acceptability (dropout due to any cause). Effect sizes were reported as odds ratios and 95% confidence intervals (CIs). Sixteen studies (2,035 patients, 57% male, mean age 20.1 years) reported on risk of transition. The treatments tested were needs-based interventions (NBI); omega-3 + NBI; ziprasidone + NBI; olanzapine + NBI; aripiprazole + NBI; integrated psychological interventions; family therapy + NBI; D-serine + NBI; cognitive behavioural therapy, French & Morrison protocol (CBT-F) + NBI; CBT-F + risperidone + NBI; and cognitive behavioural therapy, van der Gaag protocol (CBT-V) + CBT-F + NBI. The network meta-analysis showed no evidence of significantly superior efficacy of any one intervention over the others at 6 and 12 months (insufficient data were available after 12 months). Similarly, there was no evidence for intervention differences in acceptability at either time point. Tests for inconsistency were non-significant and sensitivity analyses controlling for different clustering of interventions and biases did not materially affect the interpretation of the results. In summary, this study indicates that, to date, there is no evidence that any specific intervention is particularly effective over the others in preventing transition to psychosis. Further experimental research is needed.

  View details for PubMedID 29856551

• P values in display items are ubiquitous and almost invariably significant: A survey of top science journals PLOS ONE Cristea, I., Ioannidis, J. A. 2018; 13 (5): e0197440

  Abstract

  P values represent a widely used, but pervasively misunderstood and fiercely contested method of scientific inference. Display items, such as figures and tables, often containing the main results, are an important source of P values. We conducted a survey comparing the overall use of P values and the occurrence of significant P values in display items of a sample of articles in the three top multidisciplinary journals (Nature, Science, PNAS) in 2017 and, respectively, in 1997. We also examined the reporting of multiplicity corrections and its potential influence on the proportion of statistically significant P values. Our findings demonstrated substantial and growing reliance on P values in display items, with increases of 2.5 to 14.5 times in 2017 compared to 1997. The overwhelming majority of P values (94%, 95% confidence interval [CI] 92% to 96%) were statistically significant. Methods to adjust for multiplicity were almost non-existent in 1997, but reported in many articles relying on P values in 2017 (Nature 68%, Science 48%, PNAS 38%). In their absence, almost all reported P values were statistically significant (98%, 95% CI 96% to 99%). Conversely, when any multiplicity corrections were described, 88% (95% CI 82% to 93%) of reported P values were statistically significant. Use of Bayesian methods was scant (2.5%) and rarely (0.7%) articles relied exclusively on Bayesian statistics. Overall, wider appreciation of the need for multiplicity corrections is a welcome evolution, but the rapid growth of reliance on P values and implausibly high rates of reported statistical significance are worrisome.

  View details for PubMedID 29763472

• An overview of methods for network meta-analysis using individual participant data: when do benefits arise? STATISTICAL METHODS IN MEDICAL RESEARCH Debray, T. A., Schuit, E., Efthimiou, O., Reitsma, J. B., Ioannidis, J. A., Salanti, G., Moons, K. M., GetReal Workpackage 2018; 27 (5): 1351–64

  Abstract

  Network meta-analysis (NMA) is a common approach to summarizing relative treatment effects from randomized trials with different treatment comparisons. Most NMAs are based on published aggregate data (AD) and have limited possibilities for investigating the extent of network consistency and between-study heterogeneity. Given that individual participant data (IPD) are considered the gold standard in evidence synthesis, we explored statistical methods for IPD-NMA and investigated their potential advantages and limitations, compared with AD-NMA. We discuss several one-stage random-effects NMA models that account for within-trial imbalances, treatment effect modifiers, missing response data and longitudinal responses. We illustrate all models in a case study of 18 antidepressant trials with a continuous endpoint (the Hamilton Depression Score). All trials suffered from drop-out; missingness of longitudinal responses ranged from 21 to 41% after 6 weeks follow-up. Our results indicate that NMA based on IPD may lead to increased precision of estimated treatment effects. Furthermore, it can help to improve network consistency and explain between-study heterogeneity by adjusting for participant-level effect modifiers and adopting more advanced models for dealing with missing response data. We conclude that implementation of IPD-NMA should be considered when trials are affected by substantial drop-out rate, and when treatment effects are potentially influenced by participant-level covariates.

  View details for PubMedID 27487843

• Neurosurgical Randomized Controlled Trials-Distance Travelled NEUROSURGERY Azad, T. D., Veeravagu, A., Mittal, V., Esparza, R., Johnson, E., Ioannidis, J. A., Grant, G. A. 2018; 82 (5): 604–12

  View details for DOI 10.1093/neuros/nyx319

  View details for Web of Science ID 000439693800023

• Evidence-based medicine and big genomic data HUMAN MOLECULAR GENETICS Ioannidis, J. A., Khoury, M. J. 2018; 27 (R1): R2–R7

  View details for DOI 10.1093/hmg/ddy065

  View details for Web of Science ID 000431884200002

• All science should inform policy and regulation PLOS MEDICINE Ioannidis, J. A. 2018; 15 (5)

  View details for DOI 10.1371/journal.pmed.1002576

  View details for Web of Science ID 000434236000001

• Protect us from poor-quality medical research HUMAN REPRODUCTION Ioannidis, J. A., Bhattacharya, S., Evers, J. H., van der Veen, F., Somigliana, E., Barratt, C. R., Bontempi, G., Baird, D. T., Crosignani, P., Devroey, P., Diedrich, K., Farquharson, R. G., Fraser, L. R., Geraedts, J. M., Gianaroli, L., La Vecchia, C., Lundin, K., Magli, C., Negri, E., Sunde, A., Tapanainen, J. S., Tarlatzis, B. C., Van Steirteghem, A., Veiga, A., ESHRE Capri Workshop Grp 2018; 33 (5): 770–76

  Abstract

  Much of the published medical research is apparently flawed, cannot be replicated and/or has limited or no utility. This article presents an overview of the current landscape of biomedical research, identifies problems associated with common study designs and considers potential solutions. Randomized clinical trials, observational studies, systematic reviews and meta-analyses are discussed in terms of their inherent limitations and potential ways of improving their conduct, analysis and reporting. The current emphasis on statistical significance needs to be replaced by sound design, transparency and willingness to share data with a clear commitment towards improving the quality and utility of clinical research.

  View details for DOI 10.1093/humrep/dey056

  View details for Web of Science ID 000432285200002

  View details for PubMedID 29617882

• Systematic identification of correlates of HIV infection: an X-wide association study AIDS Patel, C. J., Bhattacharya, J., Ioannidis, J. A., Bendavid, E. 2018; 32 (7): 933–43

  Abstract

  Better identification of at-risk groups could benefit HIV-1 care programmes. We systematically identified HIV-1 risk factors in two nationally representative cohorts of women in the Demographic and Health Surveys.We identified and replicated the association of 1415 social, economic, environmental, and behavioral factors with HIV-1 status. We used the 2007 and 2013-2014 surveys conducted among 5715 and 15 433 Zambian women, respectively (688 shared factors). We used false discovery rate criteria to identify factors that are strongly associated with HIV-1 in univariate and multivariate models of the entire population, as well as in subgroups stratified by wealth, residence, age, and past HIV-1 testing.In the univariate analysis, we identified 102 and 182 variables that are associated with HIV-1 in the two surveys, respectively (79 factors were associated in both). Factors that were associated with HIV-1 status in full-sample analyses and in subgroups include being formerly married (adjusted OR 2007, 2.8, P < 10; 2013-2014 2.8, P < 10), widowhood (aOR 3.7, P < 10; and 4.2, P < 10), genital ulcers within 12 months (aOR 2.4, P < 10; and 2.2, P < 10), and having a woman head of the household (aOR 1.7, P < 10; and 2.1, P < 10), while owning a bicycle (aOR 0.6, P < 10; and 0.6, P < 10) and currently breastfeeding (aOR 0.5, P < 10; and 0.4, P < 10) were associated with decreased risk. Area under the curve for HIV-1 positivity was 0.76-0.82.Our X-wide association study identifies under-recognized factors related to HIV-1 infection, including widowhood, breastfeeding, and gender of head of the household. These features could be used to improve HIV-1 identification programs.

  View details for PubMedID 29424772

• Randomized controlled trials: Often flawed, mostly useless, clearly indispensable: A commentary on Deaton and Cartwright. Social science & medicine (1982) Ioannidis, J. P. 2018

  View details for PubMedID 29776687

• Improving Disclosure of Financial Conflicts of Interest for Research on Psychosocial Interventions. JAMA psychiatry Cristea, I., Ioannidis, J. P. 2018

  View details for PubMedID 29641818

• Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis LANCET Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Levcht, S., Ruhe, H. G., Turner, E. H., Higgins, J. T., Egger, M., Takeshima, N., Hayasaka, Y., Imai, H., Shinohara, K., Tajika, A., Ioannidis, J. A., Geddes, J. R. 2018; 391 (10128): 1357–66

  Abstract

  Major depressive disorder is one of the most common, burdensome, and costly psychiatric disorders worldwide in adults. Pharmacological and non-pharmacological treatments are available; however, because of inadequate resources, antidepressants are used more frequently than psychological interventions. Prescription of these agents should be informed by the best available evidence. Therefore, we aimed to update and expand our previous work to compare and rank antidepressants for the acute treatment of adults with unipolar major depressive disorder.We did a systematic review and network meta-analysis. We searched Cochrane Central Register of Controlled Trials, CINAHL, Embase, LILACS database, MEDLINE, MEDLINE In-Process, PsycINFO, the websites of regulatory agencies, and international registers for published and unpublished, double-blind, randomised controlled trials from their inception to Jan 8, 2016. We included placebo-controlled and head-to-head trials of 21 antidepressants used for the acute treatment of adults (≥18 years old and of both sexes) with major depressive disorder diagnosed according to standard operationalised criteria. We excluded quasi-randomised trials and trials that were incomplete or included 20% or more of participants with bipolar disorder, psychotic depression, or treatment-resistant depression; or patients with a serious concomitant medical illness. We extracted data following a predefined hierarchy. In network meta-analysis, we used group-level data. We assessed the studies' risk of bias in accordance to the Cochrane Handbook for Systematic Reviews of Interventions, and certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation framework. Primary outcomes were efficacy (response rate) and acceptability (treatment discontinuations due to any cause). We estimated summary odds ratios (ORs) using pairwise and network meta-analysis with random effects. This study is registered with PROSPERO, number CRD42012002291.We identified 28 552 citations and of these included 522 trials comprising 116 477 participants. In terms of efficacy, all antidepressants were more effective than placebo, with ORs ranging between 2·13 (95% credible interval [CrI] 1·89-2·41) for amitriptyline and 1·37 (1·16-1·63) for reboxetine. For acceptability, only agomelatine (OR 0·84, 95% CrI 0·72-0·97) and fluoxetine (0·88, 0·80-0·96) were associated with fewer dropouts than placebo, whereas clomipramine was worse than placebo (1·30, 1·01-1·68). When all trials were considered, differences in ORs between antidepressants ranged from 1·15 to 1·55 for efficacy and from 0·64 to 0·83 for acceptability, with wide CrIs on most of the comparative analyses. In head-to-head studies, agomelatine, amitriptyline, escitalopram, mirtazapine, paroxetine, venlafaxine, and vortioxetine were more effective than other antidepressants (range of ORs 1·19-1·96), whereas fluoxetine, fluvoxamine, reboxetine, and trazodone were the least efficacious drugs (0·51-0·84). For acceptability, agomelatine, citalopram, escitalopram, fluoxetine, sertraline, and vortioxetine were more tolerable than other antidepressants (range of ORs 0·43-0·77), whereas amitriptyline, clomipramine, duloxetine, fluvoxamine, reboxetine, trazodone, and venlafaxine had the highest dropout rates (1·30-2·32). 46 (9%) of 522 trials were rated as high risk of bias, 380 (73%) trials as moderate, and 96 (18%) as low; and the certainty of evidence was moderate to very low.All antidepressants were more efficacious than placebo in adults with major depressive disorder. Smaller differences between active drugs were found when placebo-controlled trials were included in the analysis, whereas there was more variability in efficacy and acceptability in head-to-head trials. These results should serve evidence-based practice and inform patients, physicians, guideline developers, and policy makers on the relative merits of the different antidepressants.National Institute for Health Research Oxford Health Biomedical Research Centre and the Japan Society for the Promotion of Science.

  View details for PubMedID 29477251

• Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic? BMC MEDICINE Dal-Re, R., Janiaud, P., Ioannidis, J. A. 2018; 16: 49

  Abstract

  Pragmatic randomized controlled trials (RCTs) mimic usual clinical practice and they are critical to inform decision-making by patients, clinicians and policy-makers in real-world settings. Pragmatic RCTs assess effectiveness of available medicines, while explanatory RCTs assess efficacy of investigational medicines. Explanatory and pragmatic are the extremes of a continuum. This debate article seeks to evaluate and provide recommendation on how to characterize pragmatic RCTs in light of the current landscape of RCTs. It is supported by findings from a PubMed search conducted in August 2017, which retrieved 615 RCTs self-labeled in their titles as "pragmatic" or "naturalistic". We focused on 89 of these trials that assessed medicines (drugs or biologics).36% of these 89 trials were placebo-controlled, performed before licensing of the medicine, or done in a single-center. In our opinion, such RCTs overtly deviate from usual care and pragmatism. It follows, that the use of the term 'pragmatic' to describe them, conveys a misleading message to patients and clinicians. Furthermore, many other trials among the 615 coined as 'pragmatic' and assessing other types of intervention are plausibly not very pragmatic; however, this is impossible for a reader to tell without access to the full protocol and insider knowledge of the trial conduct. The degree of pragmatism should be evaluated by the trial investigators themselves using the PRECIS-2 tool, a tool that comprises 9 domains, each scored from 1 (very explanatory) to 5 (very pragmatic).To allow for a more appropriate characterization of the degree of pragmatism in clinical research, submissions of RCTs to funders, research ethics committees and to peer-reviewed journals should include a PRECIS-2 tool assessment done by the trial investigators. Clarity and accuracy on the extent to which a RCT is pragmatic will help understand how much it is relevant to real-world practice.

  View details for PubMedID 29615035

• The Obesity Paradox: A Misleading Term That Should Be Abandoned OBESITY Flegal, K. M., Ioannidis, J. A. 2018; 26 (4): 629–30

  Abstract

  The term "obesity paradox" is a figure of speech, not a scientific term. The term has no precise definition and has been used to describe numerous observations that have little in common other than the finding of an association of obesity with a favorable outcome. The terminology has led to misunderstandings among researchers and the public alike. It's time for authors and editors to abandon the use of this term. Simply labeling counterintuitive findings as the "obesity paradox" adds no value. Unexpected findings should not be viewed negatively; such findings can lead to new knowledge, better treatments, and scientific advances.

  View details for PubMedID 29570246

• Immunogenicity and safety of the multicomponent meningococcal B vaccine (4CMenB) in children and adolescents: a systematic review and meta-analysis LANCET INFECTIOUS DISEASES Flacco, M., Manzoli, L., Rosso, A., Marzuillo, C., Bergamini, M., Stefanati, A., Cultrera, R., Villari, P., Ricciardi, W., Ioannidis, J. A., Contopoulos-Ioannidis, D. G. 2018; 18 (4): 461–72

  Abstract

  The multicomponent meningococcal serogroup B vaccine (4CMenB) has been licensed in more than 35 countries. However, uncertainties remain about the lowest number of doses required to induce satisfactory, persistent immune responses. We did a systematic review and meta-analysis to provide quantitative estimates for the immunogenicity, persistence of immunogenicity, and safety of 4CMenB vaccine in children and adolescents.For this systematic review and meta-analyses (proportion, head to head, and network), we searched MEDLINE, Scopus, Embase, and ClinicalTrials.gov from database inception to June 30, 2017, for randomised trials that compared the immunogenicity or safety of the 4CMenB vaccine with its originator meningococcal B recombinant vaccine or routine vaccines in children or adolescents. For proportion meta-analyses, we also included single arm trials and follow-up studies of randomised controlled trials. Trials that assessed immunogenicity against at least one of four Neisseria meningitidis serogroup B reference strains (44-76/SL, 5/99, NZ98/254, and M10713) and included participants younger than 18 years who had received two or more doses of the 4CMenB vaccine were eligible for inclusion. We requested individual patient-level data from study authors and extracted data from published reports and online trial registries. We did meta-analyses to assess 4CMenB safety and immunogenicity against the four reference strains 30 days after a primary immunisation course (three doses for children, two doses for adolescents), 30 days after the primary course plus one booster dose (children only), 6 months or more after primary course, and 6 months or more after the booster dose.736 non-duplicate records were screened, and ten randomised trials and eight follow-on extension trials on 4CMenB met the inclusion criteria. In intention-to-treat analyses, the overall proportion of children and adolescents who achieved seroconversion 30 days after the primary course of 4CMenB was 92% (95% CI 89-95 [I2=95%, p<0·0001]) for the 44/76-SL strain, 91% (87-95 [I2=95%, p<0·0001]) for the 5/99 strain, 84% (77-90 [I2=97%, p<0·0001]) for the NZ98-254 strain, and 87% (68-99 [I2=97%, p<0·0001]) for the M10713 strain. 6 months after the primary course, the immunogenicity remained adequate to high against all three tested strains (5/99, 44/76-SL, and NZ98/254) in adolescents (≥77%), and against two of four strains (5/99 and 44/76-SL) in children (≥67%): the proportion of patients who achieved seroconversion substantially declined for M10713 (<50%) and NZ98/254 (<35%). A booster dose re-enhanced the proportion of patients who achieved seroconversion (≥93% for all strains). However, immunogenicity remained high 6 months after the booster dose for strains 5/99 (95%) and M10713 (75%) only, whereas the proportion of patients who achieved seroconversion against strains 44/76-SL and NZ98/254 returned to similar proportions recorded 6 months after the primary course (62% for 44/76-SL, 35% for NZ98/254). The incidence of potentially vaccine-related, acute serious adverse events in individuals receiving 4CMenB was low (5·4 per 1000 individuals), but was significantly higher than routine vaccines (1·2 per 1000 individuals).4CMenB has an acceptable short-term safety profile. The primary course is sufficient to achieve a satisfactory immune response within 30 days of vaccination. A booster dose is required for children to prolong the protection against strain M10713, and the long-term immunogenicity against strain NZ98/254 remains suboptimal.None.

  View details for PubMedID 29371070

• Are systematic reviews and meta-analyses still useful research? We are not sure INTENSIVE CARE MEDICINE Moller, M., Ioannidis, J. A., Darmon, M. 2018; 44 (4): 518–20

  View details for PubMedID 29663048

• Two Genetic Variants Associated with Plantar Fascial Disorders INTERNATIONAL JOURNAL OF SPORTS MEDICINE Kim, S. K., Ioannidis, J. A., Ahmed, M. A., Avins, A. L., Kleimeyer, J. P., Fredericson, M., Dragoo, J. L. 2018; 39 (4): 314–21

  Abstract

  Plantar fascial disorder is comprised of plantar fasciitis and plantar fibromatosis. Plantar fasciitis is the most common cause of heel pain, especially for athletes involved in running and jumping sports. Plantar fibromatosis is a rare fibrous hyperproliferation of the deep connective tissue of the foot. To identify genetic loci associated with plantar fascial disorders, a genome-wide association screen was performed using publically available data from the Research Program in Genes, Environment and Health including 21,624 cases of plantar fascial disorders and 80,879 controls. One indel (chr5:118704153:D) and one SNP (rs62051384) showed an association with plantar fascial disorders at genome-wide significance (p<5×10-8) with small effects (odds ratios=0.93 and 1.07 per allele, respectively). The indel chr5:118704153:D is located within TNFAIP8 (encodes a protein induced by TNF alpha) and rs62051384 is located within WWP2 (which is involved in proteasomal degradation). These DNA variants may be informative in explaining why some individuals are at higher risk for plantar fascial disorders than others.

  View details for PubMedID 29534260

• Improving the integrity of published science: An expanded taxonomy of retractions and corrections EUROPEAN JOURNAL OF CLINICAL INVESTIGATION Fanelli, D., Ioannidis, J. A., Goodman, S. 2018; 48 (4)

  View details for PubMedID 29369337

• Meta-analyses in environmental and occupational health. Occupational and environmental medicine Ioannidis, J. P. 2018

  Abstract

  OBJECTIVES: Meta-analyses are considered generally as the highest level of evidence, but concerns have been voiced about their massive, low-quality production. This paper aimed to evaluate the landscape of meta-analyses in the field of occupational and environmental health and medicine.METHODS: Using relevant search terms, all meta-analyses were searched for, but those published in 2015 were assessed for their origin, whether they included randomised trials and individual-level data and whether they had authors from the industry or consultancy firms.RESULTS: PubMed searches (last update February 2017) identified 1251 eligible meta-analyses in this field. There was a rapid increase over time (n=16 published in 1995 vs n=163 published in 2015). Of the 163 eligible meta-analyses published in 2015, 49 were from China, followed at a distance by the USA (n=19). Only 16 considered randomised (intervention) trials and 13 included individual-level data. Only 1 of the 150 meta-analyses had industry authors and none had consultancy firm authors. As an example of conflicting findings, 12 overlapping meta-analyses addressed mobile phones and brain cancer risk and they differed substantially in number of studies included, eligibility criteria and conclusions.CONCLUSIONS: There has been a major increase in the publication of meta-analyses in occupational and environmental health over time, with the majority of these studies focusing on observational data, while a commendable fraction used individual-level data. Authorship is still limited largely to academic and non-profit authors. With massive production of meta-analyses, redundancy needs to be anticipated and efforts should be made to safeguard quality and protect from bias.

  View details for PubMedID 29574405

• A universal standard for the validation of blood pressure measuring devices: Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO) Collaboration Statement JOURNAL OF HYPERTENSION Stergiou, G. S., Alpert, B., Mieke, S., Asmar, R., Atkins, N., Eckert, S., Frick, G., Friedman, B., Grassl, T., Ichikawa, T., Ioannidis, J. P., Lacy, P., McManus, R., Murray, A., Myers, M., Palatini, P., Parati, G., Quinn, D., Sarkis, J., Shennan, A., Usuda, T., Wang, J., Wu, C. O., O'Brien, E. 2018; 36 (3): 472–78

  Abstract

  : In the last 30 years, several organizations, such as the US Association for the Advancement of Medical Instrumentation (AAMI), the British Hypertension Society, the European Society of Hypertension (ESH) Working Group on Blood Pressure (BP) Monitoring and the International Organization for Standardization (ISO) have developed protocols for clinical validation of BP measuring devices. However, it is recognized that science, as well as patients, consumers and manufacturers would be best served if all BP measuring devices were assessed for accuracy according to an agreed single validation protocol that had global acceptance. Therefore, an international initiative was taken by AAMI, ESH and ISO experts who agreed to develop a universal standard for device validation. This statement presents the key aspects of a validation procedure, which were agreed by the AAMI, ESH and ISO representatives as the basis for a single universal validation protocol. As soon as the AAMI/ESH/ISO standard is fully developed, this will be regarded as the single universal standard and will replace all other previous standards/protocols.

  View details for PubMedID 29384983

• Assessing scientists for hiring, promotion, and tenure PLOS BIOLOGY Moher, D., Naudet, F., Cristea, I. A., Miedema, F., Ioannidis, J. A., Goodman, S. N. 2018; 16 (3): e2004089

  Abstract

  Assessment of researchers is necessary for decisions of hiring, promotion, and tenure. A burgeoning number of scientific leaders believe the current system of faculty incentives and rewards is misaligned with the needs of society and disconnected from the evidence about the causes of the reproducibility crisis and suboptimal quality of the scientific publication record. To address this issue, particularly for the clinical and life sciences, we convened a 22-member expert panel workshop in Washington, DC, in January 2017. Twenty-two academic leaders, funders, and scientists participated in the meeting. As background for the meeting, we completed a selective literature review of 22 key documents critiquing the current incentive system. From each document, we extracted how the authors perceived the problems of assessing science and scientists, the unintended consequences of maintaining the status quo for assessing scientists, and details of their proposed solutions. The resulting table was used as a seed for participant discussion. This resulted in six principles for assessing scientists and associated research and policy implications. We hope the content of this paper will serve as a basis for establishing best practices and redesigning the current approaches to assessing scientists by the many players involved in that process.

  View details for PubMedID 29596415

• Comparative evidence on harms in pediatric randomized clinical trials from less developed versus more developed countries is limited JOURNAL OF CLINICAL EPIDEMIOLOGY Tedesco, D., Farid-Kapadia, M., Offringa, M., Bhutta, Z. A., Maldonado, Y., Ioannidis, J. A., Contopoulos-Ioannidis, D. G. 2018; 95: 63–72

  Abstract

  Evaluate comparative harm rates from medical interventions in pediatric randomized clinical trials (RCTs) from more developed (MDCs) and less developed countries (LDCs).Meta-epidemiologic empirical evaluation of Cochrane Database of Systematic Reviews (June 2014) meta-analyses reporting clinically important harm-outcomes (severe adverse events [AEs], discontinuations due to AEs, any AE, and mortality) that included at least one pediatric RCT from MDCs and at least one from LDCs. We estimated relative odds ratios (RORs) for each harm, within each meta-analysis, between RCTs from MDCs and LDCs and calculated random-effects-summary-RORs (sRORs) for each harm across multiple meta-analyses.Only 1% (26/2,363) of meta-analyses with clinically important harm-outcomes in the entire Cochrane Database of Systematic Reviews included pediatric RCTs both from MDCs and LDCs. We analyzed 26 meta-analyses with 244 data sets from pediatric RCTs, 116 from MDCs and 128 from LDCs (64 and 66 unique RCTs respectively). The summary ROR was 0.92 (95% confidence intervals: 0.78-1.08) for severe AEs; 1.13 (0.54-2.34) for discontinuations due to AEs; 1.10 (0.77-1.59) for any AE; and 0.99 (0.61-1.61) for mortality and for the all-harms-combined-end point 0.96 (0.83-1.10). Differences of ROR-point-estimates ≥2-fold between MDCs and LDCs were identified in 35% of meta-analyses.We found no major systematic differences in harm rates in pediatric trials between MDCs and LDCs, but data on harms in children were overall very limited.

  View details for PubMedID 29191447

• A Universal Standard for the Validation of Blood Pressure Measuring Devices Association for the Advancement of Medical Instrumentation/European Society of Hypertension/International Organization for Standardization (AAMI/ESH/ISO) Collaboration Statement HYPERTENSION Stergiou, G. S., Alpert, B., Mieke, S., Asmar, R., Atkins, N., Eckert, S., Frick, G., Friedman, B., Grassl, T., Ichikawa, T., Ioannidis, J. P., Lacy, P., McManus, R., Murray, A., Myers, M., Palatini, P., Parati, G., Quinn, D., Sarkis, J., Shennan, A., Usuda, T., Wang, J., Wu, C. O., O'Brien, E. 2018; 71 (3): 368–74

  Abstract

  In the past 30 years, several organizations, such as the US Association for the Advancement of Medical Instrumentation (AAMI), the British Hypertension Society, the European Society of Hypertension (ESH) Working Group on Blood Pressure (BP) Monitoring, and the International Organization for Standardization (ISO), have developed protocols for clinical validation of BP measuring devices. However, it is recognized that science, as well as patients, consumers, and manufacturers, would be best served if all BP measuring devices were assessed for accuracy according to an agreed single validation protocol that had global acceptance. Therefore, an international initiative was taken by the AAMI, ESH, and ISO experts who agreed to develop a universal standard for device validation. This statement presents the key aspects of a validation procedure, which were agreed by the AAMI, ESH, and ISO representatives as the basis for a single universal validation protocol. As soon as the AAMI/ESH/ISO standard is fully developed, this will be regarded as the single universal standard and will replace all other previous standards/protocols.

  View details for PubMedID 29386350

• Evidence Based Medicine and Big Genomic Data. Human molecular genetics Ioannidis, J. P., Khoury, M. J. 2018

  Abstract

  Genomic and other related big data (Big Genomic Data, BGD for short) are ushering a new era of precision medicine. This overview discusses whether principles of evidence-based medicine (EBM) hold true for BGD and how they should be operationalized in the current era. Major EBM principles include the systematic identification, description and analysis of the validity and utility of BGD, the combination of individual clinical expertise with individual patient needs and preferences, and the focus on obtaining experimental evidence, whenever possible. BGD emphasize information of single patients with an overemphasis on N-of-1 trials to personalize treatment. However, large-scale comparative population data remain indispensable for meaningful translation of BGD personalized information. The impact of BGD on population health depends on its ability to affect large segments of the population. While several frameworks have been proposed to facilitate and standardize decision-making for use of genomic tests, there are new caveats that arise from BGD that extend beyond the limitations that were applicable for more simple genetic tests. Non-evidence-based use of BGD may be harmful and result in major waste of health care resources. Randomized controlled trials (RCTs) will continue to be the strongest arbitrator for the clinical utility of genomic technologies, including BGD. Research on BGD needs to focus not only on finding robust predictive associations (clinical validity), but more importantly on evaluating the balance of health benefits and potential harms (clinical utility), as well as implementation challenges. Appropriate features of such useful research on BGD are discussed.

  View details for PubMedID 29474574

• Effect of Low-Fat vs Low-Carbohydrate Diet on 12-Month Weight Loss in Overweight Adults and the Association With Genotype Pattern or Insulin Secretion The DIETFITS Randomized Clinical Trial JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Gardner, C. D., Trepanowski, J. F., Del Gobbo, L. C., Hauser, M. E., Rigdon, J., Ioannidis, J. A., Desai, M., King, A. C. 2018; 319 (7): 667–79

  Abstract

  Dietary modification remains key to successful weight loss. Yet, no one dietary strategy is consistently superior to others for the general population. Previous research suggests genotype or insulin-glucose dynamics may modify the effects of diets.To determine the effect of a healthy low-fat (HLF) diet vs a healthy low-carbohydrate (HLC) diet on weight change and if genotype pattern or insulin secretion are related to the dietary effects on weight loss.The Diet Intervention Examining The Factors Interacting with Treatment Success (DIETFITS) randomized clinical trial included 609 adults aged 18 to 50 years without diabetes with a body mass index between 28 and 40. The trial enrollment was from January 29, 2013, through April 14, 2015; the date of final follow-up was May 16, 2016. Participants were randomized to the 12-month HLF or HLC diet. The study also tested whether 3 single-nucleotide polymorphism multilocus genotype responsiveness patterns or insulin secretion (INS-30; blood concentration of insulin 30 minutes after a glucose challenge) were associated with weight loss.Health educators delivered the behavior modification intervention to HLF (n = 305) and HLC (n = 304) participants via 22 diet-specific small group sessions administered over 12 months. The sessions focused on ways to achieve the lowest fat or carbohydrate intake that could be maintained long-term and emphasized diet quality.Primary outcome was 12-month weight change and determination of whether there were significant interactions among diet type and genotype pattern, diet and insulin secretion, and diet and weight loss.Among 609 participants randomized (mean age, 40 [SD, 7] years; 57% women; mean body mass index, 33 [SD, 3]; 244 [40%] had a low-fat genotype; 180 [30%] had a low-carbohydrate genotype; mean baseline INS-30, 93 μIU/mL), 481 (79%) completed the trial. In the HLF vs HLC diets, respectively, the mean 12-month macronutrient distributions were 48% vs 30% for carbohydrates, 29% vs 45% for fat, and 21% vs 23% for protein. Weight change at 12 months was -5.3 kg for the HLF diet vs -6.0 kg for the HLC diet (mean between-group difference, 0.7 kg [95% CI, -0.2 to 1.6 kg]). There was no significant diet-genotype pattern interaction (P = .20) or diet-insulin secretion (INS-30) interaction (P = .47) with 12-month weight loss. There were 18 adverse events or serious adverse events that were evenly distributed across the 2 diet groups.In this 12-month weight loss diet study, there was no significant difference in weight change between a healthy low-fat diet vs a healthy low-carbohydrate diet, and neither genotype pattern nor baseline insulin secretion was associated with the dietary effects on weight loss. In the context of these 2 common weight loss diet approaches, neither of the 2 hypothesized predisposing factors was helpful in identifying which diet was better for whom.clinicaltrials.gov Identifier: NCT01826591.

  View details for PubMedID 29466592

• Disclosures in Nutrition Research Why It Is Different JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Ioannidis, J. A., Trepanowski, J. F. 2018; 319 (6): 547–48

  View details for PubMedID 29222543

• Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine BMJ-BRITISH MEDICAL JOURNAL Naudet, F., Sakarovitch, C., Janiaud, P., Cristea, I., Fanelli, D., Moher, D., Ioannidis, J. A. 2018; 360: 1–11

  Abstract

  To explore the effectiveness of data sharing by randomized controlled trials (RCTs) in journals with a full data sharing policy and to describe potential difficulties encountered in the process of performing reanalyses of the primary outcomes.Survey of published RCTs.PubMed/Medline.RCTs that had been submitted and published by The BMJ and PLOS Medicine subsequent to the adoption of data sharing policies by these journals.The primary outcome was data availability, defined as the eventual receipt of complete data with clear labelling. Primary outcomes were reanalyzed to assess to what extent studies were reproduced. Difficulties encountered were described.37 RCTs (21 from The BMJ and 16 from PLOS Medicine) published between 2013 and 2016 met the eligibility criteria. 17/37 (46%, 95% confidence interval 30% to 62%) satisfied the definition of data availability and 14