Robert Thibault

All Publications

• Post-publication critique at top-ranked journals across scientific disciplines: a cross-sectional assessment of policies and practice. Royal Society open science Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Tzavella, L., Bendixen, T., Handcock, S. A., Koneke, V. E., Ioannidis, J. P. 2022; 9 (8): 220139

  Abstract

  Journals exert considerable control over letters, commentaries and online comments that criticize prior research (post-publication critique). We assessed policies (Study One) and practice (Study Two) related to post-publication critique at 15 top-ranked journals in each of 22 scientific disciplines (N = 330 journals). Two-hundred and seven (63%) journals accepted post-publication critique and often imposed limits on length (median 1000, interquartile range (IQR) 500-1200 words) and time-to-submit (median 12, IQR 4-26 weeks). The most restrictive limits were 175 words and two weeks; some policies imposed no limits. Of 2066 randomly sampled research articles published in 2018 by journals accepting post-publication critique, 39 (1.9%, 95% confidence interval [1.4, 2.6]) were linked to at least one post-publication critique (there were 58 post-publication critiques in total). Of the 58 post-publication critiques, 44 received an author reply, of which 41 asserted that original conclusions were unchanged. Clinical Medicine had the most active culture of post-publication critique: all journals accepted post-publication critique and published the most post-publication critique overall, but also imposed the strictest limits on length (median 400, IQR 400-550 words) and time-to-submit (median 4, IQR 4-6 weeks). Our findings suggest that top-ranked academic journals often pose serious barriers to the cultivation, documentation and dissemination of post-publication critique.

  View details for DOI 10.1098/rsos.220139

  View details for PubMedID 36039285

• Discrepancy review: a feasibility study of a novel peer review intervention to reduce undisclosed discrepancies between registrations and publications ROYAL SOCIETY OPEN SCIENCE Thibault, R. T., Thompson, J., Drax, K., Munafo, M. R., Hardwicke, T. E., Pennington, C. R., Clark, R. A., O'Mahony, A., Nilsonne, G., TARG Meta-Res Grp Collaborators 2022; 9 (7): 220142

  Abstract

  Undisclosed discrepancies often exist between study registrations and their associated publications. Discrepancies can increase risk of bias, and when undisclosed, they disguise this increased risk of bias from readers. To remedy this issue, we developed an intervention called discrepancy review. We provided journals with peer reviewers specifically assigned to check for undisclosed discrepancies between registrations and manuscripts submitted to journals. We performed discrepancy review on 18 manuscripts submitted to Nicotine and Tobacco Research and three manuscripts submitted to the European Journal of Personality. We iteratively refined the discrepancy review process based on feedback from discrepancy reviewers, editors and authors. Authors addressed the majority of discrepancy reviewer comments, and there was no opposition to running a trial from authors, editors or discrepancy reviewers. Outcome measures for a trial of discrepancy review could include the presence of primary or secondary outcome discrepancies, whether publications that are not the primary report from a clinical trial registration are clearly described as such, whether registrations are permanent, and an overarching subjective assessment of the impact of discrepancies in published articles. We found that discrepancy review could feasibly be introduced as a regular practice at some journals interested in this process. A full trial of discrepancy review would be needed to evaluate its impact on reducing undisclosed discrepancies.

  View details for DOI 10.1098/rsos.220142

  View details for Web of Science ID 000885769400006

  View details for PubMedID 35911195

  View details for PubMedCentralID PMC9326291

• A many-analysts approach to the relation between religiosity and well-being RELIGION BRAIN & BEHAVIOR Hoogeveen, S., Sarafoglou, A., Aczel, B., Aditya, Y., Alayan, A. J., Allen, P. J., Altay, S., Alzahawi, S., Amir, Y., Anthony, F., Appiah, O., Atkinson, Q. D., Baimel, A., Balkaya-Ince, M., Balsamo, M., Banker, S., Bartos, F., Becerra, M., Beffara, B., Beitner, J., Bendixen, T., Berkessel, J. B., Berniunas, R., Billet, M., Billingsley, J., Bortolini, T., Breitsohl, H., Bret, A., Brown, F. L., Brown, J., Brumbaugh, C. C., Buczny, J., Bulbulia, J., Caballero, S., Carlucci, L., Carmichael, C. L., Cattaneo, M. G., Charles, S. J., Claessens, S., Panagopoulos, M. C., Costa, A., Crone, D. L., Czoschke, S., Czymara, C., D'Urso, E., Dahlstrom, O., Dalla Rosa, A., Danielsson, H., De Ron, J., de Vries, Y., Dean, K. K., Dik, B. J., Disabato, D. J., Doherty, J. K., Draws, T., Drouhot, L., Dujmovic, M., Dunham, Y., Ebert, T., Edelsbrunner, P. A., Eerland, A., Elbaek, C. T., Farahmand, S., Farahmand, H., Farias, M., Feliccia, A. A., Fischer, K., Fischer, R., Fisher-Thompson, D., Francis, Z., Frick, S., Frisch, L. K., Geraldes, D., Gerdin, E., Geven, L., Ghasemi, O., Gielens, E., Gligoric, V., Hagel, K., Hajdu, N., Hamilton, H. R., Hamzah, I., Hanel, P. P., Hawk, C. E., Himawan, K. K., Holding, B. C., Homman, L. E., Ingendahl, M., Inkila, H., Inman, M. L., Islam, C., Isler, O., Izydorczyk, D., Jaeger, B., Johnson, K. A., Jong, J., Karl, J. A., Kaszubowski, E., Katz, B. A., Keefer, L. A., Kelchtermans, S., Kelly, J. M., Klein, R. A., Kleinberg, B., Knowles, M. L., Kolczynska, M., Koller, D., Krasko, J., Kritzler, S., Krypotos, A., Kyritsis, T., Landes, T. L., Laukenmann, R., Forsyth, G., Lazar, A., Lehman, B. J., Levy, N., Lo, R. F., Lodder, P., Lorenz, J., Lowicki, P., Ly, A. L., Maassen, E., Magyar-Russell, G. M., Maier, M., Marsh, D. R., Martinez, N., Martinie, M., Martoyo, I., Mason, S. E., Mauritsen, A., McAleer, P., McCauley, T., McCullough, M., McKay, R., McMahon, C. M., McNamara, A. A., Means, K. K., Mercier, B., Mitkidis, P., Monin, B., Moon, J. W., Moreau, D., Morgan, J., Murphy, J., Muscatt, G., Nagel, C., Nagy, T., Nalborczyk, L., Nilsonne, G., Noack, P., Norenzayan, A., Nuijten, M. B., Olsson-Collentine, A., Oviedo, L., Pavlov, Y. G., Pawelski, J. O., Pearson, H., Pedder, H., Peetz, H. K., Pinus, M., Pirutinsky, S., Polito, V., Porubanova, M., Poulin, M. J., Prenoveau, J. M., Prince, M. A., Protzko, J., Pryor, C., Purzycki, B. G., Qiu, L., Putter, J., Rabelo, A., Radell, M. L., Ramsay, J. E., Reid, G., Roberts, A. J., Luna, L., Ross, R. M., Roszak, P., Roy, N., Saarelainen, S. K., Sasaki, J. Y., Schaumans, C., Schivinski, B., Schmitt, M. C., Schnitker, S. A., Schnuerch, M., Schreiner, M. R., Schuttengruber, V., Sebben, S., Segerstrom, S. C., Seryczynska, B., Shjoedt, U., Simsek, M., Sleegers, W. A., Smith, E. R., Sowden, W. J., Spath, M., Sporlein, C., Stedden, W., Stoevenbelt, A. H., Stuber, S., Sulik, J., Suwartono, C., Syropoulos, S., Szaszi, B., Szecsi, P., Tappin, B. M., Tay, L., Thibault, R. T., Thompson, B., Thurn, C. M., Torralba, J., Tuthill, S. D., Ullein, A., Van Aert, R. M., van Assen, M. M., Van Cappellen, P., van den Akker, O. R., Van der Cruyssen, I., Van der Noll, J., van Dongen, N. N., Van Lissa, C. J., van Mulukom, V., van Ravenzwaaij, D., van Zyl, C. J., Vaughn, L., Verschuere, B., Vianello, M., Vilanova, F., Vishkin, A., Vogel, V., Vogelsmeier, L. E., Watanabe, S., White, C. M., Wiebels, K., Wiechert, S., Willett, Z. Z., Witkowiak, M., Witvliet, C. O., Wiwad, D., Wuyts, R., Xygalatas, D., Yang, X., Yeo, D. J., Yilmaz, O., Zarzeczna, N., Zhao, Y., Zijlmans, J., van Elk, M., Wagenmakers, E. 2022

  View details for DOI 10.1080/2153599X.2022.2070255

  View details for Web of Science ID 000821405300001

• Excess significance and power miscalculations in neurofeedback research. NeuroImage. Clinical Thibault, R. T., Pedder, H. 2022: 103008

  View details for DOI 10.1016/j.nicl.2022.103008

  View details for PubMedID 35525708

• Rigour and reproducibility in Canadian research: call for a coordinated approach FACETS Thibault, R. T., Munafo, M. R., Moher, D. 2022; 7: 18-24

  View details for DOI 10.1139/facets-2021-0162

  View details for Web of Science ID 000741549000001

• Citation Patterns Following a Strongly Contradictory Replication Result: Four Case Studies From Psychology ADVANCES IN METHODS AND PRACTICES IN PSYCHOLOGICAL SCIENCE Hardwicke, T. E., Szucs, D., Thibault, R. T., Cruwell, S., van den Akker, O. R., Nuijten, M. B., Ioannidis, J. A. 2021; 4 (3)

  View details for DOI 10.1177/25152459211040837

  View details for Web of Science ID 000708944200001

• Estimating the Prevalence of Transparency and Reproducibility-Related Research Practices in Psychology (2014-2017). Perspectives on psychological science : a journal of the Association for Psychological Science Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., Ioannidis, J. P. 2021: 1745691620979806

  Abstract

  Psychologists are navigating an unprecedented period of introspection about the credibility and utility of their discipline. Reform initiatives emphasize the benefits of transparency and reproducibility-related research practices; however, adoption across the psychology literature is unknown. Estimating the prevalence of such practices will help to gauge the collective impact of reform initiatives, track progress over time, and calibrate future efforts. To this end, we manually examined a random sample of 250 psychology articles published between 2014 and 2017. Over half of the articles were publicly available (154/237, 65%, 95% confidence interval [CI] = [59%, 71%]); however, sharing of research materials (26/183; 14%, 95% CI = [10%, 19%]), study protocols (0/188; 0%, 95% CI = [0%, 1%]), raw data (4/188; 2%, 95% CI = [1%, 4%]), and analysis scripts (1/188; 1%, 95% CI = [0%, 1%]) was rare. Preregistration was also uncommon (5/188; 3%, 95% CI = [1%, 5%]). Many articles included a funding disclosure statement (142/228; 62%, 95% CI = [56%, 69%]), but conflict-of-interest statements were less common (88/228; 39%, 95% CI = [32%, 45%]). Replication studies were rare (10/188; 5%, 95% CI = [3%, 8%]), and few studies were included in systematic reviews (21/183; 11%, 95% CI = [8%, 16%]) or meta-analyses (12/183; 7%, 95% CI = [4%, 10%]). Overall, the results suggest that transparency and reproducibility-related research practices were far from routine. These findings establish baseline prevalence estimates against which future progress toward increasing the credibility and utility of psychology research can be compared.

  View details for DOI 10.1177/1745691620979806

  View details for PubMedID 33682488