All Publications

  • Meta-Research: Reader engagement with medical content on Wikipedia. eLife Maggio, L. A., Steinberg, R. M., Piccardi, T. n., Willinsky, J. M. 2020; 9


    Articles on Wikipedia about health and medicine are maintained by WikiProject Medicine (WPM), and are widely used by health professionals, students and others. We have compared these articles, and reader engagement with them, to other articles on Wikipedia. We found that WPM articles are longer, possess a greater density of external links, and are visited more often than other articles on Wikipedia. Readers of WPM articles are more likely to hover over and view footnotes than other readers, but are less likely to visit the hyperlinked sources in these footnotes. Our findings suggest that WPM readers appear to use links to external sources to verify and authorize Wikipedia content, rather than to examine the sources themselves.

    View details for DOI 10.7554/eLife.52426

    View details for PubMedID 32142406

  • Wikipedia as a gateway to biomedical research: The relative distribution and use of citations in the English Wikipedia. PloS one Maggio, L. A., Willinsky, J. M., Steinberg, R. M., Mietchen, D. n., Wass, J. L., Dong, T. n. 2017; 12 (12): e0190046


    Wikipedia is a gateway to knowledge. However, the extent to which this gateway ends at Wikipedia or continues via supporting citations is unknown. Wikipedia's gateway functionality has implications for information design and education, notably in medicine. This study aims to establish benchmarks for the relative distribution and referral (click) rate of citations-as indicated by presence of a Digital Object Identifier (DOI)-from Wikipedia, with a focus on medical citations. DOIs referred from the English Wikipedia in August 2016 were obtained from Next, based on a DOI's presence on a WikiProject Medicine page, all DOIs in Wikipedia were categorized as medical (WP:MED) or non-medical (non-WP:MED). Using this categorization, referred DOIs were classified as WP:MED, non-WP:MED, or BOTH, meaning the DOI may have been referred from either category. Data were analyzed using descriptive and inferential statistics. Out of 5.2 million Wikipedia pages, 4.42% (n = 229,857) included at least one DOI. 68,870 were identified as WP:MED, with 22.14% (n = 15,250) featuring one or more DOIs. WP:MED pages featured on average 8.88 DOI citations per page, whereas non-WP:MED pages had on average 4.28 DOI citations. For DOIs only on WP:MED pages, a DOI was referred every 2,283 pageviews and for non-WP:MED pages every 2,467 pageviews. DOIs from BOTH pages accounted for 12% (n = 58,475). The referral of DOI citations found in BOTH could not be assigned to WP:MED or non-WP:MED, as the page from which the referral was made was not provided with the data. While these results cannot provide evidence of greater citation referral from WP:MED than non-WP:MED, they do provide benchmarks to assess strategies for changing referral patterns. These changes might include editors adopting new methods for designing and presenting citations or the introduction of teaching strategies that address the value of consulting citations as a tool for extending learning.

    View details for PubMedID 29267345

    View details for PubMedCentralID PMC5739466

  • An Investigation of Title Ambiguity in the Health Sciences Literature CATALOGING & CLASSIFICATION QUARTERLY Miller, D. R., Banko, J., Allen, T. S., Vanderpool, A., Steinberg, R. 2017; 55 (7-8): 506–21
  • In an Age of Open Access to Research Policies: Physician and Public Health NGO Staff Research Use and Policy Awareness. PloS one Moorhead, L. L., Holzmeyer, C., Maggio, L. A., Steinberg, R. M., Willinsky, J. 2015; 10 (7)


    Through funding agency and publisher policies, an increasing proportion of the health sciences literature is being made open access. Such an increase in access raises questions about the awareness and potential utilization of this literature by those working in health fields.A sample of physicians (N=336) and public health non-governmental organization (NGO) staff (N=92) were provided with relatively complete access to the research literature indexed in PubMed, as well as access to the point-of-care service UpToDate, for up to one year, with their usage monitored through the tracking of web-log data. The physicians also participated in a one-month trial of relatively complete or limited access.The study found that participants' research interests were not satisfied by article abstracts alone nor, in the case of the physicians, by a clinical summary service such as UpToDate. On average, a third of the physicians viewed research a little more frequently than once a week, while two-thirds of the public health NGO staff viewed more than three articles a week. Those articles were published since the 2008 adoption of the NIH Public Access Policy, as well as prior to 2008 and during the maximum 12-month embargo period. A portion of the articles in each period was already open access, but complete access encouraged a viewing of more research articles.Those working in health fields will utilize more research in the course of their work as a result of (a) increasing open access to research, (b) improving awareness of and preparation for this access, and (c) adjusting public and open access policies to maximize the extent of potential access, through reduction in embargo periods and access to pre-policy literature.

    View details for DOI 10.1371/journal.pone.0129708

    View details for PubMedID 26200794

    View details for PubMedCentralID PMC4511689

  • Access of primary and secondary literature by health personnel in an academic health center: implications for open access. Journal of the Medical Library Association Maggio, L. A., Steinberg, R. M., Moorhead, L., O'Brien, B., Willinsky, J. 2013; 101 (3): 205-212


    The research sought to ascertain the types and quantity of research evidence accessed by health personnel through PubMed and UpToDate in a university medical center over the course of a year in order to better estimate the impact that increasing levels of open access to biomedical research can be expected to have on clinical practice in the years ahead.Web log data were gathered from the 5,042 health personnel working in the Stanford University Hospitals (SUH) during 2011. Data were analyzed for access to the primary literature (abstracts and full-text) through PubMed and UpToDate and to the secondary literature, represented by UpToDate (research summaries), to establish the frequency and nature of literature consulted.In 2011, SUH health personnel accessed 81,851 primary literature articles and visited UpToDate 110,336 times. Almost a third of the articles (24,529) accessed were reviews. Twenty percent (16,187) of the articles viewed were published in 2011.When it is available, health personnel in a clinical care setting frequently access the primary literature. While further studies are needed, this preliminary finding speaks to the value of the National Institutes of Health public access policy and the need for medical librarians and educators to prepare health personnel for increasing public access to medical research.

    View details for DOI 10.3163/1536-5050.101.3.010

    View details for PubMedID 23930091

    View details for PubMedCentralID PMC3738081

  • SmartSearch: automated recommendations using librarian expertise and the National Center for Biotechnology Information's Entrez Programming Utilities JOURNAL OF THE MEDICAL LIBRARY ASSOCIATION Steinberg, R. M., Zwies, R., Yates, C., Stave, C., Pouliot, Y., Heilemann, H. A. 2010; 98 (2): 171-175

    View details for DOI 10.3163/1536-5050.98.2.012

    View details for Web of Science ID 000277447300012

    View details for PubMedID 20428285

    View details for PubMedCentralID PMC2859267

  • LaneConnex: An Integrated Biomedical Digital Library Interface INFORMATION TECHNOLOGY AND LIBRARIES Ketchell, D. S., Steinberg, R. M., Yates, C., Heilemann, H. A. 2009; 28 (1): 31-40