All Publications


  • Almanac - Retrieval-Augmented Language Models for Clinical Medicine. NEJM AI Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., Boyd, J., Boyd, K., Hirsch, K., Langlotz, C., Lee, R., Melia, J., Nelson, J., Sallam, K., Tullis, S., Vogelsong, M. A., Cunningham, J. P., Hiesinger, W. 2024; 1 (2)

    Abstract

    Large language models (LLMs) have recently shown impressive zero-shot capabilities, whereby they can use auxiliary data, without the availability of task-specific training examples, to complete a variety of natural language tasks, such as summarization, dialogue generation, and question answering. However, despite many promising applications of LLMs in clinical medicine, adoption of these models has been limited by their tendency to generate incorrect and sometimes even harmful statements.We tasked a panel of eight board-certified clinicians and two health care practitioners with evaluating Almanac, an LLM framework augmented with retrieval capabilities from curated medical resources for medical guideline and treatment recommendations. The panel compared responses from Almanac and standard LLMs (ChatGPT-4, Bing, and Bard) versus a novel data set of 314 clinical questions spanning nine medical specialties.Almanac showed a significant improvement in performance compared with the standard LLMs across axes of factuality, completeness, user preference, and adversarial safety.Our results show the potential for LLMs with access to domain-specific corpora to be effective in clinical decision-making. The findings also underscore the importance of carefully testing LLMs before deployment to mitigate their shortcomings. (Funded by the National Institutes of Health, National Heart, Lung, and Blood Institute.).

    View details for DOI 10.1056/aioa2300068

    View details for PubMedID 38343631

    View details for PubMedCentralID PMC10857783

  • Almanac: Retrieval-Augmented Language Models for Clinical Medicine. Research square Zakka, C., Chaurasia, A., Shad, R., Dalal, A. R., Kim, J. L., Moor, M., Alexander, K., Ashley, E., Boyd, J., Boyd, K., Hirsch, K., Langlotz, C., Nelson, J., Hiesinger, W. 2023

    Abstract

    Large-language models have recently demonstrated impressive zero-shot capabilities in a variety of natural language tasks such as summarization, dialogue generation, and question-answering. Despite many promising applications in clinical medicine, adoption of these models in real-world settings has been largely limited by their tendency to generate incorrect and sometimes even toxic statements. In this study, we develop Almanac, a large language model framework augmented with retrieval capabilities for medical guideline and treatment recommendations. Performance on a novel dataset of clinical scenarios (n= 130) evaluated by a panel of 5 board-certified and resident physicians demonstrates significant increases in factuality (mean of 18% at p-value < 0.05) across all specialties, with improvements in completeness and safety. Our results demonstrate the potential for large language models to be effective tools in the clinical decision-making process, while also emphasizing the importance of careful testing and deployment to mitigate their shortcomings.

    View details for DOI 10.21203/rs.3.rs-2883198/v1

    View details for PubMedID 37205549

    View details for PubMedCentralID PMC10187428

  • Preloading Trifolded Grafts for Descemet Membrane Endothelial Keratoplasty Affects Scroll Formation CORNEA Solar, S. J., Deljookorani, S., Wiener, B. G., Rosen, A., Chaurasia, A., Shahmirzadi, M., Meshkin, R. S., Dzhaber, D., Chiang, E., Barnes, K., Chen, C. Y., Koo, E. H., Eghrari, A. O. 2020; 39 (8): 1062–65

    Abstract

    The trifolded, endothelium-in approach to Descemet membrane endothelial keratoplasty (DMEK) facilitates tissue insertion into the anterior chamber. We hypothesized that preloading the trifolded donor grafts in a cartridge for 48 hours before insertion would induce biomechanical changes that decrease their scrolling tendency compared with those loaded immediately before insertion.Ten Descemet membrane donor grafts, peeled and cut to 8.0 mm, were prepared by a single eye bank technician. Each graft was trifolded and pulled into a DMEK cartridge and stored for 48 hours. They were then pulled with microforceps into a petri dish filled with balanced salt solution. A video was recorded of the graft becoming a scroll over a 2-minute period. Each graft, serving as its own control, was then trifolded, pulled into the cartridge, and the process repeated. Images from 1, 5, 10, 60, and 120 seconds were extracted from video recording of the procedures. Scroll width was analyzed by graders masked to group assignment. A paired t test was used to determine differences in scroll width at each time point between the 48-hour and instant trifolding conditions.All grafts scrolled after removal from the cartridge into balanced salt solution. We measured a significant difference at all time points 1 through 120 seconds (4.02 preloaded vs. 2.91-mm instant trifold, P = 0.035).Preloading DMEK grafts in a trifolded configuration for 48 hours reduces the scrolling tendency of Descemet membrane for at least 2 minutes.

    View details for DOI 10.1097/ICO.0000000000002298

    View details for Web of Science ID 000559736500028

    View details for PubMedID 32118669

  • A Device for Preloaded, Trifolded Grafts to Facilitate Descemet Membrane Endothelial Keratoplasty JOURNAL OF MEDICAL DEVICES-TRANSACTIONS OF THE ASME Chiang, E., Chen, C., Barnes, K., Chaurasia, A., Rosen, A., Solar, S., Wiener, B., Cai, S., Subramanya, A., Vora, P., Durr, N. J., Eghrari, A. O., Allen, R. 2019; 13 (4)

    View details for DOI 10.1115/1.4043739

    View details for Web of Science ID 000506858800010

  • Comparison of Tri-folded and Scroll-based Graft Viability in Preloaded Descemet Membrane Endothelial Keratoplasty CORNEA Barnes, K., Chiang, E., Chen, C., Lohmeier, J., Christy, J., Chaurasia, A., Rosen, A., Vora, P., Cai, S., Subramanya, A., Durr, N., Allen, R., Eghrari, A. 2019; 38 (3): 392–96

    Abstract

    To compare corneal endothelial damage associated with 2 techniques for preloaded Descemet membrane endothelial keratoplasty (DMEK): a tri-folded graft stored in a plastic cartridge designed for DMEK and a scrolled graft stored in a modified Jones Tube, at the time of preparation and after shipping.DMEK grafts were prepared at the Rocky Mountain Lions Eye Bank. The grafts were either tri-folded and loaded in a plastic cartridge or scrolled and loaded into a modified Jones Tube. In each group, the grafts were then either immediately removed from the cartridges or shipped for 48 hours. The grafts were then stained with Calcein AM and imaged using a fluorescent microscope. Endothelial cell loss (ECL) was determined using trainable segmentation in Fiji by 2 graders. At each time point, rates of ECL loss were compared across the 2 groups. To explore the role of donor characteristics, a multivariable regression model was produced to account for method (tri-folding vs. scroll), donor age, donor gender, death-to-preservation time, death-to-preparation time, and shipping.A total of 40 grafts were prepared, processed, imaged, and analyzed. No significant difference in cell loss was seen between groups at either time point alone. In the multivariate model, no significant increase in cell loss was associated with either tri-folding (3.7% less ECL; P = 0.051) or shipping (4.3% less ECL; P = 0.049).All techniques used resulted in clinically acceptable levels of ECL. Tri-folded tissue in a plastic cartridge did not result in ECL inferior to a scroll when prepared either immediately or preloaded for 48 hours.

    View details for DOI 10.1097/ICO.0000000000001831

    View details for Web of Science ID 000464539300025

    View details for PubMedID 30550395

  • Viability of Descemet Membrane Endothelial Keratoplasty Grafts Folded in the Eye Bank CORNEA Lohmeier, J., Christy, J., Chiang, E., Barnes, K., Cai, S., Chen, C., Subramanya, A., Chaurasia, A., Rosen, A., Vora, P., Durr, N. J., Allen, R., Eghrari, A. 2018; 37 (11): 1474–77

    Abstract

    Preloaded, trifolded grafts in Descemet membrane endothelial keratoplasty require transfer of the trifolding process from the corneal transplant surgeon to the eye bank technician. We sought to assess whether trifolding may be safely conducted by an eye bank technician with cell loss comparable to standard peeling and lifting.A total of 10 grafts were stained, peeled, and transferred directly onto a bed of Calcein-AM and Amvisc Plus by an eye bank technician. Five grafts were removed and stained as a scroll, and 5 grafts were trifolded with the endothelium in before transfer. Photographs were acquired with an inverted fluorescence microscope, and image segmentation was performed. A t test was conducted to compare differences in endothelial cell loss across groups.Mean cell loss in the scroll group was 18.5% [95% confidence interval (CI): 15.2%-21.9%] compared with 7.6% of the trifolded group (95% CI: 1.7%-13.5%). A 2-tailed t test indicated decreased cell loss in the trifolded group (P = 0.013).Despite additional manipulation of the graft, trifolding of Descemet membrane and endothelium may be performed by an eye bank technician without significantly increased cell loss relative to graft preparation as a scroll.

    View details for DOI 10.1097/ICO.0000000000001711

    View details for Web of Science ID 000456481500030

    View details for PubMedID 30095494

    View details for PubMedCentralID PMC6173616