Thomas Robert Savage

All Publications

• Affiliation Bias in Peer Review of Abstracts. JAMA Gallo, R. J., Savage, T., Chen, J. H. 2024; 331 (14): 1234-1235

  View details for DOI 10.1001/jama.2024.3520

  View details for PubMedID 38592392

• Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine. NPJ digital medicine Savage, T., Nayak, A., Gallo, R., Rangan, E., Chen, J. H. 2024; 7 (1): 20

  Abstract

  One of the major barriers to using large language models (LLMs) in medicine is the perception they use uninterpretable methods to make clinical decisions that are inherently different from the cognitive processes of clinicians. In this manuscript we develop diagnostic reasoning prompts to study whether LLMs can imitate clinical reasoning while accurately forming a diagnosis. We find that GPT-4 can be prompted to mimic the common clinical reasoning processes of clinicians without sacrificing diagnostic accuracy. This is significant because an LLM that can imitate clinical reasoning to provide an interpretable rationale offers physicians a means to evaluate whether an LLMs response is likely correct and can be trusted for patient care. Prompting methods that use diagnostic reasoning have the potential to mitigate the "black box" limitations of LLMs, bringing them one step closer to safe and effective use in medicine.

  View details for DOI 10.1038/s41746-024-01010-1

  View details for PubMedID 38267608

  View details for PubMedCentralID 9931230

• A Large Language Model Screening Tool to Target Patients for Best Practice Alerts: Development and Validation. JMIR medical informatics Savage, T., Wang, J., Shieh, L. 2023; 11: e49886

  Abstract

  Best Practice Alerts (BPAs) are alert messages to physicians in the electronic health record that are used to encourage appropriate use of health care resources. While these alerts are helpful in both improving care and reducing costs, BPAs are often broadly applied nonselectively across entire patient populations. The development of large language models (LLMs) provides an opportunity to selectively identify patients for BPAs.In this paper, we present an example case where an LLM screening tool is used to select patients appropriate for a BPA encouraging the prescription of deep vein thrombosis (DVT) anticoagulation prophylaxis. The artificial intelligence (AI) screening tool was developed to identify patients experiencing acute bleeding and exclude them from receiving a DVT prophylaxis BPA.Our AI screening tool used a BioMed-RoBERTa (Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach; AllenAI) model to perform classification of physician notes, identifying patients without active bleeding and thus appropriate for a thromboembolism prophylaxis BPA. The BioMed-RoBERTa model was fine-tuned using 500 history and physical notes of patients from the MIMIC-III (Medical Information Mart for Intensive Care) database who were not prescribed anticoagulation. A development set of 300 MIMIC patient notes was used to determine the model's hyperparameters, and a separate test set of 300 patient notes was used to evaluate the screening tool.Our MIMIC-III test set population of 300 patients included 72 patients with bleeding (ie, were not appropriate for a DVT prophylaxis BPA) and 228 without bleeding who were appropriate for a DVT prophylaxis BPA. The AI screening tool achieved impressive accuracy with a precision-recall area under the curve of 0.82 (95% CI 0.75-0.89) and a receiver operator curve area under the curve of 0.89 (95% CI 0.84-0.94). The screening tool reduced the number of patients who would trigger an alert by 20% (240 instead of 300 alerts) and increased alert applicability by 14.8% (218 [90.8%] positive alerts from 240 total alerts instead of 228 [76%] positive alerts from 300 total alerts), compared to nonselectively sending alerts for all patients.These results show a proof of concept on how language models can be used as a screening tool for BPAs. We provide an example AI screening tool that uses a HIPAA (Health Insurance Portability and Accountability Act)-compliant BioMed-RoBERTa model deployed with minimal computing power. Larger models (eg, Generative Pre-trained Transformers-3, Generative Pre-trained Transformers-4, and Pathways Language Model) will exhibit superior performance but require data use agreements to be HIPAA compliant. We anticipate LLMs to revolutionize quality improvement in hospital medicine.

  View details for DOI 10.2196/49886

  View details for PubMedID 38010803

• Milky Way: Management of Primary Intestinal Lymphangiectasia Digestive Diseases and Sciences Norman, J. S., Testa, S., Wang, C., Savage, T. 2023

  View details for DOI 10.1007/s10620-023-08077-y

• Availability and content of clinical guidance for tobacco use and dependence treatment-United States, 2000-2019 PREVENTIVE MEDICINE VanFrank, B., Uhd, J., Savage, T. R., Shah, J. R., Twentyman, E. 2022; 164: 107276

  Abstract

  Evidence-based treatments for tobacco use and dependence can increase cessation success but remain underutilized. Health professional societies and voluntary health organizations (advising organizations) are uniquely positioned to influence the delivery of cessation treatments by providing clinical guidance for healthcare providers. This study aimed to review the guidance produced by these organizations for content and consistency with current evidence. Documents discussing healthcare providers' role in treatment of tobacco use and dependence produced by US-based advising organizations between 2000 and 2019 were identified in both peer-reviewed and grey (i.e., informally or non-commercially published) literature. Extraction of variables, defined in terms of healthcare provider role and endorsement of specific treatment(s), was completed by two independent reviewers. Review of 38 identified documents sponsored by 57 unique advising organizations revealed deficits in the direction of comprehensive care and incorporation of the most recent evidence for treatment of tobacco use and dependence. Documents endorsed: screening (74%), pharmacotherapy (68%), counseling (89%), or follow-up (37%). Few documents endorsed more recent evidence-based treatments including combination nicotine replacement therapy (18%), and text- (11%) and web-based (11%) interventions. Advising organizations have opportunities to address identified gaps and enhance clinical guidance to contribute toward expanding the provision of comprehensive tobacco cessation support.

  View details for DOI 10.1016/j.ypmed.2022.107276

  View details for Web of Science ID 000878634600019

  View details for PubMedID 36152817

• Artificial Intelligence in Medical Education ACADEMIC MEDICINE Savage, T. 2021; 96 (9): 1229-1230

  View details for Web of Science ID 000688214700011

• Enhancing patient engagement during virtual care: A conceptual model and rapid implementation at an academic medical center NEJM: Catalyst Innovations in Care Srinivasan, M., Phadke, A., Zulman, D., Thadaney, S., Madill, E., Savage, T., Downing, N., Nelligan, I., Artandi, M., Sharp, C. 2020

  View details for DOI 10.1056/CAT.20.0262

• Mathematical Modeling of Communicable Diseases: Expanding Public Health in Medical Education Medical Science Educator Savage, T. R., Shah, A., Karimi, N., Sinha, A., Holland, B. 2017

  View details for DOI 10.1007/s40670-017-0398-0

• Synthesis, characterization and structural comparisons of phosphonium and arsenic dithiocarbamates with alkyl and phenyl substituents POLYHEDRON Donahue, C. M., Black, I. K., Pecnik, S. L., Savage, T. R., Scott, B. L., Daly, S. R. 2014; 75: 110-117

  View details for DOI 10.1016/j.poly.2014.03.003

  View details for Web of Science ID 000336777400014