Kirsten Rhee Steffner

All Publications

• Red teaming ChatGPT in medicine to yield real-world insights on model behavior. NPJ digital medicine Chang, C. T., Farah, H., Gui, H., Rezaei, S. J., Bou-Khalil, C., Park, Y. J., Swaminathan, A., Omiye, J. A., Kolluri, A., Chaurasia, A., Lozano, A., Heiman, A., Jia, A. S., Kaushal, A., Jia, A., Iacovelli, A., Yang, A., Salles, A., Singhal, A., Narasimhan, B., Belai, B., Jacobson, B. H., Li, B., Poe, C. H., Sanghera, C., Zheng, C., Messer, C., Kettud, D. V., Pandya, D., Kaur, D., Hla, D., Dindoust, D., Moehrle, D., Ross, D., Chou, E., Lin, E., Haredasht, F. N., Cheng, G., Gao, I., Chang, J., Silberg, J., Fries, J. A., Xu, J., Jamison, J., Tamaresis, J. S., Chen, J. H., Lazaro, J., Banda, J. M., Lee, J. J., Matthys, K. E., Steffner, K. R., Tian, L., Pegolotti, L., Srinivasan, M., Manimaran, M., Schwede, M., Zhang, M., Nguyen, M., Fathzadeh, M., Zhao, Q., Bajra, R., Khurana, R., Azam, R., Bartlett, R., Truong, S. T., Fleming, S. L., Raj, S., Behr, S., Onyeka, S., Muppidi, S., Bandali, T., Eulalio, T. Y., Chen, W., Zhou, X., Ding, Y., Cui, Y., Tan, Y., Liu, Y., Shah, N., Daneshjou, R. 2025; 8 (1): 149

  Abstract

  Red teaming, the practice of adversarially exposing unexpected or undesired model behaviors, is critical towards improving equity and accuracy of large language models, but non-model creator-affiliated red teaming is scant in healthcare. We convened teams of clinicians, medical and engineering students, and technical professionals (80 participants total) to stress-test models with real-world clinical cases and categorize inappropriate responses along axes of safety, privacy, hallucinations/accuracy, and bias. Six medically-trained reviewers re-analyzed prompt-response pairs and added qualitative annotations. Of 376 unique prompts (1504 responses), 20.1% were inappropriate (GPT-3.5: 25.8%; GPT-4.0: 16%; GPT-4.0 with Internet: 17.8%). Subsequently, we show the utility of our benchmark by testing GPT-4o, a model released after our event (20.4% inappropriate). 21.5% of responses appropriate with GPT-3.5 were inappropriate in updated models. We share insights for constructing red teaming prompts, and present our benchmark for iterative model assessments.

  View details for DOI 10.1038/s41746-025-01542-0

  View details for PubMedID 40055532

  View details for PubMedCentralID 10564921

• The Association of Echocardiographically Measured Donor Left Ventricular Mass and 1-Year Outcomes After Heart Transplantation. JACC. Heart failure O'Donnell, C., Tapaskar, N., Sanchez, P. A., Wayda, B., Santana, E. J., Pulgrossi, R. C., Steffner, K., Zhang, S., Weng, Y., Sun, L. Y., Malinoski, D., Zaroff, J., Haddad, F., Khush, K. K. 2024

  Abstract

  Donor-recipient heart size matching is crucial in heart transplantation; however, the often-used predicted heart mass (PHM) ratio may be inaccurate in the setting of obesity.In this study, the authors sought to investigate the association between echocardiographically measured donor left ventricular mass (LVM) for heart size matching and the risk of the primary 1-year composite outcome of death or retransplantation.The Donor Heart Study was a prospective, multicenter, observational cohort study that collected echocardiograms from brain-dead donors. The measured LVM ratio (donor measured LVM/recipient predicted LVM) was defined as the exposure variable, and the association with the primary outcome was analyzed with Cox proportional hazard modeling. Secondary analyses evaluated the association of the PHM and predicted LVM (donor predicted LVM/recipient predicted LVM) ratios with the primary outcome.In 2,015 heart transplants, the measured LVM ratio demonstrated that undersized matches (<0.80) had a 47% higher risk (adjusted HR [aHR]: 1.47; 95% CI: 1.01-2.15) and oversized (>1.20) matches had a 58% increased risk (aHR: 1.58; 95% CI: 1.05-2.37) of the 1-year composite outcome compared with ideally matched transplants. However, the PHM and predicted LVM ratios were not associated with the primary outcome. Nonlinear modeling demonstrated a U-shaped relationship between the measured LVM ratio and composite outcome. The measured LVM ratio had superior predictive power for poor post-transplantation outcomes in obese recipients.Measuring donor LVM with the use of echocardiography may provide a more accurate method for donor-recipient heart size matching that could improve heart transplant outcomes, especially in obese recipients.

  View details for DOI 10.1016/j.jchf.2024.10.001

  View details for PubMedID 39570237

• Patient attitudes toward the AI doctor. Nature medicine Fanous, A., Steffner, K., Daneshjou, R. 2024

  View details for DOI 10.1038/s41591-024-03272-4

  View details for PubMedID 39313596

  View details for PubMedCentralID 9871804

• Artificial Intelligence in Perioperative Care: Opportunities and Challenges. Anesthesiology Han, L., Char, D. S., Aghaeepour, N. 2024; 141 (2): 379-387

  View details for DOI 10.1097/ALN.0000000000005013

  View details for PubMedID 38980160

• Overview and Clinical Applications of Artificial Intelligence and Machine Learning in Cardiac Anesthesiology. Journal of cardiothoracic and vascular anesthesia Mathis, M., Steffner, K. R., Subramanian, H., Gill, G. P., Girardi, N. I., Bansal, S., Bartels, K., Khanna, A. K., Huang, J. 2024

  Abstract

  Artificial intelligence- (AI) and machine learning (ML)-based applications are becoming increasingly pervasive in the healthcare setting. This has in turn challenged clinicians, hospital administrators, and health policymakers to understand such technologies and develop frameworks for safe and sustained clinical implementation. Within cardiac anesthesiology, challenges and opportunities for AI/ML to support patient care are presented by the vast amounts of electronic health data, which are collected rapidly, interpreted, and acted upon within the periprocedural area. To address such challenges and opportunities, in this article, the authors review 3 recent applications relevant to cardiac anesthesiology, including depth of anesthesia monitoring, operating room resource optimization, and transthoracic/transesophageal echocardiography, as conceptual examples to explore strengths and limitations of AI/ML within healthcare, and characterize this evolving landscape. Through reviewing such applications, the authors introduce basic AI/ML concepts and methodologies, as well as practical considerations and ethical concerns for initiating and maintaining safe clinical implementation of AI/ML-based algorithms for cardiac anesthesia patient care.

  View details for DOI 10.1053/j.jvca.2024.02.004

  View details for PubMedID 38453558

• Deep learning for transesophageal echocardiography view classification. Scientific reports Steffner, K. R., Christensen, M., Gill, G., Bowdish, M., Rhee, J., Kumaresan, A., He, B., Zou, J., Ouyang, D. 2024; 14 (1): 11

  Abstract

  Transesophageal echocardiography (TEE) imaging is a vital tool used in the evaluation of complex cardiac pathology and the management of cardiac surgery patients. A key limitation to the application of deep learning strategies to intraoperative and intraprocedural TEE data is the complexity and unstructured nature of these images. In the present study, we developed a deep learning-based, multi-category TEE view classification model that can be used to add structure to intraoperative and intraprocedural TEE imaging data. More specifically, we trained a convolutional neural network (CNN) to predict standardized TEE views using labeled intraoperative and intraprocedural TEE videos from Cedars-Sinai Medical Center (CSMC). We externally validated our model on intraoperative TEE videos from Stanford University Medical Center (SUMC). Accuracy of our model was high across all labeled views. The highest performance was achieved for the Trans-Gastric Left Ventricular Short Axis View (area under the receiver operating curve [AUC] = 0.971 at CSMC, 0.957 at SUMC), the Mid-Esophageal Long Axis View (AUC = 0.954 at CSMC, 0.905 at SUMC), the Mid-Esophageal Aortic Valve Short Axis View (AUC = 0.946 at CSMC, 0.898 at SUMC), and the Mid-Esophageal 4-Chamber View (AUC = 0.939 at CSMC, 0.902 at SUMC). Ultimately, we demonstrate that our deep learning model can accurately classify standardized TEE views, which will facilitate further downstream deep learning analyses for intraoperative and intraprocedural TEE imaging.

  View details for DOI 10.1038/s41598-023-50735-8

  View details for PubMedID 38167849

  View details for PubMedCentralID PMC10761863