Kirsti Weng Elder MD/MPH

All Publications

• Effect of an Electronic Health Record-Based Intervention on Documentation Practices. Applied clinical informatics Shah, S., Bedgood, M., Devon-Sand, A., Dolphin-Dempsey, C., Cherukuri, V., Weng, K., Lin, S., Sharp, C. 2024

  Abstract

  Please see title page and main document for latest version of abstract.

  View details for DOI 10.1055/a-2367-8564

  View details for PubMedID 39019475

• AI-Human Hybrid Workflow Enhances Teleophthalmology for the Detection of Diabetic Retinopathy. Ophthalmology science Dow, E. R., Khan, N. C., Chen, K. M., Mishra, K., Perera, C., Narala, R., Basina, M., Dang, J., Kim, M., Levine, M., Phadke, A., Tan, M., Weng, K., Do, D. V., Moshfeghi, D. M., Mahajan, V. B., Mruthyunjaya, P., Leng, T., Myung, D. 2023; 3 (4): 100330

  Abstract

  Detection of diabetic retinopathy (DR) outside of specialized eye care settings is an important means of access to vision-preserving health maintenance. Remote interpretation of fundus photographs acquired in a primary care or other nonophthalmic setting in a store-and-forward manner is a predominant paradigm of teleophthalmology screening programs. Artificial intelligence (AI)-based image interpretation offers an alternative means of DR detection. IDx-DR (Digital Diagnostics Inc) is a Food and Drug Administration-authorized autonomous testing device for DR. We evaluated the diagnostic performance of IDx-DR compared with human-based teleophthalmology over 2 and a half years. Additionally, we evaluated an AI-human hybrid workflow that combines AI-system evaluation with human expert-based assessment for referable cases.Prospective cohort study and retrospective analysis.Diabetic patients ≥ 18 years old without a prior DR diagnosis or DR examination in the past year presenting for routine DR screening in a primary care clinic.Macula-centered and optic nerve-centered fundus photographs were evaluated by an AI algorithm followed by consensus-based overreading by retina specialists at the Stanford Ophthalmic Reading Center. Detection of more-than-mild diabetic retinopathy (MTMDR) was compared with in-person examination by a retina specialist.Sensitivity, specificity, accuracy, positive predictive value, and gradability achieved by the AI algorithm and retina specialists.The AI algorithm had higher sensitivity (95.5% sensitivity; 95% confidence interval [CI], 86.7%-100%) but lower specificity (60.3% specificity; 95% CI, 47.7%-72.9%) for detection of MTMDR compared with remote image interpretation by retina specialists (69.5% sensitivity; 95% CI, 50.7%-88.3%; 96.9% specificity; 95% CI, 93.5%-100%). Gradability of encounters was also lower for the AI algorithm (62.5%) compared with retina specialists (93.1%). A 2-step AI-human hybrid workflow in which the AI algorithm initially rendered an assessment followed by overread by a retina specialist of MTMDR-positive encounters resulted in a sensitivity of 95.5% (95% CI, 86.7%-100%) and a specificity of 98.2% (95% CI, 94.6%-100%). Similarly, a 2-step overread by retina specialists of AI-ungradable encounters improved gradability from 63.5% to 95.6% of encounters.Implementation of an AI-human hybrid teleophthalmology workflow may both decrease reliance on human specialist effort and improve diagnostic accuracy.Proprietary or commercial disclosure may be found after the references.

  View details for DOI 10.1016/j.xops.2023.100330

  View details for PubMedID 37449051

  View details for PubMedCentralID PMC10336195

• Artificial Intelligence Improves Patient Follow-Up in a Diabetic Retinopathy Screening Program. Clinical ophthalmology (Auckland, N.Z.) Dow, E. R., Chen, K. M., Zhao, C. S., Knapp, A. N., Phadke, A., Weng, K., Do, D. V., Mahajan, V. B., Mruthyunjaya, P., Leng, T., Myung, D. 2023; 17: 3323-3330

  Abstract

  We examine the rate of and reasons for follow-up in an Artificial Intelligence (AI)-based workflow for diabetic retinopathy (DR) screening relative to two human-based workflows.A DR screening program initiated September 2019 between one institution and its affiliated primary care and endocrinology clinics screened 2243 adult patients with type 1 or 2 diabetes without a diagnosis of DR in the previous year in the San Francisco Bay Area. For patients who screened positive for more-than-mild-DR (MTMDR), rates of follow-up were calculated under a store-and-forward human-based DR workflow ("Human Workflow"), an AI-based workflow involving IDx-DR ("AI Workflow"), and a two-step hybrid workflow ("AI-Human Hybrid Workflow"). The AI Workflow provided results within 48 hours, whereas the other workflows took up to 7 days. Patients were surveyed by phone about follow-up decisions.Under the AI Workflow, 279 patients screened positive for MTMDR. Of these, 69.2% followed up with an ophthalmologist within 90 days. Altogether 70.5% (N=48) of patients who followed up chose their location based on primary care referral. Among the subset of patients that were seen in person at the university eye institute under the Human Workflow and AI-Human Hybrid Workflow, 12.0% (N=14/117) and 11.7% (N=12/103) of patients with a referrable screening result followed up compared to 35.5% of patients under the AI Workflow (N=99/279; χ2df=2 = 36.70, p < 0.00000001).Ophthalmology follow-up after a positive DR screening result is approximately three-fold higher under the AI Workflow than either the Human Workflow or AI-Human Hybrid Workflow. Improved follow-up behavior may be due to the decreased time to screening result.

  View details for DOI 10.2147/OPTH.S422513

  View details for PubMedID 38026608

  View details for PubMedCentralID PMC10665027

• Evaluating the Implementation of a Model of Integrated Behavioral Health in Primary Care: Perceptions of the Healthcare Team. Journal of primary care & community health Dunn, J. A., Chokron Garneau, H., Jawad, N., Zein, M., Elder, K. W., Sattler, A., McGovern, M. 2023; 14: 21501319221146918

  Abstract

  OBJECTIVES: This study aims to compare primary care providers and medical assistants in degrees of comfort, confidence, and consistency when addressing behavioral health concerns with patients before and after the implementation of a model of integrated behavioral health in primary care (IBHPC), and evaluate whether these perceptions differ based on increased access to behavioral health clinicians.METHODS: This longitudinal study was conducted at 2 primary care clinics in Northern California while implementing an IBHPC model. The Integrated Behavioral Health Staff Perceptions Survey was administered to assess the comfort, confidence, and consistency of behavioral health practices. Confidential online surveys were distributed to primary care faculty and staff members before and post-implementation. Responses from providers and medical assistants were compared between pre- and post-implementation with linear regression analyses. The relationships between accessibility to behavioral health clinicians and a change in comfort, confidence, and consistency of behavioral health practices were explored using a linear mixed-effects model.RESULTS: A total of 35 providers and medical assistants completed the survey both before and post-implementation of IBHPC. Over time, there were increasingly positive perceptions about the consistency of behavioral health screening (P=.03) and overall confidence in addressing behavioral health concerns (P=.005). Comfort in addressing behavioral health concerns did not significantly change for either providers or staff over time. Medical assistants were initially more confident and comfortable addressing behavioral health concerns than providers, but providers' attitudes increased post-IBHPC implementation. Improved access to behavioral health clinicians was associated with greater consistency of screening and referral to specialty mental health care (P<.001).CONCLUSION: The present study is the first to explore differences in provider and medical assistant perceptions during the course of an IBHPC implementation. Findings underscore the importance of integrating medical assistants, along with providers, into all phases of the implementation process.

  View details for DOI 10.1177/21501319221146918

  View details for PubMedID 36625239

• Integration of Artificial Intelligence into a Telemedicine-Based Diabetic Retinopathy Screening Program Chen, K., Dow, E. R., Khan, N. C., Levine, M., Perera, C., Phadke, A., Dang, J., Weng, K., Do, D. V., Mahajan, V. B., Mruthyunjaya, P., Mishra, K., Leng, T., Myung, D. ASSOC RESEARCH VISION OPHTHALMOLOGY INC. 2022

  View details for Web of Science ID 000844401304101