Marcie Levine

All Publications

• Primary Care 2.0: A Prospective Evaluation of a Novel Model of Advanced Team Care With Expanded Medical Assistant Support. Annals of family medicine Shaw, J. G., Winget, M., Brown-Johnson, C., Seay-Morrison, T., Garvert, D. W., Levine, M., Safaeinili, N., Mahoney, M. R. 2021; 19 (5): 411-418

  Abstract

  PURPOSE: Assess effectiveness of Primary Care 2.0: a team-based model that incorporates increased medical assistant (MA) to primary care physician (PCP) ratio, integration of advanced practice clinicians, expanded MA roles, and extended the interprofessional team.METHODS: Prospective, quasi-experimental evaluation of staff/clinician team development and wellness survey data, comparing Primary Care 2.0 to conventional clinics within our academic health care system. We surveyed before the model launch and every 6-9 months up to 24 months post implementation. Secondary outcomes (cost, quality metrics, patient satisfaction) were assessed via routinely collected operational data.RESULTS: Team development significantly increased in the Primary Care 2.0 clinic, sustained across all 3 post implementation time points (+12.2, +8.5, + 10.1 respectively, vs baseline, on the 100-point Team Development Measure) relative to the comparison clinics. Among wellness domains, only "control of work" approached significant gains (+0.5 on a 5-point Likert scale, P = .05), but was not sustained. Burnout did not have statistically significant relative changes; the Primary Care 2.0 site showed a temporal trend of improvement at 9 and 15 months. Reversal of this trend at 2 years corresponded to contextual changes, specifically, reduced MA to PCP staffing ratio. Adjusted models confirmed an inverse relationship between team development and burnout (P <.0001). Secondary outcomes generally remained stable between intervention and comparison clinics with suggestion of labor cost savings.CONCLUSIONS: The Primary Care 2.0 model of enhanced team-based primary care demonstrates team development is a plausible key to protect against burnout, but is not sufficient alone. The results reinforce that transformation to team-based care cannot be a 1-time effort and institutional commitment is integral.

  View details for DOI 10.1370/afm.2714

  View details for PubMedID 34546947

• AI-Human Hybrid Workflow Enhances Teleophthalmology for the Detection of Diabetic Retinopathy. Ophthalmology science Dow, E. R., Khan, N. C., Chen, K. M., Mishra, K., Perera, C., Narala, R., Basina, M., Dang, J., Kim, M., Levine, M., Phadke, A., Tan, M., Weng, K., Do, D. V., Moshfeghi, D. M., Mahajan, V. B., Mruthyunjaya, P., Leng, T., Myung, D. 2023; 3 (4): 100330

  Abstract

  Detection of diabetic retinopathy (DR) outside of specialized eye care settings is an important means of access to vision-preserving health maintenance. Remote interpretation of fundus photographs acquired in a primary care or other nonophthalmic setting in a store-and-forward manner is a predominant paradigm of teleophthalmology screening programs. Artificial intelligence (AI)-based image interpretation offers an alternative means of DR detection. IDx-DR (Digital Diagnostics Inc) is a Food and Drug Administration-authorized autonomous testing device for DR. We evaluated the diagnostic performance of IDx-DR compared with human-based teleophthalmology over 2 and a half years. Additionally, we evaluated an AI-human hybrid workflow that combines AI-system evaluation with human expert-based assessment for referable cases.Prospective cohort study and retrospective analysis.Diabetic patients ≥ 18 years old without a prior DR diagnosis or DR examination in the past year presenting for routine DR screening in a primary care clinic.Macula-centered and optic nerve-centered fundus photographs were evaluated by an AI algorithm followed by consensus-based overreading by retina specialists at the Stanford Ophthalmic Reading Center. Detection of more-than-mild diabetic retinopathy (MTMDR) was compared with in-person examination by a retina specialist.Sensitivity, specificity, accuracy, positive predictive value, and gradability achieved by the AI algorithm and retina specialists.The AI algorithm had higher sensitivity (95.5% sensitivity; 95% confidence interval [CI], 86.7%-100%) but lower specificity (60.3% specificity; 95% CI, 47.7%-72.9%) for detection of MTMDR compared with remote image interpretation by retina specialists (69.5% sensitivity; 95% CI, 50.7%-88.3%; 96.9% specificity; 95% CI, 93.5%-100%). Gradability of encounters was also lower for the AI algorithm (62.5%) compared with retina specialists (93.1%). A 2-step AI-human hybrid workflow in which the AI algorithm initially rendered an assessment followed by overread by a retina specialist of MTMDR-positive encounters resulted in a sensitivity of 95.5% (95% CI, 86.7%-100%) and a specificity of 98.2% (95% CI, 94.6%-100%). Similarly, a 2-step overread by retina specialists of AI-ungradable encounters improved gradability from 63.5% to 95.6% of encounters.Implementation of an AI-human hybrid teleophthalmology workflow may both decrease reliance on human specialist effort and improve diagnostic accuracy.Proprietary or commercial disclosure may be found after the references.

  View details for DOI 10.1016/j.xops.2023.100330

  View details for PubMedID 37449051

  View details for PubMedCentralID PMC10336195

• Integration of Artificial Intelligence into a Telemedicine-Based Diabetic Retinopathy Screening Program Chen, K., Dow, E. R., Khan, N. C., Levine, M., Perera, C., Phadke, A., Dang, J., Weng, K., Do, D. V., Mahajan, V. B., Mruthyunjaya, P., Mishra, K., Leng, T., Myung, D. ASSOC RESEARCH VISION OPHTHALMOLOGY INC. 2022

  View details for Web of Science ID 000844401304101