Olivia Jee

All Publications

• Graph-based clinical recommender: Predicting specialists procedure orders using graph representation learning. Journal of biomedical informatics Fouladvand, S., Gomez, F. R., Nilforoshan, H., Schwede, M., Noshad, M., Jee, O., You, J., Sosic, R., Leskovec, J., Chen, J. 2023: 104407

  Abstract

  To determine whether graph neural network based models of electronic health records can predict specialty consultation care needs for endocrinology and hematology more accurately than the standard of care checklists and other conventional medical recommendation algorithms in the literature.Demand for medical expertise far outstrips supply, with tens of millions in the US alone with deficient access to specialty care. Rather than potentially months long delays to initiate diagnostic workup and medical treatment with a specialist, referring primary care supported by an automated recommender algorithm could anticipate and directly initiate patient evaluation that would otherwise be needed at subsequent a specialist appointment. We propose a novel graph representation learning approach with a heterogeneous graph neural network to model structured electronic health records and formulate recommendation/prediction of subsequent specialist orders as a link prediction problem.Models are trained and assessed in two specialty care sites: endocrinology and hematology. Our experimental results show that our model achieves an 8% improvement in ROC-AUC for endocrinology (ROC-AUC = 0.88) and 5% improvement for hematology (ROC-AUC = 0.84) personalized procedure recommendations over prior medical recommender systems. These recommender algorithm approaches provide medical procedure recommendations for endocrinology referrals more effectively than manual clinical checklists (recommender: precision = 0.60, recall = 0.27, F1-score = 0.37) vs. (checklist: precision = 0.16, recall = 0.28, F1-score = 0.20), and similarly for hematology referrals (recommender: precision = 0.44, recall = 0.38, F1-score = 0.41) vs. (checklist: precision = 0.27, recall = 0.71, F1-score = 0.39).Embedding graph neural network models into clinical care can improve digital specialty consultation systems and expand the access to medical experience of prior similar cases.

  View details for DOI 10.1016/j.jbi.2023.104407

  View details for PubMedID 37271308

• Transforming Interprofessional Roles During Virtual Health Care: The Evolving Role of the Medical Assistant, in Relationship to National Health Profession Competency Standards. Journal of primary care & community health Rokicki-Parashar, J., Phadke, A., Brown-Johnson, C., Jee, O., Sattler, A., Torres, E., Srinivasan, M. 2021; 12: 21501327211004285

  Abstract

  INTRODUCTION: Medical assistants (MAs) were once limited to obtaining vital signs and office work. Now, MAs are foundational to team-based care, interacting with patients, systems, and teams in many ways. The transition to Virtual Health during the COVID-19 pandemic resulted in a further rapid and unique shift of MA roles and responsibilities. We sought to understand the impact of this shift and to place their new roles in the context of national professional competency standards.METHODS: In this qualitative, grounded theory study we conducted semi-structured interviews with 24 MAs at 10 primary care sites at a major academic medical center on their experiences during the shift from in-person to virtual care. MAs were selected by convenience sample. Coding was done in Dedoose version 8.335. Consensus-based inductive and deductive approaches were used for interview analysis. Identified MA roles were compared to national MA, Institute of Medicine, physician, and nursing professional competency domains.RESULTS: Three main themes emerged: Role Apprehension, Role Expansion, and Adaptability/Professionalism. Nine key roles emerged in the context of virtual visits: direct patient care (pre-visit and physical care), panel management, health systems ambassador, care coordination, patient flow coordination, scribing, quality improvement, and technology support. While some prior MA roles were limited by the virtual care shift, the majority translated directly or expanded in virtual care. Identified roles aligned better with Institute of Medicine, physician, and nursing professional competencies, than current national MA curricula.CONCLUSIONS: The transition to Virtual Health decreased MA's direct clinical work and expanded other roles within interprofessional care, notably quality improvement and technology support. Comparison of the current MA roles with national training program competencies identified new leadership and teamwork competencies which could be expanded during MA training to better support MA roles on inter-professional teams.

  View details for DOI 10.1177/21501327211004285

  View details for PubMedID 33764223

• Qualitative Assessment of Rapid System Transformation to Primary Care Video Visits at an Academic Medical Center. Annals of internal medicine Srinivasan, M. n., Asch, S. n., Vilendrer, S. n., Thomas, S. C., Bajra, R. n., Barman, L. n., Edwards, L. M., Filipowicz, H. n., Giang, L. n., Jee, O. n., Mahoney, M. n., Nelligan, I. n., Phadke, A. J., Torres, E. n., Artandi, M. n. 2020

  Abstract

  The coronavirus disease 2019 pandemic spurred health systems across the world to quickly shift from in-person visits to safer video visits.To seek stakeholder perspectives on video visits' acceptability and effect 3 weeks after near-total transition to video visits.Semistructured qualitative interviews.6 Stanford general primary care and express care clinics at 6 northern California sites, with 81 providers, 123 staff, and 97 614 patient visits in 2019.Fifty-three program participants (overlapping roles as medical providers [n = 20], medical assistants [n = 16], nurses [n = 4], technologists [n = 4], and administrators [n = 13]) were interviewed about video visit transition and challenges.In 3 weeks, express care and primary care video visits increased from less than 10% to greater than 80% and from less than 10% to greater than 75%, respectively. New video visit providers received video visit training and care quality feedback. New system workflows were created to accommodate the new visit method.Nine faculty, trained in qualitative research methods, conducted 53 stakeholder interviews in 4 days using purposeful (administrators and technologists) and convenience (medical assistant, nurses, and providers) sampling. A rapid qualitative analytic approach for thematic analysis was used.The analysis revealed 12 themes, including Pandemic as Catalyst; Joy in Medicine; Safety in Medicine; Slipping Through the Cracks; My Role, Redefined; and The New Normal. Themes were analyzed using the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to identify critical issues for continued program utilization.Evaluation was done immediately after deployment. Although viewpoints may have evolved later, immediate evaluation allowed for prompt program changes and identified broader issues to address for program sustainability.After pandemic-related systems transformation at Stanford, critical issues to sustain video visit long-term viability were identified. Specifically, technology ease of use must improve and support multiparty videoconferencing. Providers should be able to care for their patients, regardless of geography. Providers need decision-making support with virtual examination training and home-based patient diagnostics. Finally, ongoing video visit reimbursement should be commensurate with value to the patients' health and well-being.Stanford Department of Medicine and Stanford Health Care.

  View details for DOI 10.7326/M20-1814

  View details for PubMedID 32628536