Bio

Julie Lee is a board-certified internal medicine physician and clinical informaticist at Stanford University. Dr. Lee's expertise in clinical informatics enables her to use informatics-driven approaches and clinically integrate AI models to improve patient health outcomes, alleviate physician burnout by streamlining workflows, and champion health equity at all levels. Dr. Lee focuses on clinical feasibility of AI implementation in healthcare systems and also leveraging patient data and AI models to identify/mitigate health disparities, making certain they function as instruments of equity rather than increasing gaps.

Dr. Lee has been key to several initiatives in improving operational processes within Stanford. Her efforts include: 1) advancing the governance and operations of Clinical Decision Support, 2) strategic integration of the Prescription Drug Monitoring Program into the electronic health record (EHR) to address the opioid crisis 3) human factors analysis of the usability of health information technology on physicians and patient outcomes. Additionally, she has worked on innovative solutions to improve patient-physician communications--including the creation of a dynamic EHR tool for better triage and processing by medical staff before a medical advice request reaches the doctors.

Health equity is her north star, informing Dr. Lee to dedicated engagement with historically underrepresented populations in medical research and collaborative partnerships between academia and community healthcare practitioners. Her previous role as an EpiScholar with the Los Angeles Department of Public Health involved researching the impact of language and acculturation on the Latino population's dietary habits and health behaviors, with a particular focus on diabetes. She has also worked with community health centers in east Los Angeles to bridge the translational gap between academic research and frontline healthcare workers, facilitating the transfer of cutting-edge liver disease research to those treating patients with substance abuse-related liver conditions.

Of major clinical interest is cardiovascular disease—she has published several papers including a landmark article on the impact of sex-specific risk factors for cardiovascular disease in women and transgender population. She is interested in improving precision health for Asians and NHPI.

Academic Appointments

Honors & Awards

  • Physician Wellness Poster Finalist, American College of Physicians (04/2024)
  • Center of Digital Health Student Grant, Stanford Center of Digital Health (04/2024)
  • Fellow Abstract Award, American Society of Addiction Medicine (02/2024)
  • ODME Conference Scholarship, Stanford School of Medicine Office of Diversity Equity and Inclusion (02/2024)
  • Fellowship Research Scholarship, Stanford Pediatrics Fellowship (11/2023)
  • Quality Improvement Abstract Award, American College of Physicians (10/2023)
  • Overall Innovation Award, Stanford Quality Improvement & Patient Safety Symposium (05/2023)
  • Future Physician Leader, High Value Practice Academic Alliance (02/2021)
  • Research Honors, University at Buffalo (05/2019)
  • Clinical Research First Place Award, Academy of Women's Health (04/2017)
  • Clarence Darrow Leadership Award, Rotaract Club of Columbia University (10/2010)
  • Milken Scholar, Milken Family Foundation - Milken Institute (05/2007)

Professional Education

  • Fellowship, Stanford University, Clinical Informatics
  • Residency, University of California, Riverside, Internal Medicine
  • Internship, University of California, Riverside, Internal Medicine
  • MD, University at Buffalo, Medicine
  • MPH, Yale University, Epidemiology
  • BA, Columbia University, Psychology

Contact

  • Academic
    juliejl@stanford.edu
    University - Academic staff Department: Med/Primary Care and Population Health Position: Clinical Assistant Professor

Additional Info

  • Mail Code: 5411
  • Profile:
    Clinical Informatics fellow (Internal Medicine trained), broadly interested in projects to improve physician wellness.
  • ORCID:
    https://orcid.org/0009-0002-6640-8597

Links

Graduate and Fellowship Programs

All Publications

  • Clinical decision support in the electronic health record: a primer for antimicrobial stewards and infection preventionists: work smarter so end users don't work harder. Antimicrobial stewardship & healthcare epidemiology : ASHE Smith, M. R., Lee, J. J., Holubar, M., Salinas, J. L., Sampson, M. M., Medford, R. J., Chang, A. 2024; 4 (1): e204

    Abstract

    Computerized clinical decision support (CDS) assists healthcare professionals in making decisions to improve patient care. In the realms of antimicrobial stewardship (ASP) and infection prevention (IP) programs, CDS interventions can play a crucial role in optimizing antibiotic prescribing practices, reducing healthcare-associated infections, and promoting diagnostic stewardship when optimally designed. This primer article aims to provide ASP and IP professionals with a practical framework for the development, design, and evaluation of CDS interventions.Large academic medical center design: Established frameworks of CDS evaluation, "Five Rights" of CDS and the "Ten Commandments of Effective Clinical Decision Support", were applied to two real-world examples of CDS tools, a Vancomycin Best Practice Advisory and a Clostridioides Difficile order panel, to demonstrate a structured approach to developing and enhancing the functionality of ASP/IP CDS interventions to promote efficacy and reduce unintended consequences of CDS.By outlining a structured approach for the development and evaluation of CDS interventions, with focus on end user engagement, efficiency and feasibility, ASP and IP professionals can leverage CDS to enhance IP/ASP quality improvement initiatives aimed to improve antibiotic utilization, diagnostic stewardship, and adherence to IP protocols.

    View details for DOI 10.1017/ash.2024.448

    View details for PubMedID 39563933

    View details for PubMedCentralID PMC11574583

  • Clinical Decision Support in the Electronic Health Record: A Primer for Antimicrobial Stewards and Infection Preventionists: Work Smarter so End Users Don’t Work Harder Antimicrobial Stewardship & Healthcare Epidemiology Smith, M., Lee, J. J., Holubar, M., Salinas, J., Sampson, M., Medford, R. J., Chang, A. 2024; 4 (1): e204

    Abstract

    Computerized clinical decision support (CDS) assists healthcare professionals in making decisions to improve patient care. In the realms of antimicrobial stewardship (ASP) and infection prevention (IP) programs, CDS interventions can play a crucial role in optimizing antibiotic prescribing practices, reducing healthcare-associated infections, and promoting diagnostic stewardship when optimally designed. This primer article aims to provide ASP and IP professionals with a practical framework for the development, design, and evaluation of CDS interventions.Large academic medical center design: Established frameworks of CDS evaluation, "Five Rights" of CDS and the "Ten Commandments of Effective Clinical Decision Support", were applied to two real-world examples of CDS tools, a Vancomycin Best Practice Advisory and a Clostridioides Difficile order panel, to demonstrate a structured approach to developing and enhancing the functionality of ASP/IP CDS interventions to promote efficacy and reduce unintended consequences of CDS.By outlining a structured approach for the development and evaluation of CDS interventions, with focus on end user engagement, efficiency and feasibility, ASP and IP professionals can leverage CDS to enhance IP/ASP quality improvement initiatives aimed to improve antibiotic utilization, diagnostic stewardship, and adherence to IP protocols.

    View details for DOI 10.1017/ash.2024.448

    View details for PubMedCentralID PMC11574583

  • Perspectives on Artificial Intelligence-Generated Responses to Patient Messages. JAMA network open Kim, J., Chen, M. L., Rezaei, S. J., Liang, A. S., Seav, S. M., Onyeka, S., Lee, J. J., Vedak, S. C., Mui, D., Lal, R. A., Pfeffer, M. A., Sharp, C., Pageler, N. M., Asch, S. M., Linos, E. 2024; 7 (10): e2438535

    View details for DOI 10.1001/jamanetworkopen.2024.38535

    View details for PubMedID 39412810

  • Empowering Hospitalized Patients With Diabetes: Implementation of a Hospital-wide CGM Policy With EHR-Integrated Validation for Dosing Insulin. Diabetes care Lee, M. Y., Seav, S. M., Ongwela, L., Lee, J. J., Aubyrn, R., Cao, F. Y., Kalinsky, A., Aparicio Ramos, O., Gu, Y., Kingston, K., Ivanovic, M., Buckingham, B. A., Desai, D., Lal, R. A., Tan, M., Basina, M., Hughes, M. S. 2024

    Abstract

    We aimed to assess the feasibility, clinical accuracy, and acceptance of a hospital-wide continuous glucose monitoring (CGM) policy with electronic health record (EHR)-integrated validation for insulin dosing.A hospital policy was developed and implemented at Stanford Health Care for using personal CGMs in lieu of fingerstick blood glucose (FSBG) monitoring. It included requirements specific to each CGM, accuracy monitoring protocols, and EHR integration. User experience surveys were conducted among a subset of patients and nurses.From November 2022 to August 2023, 135 patients used the CGM protocol in 185 inpatient encounters. This included 27% with type 1 diabetes and 24% with automated insulin delivery systems. The most-used CGMs were Dexcom G6 (44%) and FreeStyle Libre 2 (43%). Of 1,506 CGM validation attempts, 87.8% met the %20/20 criterion for CGM-based insulin dosing and 99.3% fell within Clarke zones A or B. User experience surveys were completed by 27 nurses and 46 patients. Most nurses found glucose management under the protocol effective (74%), easy to use (67%), and efficient (63%); 80% of nurses preferred inpatient CGM to FSBG. Most patients liked the CGM protocol (63%), reported positive CGM interactions with nursing staff (63%), and felt no significant interruptions to their diabetes management (63%).Implementation of a hospital-wide inpatient CGM policy supporting multiple CGM types with real-time accuracy monitoring and integration into the EHR is feasible. Initial feedback from nurses and patients was favorable, and further investigation toward broader use and sustainability is needed.

    View details for DOI 10.2337/dc24-0626

    View details for PubMedID 39140891

  • Reimagining Primary Care With AI: A Future Within Reach Shah, S., et al Practice UPdate. 2024
  • Red Teaming Large Language Models in Medicine: Real-World Insights on Model Behavior Chang, C. medxriv. 2024

    Abstract

    Background The integration of large language models (LLMs) in healthcare offers immense opportunity to streamline healthcare tasks, but also carries risks such as response accuracy and bias perpetration. To address this, we conducted a red-teaming exercise to assess LLMs in healthcare and developed a dataset of clinically relevant scenarios for future teams to use. Methods We convened 80 multi-disciplinary experts to evaluate the performance of popular LLMs across multiple medical scenarios. Teams composed of clinicians, medical and engineering students, and technical professionals stress-tested LLMs with real world clinical use cases. Teams were given a framework comprising four categories to analyze for inappropriate responses: Safety, Privacy, Hallucinations, and Bias. Prompts were tested on GPT-3.5, GPT-4.0, and GPT-4.0 with the Internet. Six medically trained reviewers subsequently reanalyzed the prompt-response pairs, with dual reviewers for each prompt and a third to resolve discrepancies. This process allowed for the accurate identification and categorization of inappropriate or inaccurate content within the responses. Results There were a total of 382 unique prompts, with 1146 total responses across three iterations of ChatGPT (GPT-3.5, GPT-4.0, GPT-4.0 with Internet). 19.8% of the responses were labeled as inappropriate, with GPT-3.5 accounting for the highest percentage at 25.7% while GPT-4.0 and GPT-4.0 with internet performing comparably at 16.2% and 17.5% respectively. Interestingly, 11.8% of responses were deemed appropriate with GPT-3.5 but inappropriate in updated models, highlighting the ongoing need to evaluate evolving LLMs. Conclusion The red-teaming exercise underscored the benefits of interdisciplinary efforts, as this collaborative model fosters a deeper understanding of the potential limitations of LLMs in healthcare and sets a precedent for future red teaming events in the field. Additionally, we present all prompts and outputs as a benchmark for future LLM model evaluations.

  • Paging the Clinical Informatics Community: Respond STAT to Dobbs v Jackson's Women's Health Organization. Applied clinical informatics Arvisais-Anhalt, S., Ravi, A., Weia, B., Aarts, J., Ahmad, H. B., Araj, E., Bauml, J. A., Benham-Hutchins, M., Boyd, A. D., Brecht-Doscher, A., Butler-Henderson, K., Butte, A., Cardillo, A. B., Chilukuri, N., Cho, M. K., Cohen, J. K., Craven, C. K., Crusco, S. J., Dadabhoy, F., Dash, D., DeBolt, C., Elkin, P. L., Fayanju, O. A., Fochtmann, L., Graham, J. V., Hanna, J., Hersh, W., Hoffard, M. R., Hron, J., Huang, S. S., Jackson, B. R., Kaplan, B., Kelly, W., Ko, K., Koppel, R., Kurapati, N., Labbad, G., Lee, J., Lehmann, C. U., Leitner, S., Liao, Z. C., Medford, R. J., Melnick, E. R., Muniyappa, A. N., Murray, S., Neinstein, A., Nichols-Johnson, V., Novak, L., Ogan, W. S., Ozeran, L., Pageler, N., Pandita, D., Perumbeti, A., Petersen, C., Pierce, L., Puttagunta, R., Ramaswamy, P., Rogers, K. M., Rosenbloom, T., Ryan, A., Saleh, S., Sarabu, C., Schreiber, R., Shaw, K. A., Sim, I., Sirintrapun, S. J., Solomonides, A., Spector, J. D., Starren, J. B., Stoffel, M., Subbian, V., Swanson, K., Tomes, A., Trang, K., Unertl, K. M., Weon, J. L., Whooley, M., Wiley, K., Williamson, D. F., Winkelstein, P., Wong, J., Xie, J., Yarahuan, J. K., Yung, N., Zera, C., Ratanawongsa, N., Sadasivaiah, S. 2022

    Abstract

    n/a.

    View details for DOI 10.1055/a-2000-7590

    View details for PubMedID 36535703

  • After menopause, is an enlarging middle, an enlarging cardiovascular risk factor? Menopause (New York, N.Y.) Lee, J. J., Shufelt, C. L. 2020; 27 (9): 974-975

    View details for DOI 10.1097/GME.0000000000001620

    View details for PubMedID 32852448

  • Cardiovascular implications of gender-affirming hormone treatment in the transgender population. Maturitas Dutra, E., Lee, J., Torbati, T., Garcia, M., Merz, C. N., Shufelt, C. 2019; 129: 45-49

    Abstract

    Transgender men and women represent a growing population in the United States and Europe, with 0.5% of adults and 3% of youth identifying as transgender. Globally, an estimated 0.3-0.5% of the population identify as transgender. Despite the increasing percentage of individuals whose gender identity, gender expression and behavior differ from their assigned sex at birth, health outcomes in transgenders have been understudied. Many transgender people seek treatment with cross-sex hormone therapy starting from a young age and frequently at high doses in order to obtain the secondary sex characteristics of the desired gender. There is a need to understand the potential long-term health consequences of cross-sex hormone therapy, given that cardiovascular disease is the leading disease-specific cause of death in this population. This review discusses the cardiovascular risks of gender-affirming hormone treatments with respect to transgender women and transgender men.

    View details for DOI 10.1016/j.maturitas.2019.08.010

    View details for PubMedID 31547912

    View details for PubMedCentralID PMC6761990

  • Age at Menarche and Risk of Cardiovascular Disease Outcomes: Findings From the National Heart Lung and Blood Institute-Sponsored Women's Ischemia Syndrome Evaluation. Journal of the American Heart Association Lee, J. J., Cook-Wiens, G., Johnson, B. D., Braunstein, G. D., Berga, S. L., Stanczyk, F. Z., Pepine, C. J., Bairey Merz, C. N., Shufelt, C. L. 2019; 8 (12): e012406

    Abstract

    Background Previous studies have reported an association between the timing of menarche and cardiovascular disease ( CVD ). However, emerging studies have not examined the timing of menarche in relation to role of estrogen over a lifetime and major adverse cardiac events ( MACE ). Methods and Results A total of 648 women without surgical menopause undergoing coronary angiography for suspected ischemia in the WISE (Women's Ischemia Syndrome Evaluation) study were evaluated at baseline and followed for 6 years (median) to assess major adverse CVD outcomes. MACE was defined as the first occurrence of all-cause death, nonfatal myocardial infarction, nonfatal stroke, or heart failure hospitalization. Age at menarche was self-reported and categorized (≤10, 11, 12, 13, 14, ≥15 years) with age 12 as reference. Total estrogen time and supra-total estrogen time were calculated. Cox regression analysis was performed adjusting for CVD risk factors. Baseline age was 57.9 ± 12 years (mean ± SD ), body mass index was 29.5 ± 6.5 kg/m2, total estrogen time was 32.2 ± 8.9 years, and supra-total estrogen time was 41.4 ± 8.8 years. MACE occurred in 172 (27%), and its adjusted regression model was J-shaped. Compared with women with menarche at age 12 years, the adjusted MACE hazard ratio for menarche at ≤10 years was 4.53 (95% CI 2.13-9.63); and at ≥15 years risk for MACE was 2.58 (95% CI , 1.28-5.21). Conclusions History of early or late menarche was associated with a higher risk for adverse CVD outcomes. These findings highlight age at menarche as a potential screening tool for women at risk of adverse CVD events. Clinical Trial Registration URL : http://www.clinicaltrials.gov . Unique identifier: NCT00000554.

    View details for DOI 10.1161/JAHA.119.012406

    View details for PubMedID 31165670

    View details for PubMedCentralID PMC6645646

  • Lagged Versus Difference Score Regression: An Example From a Community-Based Educational Seminar Evaluation Pedagogy in Health Promotion Valente, T., Lee, J., et al 2017
  • Clinical characteristics associated with Spitz nevi and Spitzoid malignant melanomas: the Yale University Spitzoid Neoplasm Repository experience, 1991 to 2008. Journal of the American Academy of Dermatology Lott, J. P., Wititsuwannakul, J., Lee, J. J., Ariyan, S., Narayan, D., Kluger, H. H., Lazova, R. 2014; 71 (6): 1077-82

    Abstract

    Spitz nevi and Spitzoid malignant melanomas are uncommon and may be difficult to distinguish histopathologically. Identification of clinical features associated with these lesions may aid in diagnosis.We sought to identify clinical characteristics associated with Spitz nevi and Spitzoid malignant melanomas.We conducted a retrospective cohort study of Spitz nevi and Spitzoid malignant melanomas from the Yale University Spitzoid Neoplasm Repository diagnosed from years 1991 through 2008. Descriptive statistics and multivariate logistic regression were used to compare select patient- and tumor-level factors associated with each lesion.Our cohort included 484 Spitz nevi and 54 Spitzoid malignant melanomas. Spitz nevi were more common (P = .03) in females (65%; n = 316) compared with Spitzoid malignant melanomas (50%; n = 27), occurred more frequently in younger patients (mean age at diagnosis 22 vs 55 years; P < .001), and more likely presented as smaller lesions (diameter 7.6 vs 10.5 mm; P < .001). Increasing age (odds ratio 1.16, 95% CI [1.09, 1.14]; P< .001) and male gender (odds ratio 2.77, 95% CI [1.17, 6.55]; P< .02) predicted Spitzoid malignant melanoma diagnosis.Small sample size, unmeasured confounding, and restriction to a single institution may limit the accuracy and generalizability of our findings.Age and gender help predict diagnosis of Spitz nevi and Spitzoid malignant melanomas.

    View details for DOI 10.1016/j.jaad.2014.08.026

    View details for PubMedID 25308882

    View details for PubMedCentralID PMC6133655

  • The Association Of Acculturation And Poor Nutritional Behavior Among Latinos In The Los Angeles County Health Survey 2007 Lee, J. J. Public Theses. 2013
  • Artificial Physics, Swarm Engineering, and the Hamiltonian Method World Congress on Engineering and Computer Science Kazadi, S., Lee, J. J., Lee, J. 2007
https://orcid.org/0009-0002-6640-8597