Ron Li is a Clinical Assistant Professor of Medicine in the Division of Hospital Medicine and Center for Biomedical Informatics Research at Stanford University School of Medicine. He is also the Medical Informatics Director for Artificial Intelligence Clinical Integration at Stanford Health Care. Ron's work is centered around the design, implementation, and evaluation of novel systems of care delivery that can be enabled by artificial intelligence. His work spans across multiple disciplines, including clinical medicine, data science, digital health, information technology, design thinking, process improvement, and implementation science. Current areas of focus include using machine learning to improve advance care planning, care of clinically deteriorating patients, and e-consults for the health system. He has consulted for various companies in the digital health and artificial intelligence space. He is a practicing hospitalist and attends on the inpatient medicine teaching service at Stanford Hospital.
- Clinical Informatics
- Hospital Medicine
- Internal Medicine
Clinical Assistant Professor, Medicine
Medical Informatics Director for AI Clinical Integration, Stanford Health Care (2019 - Present)
Honors & Awards
Stanford Human Centered AI Seed Grant Recipient, Stanford (2018)
Fellowship: Stanford University Clinical Informatics Fellowship (2019) CA
Board Certification: American Board of Internal Medicine, Internal Medicine (2018)
Residency: Stanford University Internal Medicine Residency (2017) CA
Medical Education: Northwestern University Feinberg School of Medicine (2014) IL
Internal Medicine Residency, Stanford University
MD, Northwestern University
- Annals for Hospitalists Inpatient Notes - Realizing the Promises of Hospital Electronic Order Sets. Annals of internal medicine 2020; 173 (4): HO2–HO3
- Assessment of a Real-Time Locator System to Identify Physician and Nurse Work Locations. JAMA network open 2020; 3 (2): e1920352
Developing a delivery science for artificial intelligence in healthcare.
NPJ digital medicine
2020; 3: 107
Artificial Intelligence (AI) has generated a large amount of excitement in healthcare, mostly driven by the emergence of increasingly accurate machine learning models. However, the promise of AI delivering scalable and sustained value for patient care in the real world setting has yet to be realized. In order to safely and effectively bring AI into use in healthcare, there needs to be a concerted effort around not just the creation, but also the delivery of AI. This AI "delivery science" will require a broader set of tools, such as design thinking, process improvement, and implementation science, as well as a broader definition of what AI will look like in practice, which includes not just machine learning models and their predictions, but also the new systems for care delivery that they enable. The careful design, implementation, and evaluation of these AI enabled systems will be important in the effort to understand how AI can improve healthcare.
View details for DOI 10.1038/s41746-020-00318-y
View details for PubMedID 32885053
View details for PubMedCentralID PMC7443141
When order sets do not align with clinician workflow: assessing practice patterns in the electronic health record.
BMJ quality & safety
Order sets are widely used tools in the electronic health record (EHR) for improving healthcare quality. However, there is limited insight into how well they facilitate clinician workflow. We assessed four indicators based on order set usage patterns in the EHR that reflect potential misalignment between order set design and clinician workflow needs.We used data from the EHR on all orders of medication, laboratory, imaging and blood product items at an academic hospital and an itemset mining approach to extract orders that frequently co-occurred with order set use. We identified the following four indicators: infrequent ordering of order set items, rapid retraction of medication orders from order sets, additional a la carte ordering of items not included in order sets and a la carte ordering of items despite being listed in the order set.There was significant variability in workflow alignment across the 11 762 order set items used in the 77 421 inpatient encounters from 2014 to 2017. The median ordering rate was 4.1% (IQR 0.6%-18%) and median medication retraction rate was 4% (IQR 2%-10%). 143 (5%) medications were significantly less likely while 68 (3%) were significantly more likely to be retracted than if the same medication was ordered a la carte. 214 (39%) order sets were associated with least one additional item frequently ordered a la carte and 243 (45%) order sets contained at least one item that was instead more often ordered a la carte.Order sets often do not align with what clinicians need at the point of care. Quantitative insights from EHRs may inform how order sets can be optimised to facilitate clinician workflow.
View details for DOI 10.1136/bmjqs-2018-008968
View details for PubMedID 31164486
- Impact of problem-based charting on the utilization and accuracy of the electronic problem list JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION 2018; 25 (5): 548–54
- The Impact of Big Data on the Physician GUIDE TO BIG DATA APPLICATIONS 2018; 26: 415–48
Discordance Between Apolipoprotein B and LDL-Cholesterol in Young Adults Predicts Coronary Artery Calcification The CARDIA Study
JOURNAL OF THE AMERICAN COLLEGE OF CARDIOLOGY
2016; 67 (2): 193–201
High levels of apolipoprotein B (apoB) have been shown to predict atherosclerotic cardiovascular disease (CVD) in adults even in the context of low levels of low-density lipoprotein cholesterol (LDL-C) or non-high-density lipoprotein cholesterol (non-HDL-C).This study aimed to quantify the associations between apoB and the discordance between apoB and LDL-C or non-HDL-C in young adults and measured coronary artery calcium (CAC) in midlife.Data were derived from a multicenter cohort study of young adults recruited at ages 18 to 30 years. All participants with complete baseline CVD risk factor data, including apoB and year 25 (Y25) CAC score, were entered into this study. Presence of CAC was defined as having a positive, nonzero Agatston score as determined by computed tomography. Baseline apoB values were divided into tertiles of 4 mutually exclusive concordant/discordant groups, based on median apoB and LDL-C or non-HDL-C.Analysis included 2,794 participants (mean age: 25 ± 3.6 years; body mass index: 24.5 ± 5 kg/m(2); and 44.4% male). Mean lipid values were as follows: total cholesterol: 177.3 ± 33.1 mg/dl; LDL-C: 109.9 ± 31.1 mg/dl; non-HDL-C: 124.0 ± 33.5 mg/dl; HDL-C: 53 ± 12.8 mg/dl; and apoB: 90.7 ± 24 mg/dl; median triglycerides were 61 mg/dl. Compared with the lowest apoB tertile, higher odds of developing Y25 CAC were seen in the middle (odds ratio [OR]: 1.53) and high (OR: 2.28) tertiles based on traditional risk factor-adjusted models. High apoB and low LDL-C or non-HDL-C discordance was also associated with Y25 CAC in adjusted models (OR: 1.55 and OR: 1.45, respectively).These data suggest a dose-response association between apoB in young adults and the presence of midlife CAC independent of baseline traditional CVD risk factors.
View details for DOI 10.1016/j.jacc.2015.10.055
View details for Web of Science ID 000368114400011
View details for PubMedID 26791067
Validation of Test Performance and Clinical Time Zero for an Electronic Health Record Embedded Severe Sepsis Alert.
Applied clinical informatics
2016; 7 (2): 560-572
Increasing use of EHRs has generated interest in the potential of computerized clinical decision support to improve treatment of sepsis. Electronic sepsis alerts have had mixed results due to poor test characteristics, the inability to detect sepsis in a timely fashion and the use of outside software limiting widespread adoption. We describe the development, evaluation and validation of an accurate and timely severe sepsis alert with the potential to impact sepsis management.To develop, evaluate, and validate an accurate and timely severe sepsis alert embedded in a commercial EHR.The sepsis alert was developed by identifying the most common severe sepsis criteria among a cohort of patients with ICD 9 codes indicating a diagnosis of sepsis. This alert requires criteria in three categories: indicators of a systemic inflammatory response, evidence of suspected infection from physician orders, and markers of organ dysfunction. Chart review was used to evaluate test performance and the ability to detect clinical time zero, the point in time when a patient develops severe sepsis.Two physicians reviewed 100 positive cases and 75 negative cases. Based on this review, sensitivity was 74.5%, specificity was 86.0%, the positive predictive value was 50.3%, and the negative predictive value was 94.7%. The most common source of end-organ dysfunction was MAP less than 70 mm/Hg (59%). The alert was triggered at clinical time zero in 41% of cases and within three hours in 53.6% of cases. 96% of alerts triggered before a manual nurse screen.We are the first to report the time between a sepsis alert and physician chart-review clinical time zero. Incorporating physician orders in the alert criteria improves specificity while maintaining sensitivity, which is important to reduce alert fatigue. By leveraging standard EHR functionality, this alert could be implemented by other healthcare systems.
View details for DOI 10.4338/ACI-2015-11-RA-0159
View details for PubMedID 27437061
View details for PubMedCentralID PMC4941860
- Psoriasis is associated with decreased plasma adiponectin levels beyond cardiometabolic risk factors Clinical and Experimental Dermatology 2014; 39 (1)
- Dietary vitamin K intake and anticoagulation control during the initiation phase of warfarin therapy: A prospective cohort study THROMBOSIS AND HAEMOSTASIS 2013; 110 (1): 195–96
- Abnormal lipoprotein particles and cholesterol efflux capacity in patients with psoriasis Atherosclerosis 2012; 224 (1)