Contact
https://orcid.org/0000-0002-1302-4875All Publications
-
Investigating real-world consequences of biases in commonly used clinical calculators.
The American journal of managed care
2023; 29 (1): e1-e7
Abstract
OBJECTIVES: To evaluate whether one summary metric of calculator performance sufficiently conveys equity across different demographic subgroups, as well as to evaluate how calculator predictive performance affects downstream health outcomes.STUDY DESIGN: We evaluate 3 commonly used clinical calculators-Model for End-Stage Liver Disease (MELD), CHA2DS2-VASc, and simplified Pulmonary Embolism Severity Index (sPESI)-on the cohort extracted from the Stanford Medicine Research Data Repository, following the cohort selection process as described in respective calculator derivation papers.METHODS: We quantified the predictive performance of the 3 clinical calculators across sex and race. Then, using the clinical guidelines that guide care based on these calculators' output, we quantified potential disparities in subsequent health outcomes.RESULTS: Across the examined subgroups, the MELD calculator exhibited worse performance for female and White populations, CHA2DS2-VASc calculator for the male population, and sPESI for the Black population. The extent to which such performance differences translated into differential health outcomes depended on the distribution of the calculators' scores around the thresholds used to trigger a care action via the corresponding guidelines. In particular, under the old guideline for CHA2DS2-VASc, among those who would not have been offered anticoagulant therapy, the Hispanic subgroup exhibited the highest rate of stroke.CONCLUSIONS: Clinical calculators, even when they do not include variables such as sex and race as inputs, can have very different care consequences across those subgroups. These differences in health care outcomes across subgroups can be explained by examining the distribution of scores and their calibration around the thresholds encoded in the accompanying care guidelines.
View details for DOI 10.37765/ajmc.2023.89306
View details for PubMedID 36716157
-
A model for comprehensive climate and medical education.
The Lancet. Planetary health
2023; 7 (1): e2-e3
View details for DOI 10.1016/S2542-5196(22)00215-7
View details for PubMedID 36608944
-
A model for comprehensive climate and medical education
LANCET PLANETARY HEALTH
2023; 7 (1): E2-E3
View details for Web of Science ID 001031242800001
-
Assessment of Adherence to Reporting Guidelines by Commonly Used Clinical Prediction Models From a Single Vendor: A Systematic Review.
JAMA network open
2022; 5 (8): e2227779
Abstract
Importance: Various model reporting guidelines have been proposed to ensure clinical prediction models are reliable and fair. However, no consensus exists about which model details are essential to report, and commonalities and differences among reporting guidelines have not been characterized. Furthermore, how well documentation of deployed models adheres to these guidelines has not been studied.Objectives: To assess information requested by model reporting guidelines and whether the documentation for commonly used machine learning models developed by a single vendor provides the information requested.Evidence Review: MEDLINE was queried using machine learning model card and reporting machine learning from November 4 to December 6, 2020. References were reviewed to find additional publications, and publications without specific reporting recommendations were excluded. Similar elements requested for reporting were merged into representative items. Four independent reviewers and 1 adjudicator assessed how often documentation for the most commonly used models developed by a single vendor reported the items.Findings: From 15 model reporting guidelines, 220 unique items were identified that represented the collective reporting requirements. Although 12 items were commonly requested (requested by 10 or more guidelines), 77 items were requested by just 1 guideline. Documentation for 12 commonly used models from a single vendor reported a median of 39% (IQR, 37%-43%; range, 31%-47%) of items from the collective reporting requirements. Many of the commonly requested items had 100% reporting rates, including items concerning outcome definition, area under the receiver operating characteristics curve, internal validation, and intended clinical use. Several items reported half the time or less related to reliability, such as external validation, uncertainty measures, and strategy for handling missing data. Other frequently unreported items related to fairness (summary statistics and subgroup analyses, including for race and ethnicity or sex).Conclusions and Relevance: These findings suggest that consistent reporting recommendations for clinical predictive models are needed for model developers to share necessary information for model deployment. The many published guidelines would, collectively, require reporting more than 200 items. Model documentation from 1 vendor reported the most commonly requested items from model reporting guidelines. However, areas for improvement were identified in reporting items related to model reliability and fairness. This analysis led to feedback to the vendor, which motivated updates to the documentation for future users.
View details for DOI 10.1001/jamanetworkopen.2022.27779
View details for PubMedID 35984654
-
Nursing Workflow Change in a COVID-19 Inpatient Unit Following the Deployment of Inpatient Telehealth: An Observational Study Using a Real-Time Locating System.
Journal of medical Internet research
2022
Abstract
BACKGROUND: The COVID-19 pandemic prompted widespread implementation of telehealth, including in the inpatient setting with the goals to reduce potential pathogen exposure events and personal protective equipment (PPE) utilization. Nursing workflow adaptations in these novel environments is of particular interest given the association between nursing time at the bedside and patient safety. Understanding the frequency and duration of nurse-patient encounters following the introduction of a novel telehealth platform in the context of COVID-19 may therefore provide insight into downstream impacts on patient safety, pathogen exposure, and PPE utilization.OBJECTIVE: To evaluate changes in nursing workflow relative to pre-pandemic levels using real-time locating system (RTLS) following the deployment of inpatient telehealth on a COVID-19 unit.METHODS: In March 2020, telehealth was installed in patient rooms in a COVID-19 unit and on movable carts in 3 comparison units. Existing RTLS captured nurse movement during 1 pre- and 5 post-pandemic stages (January-December 2020). Change in direct nurse-patient encounters, time spent in patient rooms per encounter, and total time spent with patients per shift relative to baseline were calculated. Generalized linear models assessed difference-in-differences in outcomes between COVID-19 and comparison units. Telehealth adoption was captured and reported at the unit level.RESULTS: Change in frequency of encounters and time spent per encounter from baseline differed between the COVID-19 and comparison units at all stages of the pandemic (all P's<0.0001). Frequency of encounters decreased (difference-in-differences range: -6.6 to -14.1 encounters) and duration of encounters increased (difference-in-differences range: 1.8 to 6.2 minutes) from baseline to a greater extent in the COVID-19 units compared to the comparison units. At most stages of the pandemic, the change in total time nurses spent in patient rooms per patient per shift from baseline did not differ between the COVID-19 and comparison units (p's>0.17). The primary COVID-19 unit quickly adopted telehealth technology during the observation period, initiating 15,088 encounters that averaged 6.6 minutes (standard deviation = 13.6) each.CONCLUSIONS: RTLS movement data suggests total nursing time at the bedside remained unchanged following the deployment of inpatient telehealth in a COVID-19 unit. Compared to other units with shared mobile telehealth units, frequency of nurse-patient in-person encounters decreased and duration lengthened on a COVID-19 unit with in-room telehealth availability, indicating "batched" redistribution of work to maintain total time at bedside relative to pre-pandemic periods. The simultaneous adoption of telehealth suggests virtual care was a complement to, rather than a replacement for, in-person care. Study limitations, however, preclude our ability to draw a causal link between nursing workflow change and telehealth adoption, and further evaluation is needed to determine potential downstream implications on disease transmission, PPE utilization, and patient safety.CLINICALTRIAL:
View details for DOI 10.2196/36882
View details for PubMedID 35635840
-
Monitoring Approaches for a Pediatric Chronic Kidney Disease Machine Learning Model.
Applied clinical informatics
2022; 13 (2): 431-438
Abstract
OBJECTIVE: The purpose of this study is to evaluate the ability of three metrics to monitor for a reduction in performance of a chronic kidney disease (CKD) model deployed at a pediatric hospital.METHODS: The CKD risk model estimates a patient's risk of developing CKD 3 to 12 months following an inpatient admission. The model was developed on a retrospective dataset of 4,879 admissions from 2014 to 2018, then run silently on 1,270 admissions from April to October, 2019. Three metrics were used to monitor its performance during the silent phase: (1) standardized mean differences (SMDs); (2) performance of a "membership model"; and (3) response distribution analysis. Observed patient outcomes for the 1,270 admissions were used to calculate prospective model performance and the ability of the three metrics to detect performance changes.RESULTS: The deployed model had an area under the receiver-operator curve (AUROC) of 0.63 in the prospective evaluation, which was a significant decrease from an AUROC of 0.76 on retrospective data (p=0.033). Among the three metrics, SMDs were significantly different for 66/75 (88%) of the model's input variables (p <0.05) between retrospective and deployment data. The membership model was able to discriminate between the two settings (AUROC=0.71, p <0.0001) and the response distributions were significantly different (p <0.0001) for the two settings.CONCLUSION: This study suggests that the three metrics examined could provide early indication of performance deterioration in deployed models' performance.
View details for DOI 10.1055/s-0042-1746168
View details for PubMedID 35508197
-
Considerations in the reliability and fairness audits of predictive models for advance care planning
Frontiers in Digital Health
2022: 943768
View details for DOI 10.3389/fdgth.2022.943768
-
Causal network inference from gene transcriptional time-series response to glucocorticoids
PLOS COMPUTATIONAL BIOLOGY
2021; 17 (1): e1008223
Abstract
Gene regulatory network inference is essential to uncover complex relationships among gene pathways and inform downstream experiments, ultimately enabling regulatory network re-engineering. Network inference from transcriptional time-series data requires accurate, interpretable, and efficient determination of causal relationships among thousands of genes. Here, we develop Bootstrap Elastic net regression from Time Series (BETS), a statistical framework based on Granger causality for the recovery of a directed gene network from transcriptional time-series data. BETS uses elastic net regression and stability selection from bootstrapped samples to infer causal relationships among genes. BETS is highly parallelized, enabling efficient analysis of large transcriptional data sets. We show competitive accuracy on a community benchmark, the DREAM4 100-gene network inference challenge, where BETS is one of the fastest among methods of similar performance and additionally infers whether causal effects are activating or inhibitory. We apply BETS to transcriptional time-series data of differentially-expressed genes from A549 cells exposed to glucocorticoids over a period of 12 hours. We identify a network of 2768 genes and 31,945 directed edges (FDR ≤ 0.2). We validate inferred causal network edges using two external data sources: Overexpression experiments on the same glucocorticoid system, and genetic variants associated with inferred edges in primary lung tissue in the Genotype-Tissue Expression (GTEx) v6 project. BETS is available as an open source software package at https://github.com/lujonathanh/BETS.
View details for DOI 10.1371/journal.pcbi.1008223
View details for Web of Science ID 000613893600001
View details for PubMedID 33513136
View details for PubMedCentralID PMC7875426
-
COVID-19 Solutions Are Climate Solutions: Lessons From Reusable Gowns
Frontiers in Public Health
2020
View details for DOI 10.3389/fpubh.2020.590275
-
Improved Peak Detection and Deconvolution of Native Electrospray Mass Spectra from Large Protein Complexes
JOURNAL OF THE AMERICAN SOCIETY FOR MASS SPECTROMETRY
2015; 26 (12): 2141-2151
Abstract
Native electrospray-ionization mass spectrometry (native MS) measures biomolecules under conditions that preserve most aspects of protein tertiary and quaternary structure, enabling direct characterization of large intact protein assemblies. However, native spectra derived from these assemblies are often partially obscured by low signal-to-noise as well as broad peak shapes because of residual solvation and adduction after the electrospray process. The wide peak widths together with the fact that sequential charge state series from highly charged ions are closely spaced means that native spectra containing multiple species often suffer from high degrees of peak overlap or else contain highly interleaved charge envelopes. This situation presents a challenge for peak detection, correct charge state and charge envelope assignment, and ultimately extraction of the relevant underlying mass values of the noncovalent assemblages being investigated. In this report, we describe a comprehensive algorithm developed for addressing peak detection, peak overlap, and charge state assignment in native mass spectra, called PeakSeeker. Overlapped peaks are detected by examination of the second derivative of the raw mass spectrum. Charge state distributions of the molecular species are determined by fitting linear combinations of charge envelopes to the overall experimental mass spectrum. This software is capable of deconvoluting heterogeneous, complex, and noisy native mass spectra of large protein assemblies as demonstrated by analysis of (1) synthetic mononucleosomes containing severely overlapping peaks, (2) an RNA polymerase II/α-amanitin complex with many closely interleaved ion signals, and (3) human TriC complex containing high levels of background noise. Graphical Abstract ᅟ.
View details for DOI 10.1007/s13361-015-1235-6
View details for Web of Science ID 000365116500020
View details for PubMedID 26323614