Steven Bagley

All Publications

• A framework for making predictive models useful in practice. Journal of the American Medical Informatics Association : JAMIA Jung, K., Kashyap, S., Avati, A., Harman, S., Shaw, H., Li, R., Smith, M., Shum, K., Javitz, J., Vetteth, Y., Seto, T., Bagley, S. C., Shah, N. H. 2020

  Abstract

  OBJECTIVE: To analyze the impact of factors in healthcare delivery on the net benefit of triggering an Advanced Care Planning (ACP) workflow based on predictions of 12-month mortality.MATERIALS AND METHODS: We built a predictive model of 12-month mortality using electronic health record data and evaluated the impact of healthcare delivery factors on the net benefit of triggering an ACP workflow based on the models' predictions. Factors included nonclinical reasons that make ACP inappropriate: limited capacity for ACP, inability to follow up due to patient discharge, and availability of an outpatient workflow to follow up on missed cases. We also quantified the relative benefits of increasing capacity for inpatient ACP versus outpatient ACP.RESULTS: Work capacity constraints and discharge timing can significantly reduce the net benefit of triggering the ACP workflow based on a model's predictions. However, the reduction can be mitigated by creating an outpatient ACP workflow. Given limited resources to either add capacity for inpatient ACP versus developing outpatient ACP capability, the latter is likely to provide more benefit to patient care.DISCUSSION: The benefit of using a predictive model for identifying patients for interventions is highly dependent on the capacity to execute the workflow triggered by the model. We provide a framework for quantifying the impact of healthcare delivery factors and work capacity constraints on achieved benefit.CONCLUSION: An analysis of the sensitivity of the net benefit realized by a predictive model triggered clinical workflow to various healthcare delivery factors is necessary for making predictive models useful in practice.

  View details for DOI 10.1093/jamia/ocaa318

  View details for PubMedID 33355350

• Estimate the hidden deployment cost of predictive models to improve patient care. Nature medicine Morse, K. E., Bagely, S. C., Shah, N. H. 2020; 26 (1): 18–19

  View details for DOI 10.1038/s41591-019-0651-8

  View details for PubMedID 31932778

• Making Machine Learning Models Clinically Useful. JAMA Shah, N. H., Milstein, A. n., Bagley PhD, S. C. 2019

  View details for DOI 10.1001/jama.2019.10306

  View details for PubMedID 31393527

• Constraints on Biological Mechanism from Disease Comorbidity Using Electronic Medical Records and Database of Genetic Variants PLOS COMPUTATIONAL BIOLOGY Bagley, S. C., Sirota, M., Chen, R., Butte, A. J., Altman, R. B. 2016; 12 (4)

  Abstract

  Patterns of disease co-occurrence that deviate from statistical independence may represent important constraints on biological mechanism, which sometimes can be explained by shared genetics. In this work we study the relationship between disease co-occurrence and commonly shared genetic architecture of disease. Records of pairs of diseases were combined from two different electronic medical systems (Columbia, Stanford), and compared to a large database of published disease-associated genetic variants (VARIMED); data on 35 disorders were available across all three sources, which include medical records for over 1.2 million patients and variants from over 17,000 publications. Based on the sources in which they appeared, disease pairs were categorized as having predominant clinical, genetic, or both kinds of manifestations. Confounding effects of age on disease incidence were controlled for by only comparing diseases when they fall in the same cluster of similarly shaped incidence patterns. We find that disease pairs that are overrepresented in both electronic medical record systems and in VARIMED come from two main disease classes, autoimmune and neuropsychiatric. We furthermore identify specific genes that are shared within these disease groups.

  View details for DOI 10.1371/journal.pcbi.1004885

  View details for Web of Science ID 000376584400019

  View details for PubMedID 27115429

  View details for PubMedCentralID PMC4846031

• Computing disease incidence, prevalence and comorbidity from electronic medical records Journal of Biomedical Informatics Bagley, S. C., Altman, R. B. 2016; 63 (Oct): 108-111

  View details for DOI 10.1016/j.jbi.2016.08.005

• An ontology for Autism Spectrum Disorder (ASD) to infer ASD phenotypes from Autism Diagnostic Interview-Revised data JOURNAL OF BIOMEDICAL INFORMATICS Mugzach, O., Peleg, M., Bagley, S. C., Guter, S. J., Cook, E. H., Altman, R. B. 2015; 56: 333-347

  Abstract

  Our goal is to create an ontology that will allow data integration and reasoning with subject data to classify subjects, and based on this classification, to infer new knowledge on Autism Spectrum Disorder (ASD) and related neurodevelopmental disorders (NDD). We take a first step toward this goal by extending an existing autism ontology to allow automatic inference of ASD phenotypes and Diagnostic & Statistical Manual of Mental Disorders (DSM) criteria based on subjects' Autism Diagnostic Interview-Revised (ADI-R) assessment data.Knowledge regarding diagnostic instruments, ASD phenotypes and risk factors was added to augment an existing autism ontology via Ontology Web Language class definitions and semantic web rules. We developed a custom Protégé plugin for enumerating combinatorial OWL axioms to support the many-to-many relations of ADI-R items to diagnostic categories in the DSM. We utilized a reasoner to infer whether 2642 subjects, whose data was obtained from the Simons Foundation Autism Research Initiative, meet DSM-IV-TR (DSM-IV) and DSM-5 diagnostic criteria based on their ADI-R data.We extended the ontology by adding 443 classes and 632 rules that represent phenotypes, along with their synonyms, environmental risk factors, and frequency of comorbidities. Applying the rules on the data set showed that the method produced accurate results: the true positive and true negative rates for inferring autistic disorder diagnosis according to DSM-IV criteria were 1 and 0.065, respectively; the true positive rate for inferring ASD based on DSM-5 criteria was 0.94.The ontology allows automatic inference of subjects' disease phenotypes and diagnosis with high accuracy.The ontology may benefit future studies by serving as a knowledge base for ASD. In addition, by adding knowledge of related NDDs, commonalities and differences in manifestations and risk factors could be automatically inferred, contributing to the understanding of ASD pathophysiology.

  View details for DOI 10.1016/j.jbi.2015.06.026

  View details for Web of Science ID 000359752100030

  View details for PubMedID 26151311

  View details for PubMedCentralID PMC4532604

• Environmental and State-Level Regulatory Factors Affect the Incidence of Autism and Intellectual Disability PLOS COMPUTATIONAL BIOLOGY Rzhetsky, A., Bagley, S. C., Wang, K., Lyttle, C. S., Cook, E. H., Altman, R. B., Gibbons, R. D. 2014; 10 (3)

  Abstract

  Many factors affect the risks for neurodevelopmental maladies such as autism spectrum disorders (ASD) and intellectual disability (ID). To compare environmental, phenotypic, socioeconomic and state-policy factors in a unified geospatial framework, we analyzed the spatial incidence patterns of ASD and ID using an insurance claims dataset covering nearly one third of the US population. Following epidemiologic evidence, we used the rate of congenital malformations of the reproductive system as a surrogate for environmental exposure of parents to unmeasured developmental risk factors, including toxins. Adjusted for gender, ethnic, socioeconomic, and geopolitical factors, the ASD incidence rates were strongly linked to population-normalized rates of congenital malformations of the reproductive system in males (an increase in ASD incidence by 283% for every percent increase in incidence of malformations, 95% CI: [91%, 576%], p<6×10(-5)). Such congenital malformations were barely significant for ID (94% increase, 95% CI: [1%, 250%], p = 0.0384). Other congenital malformations in males (excluding those affecting the reproductive system) appeared to significantly affect both phenotypes: 31.8% ASD rate increase (CI: [12%, 52%], p<6×10(-5)), and 43% ID rate increase (CI: [23%, 67%], p<6×10(-5)). Furthermore, the state-mandated rigor of diagnosis of ASD by a pediatrician or clinician for consideration in the special education system was predictive of a considerable decrease in ASD and ID incidence rates (98.6%, CI: [28%, 99.99%], p = 0.02475 and 99% CI: [68%, 99.99%], p = 0.00637 respectively). Thus, the observed spatial variability of both ID and ASD rates is associated with environmental and state-level regulatory factors; the magnitude of influence of compound environmental predictors was approximately three times greater than that of state-level incentives. The estimated county-level random effects exhibited marked spatial clustering, strongly indicating existence of as yet unidentified localized factors driving apparent disease incidence. Finally, we found that the rates of ASD and ID at the county level were weakly but significantly correlated (Pearson product-moment correlation 0.0589, p = 0.00101), while for females the correlation was much stronger (0.197, p<2.26×10(-16)).

  View details for DOI 10.1371/journal.pcbi.1003518

  View details for Web of Science ID 000336509000034

  View details for PubMedID 24625521

  View details for PubMedCentralID PMC3952819

• Identifying Patients at Risk for Suicide: Brief Review Making Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient Safety Practices Bagley, S. C. AHRQ. 2013: 287–296
• Predicting Suicide Attempt Risk: Logistic Regression Requires Large Sample Sizes JOURNAL OF CLINICAL PSYCHIATRY Bagley, S. C. 2011; 72 (12): 1698-1698

  View details for DOI 10.4088/JCP.11lr07393

  View details for Web of Science ID 000298778500020

  View details for PubMedID 22244029

• Efficacy and Comparative Effectiveness of Atypical Antipsychotic Medications for Off-Label Uses in Adults A Systematic Review and Meta-analysis JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION Maher, A. R., Maglione, M., Bagley, S., Suttorp, M., Hu, J., Ewing, B., Wang, Z., Timmer, M., Sultzer, D., Shekelle, P. G. 2011; 306 (12): 1359-1369

  Abstract

  Atypical antipsychotic medications are commonly used for off-label conditions such as agitation in dementia, anxiety, and obsessive-compulsive disorder.To perform a systematic review on the efficacy and safety of atypical antipsychotic medications for use in conditions lacking approval for labeling and marketing by the US Food and Drug Administration.Relevant studies published in the English language were identified by searches of 6 databases (PubMed, EMBASE, CINAHL, PsycInfo, Cochrane DARE, and CENTRAL) from inception through May 2011. Controlled trials comparing an atypical antipsychotic medication (risperidone, olanzapine, quetiapine, aripiprazole, ziprasidone, asenapine, iloperidone, or paliperidone) with placebo, another atypical antipsychotic medication, or other pharmacotherapy for adult off-label conditions were included. Observational studies with sample sizes of greater than 1000 patients were included to assess adverse events.Independent article review and study quality assessment by 2 investigators.Of 12 228 citations identified, 162 contributed data to the efficacy review. Among 14 placebo-controlled trials of elderly patients with dementia reporting a total global outcome score that includes symptoms such as psychosis, mood alterations, and aggression, small but statistically significant effects sizes ranging from 0.12 and 0.20 were observed for aripiprazole, olanzapine, and risperidone. For generalized anxiety disorder, a pooled analysis of 3 trials showed that quetiapine was associated with a 26% greater likelihood of a favorable response (defined as at least 50% improvement on the Hamilton Anxiety Scale) compared with placebo. For obsessive-compulsive disorder, risperidone was associated with a 3.9-fold greater likelihood of a favorable response (defined as a 25% improvement on the Yale-Brown Obsessive Compulsive Scale) compared with placebo. In elderly patients, adverse events included an increased risk of death (number needed to harm [NNH] = 87), stroke (NNH = 53 for risperidone), extrapyramidal symptoms (NNH = 10 for olanzapine; NNH = 20 for risperidone), and urinary tract symptoms (NNH range = 16-36). In nonelderly adults, adverse events included weight gain (particularly with olanzapine), fatigue, sedation, akathisia (for aripiprazole), and extrapyramidal symptoms.Benefits and harms vary among atypical antipsychotic medications for off-label use. For global behavioral symptom scores associated with dementia in elderly patients, small but statistically significant benefits were observed for aripiprazole, olanzapine, and risperidone. Quetiapine was associated with benefits in the treatment of generalized anxiety disorder, and risperidone was associated with benefits in the treatment of obsessive-compulsive disorder; however, adverse events were common.

  View details for Web of Science ID 000295257000025

  View details for PubMedID 21954480

• A Systematic Review of Suicide Prevention Programs for Military or Veterans SUICIDE AND LIFE-THREATENING BEHAVIOR Bagley, S. C., Munjas, B., Shekelle, P. 2010; 40 (3): 257-265

  Abstract

  Military personnel and veterans have important suicide risk factors. After a systematic review of the literature on suicide prevention, seven (five in the U.S.) studies of military personnel were identified containing interventions that may reduce the risk of suicide. The effectiveness of the individual components was not assessed, and problems in methodology or reporting of data were common. Overall, multifaceted interventions for active duty military personnel are supported by consistent evidence, although of very mixed quality, and in some cases during intervals of declines in suicide rates in the general population. There were insufficient studies of U.S. Veterans to reach conclusions.

  View details for Web of Science ID 000278740200005

  View details for PubMedID 20560747

• Logistic regression in the medical literature: Standards for use and reporting, with particular attention to one medical domain JOURNAL OF CLINICAL EPIDEMIOLOGY Bagley, S. C., White, H., Golomb, B. A. 2001; 54 (10): 979-985

  Abstract

  Logistic regression (LR) is a widely used multivariable method for modeling dichotomous outcomes. This article examines use and reporting of LR in the medical literature by comprehensively assessing its use in a selected area of medical study. Medline, followed by bibliography searches, identified 15 peer-reviewed English-language articles with original data, employing LR, published between 1985 and 1999, pertaining to patient interest in genetic testing for cancer susceptibility. Articles were examined for each of 10 criteria for proper use and reporting of LR models. Substantial shortcomings were found in both use of LR and reporting of results. For many studies, the ratio of the number of outcome events to predictor variables (events per variable) was sufficiently small to call into question the accuracy of the regression model. Additionally, no studies reported validation analysis, regression diagnostics, or goodness-of-fit measures. It is recommended that authors, reviewers, and editors pay greater attention to guidelines concerning the use and reporting of LR models.

  View details for Web of Science ID 000171047800003

  View details for PubMedID 11576808

• CHARACTERIZING THE MICROENVIRONMENT SURROUNDING PROTEIN SITES PROTEIN SCIENCE Bagley, S. C., Altman, R. B. 1995; 4 (4): 622-635

  Abstract

  Sites are microenvironments within a biomolecular structure, distinguished by their structural or functional role. A site can be defined by a three-dimensional location and a local neighborhood around this location in which the structure or function exists. We have developed a computer system to facilitate structural analysis (both qualitative and quantitative) of biomolecular sites. Our system automatically examines the spatial distributions of biophysical and biochemical properties, and reports those regions within a site where the distribution of these properties differs significantly from control nonsites. The properties range from simple atom-based characteristics such as charge to polypeptide-based characteristics such as type of secondary structure. Our analysis of sites uses non-sites as controls, providing a baseline for the quantitative assessment of the significance of the features that are uncovered. In this paper, we use radial distributions of properties to study three well-known sites (the binding sites for calcium, the milieu of disulfide bridges, and the serine protease active site). We demonstrate that the system automatically finds many of the previously described features of these sites and augments these features with some new details. In some cases, we cannot confirm the statistical significance of previously reported features. Our results demonstrate that analysis of protein structure is sensitive to assumptions about background distributions, and that these distributions should be considered explicitly during structural analyses.

  View details for Web of Science ID A1995QU44000004

  View details for PubMedID 7613462