Patrick Sullivan

All Publications

• Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials. JAMA internal medicine Wallach, J. D., Sullivan, P. G., Trepanowski, J. F., Sainani, K. L., Steyerberg, E. W., Ioannidis, J. P. 2017

  Abstract

  Many published randomized clinical trials (RCTs) make claims for subgroup differences.To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P < .05 from an interaction test) and corroborated by subsequent RCTs and meta-analyses.This meta-epidemiological survey examines data sets of trials with at least 1 subgroup claim, including Subgroup Analysis of Trials Is Rarely Easy (SATIRE) articles and Discontinuation of Randomized Trials (DISCO) articles. We used Scopus (updated July 2016) to search for English-language articles citing each of the eligible index articles with at least 1 subgroup finding in the abstract.Articles with a subgroup claim in the abstract with or without evidence of statistical heterogeneity (P < .05 from an interaction test) in the text and articles attempting to corroborate the subgroup findings.Study characteristics of trials with at least 1 subgroup claim in the abstract were recorded. Two reviewers extracted the data necessary to calculate subgroup-level effect sizes, standard errors, and the P values for interaction. For individual RCTs and meta-analyses that attempted to corroborate the subgroup findings from the index articles, trial characteristics were extracted. Cochran Q test was used to reevaluate heterogeneity with the data from all available trials.The number of subgroup claims in the abstracts of RCTs, the number of subgroup claims in the abstracts of RCTs with statistical support (subgroup findings), and the number of subgroup findings corroborated by subsequent RCTs and meta-analyses.Sixty-four eligible RCTs made a total of 117 subgroup claims in their abstracts. Of these 117 claims, only 46 (39.3%) in 33 articles had evidence of statistically significant heterogeneity from a test for interaction. In addition, out of these 46 subgroup findings, only 16 (34.8%) ensured balance between randomization groups within the subgroups (eg, through stratified randomization), 13 (28.3%) entailed a prespecified subgroup analysis, and 1 (2.2%) was adjusted for multiple testing. Only 5 (10.9%) of the 46 subgroup findings had at least 1 subsequent pure corroboration attempt by a meta-analysis or an RCT. In all 5 cases, the corroboration attempts found no evidence of a statistically significant subgroup effect. In addition, all effect sizes from meta-analyses were attenuated toward the null.A minority of subgroup claims made in the abstracts of RCTs are supported by their own data (ie, a significant interaction effect). For those that have statistical support (P < .05 from an interaction test), most fail to meet other best practices for subgroup tests, including prespecification, stratified randomization, and adjustment for multiple testing. Attempts to corroborate statistically significant subgroup differences are rare; when done, the initially observed subgroup differences are not reproduced.

  View details for DOI 10.1001/jamainternmed.2016.9125

  View details for PubMedID 28192563

• Sex based subgroup differences in randomized controlled trials: empirical evidence from Cochrane meta-analyses BMJ-BRITISH MEDICAL JOURNAL Wallach, J. D., Sullivan, P. G., Trepanowski, J. F., Steyerberg, E. W., Ioannidis, J. P. 2016; 355

  Abstract

   To evaluate the frequency, validity, and relevance of statistically significant (P<0.05) sex-treatment interactions in randomized controlled trials in Cochrane meta-analyses. Meta-epidemiological study. Cochrane Database of Systematic Reviews (CDSR) and PubMed. Reviews published in the CDSR with sex-treatment subgroup analyses in the forest plots, using data from randomized controlled trials. Information on the study design and sex subgroup data were extracted from reviews and forest plots that met inclusion criteria. For each statistically significant sex-treatment interaction, the potential for biological plausibility and clinical significance was considered. Among the 41 reviews with relevant data, there were 109 separate treatment-outcome analyses ("topics"). Among the 109 topics, eight (7%) had a statistically significant sex-treatment interaction. The 109 topics included 311 randomized controlled trials (162 with both sexes, 46 with males only, 103 with females only). Of the 162 individual randomized controlled trials that included both sexes, 15 (9%) had a statistically significant sex-treatment interaction. Of four topics where the first published randomized controlled trial had a statistically significant sex-treatment interaction, no meta-analyses that included other randomized controlled trials retained the statistical significance and no meta-analyses showed statistical significance when data from the first published randomized controlled trial were excluded. Of the eight statistically significant sex-treatment interactions from the overall analyses, only three were discussed by the CDSR reviewers for a potential impact on different clinical management for males compared with females. None of these topics had a sex-treatment interaction that influenced treatment recommendations in recent guidelines. UpToDate, an online physician-authored clinical decision support resource, suggested differential management of men and women for one of these sex-treatment interactions. Statistically significant sex-treatment interactions are only slightly more frequent than what would be expected by chance and there is little evidence of subsequent corroboration or clinical relevance of sex-treatment interactions.

  View details for DOI 10.1136/bmj.i5826

  View details for PubMedID 27884869

• Meta-Analysis Comparing Established Risk Prediction Models (EuroSCORE II, STS Score, and ACEF Score) for Perioperative Mortality During Cardiac Surgery. American journal of cardiology Sullivan, P. G., Wallach, J. D., Ioannidis, J. P. 2016; 118 (10): 1574-1582

  Abstract

  A wide variety of multivariable risk models have been developed to predict mortality in the setting of cardiac surgery; however, the relative utility of these models is unknown. This study investigated the literature related to comparisons made between established risk prediction models for perioperative mortality used in the setting of cardiac surgery. A systematic review was conducted to capture studies in cardiac surgery comparing the relative performance of at least 2 prediction models cited in recent guidelines (European System for Cardiac Operative Risk Evaluation [EuroSCORE II], Society for Thoracic Surgeons 2008 Cardiac Surgery Risk Models [STS] score, and Age, Creatinine, Ejection Fraction [ACEF] score) for the outcomes of 1-month or inhospital mortality. For articles that met inclusion criteria, we extracted information on study design, predictive performance of risk models, and potential for bias. Meta-analyses were conducted to calculate a summary estimate of the difference in AUCs between models. We identified 22 eligible studies that contained 33 comparisons among the above models. Meta-analysis of differences in AUCs revealed that the EuroSCORE II and STS score performed similarly (with a summary difference in AUC = 0.00), while outperforming the ACEF score (with summary differences in AUC of 0.10 and 0.08, respectively, p <0.05). Other metrics of discrimination and calibration were presented less consistently, and no study presented any metric of reclassification. Small sample size and absent descriptions of missing data were common in these studies. In conclusion, the EuroSCORE II and STS score outperform the ACEF score on discrimination.

  View details for DOI 10.1016/j.amjcard.2016.08.024

  View details for PubMedID 27687052

• Watershed Hepatocellular Carcinomas: The Risk of Incomplete Response following Transhepatic Arterial Chemoembolization. Journal of vascular and interventional radiology Kothary, N., Takehana, C., Mueller, K., Sullivan, P., Tahvildari, A., Sidhar, V., Rosenberg, J., Louie, J. D., Sze, D. Y. 2015; 26 (8): 1122-1129

  Abstract

  Hepatocellular carcinomas (HCCs) bridging two or more Couinaud-Bismuth segments of the liver ("watershed tumors") can recruit multiple segmental arteries. The primary hypothesis of this study was that fewer watershed tumors show complete response (CR) after chemoembolization, with shorter time to local recurrence. Secondary analysis on the impact on transplantation eligibility in the presence of progressive disease was also performed.A total of 155 transplantation-eligible patients whose HCC met Milan criteria (watershed, n = 83; nonwatershed, n = 72) and was treated with chemoembolization were included. Cone-beam computed tomography (CT) was used for guidance and for confirmation of circumferential uptake. Local response to chemoembolization per modified Response Evaluation Criteria In Solid Tumors and local disease-free survival (DFS) for the index tumor were calculated. Differences were assessed by univariate and multivariate analyses.CR after a single of chemoembolization was observed in 55.4% of watershed tumors and in 72.2% of nonwatershed tumors (P = .045). Estimated DFS intervals were 151 days (95% confidence interval [CI], 93-245 d) and 336 days (95% CI, 231-747 d; P = .040) in the watershed and nonwatershed groups, respectively. Worse DFS was observed with a Model for End-Stage Liver Disease score > 20 (P = .0001), higher Child-Pugh-Turcotte score (P = .049), and watershed location (P = .040). Waiting list drop-off rates were statistically similar between groups.Hepatocellular carcinomas located in the watershed region of the liver have a poorer response to chemoembolization than those located elsewhere. These tumors are associated with worse DFS and require additional treatments to maintain transplantation eligibility per Milan criteria. Cone-beam CT can identify crossover supply and confirm complete geographic drug uptake, possibly reducing (but not eliminating) the risk of incomplete response.

  View details for DOI 10.1016/j.jvir.2015.04.030

  View details for PubMedID 26091800

• TORC1 and class I HDAC inhibitors synergize to suppress mature B cell neoplasms MOLECULAR ONCOLOGY Simmons, J. K., Patel, J., Michalowski, A., Zhang, S., Wei, B., Sullivan, P., Gamache, B., Felsenstein, K., Kuehl, W. M., Simpson, R. M., Zingone, A., Landgren, O., Mock, B. A. 2014; 8 (2): 261-272

  Abstract

  Enhanced proliferative signaling and loss of cell cycle regulation are essential for cancer progression. Increased mitogenic signaling through activation of the mTOR pathway, coupled with deregulation of the Cyclin D/retinoblastoma (Rb) pathway is a common feature of lymphoid malignancies, including plasmacytoma (PCT), multiple myeloma (MM), Burkitt's lymphoma (BL), and mantle cell lymphoma (MCL). Here we evaluate the synergy of pharmacologically affecting both of these critical pathways using the mTOR inhibitor sirolimus and the histone deacetylase inhibitor entinostat. A dose-matrix screening approach found this combination to be highly active and synergistic in a panel of genetically diverse human MM cell lines. Synergy and activity was observed in mouse PCT and human BL and MCL cell lines tested in vitro, as well as in freshly isolated primary MM patient samples tested ex vivo. This combination had minimal effects on healthy donor cells and retained activity when tested in a co-culture system simulating the protective interaction of cancer cells with the tumor microenvironment. Combining sirolimus with entinostat enhanced cell cycle arrest and apoptosis. At the molecular level, entinostat increased the expression of cell cycle negative regulators including CDKN1A (p21) and CDKN2A (p16), while the combination decreased critical growth and survival effectors including Cyclin D, BCL-XL, BIRC5, and activated MAPK.

  View details for DOI 10.1016/j.molonc.2013.11.007

  View details for Web of Science ID 000333507000009

  View details for PubMedID 24429254

  View details for PubMedCentralID PMC3969848

• Molecular genetic variation and population structure in morphologically differentiated cave and surface populations of the freshwater amphipod Gammarus minus MOLECULAR ECOLOGY Carlini, D. B., Manning, J., Sullivan, P. G., Fong, D. W. 2009; 18 (9): 1932-1945

  Abstract

  Gammarus minus is an important component of surface spring and cave ecosystems throughout Appalachia, and is a useful indicator of the hydrology and gene flow in freshwater communities. Gammarus minus populations occupying large cave passages (> 2 km) are usually troglomorphic, having reduced eyes, fewer ommatidia, larger body size, longer antennae, and reduced pigmentation relative to surface populations. We surveyed five cave stream and 10 surface spring populations for DNA sequence variation in the cytochromec oxidase I (COI) and internal transcribed spacer 1 (ITS-1) genes with an aim towards characterizing phylogeographical structure and comparing the nature of genetic variation in cave vs. surface populations. Although standing variation at both loci was rather low within populations, a significant degree of divergence and spatial structuring of populations was observed. Levels of genetic variation within cave and spring populations differed substantially, with caves harbouring significantly less variation at the COI locus than surface springs. Codon usage bias was significantly lower in caves, indicating that cave streams harbour smaller and/or more recently colonized populations. Overall these data indicate limited gene flow among populations and suggest that the cave populations sampled in this study are prone to bottlenecks, possibly due to larger temperature fluctuations and more frequent incidence of drought conditions associated with these particular cave habitats.

  View details for DOI 10.1111/j.1365-294X.2009.04161.x

  View details for Web of Science ID 000265189400012

  View details for PubMedID 19434810