Joseph Romano
Professor of Statistics and of Economics
Bio
PERSONAL BACKGROUND:
I grew up in an Italian New Jersey family, and attended Princeton where I was influenced by John Tukey and terrific mentor Nick Jewell. I got my Ph.D. from Berkeley in 1986, where I was fortunate enough to meet many of the great statisticians of the 20th century, including Erich Lehmann, Lucien LeCam, David Freedman, Rudy Beran, and others. I joined the faculty at Stanford in 1986 when I was 25 and have been at Stanford ever since. My professional life combines intellectual advancement, teaching, and mentoring of young students and researchers by sharing of knowledge and promoting academic integrity. I am proud to be part of the 500 Queer Scientists visibility campaign. I also lead a balanced life with passions in music (having performed at Carnegie Hall), tennis (ranked nationally in my age group), cooking, architecture, and other interests.
RESEARCH GOALS:
Statistics is concerned with making sense or inferences about the world based on limited information and uncertainties. In contrast, mathematics is exact, where the goal is to prove theorems based on a well-defined set of assumptions. It is the juxtaposition of statistics and mathematics that I find intriguing and challenging. Mathematical statistics serves to precisely quantify and explain what can be learned from data in spite of having to acknowledge our uncertainty in the process.
While much of my own research has been theoretically oriented, I have been motivated by a desire to develop practical statistical methodology in order to construct techniques that may be applied safely in practice. I have been particularly interested in advancing "nonparametric" techniques that do not rely on the statistician having to invoke unverifiable assumptions. In my work, I have tried to explore the extent of applicability of bootstrap, subsampling, and other resampling methods, as well as understanding their limitations.
In recent years, I have been interested in developing new methods for multiple testing and multivariate inference, especially driven by the availability of massive data sets. For example current methods in biotechnology generate ultra high throughput data, where expression levels in tens of thousands of genes or SNP data with hundreds of thousands of locations must be analyzed simultaneously. Multiple testing methods can be used to understand the hidden structure in the data rather than random artifacts (due to "data snooping"). In addition, the analysis of data is complicated by large number of features with unknown dependence structures, heterogeneity, model fitting, high dimensionality and other unknown sources of variation. The statistician is then faced with the challenge of accounting for all possible errors resulting from a complex data analysis, so that any resulting inferences or interesting conclusions can reliable be viewed as real structure (and is reproducible or has predictive power). Thus, my goals are the development of universal statistical tools that can be applied to such diverse fields as econometrics, climate science, genetics, clinical trials, finance, education, etc. The many burgeoning fields of applications demand new statistical methods, creating exciting opportunities for statisticians and data scientists.
Administrative Appointments
-
Assistant Professor, Department of Statistics, Stanford University (1986 - 1994)
-
Associate Professor, Department of Statistics, Stanford University (1994 - 2000)
-
Full Professorship in Statistics, Stanford University (2000 - Present)
-
Joint Professorship, Stanford University (2007 - Present)
Honors & Awards
-
2021 LGBTQ+ Scientist of the Year, Out to Innovate (National Organization of Gay and Lesbian Scientists and Technical Professionals) (2021)
-
Fellow, International Association of Applied Econometrics (2020)
-
Computer-intensive Inference with Applications to Social Sciences, National Science Foundation Grant (July 2020-June 2023)
-
Randomization Inference for Contemporary Problems in Statistics, National Science Foundation Research Grant (July 2013-June 2016)
-
Multiple Problems in Multiple Testing and Simultaneous Inference, National Science Foundation Research Grant (July 2010 - June 2013)
-
New Methodology for Multiple Testing and Simultaneous Inference, National Science Foundation Research Grant (July 2007 - June 2010)
-
Theory and Methods for Multiple Testing and Inference, National Science Foundation Research Grant (July 2004 - June 2007)
-
Approximate and Exact Inference Via Computer Intensive Methods, National Science Foundation Research Grant (July 2001 - June 2004)
-
Computer-Intensive Methods for the Statistical Analysis of Dependent Data, National Science Foundation Research Grant (September 1997 - August 2000)
-
Fellow, Institute of Mathematical Statistics (2000)
-
Computer Intensive Methods for the Statistical Analysis of Time Series and Random Fields, National Science Foundation Research Grant (September 1994 - August 1997)
-
Presidential Young Investigator Award, National Science Foundation (1989-1994)
-
The Canadian Journal of Statistics Award, Statistical Society of Canada (1989)
-
Postdoctoral Fellowship, National Science Foundation (1986-1988)
-
Graduate Fellowship, National Science Foundation (1982-1984)
-
Collegiate Award, Northern New Jersey Chapter of the American Statistical Association (1982)
-
Graduated Summa Cum Laude in Statistics, Princeton University (1982)
-
Member, Phi Beta Kappa Society (1982)
Boards, Advisory Committees, Professional Organizations
-
Fellow, Institute of Mathematical Statistics
-
Grant Proposal Reviewer, National Security Agency Mathematical Sciences Program
-
Grant Proposal Reviewer, National Science Foundation
-
Grant Proposal Reviewer, Natural Science and Engineering Research Council of Canada
-
Member, American Statistical Association
-
Referee, Algorithmic Learning Theory
-
Referee, Biometrics
-
Referee, Journal of Statistical Planning and Inference
-
Referee, Journal of Statistical Computation and Simulation
-
Referee, The Scandinavian Journal of Statistics
-
Referee, Journal of Econometrics
-
Referee, British Journal of Mathematical and Statistical Psychology
-
Referee, Statistical Science
-
Referee, Biometrika
-
Referee, The American Statistician
-
Referee, Journal of the Italian Statistical Society
-
Referee, Transactions on Signal Processing
-
Referee, Technometrics
-
Referee, Proceedings of the American Mathematical Society
-
Referee, Communications in Statistics
-
Referee, Journal of the American Statistical Association
-
Referee, Bernoulli
-
Referee, Journal of the Royal Statistical Association
-
Referee, The British Journal of Mathematical and Statistical Psychology
-
Referee, Statistica Sinica
-
Referee, Journal of Time Series Analysis
-
Referee, Psychometrika
-
Referee, Annals of Probability
-
Referee, Econometrica
-
Referee, International Statistical Review
-
Referee, Annals of Statistics
-
Referee, Annals of the Institute of Statistical Mathematics
-
Referee, The Canadian Journal of Statistics
-
Referee, The Journal of Nonparametric Statistics
-
Fellow, International Association of Applied Econometrics (2020 - Present)
-
Associate Chairman, Stanford University (2013 - 2014)
-
Chair of Qualifying Exams, Stanford University (2012 - 2013)
-
Master's Advisor, Stanford University (2012 - 2013)
-
Member, Ph.D. Admissions Committee, Stanford University (2011 - 2012)
-
Chair of Qualifying Exams, Stanford University (2010 - 2011)
-
Advisor to Master's Degree Students, Stanford University (2009 - 2010)
-
Advisor to Master's Degree Students, Stanford University (2008 - 2009)
-
Member, Faculty Affairs Committee, Stanford University (2008 - 2009)
-
Chair of Committee on Faculty Affairs, Stanford University (2007 - 2008)
-
Vice Chairman of the Department of Statistics, Stanford University (2007 - 2008)
-
Vice Chairman, Noether Award Committee, American Statistical Association (2007 - 2007)
-
Member, Noether Award Committee, American Statistical Association (2006 - 2011)
-
Associate Editor, The Annals of Applied Statistics (2006 - 2010)
-
Master's Degree Advisor, Stanford University (2006 - 2007)
-
Member, Committee on Faculty Affairs, Stanford University (2006 - 2007)
-
Master's Degree Advisor, Stanford University (2005 - 2006)
-
Member, Ph.D. Admissions Committee, Stanford University (2005 - 2006)
-
Advisor to all students in the Master's Degree and Ph.D. Minor Programs, Stanford University (2004 - 2005)
-
Chair of Qualifying Exam Committee, Stanford University (2004 - 2005)
-
Member, Grant Proposal Panel, National Science Foundation (2004 - 2004)
-
Advisor to all students in the Master's Degree and Ph.D. Minor Programs, Stanford University (2002 - 2004)
-
Associate Editor, The Annals of Statistics (2001 - 2004)
-
Chair of the Qualifying Exam Committee, Stanford University (2001 - 2002)
-
Member, Ph.D. Program Committee, Stanford University (2001 - 2002)
-
Chair of Qualifying Exam Committee, Stanford University (2000 - 2001)
-
Member, Judicial Panel, Stanford University (2000 - 2001)
-
Ph.D. advisor to first and second year Ph.D. students, Stanford University (1999 - 2000)
-
Chair of Qualifying Exam Committee, Stanford University (1998 - 1999)
-
Ph.D. advisor to first and second year Ph.D. students, Stanford University (1998 - 1999)
-
Chair of Student Selection Committee, Stanford University (1997 - 1998)
-
Ph.D. advisor to first and second year Ph.D. students, Stanford University (1997 - 1998)
-
Chair of Contributed Papers, Annual Meeting of the Institute of Mathematical Statistics (1997 - 1997)
-
Associate Editor, The Journal of Statistical Planning and Inference (1996 - 1999)
-
Ph.D. advisor to to first and second year Ph.D. students, Stanford University (1996 - 1997)
-
Advisor to Ph.D students without thesis advisors, Stanford University (1995 - 1996)
-
Member, Search Committee, Stanford University (1995 - 1996)
-
First Year Ph.D. Student Advisor, Stanford University (1994 - 1995)
-
Qualifying Exam Advisor, Stanford University (1994 - 1995)
-
Member, Ph.D. Exam Committee, Stanford University (1993 - 1994)
-
Ph.D. Student Advisor, Stanford University (1992 - 1993)
-
Masters Student Advisor, Stanford University (1991 - 1992)
-
Member, Ph.D. Exam Committee, Stanford University (1991 - 1992)
-
Member, Affirmative Action Committee, Stanford University (1990 - 1991)
-
Member, Ph.D. Exam Committee, Stanford University (1990 - 1991)
-
Masters Student Advisor, Stanford University (1989 - 1990)
-
Member, Ph.D. Exam Committee, Stanford University (1989 - 1990)
-
Ph.D. Student Selection, Stanford University (1989 - 1990)
-
Member, Curriculum Committee, Stanford University (1988 - 1989)
-
Masters Student Advisor, Stanford University (1987 - 1988)
-
Seminar Chairperson, Ph.D. Exam Committee, Stanford University (1986 - 1987)
Professional Education
-
Ph.D., University of California, Berkeley (1986)
-
M.S., University of California, Berkeley (1983)
-
A.B., Princeton University (1982)
Current Research and Scholarly Interests
Work in progress is described under "Projects"
Projects
-
Control of directional errors in fixed sequence multiple testing (with Anjana Grandhi and Wenge Guo)
-
Testing for differences between random processes in sample-starved regimes (with Bala Rajaratnam and Michael Tsiang)
Location
Stanford, CA
-
Randomization tests under an approximate symmetry assumption (with Ivan Canay and Azeem Shaikh)
-
Improved weighted least squares inference (with Cyrus DiCiccio and Michael Wolf)
Location
CA
-
Analysis of error control in large scale two stage multiple testing (with Wenge Guo)
Location
CA
2024-25 Courses
- Econometric Methods I
MGTECON 603 (Aut) - Econometrics Seminar
ECON 370 (Aut, Win, Spr) - Intermediate Econometrics I
ECON 270 (Aut) - Theory of Statistics I
STATS 300A (Aut) -
Independent Studies (7)
- Directed Reading
ECON 139D (Aut, Win, Spr) - Directed Reading
ECON 239D (Aut, Win, Spr) - Honors Thesis Research
ECON 199D (Aut, Win, Spr) - Independent Study
STATS 299 (Aut, Win, Spr) - Industrial Research for Statisticians
STATS 398 (Aut, Win, Spr) - Practical Training
ECON 299 (Aut, Win, Spr) - Research
STATS 399 (Aut, Win, Spr)
- Directed Reading
-
Prior Year Courses
2023-24 Courses
- Advanced Econometrics II
ECON 274 (Spr) - Econometrics Seminar
ECON 370 (Aut, Win, Spr) - Theory of Statistics I
STATS 300A (Aut)
2022-23 Courses
- Advanced Econometrics II
ECON 274 (Spr) - Econometric Methods I
MGTECON 603 (Aut) - Econometrics Workshop
ECON 370 (Aut, Win, Spr) - Intermediate Econometrics I
ECON 270 (Aut) - Theory of Statistics I
STATS 300A (Aut)
2021-22 Courses
- Econometric Methods I
MGTECON 603 (Aut) - Econometrics Workshop
ECON 370 (Aut, Win, Spr) - Intermediate Econometrics I
ECON 270 (Aut) - Introduction to Statistical Inference
STATS 200 (Aut) - Statistical Methods in Engineering and the Physical Sciences
STATS 110 (Spr)
- Advanced Econometrics II
Stanford Advisees
-
Doctoral Dissertation Reader (AC)
Annette Jing -
Doctoral Dissertation Co-Advisor (AC)
David Ritzwoller -
Doctoral (Program)
Zhaomeng Chen, John Cherian, Noah Cowan, Paula Gablenz, Disha Ghandwani, Aditya Ghosh, Xavier Gonzalez, Dileka Gunawardana, Will Hartog, Valérie, Ngoc Phuc Ho, Michael Howes, Amber Hu, Yujin Jeong, Wenlong Ji, Rahul Raphael Kanekar, Etaash Katiyar, Josh Kazdan, Jack Krew, Joonhyuk Lee, Harrison Li, Puheng Li, Leda Liang, Sophia Lu, Ginnie Ma, Matthew MacKay, Tim Morrison, Yash Nair, Michael Salerno, Jing Shang, Rex Shen, Henry Smith, Ziang Song, Anav Sood, Asher Spector, Timothy Sudijono, Ian Christopher Tanoh, Nathan Tung, Viet Vu, Ran Xie, Allison Xu, James Yang, Zitong Yang, Ivy Zhang, Julie Zhang, Sarah Zhao, Yanxin Zhou
All Publications
-
Covariate adjustment in experiments with matched pairs
JOURNAL OF ECONOMETRICS
2024; 241 (1)
View details for DOI 10.1016/j.jeconom.2024.105740
View details for Web of Science ID 001232620400001
-
The choice-wide behavioral association study: data-driven identification of interpretable behavioral components.
bioRxiv : the preprint server for biology
2024
Abstract
Behavior contains rich structure across many timescales, but there is a dearth of methods to identify relevant components, especially over the longer periods required for learning and decision-making. Inspired by the goals and techniques of genome-wide association studies, we present a data-driven method-the choice-wide behavioral association study: CBAS-that systematically identifies such behavioral features. CBAS uses powerful, resampling-based, methods of multiple comparisons correction1-3 to identify sequences of actions or choices that either differ significantly between groups or significantly correlate with a covariate of interest. We apply CBAS to different tasks and species (flies4, rats5, and humans6) and find, in all instances, that it provides interpretable information about each behavioral task.
View details for DOI 10.1101/2024.02.26.582115
View details for PubMedID 38464037
-
Inference for Ranks with Applications to Mobility across Neighbourhoods and Academic Achievement across Countries
REVIEW OF ECONOMIC STUDIES
2023
View details for DOI 10.1093/restud/rdad006
View details for Web of Science ID 000950131400001
-
A COMMENT ON: "Invidious Comparisons: Ranking and Selection as Compound Decisions" by Jiaying Gu and Roger Koenker
ECONOMETRICA
2023; 91 (1): 53-60
View details for DOI 10.3982/ECTA20460
View details for Web of Science ID 000967592600004
-
Confidence Intervals for Seroprevalence
STATISTICAL SCIENCE
2022; 37 (3): 306-321
View details for DOI 10.1214/21-STS844
View details for Web of Science ID 000851465300002
-
Statistical Uncertainty in the Ranking of Journals and Universities
AMER ECONOMIC ASSOC. 2022: 630-634
View details for DOI 10.1257/pandp.20221064
View details for Web of Science ID 000832321200030
-
Permutation testing for dependence in time series
JOURNAL OF TIME SERIES ANALYSIS
2022
View details for DOI 10.1111/jtsa.12638
View details for Web of Science ID 000746816600001
-
CLT FOR U-STATISTICS WITH GROWING DIMENSION
STATISTICA SINICA
2022; 32 (1): 323-344
View details for DOI 10.5705/ss.202020.0048
View details for Web of Science ID 000739764600016
-
Uncertainty in the Hot Hand Fallacy: Detecting Streaky Alternatives to Random Bernoulli Sequences
REVIEW OF ECONOMIC STUDIES
2021
View details for DOI 10.1093/restud/rdab020
View details for Web of Science ID 000759037500001
-
Inference in Experiments With Matched Pairs
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
2021
View details for DOI 10.1080/01621459.2021.1883437
View details for Web of Science ID 000646152300001
-
The Romano-Wolf multiple-hypothesis correction in Stata
STATA JOURNAL
2020; 20 (4): 812–43
View details for DOI 10.1177/1536867X20976314
View details for Web of Science ID 000603072600005
-
Exact tests via multiple data splitting
STATISTICS & PROBABILITY LETTERS
2020; 166
View details for DOI 10.1016/j.spl.2020.108865
View details for Web of Science ID 000568993400018
- Permutation testing for dependence in time series Stanford Statistics Department. 2020 ; Stanford Statistics Technical Reports (2020-11):
- Uncertainty in the hot hand fallacy: Tests of randomness against steady alternatives to Bernoulli sequences Stanford Statistics Technical Report. 2020 (2020-02):
- Inference for ranks with applications to mobility across neighborhoods and academic achievement across countries Stanford Statistics Technical Report 2020-03. 2020
- CLT for U-statistics with growing dimension Statistica Sinica 2020
-
CONTROL OF DIRECTIONAL ERRORS IN FIXED SEQUENCE MULTIPLE TESTING
STATISTICA SINICA
2019; 29 (2): 1047–64
View details for DOI 10.5705/ss.202017.0072
View details for Web of Science ID 000462741700024
-
Improving weighted least squares inference
ECONOMETRICS AND STATISTICS
2019; 10: 96–119
View details for DOI 10.1016/j.ecosta.2018.06.005
View details for Web of Science ID 000461808600006
- Multiple data splitting for testing. Stanford. 2019 ; Stanford Statistics Department Techincal Report (2019-3):
- Inference in experiments with matched pairs Stanford. 2019 ; Stanford University Statistics Department (2019-4):
- A new approach for large scale multiple testing with application to FDR control of graphically structured hypotheses Stanford University. 2018 ; Stanford Statistics Department (2018-6):
-
Resurrecting weighted least squares
JOURNAL OF ECONOMETRICS
2017; 197 (1): 1-19
View details for DOI 10.1016/j.jeconom.2016.10.003
View details for Web of Science ID 000393932700001
- Improving weighted least squares inference Department of Statistics, Stanford University. 2017 ; Technical Report (2017-04):
-
Robust Permutation Tests For Correlation And Regression Coefficients
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
2017; 112 (519): 1211–20
View details for DOI 10.1080/01621459.2016.1202117
View details for Web of Science ID 000416611500029
- Analysis of error control in large scale two-stage multiple testing Department of Statistics, Stanford University. 2017 ; Technical Report (2017-03):
- Multiple testing of one-sided hypotheses: combining Bonferroni and the bootstrap University of Zurich. 2017 ; Working Paper ECON 254
-
Supplement to Approximation randomization tests under an approximate symmetry assumption
Econometrica
2017
View details for DOI 10.3982/ECTA12974
- Randomization tests under an approximate symmetry assumption Econometrica 2017; 85: 1013-1030
-
Supplement to "Robust permutation tests for correlation and regression coefficients"
Robust permutation tests for correlation and regression coefficients
2017
View details for DOI 10.1080/01621459.2016.1212117
-
Multivariate and multiple permutation tests
JOURNAL OF ECONOMETRICS
2016; 193 (1): 76-91
View details for DOI 10.1016/j.jeconom.2016.01.003
View details for Web of Science ID 000377310100005
-
Efficient computation of adjusted p-values for resampling-based stepdown multiple testing
STATISTICS & PROBABILITY LETTERS
2016; 113: 38-40
View details for DOI 10.1016/j.spl.2016.02.012
View details for Web of Science ID 000374808900006
-
Asymptotically valid and exact permutation tests based on two-sample U-statistics
JOURNAL OF STATISTICAL PLANNING AND INFERENCE
2016; 168: 97-105
View details for DOI 10.1016/j.jspi.2015.07.004
View details for Web of Science ID 000364244500007
- Resurrecting weighted least squares Journal of Econometrics 2016: to appear
-
Debunking the climate hiatus
CLIMATIC CHANGE
2015; 133 (2): 129-140
View details for DOI 10.1007/s10584-015-1495-y
View details for Web of Science ID 000363483800001
-
On stepwise control of directional errors under independence and some dependence
JOURNAL OF STATISTICAL PLANNING AND INFERENCE
2015; 163: 21-33
View details for DOI 10.1016/j.jspi.2015.02.009
View details for Web of Science ID 000355025200002
-
A PRACTICAL TWO-STEP METHOD FOR TESTING MOMENT INEQUALITIES
ECONOMETRICA
2014; 82 (5): 1979-2002
View details for DOI 10.3982/ECTA11011
View details for Web of Science ID 000342905900012
-
Testing for monotonicity in expected asset returns
JOURNAL OF EMPIRICAL FINANCE
2013; 23: 93-116
View details for DOI 10.1016/j.jempfin.2013.05.001
View details for Web of Science ID 000324083600007
-
EXACT AND ASYMPTOTICALLY ROBUST PERMUTATION TESTS
ANNALS OF STATISTICS
2013; 41 (2): 484-507
View details for DOI 10.1214/13-AOS1090
View details for Web of Science ID 000320488200004
-
Supplement to "Exact and asymptotically robust permutation tests"
Annals of Statistics
2013; 41
View details for DOI 10.1214/13-AOS1090SUPP
-
ON THE UNIFORM ASYMPTOTIC VALIDITY OF SUBSAMPLING AND THE BOOTSTRAP
ANNALS OF STATISTICS
2012; 40 (6): 2798-2822
View details for DOI 10.1214/12-AOS1051
View details for Web of Science ID 000321845400002
-
Subsampling Inference with K Populations and a Non-standard Behrens-Fisher Problem
INTERNATIONAL STATISTICAL REVIEW
2012; 80 (1): 149-175
View details for DOI 10.1111/j.1751-5823.2012.00177.x
View details for Web of Science ID 000302611600012
- On the third edition of Testing Statistical Hypotheses Selected Works of E.L. Lehmann edited by Rojo, J. New York: Springer-Verlag. 2012: 1089–1092
-
Supplement to "On the uniform asymptotic validity of subsampling and the bootstrap"
Annals of Statistics
2012; 40
View details for DOI 10.1214/12-AOS1041SUPP
-
Consonance and the Closure Method in Multiple Testing
INTERNATIONAL JOURNAL OF BIOSTATISTICS
2011; 7 (1)
View details for DOI 10.2202/1557-4679.1300
View details for Web of Science ID 000295789600012
-
K-sample subsampling in general spaces: The case of independent time series
JOURNAL OF MULTIVARIATE ANALYSIS
2010; 101 (2): 316-326
View details for DOI 10.1016/j.jmva.2008.11.005
View details for Web of Science ID 000272526300003
-
BALANCED CONTROL OF GENERALIZED ERROR RATES
ANNALS OF STATISTICS
2010; 38 (1): 598-633
View details for DOI 10.1214/09-AOS734
View details for Web of Science ID 000273800100018
-
Inference for the Identified Set in Partially Identified Econometric Models
ECONOMETRICA
2010; 78 (1): 169-211
View details for DOI 10.3982/ECTA6706
View details for Web of Science ID 000274388800006
- Multiple Testing New Palgrave Dictionary of Economics (Online Edition). 2010
-
Hypothesis Testing in Econometrics
ANNUAL REVIEW OF ECONOMICS, VOL 2
2010; 2: 75-104
View details for DOI 10.1146/annurev.economics.102308.124342
View details for Web of Science ID 000290636900004
-
Optimal testing of multiple hypotheses with common effect direction
BIOMETRIKA
2009; 96 (2): 399-410
View details for DOI 10.1093/biomet/asp006
View details for Web of Science ID 000266344300011
-
Discussion of 'Parametric versus nonparametrics: two alternative methodologies'
JOURNAL OF NONPARAMETRIC STATISTICS
2009; 21 (4): 419-424
View details for DOI 10.1080/10485250902846900
View details for Web of Science ID 000265336200005
-
Control of the false discovery rate under dependence using the bootstrap and subsampling
TEST
2008; 17 (3): 417-442
View details for DOI 10.1007/s11749-008-0126-6
View details for Web of Science ID 000261693400001
-
Inference for identifiable parameters in partially identified econometric models
JOURNAL OF STATISTICAL PLANNING AND INFERENCE
2008; 138 (9): 2786-2807
View details for DOI 10.1016/j.jspi.2008.03.015
View details for Web of Science ID 000256602600017
-
Formalized data snooping based on generalized error rates
ECONOMETRIC THEORY
2008; 24 (2): 404-447
View details for DOI 10.1017/S0266466608080171
View details for Web of Science ID 000253574300005
-
K-sample subsampling
1st International Workshop on Functional and Operatorial Statistics
PHYSICA-VERLAG GMBH & CO. 2008: 247–253
View details for Web of Science ID 000258056000038
- Discussion: On methods controlling the false discover rate Sankya 2008; 70: 169-176
-
Control of generalized error rates in multiple testing
ANNALS OF STATISTICS
2007; 35 (4): 1378-1408
View details for DOI 10.1214/009053606000001622
View details for Web of Science ID 000249568000002
-
A generalized Sidak-Holm procedure and control of generalized error rates under independence
STATISTICAL APPLICATIONS IN GENETICS AND MOLECULAR BIOLOGY
2007; 6
Abstract
For testing multiple null hypotheses, the classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. In many applications, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which is called the k-FWER. In Hommel and Hoffmann (1987) and Lehmann and Romano (2005a), single step and stepdown procedures are derived that control the k-FWER, without making any assumptions concerning the dependence structure of the p-values of the individual tests. However, if the p-values are mutually independent, one can improve the procedures. In fact, Sarkar (2005) provided such an improvement. However, we show other improvements are possible which appear to be generally much better, and are sometimes unimprovable. When k=1, the procedure reduces to the classical method of Sidak, and the stepdown procedure is unimprovable and strictly dominates that of Sarkar. Under a monotonicity condition, an unimprovable procedure is obtained. In the case k=2, the monotonicity condition is satisfied, and the condition can be checked numerically in general. We then develop a stepdown method that controls the false discovery proportion. Except for the case of k-FWER control with k=1, the gains are surprisingly dramatic, and theoretical and numerical evidence is given.
View details for Web of Science ID 000245335600004
View details for PubMedID 17402918
-
Stepup procedures for control of generalizations of the familywise error rate
ANNALS OF STATISTICS
2006; 34 (4): 1850-1873
View details for DOI 10.1214/009053606000000461
View details for Web of Science ID 000242314100015
-
Improved nonparametric confidence intervals in time series regressions
JOURNAL OF NONPARAMETRIC STATISTICS
2006; 18 (2): 199-214
View details for DOI 10.1080/10485250600687812
View details for Web of Science ID 000238876000006
- On stepdown control of the false discovery proportion 2nd Lehmann Symposium-Optimality edited by Rojo, J. IMS. 2006: 33–50
- A generalized Sidák procedure and control of generalized error rates under independence Statistical Applications in Genetics and Molecular Biology 2006; 6 (1)
-
Stepwise multiple testing as formalized data snooping
ECONOMETRICA
2005; 73 (4): 1237-1282
View details for Web of Science ID 000230240500007
-
Optimal testing of equivalence hypotheses
ANNALS OF STATISTICS
2005; 33 (3): 1036-1047
View details for DOI 10.1214/009053605000000048
View details for Web of Science ID 000230520500009
-
On optimality of stepdown and stepup multiple test procedures
ANNALS OF STATISTICS
2005; 33 (3): 1084-1108
View details for DOI 10.1214/009053605000000066
View details for Web of Science ID 000230520500011
-
Generalizations of the familywise error rate
ANNALS OF STATISTICS
2005; 33 (3): 1138-1154
View details for DOI 10.1214/009053605000000084
View details for Web of Science ID 000230520500013
-
Exact and approximate stepdown methods for multiple hypothesis testing
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
2005; 100 (469): 94-108
View details for DOI 10.1198/016214504000000539
View details for Web of Science ID 000233311100015
- Testing Statistical Hypotheses New York: Springer-Verlag. 2005
-
On non-parametric testing, the uniform behaviour of the t-test, and related problems
SCANDINAVIAN JOURNAL OF STATISTICS
2004; 31 (4): 567-584
View details for Web of Science ID 000225265200005
-
Inference for autocorrelations in the possible presence of a unit root
JOURNAL OF TIME SERIES ANALYSIS
2004; 25 (2): 251-263
View details for Web of Science ID 000220051100006
-
Explicit nonparametric confidence intervals for the variance with guaranteed coverage
COMMUNICATIONS IN STATISTICS-THEORY AND METHODS
2002; 31 (8): 1231-1250
View details for Web of Science ID 000177293100001
- Automatic adaptive estimation via the boostrap Technical Report 2000-01, Department of Statistics, Stanford University 2002
-
On the asymptotic theory of subsampling
STATISTICA SINICA
2001; 11 (4): 1105-1124
View details for Web of Science ID 000172094600018
-
Subsampling intervals in autoregressive models with linear time trend
ECONOMETRICA
2001; 69 (5): 1283-1314
View details for Web of Science ID 000170659200006
-
Finite sample nonparametric inference and large sample efficiency
ANNALS OF STATISTICS
2000; 28 (3): 756-778
View details for Web of Science ID 000165456000005
-
A more general central limit theorem for m-dependent random variables with unbounded m
STATISTICS & PROBABILITY LETTERS
2000; 47 (2): 115-124
View details for Web of Science ID 000085552500002
-
Subsampling, symmetrization, and robust interpolation
COMMUNICATIONS IN STATISTICS-THEORY AND METHODS
2000; 29 (8): 1741-1757
View details for Web of Science ID 000088283000004
-
Weak convergence of dependent empirical measures with application to subsampling in function spaces
JOURNAL OF STATISTICAL PLANNING AND INFERENCE
1999; 79 (2): 179-190
View details for Web of Science ID 000081440200001
-
On subsampling estimators with unknown rate of convergence
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
1999; 94 (446): 569-579
View details for Web of Science ID 000081058500026
-
An invariance principle for triangular arrays of dependent variables with application to autocovariance estimation
CANADIAN JOURNAL OF STATISTICS-REVUE CANADIENNE DE STATISTIQUE
1999; 27 (2): 329-343
View details for Web of Science ID 000081537800009
-
Multivariate density estimation with general flat-top kernels of infinite order
JOURNAL OF MULTIVARIATE ANALYSIS
1999; 68 (1): 1-25
View details for Web of Science ID 000078185500001
- Resampling marked point processes Multivariate Analysis, Design of Experiments, and Survey Sampling edited by Ghosh, S. New York: Marcel Dekker. 1999: 163–185
- Bootstrap goodness of fit tests in the frequency domain The Journal of Time Series 1999; 20: 619-654
- Subsampling New York: Springer-Verlag. 1999
-
Subsampling inference for the mean in the heavy-tailed case
METRIKA
1999; 50 (1): 55-69
View details for Web of Science ID 000083990100004
- Large sample inference for irregularly spaced dependent observations based on subsamples Sankhya Series A 1998; 60: 274-292
- Subsampling confidence intervals for the autoregressive root Technical Report 5, Department of Statistics, Stanford University 1998
-
Subsampling for heteroskedastic time series
JOURNAL OF ECONOMETRICS
1997; 81 (2): 281-317
View details for Web of Science ID A1997YD79800001
-
Inference for autocorrelations under weak assumptions
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
1996; 91 (434): 590-600
View details for Web of Science ID A1996UP55200028
-
On flat-top kernel spectral density estimators for homogeneous random fields
JOURNAL OF STATISTICAL PLANNING AND INFERENCE
1996; 51 (1): 41-53
View details for Web of Science ID A1996UC36000003
- Subsampling for econometric models Econometric Reviews 1996; 15 (2): 169-176
-
ON BOOTSTRAP PROCEDURES FOR 2ND-ORDER ACCURATE CONFIDENCE-LIMITS IN PARAMETRIC MODELS
STATISTICA SINICA
1995; 5 (1): 141-160
View details for Web of Science ID A1995QE67800009
- Bias-corrected nonparametric spectral estimation Journal of Time Series Analysis 1995; 16: 67-103
-
THE STATIONARY BOOTSTRAP
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
1994; 89 (428): 1303-1313
View details for Web of Science ID A1994PU33000016
-
LARGE-SAMPLE CONFIDENCE-REGIONS BASED ON SUBSAMPLES UNDER MINIMAL ASSUMPTIONS
ANNALS OF STATISTICS
1994; 22 (4): 2031-2050
View details for Web of Science ID A1994QX01100025
-
LIMIT-THEOREMS FOR WEAKLY DEPENDENT HILBERT-SPACE VALUED RANDOM-VARIABLES WITH APPLICATION TO THE STATIONARY BOOTSTRAP
STATISTICA SINICA
1994; 4 (2): 461-476
View details for Web of Science ID A1994PK25300005
-
Inference for autocorrelations by resampling
Annual Meeting of the American-Statistical-Association, Statistical-Computing-Section
AMER STATISTICAL ASSOC. 1994: 1–10
View details for Web of Science ID A1994BD83T00001
-
NONPARAMETRIC RESAMPLING FOR HOMOGENEOUS STRONG MIXING RANDOM-FIELDS
JOURNAL OF MULTIVARIATE ANALYSIS
1993; 47 (2): 301-328
View details for Web of Science ID A1993MJ98600008
-
ON THE SAMPLE VARIANCE OF LINEAR STATISTICS DERIVED FROM MIXING SEQUENCES
STOCHASTIC PROCESSES AND THEIR APPLICATIONS
1993; 45 (1): 155-167
View details for Web of Science ID A1993KQ45400011
-
ON A FAMILY OF SMOOTHING KERNELS OF INFINITE-ORDER
25th Symposium on the Interface of Computing Science and Statistics - Statistical Applications of Expanding Computer Capabilities
INTERFACE FOUNDATION NORTH AMERICA. 1993: 141–145
View details for Web of Science ID A1993BB70B00021
- Estimating the distribution of a studentized statistic by subsampling Bulletin of the International Statistical Institute 1993; 49: 315-316
-
UNIFORM CONFIDENCE BANDS FOR THE SPECTRUM BASED ON SUBSAMPLES
25th Symposium on the Interface of Computing Science and Statistics - Statistical Applications of Expanding Computer Capabilities
INTERFACE FOUNDATION NORTH AMERICA. 1993: 346–351
View details for Web of Science ID A1993BB70B00054
-
A GENERAL RESAMPLING SCHEME FOR TRIANGULAR ARRAYS OF ALPHA-MIXING RANDOM-VARIABLES WITH APPLICATION TO THE PROBLEM OF SPECTRAL DENSITY-ESTIMATION
ANNALS OF STATISTICS
1992; 20 (4): 1985-2007
View details for Web of Science ID A1992KH71800015
-
BOOTSTRAP TECHNOLOGY AND APPLICATIONS
TECHNOMETRICS
1992; 34 (4): 378-398
View details for Web of Science ID A1992JU48500001
-
BOOTSTRAP CONFIDENCE BANDS FOR SPECTRA AND CROSS-SPECTRA
IEEE TRANSACTIONS ON SIGNAL PROCESSING
1992; 40 (5): 1206-1215
View details for Web of Science ID A1992HR45700017
- A nonparametric resampling procedure for multivariate confidence regions in time series analysis Proceedings of the 22nd Symposium on the Interface edited by Page, C., LePage, R. 1992: 98–103
- A circular block-resampling procedure for stationary data Exploring the Limits of Bootstrap edited by LePage, R., Billard, L. John Wiley. 1992: 263–270
-
EMPIRICAL LIKELIHOOD IS BARTLETT-CORRECTABLE
ANNALS OF STATISTICS
1991; 19 (2): 1053-1061
View details for Web of Science ID A1991FU11400029
-
BOOTSTRAP ADAPTIVE ESTIMATION - THE TRIMMED-MEAN EXAMPLE
CANADIAN JOURNAL OF STATISTICS-REVUE CANADIENNE DE STATISTIQUE
1990; 18 (4): 297-314
View details for Web of Science ID A1990EX01800002
-
BOOTSTRAP CHOICE OF TUNING PARAMETERS
ANNALS OF THE INSTITUTE OF STATISTICAL MATHEMATICS
1990; 42 (4): 709-735
View details for Web of Science ID A1990ER55900007
-
ON THE BEHAVIOR OF RANDOMIZATION TESTS WITHOUT A GROUP INVARIANCE ASSUMPTION
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
1990; 85 (411): 686-692
View details for Web of Science ID A1990DZ46800012
-
NONPARAMETRIC CONFIDENCE-LIMITS BY RESAMPLING METHODS AND LEAST FAVORABLE FAMILIES
INTERNATIONAL STATISTICAL REVIEW
1990; 58 (1): 59-76
View details for Web of Science ID A1990DE65600006
-
ON ADJUSTMENTS BASED ON THE SIGNED ROOT OF THE EMPIRICAL LIKELIHOOD RATIO STATISTIC
BIOMETRIKA
1989; 76 (3): 447-456
View details for Web of Science ID A1989AR09100004
-
COMPARISON OF PARAMETRIC AND EMPIRICAL LIKELIHOOD FUNCTIONS
BIOMETRIKA
1989; 76 (3): 465-476
View details for Web of Science ID A1989AR09100006
-
ON SMOOTHING AND THE BOOTSTRAP
ANNALS OF STATISTICS
1989; 17 (2): 692-704
View details for Web of Science ID A1989AB89300021
-
THE AUTOMATIC PERCENTILE METHOD - ACCURATE CONFIDENCE-LIMITS IN PARAMETRIC MODELS
CANADIAN JOURNAL OF STATISTICS-REVUE CANADIENNE DE STATISTIQUE
1989; 17 (2): 155-169
View details for Web of Science ID A1989AT50800003
-
BOOTSTRAP AND RANDOMIZATION TESTS OF SOME NONPARAMETRIC HYPOTHESES
ANNALS OF STATISTICS
1989; 17 (1): 141-159
View details for Web of Science ID A1989T614100006
-
DO BOOTSTRAP CONFIDENCE PROCEDURES BEHAVE WELL UNIFORMLY IN P
CANADIAN JOURNAL OF STATISTICS-REVUE CANADIENNE DE STATISTIQUE
1989; 17 (1): 75-80
View details for Web of Science ID A1989AL06200008
-
A BOOTSTRAP REVIVAL OF SOME NONPARAMETRIC DISTANCE TESTS
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
1988; 83 (403): 698-708
View details for Web of Science ID A1988Q348300014
-
THEORETICAL COMPARISON OF BOOTSTRAP CONFIDENCE-INTERVALS - DISCUSSION
ANNALS OF STATISTICS
1988; 16 (3): 965-969
View details for Web of Science ID A1988Q161700006
-
ON WEAK-CONVERGENCE AND OPTIMALITY OF KERNEL DENSITY ESTIMATES OF THE MODE
ANNALS OF STATISTICS
1988; 16 (2): 629-647
View details for Web of Science ID A1988P078900008
-
BOOTSTRAP METHODS - A REVIEW OF BOOTSTRAP CONFIDENCE-INTERVALS - DISCUSSION
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL
1988; 50 (3): 355-370
View details for Web of Science ID A1988Q939300003
-
A REVIEW OF BOOTSTRAP CONFIDENCE-INTERVALS
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-METHODOLOGICAL
1988; 50 (3): 338-354
View details for Web of Science ID A1988Q939300002
-
BOOTSTRAPPING THE MODE
ANNALS OF THE INSTITUTE OF STATISTICAL MATHEMATICS
1988; 40 (3): 565-586
View details for Web of Science ID A1988Q407100011
- Counterexamples in Probability and Statistics Monterey, CA: Wadsworth Publishing Company. 1986
-
BOOTSTRAP CONFIDENCE CONES FOR DIRECTIONAL-DATA
BIOMETRIKA
1985; 72 (3): 637-645
View details for Web of Science ID A1985AUT4400016
- Evaluating inclusion functionals for random convex hulls Z. Wahrscheinlichkeitsth 1985; 68: 415-424
-
COVERAGE PROBLEMS AND RANDOM CONVEX HULLS
JOURNAL OF APPLIED PROBABILITY
1982; 19 (3): 546-561
View details for Web of Science ID A1982PK17800007