Bio


John Duchi (http://web.stanford.edu/~jduchi/) is an assistant professor of Statistics and Electrical Engineering and (by courtesy) Computer Science at Stanford University. His work spans statistical learning, optimization, information theory, and computation, with a few driving goals. (1) To discover statistical learning procedures that optimally trade between resources--computation, communication, privacy provided to study participants--while maintaining good statistical performance. (2) To build efficient large-scale optimization methods that address the spectrum of optimization, machine learning, and data analysis problems we face, allowing us to move beyond bespoke solutions to methods that robustly work. (3) To develop tools to assess and guarantee the validity of--and confidence we should have in--machine-learned systems.

Academic Appointments


Honors & Awards


  • SIAM/OPT Early Career Prize, Society for Industrial and Applied Mathematics, Optimization Group (2020)
  • Young Investigator Award, Office of Naval Research (2019)
  • Best Paper Award, Neural Information Processing Systems (2017)
  • Sloan Foundation Fellow in Mathematics, Sloan Foundation (2016)
  • Doctoral Dissertation Award (Honorable Mention), Association for Computing Machinery (2015)
  • Okawa Foundation Award, Okawa Foundation (2015)
  • C.V. Ramamoorthy Distinguished Research Award, University of California, Berkeley (2014)
  • Best Student Paper Award, International Conference on Machine Learning (2010)

Professional Education


  • BS, Stanford University, Computer Science (2007)
  • MS, Stanford University, Computer Science (2008)
  • MA, University of California, Berkeley, Statistics (2012)
  • PhD, University of California, Berkeley, EECS (2014)

Current Research and Scholarly Interests


My work spans statistical learning, optimization, information theory, and computation, with a few driving goals: 1. To discover statistical learning procedures that optimally trade between real-world resources while maintaining statistical efficiency. 2. To build efficient large-scale optimization methods that move beyond bespoke solutions to methods that robustly work. 3. To develop tools to assess and guarantee the validity of---and confidence we should have in---machine-learned systems.

2024-25 Courses


Stanford Advisees


All Publications


  • THE RIGHT COMPLEXITY MEASURE IN LOCALLY PRIVATE ESTIMATION: IT IS NOT THE FISHER INFORMATION ANNALS OF STATISTICS Duchi, J. C., Ruan, F. 2024; 52 (1): 1-51

    View details for DOI 10.1214/22-AOS2227

    View details for Web of Science ID 001181807000014

  • Robust Validation: Confident Predictions Even When Distributions Shift JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION Cauchois, M., Gupta, S., Ali, A., Duchi, J. C. 2024
  • Predictive Inference with Weak Supervision JOURNAL OF MACHINE LEARNING RESEARCH Cauchois, M., Gupta, S., Ali, A., Duchi, J. 2024; 25
  • BOUNDS ON THE CONDITIONAL AND AVERAGE TREATMENT EFFECT WITH UNOBSERVED CONFOUNDING FACTORS ANNALS OF STATISTICS Yadlowsky, S., Namkoong, H., Basu, S., Duchi, J., Tian, L. 2022; 50 (5): 2587-2615

    View details for DOI 10.1214/22-AOS2195

    View details for Web of Science ID 000964342400006

  • BOUNDS ON THE CONDITIONAL AND AVERAGE TREATMENT EFFECT WITH UNOBSERVED CONFOUNDING FACTORS. Annals of statistics Yadlowsky, S., Namkoong, H., Basu, S., Duchi, J., Tian, L. 2022; 50 (5): 2587-2615

    Abstract

    For observational studies, we study the sensitivity of causal inference when treatment assignments may depend on unobserved confounders. We develop a loss minimization approach for estimating bounds on the conditional average treatment effect (CATE) when unobserved confounders have a bounded effect on the odds ratio of treatment selection. Our approach is scalable and allows flexible use of model classes in estimation, including nonparametric and black-box machine learning methods. Based on these bounds for the CATE, we propose a sensitivity analysis for the average treatment effect (ATE). Our semiparametric estimator extends/bounds the augmented inverse propensity weighted (AIPW) estimator for the ATE under bounded unobserved confounding. By constructing a Neyman orthogonal score, our estimator of the bound for the ATE is a regular root-n estimator so long as the nuisance parameters are estimated at the opn-1/4 rate. We complement our methodology with optimality results showing that our proposed bounds are tight in certain cases. We demonstrate our method on simulated and real data examples, and show accurate coverage of our confidence intervals in practical finite sample regimes with rich covariate information.

    View details for DOI 10.1214/22-aos2195

    View details for PubMedID 38050638

    View details for PubMedCentralID PMC10694186

  • Distributionally Robust Losses for Latent Covariate Mixtures OPERATIONS RESEARCH Duchi, J., Hashimoto, T., Namkoong, H. 2022
  • Mean Estimation From One-Bit Measurements IEEE TRANSACTIONS ON INFORMATION THEORY Kipnis, A., Duchi, J. C. 2022; 68 (9): 6276-6296
  • Lower bounds for non-convex stochastic optimization MATHEMATICAL PROGRAMMING Arjevani, Y., Carmon, Y., Duchi, J. C., Foster, D. J., Srebro, N., Woodworth, B. 2022
  • Private optimization in the interpolation regime: faster rates and hardness results Asi, H., Chadha, K., Cheng, G., Duchi, J., Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., Sabato, S. JMLR-JOURNAL MACHINE LEARNING RESEARCH. 2022
  • Accelerated, Optimal and Parallel: Some results on model-based stochastic optimization Chadha, K., Cheng, G., Duchi, J., Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., Sabato, S. JMLR-JOURNAL MACHINE LEARNING RESEARCH. 2022
  • Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach MATHEMATICS OF OPERATIONS RESEARCH Duchi, J. C., Glynn, P. W., Namkoong, H. 2021; 46 (3): 946-969
  • LEARNING MODELS WITH UNIFORM PERFORMANCE VIA DISTRIBUTIONALLY ROBUST OPTIMIZATION ANNALS OF STATISTICS Duchi, J. C., Namkoong, H. 2021; 49 (3): 1378-1406

    View details for DOI 10.1214/20-AOS2004

    View details for Web of Science ID 000684378300006

  • ASYMPTOTIC OPTIMALITY IN STOCHASTIC OPTIMIZATION ANNALS OF STATISTICS Duchi, J. C., Ruan, F. 2021; 49 (1): 21–48

    View details for DOI 10.1214/19-AOS1831

    View details for Web of Science ID 000614187400002

  • Misspecification in Prediction Problems and Robustness via Improper Learning Marsden, A., Duchi, J., Valiant, G., Banerjee, A., Fukumizu, K. MICROTOME PUBLISHING. 2021
  • Knowing what You Know: valid and validated confidence sets in multiclass and multilabel prediction JOURNAL OF MACHINE LEARNING RESEARCH Cauchois, M., Gupta, S., Duchi, J. C. 2021; 22
  • Private Adaptive Gradient Methods for Convex Optimization Asi, H., Duchi, J., Fallah, A., Javidbakht, O., Talwar, K., Meila, M., Zhang, T. JMLR-JOURNAL MACHINE LEARNING RESEARCH. 2021
  • A constrained risk inequality for general losses Duchi, J., Ruan, F., Banerjee, A., Fukumizu, K. MICROTOME PUBLISHING. 2021
  • Lower bounds for finding stationary points II: first-order methods MATHEMATICAL PROGRAMMING Carmon, Y., Duchi, J. C., Hinder, O., Sidford, A. 2021; 185 (1-2): 315–55
  • Lower bounds for finding stationary points I MATHEMATICAL PROGRAMMING Carmon, Y., Duchi, J. C., Hinder, O., Sidford, A. 2020; 184 (1-2): 71–120
  • First-Order Methods for Nonconvex Quadratic Minimization SIAM REVIEW Carmon, Y., Duchi, J. C. 2020; 62 (2): 395–436

    View details for DOI 10.1137/20M1321759

    View details for Web of Science ID 000551389300003

  • Solving (most) of a set of quadratic equalities: composite optimization for robust phase retrieval INFORMATION AND INFERENCE-A JOURNAL OF THE IMA Duchi, J. C., Ruan, F. 2019; 8 (3): 471–529
  • GRADIENT DESCENT FINDS THE CUBIC-REGULARIZED NONCONVEX NEWTON STEP SIAM JOURNAL ON OPTIMIZATION Carmon, Y., Duchi, J. 2019; 29 (3): 2146–78

    View details for DOI 10.1137/17M1113898

    View details for Web of Science ID 000487929500017

  • The importance of better models in stochastic optimization. Proceedings of the National Academy of Sciences of the United States of America Asi, H. n., Duchi, J. C. 2019

    Abstract

    Standard stochastic optimization methods are brittle, sensitive to stepsize choice and other algorithmic parameters, and they exhibit instability outside of well-behaved families of objectives. To address these challenges, we investigate models for stochastic optimization and learning problems that exhibit better robustness to problem families and algorithmic parameters. With appropriately accurate models-which we call the aprox family-stochastic methods can be made stable, provably convergent, and asymptotically optimal; even modeling that the objective is nonnegative is sufficient for this stability. We extend these results beyond convexity to weakly convex objectives, which include compositions of convex losses with smooth functions common in modern machine learning. We highlight the importance of robustness and accurate modeling with experimental evaluation of convergence time and algorithm sensitivity.

    View details for DOI 10.1073/pnas.1908018116

    View details for PubMedID 31666325

  • Modeling simple structures and geometry for better stochastic optimization algorithms Asi, H., Duchi, J. C., Chaudhuri, K., Sugiyama, M. MICROTOME PUBLISHING. 2019
  • Necessary and Sufficient Geometries for Gradient Methods Levy, D., Duchi, J. C., Wallach, H., Larochelle, H., Beygelzimer, A., d'Alche-Buc, F., Fox, E., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2019
  • Unlabeled Data Improves Adversarial Robustness Carmon, Y., Raghunathan, A., Schmidt, L., Liang, P., Duchi, J. C., Wallach, H., Larochelle, H., Beygelzimer, A., d'Alche-Buc, F., Fox, E., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2019
  • Variance-based Regularization with Convex Objectives JOURNAL OF MACHINE LEARNING RESEARCH Duchi, J., Namkoong, H. 2019; 20
  • STOCHASTIC (APPROXIMATE) PROXIMAL POINT METHODS: CONVERGENCE, OPTIMALITY, AND ADAPTIVITY SIAM JOURNAL ON OPTIMIZATION Asi, H., Duchi, J. C. 2019; 29 (3): 2257–90

    View details for DOI 10.1137/18M1230323

    View details for Web of Science ID 000487929500021

  • ACCELERATED METHODS FOR NONCONVEX OPTIMIZATION SIAM JOURNAL ON OPTIMIZATION Carmon, Y., Duchi, J. C., Hinder, O., Sidford, A. 2018; 28 (2): 1751–72

    View details for DOI 10.1137/17M1114296

    View details for Web of Science ID 000436991600031

  • STOCHASTIC METHODS FOR COMPOSITE AND WEAKLY CONVEX OPTIMIZATION PROBLEMS SIAM JOURNAL ON OPTIMIZATION Duchi, J. C., Ruan, F. 2018; 28 (4): 3229–59

    View details for DOI 10.1137/17M1135086

    View details for Web of Science ID 000453750000019

  • Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation O'Kelly, M., Sinha, A., Namkoong, H., Duchi, J., Tedrake, R., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018
  • Generalizing to Unseen Domains via Adversarial Data Augmentation Volpi, R., Namkoong, H., Sener, O., Duchi, J., Murino, V., Savarese, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018
  • Analysis of Krylov Subspace Solutions of Regularized Nonconvex Quadratic Problems Carmon, Y., Duchi, J. C., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018
  • Minimax Optimal Procedures for Locally Private Estimation JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2018; 113 (521): 182–201
  • Minimax Optimal Procedures for Locally Private Estimation Rejoinder JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2018; 113 (521): 212–15
  • Unsupervised Transformation Learning via Convex Relaxations Hashimoto, T. B., Duchi, J. C., Liang, P., Guyon, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017
  • Variance-based Regularization with Convex Objectives Namkoong, H., Duchi, J. C., Guyon, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017
  • Mean Estimation from Adaptive One-bit Measurements Kipnis, A., Duchi, J. C., IEEE IEEE. 2017: 1000–1007
  • Simultaneous dimension reduction and adjustment for confounding variation PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA Lin, Z., Yang, C., Zhu, Y., Duchi, J., Fu, Y., Wang, Y., Jiang, B., Zamanighomi, M., Xu, X., Li, M., Sestan, N., Zhao, H., Wong, W. H. 2016; 113 (51): 14662-14667

    Abstract

    Dimension reduction methods are commonly applied to high-throughput biological datasets. However, the results can be hindered by confounding factors, either biological or technical in origin. In this study, we extend principal component analysis (PCA) to propose AC-PCA for simultaneous dimension reduction and adjustment for confounding (AC) variation. We show that AC-PCA can adjust for (i) variations across individual donors present in a human brain exon array dataset and (ii) variations of different species in a model organism ENCODE RNA sequencing dataset. Our approach is able to recover the anatomical structure of neocortical regions and to capture the shared variation among species during embryonic development. For gene selection purposes, we extend AC-PCA with sparsity constraints and propose and implement an efficient algorithm. The methods developed in this paper can also be applied to more general settings. The R package and MATLAB source code are available at https://github.com/linzx06/AC-PCA.

    View details for DOI 10.1073/pnas.1617317113

    View details for PubMedID 27930330

  • Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates JOURNAL OF MACHINE LEARNING RESEARCH Zhang, Y., Duchi, J., Wainwright, M. 2015; 16: 3299-3340
  • Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function Evaluations IEEE TRANSACTIONS ON INFORMATION THEORY Duchi, J. C., Jordan, M. I., Wainwright, M. J., Wibisono, A. 2015; 61 (5): 2788-2806
  • Dynamic Management of Network Risk from Epidemic Phenomena Sinha, A., Duchi, J., Bambos, N., IEEE IEEE. 2015: 1583–88
  • Privacy Aware Learning JOURNAL OF THE ACM Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2014; 61 (6)

    View details for DOI 10.1145/2666468

    View details for Web of Science ID 000347051200006

  • Privacy: a few definitional aspects and consequences for minimax mean-squared error Barber, R., Duchi, J., IEEE IEEE. 2014: 1365–69
  • THE ASYMPTOTICS OF RANKING ALGORITHMS ANNALS OF STATISTICS Duchi, J. C., Mackey, L., Jordan, M. I. 2013; 41 (5): 2292-2323

    View details for DOI 10.1214/13-AOS1142

    View details for Web of Science ID 000327746100002