John Duchi

Associate Professor of Statistics, of Electrical Engineering and, by courtesy, of Computer Science

Bio

John Duchi (http://web.stanford.edu/~jduchi/) is an assistant professor of Statistics and Electrical Engineering and (by courtesy) Computer Science at Stanford University. His work spans statistical learning, optimization, information theory, and computation, with a few driving goals. (1) To discover statistical learning procedures that optimally trade between resources--computation, communication, privacy provided to study participants--while maintaining good statistical performance. (2) To build efficient large-scale optimization methods that address the spectrum of optimization, machine learning, and data analysis problems we face, allowing us to move beyond bespoke solutions to methods that robustly work. (3) To develop tools to assess and guarantee the validity of--and confidence we should have in--machine-learned systems.

Academic Appointments

Associate Professor, Statistics
Associate Professor, Electrical Engineering
Associate Professor (By courtesy), Computer Science
Member, Bio-X

Honors & Awards

SIAM/OPT Early Career Prize, Society for Industrial and Applied Mathematics, Optimization Group (2020)
Young Investigator Award, Office of Naval Research (2019)
Best Paper Award, Neural Information Processing Systems (2017)
Sloan Foundation Fellow in Mathematics, Sloan Foundation (2016)
Doctoral Dissertation Award (Honorable Mention), Association for Computing Machinery (2015)
Okawa Foundation Award, Okawa Foundation (2015)
C.V. Ramamoorthy Distinguished Research Award, University of California, Berkeley (2014)
Best Student Paper Award, International Conference on Machine Learning (2010)

Professional Education

BS, Stanford University, Computer Science (2007)
MS, Stanford University, Computer Science (2008)
MA, University of California, Berkeley, Statistics (2012)
PhD, University of California, Berkeley, EECS (2014)

Contact

Academic
jduchi@stanford.edu
University - Faculty Department: Statistics Position: Assoc Professor
- 390 Jane Stanford Way
- 126 Sequoia Hall
- Stanford, California 94305-4020
- (650) 498-5224 (office)
University - Faculty Department: Electrical Engineering Position: Assoc Professor
- (650) 498-5224 (office)

Additional Info

Mail Code: 4065

Current Research and Scholarly Interests

My work spans statistical learning, optimization, information theory, and computation, with a few driving goals: 1. To discover statistical learning procedures that optimally trade between real-world resources while maintaining statistical efficiency. 2. To build efficient large-scale optimization methods that move beyond bespoke solutions to methods that robustly work. 3. To develop tools to assess and guarantee the validity of---and confidence we should have in---machine-learned systems.

2023-24 Courses

Information Theory and Statistics
EE 377, STATS 311 (Aut)
Introduction to Matrix Methods
ENGR 108 (Aut)
Mathematics of Convexity
EE 364M (Win)
Modern Applied Statistics: Learning
STATS 315A (Win)
Independent Studies (18)
- Advanced Reading and Research
  CS 499 (Aut, Win, Spr)
- Advanced Reading and Research
  CS 499P (Aut, Win, Spr)
- Curricular Practical Training
  CS 390A (Aut, Win, Sum)
- Curricular Practical Training
  CS 390B (Win)
- Independent Study
  DATASCI 199 (Win, Spr, Sum)
- Independent Study
  STATS 199 (Aut, Win, Spr, Sum)
- Independent Study
  STATS 299 (Aut, Win, Spr, Sum)
- Industrial Research for Statisticians
  STATS 398 (Sum)
- Master's Thesis and Thesis Research
  EE 300 (Win)
- Part-time Curricular Practical Training
  CS 390D (Aut, Win)
- Practical Training
  MCS 198 (Sum)
- Research
  STATS 399 (Aut, Win, Spr, Sum)
- Special Studies and Reports in Electrical Engineering
  EE 191 (Aut, Win, Spr, Sum)
- Special Studies and Reports in Electrical Engineering
  EE 191A (Aut)
- Special Studies and Reports in Electrical Engineering
  EE 391 (Aut, Win, Spr, Sum)
- Special Studies and Reports in Electrical Engineering (WIM)
  EE 191W (Aut, Win, Spr, Sum)
- Special Studies or Projects in Electrical Engineering
  EE 190 (Aut, Win, Spr, Sum)
- Special Studies or Projects in Electrical Engineering
  EE 390 (Aut, Win, Spr, Sum)
Prior Year Courses
2022-23 Courses
- Applied Statistics I
  STATS 305A (Aut)
- Introduction to Matrix Methods
  ENGR 108 (Spr)
2021-22 Courses
- Applied Statistics I
  STATS 305A (Aut)
- Information Theory and Statistics
  EE 377, STATS 311 (Aut)
- Theory of Probability
  STATS 116 (Spr)
2020-21 Courses
- Convex Optimization I
  CME 364A, EE 364A (Win)
- Theory of Statistics II
  STATS 300B (Win)

Stanford Advisees

Doctoral Dissertation Reader (AC)
John Cherian, Isaac Gibbs, Anmol Kagrecha, Basil Saeed
Doctoral Dissertation Advisor (AC)
Karan Chadha, Gary Cheng, Nicole Meister
Doctoral Dissertation Co-Advisor (AC)
Chen Cheng
Master's Program Advisor
Haoyi Duan, Jake Hofgard, Sneha Jayaganthan, Yichen Jiang, Zixin Li, Yichun Qian, John Siddiqui
Doctoral (Program)
Felipe Areces, Andy Dong, Saminul Haque, Rohith Kuditipudi, Pratik Rathore, Alan Yang, Fangzhao Zhang

All Publications

Lower bounds for finding stationary points I MATHEMATICAL PROGRAMMING Carmon, Y., Duchi, J. C., Hinder, O., Sidford, A. 2020; 184 (1-2): 71–120

View details for DOI 10.1007/s10107-019-01406-y

View details for Web of Science ID 000578114700003
First-Order Methods for Nonconvex Quadratic Minimization SIAM REVIEW Carmon, Y., Duchi, J. C. 2020; 62 (2): 395–436

View details for DOI 10.1137/20M1321759

View details for Web of Science ID 000551389300003
Solving (most) of a set of quadratic equalities: composite optimization for robust phase retrieval INFORMATION AND INFERENCE-A JOURNAL OF THE IMA Duchi, J. C., Ruan, F. 2019; 8 (3): 471–529

View details for DOI 10.1093/imaiai/iay015

View details for Web of Science ID 000493304200003
Unlabeled Data Improves Adversarial Robustness Carmon, Y., Raghunathan, A., Schmidt, L., Liang, P., Duchi, J. C., Wallach, H., Larochelle, H., Beygelzimer, A., d'Alche-Buc, F., Fox, E., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2019

View details for Web of Science ID 000535866902078
The importance of better models in stochastic optimization. Proceedings of the National Academy of Sciences of the United States of America Asi, H. n., Duchi, J. C. 2019

Abstract

Standard stochastic optimization methods are brittle, sensitive to stepsize choice and other algorithmic parameters, and they exhibit instability outside of well-behaved families of objectives. To address these challenges, we investigate models for stochastic optimization and learning problems that exhibit better robustness to problem families and algorithmic parameters. With appropriately accurate models-which we call the aprox family-stochastic methods can be made stable, provably convergent, and asymptotically optimal; even modeling that the objective is nonnegative is sufficient for this stability. We extend these results beyond convexity to weakly convex objectives, which include compositions of convex losses with smooth functions common in modern machine learning. We highlight the importance of robustness and accurate modeling with experimental evaluation of convergence time and algorithm sensitivity.

View details for DOI 10.1073/pnas.1908018116

View details for PubMedID 31666325
Necessary and Sufficient Geometries for Gradient Methods Levy, D., Duchi, J. C., Wallach, H., Larochelle, H., Beygelzimer, A., d'Alche-Buc, F., Fox, E., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2019

View details for Web of Science ID 000535866903016
Variance-based Regularization with Convex Objectives JOURNAL OF MACHINE LEARNING RESEARCH Duchi, J., Namkoong, H. 2019; 20

View details for Web of Science ID 000467894600001
STOCHASTIC (APPROXIMATE) PROXIMAL POINT METHODS: CONVERGENCE, OPTIMALITY, AND ADAPTIVITY SIAM JOURNAL ON OPTIMIZATION Asi, H., Duchi, J. C. 2019; 29 (3): 2257–90

View details for DOI 10.1137/18M1230323

View details for Web of Science ID 000487929500021
GRADIENT DESCENT FINDS THE CUBIC-REGULARIZED NONCONVEX NEWTON STEP SIAM JOURNAL ON OPTIMIZATION Carmon, Y., Duchi, J. 2019; 29 (3): 2146–78

View details for DOI 10.1137/17M1113898

View details for Web of Science ID 000487929500017
Modeling simple structures and geometry for better stochastic optimization algorithms Asi, H., Duchi, J. C., Chaudhuri, K., Sugiyama, M. MICROTOME PUBLISHING. 2019

View details for Web of Science ID 000509687902049
ACCELERATED METHODS FOR NONCONVEX OPTIMIZATION SIAM JOURNAL ON OPTIMIZATION Carmon, Y., Duchi, J. C., Hinder, O., Sidford, A. 2018; 28 (2): 1751–72

View details for DOI 10.1137/17M1114296

View details for Web of Science ID 000436991600031
STOCHASTIC METHODS FOR COMPOSITE AND WEAKLY CONVEX OPTIMIZATION PROBLEMS SIAM JOURNAL ON OPTIMIZATION Duchi, J. C., Ruan, F. 2018; 28 (4): 3229–59

View details for DOI 10.1137/17M1135086

View details for Web of Science ID 000453750000019
Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation O'Kelly, M., Sinha, A., Namkoong, H., Duchi, J., Tedrake, R., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

View details for Web of Science ID 000461852004039
Generalizing to Unseen Domains via Adversarial Data Augmentation Volpi, R., Namkoong, H., Sener, O., Duchi, J., Murino, V., Savarese, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

View details for Web of Science ID 000461823305036
Analysis of Krylov Subspace Solutions of Regularized Nonconvex Quadratic Problems Carmon, Y., Duchi, J. C., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

View details for Web of Science ID 000461852005030
Minimax Optimal Procedures for Locally Private Estimation JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2018; 113 (521): 182–201

View details for DOI 10.1080/01621459.2017.1389735

View details for Web of Science ID 000438960500018
Minimax Optimal Procedures for Locally Private Estimation Rejoinder JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2018; 113 (521): 212–15

View details for DOI 10.1080/01621459.2018.1442611

View details for Web of Science ID 000438960500024
Unsupervised Transformation Learning via Convex Relaxations Hashimoto, T. B., Duchi, J. C., Liang, P., Guyon, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017

View details for Web of Science ID 000452649406090
Variance-based Regularization with Convex Objectives Namkoong, H., Duchi, J. C., Guyon, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017

View details for Web of Science ID 000452649403004
Mean Estimation from Adaptive One-bit Measurements Kipnis, A., Duchi, J. C., IEEE IEEE. 2017: 1000–1007

View details for Web of Science ID 000428047800137
Simultaneous dimension reduction and adjustment for confounding variation PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA Lin, Z., Yang, C., Zhu, Y., Duchi, J., Fu, Y., Wang, Y., Jiang, B., Zamanighomi, M., Xu, X., Li, M., Sestan, N., Zhao, H., Wong, W. H. 2016; 113 (51): 14662-14667

Abstract

Dimension reduction methods are commonly applied to high-throughput biological datasets. However, the results can be hindered by confounding factors, either biological or technical in origin. In this study, we extend principal component analysis (PCA) to propose AC-PCA for simultaneous dimension reduction and adjustment for confounding (AC) variation. We show that AC-PCA can adjust for (i) variations across individual donors present in a human brain exon array dataset and (ii) variations of different species in a model organism ENCODE RNA sequencing dataset. Our approach is able to recover the anatomical structure of neocortical regions and to capture the shared variation among species during embryonic development. For gene selection purposes, we extend AC-PCA with sparsity constraints and propose and implement an efficient algorithm. The methods developed in this paper can also be applied to more general settings. The R package and MATLAB source code are available at https://github.com/linzx06/AC-PCA.

View details for DOI 10.1073/pnas.1617317113

View details for PubMedID 27930330
Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates JOURNAL OF MACHINE LEARNING RESEARCH Zhang, Y., Duchi, J., Wainwright, M. 2015; 16: 3299-3340

View details for Web of Science ID 000369888000031
Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function Evaluations IEEE TRANSACTIONS ON INFORMATION THEORY Duchi, J. C., Jordan, M. I., Wainwright, M. J., Wibisono, A. 2015; 61 (5): 2788-2806

View details for DOI 10.1109/TIT.2015.2409256

View details for Web of Science ID 000353515700036
Privacy Aware Learning JOURNAL OF THE ACM Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2014; 61 (6)

View details for DOI 10.1145/2666468

View details for Web of Science ID 000347051200006
Privacy: a few definitional aspects and consequences for minimax mean-squared error Barber, R., Duchi, J., IEEE IEEE. 2014: 1365–69

View details for Web of Science ID 000370073801085
THE ASYMPTOTICS OF RANKING ALGORITHMS ANNALS OF STATISTICS Duchi, J. C., Mackey, L., Jordan, M. I. 2013; 41 (5): 2292-2323

View details for DOI 10.1214/13-AOS1142

View details for Web of Science ID 000327746100002