Bio


John Duchi (http://web.stanford.edu/~jduchi/) is an assistant professor of Statistics and Electrical Engineering and (by courtesy) Computer Science at Stanford University. His work spans statistical learning, optimization, information theory, and computation, with a few driving goals. (1) To discover statistical learning procedures that optimally trade between resources--computation, communication, privacy provided to study participants--while maintaining good statistical performance. (2) To build efficient large-scale optimization methods that address the spectrum of optimization, machine learning, and data analysis problems we face, allowing us to move beyond bespoke solutions to methods that robustly work. (3) To develop tools to assess and guarantee the validity of--and confidence we should have in--machine-learned systems.

Academic Appointments


Honors & Awards


  • SIAM/OPT Early Career Prize, Society for Industrial and Applied Mathematics, Optimization Group (2020)
  • Young Investigator Award, Office of Naval Research (2019)
  • Best Paper Award, Neural Information Processing Systems (2017)
  • Sloan Foundation Fellow in Mathematics, Sloan Foundation (2016)
  • Doctoral Dissertation Award (Honorable Mention), Association for Computing Machinery (2015)
  • Okawa Foundation Award, Okawa Foundation (2015)
  • C.V. Ramamoorthy Distinguished Research Award, University of California, Berkeley (2014)
  • Best Student Paper Award, International Conference on Machine Learning (2010)

Professional Education


  • BS, Stanford University, Computer Science (2007)
  • MS, Stanford University, Computer Science (2008)
  • MA, University of California, Berkeley, Statistics (2012)
  • PhD, University of California, Berkeley, EECS (2014)

Current Research and Scholarly Interests


My work spans statistical learning, optimization, information theory, and computation, with a few driving goals: 1. To discover statistical learning procedures that optimally trade between real-world resources while maintaining statistical efficiency. 2. To build efficient large-scale optimization methods that move beyond bespoke solutions to methods that robustly work. 3. To develop tools to assess and guarantee the validity of---and confidence we should have in---machine-learned systems.

2019-20 Courses


Stanford Advisees


  • Doctoral Dissertation Reader (AC)
    Anqi Fu, Blaine Rister, Vatsal Sharan, Matthew Tsao, Andrew Ward
  • Orals Chair
    Zhihao Jia
  • Doctoral Dissertation Advisor (AC)
    Hilal Asi, Yair Carmon, Maxime Cauchois, Suyash Gupta, Andrew Naber, Aman Sinha
  • Master's Program Advisor
    Kevin Lee, Zucks Liu, Isaac Scheinfeld, Nicholas Tan, William Yin, Will Zhuk
  • Doctoral (Program)
    Daniel Levy, Ajay Mandlekar, Mark Nishimura, Matthew Tsao

All Publications


  • Solving (most) of a set of quadratic equalities: composite optimization for robust phase retrieval INFORMATION AND INFERENCE-A JOURNAL OF THE IMA Duchi, J. C., Ruan, F. 2019; 8 (3): 471–529
  • GRADIENT DESCENT FINDS THE CUBIC-REGULARIZED NONCONVEX NEWTON STEP SIAM JOURNAL ON OPTIMIZATION Carmon, Y., Duchi, J. 2019; 29 (3): 2146–78

    View details for DOI 10.1137/17M1113898

    View details for Web of Science ID 000487929500017

  • The importance of better models in stochastic optimization. Proceedings of the National Academy of Sciences of the United States of America Asi, H., Duchi, J. C. 2019

    Abstract

    Standard stochastic optimization methods are brittle, sensitive to stepsize choice and other algorithmic parameters, and they exhibit instability outside of well-behaved families of objectives. To address these challenges, we investigate models for stochastic optimization and learning problems that exhibit better robustness to problem families and algorithmic parameters. With appropriately accurate models-which we call the aprox family-stochastic methods can be made stable, provably convergent, and asymptotically optimal; even modeling that the objective is nonnegative is sufficient for this stability. We extend these results beyond convexity to weakly convex objectives, which include compositions of convex losses with smooth functions common in modern machine learning. We highlight the importance of robustness and accurate modeling with experimental evaluation of convergence time and algorithm sensitivity.

    View details for DOI 10.1073/pnas.1908018116

    View details for PubMedID 31666325

  • Modeling simple structures and geometry for better stochastic optimization algorithms Asi, H., Duchi, J. C., Chaudhuri, K., Sugiyama, M. MICROTOME PUBLISHING. 2019
  • Necessary and Sufficient Geometries for Gradient Methods Levy, D., Duchi, J. C., Wallach, H., Larochelle, H., Beygelzimer, A., d'Alche-Buc, F., Fox, E., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2019
  • Unlabeled Data Improves Adversarial Robustness Carmon, Y., Raghunathan, A., Schmidt, L., Liang, P., Duchi, J. C., Wallach, H., Larochelle, H., Beygelzimer, A., d'Alche-Buc, F., Fox, E., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2019
  • Variance-based Regularization with Convex Objectives JOURNAL OF MACHINE LEARNING RESEARCH Duchi, J., Namkoong, H. 2019; 20
  • STOCHASTIC (APPROXIMATE) PROXIMAL POINT METHODS: CONVERGENCE, OPTIMALITY, AND ADAPTIVITY SIAM JOURNAL ON OPTIMIZATION Asi, H., Duchi, J. C. 2019; 29 (3): 2257–90

    View details for DOI 10.1137/18M1230323

    View details for Web of Science ID 000487929500021

  • ACCELERATED METHODS FOR NONCONVEX OPTIMIZATION SIAM JOURNAL ON OPTIMIZATION Carmon, Y., Duchi, J. C., Hinder, O., Sidford, A. 2018; 28 (2): 1751–72

    View details for DOI 10.1137/17M1114296

    View details for Web of Science ID 000436991600031

  • STOCHASTIC METHODS FOR COMPOSITE AND WEAKLY CONVEX OPTIMIZATION PROBLEMS SIAM JOURNAL ON OPTIMIZATION Duchi, J. C., Ruan, F. 2018; 28 (4): 3229–59

    View details for DOI 10.1137/17M1135086

    View details for Web of Science ID 000453750000019

  • Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation O'Kelly, M., Sinha, A., Namkoong, H., Duchi, J., Tedrake, R., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018
  • Generalizing to Unseen Domains via Adversarial Data Augmentation Volpi, R., Namkoong, H., Sener, O., Duchi, J., Murino, V., Savarese, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018
  • Analysis of Krylov Subspace Solutions of Regularized Nonconvex Quadratic Problems Carmon, Y., Duchi, J. C., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018
  • Minimax Optimal Procedures for Locally Private Estimation JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2018; 113 (521): 182–201
  • Minimax Optimal Procedures for Locally Private Estimation Rejoinder JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2018; 113 (521): 212–15
  • Mean Estimation from Adaptive One-bit Measurements Kipnis, A., Duchi, J. C., IEEE IEEE. 2017: 1000–1007
  • Variance-based Regularization with Convex Objectives Namkoong, H., Duchi, J. C., Guyon, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017
  • Unsupervised Transformation Learning via Convex Relaxations Hashimoto, T. B., Duchi, J. C., Liang, P., Guyon, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017
  • Simultaneous dimension reduction and adjustment for confounding variation PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA Lin, Z., Yang, C., Zhu, Y., Duchi, J., Fu, Y., Wang, Y., Jiang, B., Zamanighomi, M., Xu, X., Li, M., Sestan, N., Zhao, H., Wong, W. H. 2016; 113 (51): 14662-14667

    Abstract

    Dimension reduction methods are commonly applied to high-throughput biological datasets. However, the results can be hindered by confounding factors, either biological or technical in origin. In this study, we extend principal component analysis (PCA) to propose AC-PCA for simultaneous dimension reduction and adjustment for confounding (AC) variation. We show that AC-PCA can adjust for (i) variations across individual donors present in a human brain exon array dataset and (ii) variations of different species in a model organism ENCODE RNA sequencing dataset. Our approach is able to recover the anatomical structure of neocortical regions and to capture the shared variation among species during embryonic development. For gene selection purposes, we extend AC-PCA with sparsity constraints and propose and implement an efficient algorithm. The methods developed in this paper can also be applied to more general settings. The R package and MATLAB source code are available at https://github.com/linzx06/AC-PCA.

    View details for DOI 10.1073/pnas.1617317113

    View details for PubMedID 27930330

  • Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates JOURNAL OF MACHINE LEARNING RESEARCH Zhang, Y., Duchi, J., Wainwright, M. 2015; 16: 3299-3340
  • Optimal Rates for Zero-Order Convex Optimization: The Power of Two Function Evaluations IEEE TRANSACTIONS ON INFORMATION THEORY Duchi, J. C., Jordan, M. I., Wainwright, M. J., Wibisono, A. 2015; 61 (5): 2788-2806
  • Privacy Aware Learning JOURNAL OF THE ACM Duchi, J. C., Jordan, M. I., Wainwright, M. J. 2014; 61 (6)

    View details for DOI 10.1145/2666468

    View details for Web of Science ID 000347051200006

  • Privacy: a few definitional aspects and consequences for minimax mean-squared error Barber, R., Duchi, J., IEEE IEEE. 2014: 1365–69
  • THE ASYMPTOTICS OF RANKING ALGORITHMS ANNALS OF STATISTICS Duchi, J. C., Mackey, L., Jordan, M. I. 2013; 41 (5): 2292-2323

    View details for DOI 10.1214/13-AOS1142

    View details for Web of Science ID 000327746100002