All Publications


  • STOCHASTIC (APPROXIMATE) PROXIMAL POINT METHODS: CONVERGENCE, OPTIMALITY, AND ADAPTIVITY SIAM JOURNAL ON OPTIMIZATION Asi, H., Duchi, J. C. 2019; 29 (3): 2257–90

    View details for DOI 10.1137/18M1230323

    View details for Web of Science ID 000487929500021

  • The importance of better models in stochastic optimization. Proceedings of the National Academy of Sciences of the United States of America Asi, H. n., Duchi, J. C. 2019

    Abstract

    Standard stochastic optimization methods are brittle, sensitive to stepsize choice and other algorithmic parameters, and they exhibit instability outside of well-behaved families of objectives. To address these challenges, we investigate models for stochastic optimization and learning problems that exhibit better robustness to problem families and algorithmic parameters. With appropriately accurate models-which we call the aprox family-stochastic methods can be made stable, provably convergent, and asymptotically optimal; even modeling that the objective is nonnegative is sufficient for this stability. We extend these results beyond convexity to weakly convex objectives, which include compositions of convex losses with smooth functions common in modern machine learning. We highlight the importance of robustness and accurate modeling with experimental evaluation of convergence time and algorithm sensitivity.

    View details for DOI 10.1073/pnas.1908018116

    View details for PubMedID 31666325