Stefano Ermon

All Publications

• Computational Sustainability: Computing for a Better World and a Sustainable Future COMMUNICATIONS OF THE ACM Gomes, C., Dietterich, T., Barrett, C., Conrad, J., Dilkina, B., Ermon, S., Fang, F., Farnsworth, A., Fern, A., Fern, X., Fink, D., Fisher, D., Flecker, A., Freund, D., Fuller, A., Gregoire, J., Hoperoft, J., Kelling, S., Kolter, Z., Powell, W., Sintov, N., Selker, J., Selman, B., Sheldon, D., Shmoys, D., Tambe, M., Wong, W., Wood, C., Wu, X., Xue, Y., Yadav, A., Yakubu, A., Zeeman, M. 2019; 62 (9): 56–65

  View details for DOI 10.1145/3339399

  View details for Web of Science ID 000483033900019

• High-Voltage Charging-Induced Strain, Heterogeneity, and Micro-Cracks in Secondary Particles of a Nickel-Rich Layered Cathode Material ADVANCED FUNCTIONAL MATERIALS Mao, Y., Wang, X., Xia, S., Zhang, K., Wei, C., Bak, S., Shadike, Z., Liu, X., Yang, Y., Xu, R., Pianetta, P., Ermon, S., Stavitski, E., Zhao, K., Xu, Z., Lin, F., Yang, X., Hu, E., Liu, Y. 2019; 29 (18)

  View details for DOI 10.1002/adfm.201900247

  View details for Web of Science ID 000471330500021

• InfoVAE: Balancing Learning and Inference in Variational Autoencoders Zhao, S., Song, J., Ermon, S., AAAI ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE. 2019: 5885–92

  View details for Web of Science ID 000486572500051

• Tile2Vec: Unsupervised Representation Learning for Spatially Distributed Data Jean, N., Wang, S., Samar, A., Azzari, G., Lobell, D., Ermon, S., AAAI ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE. 2019: 3967–74

  View details for Web of Science ID 000485292603121

• Rapid identification of pathogenic bacteria using Raman spectroscopy and deep learning. Nature communications Ho, C. S., Jean, N., Hogan, C. A., Blackmon, L., Jeffrey, S. S., Holodniy, M., Banaei, N., Saleh, A. A., Ermon, S., Dionne, J. 2019; 10 (1): 4927

  Abstract

  Raman optical spectroscopy promises label-free bacterial detection, identification, and antibiotic susceptibility testing in a single step. However, achieving clinically relevant speeds and accuracies remains challenging due to weak Raman signal from bacterial cells and numerous bacterial species and phenotypes. Here we generate an extensive dataset of bacterial Raman spectra and apply deep learning approaches to accurately identify 30 common bacterial pathogens. Even on low signal-to-noise spectra, we achieve average isolate-level accuracies exceeding 82% and antibiotic treatment identification accuracies of 97.0±0.3%. We also show that this approach distinguishes between methicillin-resistant and -susceptible isolates of Staphylococcus aureus (MRSA and MSSA) with 89±0.1% accuracy. We validate our results on clinical isolates from 50 patients. Using just 10 bacterial spectra from each patient isolate, we achieve treatment identification accuracies of 99.7%. Our approach has potential for culture-free pathogen identification and antibiotic susceptibility testing, and could be readily extended for diagnostics on blood, urine, and sputum.

  View details for DOI 10.1038/s41467-019-12898-9

  View details for PubMedID 31666527

• Predicting Economic Development using Geolocated Wikipedia Articles Sheehan, E., Meng, C., Tan, M., Uzkent, B., Jean, N., Burke, M., Lobell, D., Ermon, S., Assoc Comp Machinery ASSOC COMPUTING MACHINERY. 2019: 2698–2706

  View details for DOI 10.1145/3292500.3330784

  View details for Web of Science ID 000485562502077

• Using machine learning to discover shape descriptors for predicting emulsion stability in a microfluidic channel. Soft matter Khor, J. W., Jean, N., Luxenberg, E. S., Ermon, S., Tang, S. K. 2018

  Abstract

  In soft matter consisting of many deformable objects, object shapes often carry important information about local forces and their interactions with the local environment, and can be tightly coupled to the bulk properties and functions. In a concentrated emulsion, for example, the shapes of individual droplets are directly related to the local stress arising from interactions with neighboring drops, which in turn determine their stability and the resulting rheological properties. Shape descriptors used in prior work on single drops and dilute emulsions, where droplet-droplet interactions are largely negligible and the drop shapes are simple, are insufficient to fully capture the broad range of droplet shapes in a concentrated system. This paper describes the application of a machine learning method, specifically a convolutional autoencoder model, that learns to: (1) discover a low-dimensional code (8-dimensional) to describe droplet shapes within a concentrated emulsion, and (2) predict whether the drop will become unstable and undergo break-up. The input consists of images (N = 500002) of two-dimensional droplet boundaries extracted from movies of a concentrated emulsion flowing through a confined microfluidic channel as a monolayer. The model is able to faithfully reconstruct droplet shapes, as well as to achieve a classification accuracy of 91.7% in the prediction of droplet break-up, compared with 60% using conventional scalar descriptors based on droplet elongation. It is observed that 4 out of the 8 dimensions of the code are interpretable, corresponding to drop skewness, elongation, throat size, and surface curvature, respectively. Furthermore, the results show that drop elongation, throat size, and surface curvature are dominant factors in predicting droplet break-up for the flow conditions tested. The method presented is expected to facilitate follow-on work to identify the relationship between drop shapes and the interactions with other drops, and to identify potentially new modes of break-up mechanisms in a concentrated system. Finally, the method developed here should also apply to other soft materials such as foams, gels, and cells and tissues.

  View details for PubMedID 30570628

• Learning with Weak Supervision from Physics and Data-Driven Constraints AI MAGAZINE Ren, H., Stewart, R., Song, J., Kuleshov, V., Ermon, S. 2018; 39 (1): 27–38

  View details for Web of Science ID 000435949900005

• Bias and Generalization in Deep Generative Models: An Empirical Study Zhao, S., Ren, H., Yuan, A., Song, J., Goodman, N., Ermon, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

  View details for Web of Science ID 000461852005038

• Amortized Inference Regularization Shu, R., Bui, H. H., Zhao, S., Kochenderfer, M. J., Ermon, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

  View details for Web of Science ID 000461823304041

• Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance Jean, N., Xie, S., Ermon, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

  View details for Web of Science ID 000461823305035

• The Information Autoencoding Family: A Lagrangian Perspective on Latent Variable Generative Models Zhao, S., Song, J., Ermon, S., Globerson, A., Silva, R. AUAI PRESS. 2018: 1031–41

  View details for Web of Science ID 000493119200101

• Bayesian optimization and attribute adjustment Eismann, S., Levy, D., Shu, R., Bartzsch, S., Ermon, S., Globerson, A., Silva, R. AUAI PRESS. 2018: 1042–52

  View details for Web of Science ID 000493119200102

• End-to-End Learning of Motion Representation for Video Understanding Fan, L., Huang, W., Gan, C., Ermon, S., Gong, B., Huang, J., IEEE IEEE. 2018: 6016–25

  View details for DOI 10.1109/CVPR.2018.00630

  View details for Web of Science ID 000457843606018

• Infrastructure Quality Assessment in Africa using Satellite Imagery and Deep Learning Oshri, B., Hu, A., Adelson, P., Chen, X., Dupas, P., Weinstein, J., Burke, M., Lobell, D., Ermon, S., ACM ASSOC COMPUTING MACHINERY. 2018: 616–25

  View details for DOI 10.1145/3219819.3219924

  View details for Web of Science ID 000455346400065

• Deep Transfer Learning for Crop Yield Prediction with Remote Sensing Data Wang, A. X., Tran, C., Desai, N., Lobell, D., Ermon, S., Assoc Comp Machinery ASSOC COMPUTING MACHINERY. 2018

  View details for DOI 10.1145/3209811.3212707

  View details for Web of Science ID 000455345900050

• Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models Grover, A., Dhar, M., Ermon, S., AAAI ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE. 2018: 3069–76

  View details for Web of Science ID 000485488903019

• Boosted Generative Models Grover, A., Ermon, S., AAAI ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE. 2018: 3077–84

  View details for Web of Science ID 000485488903020

• Approximate Inference via Weighted Rademacher Complexity Kuck, J., Sabharwal, A., Ermon, S., AAAI ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE. 2018: 6376–83

  View details for Web of Science ID 000485488906057

• Deterministic Policy Optimization by Combining Pathwise and Score Function Estimators for Discrete Action Spaces Levy, D., Ermon, S., AAAI ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE. 2018: 3474–81

  View details for Web of Science ID 000485488903069

• Constructing Unrestricted Adversarial Examples with Generative Models Song, Y., Shu, R., Kushman, N., Ermon, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

  View details for Web of Science ID 000461852002082

• Multi-Agent Generative Adversarial Imitation Learning Song, J., Ren, H., Sadigh, D., Ermon, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

  View details for Web of Science ID 000461852002005

• Streamlining Variational Inference for Constraint Satisfaction Problems Grover, A., Achim, T., Ermon, S., Bengio, S., Wallach, H., Larochelle, H., Grauman, K., CesaBianchi, N., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018

  View details for Web of Science ID 000461852005016

• Coupling between oxygen redox and cation migration explains unusual electrochemistry in lithium-rich layered oxides NATURE COMMUNICATIONS Gent, W. E., Lim, K., Liang, Y., Li, Q., Barnes, T., Ahn, S., Stone, K. H., McIntire, M., Hong, J., Song, J., Li, Y., Mehta, A., Ermon, S., Tyliszczak, T., Kilcoyne, D., Vine, D., Park, J., Doo, S., Toney, M. F., Yang, W., Prendergast, D., Chueh, W. C. 2017; 8

  View details for DOI 10.1038/s41467-017-02041-x

  View details for Web of Science ID 000417702300042

• Autotuning Stencil Computations with Structural Ordinal Regression Learning Cosenza, B., Durillo, J. J., Ermon, S., Juurlink, B., IEEE IEEE. 2017: 287–96

  View details for DOI 10.1109/IPDPS.2017.102

  View details for Web of Science ID 000427044800030

• A-NICE-MC: Adversarial Training for MCMC Song, J., Zhao, S., Ermon, S., Guyon, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017

  View details for Web of Science ID 000452649405022

• InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations Li, Y., Song, J., Ermon, S., Guyon, Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017

  View details for Web of Science ID 000452649403085

• Coupling between oxygen redox and cation migration explains unusual electrochemistry in lithium-rich layered oxides. Nature communications Gent, W. E., Lim, K., Liang, Y., Li, Q., Barnes, T., Ahn, S. J., Stone, K. H., McIntire, M., Hong, J., Song, J. H., Li, Y., Mehta, A., Ermon, S., Tyliszczak, T., Kilcoyne, D., Vine, D., Park, J. H., Doo, S. K., Toney, M. F., Yang, W., Prendergast, D., Chueh, W. C. 2017; 8 (1): 2091

  Abstract

  Lithium-rich layered transition metal oxide positive electrodes offer access to anion redox at high potentials, thereby promising high energy densities for lithium-ion batteries. However, anion redox is also associated with several unfavorable electrochemical properties, such as open-circuit voltage hysteresis. Here we reveal that in Li1.17-x Ni0.21Co0.08Mn0.54O2, these properties arise from a strong coupling between anion redox and cation migration. We combine various X-ray spectroscopic, microscopic, and structural probes to show that partially reversible transition metal migration decreases the potential of the bulk oxygen redox couple by > 1 V, leading to a reordering in the anionic and cationic redox potentials during cycling. First principles calculations show that this is due to the drastic change in the local oxygen coordination environments associated with the transition metal migration. We propose that this mechanism is involved in stabilizing the oxygen redox couple, which we observe spectroscopically to persist for 500 charge/discharge cycles.

  View details for PubMedID 29233965

  View details for PubMedCentralID PMC5727078

• Monitoring Ethiopian Wheat Fungus with Satellite Imagery and Deep Feature Learning Pryzant, R., Ermon, S., Lobell, D., IEEE IEEE. 2017: 1524–32

  View details for DOI 10.1109/CVPRW.2017.196

  View details for Web of Science ID 000426448300189

• Unsupervised Data Mining in nanoscale X-ray Spectro-Microscopic Study of NdFeB Magnet SCIENTIFIC REPORTS Duan, X., Yang, F., Antono, E., Yang, W., Pianetta, P., Ermon, S., Mehta, A., Liu, Y. 2016; 6

  Abstract

  Novel developments in X-ray based spectro-microscopic characterization techniques have increased the rate of acquisition of spatially resolved spectroscopic data by several orders of magnitude over what was possible a few years ago. This accelerated data acquisition, with high spatial resolution at nanoscale and sensitivity to subtle differences in chemistry and atomic structure, provides a unique opportunity to investigate hierarchically complex and structurally heterogeneous systems found in functional devices and materials systems. However, handling and analyzing the large volume data generated poses significant challenges. Here we apply an unsupervised data-mining algorithm known as DBSCAN to study a rare-earth element based permanent magnet material, Nd2Fe14B. We are able to reduce a large spectro-microscopic dataset of over 300,000 spectra to 3, preserving much of the underlying information. Scientists can easily and quickly analyze in detail three characteristic spectra. Our approach can rapidly provide a concise representation of a large and complex dataset to materials scientists and chemists. For example, it shows that the surface of common Nd2Fe14B magnet is chemically and structurally very different from the bulk, suggesting a possible surface alteration effect possibly due to the corrosion, which could affect the material's overall properties.

  View details for DOI 10.1038/srep34406

  View details for Web of Science ID 000384188300001

  View details for PubMedID 27680388

  View details for PubMedCentralID PMC5041149

• Combining satellite imagery and machine learning to predict poverty. Science Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., Ermon, S. 2016; 353 (6301): 790-794

  Abstract

  Reliable data on economic livelihoods remain scarce in the developing world, hampering efforts to study these outcomes and to design policies that improve them. Here we demonstrate an accurate, inexpensive, and scalable method for estimating consumption expenditure and asset wealth from high-resolution satellite imagery. Using survey and satellite data from five African countries--Nigeria, Tanzania, Uganda, Malawi, and Rwanda--we show how a convolutional neural network can be trained to identify image features that can explain up to 75% of the variation in local-level economic outcomes. Our method, which requires only publicly available data, could transform efforts to track and target poverty in developing countries. It also demonstrates how powerful machine learning techniques can be applied in a setting with limited training data, suggesting broad potential application across many scientific domains.

  View details for DOI 10.1126/science.aaf7894

  View details for PubMedID 27540167