Andrea Montanari
John D. and Sigrid Banks Professor and Professor of Mathematics
Statistics
Bio
I am interested in developing efficient algorithms to make sense of large amounts of noisy data, extract information from observations, estimate signals from measurements. This effort spans several disciplines including statistics, computer science, information theory, machine learning.
I am also working on applications of these techniques to healthcare data analytics.
Academic Appointments
-
Professor, Statistics
-
Professor, Mathematics
-
Member, Bio-X
-
Member, Wu Tsai Neurosciences Institute
Professional Education
-
PhD, Scuola Normale Superiore, Pisa, Italy (2001)
2024-25 Courses
- Advanced Statistical Theory
STATS 314A (Aut) - Real Analysis
MATH 205A (Aut) - Statistics Faculty Research Presentations
STATS 303 (Aut) -
Independent Studies (15)
- Advanced Reading and Research
CS 499 (Aut, Win, Spr, Sum) - Advanced Reading and Research
MATH 360 (Aut, Win, Spr, Sum) - Industrial Research for Statisticians
STATS 398 (Aut, Win, Spr, Sum) - Master's Thesis and Thesis Research
EE 300 (Aut, Win, Sum) - Part-time Curricular Practical Training
CS 390D (Aut, Win, Spr, Sum) - Ph.D. Research
CME 400 (Aut, Win, Spr, Sum) - Research
STATS 399 (Aut, Win, Spr, Sum) - Senior Honors Thesis
MATH 197 (Spr) - Senior Project
CS 191 (Aut, Win, Spr, Sum) - Special Studies and Reports in Electrical Engineering
EE 191 (Aut, Win, Sum) - Special Studies and Reports in Electrical Engineering
EE 391 (Aut, Win, Sum) - Special Studies and Reports in Electrical Engineering (WIM)
EE 191W (Aut, Win, Sum) - Special Studies or Projects in Electrical Engineering
EE 190 (Aut, Win, Sum) - Special Studies or Projects in Electrical Engineering
EE 390 (Aut, Win, Sum) - Writing Intensive Senior Research Project
CS 191W (Aut, Win, Spr)
- Advanced Reading and Research
-
Prior Year Courses
2023-24 Courses
- Mathematical Problems in Machine Learning
MATH 276, STATS 375 (Spr) - Theory of Probability II
MATH 230B, STATS 310B (Win)
2021-22 Courses
- Introduction to Stochastic Processes II
STATS 218 (Spr) - Methods from Statistical Physics
STATS 369 (Aut) - Statistical Signal Processing
EE 378A (Spr)
- Mathematical Problems in Machine Learning
Stanford Advisees
-
Doctoral Dissertation Reader (AC)
Apratim Dey, Anmol Kagrecha, Timothy Sudijono, Yifei Wang -
Doctoral Dissertation Advisor (AC)
Basil Saeed -
Doctoral Dissertation Co-Advisor (AC)
Chen Cheng -
Doctoral (Program)
Basil Saeed
All Publications
-
The Landscape of the Spiked Tensor Model
COMMUNICATIONS ON PURE AND APPLIED MATHEMATICS
2019; 72 (11): 2282–2330
View details for DOI 10.1002/cpa.21861
View details for Web of Science ID 000486251500002
-
Fundamental Limits of Weak Recovery with Applications to Phase Retrieval
FOUNDATIONS OF COMPUTATIONAL MATHEMATICS
2019; 19 (3): 703–73
View details for DOI 10.1007/s10208-018-9395-y
View details for Web of Science ID 000470144600006
-
The spectral norm of random inner-product kernel matrices
PROBABILITY THEORY AND RELATED FIELDS
2019; 173 (1-2): 27–85
View details for DOI 10.1007/s00440-018-0830-4
View details for Web of Science ID 000459396500002
-
Optimization of the Sherrington-Kirkpatrick Hamiltonian
IEEE COMPUTER SOC. 2019: 1417–33
View details for DOI 10.1109/FOCS.2019.00087
View details for Web of Science ID 000510015300078
-
A Statistical Model for Motifs Detection
IEEE TRANSACTIONS ON INFORMATION THEORY
2018; 64 (12): 7594–7612
View details for DOI 10.1109/TIT.2018.2870853
View details for Web of Science ID 000451257100009
-
THE LANDSCAPE OF EMPIRICAL RISK FOR NONCONVEX LOSSES
ANNALS OF STATISTICS
2018; 46 (6): 2747–74
View details for DOI 10.1214/17-AOS1637
View details for Web of Science ID 000443987700008
-
Spectral Algorithms for Tensor Completion
COMMUNICATIONS ON PURE AND APPLIED MATHEMATICS
2018; 71 (11): 2381–2425
View details for DOI 10.1002/cpa.21748
View details for Web of Science ID 000444411800006
-
Generating Random Networks Without Short Cycles
OPERATIONS RESEARCH
2018; 66 (5): 1227–46
View details for DOI 10.1287/opre.2018.1730
View details for Web of Science ID 000446179500004
-
A mean field view of the landscape of two-layer neural networks
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
2018; 115 (33): E7665–E7671
View details for DOI 10.1073/pnas.1806579115
View details for Web of Science ID 000441638200005
-
A mean field view of the landscape of two-layer neural networks.
Proceedings of the National Academy of Sciences of the United States of America
2018
Abstract
Multilayer neural networks are among the most powerful models in machine learning, yet the fundamental reasons for this success defy mathematical understanding. Learning a neural network requires optimizing a nonconvex high-dimensional objective (risk function), a problem that is usually attacked using stochastic gradient descent (SGD). Does SGD converge to a global optimum of the risk or only to a local optimum? In the former case, does this happen because local minima are absent or because SGD somehow avoids them? In the latter, why do local minima reached by SGD have good generalization properties? In this paper, we consider a simple case, namely two-layer neural networks, and prove that-in a suitable scaling limit-SGD dynamics is captured by a certain nonlinear partial differential equation (PDE) that we call distributional dynamics (DD). We then consider several specific examples and show how DD can be used to prove convergence of SGD to networks with nearly ideal generalization error. This description allows for "averaging out" some of the complexities of the landscape of neural networks and can be used to prove a general convergence result for noisy SGD.
View details for PubMedID 30054315
-
ONLINE RULES FOR CONTROL OF FALSE DISCOVERY RATE AND FALSE DISCOVERY EXCEEDANCE
ANNALS OF STATISTICS
2018; 46 (2): 526–54
View details for DOI 10.1214/17-AOS1559
View details for Web of Science ID 000431125400004
-
Contextual Stochastic Block Models
NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2018
View details for Web of Science ID 000461852003017
-
EXTREMAL CUTS OF SPARSE RANDOM GRAPHS
ANNALS OF PROBABILITY
2017; 45 (2): 1190-1217
View details for DOI 10.1214/15-AOP1084
View details for Web of Science ID 000398966500013
-
On the Limitation of Spectral Methods: From the Gaussian Hidden Clique Problem to Rank One Perturbations of Gaussian Tensors
IEEE TRANSACTIONS ON INFORMATION THEORY
2017; 63 (3): 1572-1579
View details for DOI 10.1109/TIT.2016.2637959
View details for Web of Science ID 000395822500015
-
Universality of the Elastic Net Error
IEEE. 2017: 2338–42
View details for Web of Science ID 000430345202081
-
Inference in Graphical Models via Semidefinite Programming Hierarchies
NEURAL INFORMATION PROCESSING SYSTEMS (NIPS). 2017
View details for Web of Science ID 000452649400040
-
How Well Do Local Algorithms Solve Semidefinite Programs?
ASSOC COMPUTING MACHINERY. 2017: 604–14
View details for DOI 10.1145/3055399.3055451
View details for Web of Science ID 000440317600056
-
High dimensional robust M-estimation: asymptotic variance via approximate message passing
PROBABILITY THEORY AND RELATED FIELDS
2016; 166 (3-4): 935-969
View details for DOI 10.1007/s00440-015-0675-z
View details for Web of Science ID 000387502400008
-
Phase transitions in semidefinite relaxations
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
2016; 113 (16): E2218-E2223
Abstract
Statistical inference problems arising within signal processing, data mining, and machine learning naturally give rise to hard combinatorial optimization problems. These problems become intractable when the dimensionality of the data is large, as is often the case for modern datasets. A popular idea is to construct convex relaxations of these combinatorial problems, which can be solved efficiently for large-scale datasets. Semidefinite programming (SDP) relaxations are among the most powerful methods in this family and are surprisingly well suited for a broad range of problems where data take the form of matrices or graphs. It has been observed several times that when the statistical noise is small enough, SDP relaxations correctly detect the underlying combinatorial structures. In this paper we develop asymptotic predictions for several detection thresholds, as well as for the estimation error above these thresholds. We study some classical SDP relaxations for statistical problems motivated by graph synchronization and community detection in networks. We map these optimization problems to statistical mechanics models with vector spins and use nonrigorous techniques from statistical mechanics to characterize the corresponding phase transitions. Our results clarify the effectiveness of SDP relaxations in solving high-dimensional statistical problems.
View details for DOI 10.1073/pnas.1523097113
View details for Web of Science ID 000374393800005
View details for PubMedID 27001856
View details for PubMedCentralID PMC4843421
-
Non-Negative Principal Component Analysis: Message Passing Algorithms and Sharp Asymptotics
IEEE TRANSACTIONS ON INFORMATION THEORY
2016; 62 (3): 1458-1484
View details for DOI 10.1109/TIT.2015.2457942
View details for Web of Science ID 000370954900025
-
Sparse PCA via Covariance Thresholding
JOURNAL OF MACHINE LEARNING RESEARCH
2016; 17
View details for Web of Science ID 000391660400001
-
Statistical analysis of a low cost method for multiple disease prediction.
Statistical methods in medical research
2016: 962280216680242-?
Abstract
Early identification of individuals at risk for chronic diseases is of significant clinical value. Early detection provides the opportunity to slow the pace of a condition, and thus help individuals to improve or maintain their quality of life. Additionally, it can lessen the financial burden on health insurers and self-insured employers. As a solution to mitigate the rise in chronic conditions and related costs, an increasing number of employers have recently begun using wellness programs, which typically involve an annual health risk assessment. Unfortunately, these risk assessments have low detection capability, as they should be low-cost and hence rely on collecting relatively few basic biomarkers. Thus one may ask, how can we select a low-cost set of biomarkers that would be the most predictive of multiple chronic diseases? In this paper, we propose a statistical data-driven method to address this challenge by minimizing the number of biomarkers in the screening procedure while maximizing the predictive power over a broad spectrum of diseases. Our solution uses multi-task learning and group dimensionality reduction from machine learning and statistics. We provide empirical validation of the proposed solution using data from two different electronic medical records systems, with comparisons over a statistical benchmark.
View details for DOI 10.1177/0962280216680242
View details for PubMedID 27932665
-
THE SET OF SOLUTIONS OF RANDOM XORSAT FORMULAE
ANNALS OF APPLIED PROBABILITY
2015; 25 (5): 2743-2808
View details for DOI 10.1214/14-AAP1060
View details for Web of Science ID 000360868800010
-
Finding One Community in a Sparse Graph
JOURNAL OF STATISTICAL PHYSICS
2015; 161 (2): 273-299
View details for DOI 10.1007/s10955-015-1338-2
View details for Web of Science ID 000361814400001
-
Finding Hidden Cliques of Size root N/e in Nearly Linear Time
FOUNDATIONS OF COMPUTATIONAL MATHEMATICS
2015; 15 (4): 1069-1128
View details for DOI 10.1007/s10208-014-9215-y
View details for Web of Science ID 000358262500007
-
UNIVERSALITY IN POLYTOPE PHASE TRANSITIONS AND MESSAGE PASSING ALGORITHMS
ANNALS OF APPLIED PROBABILITY
2015; 25 (2): 753-822
View details for DOI 10.1214/14-AAP1010
View details for Web of Science ID 000350708000012
-
Bargaining dynamics in exchange networks
JOURNAL OF ECONOMIC THEORY
2015; 156: 417-454
View details for DOI 10.1016/j.jet.2014.02.007
View details for Web of Science ID 000349728700014
-
Computational implications of reducing data to sufficient statistics
ELECTRONIC JOURNAL OF STATISTICS
2015; 9 (2): 2370-2390
View details for DOI 10.1214/15-EJS1059
View details for Web of Science ID 000366270900030
-
A Low-Cost Method for Multiple Disease Prediction.
AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium
2015; 2015: 329-338
Abstract
Recently, in response to the rising costs of healthcare services, employers that are financially responsible for the healthcare costs of their workforce have been investing in health improvement programs for their employees. A main objective of these so called "wellness programs" is to reduce the incidence of chronic illnesses such as cardiovascular disease, cancer, diabetes, and obesity, with the goal of reducing future medical costs. The majority of these wellness programs include an annual screening to detect individuals with the highest risk of developing chronic disease. Once these individuals are identified, the company can invest in interventions to reduce the risk of those individuals. However, capturing many biomarkers per employee creates a costly screening procedure. We propose a statistical data-driven method to address this challenge by minimizing the number of biomarkers in the screening procedure while maximizing the predictive power over a broad spectrum of diseases. Our solution uses multi-task learning and group dimensionality reduction from machine learning and statistics. We provide empirical validation of the proposed solution using data from two different electronic medical records systems, with comparisons to a statistical benchmark.
View details for PubMedID 26958164
-
On the Concentration of the Number of Solutions of Random Satisfiability Formulas
RANDOM STRUCTURES & ALGORITHMS
2014; 45 (3): 362-382
View details for DOI 10.1002/rsa.20501
View details for Web of Science ID 000341194800002
-
Hypothesis Testing in High-Dimensional Regression Under the Gaussian Random Design Model: Asymptotic Theory
IEEE TRANSACTIONS ON INFORMATION THEORY
2014; 60 (10): 6522-6554
View details for DOI 10.1109/TIT.2014.2343629
View details for Web of Science ID 000342416900042
-
Confidence Intervals and Hypothesis Testing for High-Dimensional Regression
JOURNAL OF MACHINE LEARNING RESEARCH
2014; 15: 2869-2909
View details for Web of Science ID 000344638800002
-
Accelerated Time-of-Flight Mass Spectrometry
IEEE TRANSACTIONS ON SIGNAL PROCESSING
2014; 62 (15): 3784-3798
View details for DOI 10.1109/TSP.2014.2329644
View details for Web of Science ID 000340092600005
-
The Replica Symmetric Solution for Potts Models on d-Regular Graphs
COMMUNICATIONS IN MATHEMATICAL PHYSICS
2014; 327 (2): 551-575
View details for DOI 10.1007/s00220-014-1956-6
View details for Web of Science ID 000334052300007
-
Information-Theoretically Optimal Compressed Sensing via Spatial Coupling and Approximate Message Passing
IEEE TRANSACTIONS ON INFORMATION THEORY
2013; 59 (11): 7434-7464
View details for DOI 10.1109/TIT.2013.2274513
View details for Web of Science ID 000325981100031
-
Optimal Coding for the Binary Deletion Channel With Small Deletion Probability
IEEE TRANSACTIONS ON INFORMATION THEORY
2013; 59 (10): 6192-6219
View details for DOI 10.1109/TIT.2013.2262020
View details for Web of Science ID 000324573500003
-
Localization from Incomplete Noisy Distance Measurements
FOUNDATIONS OF COMPUTATIONAL MATHEMATICS
2013; 13 (3): 297-345
View details for DOI 10.1007/s10208-012-9129-5
View details for Web of Science ID 000318174900001
-
Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising
IEEE TRANSACTIONS ON INFORMATION THEORY
2013; 59 (6): 3396-3433
View details for DOI 10.1109/TIT.2013.2239356
View details for Web of Science ID 000320709800013
-
The phase transition of matrix recovery from Gaussian measurements matches the minimax MSE of matrix denoising.
Proceedings of the National Academy of Sciences of the United States of America
2013; 110 (21): 8405-8410
Abstract
Let X(0) be an unknown M by N matrix. In matrix recovery, one takes n < MN linear measurements y(1),…,y(n) of X(0), where y(i) = Tr(A(T)iX(0)) and each A(i) is an M by N matrix. A popular approach for matrix recovery is nuclear norm minimization (NNM): solving the convex optimization problem min ||X||*subject to y(i) =Tr(A(T)(i)X) for all 1 ≤ i ≤ n, where || · ||* denotes the nuclear norm, namely, the sum of singular values. Empirical work reveals a phase transition curve, stated in terms of the undersampling fraction δ(n,M,N) = n/(MN), rank fraction ρ=rank(X0)/min {M,N}, and aspect ratio β=M/N. Specifically when the measurement matrices Ai have independent standard Gaussian random entries, a curve δ*(ρ) = δ*(ρ;β) exists such that, if δ > δ*(ρ), NNM typically succeeds for large M,N, whereas if δ < δ*(ρ), it typically fails. An apparently quite different problem is matrix denoising in Gaussian noise, in which an unknown M by N matrix X(0) is to be estimated based on direct noisy measurements Y =X(0) + Z, where the matrix Z has independent and identically distributed Gaussian entries. A popular matrix denoising scheme solves the unconstrained optimization problem min|| Y-X||(2)(F)/2+λ||X||*. When optimally tuned, this scheme achieves the asymptotic minimax mean-squared error M(ρ;β) = lim(M,N → ∞)inf(λ)sup(rank(X) ≤ ρ · M)MSE(X,X(λ)), where M/N → . We report extensive experiments showing that the phase transition δ*(ρ) in the first problem, matrix recovery from Gaussian measurements, coincides with the minimax risk curve M(ρ)=M(ρ;β) in the second problem, matrix denoising in Gaussian noise: δ*(ρ)=M(ρ), for any rank fraction 0 < ρ < 1 (at each common aspect ratio β). Our experiments considered matrices belonging to two constraint classes: real M by N matrices, of various ranks and aspect ratios, and real symmetric positive-semidefinite N by N matrices, of various ranks.
View details for DOI 10.1073/pnas.1306110110
View details for PubMedID 23650360
View details for PubMedCentralID PMC3666686
-
Iterative Coding for Network Coding
IEEE TRANSACTIONS ON INFORMATION THEORY
2013; 59 (3): 1563-1572
View details for DOI 10.1109/TIT.2012.2236912
View details for Web of Science ID 000315120400021
-
The mutual information of a class of graphical channels
51st Annual Allerton Conference on Communication, Control, and Computing
IEEE. 2013: 20–25
View details for Web of Science ID 000350802400004
-
Linear Bandits in High Dimension and Recommendation Systems
50th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
IEEE. 2013: 1750–1754
View details for Web of Science ID 000320654000242
-
Nearly Optimal Sample Size in Hypothesis Testing for High-Dimensional Regression
51st Annual Allerton Conference on Communication, Control, and Computing
IEEE. 2013: 1427–1434
View details for Web of Science ID 000350802400199
- Finding Hidden Cliques of Size in Nearly Linear Time 2013
- Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theory 2013
- Confidence Intervals and Hypothesis Testing for High-Dimensional Regression 2013
- High Dimensional Robust M-Estimation: Asymptotic Variance via Approximate Message Passing 2013
- Factor models on locally tree-like graphs Annals of Probability 2013
-
Lossy Compression of Discrete Sources via the Viterbi Algorithm
IEEE TRANSACTIONS ON INFORMATION THEORY
2012; 58 (4): 2475-2489
View details for DOI 10.1109/TIT.2011.2178059
View details for Web of Science ID 000302079800033
-
The LASSO Risk for Gaussian Matrices
IEEE TRANSACTIONS ON INFORMATION THEORY
2012; 58 (4): 1997-2017
View details for DOI 10.1109/TIT.2011.2174612
View details for Web of Science ID 000302079800001
-
The weak limit of Ising models on locally tree-like graphs
PROBABILITY THEORY AND RELATED FIELDS
2012; 152 (1-2): 31-51
View details for DOI 10.1007/s00440-010-0315-6
View details for Web of Science ID 000299131700002
-
Information-Theoretically Optimal Compressed Sensing via Spatial Coupling and Approximate Message Passing
IEEE International Symposium on Information Theory
IEEE. 2012
View details for Web of Science ID 000312544301066
- Universality in Polytope Phase Transitions and Message Passing Algorithms 2012
- The noise sensitivity phase transition in compressed sensing IEEE Transactions on Information Theory 2012
- Graphical Models Concepts in Compressed Sensing in Compressed Sensing Cambridge university PRess. 2012: 1
-
ACCELERATED TIME-OF-FLIGHT MASS SPECTROMETRY
IEEE Statistical Signal Processing Workshop (SSP)
IEEE. 2012: 432–435
View details for Web of Science ID 000309943200109
-
Subsampling at Information Theoretically Optimal Rates
IEEE International Symposium on Information Theory
IEEE. 2012
View details for Web of Science ID 000312544302106
-
Universality in Polytope Phase Transitions and Iterative Algorithms
IEEE International Symposium on Information Theory
IEEE. 2012
View details for Web of Science ID 000312544301149
-
The Noise-Sensitivity Phase Transition in Compressed Sensing
IEEE TRANSACTIONS ON INFORMATION THEORY
2011; 57 (10): 6920-6941
View details for DOI 10.1109/TIT.2011.2165823
View details for Web of Science ID 000295739000043
-
MAJORITY DYNAMICS ON TREES AND THE DYNAMIC CAVITY METHOD
ANNALS OF APPLIED PROBABILITY
2011; 21 (5): 1694-1748
View details for DOI 10.1214/10-AAP729
View details for Web of Science ID 000297027800003
-
Applications of the Lindeberg Principle in Communications and Statistical Learning
IEEE TRANSACTIONS ON INFORMATION THEORY
2011; 57 (4): 2440-2450
View details for DOI 10.1109/TIT.2011.2112231
View details for Web of Science ID 000288459100044
-
The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
IEEE International Symposium on Information Theory
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC. 2011: 764–85
View details for DOI 10.1109/TIT.2010.2094817
View details for Web of Science ID 000286514200014
-
Subexponential convergence for information aggregation on regular trees
50th IEEE Conference of Decision and Control (CDC)/European Control Conference (ECC)
IEEE. 2011: 5317–5322
View details for Web of Science ID 000303506205152
- Gossip PCA 2011
- Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising 2011
- Information-Theoretically Optimal Compressed Sensing via Spatial Coupling and Approximate Message Passing 2011
- Distributed Storage for Intermittent Energy Sources: Control Design and Performance Limits 2011
-
Fast Convergence of Natural Bargaining Dynamics in Exchange Networks
22nd Annual ACM/SIAM Symposium on Discrete Algorithms
SIAM. 2011: 1518–1537
View details for Web of Science ID 000296182400118
-
Localization from Incomplete Noisy Distance Measurements
IEEE International Symposium on Information Theory (ISIT)
IEEE. 2011: 1584–1588
View details for Web of Science ID 000297465101155
-
Information Theoretic Limits on Learning Stochastic Differential Equations
IEEE International Symposium on Information Theory (ISIT)
IEEE. 2011: 855–859
View details for Web of Science ID 000297465101007
- Factor models on locally tree-like graphs 2011
-
Compressed Sensing over l(p)-balls: Minimax Mean Square Error
IEEE International Symposium on Information Theory (ISIT)
IEEE. 2011: 129–133
View details for Web of Science ID 000297465100027
-
RECONSTRUCTION AND CLUSTERING IN RANDOM CONSTRAINT SATISFACTION PROBLEMS
SIAM JOURNAL ON DISCRETE MATHEMATICS
2011; 25 (2): 771-808
View details for DOI 10.1137/090755862
View details for Web of Science ID 000292302000024
-
The spread of innovations in social networks
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
2010; 107 (47): 20196-20201
Abstract
Which network structures favor the rapid spread of new ideas, behaviors, or technologies? This question has been studied extensively using epidemic models. Here we consider a complementary point of view and consider scenarios where the individuals' behavior is the result of a strategic choice among competing alternatives. In particular, we study models that are based on the dynamics of coordination games. Classical results in game theory studying this model provide a simple condition for a new action or innovation to become widespread in the network. The present paper characterizes the rate of convergence as a function of the structure of the interaction network. The resulting predictions differ strongly from the ones provided by epidemic models. In particular, it appears that innovation spreads much more slowly on well-connected network structures dominated by long-range links than in low-dimensional ones dominated, for example, by geographic proximity.
View details for DOI 10.1073/pnas.1004098107
View details for Web of Science ID 000284529000013
View details for PubMedID 21076030
View details for PubMedCentralID PMC2996710
-
Matrix Completion from Noisy Entries
JOURNAL OF MACHINE LEARNING RESEARCH
2010; 11: 2057-2078
View details for Web of Science ID 000282523000005
-
Matrix Completion From a Few Entries
IEEE TRANSACTIONS ON INFORMATION THEORY
2010; 56 (6): 2980-2998
View details for DOI 10.1109/TIT.2010.2046205
View details for Web of Science ID 000277880200039
-
ISING MODELS ON LOCALLY TREE-LIKE GRAPHS
ANNALS OF APPLIED PROBABILITY
2010; 20 (2): 565-592
View details for DOI 10.1214/09-AAP627
View details for Web of Science ID 000283529500007
-
The dynamics of message passing on dense graphs, with applications to compressed sensing
2010 IEEE International Symposium on Information Theory
IEEE. 2010: 1528–1532
View details for Web of Science ID 000287512700308
- Learning Networks of Stochastic Differential Equations 2010
- Tight Thresholds for Cuckoo Hashing via XORSAT 2010
- Gibbs Measures and Phase Transitions on Sparse Random Graphs Brazilian Journal of Probability and Statistics 2008 Brazilian School of Probability. 2010: 1
- Ising models on locally tree-like graphs Annals of Applied Probability 2010
- Fast Convergence of Natural Bargaining Dynamics in Exchange Networks 2010
- The LASSO risk: asymptotic results and real world examples 2010
- The Spread of Innovations in Social Networks 2010
- Graphical Models Concepts in Compressed Sensing 2010
-
On the deletion channel with small deletion probability
2010 IEEE International Symposium on Information Theory
IEEE. 2010: 1002–1006
View details for Web of Science ID 000287512700201
-
Tight Thresholds for Cuckoo Hashing via XORSAT
37th International Colloquium on Automata, Languages and Programming
SPRINGER-VERLAG BERLIN. 2010: 213–225
View details for Web of Science ID 000286345400019
-
An Empirical Scaling Law for Polar Codes
2010 IEEE International Symposium on Information Theory
IEEE. 2010: 884–888
View details for Web of Science ID 000287512700177
-
Regularization for Matrix Completion
2010 IEEE International Symposium on Information Theory
IEEE. 2010: 1503–1507
View details for Web of Science ID 000287512700303
- Majority dynamics on trees and the dynamic cavity method Annals of Applied Probability 2010
-
Message-passing algorithms for compressed sensing
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
2009; 106 (45): 18914-18919
Abstract
Compressed sensing aims to undersample certain high-dimensional signals yet accurately reconstruct them by exploiting signal characteristics. Accurate reconstruction is possible when the object to be recovered is sufficiently sparse in a known basis. Currently, the best known sparsity-undersampling tradeoff is achieved when reconstructing by convex optimization, which is expensive in important large-scale applications. Fast iterative thresholding algorithms have been intensively studied as alternatives to convex optimization for large-scale problems. Unfortunately known fast algorithms offer substantially worse sparsity-undersampling tradeoffs than convex optimization. We introduce a simple costless modification to iterative thresholding making the sparsity-undersampling tradeoff of the new algorithms equivalent to that of the corresponding convex optimization procedures. The new iterative-thresholding algorithms are inspired by belief propagation in graphical models. Our empirical measurements of the sparsity-undersampling tradeoff for the new algorithms agree with theoretical calculations. We show that a state evolution formalism correctly derives the true sparsity-undersampling tradeoff. There is a surprising agreement between earlier calculations based on random convex polytopes and this apparently very different theoretical formalism.
View details for DOI 10.1073/pnas.0909892106
View details for Web of Science ID 000271637500010
View details for PubMedID 19858495
View details for PubMedCentralID PMC2767368
-
The Generalized Area Theorem and Some of its Consequences
IEEE TRANSACTIONS ON INFORMATION THEORY
2009; 55 (11): 4793-4821
View details for DOI 10.1109/TIT.2009.2030457
View details for Web of Science ID 000271019700001
-
Finite-Length Scaling for Iteratively Decoded LDPC Ensembles
IEEE TRANSACTIONS ON INFORMATION THEORY
2009; 55 (2): 473-498
View details for DOI 10.1109/TIT.2008.2009580
View details for Web of Science ID 000263375500001
-
Low-rank Matrix Completion with Noisy Observations: a Quantitative Comparison
47th Annual Allerton Conference on Communication, Control, and Computing
IEEE. 2009: 1216–1222
View details for Web of Science ID 000279627100166
- Generating Random Graphs with Large Girth 2009
- Matrix Completion from a Few Entries 2009
- Which graphical models are difficult to learn? 2009
- An Implementable Scheme for Universal Lossy Compression of Discrete Markov Sources 2009
-
Matrix Completion from a Few Entries
IEEE International Symposium on Information Theory (ISIT 2009)
IEEE. 2009: 324–328
View details for Web of Science ID 000280141400067
-
Convergence to Equilibrium in Local Interaction Games
50th Annual IEEE Symposium on Foundations of Computer Science
IEEE COMPUTER SOC. 2009: 303–312
View details for DOI 10.1109/FOCS.2009.64
View details for Web of Science ID 000278330300030
-
An Iterative Scheme for Near Optimal and Universal Lossy Compression
IEEE Information Theory Workshop on Networking and Information Theory
IEEE. 2009: 231–235
View details for Web of Science ID 000273966100049
-
Generating random Tanner-graphs with large girth
IEEE Information Theory Workshop (ITW)
IEEE. 2009: 154–157
View details for Web of Science ID 000305809200032
- Matrix Completion from Noisy Entries 2009
- Information, Physics, and Computation Oxford University Press. 2009
-
Generating Random Graphs with Large Girth
20th Annual ACM-SIAM Symposium on Discrete Algorithms
SIAM. 2009: 566–575
View details for Web of Science ID 000281447000063
-
Maxwell Construction: The Hidden Bridge Between Iterative and Maximum a Posteriori Decoding
IEEE TRANSACTIONS ON INFORMATION THEORY
2008; 54 (12): 5277-5307
View details for DOI 10.1109/TIT.2008.2006466
View details for Web of Science ID 000261648100001
-
Estimating random variables from random sparse observations
EUROPEAN TRANSACTIONS ON TELECOMMUNICATIONS
2008; 19 (4): 385-403
View details for DOI 10.1002/ett.1289
View details for Web of Science ID 000257012200005
-
Clusters of solutions and replica symmetry breaking in random k-satisfi ability
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT
2008
View details for DOI 10.1088/1742-5468/2008/04/P04004
View details for Web of Science ID 000255662000008
-
The Slope Scaling Parameter for General Channels, Decoders, and Ensembles
IEEE International Symposium on Information Theory
IEEE. 2008: 1443–1447
View details for Web of Science ID 000260364401009
- Finite Size Scaling for the Core of Large Random Hypergraphs Annals of Applied Probability 2008
-
Counter Braids: A Novel Counter Architecture for Per-Flow Measurement
International Conference on Measurement and Modeling of Computer Systems
ASSOC COMPUTING MACHINERY. 2008: 121–32
View details for Web of Science ID 000266429500011
-
Counter Braids: Asymptotic Optimality of the Message Passing Decoding Algorithm
46th Annual Allerton Conference on Communication, Control and Computing
IEEE. 2008: 209–216
View details for Web of Science ID 000268229600032
-
Computing the threshold shift for general channels
IEEE International Symposium on Information Theory
IEEE. 2008: 1448–1452
View details for Web of Science ID 000260364401010
-
An Implementable Scheme for Universal Lossy Compression of Discrete Markov Sources
19th Data Compression Conference
IEEE COMPUTER SOC. 2008: 292–301
View details for DOI 10.1109/DCC.2009.72
View details for Web of Science ID 000268029900030
-
Smooth compression, Gallager bound and Nonlinear sparse-graph codes
IEEE International Symposium on Information Theory
IEEE. 2008: 2474–2478
View details for Web of Science ID 000260364401219
-
Counter Braids
IEEE Information Theory Workshop
IEEE. 2008: 220–221
View details for Web of Science ID 000259299000048
-
Learning Low Rank Matrices from O(n) Entries
46th Annual Allerton Conference on Communication, Control and Computing
IEEE. 2008: 1365–1372
View details for Web of Science ID 000268229600194
-
How to find good finite-length codes: from art towards science
4th International Symposium on Turbo Codes and Related Topics
WILEY-BLACKWELL. 2007: 491–508
View details for DOI 10.1002/ett.1182
View details for Web of Science ID 000248443400007
-
A simple one dimensional glassy Kac model
JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT
2007
View details for DOI 10.1088/1742-5468/2007/08/P08004
View details for Web of Science ID 000249277600007
-
Gibbs states and the set of solutions of random constraint satisfaction problems
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
2007; 104 (25): 10318-10323
Abstract
An instance of a random constraint satisfaction problem defines a random subset (the set of solutions) of a large product space chiN (the set of assignments). We consider two prototypical problem ensembles (random k-satisfiability and q-coloring of random regular graphs) and study the uniform measure with support on S. As the number of constraints per variable increases, this measure first decomposes into an exponential number of pure states ("clusters") and subsequently condensates over the largest such states. Above the condensation point, the mass carried by the n largest states follows a Poisson-Dirichlet process. For typical large instances, the two transitions are sharp. We determine their precise location. Further, we provide a formal definition of each phase transition in terms of different notions of correlation between distinct variables in the problem. The degree of correlation naturally affects the performances of many search/sampling algorithms. Empirical evidence suggests that local Monte Carlo Markov chain strategies are effective up to the clustering phase transition and belief propagation up to the condensation point. Finally, refined message passing techniques (such as survey propagation) may also beat this threshold.
View details for DOI 10.1073/pnas.0703685104
View details for Web of Science ID 000247500000006
View details for PubMedID 17567754
View details for PubMedCentralID PMC1965511
-
Reconstruction for models on random graphs
48th Annual IEEE Symposium on Foundations of Computer Science
IEEE COMPUTER SOC. 2007: 194–204
View details for DOI 10.1109/FOCS.2007.58
View details for Web of Science ID 000252161900018
- Detailed Network Measurements Using Sparse Graph Counters: The Theory 2007
- TP Decoding 2007
- Counting Good Truth Assignmants for Random Satisfiability Formulae 2007
- Solving Constraint Satisfaction Problems through Belief Propagation-guided Decimation 2007
- Modern Coding Theory: The Statistical Mechanics and Computer Science Point of View Les Houches Summer School on Mathematical Statistical Physics Elsevier. 2007: 1
-
A generalization of the finite-length scaling approach beyond the BEC
IEEE International Symposium on Information Theory
IEEE. 2007: 1011–1015
View details for Web of Science ID 000257010201034
-
Asymptotic rate versus design rate
IEEE International Symposium on Information Theory
IEEE. 2007: 1541–1545
View details for Web of Science ID 000257010201140
-
Rigorous inequalities between length and time scales in glassy systems
JOURNAL OF STATISTICAL PHYSICS
2006; 125 (1): 22-54
View details for DOI 10.1007/s10955-006-9175-y
View details for Web of Science ID 000241842100002
-
On the dynamics of the glass transition on Bethe lattices
JOURNAL OF STATISTICAL PHYSICS
2006; 124 (1): 103-189
View details for DOI 10.1007/s10955-006-9103-1
View details for Web of Science ID 000240126900006
-
Precision electroweak measurements on the Z resonance
PHYSICS REPORTS-REVIEW SECTION OF PHYSICS LETTERS
2006; 427 (5-6): 257-454
View details for DOI 10.1016/j.physrep.2005.12.006
View details for Web of Science ID 000236815000001
- The Asymptotic Error Floor of LDPC Ensembles Under BP Decoding 2006
- Finite-Length Scaling for Gallager A 2006
- Two Lectures on Iterative Coding and Statistical Mechanics Les Houches Summer School on Mathematical Statistical Physics Elsevier. 2006: 1
- Analysis of Belief propagation for Non-Linear Problems: The Example of CDMA (or: How to Prove Tanaka's Formula) 2006
- Approximate analysis of search algorithms with “physical” methods Phase Transitions and Algorithmic Complexity Oxford. 2006: 1
- How to Find Good Finite-Length Codes: From Art Towards Science European Transactions on Telecommunications 2006
-
From large scale rearrangements to mode coupling phenomenology in model glasses
PHYSICAL REVIEW LETTERS
2005; 94 (24)
View details for DOI 10.1103/PhysRevLett.94.247201
View details for Web of Science ID 000230092000053
- Why We Can Not Surpass Capacity: The Matching Condition 2005
- From Large Scale Rearrangements to Mode Coupling Phenomenology Physical Review Letters 2005
- How to Compute Loop Corrections to Bethe Approximation Journal of Statistical Mechanics 2005
- Tight bounds for LDPC and LDGM codes under MAP decoding IEEE Transactions on Information Theory 2005
- Belief Propagation Based Multi–User Detection 2005
-
Further results on finite-length scaling for iteratively decoded LDPC ensembles
IEEE International Symposium on Information Theory
IEEE. 2004: 103–103
View details for Web of Science ID 000223202600103
- Finite-Length Scaling and Finite-Length Shift for Low-Density Parity-Check Codes 2004
- Instability of one-step replica-symmetry-broken phase in satisfiability problems Journal of Physics A 2004
- The Phase Diagram of Random Heteropolymers Physical Review Letters 2004
- Glassy phases in Random Heteropolymers with correlated sequences Journal of Chemical Physics 2004
- On the cooling-schedule dependence of the dynamics of mean-field glasses Physical Review B 2004
- Life Above Threshold: From List Decoding to Area Theorem and MSE 2004
-
Maxwell's construction: The hidden bridge between maximum-likelihood and iterative decoding
IEEE International Symposium on Information Theory
IEEE. 2004: 225–225
View details for Web of Science ID 000223202600225
-
Weight distributions of LDPC code ensembles: Combinatorics meets statistical physics
IEEE International Symposium on Information Theory
IEEE. 2004: 102–102
View details for Web of Science ID 000223202600102
- Tight bounds for LDPC codes under MAP decoding 2004
- On the stochastic dynamics of disordered spin models Journal of Statistical Physics 2004
-
On the nature of the low-temperature phase in discontinuous mean-field spin glasses
EUROPEAN PHYSICAL JOURNAL B
2003; 33 (3): 339-346
View details for DOI 10.1140/epjb/e2003-00174-7
View details for Web of Science ID 000183776500012
- Finite-length scaling for iteratively decoded LDPC ensembles 2003
- A microscopic description of the aging dynamics: fluctuation-dissipation relations, effective temperature and heterogeneities Physical Review Letters 2003
-
Alternative translation techniques for propositional and first-order modal logics
JOURNAL OF AUTOMATED REASONING
2002; 28 (4): 397-415
View details for Web of Science ID 000176183400003
- The dynamic phase transition for decoding algorithms Physical Review E 2002; 66
- Optimizing searches via rare events Physical Review Letters 2002
- Finite-size scaling and metastable states of good codes 2001
- Statistical mechanics and turbo codes 2001
- Asymptotically free models and discrete non-Abelian groups Physics Letters B 2001
- Hairpin formation and elongation of biomolecules Physical Review Letters 2001
- Spin models on Platonic solids and asymptotic freedom 2001
- Discrete non-Abelian groups and asymptotically free models 2001
- The glassy phase of Gallager codes The European Physical Journal B 2001
-
A guided tour through some extensions of the event calculus
COMPUTATIONAL INTELLIGENCE
2000; 16 (2): 307-347
View details for Web of Science ID 000086463100007
- Turbo codes: the phase transition The European Physical Journal B 2000
- The statistical mechanics of turbo codes The European Physical Journal B 2000
- Operator product expansion on the lattice: a numerical test in the two-dimensional non-linear sigma-model Journal of High Energy Physics 2000
-
A general modal framework for the Event Calculus and its skeptical and credulous variants
JOURNAL OF LOGIC PROGRAMMING
1999; 38 (2): 111-164
View details for Web of Science ID 000077705000001
- Testing the efficiency of different improvement programs Nuclear Physics B 1999
- Composite operators from operator product expansion: what can go wrong? 1999
-
Event Calculus with explicit quantifiers
5th International Workshop on Temporal Representation and Reasoning
I E E E, COMPUTER SOC PRESS. 1998: 81–88
View details for Web of Science ID 000074225400013
- Operator product expansion and non-perturbative renormalization 1998
- Improved actions for the two-dimensional sigma-model 1997
-
COEVAL AR-40/AR-39 AGES OF 65.0 MILLION YEARS AGO FROM CHICXULUB CRATER MELT ROCK AND CRETACEOUS-TERTIARY BOUNDARY TEKTITES
SCIENCE
1992; 257 (5072): 954-958
Abstract
(40)Ar/(39)Ar dating of drill core samples of a glassy melt rock recovered from beneath a massive impact breccia contained within the 180-kilometer subsurface Chicxulub crater in Yucatán, Mexico, has yielded well-behaved incremental heating spectra with a mean plateau age of 64.98 +/- 0.05 million years ago (Ma). The glassy melt rock of andesitic composition was obtained from core 9 (1390 to 1393 meters) in the Chicxulub 1 well. The age of the melt rock is virtually indistinguishable from (40)Ar/(39)Ar ages obtained on tektite glass from Beloc, Haiti, and Arroyo el Mimbral, northeastern Mexico, of 65.01 +/- 0.08 Ma (mean plateau age for Beloc) and 65.07 +/- 0.10 Ma (mean total fusion age for both sites). The (40)Ar/(39)Ar ages, in conjunction with geochemical and petrological similarities, strengthen the recent suggestion that the Chicxulub structure is the source for the Haitian and Mexican tektites and is a viable candidate for the Cretaceous-Tertiary boundary impact site.
View details for Web of Science ID A1992JH82700032
View details for PubMedID 17789640