In this setting, the instances are in r d, and the labels are from. Asymptotic mmse analysis under sparse representation. Donoho y, adel javanmard and andrea montanari december 3, 2011 abstract we study the compressed sensing reconstruction problem for a. Informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing authors. At the gathering, we will be entertained by a 1530 minute mini chalktalk on interesting topics in ml or related fields. Amp is currently acknowledged as one of the foremost algorithms for such problems in terms of both its computational ef.
Efficient active learning of sparse halfspaces deepai. One of the main goal in compressed sensing is the recovery process. Statisticalphysicsbased reconstruction in compressed. Citeseerx document details isaac councill, lee giles, pradeep teregowda. Verificationbased messagepassing algorithms vbmpas in compressed sensing cs, a branch of digital signal processing that deals with measuring sparse signals, are some methods to efficiently solve the recovery problem in compressed sensing. Dmm09 in the setting of compressed sensing, based on ideas from bol12. Informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing dl donoho, a javanmard, a montanari information theory. R d, the class of homogeneous linear classifiers, to predict labels from instances. Machine learning tea is a weekly informal gathering for researchers in machine learning and those in the related fields such as computational neuroscience, ai, statistics, etc. Using an interdisciplinary approach, we have recently proposed in arxiv.
Phase retrieval using untrained neural network priors. Motivated by this puzzling remark, wu and verdu 15 introduced a shannontheoretic. Informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing abstract. Information theoretically optimal compressed sensing via spatial coupling and approximate message passing by david l. Information theoretically optimal compressed sensing via spatial coupling and approximate message passing dl donoho, a javanmard, a montanari information theory, ieee transactions on 59 11, 74347464, 20. We propose a novel spectralbased estimation procedure and show that. Computational hardness of fast rates for online sparse pca. A core tenet of signal processing and information theory is that signals.
As demonstrated heuristically and numerically by krzakala et al. Donoho, adel javanmard, andrea montanari mathematics, computer science, physics. We establish rigorously the intrinsic information theoretic limitations of inference and learning for a class of randomly generated instances. Outline the remainder of the paper is organized as follows. A note on optimal support recovery in compressed sensing. Some previous works on compressed sensing have used these ingredients separately. We will focus on the rademacherspiked wigner model. Pdf approximate messagepassing with spatially coupled. Donoho, adel javanmard and andrea montanari abstract. Informationtheoretically optimal compressed sensing via spatial.
The signsketch procedure proposed here provides an af. Blind calibration in compressed sensing using message passing. Donoho y, adel javanmard and andrea montanari december 3, 2011 abstract we study the compressed sensing reconstruction problem for a broad class of random, banddiagonal sensing matrices. Sparse statistics, optimization and machine learning.
In this paper, bayesian compressive sensing algorithm is studied. Balancing fundamental theory with engineering applications, it describes the mechanism and limits for the representation and communication of information using electromagnetic waves. Furthermore, in the case of 1bit compressed sensing, we require the number of measurements x. This paper proposes to use preconditioning to modified the sensing matrix such that it satisfies the assumptions of the original amp algorithm. They arise in signal processing, statistical inference, machine learning, communication theory, and other fields. From this point of view, we can encode all the information about xin a single real number y2r i. We establish rigorously the intrinsic informationtheoretic limitations of inference and learning for a class of randomly generated instances. Informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing david l. Taking an informationtheoretic viewpoint, wu and verd.
On the recovery conditions for practical ghost imaging. Montanariinformationtheoretically optimal compressed sensing via spatial coupling and approximate message passing ieee trans. Optimal errors and phase transitions in highdimensional. Optimal linear estimation under unknown nonlinear transform. It has been shown recently 3, 4 that for compressed sensing of sparse signals with known empirical distribution of components the theoretically optimal reconstruction can be achieved with the combined use of gamp algorithm 5, 6 and seeding spatially coupled measurement matrices. Verificationbased messagepassing algorithms in compressed. Informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing article pdf available in ieee transactions on information theory 5911 december 2011. One example is the noiseless onebit compressed sensing model defined in, for which we have fz signz. In particular, adaptations of belief propagation have been developed for the compressedsensing reconstruction, both in the context of 1 reconstruction 11,22,23 and in a probabilistic approach 24. This construction is inspired by the idea of spatial.
Montanariinformation theoretically optimal compressed sensing via spatial coupling and approximate message passing ieee trans. Osa on the recovery conditions for practical ghost imaging. Subsampling at information theoretically optimal rates. Donoho, adel javanmard, andrea montanari submitted on 4 dec 2011 v1, last revised 19 jan 20 this version, v2. There are two modeling approaches that can be considered.
Analogous to onebit compressed sensing, we consider a new model named onebit phase retrieval where the linear measurement with phase information erased is quantized to one bit. Oribc theory we know that the socalled central algorithm is optimal. Phase transitions, optimal errors and optimality of message. Phase transitions, optimal errors and optimality of messagepassing in generalized linear models. Phase transitions, optimal errors and optimality of.
Swept approximate message passing for sparse estimation. We evaluate the mutual information or free entropy from which we deduce the bayesoptimal inference and generalization errors. Informationtheoretically optimal compressed sensing. Doa estimation based on bayesian compressive sensing. The approximate message passing algorithm amp is introduced for practical ghost imaging gi. Key background information and clear definitions make this an ideal resource for. Informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing.
Sparse statistics, optimization and machine learning alexandre daspremont princeton university, francis bach ensinria, martin wainwright u. Performance regions in compressed sensing from noisy. Informationtheoretically optimal compressed sensinguniversity of southern california usc, jan 2014 invited talkinformation theory forum, stanford, 20asilomar conference on signals, systems, and computers, 2012 invited talk. We show information theoretically optimal sample complexity2 for structured. Donoho, adel javanmard, and andrea montanari, \ information theoretically optimal com pressed sensing via spatial coupling and approximate message passing, proc.
Performance regions in compressed sensing from noisy measurements junan zhu and dror baron department of electrical and computer engineering north carolina state university. To deal with signals with multiple snapshots, we extend traditional bayesian algorithm under the condition of single snapshot to multisnapshot bayesian compressed sensing mbcs algorithm and apply mbcs algorithm to direction of arrival doa estimation of narrowband signals and wideband signals. Optimal errors and phase transitions in highdimensional generalized linear models jean barbiera,b,1,2. Informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing author. The challenge in both cases is to design computationally ef. Statisticalphysicsbased reconstruction in compressed sensing. Messagepassing algorithms for synchronization problems alex wein mit mathematics with amelia perry, afonso bandeira, and ankur moitra. Furthermore, this paper points out that the speckles used in practical gi system are spatially correlated, and this will degrade the imaging performance. In this paper we introduce a new theory for distributed compressed sensing.
Informationtheoretically optimal compressed sensing via. We study compressed sensing of real and complex sparse signals using an optimal and computationally efficient procedure based on the combined use of approximate messagepassing and spatially. Montanari, informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing, ieee transactions on inf. Contribution using spatial coupling and approximate message passing, the approaches of 7, 4 allow successful compressed. An informationtheoretic approach to distributed compressed. Abstractwe study the compressed sensing reconstruction. There have been many exciting works on active halfspace learning in the literature. Learning and 1bit compressed sensing under asymmetric. A note on optimal support recovery in compressed sensing galen reeves and michael gastpar department of electrical engineering and computer sciences university of california, berkeley email. Sparsity assumptions for inverting quadratic or magnitudeonly equations of the form 1 has similarly helped lower computational.
We show information theoretically optimal sample complexity2 for. A random matrix approach to neural networks louart, cosme, liao, zhenyu, and couillet, romain, the annals of applied probability, 2018. Adel javanmard with david donoho and andrea montanari created date. Wave theory of information by massimo franceschetti. Statisticalphysicsbased reconstruction in compressed sensing f.
Joseph, least squares superposition codes of moderate dictionary size are reliable at rates up to capacity, ieee trans. Trained generative priors have successfully cirvumvented this problem 5, 6, 21, 22, 7. Mar 19, 2019 highdimensional generalized linear models are basic building blocks of current data analysis tools including multilayers neural networks. Informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing dl donoho, a javanmard, a montanari ieee transaction on information theory 59. Donoho, adel javanmard, and andrea montanari, \informationtheoretically optimal com pressed sensing via spatial coupling and approximate message passing, proc.
Learning and 1bit compressed sensing under asymmetric noise. This construction is inspired by the idea of spatial coupling in coding theory. Identical fixed points in state evolutions of amp and vamp. Gaps in vn mod 1 and ergodic theory elkies, noam d. Asymptotic mmse analysis under sparse representation modeling. Compressed sensing is a signal processing technique in which data is acquired directly in a compressed form. Montanari, informationtheoretically optimal compressed sensing via spatial coupling and approximate message passing, in proceedings of ieee international symposium on information theory proceedings ieee, 2012, 12311235. The results of 7 were based on statistical mechanics methods and numerical simulations. Highdimensional generalized linear models are basic building blocks of current data analysis tools including multilayers neural networks.
Generally speaking, recovery process in compressed sensing is a method by which the. We study the compressed sensing reconstruction problem for a broad class of random, banddiagonal sensing matrices. Understand the relationship between information theory and the physics of wave propagation with this expert guide. Cambridge core general and classical physics wave theory of information by massimo franceschetti. Efficient active halfspace learning algorithms that work under different. This is a surprising result, given that standard compressed sensing methods achieve successful recovery from klognk measurements. Towards sampleoptimal methods for solving random quadratic. The workshop was designed to bring together scientists from three disciplines. Pdf compressed sensing is an exciting, rapidly growing field, attracting considerable. Information theoretically optimal compressed sensing via spatial coupling and approximate message passing dl donoho, a javanmard, a montanari ieee transaction on information theory 59 11, 74347464, 20. A core tenet o f signal processing and information theory is that signals, images, and. Abstract we study the compressed sensing reconstruction problem for a broad class of random, banddiagonal sensing matrices. Information theoretically optimal compressed sensing via spatial coupling and approximate message passing authors.
140 729 673 550 475 1339 534 646 1323 938 1516 509 1544 946 1405 373 747 1465 346 411 1068 1246 1169 965 757 312 1436 57 1408 1025 559 360 1159 268 902 1468