CAREER: Entropy Geometry in Variational Inference Signal Processing

点击翻译按钮获取中文摘要

The past decade has proven that variational approaches to approximate Bayesian inference holdthe key to pragmatic solutions to many signal processing problems previously thought to be fundamentally difficult. Important signal processing problems where the application of appropriatevariational approximate Bayesian inference techniques led to significant breakthroughs include the decoding of capacity approaching codes for noisy channels, underdetermined speech source separation, and distributed estimation over networks. These applications provide evidence that variational inference techniques have revolutionized the state of the art in signal processing because of their ability to provide high performance estimates at reasonable complexity and communication (equivalently, energy consumption) costs. However, the tradeoff between their performance and required complexity and communication, the very phenomenon leading to their widespread and growing adoption, remains incompletely understood.The underlying thesis of this project is that entropy & information geometry lies at the heartof the tradeo^ff between performance, complexity, and communication in variational inference basedsignal processing. The research being performed improves our understanding of entropy geometry,which primarily dictates the fundamental relationship between the performance of a collaborativeestimation algorithm and its communication cost. This research also develops the information geometric relationship between a variational inference signal processing technique's performance andcomplexity. Together, these insights allow algorithms for robust speech processing and collaborative estimation over networks to be developed that provide optimal performance at a tunable complexity and communication cost. In addition to its benefits in each of these areas, the work has a synergistic aspect in that it contributes directly to the creation of an overall design science forefficient distributed variational inference signal processing algorithms.