CAREER: Entropy Geometry in Variational Inference Signal Processing

过去的十年已经证明，近似贝叶斯推理的变分方法是解决许多信号处理问题的关键，这些问题以前被认为是根本困难的。适当变分近似贝叶斯推理技术的应用带来了重大突破的重要信号处理问题包括对噪声信道的容量逼近码的解码、欠定语音源分离和网络上的分布式估计。这些应用证明，变分推理技术已经彻底改变了信号处理的现状，因为它们能够在合理的复杂性和通信（相当于能耗）成本下提供高性能估计。然而，它们的性能与所需的复杂性和通信之间的权衡，导致它们广泛和不断增长的采用的现象，仍然没有完全理解。这个项目的基本论点是，在基于变分推理的信号处理中，熵信息几何是性能、复杂性和通信之间权衡的核心。正在进行的研究提高了我们对熵几何的理解，熵几何主要指示了协作估计算法的性能与其通信成本之间的基本关系。研究还揭示了变分推理信号处理技术的性能与复杂度之间的信息几何关系。总之，这些见解允许在网络上开发健壮的语音处理和协作估计算法，在可调的复杂性和通信成本下提供最佳性能。除了在这些领域的好处之外，这项工作还具有协同作用，因为它直接有助于创建高效的分布式变分推理信号处理算法的整体设计科学。

The past decade has proven that variational approaches to approximate Bayesian inference holdthe key to pragmatic solutions to many signal processing problems previously thought to be fundamentally difficult. Important signal processing problems where the application of appropriatevariational approximate Bayesian inference techniques led to significant breakthroughs include the decoding of capacity approaching codes for noisy channels, underdetermined speech source separation, and distributed estimation over networks. These applications provide evidence that variational inference techniques have revolutionized the state of the art in signal processing because of their ability to provide high performance estimates at reasonable complexity and communication (equivalently, energy consumption) costs. However, the tradeoff between their performance and required complexity and communication, the very phenomenon leading to their widespread and growing adoption, remains incompletely understood.The underlying thesis of this project is that entropy & information geometry lies at the heartof the tradeo^ff between performance, complexity, and communication in variational inference basedsignal processing. The research being performed improves our understanding of entropy geometry,which primarily dictates the fundamental relationship between the performance of a collaborativeestimation algorithm and its communication cost. This research also develops the information geometric relationship between a variational inference signal processing technique's performance andcomplexity. Together, these insights allow algorithms for robust speech processing and collaborative estimation over networks to be developed that provide optimal performance at a tunable complexity and communication cost. In addition to its benefits in each of these areas, the work has a synergistic aspect in that it contributes directly to the creation of an overall design science forefficient distributed variational inference signal processing algorithms.