From Boltzmann Machines to Neural Networks and Back Again
From Boltzmann Machines to Neural Networks and Back Again
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Surbhi Goel;Adam R. Klivans;Frederic Koehler
中科院分区：
文献类型：
--
作者：
Surbhi Goel;Adam R. Klivans;Frederic Koehler
图形模型是对高维数据建模的有力工具，但在存在潜在变量的情况下学习图形模型是众所周知的困难。在这项工作中，我们给出了新的结果学习限制玻尔兹曼机，可能是最好的研究类的潜变量模型。我们的研究结果是基于新的连接学习两层神经网络下$\ell_{\infty}$有界输入，对于这两个问题，我们给出了几乎最优的结果下的稀疏奇偶噪声的约束硬度。利用RBM和前馈网络之间的联系，我们还启动了$supervised~ RBM $ [欣顿，2012]的理论研究，这是一种神经网络学习，将从底层图形模型中导出的分布假设与未知函数类的架构相结合。然后，我们给出了一个算法，学习一个自然类的监督RBM更好的运行时间比它的相关类的网络没有分布的假设。
Graphical models are powerful tools for modeling high-dimensional data, but learning graphical models in the presence of latent variables is well-known to be difficult. In this work we give new results for learning Restricted Boltzmann Machines, probably the most well-studied class of latent variable models. Our results are based on new connections to learning two-layer neural networks under $\ell_{\infty}$ bounded input; for both problems, we give nearly optimal results under the conjectured hardness of sparse parity with noise. Using the connection between RBMs and feedforward networks, we also initiate the theoretical study of $supervised~RBMs$ [Hinton, 2012], a version of neural-network learning that couples distributional assumptions induced from the underlying graphical model with the architecture of the unknown function class. We then give an algorithm for learning a natural class of supervised RBMs with better runtime than what is possible for its related class of networks without distributional assumptions.