RI: Small: Efficient Learning Algorithms for Modeling Natural Data

能够准确地模拟诸如照片或电影中固有的自然场景统计数据，或大脑皮层中数百个同时记录的神经元的集体行为等复杂现象，将对我们理解自然世界和人类思维产生变革性影响。 所获得的见解不仅会增强我们对大脑及其所能处理的感官刺激的理解，而且还会带来实际优势-例如，导致自动语音识别和实时视频有意义分析的改进。 这些研究所需的各种数据正在快速上线，但这些庞大而复杂的数据集无视传统的建模和分析技术。 不幸的是，生物学、物理学和工程学领域的许多最近获得的语料库的复杂性和规模使得它们无法被强大的数学模型拟合，除非它们受到关于数据的强有力和不合理的假设的约束。 这一点，再加上开发通用机器学习算法的普遍困难，促使大多数当代科学家和工程师专注于为狭窄的问题空间量身定制的算法，而不是解决更一般的机器学习问题。 幸运的是，一些研究人员继续推动具有更类似于人类智能的能力的通用学习算法，但他们通常不得不依赖于临时假设或不受控制的近似，以便在这个令人生畏的问题上取得进展。 这个建议是为了进一步发展最近引入的机器学习技术，称为最小概率流学习，以便它能够将非常一般的参数模型拟合到比以往任何时候都可能的更大的数据集。 此外，这项建议是开发新的，互补的方法，有效地从模型分布的采样，一旦参数已经适合数据，使模型可以理解和有意义的相互比较。 这些技术将用于研究自然场景的统计结构，方法是将一个新的强大的数学模型与由大量照片组成的数据库相匹配。 这里提出的计划是高度跨学科的，从物理学，工程学，计算机科学和系统神经科学中汲取思想和方法。

The ability to accurately model such complex phenomena as the natural scene statistics inherent in stacks of photographs or movies, or the collective behavior of hundreds of simultaneously recorded neurons in the cerebral cortex, would be transformative for our understanding of the natural world and of human thought. The insights gained would not only enhance our understanding of the brain and the sensory stimuli it can process, but they would confer practical advantages as well -- leading to improvements in automated speech recognition and meaningful analysis of real-time video, for example. The various data needed for these studies is coming online at a rapid pace, but these large and complex data sets defy traditional modeling and analysis techniques. Unfortunately, the complexity and size of many recently acquired corpora in biology, physics, and engineering domains render them incapable of being fit by powerful mathematical models unless they are constrained by strong and unjustified assumptions about the data. This, coupled with the general difficulty of developing general purpose machine learning algorithms has driven most contemporary scientists and engineers to focus on algorithms tailored to narrow problem spaces rather than tackling the more general machine learning problem. Fortunately, some researchers have continued to push for general learning algorithms with capabilities more similar to human intelligence, but they have typically had to rely on ad hoc assumptions or uncontrolled approximations in order to make progress on this daunting problem. This proposal is to further develop a recently introduced machine learning technique, called Minimum Probability Flow learning, so that it is capable of fitting exceedingly general parametric models to much larger data sets than has ever been possible before. In addition, this proposal is to develop novel, complimentary methods for sampling efficiently from a model distribution once the parameters have been fit to data, so that the models can be understood and meaningfully compared with one another. These techniques will be used to study the statistical structure of natural scenes by fitting a new and powerful mathematical model to a database consisting of a large number of photographs. The program proposed here is highly interdisciplinary, drawing ideas and approaches from physics, engineering, computer science, and systems neuroscience.