IIS: RI: Small: Nonlinear Dynamical System Theory for Machine Learning

对于许多感兴趣的模型来说，从数据中学习复杂的统计模型是难以处理的。pi正在研究一种从数据中学习的新方法，该方法将学习表述为弱混沌非线性动力系统。他们展示了这种动力系统，他们称之为“羊群”将学习和推理结合到一个可处理的前向映射中。他们研究了这种非线性映射的抽象数学性质，例如它的吸引子集的性质以及映射的拓扑熵和度量熵。然后他们将这些与学习系统的属性联系起来。pi将羊群系统应用于机器学习中的广泛应用。在监督式学习中，他们表明群居表明了对大脑的自然延伸。投票感知器算法？通过包含隐藏变量。在无监督学习中，羊群被用于从数据中训练马尔可夫随机场模型。羊群也被扩展到希尔伯特空间，在那里它自然地导致一个确定性的抽样算法。由于负自相关，这？内核放牧吗?生成比随机抽样具有优越收敛特性的样本。他们还将羊群效应应用于主动学习问题。羊群效应有可能从根本上改变我们看待学习系统的方式。它将学习与非线性动力系统和混沌理论的广阔领域联系起来。因此，它对机器学习的影响是巨大的。科学成果将通过期刊出版物和会议记录进行传播。pi还引入了一门关于学习、混沌和分形的新课程，让学生们了解这些领域之间有趣的联系。

Learning complex statistical models from data is intractable for many models of interest. The PIs are studying a new approach to learning from data that formulates learning as a weakly chaotic nonlinear dynamical system. They show that this dynamical system, which they call ?herding?, combines learning and inference into one tractable forward mapping. They study the abstract mathematical properties of this nonlinear mapping, such as the properties of its attractor set and the topological and metric entropy of the mapping. They then relate these to properties of learning systems. The PIs apply herding systems to a wide range of applications in machine learning. In supervised learning they show that herding suggests a natural extension to the ?voted perceptron algorithm? by including hidden variables. In unsupervised learning, herding is used to train Markov random field models from data. Herding is also extended to Hilbert spaces where it naturally leads to a deterministic sampling algorithm. Due to negative autocorrelations, this ?kernel herding? generates samples that have superior convergence properties than random sampling. They also apply herding to active learning problems. Herding has the potential to radically transform the way we view learning systems. It connects learning to the vast field of nonlinear dynamical systems and chaos theory. As such the impact on machine learning is significant. Scientific results will be disseminated through journal publications and conference proceedings. The PIs also introduce a new course on learning, chaos and fractals to expose students to the intriguing connections between these fields.