IIS: RI: Small: Nonlinear Dynamical System Theory for Machine Learning

IIS：RI：小型：机器学习的非线性动力系统理论

• 批准号: 1018433
• 负责人: Max Welling
• 金额: $ 45万
• 依托单位: University of California-Irvine
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-09-01 至 2014-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1018433&HistoricalAwards=false
• 关键词: IIS; RI; Small; Nonlinear; Dynamical

Learning complex statistical models from data is intractable for many models of interest. The PIs are studying a new approach to learning from data that formulates learning as a weakly chaotic nonlinear dynamical system. They show that this dynamical system, which they call ?herding?, combines learning and inference into one tractable forward mapping. They study the abstract mathematical properties of this nonlinear mapping, such as the properties of its attractor set and the topological and metric entropy of the mapping. They then relate these to properties of learning systems. The PIs apply herding systems to a wide range of applications in machine learning. In supervised learning they show that herding suggests a natural extension to the ?voted perceptron algorithm? by including hidden variables. In unsupervised learning, herding is used to train Markov random field models from data. Herding is also extended to Hilbert spaces where it naturally leads to a deterministic sampling algorithm. Due to negative autocorrelations, this ?kernel herding? generates samples that have superior convergence properties than random sampling. They also apply herding to active learning problems. Herding has the potential to radically transform the way we view learning systems. It connects learning to the vast field of nonlinear dynamical systems and chaos theory. As such the impact on machine learning is significant. Scientific results will be disseminated through journal publications and conference proceedings. The PIs also introduce a new course on learning, chaos and fractals to expose students to the intriguing connections between these fields.

从数据中学习复杂的统计模型对于许多感兴趣的模型来说是棘手的。PI正在研究一种从数据中学习的新方法，该方法将学习描述为弱混沌非线性动力系统。他们展示了这个动力系统，他们称之为？放牧？将学习和推理结合到一个易于处理的前向映射中。他们研究了这种非线性映射的抽象数学性质，如其吸引子集的性质以及映射的拓扑熵和度量熵。然后，他们将这些与学习系统的属性联系起来。PI将羊群系统应用于机器学习中的广泛应用。在监督学习，他们表明，羊群效应表明一个自然的延伸？投票感知器算法通过包含隐藏变量。在无监督学习中，羊群用于从数据中训练马尔可夫随机场模型。羊群也扩展到希尔伯特空间，它自然会导致一个确定性的采样算法。由于负相关性，这？核心羊群效应？生成比随机采样具有上级收敛特性的采样。他们还将羊群效应应用于主动学习问题。羊群效应有可能从根本上改变我们看待学习系统的方式。它将学习与非线性动力系统和混沌理论的广阔领域联系起来。因此，对机器学习的影响是巨大的。科学成果将通过期刊出版物和会议记录传播。PI还引入了一门关于学习，混沌和分形的新课程，让学生了解这些领域之间有趣的联系。