Learning Predictive Representations from Incomplete Data

从不完整的数据中学习预测表示

• 批准号: 217337-2013
• 负责人: Schuurmans, Dale
• 金额: $ 4.52万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2017
• 资助国家: 加拿大
• 起止时间: 2017-01-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=635097
• 关键词: Learning; Predictive; Representations; Incomplete; Data

Almost all data is now digitally stored, with unprecedented computing power available to process it. These developments have created new opportunities to advance computer interpretation (e.g. natural language processing, computer perception) and intelligent data-analysis (e.g. bio-informatics) by analyzing the massive amounts of complex data stored in text, multimedia, and scientific repositories. My research addresses the fundamental challenge of synthesizing predictive models from data, focusing on the problem of learning predictors in the presence of latent variables and incomplete observations. These challenges are heightened by the fact that predictions in complex domains are not just simple class labels or scalar values, but are complex structured outputs---such as parse trees, scene labellings or graph labellings---that involve multiple outputs to be predicted in a coordinated fashion, usually with intervening latent variables. The key problem is training complex predictors when some of the output or intervening latent variables are unobserved. To tackle these problems, I will develop convex training principles that combine model optimization with inference of missing components. Convexity decouples parameter optimization from model quality: a poor result arises from poor modeling choices, not a poor local minimum---thus separating specification from implementation. A key insight is that training with incomplete data can be tackled by treating missing components as auxiliary variables to be optimized (i.e. inferred) simultaneously with parameter optimization. Convex formulations of joint training and inference can then be obtained by one of two strategies that I have been developing: working with relaxed equivalence relations over missing components, or deriving implicitly induced regularizers. These approaches have already led to fundamental advances in unsupervised and semi-supervised training, including state of the art methods for dimensionality reduction, robust estimation, and latent large margin models. One of my long term goals is to commoditize solution methods for challeng- ing machine learning formulations, such as predictive representation learning and data component discovery.

现在几乎所有的数据都以数字方式存储，拥有前所未有的计算能力来处理这些数据。这些发展为通过分析存储在文本、多媒体和科学知识库中的大量复杂数据来推进计算机解释（例如自然语言处理、计算机感知）和智能数据分析（例如生物信息学）创造了新的机会。我的研究解决了从数据中合成预测模型的根本挑战，重点是在存在潜在变量和不完整观察的情况下学习预测器的问题。复杂域中的预测不仅仅是简单的类标签或标量值，而且是复杂的结构化输出-例如解析树，场景标签或图形标签-涉及以协调方式预测的多个输出，通常具有介入的潜在变量。关键问题是当一些输出或干预潜变量未被观察时训练复杂的预测器。为了解决这些问题，我将开发将联合收割机模型优化与缺失组件推理相结合的凸训练原则。Convexity将参数优化与模型质量相结合：糟糕的结果来自糟糕的建模选择，而不是糟糕的局部最小值-从而将规范与实现分离。一个关键的见解是，不完整数据的训练可以通过将缺失的组件视为与参数优化同时优化（即推断）的辅助变量来解决。然后，联合训练和推理的凸公式可以通过我一直在开发的两种策略之一来获得：在缺失的组件上使用松弛的等价关系，或者导出隐式诱导正则化器。这些方法已经在无监督和半监督训练方面取得了根本性的进展，包括最先进的降维方法，鲁棒估计和潜在的大边际模型。我的长期目标之一是将用于验证机器学习公式的解决方案方法商品化，例如预测表示学习和数据组件发现。