Learning Predictive Representations from Incomplete Data

现在几乎所有的数据都是数字化存储的，拥有前所未有的计算能力来处理这些数据。这些发展通过分析存储在文本、多媒体和科学知识库中的大量复杂数据，为推进计算机解释（如自然语言处理、计算机感知）和智能数据分析（如生物信息学）创造了新的机会。我的研究解决了从数据中合成预测模型的基本挑战，重点是在潜在变量和不完整观察存在的情况下学习预测器的问题。复杂领域的预测不仅仅是简单的类标签或标量值，而是复杂的结构化输出——如解析树、场景标记或图形标记——涉及以协调的方式预测的多个输出，通常带有干预的潜在变量，这一事实加剧了这些挑战。关键问题是训练复杂的预测器时，一些输出或干预潜在变量是未观察到的。为了解决这些问题，我将开发将模型优化与缺失组件推理相结合的凸训练原理。凸性将参数优化与模型质量解耦：糟糕的结果源于糟糕的建模选择，而不是糟糕的局部最小值——从而将规范与实现分离。一个关键的见解是，不完整数据的训练可以通过将缺失的组件作为辅助变量来处理，与参数优化同时进行优化（即推断）。然后，联合训练和推理的凸公式可以通过我一直在开发的两种策略之一来获得：在缺失组件上使用宽松的等价关系，或者推导隐式诱导正则器。这些方法已经导致了无监督和半监督训练的基本进展，包括最先进的降维方法、鲁棒估计和潜在大边际模型。我的长期目标之一是将具有挑战性的机器学习公式的解决方案方法商品化，例如预测表示学习和数据组件发现。

Almost all data is now digitally stored, with unprecedented computing power available to process it. These developments have created new opportunities to advance computer interpretation (e.g. natural language processing, computer perception) and intelligent data-analysis (e.g. bio-informatics) by analyzing the massive amounts of complex data stored in text, multimedia, and scientific repositories. My research addresses the fundamental challenge of synthesizing predictive models from data, focusing on the problem of learning predictors in the presence of latent variables and incomplete observations. These challenges are heightened by the fact that predictions in complex domains are not just simple class labels or scalar values, but are complex structured outputs---such as parse trees, scene labellings or graph labellings---that involve multiple outputs to be predicted in a coordinated fashion, usually with intervening latent variables. The key problem is training complex predictors when some of the output or intervening latent variables are unobserved. To tackle these problems, I will develop convex training principles that combine model optimization with inference of missing components. Convexity decouples parameter optimization from model quality: a poor result arises from poor modeling choices, not a poor local minimum---thus separating specification from implementation. A key insight is that training with incomplete data can be tackled by treating missing components as auxiliary variables to be optimized (i.e. inferred) simultaneously with parameter optimization. Convex formulations of joint training and inference can then be obtained by one of two strategies that I have been developing: working with relaxed equivalence relations over missing components, or deriving implicitly induced regularizers. These approaches have already led to fundamental advances in unsupervised and semi-supervised training, including state of the art methods for dimensionality reduction, robust estimation, and latent large margin models. One of my long term goals is to commoditize solution methods for challeng- ing machine learning formulations, such as predictive representation learning and data component discovery.