Learning Predictive Representations from Incomplete Data

现在，几乎所有的数据都以数字方式存储，拥有前所未有的计算能力来处理这些数据。这些发展创造了新的机会，通过分析存储在文本、多媒体和科学资源库中的大量复杂数据来推进计算机解释(例如自然语言处理、计算机感知)和智能数据分析(例如生物信息学)。我的研究解决了从数据中合成预测模型的根本挑战，重点是在存在潜在变量和不完全观测的情况下学习预测者的问题。复杂领域中的预测不仅是简单的类别标签或标量值，而且是复杂的结构化输出-如分析树、场景标签或图标签--这一事实加剧了这些挑战，这些输出涉及以协调方式预测的多个输出，通常带有介入的潜在变量。关键问题是当一些输出或干预的潜在变量未被观察到时，训练复杂的预测者。 为了解决这些问题，我将制定凸训练原则，将模型优化与缺失组件的推断相结合。凸性将参数优化与模型质量分离：糟糕的结果来自糟糕的建模选择，而不是糟糕的局部最小值-从而将规范与实现分开。一个关键的见解是，数据不完整的训练可以通过将缺失的分量视为辅助变量来处理，以便在参数优化的同时进行优化(即推断)。然后，联合训练和推理的凸公式可以通过我一直在开发的两种策略之一获得：在缺失组件上处理松散的等价关系，或者导出隐式诱导的正则化。这些方法已经在无监督和半监督训练方面取得了根本性的进展，包括最新的降维方法、稳健估计和潜在的大间隔模型。我的长期目标之一是将用于挑战机器学习公式的解决方案方法商品化，例如预测表示学习和数据组件发现。

Almost all data is now digitally stored, with unprecedented computing power available to process it. These developments have created new opportunities to advance computer interpretation (e.g. natural language processing, computer perception) and intelligent data-analysis (e.g. bio-informatics) by analyzing the massive amounts of complex data stored in text, multimedia, and scientific repositories. My research addresses the fundamental challenge of synthesizing predictive models from data, focusing on the problem of learning predictors in the presence of latent variables and incomplete observations. These challenges are heightened by the fact that predictions in complex domains are not just simple class labels or scalar values, but are complex structured outputs---such as parse trees, scene labellings or graph labellings---that involve multiple outputs to be predicted in a coordinated fashion, usually with intervening latent variables. The key problem is training complex predictors when some of the output or intervening latent variables are unobserved. To tackle these problems, I will develop convex training principles that combine model optimization with inference of missing components. Convexity decouples parameter optimization from model quality: a poor result arises from poor modeling choices, not a poor local minimum---thus separating specification from implementation. A key insight is that training with incomplete data can be tackled by treating missing components as auxiliary variables to be optimized (i.e. inferred) simultaneously with parameter optimization. Convex formulations of joint training and inference can then be obtained by one of two strategies that I have been developing: working with relaxed equivalence relations over missing components, or deriving implicitly induced regularizers. These approaches have already led to fundamental advances in unsupervised and semi-supervised training, including state of the art methods for dimensionality reduction, robust estimation, and latent large margin models. One of my long term goals is to commoditize solution methods for challeng- ing machine learning formulations, such as predictive representation learning and data component discovery.