Learning Predictive Representations from Incomplete Data

点击翻译按钮获取中文摘要

Almost all data is now digitally stored, with unprecedented computing power available to process it. These developments have created new opportunities to advance computer interpretation (e.g. natural language processing, computer perception) and intelligent data-analysis (e.g. bio-informatics) by analyzing the massive amounts of complex data stored in text, multimedia, and scientific repositories. My research addresses the fundamental challenge of synthesizing predictive models from data, focusing on the problem of learning predictors in the presence of latent variables and incomplete observations. These challenges are heightened by the fact that predictions in complex domains are not just simple class labels or scalar values, but are complex structured outputs---such as parse trees, scene labellings or graph labellings---that involve multiple outputs to be predicted in a coordinated fashion, usually with intervening latent variables. The key problem is training complex predictors when some of the output or intervening latent variables are unobserved. To tackle these problems, I will develop convex training principles that combine model optimization with inference of missing components. Convexity decouples parameter optimization from model quality: a poor result arises from poor modeling choices, not a poor local minimum---thus separating specification from implementation. A key insight is that training with incomplete data can be tackled by treating missing components as auxiliary variables to be optimized (i.e. inferred) simultaneously with parameter optimization. Convex formulations of joint training and inference can then be obtained by one of two strategies that I have been developing: working with relaxed equivalence relations over missing components, or deriving implicitly induced regularizers. These approaches have already led to fundamental advances in unsupervised and semi-supervised training, including state of the art methods for dimensionality reduction, robust estimation, and latent large margin models. One of my long term goals is to commoditize solution methods for challeng- ing machine learning formulations, such as predictive representation learning and data component discovery.