Leveraging structure in privileged information

点击翻译按钮获取中文摘要

This research program will focus on leveraging privileged information to develop new algorithms for learning (parameter estimation and probabilistic inference) and decision making with deep generative models. There are three key objectives to this research program:  Privileged information from large stores of text data: Text data is a rich store of contextual information about many different predictive tasks. Clinical textbooks contain vital information about the manifestation of the disease that can inform predictive models of disease onset. We will develop new learning algorithms for deep generative models that leverage context about predictive tasks presented in text data at training time. We will use neural network based language models to learn which aspects of text data are most helpful to solve a predictive problem at hand. Privileged information in time-series data: Predicting events that occur in the future via survival analysis is a powerful technique used in fields like economics and healthcare, where the goal could be identifying when a patient will develop a complication. But between a patient's baseline data and an event of interest, lie data that characterize the patient's physiological state over time. We will develop new hierarchical latent variable models that uncover structure in time-series privileged information and use this structure to improve the predictive quality and sample complexity of survival analysis models. Privileged information in causal graphs: Latent variable deep generative models use inference networks in the inner loop of learning to predict parameters of the variational distribution. This prediction problem is challenging when data are missing, and little is known about the bias incurred in the variational parameters when inference networks make predictions from missing data. We will study how causal graphs, which characterize the causal relationships among the random variables, can serve as privileged information for inference networks and be used to improve variational inference (and consequently parameter estimation) in deep generative models. This program will apply the algorithms developed in this research program to predictive problems that arise in the natural sciences and engineering including (but not limited to) economics, social sciences and healthcare. The design of sample efficient predictive models can open the door to incorporating emerging data modalities such as proteomics in computational biology. Hierarchical models that can cluster longitudinal, high-dimensional data with time-varying interventions can find patterns of human driving behavior using data from self-driving cars. Finally, the utilization of causal knowledge (in the form of graphs) can lead to novel unsupervised learning algorithms. In summary, this research agenda will open up the utility of machine learning to domains where data is scarce and new problems where labels are expensive to obtain.