Leveraging structure in privileged information

该研究计划将专注于利用特权信息开发新的学习算法（参数估计和概率推理）和使用深度生成模型进行决策。 这个研究项目有三个关键目标：从大量的文本数据中提取信息：文本数据是关于许多不同预测任务的丰富的上下文信息存储。临床教科书包含有关疾病表现的重要信息，可以为疾病发作的预测模型提供信息。我们将为深度生成模型开发新的学习算法，这些算法在训练时利用文本数据中呈现的预测任务的上下文。我们将使用基于神经网络的语言模型来学习文本数据的哪些方面最有助于解决手头的预测问题。 通过生存分析预测未来发生的事件是经济学和医疗保健等领域使用的强大技术，其目标可能是确定患者何时会出现并发症。但是在患者的基线数据和感兴趣的事件之间，存在表征患者随时间的生理状态的数据。我们将开发新的分层潜变量模型，揭示时间序列特权信息的结构，并使用这种结构来提高生存分析模型的预测质量和样本复杂性。 因果图中的隐藏信息：潜在变量深度生成模型在学习的内部循环中使用推理网络来预测变分分布的参数。当数据缺失时，这个预测问题是具有挑战性的，并且当推理网络从缺失数据进行预测时，对变分参数中产生的偏差知之甚少。我们将研究因果图如何表征随机变量之间的因果关系，可以作为推理网络的特权信息，并用于改进深度生成模型中的变分推理（以及随后的参数估计）。该计划将应用本研究计划中开发的算法来预测自然科学和工程中出现的问题，包括（但不限于）经济学，社会科学和医疗保健。设计样本有效的预测模型可以打开大门，将新兴的数据模式，如蛋白质组学在计算生物学。分层模型可以对纵向、高维数据进行聚类，并带有随时间变化的干预，可以使用自动驾驶汽车的数据来发现人类驾驶行为的模式。最后，利用因果知识（以图形的形式）可以产生新的无监督学习算法。总之，这项研究议程将开放机器学习的实用性，以解决数据稀缺的领域和标签获取成本高昂的新问题。

This research program will focus on leveraging privileged information to develop new algorithms for learning (parameter estimation and probabilistic inference) and decision making with deep generative models. There are three key objectives to this research program:  Privileged information from large stores of text data: Text data is a rich store of contextual information about many different predictive tasks. Clinical textbooks contain vital information about the manifestation of the disease that can inform predictive models of disease onset. We will develop new learning algorithms for deep generative models that leverage context about predictive tasks presented in text data at training time. We will use neural network based language models to learn which aspects of text data are most helpful to solve a predictive problem at hand. Privileged information in time-series data: Predicting events that occur in the future via survival analysis is a powerful technique used in fields like economics and healthcare, where the goal could be identifying when a patient will develop a complication. But between a patient's baseline data and an event of interest, lie data that characterize the patient's physiological state over time. We will develop new hierarchical latent variable models that uncover structure in time-series privileged information and use this structure to improve the predictive quality and sample complexity of survival analysis models. Privileged information in causal graphs: Latent variable deep generative models use inference networks in the inner loop of learning to predict parameters of the variational distribution. This prediction problem is challenging when data are missing, and little is known about the bias incurred in the variational parameters when inference networks make predictions from missing data. We will study how causal graphs, which characterize the causal relationships among the random variables, can serve as privileged information for inference networks and be used to improve variational inference (and consequently parameter estimation) in deep generative models. This program will apply the algorithms developed in this research program to predictive problems that arise in the natural sciences and engineering including (but not limited to) economics, social sciences and healthcare. The design of sample efficient predictive models can open the door to incorporating emerging data modalities such as proteomics in computational biology. Hierarchical models that can cluster longitudinal, high-dimensional data with time-varying interventions can find patterns of human driving behavior using data from self-driving cars. Finally, the utilization of causal knowledge (in the form of graphs) can lead to novel unsupervised learning algorithms. In summary, this research agenda will open up the utility of machine learning to domains where data is scarce and new problems where labels are expensive to obtain.