Leveraging structure in privileged information

利用特权信息的结构

• 批准号: RGPIN-2022-04546
• 负责人: Gopalkrishnan, Rahul
• 金额: $ 1.82万
• 依托单位: University of Toronto
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750037
• 关键词: Leveraging; structure; privileged; information

This research program will focus on leveraging privileged information to develop new algorithms for learning (parameter estimation and probabilistic inference) and decision making with deep generative models. There are three key objectives to this research program:  Privileged information from large stores of text data: Text data is a rich store of contextual information about many different predictive tasks. Clinical textbooks contain vital information about the manifestation of the disease that can inform predictive models of disease onset. We will develop new learning algorithms for deep generative models that leverage context about predictive tasks presented in text data at training time. We will use neural network based language models to learn which aspects of text data are most helpful to solve a predictive problem at hand. Privileged information in time-series data: Predicting events that occur in the future via survival analysis is a powerful technique used in fields like economics and healthcare, where the goal could be identifying when a patient will develop a complication. But between a patient's baseline data and an event of interest, lie data that characterize the patient's physiological state over time. We will develop new hierarchical latent variable models that uncover structure in time-series privileged information and use this structure to improve the predictive quality and sample complexity of survival analysis models. Privileged information in causal graphs: Latent variable deep generative models use inference networks in the inner loop of learning to predict parameters of the variational distribution. This prediction problem is challenging when data are missing, and little is known about the bias incurred in the variational parameters when inference networks make predictions from missing data. We will study how causal graphs, which characterize the causal relationships among the random variables, can serve as privileged information for inference networks and be used to improve variational inference (and consequently parameter estimation) in deep generative models. This program will apply the algorithms developed in this research program to predictive problems that arise in the natural sciences and engineering including (but not limited to) economics, social sciences and healthcare. The design of sample efficient predictive models can open the door to incorporating emerging data modalities such as proteomics in computational biology. Hierarchical models that can cluster longitudinal, high-dimensional data with time-varying interventions can find patterns of human driving behavior using data from self-driving cars. Finally, the utilization of causal knowledge (in the form of graphs) can lead to novel unsupervised learning algorithms. In summary, this research agenda will open up the utility of machine learning to domains where data is scarce and new problems where labels are expensive to obtain.

该研究项目将专注于利用特权信息开发新的学习算法（参数估计和概率推理）和深度生成模型决策。本研究计划有三个关键目标：来自大量文本数据的特权信息：文本数据是关于许多不同预测任务的上下文信息的丰富存储。临床教科书包含疾病表现的重要信息，可以告知疾病发病的预测模型。我们将为深度生成模型开发新的学习算法，利用训练时文本数据中呈现的预测任务的上下文。我们将使用基于神经网络的语言模型来学习文本数据的哪些方面对解决手头的预测问题最有帮助。时间序列数据中的特权信息：通过生存分析预测未来发生的事件是一项强大的技术，用于经济学和医疗保健等领域，其目标可能是确定患者何时会出现并发症。但在患者的基线数据和感兴趣的事件之间，存在着表征患者长期生理状态的数据。我们将开发新的分层潜变量模型，揭示时间序列特权信息的结构，并使用这种结构来提高生存分析模型的预测质量和样本复杂性。因果图中的特权信息：潜变量深度生成模型在学习的内环中使用推理网络来预测变分分布的参数。当数据缺失时，这个预测问题是具有挑战性的，并且当推理网络根据缺失的数据进行预测时，对变分参数产生的偏差知之甚少。我们将研究表征随机变量之间因果关系的因果图如何作为推理网络的特权信息，并用于改进深度生成模型中的变分推理（以及由此产生的参数估计）。本课程将应用本研究项目中开发的算法来预测自然科学和工程中出现的问题，包括（但不限于）经济学、社会科学和医疗保健。样本高效预测模型的设计可以为整合新兴的数据模式（如计算生物学中的蛋白质组学）打开大门。分层模型可以将纵向、高维数据与时变干预聚类，可以利用自动驾驶汽车的数据找到人类驾驶行为的模式。最后，利用因果知识（以图的形式）可以产生新的无监督学习算法。总之，这个研究议程将打开机器学习的应用领域，在数据稀缺的领域和新问题，标签是昂贵的获得。