Breaking the curse of dimensionality in low-data tasks

打破低数据任务中的维度诅咒

• 批准号: 2615996
• 金额: --
• 依托单位: University of Cambridge
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2615996
• 关键词: Breaking; curse; dimensionality; low; data

This research seeks to create data-efficient machine learning algorithms that can learn in complex domains with low-data and high dimensionality. Specifically, the research will focus on tasks with - 100s samples and 1000 - 20; 000 features, such as diagnosing patients from clinical trials where sequencing data is available. Learning from little data usually requires creating machine learning models that incorporate adequate invariances and inductive biases. The Bayesian machine learning framework allows inputting domain knowledge through specifying prior distributions and kernel functions. However, specifying kernels for complex domains remains challenging, and it is common practice to use uninformative priors and rely almost entirely on learning from the data.This research aims to circumvent the apparent limitations of learning from low-data by designing methods to learn priors that capture the rich interactions between features, and to incorporate human knowledge. It will investigate ways to learn kernel functions for Bayesian models. Using data-driven kernel could enable integrating learned information about the complex feature interactions (e.g., gene interactions in particular diseases) and facilitate reliable predicting from small datasets. The first research direction is learning rich kernel functions for semi-parametric models via the framework of deep kernel learning in the context of Gaussian Processes. A second research direction will investigate transfer learning for the 'kernel function' stored in the parameters of the recently introduced neural processes.Potential impact: This research has the potential to enable reliable estimation from small, high-dimensional datasets from various domains such as Medicine, drug discovery and beyond. Ultimately, the proposed advancements could capture rich interaction between variables and transfer this knowledge to similar scenarios in which relying on data alone is insufficient. I hope the proposed approach will become standard practice in enabling fast learning, similar to the transfer learning framework in domains such as computer vision or natural language processing.

这项研究旨在创建数据高效的机器学习算法，可以在低数据和高维的复杂领域中学习。具体来说，该研究将专注于具有约100个样本和1000 - 20; 000个特征的任务，例如从测序数据可用的临床试验中诊断患者。从少量数据中学习通常需要创建包含足够的不变性和归纳偏差的机器学习模型。贝叶斯机器学习框架允许通过指定先验分布和核函数来输入领域知识。然而，为复杂域指定核仍然具有挑战性，并且通常的做法是使用无信息的先验知识，并且几乎完全依赖于从数据中学习，本研究旨在通过设计方法来学习捕获特征之间丰富的相互作用的先验知识，并结合人类知识，从而规避从低数据中学习的明显局限性。它将研究学习贝叶斯模型的核函数的方法。使用数据驱动的内核可以集成关于复杂特征交互的学习信息（例如，特定疾病中的基因相互作用），并有助于从小数据集进行可靠的预测。第一个研究方向是在高斯过程的背景下，通过深度核学习框架学习半参数模型的丰富核函数。第二个研究方向将研究最近引入的神经过程参数中存储的“核函数”的迁移学习。潜在影响：这项研究有可能从医学，药物发现等各个领域的小型高维数据集进行可靠的估计。最终，拟议的进步可以捕捉变量之间的丰富互动，并将这些知识转移到仅依靠数据是不够的类似场景中。我希望所提出的方法将成为实现快速学习的标准实践，类似于计算机视觉或自然语言处理等领域的迁移学习框架。