Breaking the curse of dimensionality in low-data tasks

本研究旨在创建数据高效的机器学习算法，可以在具有低数据和高维的复杂领域中学习。具体来说，研究将集中在- 100个样本和1000 - 20个样本的任务上；000个特征，例如从可获得测序数据的临床试验中诊断患者。从小数据中学习通常需要创建包含足够不变性和归纳偏差的机器学习模型。贝叶斯机器学习框架允许通过指定先验分布和核函数输入领域知识。然而，为复杂的领域指定核仍然具有挑战性，并且使用无信息的先验和几乎完全依赖于从数据中学习是常见的做法。本研究旨在通过设计方法来学习捕获特征之间丰富交互的先验，并结合人类知识，从而规避从低数据中学习的明显局限性。它将研究如何学习贝叶斯模型的核函数。使用数据驱动的内核可以整合关于复杂特征相互作用的学习信息（例如，特定疾病的基因相互作用），并有助于从小数据集进行可靠的预测。第一个研究方向是利用高斯过程背景下的深度核学习框架学习半参数模型的富核函数。第二个研究方向将研究存储在最近引入的神经过程参数中的“核函数”的迁移学习。潜在影响：这项研究有可能对来自医学、药物发现等各个领域的小型高维数据集进行可靠的估计。最终，提出的进步可以捕获变量之间丰富的相互作用，并将这些知识转移到仅依赖数据是不够的类似场景中。我希望所提出的方法将成为实现快速学习的标准实践，类似于计算机视觉或自然语言处理等领域的迁移学习框架。

This research seeks to create data-efficient machine learning algorithms that can learn in complex domains with low-data and high dimensionality. Specifically, the research will focus on tasks with - 100s samples and 1000 - 20; 000 features, such as diagnosing patients from clinical trials where sequencing data is available. Learning from little data usually requires creating machine learning models that incorporate adequate invariances and inductive biases. The Bayesian machine learning framework allows inputting domain knowledge through specifying prior distributions and kernel functions. However, specifying kernels for complex domains remains challenging, and it is common practice to use uninformative priors and rely almost entirely on learning from the data.This research aims to circumvent the apparent limitations of learning from low-data by designing methods to learn priors that capture the rich interactions between features, and to incorporate human knowledge. It will investigate ways to learn kernel functions for Bayesian models. Using data-driven kernel could enable integrating learned information about the complex feature interactions (e.g., gene interactions in particular diseases) and facilitate reliable predicting from small datasets. The first research direction is learning rich kernel functions for semi-parametric models via the framework of deep kernel learning in the context of Gaussian Processes. A second research direction will investigate transfer learning for the 'kernel function' stored in the parameters of the recently introduced neural processes.Potential impact: This research has the potential to enable reliable estimation from small, high-dimensional datasets from various domains such as Medicine, drug discovery and beyond. Ultimately, the proposed advancements could capture rich interaction between variables and transfer this knowledge to similar scenarios in which relying on data alone is insufficient. I hope the proposed approach will become standard practice in enabling fast learning, similar to the transfer learning framework in domains such as computer vision or natural language processing.