Bayesian Nonparametric Methods for Aggregated and Multivariate Outputs

用于聚合和多元输出的贝叶斯非参数方法

• 批准号: 2283505
• 金额: --
• 依托单位: Imperial College London
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2019
• 资助国家: 英国
• 起止时间: 2019 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2283505
• 关键词: Bayesian; Nonparametric; Methods; Aggregated; Multivariate

This project investigates 2 types of situations under data-scarce labelled data that are expensive to obtain, situations which are problems that occur in many environmental and social sciences problems. We aim to develop novel methods that tackle these situations using flexible proxy models that encode prior beliefs and interpretable uncertainty quantifications. This project falls within the EPSRC Mathematical Sciences research area and is partly funded by and in collaboration with Cervest Limited, an artificial intelligence start-up focusing on Earth Science AI, and Imperial College London. This collaboration between industry and academia will allow our research to have access to a wide array of Earth observation datasets from the industry as well as for the industry to gain access to novel methodologies for their own work. The first part involving aggregated outputs address the situation where we typically observe or must average out quantities over large groups of individuals or geographical areas. An important application where this type of problem occurs is in computing the average treatment effect of administering a pharmaceutical or policy intervention. When labelled data is scarce, this type of problem is even more complex. For instance, how do we model crop yields over a large geographical region when we only know what the yield is for the entire region? The second part of the project involves modelling multiple quantities, such as precipitation and temperature, jointly in a way that exploits their inter-dependence. Again, when labelled data is scarce modelling multiple quantities can allow for additional signals to be extracted. To capture complex interactions between covariates and outputs, nonparametric methods, ones that assume infinitely many model parameters such as Gaussian processes (GP), provide a flexible way for encoding prior beliefs, and there is also a rich literature on using GPs for label-scarce and feature-rich situations (Law et al. (2018); Hamelijnck et al. (2019)). GPs encode prior beliefs using normal distributions and can also give uncertainty quantification, which is highly desirable for situations when this is important. Recently, tree-based models (Chipman et al. (2010); Lakshminarayanan et al. (2016)), where the prior belief is broken down into subgroups of individuals or subregions, have been of interest to the machine learning community, yielding highly competitive results to GPs. Like GPs, tree-based models also provide a flexible nonparametric model that can provide uncertainty quantification, but properties of tree-based priors have yet to have been fully exploited for more complex applications. We hope to work on the development of novel nonparametric methodologies as solutions for our project aims. We will first develop novel nonparametric modelling approaches for applications that involve aggregated quantities of interest and outputs. We will then work on developing flexible models for multiple outputs with broad applications for environmental sciences in mind. References:Chipman, H.A., George, E.I. and McCulloch, R.E., 2010. BART: Bayesian additive regression trees. The Annals of Applied Statistics, 4(1), pp.266-298. Hamelijnck, O., Damoulas, T., Wang, K. and Girolami, M., 2019. Multi-resolution multi-task Gaussian processes. In Advances in Neural Information Processing Systems (pp. 14025-14035). Lakshminarayanan, B., Roy, D.M. and Teh, Y.W., 2016, May. Mondrian forests for large-scale regression when uncertainty matters. In Artificial Intelligence and Statistics (pp. 1478-1487). Law, H.C., Sejdinovic, D., Cameron, E., Lucas, T., Flaxman, S., Battle, K. and Fukumizu, K., 2018. Variational learning on aggregate outputs with Gaussian processes. In Advances in Neural Information Processing Systems (pp. 6081-6091).

该项目研究了数据稀缺且获取成本昂贵的标记数据下的两种情况，这些情况是许多环境和社会科学问题中出现的问题。我们的目标是开发新的方法来解决这些情况，使用灵活的代理模型来编码先验信念和可解释的不确定性量化。该项目属于 EPSRC 数学科学研究领域，部分由 Cervest Limited（一家专注于地球科学 AI 的人工智能初创公司）和伦敦帝国学院资助并与之合作。工业界和学术界之间的这种合作将使我们的研究能够访问来自工业界的各种地球观测数据集，并使工业界能够为自己的工作获得新颖的方法。第一部分涉及汇总产出，解决了我们通常观察或必须对大量个人或地理区域的数量进行平均的情况。发生此类问题的一个重要应用是计算药物或政策干预的平均治疗效果。当标记数据稀缺时，此类问题就更加复杂。例如，当我们只知道整个地区的产量时，如何对一个大地理区域的农作物产量进行建模？该项目的第二部分涉及对多个量（例如降水量和温度）进行联合建模，以利用它们的相互依赖性。同样，当标记数据稀缺时，对多个量进行建模可以允许提取额外的信号。为了捕获协变量和输出之间的复杂相互作用，非参数方法（假设无限多个模型参数，例如高斯过程（GP））提供了一种灵活的方式来编码先验信念，并且还有大量关于在标签稀缺和特征丰富的情况下使用 GP 的文献（Law 等人（2018）；Hamelijnck 等人（2019））。 GP 使用正态分布对先验信念进行编码，并且还可以给出不确定性量化，这对于非常重要的情况非常理想。最近，基于树的模型（Chipman 等人（2010）；Lakshminarayanan 等人（2016））将先验信念分解为个体或子区域的子组，引起了机器学习界的兴趣，为 GP 带来了极具竞争力的结果。与 GP 一样，基于树的模型也提供了灵活的非参数模型，可以提供不确定性量化，但基于树的先验的属性尚未充分利用于更复杂的应用程序。我们希望致力于开发新颖的非参数方法作为我们项目目标的解决方案。我们将首先为涉及感兴趣和输出的聚合数量的应用程序开发新颖的非参数建模方法。然后，我们将致力于开发多种输出的灵活模型，并考虑到环境科学的广泛应用。参考文献：Chipman, H.A.、George, E.I.和 McCulloch, R.E.，2010。BART：贝叶斯加性回归树。 《应用统计年鉴》，4(1)，第 266-298 页。 Hamelijnck, O.、Damoulas, T.、Wang, K. 和 Girolami, M.，2019。多分辨率多任务高斯过程。神经信息处理系统的进展（第 14025-14035 页）。拉克什米纳拉亚南 (Lakshminarayanan)，B.，罗伊 (Roy)，D.M.和 Teh, Y.W.，2016 年 5 月。当不确定性很重要时，蒙德里安森林可以进行大规模回归。人工智能和统计（第 1478-1487 页）。 Law, H.C.、Sejdinovic, D.、Cameron, E.、Lucas, T.、Flaxman, S.、Battle, K. 和 Fukumizu, K.，2018。高斯过程聚合输出的变分学习。神经信息处理系统的进展（第 6081-6091 页）。

项目成果

Markovian Gaussian Process Variational Autoencoders

• DOI: 10.48550/arxiv.2207.05543
• 发表时间: 2022-07
• 期刊: ArXiv
• 作者: Harrison Zhu;Carles Balsells Rodas;Yingzhen Li

Bayesian Probabilistic Numerical Integration with Tree-Based Models

贝叶斯概率数值积分与基于树的模型

• 发表时间: 2020
• 作者: Zhu H

Grassmann Stein Variational Gradient Descent

• 发表时间: 2022-02
• 期刊: ArXiv
• 作者: Xingtu Liu;Harrison Zhu;Jean-Francois Ton;George Wynne;A. Duncan

Convolutional Neural Processes for Inpainting Satellite Images

• DOI: 10.48550/arxiv.2205.12407
• 发表时间: 2022-05
• 期刊: ArXiv
• 作者: Alexander Pondaven;M. Bakler;D. Guo;Hamzah Hashim;Martin Ignatov;Harrison Zhu

Aggregated Gaussian Processes with Multiresolution Earth Observation Covariates

具有多分辨率地球观测协变量的聚合高斯过程

• 发表时间: 2022
• 作者: Zhu H