Methods for Dealing with Misspecification in Bayesian Experimental Design

贝叶斯实验设计中处理错误指定的方法

• 批准号: 2740638
• 金额: --
• 依托单位: University of Oxford
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2740638
• 关键词: Methods; Dealing; Misspecification; Bayesian; Experimental

Much of the research in statistics and machine learning has focussed on methods of analysing data once it has already been acquired, but the question of how to best collect data in the first place has been under-explored. Gathering data can be expensive, therefore practitioners are limited by the amount of data they can collect. When this is done without care, it can produce poor quality data - potentially leading to inaccurate results and incorrect conclusions, regardless of how advanced one's analytical toolkit is. It is therefore vital to endeavour to gather good quality data before analysis. Research in experimental design aims to address this issue, providing practitioners with methods of collecting informative data that will lead to reliable results and strong conclusions. To outline Bayesian experimental design (BED), we consider the following setting: there are several beacons within an area, each emitting a signal, and a practitioner wishes to locate the beacons. The data-gathering process involves the practitioner choosing locations in which to probe the signal, then recording the strength of that signal at these locations. With infinite resources, the practitioner would be able to perfectly locate the beacons, but in practice they are constrained to performing only a finite number of experiments. BED then works as follows: before collecting any data, the practitioner will first form a statistical model of the strength of the signal at a given location in terms of the unknown locations of the beacons, and they will specify their prior beliefs about the locations of the beacons with a prior distribution on these locations. BED procedures can then provide the practitioner with the best locations to probe the signal - where "best" is defined as the locations that will lead to the largest increase in information about the beacon locations. The above can be easily generalised to other settings. BED is both theoretically sound and performs well practically, however, it can break down if our statistical model of the data is misspecified, i.e., if the true data-generating process is different from the model that we specified. Bayesian statistical methodology is always vulnerable to model misspecification, but unfortunately this fact is particularly problematic for experimental design, where we are not just using our model to analyse the data, but also to collect it in the first place. In the worst case, there are models in which the optimal course of action is to pick all your designs in exactly the same place, regardless of the outcomes you observe. However, unless your model is correct, this will produce an extremely poor quality dataset.In collaboration with my supervisor, Dr Tom Rainforth, we will first aim to deepen understanding of this problem: categorising the ways in which misspecification causes BED to fail; forming metrics to diagnose this failure and best practices to avoid its occurrence; and providing theoretical guarantees for when failure will occur. Following this, we will develop methods to counteract misspecification, ideally expanding the theoretical elegance and empirical performance of BED to cases when our model is misspecified. BED has enormous potential application, including quantum information experiments, psychology trials, constructing lifelike police sketches, and guiding drug discovery. As these applications become more complex, model misspecification becomes more prevalent; it is therefore pertinent to further investigate misspecification in BED. This project falls within the EPSRC 'statistics and applied probability' research area.

统计学和机器学习领域的大部分研究都集中在数据获取后的分析方法上，但对如何最好地收集数据这一问题一直没有得到充分的探讨。收集数据的成本可能很高，因此从业者受到他们可以收集的数据量的限制。如果不小心这样做，可能会产生质量不佳的数据--无论一个人的分析工具有多先进，都可能导致不准确的结果和不正确的结论。因此，在分析之前努力收集高质量的数据是至关重要的。实验设计研究旨在解决这一问题，为从业者提供收集信息数据的方法，这些数据将导致可靠的结果和强有力的结论。为了概述贝叶斯实验设计(BED)，我们考虑以下设置：在一个区域内有几个信标，每个信标都发出一个信号，从业者希望定位信标。数据收集过程包括从业者选择探测信号的位置，然后记录这些位置的信号强度。有了无限的资源，从业者将能够完美地定位信标，但在实践中，他们只能进行有限数量的实验。Bed然后工作如下：在收集任何数据之前，从业者首先根据信标的未知位置形成在给定位置的信号强度的统计模型，并且他们将用这些位置上的先验分布来指定他们对信标位置的先验信念。然后，BED程序可以为从业者提供探测信号的最佳位置--其中“最佳”被定义为将导致有关信标位置的信息增加最多的位置。以上内容可以很容易地推广到其他环境。BED在理论上是可靠的，在实践中也表现得很好，但是，如果我们错误地指定了数据的统计模型，即如果真正的数据生成过程与我们指定的模型不同，它可能会崩溃。贝叶斯统计方法总是容易受到模型错误说明的影响，但不幸的是，这一事实对于实验设计尤其成问题，在实验设计中，我们不仅使用我们的模型分析数据，而且首先收集数据。在最糟糕的情况下，有一些模型的最佳行动路线是在完全相同的地方挑选所有设计，而不管你观察到的结果如何。然而，除非你的模型是正确的，否则这将产生一个质量极差的数据集。我们将首先与我的主管汤姆·雷恩福斯博士合作，旨在加深对这个问题的理解：对错误规范导致床故障的方式进行分类；形成诊断这种故障的指标和避免其发生的最佳实践；以及为何时发生故障提供理论保证。在此之后，我们将开发出抵消错误指定的方法，理想地将BED的理论优雅和经验性能扩展到我们的模型错误指定的情况。Bed具有巨大的潜在应用，包括量子信息实验、心理学试验、构建栩栩如生的警察草图以及指导药物发现。随着这些应用变得越来越复杂，模型错误说明变得更加普遍；因此，在BED中进一步研究错误说明是有意义的。该项目属于EPSRC的统计和应用概率研究领域。