Data-Driven Algorithms for Data Acquisition

机器学习的进步改变了我们利用数据的能力。但在智能获取此类数据方面，首先取得的进展要少得多。因此，尽管数据驱动的方法现在在科学和工业中无处不在，但手工制作和启发式方法通常仍然是数据采集本身的标准。我的目标是通过开发有原则的定量数据采集方法来解决这一不足。特别是，我将构建自适应算法，利用来自以前数据的信息来指导未来的数据获取。这样做的基础将是贝叶斯自适应设计(BAD)的框架，该框架通过数据提供的信息来形式化数据的效用，然后利用这一点来优化收购过程的可控方面。尽管有原则上的基础，但由于部署中的一些关键挑战，BAD尚未看到实质性的采用。最值得注意的是，它存在严重的计算瓶颈，破坏了它的使用。通过使用一种新的基于政策的方法来克服这些问题，我希望将BAD的潜力变成现实，为从互动调查和虚拟助理到实验室实验和心理学试验等各种领域的智能数据获取提供强大的基础。其中一个特别关注的领域将是主动学习，即一个人反复从未标记的池中选择要标记的点。BAD在这方面已经取得了一些成功，但我认为它目前从根本上被滥用了。我希望通过各种创新，如以预测而不是参数为目标的信息收益，适当利用未标记的数据，以及开发基于政策的方法，大幅提高该领域的最新水平。我还建议重新审视经常在这种情况下使用的贝叶斯神经网络模型的基础，质疑它们的基本假设，并开发出全新的方法。

Advances in machine learning have transformed our ability to utilize data. But far less progress has been made on intelligently acquiring such data in the first place. Consequently, though data-driven approaches are now ubiquitous across science and industry, hand-crafted and heuristic approaches are typically still the norm for data acquisition itself.My goal is to address this shortfall by developing principled quantitative methods for data acquisition. In particular, I will construct adaptive algorithms that leverage information from previous data to guide future data acquisition. The basis for doing this will be the framework of Bayesian adaptive design (BAD), which formalizes the utility of data through the information it provides, then exploits this to optimize the controllable aspects of the acquisition process.Despite its principled foundations, BAD has not yet seen substantial uptake due to some key challenges in its deployment. Most notably, it has crippling computational bottlenecks that undermine its usage. By overcoming these with a new policy-based approach, I hope to turn BAD's potential into a reality, providing a powerful basis for intelligent data acquisition in domains as diverse as interactive surveys and virtual assistants, to laboratory experiments and psychology trials.One area of particular focus will be active learning, wherein one iteratively selects points to label from an unlabelled pool. Here BAD has already provided some success, but I believe it is currently fundamentally misapplied. I hope to substantially improve state-of-the-art in the area through various innovations, such as targeting information gain in predictions rather than parameters, properly utilizing unlabelled data, and developing policy-based approaches. I further propose to revisit the foundations of the Bayesian neural network models often used in such settings, questioning their fundamental assumptions and developing radically new approaches.