Active Learning of Bayesian Linear Models with High-Dimensional Binary Features by Parameter Confidence-Region Estimation
Active Learning of Bayesian Linear Models with High-Dimensional Binary Features by Parameter Confidence-Region Estimation
复制标题
DOI：
10.1162/neco_a_01310
复制
发表时间：
2020-08
影响因子：
2.9
通讯作者：
Yu Inatsu;Masayuki Karasuyama;Keiichi Inoue;H. Kandori;I. Takeuchi
中科院分区：
文献类型：
--
作者：
Yu Inatsu;Masayuki Karasuyama;Keiichi Inoue;H. Kandori;I. Takeuchi
在这封信中，我们研究了一个主动学习问题，最大化一个未知的线性函数与高维二进制功能。这个问题是众所周知的复杂，但在许多重要的情况下出现。当采样预算，即可能的函数求值的数量小于维数时，往往不可能识别所有的最佳二进制特征。因此，在实践中，只有少数这样的功能被考虑，与大多数保持固定在某些默认值，我们称之为工作集启发式。这封信的主要贡献是正式研究工作集启发式，并提出了一套理论上强大的算法，更有效地利用抽样预算。从技术上讲，我们引入了一种新的方法来估计模型参数的置信区域，该方法适用于具有高维二进制特征的主动学习。我们提供了一个严格的理论分析，这些算法，并证明了一个常用的工作集启发式可以确定最佳的二进制功能，有利的样本复杂度。我们探讨所提出的方法，通过数值模拟和应用程序的功能蛋白质设计问题的性能。
In this letter, we study an active learning problem for maximizing an unknown linear function with high-dimensional binary features. This problem is notoriously complex but arises in many important contexts. When the sampling budget, that is, the number of possible function evaluations, is smaller than the number of dimensions, it tends to be impossible to identify all of the optimal binary features. Therefore, in practice, only a small number of such features are considered, with the majority kept fixed at certain default values, which we call the working set heuristic. The main contribution of this letter is to formally study the working set heuristic and present a suite of theoretically robust algorithms for more efficient use of the sampling budget. Technically, we introduce a novel method for estimating the confidence regions of model parameters that is tailored to active learning with high-dimensional binary features. We provide a rigorous theoretical analysis of these algorithms and prove that a commonly used working set heuristic can identify optimal binary features with favorable sample complexity. We explore the performance of the proposed approach through numerical simulations and an application to a functional protein design problem.