CIF: Small: Collaborative Research:Algorithms and Information-Theoretic Limits for Data-Limited Inference

这是我们这个时代的一个讽刺，尽管生活在“信息时代”，我们往往是有限的数据。经过几十年的研究，科学家们仍然在争论气候变化的原因和影响，最近的工作表明，过去13年中最有影响力的医学研究中有很大一部分后来被发现是不准确的，主要是由于数据不足。这种明显矛盾的一个原因是，对复杂的真实世界信息源建模需要丰富的概率模型，即使从非常大的数据集也无法准确地学习。在更深层次上，研究本质上处于可能性的边缘，并试图解决现有数据只能部分回答的问题。因此，我们有理由期望我们将永远是数据有限的。本研究涉及开发新的算法和性能界限的数据有限的推理。PI的先前工作表明，通过采用信息理论方法，可以开发出专门针对数据有限制度的新算法，并且比以前已知的性能更好，并且在某些情况下可以证明是最佳的。该项目通过考虑一系列跨越多个应用领域的问题，如分类，确定两个数据集是由相同的分布还是不同的分布生成的，从事件定时进行分布估计，熵估计，以及复杂和未知信道上的通信，来推进开发数据有限推理的通用理论的目标。尽管这些问题之前都被孤立地研究过，但PI的先前工作表明，将它们视为同一个潜在问题的实例是富有成效的：数据有限的推理。

It is an irony of our time that despite living in the 'information age' we are often data-limited. After decades of research, scientists still debate the causes and effects of climate change, and recent work has shown that a significant fraction of the most influential medical studies over the past 13 years have been subsequently found to be inaccurate, largely due to insufficient data. One reason for this apparent paradox is that modeling complex, real-world information sources requires rich probabilistic models that cannot be accurately learned even from very large data sets. On a deeper level, research inherently resides at the edge of the possible, and seeks to address questions that available data can only partially answer. It is therefore reasonable to expect that we will always be data-limited.This research involves developing new algorithms and performance bounds for data-limited inference. Prior work of the PIs has shown that, by taking an information-theoretic approach, one can develop new algorithms that are tailored specifically to the data-limited regime and perform better than was previously known, and in some cases are provably optimal. This project advances the goal of developing a general theory for data-limited inference by considering a suite of problems spanning multiple application areas, such as classification; determining whether two data sets were generated by the same distribution or by different distributions; distribution estimation from event timings; entropy estimation; and communication over complex and unknown channels. Whereas these problems have all been studied before in isolation, prior work of the PIs has shown it is fruitful to view them as instances of the same underlying problem: data-limited inference.