Optimal Shrinkage and Empirical Bayes Prediction under Asymmetry, Censoring and Nonexchangeability

在大数据分析的每个分支中，现在在构建稳健的算法和预测器时使用收缩概念是司空见惯的。收缩的概念很重要，因为它为组合来自相关种群的信息提供了一个优雅的框架，并且经常导致用于同时推理的算法性能的实质性改进。在过去十年中，在广泛的科学问题应用的推动下，随着引入新的视角来解决和利用现代数据集的复杂、潜在的结构特性，统计收缩的传统角色已经迅速演变。这些新应用通常涉及非标准的推断属性，如不对称预测目标，以及复杂的建模警告，如不可交换的先验结构和审查的观察。这些新时代的统计问题不仅对开发灵活的收缩算法提出了挑战，而且对它们进行优化以获得有效的收缩特性也提出了挑战。该项目将开发新的经验贝叶斯预测方法，该方法具有最佳的收缩特性，并且可以显著增强基于对称损失函数和可交换先验结构的数学便利性的现有算法。这个项目背后的共同主题是使用最佳收缩特性来开发有效的预测方法。现有的收缩算法严重依赖于不对称和不可交换性下的决策理论恒等式，因此，迫切需要开发新的统计方法、理论和算法。PI将开发新的条件线性决策规则，用于非交换高斯分层模型中不对称下的预测，并将提出的方法扩展到非高斯模型以及具有非线性结构约束的设置。此外，将开发最优经验贝叶斯规则，该规则将与审查数据一起工作，并可用于具有不对称目标的多阶段决策场景的预测。本项目开发的结果将使从业者更好地理解现有的预测方法在不对称、审查和不可交换的情况下失败的地方，以及为什么应该使用专门开发的算法来在这些条件下运行。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In every branch of big-data analytics, it is now commonplace to use notions of shrinkage in the construction of robust algorithms and predictors. The concept of shrinkage is important because it provides an elegant framework for combining information from related populations and often leads to substantial improvements in the performances of algorithms used for simultaneous inference. Driven by applications in a wide range of scientific problems over the last decade, the traditional roles of statistical shrinkage have rapidly evolved as new perspectives have been introduced to address and exploit complex, latent structural properties of modern datasets. These new applications often involve non-standard inferential attributes such as asymmetric predictive objectives as well as intricate modeling caveats, such as nonexchangeable prior structures and censored observations. These new age statistical problems pose challenges not only in developing flexible shrinkage algorithms but also in optimally tuning them to obtain efficient shrinkage properties. This project will develop new empirical Bayes predictive methods that possess optimal shrinkage properties and can produce significant enhancements over existing algorithms built on the mathematical convenience of symmetric loss functions and exchangeable prior structures.The common theme underlying this project is that of using optimal shrinkage properties to develop efficient predictive methods. Existing shrinkage algorithms rely heavily on decision theoretic identities that break down under asymmetry and nonexchangeability, and so, there is an urgent need to develop new statistical methodologies, theories, and algorithms. The PI will develop new conditionally linear decision rules for prediction under asymmetry in nonexchangeable Gaussian hierarchical models and will extend the proposed methodologies to non-Gaussian models as well as to settings with non-linear structural constraints. Additionally, optimal empirical Bayes rules will be developed that will work with censored data and can be used for prediction in multi-stage decision making scenarios with asymmetric objectives. The results developed in this project will provide practitioners with an improved understanding of where existing prediction approaches fail under asymmetry, censoring, and nonexchangeability and why algorithms specifically developed to operate under these conditions should be used.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.