Nonparametric Inference and Prediction for Complex Data by Data Depth, Confidence Distribution and Monte Carlo Method

在信息和数据爆炸的时代，解决问题和辅助决策的有效数据分析方法的需求从未像现在这样大。这种需求来自各个领域，从现代科学努力、政府和行业政策制定、金融和商业战略规划，甚至是最基本的社会经济研究。尽管最近在数学和统计科学方面取得了长足的进步，但由于需要面对无处不在的海量、多样化和复杂的数据，许多新的挑战也浮出水面。这个项目的PI将开发几种新的方法来解决在数据源多样化或传统的统计大样本理论无法应用的情况下的一般推理和预测问题。在几个实际应用的推动下，该项目将开发非参数方法来从不同的数据源进行个性化推理(简称I-Fusion)，对复杂数据进行预测，以及对估计方程进行精确推理。这些方法的基础是由数据深度、置信度分布和蒙特卡罗方法组成的通用工具包。所提出的方法可望具有广泛的适用性、高效性和计算可行性。三个具体项目是：a.开发新的i-Fusion，通过有效结合相关数据源的学习来得出有效的个体化推理；b.开发CD蒙特卡罗方法，用于估计方程的精确推理；c.开发非参数预测分布，用于对复杂数据进行有效预测。拟议的方法将在理论支持下开发并应用于以下领域：i)预测向政府机构相关单位提交的申请量；ii)通过借用可能与其他公司共享的信息对个别公司的业绩进行预测；以及iii)确定跟踪全球冰川条纹的热点。这些应用程序是由PIS与国土安全部CCICADA正在进行的合作项目以及可能的罗格斯气候风险和复原力倡议推动的。这些项目涉及真实的数据库，非常适合吸引和培训学生和新的研究人员。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In the era of information and data explosion, the demand of effective data analysis methods for solving problems and assisting decision-making has never been greater. This demand comes from all domains, from modern scientific endeavors, government and industry policy-making, financial and business strategic planning, to even the most basic social-economic studies. Despite recent great strides made in mathematical and statistical sciences, many new challenges have been brought to the fore by the need of confronting the pervasive massive, diverse and complex data. The PIs of this project will develop several novel approaches to addresses general inference and prediction problems in settings where data sources are diverse or where the conventional statistical large sample theory fails to apply.Motivated by several real applications, this project will develop nonparametric approaches for: individualized inference from diverse data sources (referring to as i-Fusion), prediction for complex data, and exact inference for estimating equations. Underlying these proposed approaches is the common tool kit consisting of data depth, confidence distribution and Monte Carlo methods. The proposed approaches are expected to be broadly applicable, efficient and computationally feasible. Three specific projects are: A. Develop the new i-Fusion for drawing efficient individualized inference by effectively combining learnings from relevant data sources; B. Develop CD Monte-Carlo methods for the exact inference for estimating equations; C. Develop nonparametric predictive distributions for efficient prediction with complex data. The proposed methodologies will be developed with theoretical support and applied to the areas: i) prediction of volumes of application submissions to interrelated units in a government agency; and ii) performance forecast for individual companies by borrowing information possibly shared by others, and, potentially, iii) identification of hot spots in tracking glacial striation around the globe. These applications are motivated by the PIs' ongoing collaborative projects with the CCICADA of Department of Homeland Security, and possibly Rutgers Climate Risk and Resilience Initiative. These projects involve real databases and are ideally suited for engaging and training students and new researchers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.