Multivariate Statistical Methodology for the Virtual Observatory

NAS Taylor/McKee的2000-2010年天文十年报告建议作为最优先事项建立一个国家虚拟天文台，将许多现有天文调查的档案数据集和星表连接起来。有效利用这种综合的海量数据集涉及的不仅仅是信息的获取和提取--科学理解需要对选定的数据进行复杂的统计建模。这项工作属于统计推断的范畴，包括多变量分析、非参数分析、贝叶斯分析、空间点过程、密度估计和数据挖掘。大规模多波长天文调查提出了各种新的具有挑战性的统计和算法问题，需要方法论的进步。首席调查员和他的同事们讨论了NVO提出的一些至关重要的统计挑战。具体办法包括：对大数据集进行低存储百分位数估计，对多分辨率K维树进行分组和异常值检测，以及对多变量天文数据集与天体物理模型和模拟进行比较的多维拟合度检验。这样的努力需要居住在不同机构的统计学家、天文学家和NVO专家的密切合作。在NVO软件环境中开发一个执行新的和现有方法的统计工具包是该项目的中心目标之一。随着近几十年来天文发现的数据量和复杂性的极大增加，观测天文学的本质正在发生范式的转变。在过去，一位天文学家可能会观测几个天体，而今天，对在所有波长的光获得的大型数字天空档案进行数据挖掘正在成为一种主要的研究模式。因此，天文学界面临着一项关键任务：能够有效和客观地科学利用巨大的多方面数据集。认识到这一需要，最近出现了国家虚拟天文台(NVO)倡议，以联合众多大型数字天空档案，并开发工具来探索和理解这些海量数据。这里的调查旨在开发统计和计算方法来实现这些目标。由天文学家和统计学家组成的跨学科团队将这些领域的进展带入观测天文学的工具箱。该项目不仅寻求制定有效的技术来解决NVO问题，而且还将这些方法编码成NVO软件环境中的统计工具包，供整个天文学团体使用。合作包括两个精通天体统计学的机构(宾夕法尼亚州立大学和卡内基梅隆大学)和一个处于NVO工作中心的机构(加州理工学院)。研究生和博士后的参与给了他们一个难得的机会来发展跨学科工作所需的技能。

The NAS Taylor/McKee Decadal Report on astronomy for 2000-2010 recommends as a top priority the formation of a National Virtual Observatory (NVO) to link archival data sets and catalogues from many existing astronomical surveys. The effective use of such integrated massive data sets involves more than just access and extraction of information -- scientific understanding requires sophisticated statistical modeling of the selected data. This effort falls under the rubric of statistical inference and includes the fields of multivariate analysis, nonparametrics, Bayesian analysis, spatial point processes, density estimation and data mining.Large-scale multiwavelength astronomical surveys present a variety of new challenging statistical and algorithmic problems that require methodological advances. The principal investigator and his colleagues address some of the critically important statistical challenges raised by the NVO. Specific approaches include: low-storage percentile estimation for large data sets, multi-resolutional K-Dimensional trees for clustering and outlier detection, and multi-dimensional goodness-of-fit tests for comparison of multivariate astronomical data sets with astrophysical models and simulations. Such an endeavor needs close collaboration of statisticians, astronomers and NVO specialists who reside at different institutions. Developing a statistical toolkit within the NVO software environment implementing both new and existing methods is one of the central goals of this project.As the data volume and complexity of astronomical findings have enormously increased in recent decades, a paradigm shift is underway in the very nature of observational astronomy. While in the past a single astronomer might observe a handful of objects, today data mining of large digital sky archives obtained at all wavelengths of light is becoming a major mode of study. The astronomical community thus faces a key task: to enable efficient and objective scientific exploitation of enormous multifaceted data sets. In recognition of this need, the NationalVirtual Observatory (NVO) initiative has recently emerged to federate numerous large digital sky archives and develop tools to explore and understand these vast volumes of data. The investigation here aims at developing statistical and computational methods to achieve these goals. The cross-disciplinary team, of astronomers and statisticians, brings advances in these fields into the toolbox of observational astronomy. The project seeks not only to formulate effective techniques to address NVO problems, but also to code these methods into statistical toolkits within NVO software environments for the entire astronomical community. The collaboration includes two institutions skilled in astrostatistics (Penn State and Carnegie Mellon) and an institution at the center of the NVO effort (California Institute of Technology). The participation by graduate students and postdocs give them a rare opportunity to develop skills needed for cross-disciplinary work.