一类最优的子抽样设计及其统计分析方法

大数据时代，数据体量大幅增加，分析方法的计算复杂度显著提高，导致数据采集耗费的经济成本与日俱增，数据分析对于计算机存储与运算能力的要求也越来越高。当抽样成本与计算资源有限，或者对于数据分析的时效性要求较高时，子抽样方法可以有效地降低待分析数据体量，从而缩短分析时间，节省经济成本与计算资源。然而，子抽样因只使用原始数据的一部分，可能会影响数据分析方法的正确性与效率，因此，如何尽量优选数据、保证分析方法正确、减少效率损失是子抽样设计的重点。本项目提出一类新的子抽样设计思想，可以根据不同的数据分析目标对应地给出最优抽样设计，适用于多种常用的模型框架与不同的数据类型，对模型错误假定具备稳健性。除抽样设计外，本项目结合逆概率加权技术提出基于子抽样数据的统计分析方法，抽样设计的最优性使得新抽样方法在相同的抽样成本下渐近地达到最优效率。本项目的研究将丰富大规模数据处理的抽样与分析手段。

In the era of big data, with the rapid growth of data volume and significant raise of computation complexity, the economic cost of data collection is increasing rapidly and the requirement of computer storage and computation ability for data analysis is also getting higher and higher. When there is limited sampling cost and computation capacity, or when there is requirement for the speed of data analysis, sumsampling can help to reduce the data volume significantly, resulting in reduction in computational time and cost on data collection. However, since subsampling only use a fraction of the original data, it may affect the validity and efficiency of the analysis methods. Thus, it is quite important to consider how to optimally selection the data, to keep the validity of the methods, and to reduce the efficiency lose. In this project, we propose an optimal subsampling design which can adaptively give out optimal design according to different purpose of data analysis. The proposed design can be used in various model frameworks and different kinds of data. It possesses robustness against model misspecification. Beside the design itself, by incorporating the idea of inverse probability weighting, we propose the related statistical methods based on the subsample data. The optimality of the proposed method helps the design to reach the optimal efficiency asymptotically under the same sampling cost. The research of this project will enrich the sampling and data analysis techniques in dealing with large scale data.