大数据应用中非概率性抽样的评估及算法研究

非概率性抽样指样本选择受限条件下的抽样。尽管很多大数据计算方法要求独立同分布抽样（i.i.d.抽样），但是这一要求往往受到各种限制而难以满足。这些限制可能由各种主观或客观因素导致，如由于成本、隐私、存储环境等限制，部分样本难以访问，由于数据开放中信息保护等限制，部分属性的分布信息需要脱敏等，均限制了样本选择的独立性和随机性，导致非概率性抽样。为此，本项目开展非概率性抽样的评估及算法研究，具体包括：1）通过高维分布分解与假设检验构建高维大数据非概率性抽样的分布评估模型，实现抽样结果的分布检验；2）基于评估模型分析满足高维同分布约束的抽样下界与上界，指导抽样算法设计；3）样本约束及部分属性分布约束下的非概率性抽样算法设计，获取满足约束条件的最小抽样；4）集成前述成果开发高维大数据抽样的评估及算法工具包。本项目可为复杂环境下数据的共享和分析提供新的理论和工具支撑，提高数据共享利用效率。

Nonprobability sampling is to select samples under constraints. Although many big data analysis methods require independent identically distributed sampling (i.i.d. sampling), this requirement is hardly to meet due to various constraints. These constraints may be caused by objective or subjective factors, e.g. due to the limitation of cost, privacy, storage environments, some samples are hard to access; due to the limitation of information protection in data publication, the distributions of some features need to be masked. These factors restrict the independence and randomness for sample selection, resulting in nonprobability sampling. Therefore, this project conducts research on evaluation and algorithms for nonprobability sampling. First, we establish an evaluation model for nonprobability sampling by distribution decomposition and hypothesis testing to realize the distribution test for sampling results. Second, we analyze the upper boundary and the lower boundary under the identical distribution constraint for high dimensional data based on the evaluation model to guide the design of sampling algorithms. Third, nonprobability sampling algorithms are designed to find the minimum sampled subsets under samples constraints and partially distribution constraints. Fourth, we develop the corresponding tools for big data sampling. The project will provide novel theories and methods for data share and analysis under complex environment, which will improve the efficiency for data share and utilization.