Confidentiality and Estimation for Large Sparse Multi-Dimensional Contingency Tables

本研究项目涉及处理大型稀疏列联表的两个关键方面：与其他研究人员共享数据时保护响应的机密性，以及对数线性模型中最大似然估计的稀疏性的含义。第一个问题涉及评价从分类数据库（例如以涉及变量子集的边缘表的形式）部分公布资料所涉及的披露风险。第二个问题是关于在对数线性模型分析中开发适用于稀疏分类数据的模型选择和估计/测试的通用推理方法。这些看似独立的问题之间的联系源于代数统计的共同统计和数学形式。这项研究将产生新的计算算法和可共享的计算机代码，供行为和社会科学研究人员使用，以及将使用最大似然和对数线性模型和保密保护的细胞估计问题联系起来的基础方法和理论。这项活动的预期结果将包括：(1)更有效的推理程序，用于定量分析和解释行为和社会科学数据，并确定披露的风险；(2)面向广大从业人员和研究人员的分类数据分析统计软件，这些软件将以计算机源代码和模块化可执行文件的形式开发和免费分发；对数线性模型分析形成了一套完善而强大的统计工具，用于研究分类数据，特别是以多维交叉分类或多向列联表的形式。这些模型已被证明对分析来自社会和行为科学许多领域的数据至关重要。在其他科学领域也是如此。例如，在一次典型的抽样调查中，为数千人就大量分类变量生成数据，测量就业、收入、健康状况等方面的信息。这些变量的交叉分类结果很大，即涉及成千上万个单元格，也很稀疏，即大多数单元格条目要么非常小，要么包含零计数。类似的问题也出现在社会网络的研究、公共卫生和医学以及基因数据库的分析中。代数几何数学领域的最新发展为与列联表数据相关的对数线性模型的表示提供了一种新颖而强大的形式。本项目将使用这种数学形式来关注大型稀疏列联表的两个不同方面：(1)在与其他用户共享数据时保护数据提供者的隐私，同时(2)通过开发对数线性模型计算的新方法来确保这些表对统计分析有用。该项目的结果将改善获取数据进行二次分析的机会，并提高研究人员和分析人员利用大型稀疏数据库中的信息的能力。

This research project deals with two crucial aspects of working with large sparse contingency tables: protecting the confidentiality of responses when data are shared with other researchers, and the implications of sparsity for maximum likelihood estimation in log-linear models. The first problem entails the evaluation of the disclosure risk associated with the partial release of information from a classified database, e.g., in the form of marginal tables involving subsets of variables. The second problem is concerned with developing general-purpose inferential methodologies for model selection and estimation/testing in log-linear model analysis that are appropriate for sparse categorical data. The links between these seemingly separate problems emanate from the common statistical and mathematical formalism of algebraic statistics. This research will produce new computational algorithms and sharable computer code for use by behavioral and social science researchers, as well as foundational methods and theory linking the problems of cell estimation using maximum likelihood and log-linear models and confidentiality protection. The expected outcomes of this activity will include: (1) more effective inferential procedures for the quantitative analysis and interpretation of behavioral and social science data and for the determination of the risk of disclosure; (2) statistical software for the analysis of categorical data targeted at a large audience of practitioners and researchers, which will be developed and freely distributed in the form of both computer source codes and modular, executable files; (3) more efficient numerical procedures for assessing the disclosure risk associated with the release of marginal totals.Log-linear models analysis forms a well-established and powerful set of statistical tools for the study of categorical data, especially in the form of multi-dimentional cross-classifications or multi-way contingency tables, These models have proved to be essential for the analysis of data emanating from many areas of the social and behavioral sciences, as well as in other scientific areas. For example, in a typical sample survey,data are generated for several thousand individuals on a large number of categorical variables, measuring such information on employment, income, health status, etc. The resulting cross-classification of these variables is large, i.e., involving many thousands of cells, and sparse, i.e., most of the cell entries are either very small or contain zero counts. Similar problems arise in the study of social networks, in public health and medicine, and in the analysis of genetics databases. Recent developments in the mathematical area of algebraic geometry have provided a novel and powerful formalism for the representation of log-linear models relevant for such contingency table data. This project will use this mathematical formalism to focus on two different aspects of large sparse contingency tables: (1) Protecting the privacy of the data providers when data are shared with other users, while at the same time (2) Ensuring that such tables are useful for statistical analysis by developing new methods for log-linear model computation. The results of the project will improve access to data for secondary analysis and enhance the capacity of researchers and analysts to exploit the information in large sparse databases.