Monte Carlo and Quasi-Monte Carlo Methods for Statistics

该项目有两个主要组成部分。 第一个是调查的可视化和分析方法的数据集在高维度，重点是分类变量的独特水平的数量是可比的总样本量。这些变量的示例包括搜索查询字符串、ISBN、歌曲标题、作者姓名、URL、基因型、环境和客户ID号。 可视化方法旨在显示广泛的趋势和突出异常。推理方法是样本重用类型：自助法和交叉验证。新的方法是必要的，因为数据集有复杂的连锁模式，无效的任何IID抽样假设。第二个组成部分是更好的统计推断，通过改进他们的数值方法。 这包括校准经验似然方法，以获得更好的覆盖范围，并将平均值的置信区域扩展到数据点的凸船体之外。 它还包括准蒙特卡罗抽样方法嵌入到马尔可夫链蒙特卡罗算法联合收割机结合前者的准确性和后者的广泛适用性。分类变量的探索性数据分析是有用的，看到广泛的模式，包括小群体的客户有类似的口味，一个小名单的歌曲或书籍或电影。 识别可能表明滥用行为的异常也很有用，包括网络攻击和在线环境中通常称为垃圾邮件的内容。样品再利用方法的最初动机之一是在作物科学中。 在这些问题中的一些中，大量的植物品种（基因型）在许多不同的环境条件下生长。 使用统计模型来确定在每个环境中使用哪些品种。 早期的统计方法是基于不适合这种设置的假设，他们往往没有选择最好的模型。 因此，该项目的新方法可用于选择更好的模型，从而增加食物和纤维的产量。经验似然工作是基础研究，旨在从统计模型中删除不必要的数学假设，以扩大其适用性。该项目的蒙特卡罗抽样部分是对物理学中广泛使用的计算技术以及贝叶斯统计推断的基础研究。

This project has two major components. The first is an investigation of visualization and analysis methods for data sets in high dimensions, with a focus on categorical variables whose number of unique levels is comparable to the total sample size. Examples of such variables include search query strings, ISBNs, song titles, author names, URLs, genotypes, environments, and customer ID numbers. The visualization methods are designed to show broad trends and to highlight anomalies. The inferential methods are of the sample reuse type: the bootstrap and cross-validation. New methods are necessary here because the data sets have complicated interlocking patterns that invalidate any IID sampling assumptions. The second component is better statistical inference by improving on their numerical methods. This includes calibration of empirical likelihood methods to get better coverage and to extend confidence regions for the mean beyond the convex hull of the data points. It also includes the embedding of quasi-Monte Carlo sampling methods into Markov chain Monte Carlo algorithms to combine the accuracy of the former and the wide applicability of the latter.Exploratory data analysis of categorical variables is useful to see broad patterns including small groups of customers that have similar tastes for a small list of songs or books or movies. It is also useful to identify anomalies that may indicate abusive behavior, including cyber-attack, and what is commonly called spam in the online context. One of the original motivations for the sample reuse methods is in crop science. In some of those problems, a large number of plant varieties (genotypes) are grown under many different environmental conditions. A statistical model is used to determine which varieties to use in each environment. Earlier statistical methods were based on assumptions that don't fit this setting and they often did not select the best model. New methods from this project may therefore be used to select better models which then result in increased production of food and fiber. The empirical likelihood work is basic research aimed at removing unnecessary mathematical assumptions from statistical models in order to widen their applicability. The Monte Carlo sampling component of the project is basic research on a computational technique used extensively in physics as well Bayesian statistical inference.