Messy Data Modeling and Related Topics

-----------------------------------------------------------------------提案编号：DMS 9803273 PI：谢明戈单位：罗格斯大学项目：凌乱数据建模及相关主题摘要：本研究的主要目的是探讨在许多科学学科中由于处理杂乱数据集而产生的一些问题。本研究中研究的数据违反了传统的假设，如独立性、同质性等，这些假设在更多的标准设置下被采用。提出了六个具体主题；每一种都至少对应一种违反常规假设的情况，而一个混乱的数据集可能有一种或多种违反常规假设的情况。根据它们来自两个不同的激励数据集，六个主题中的两个解决了群体测试方案（Dorfman, 1943）及其变体的问题，包括假阴性建模和放松对个体独立性的假设的问题。其余主题研究与批量相关回归数据建模相关的实践和理论问题，并为异质观测开发新的模型和方法。这些发展不仅将解决特定类型的问题，而且还将激发新的研究来开发更通用的方法。本研究发展统计方法、模型和相关理论，以解决混乱数据的建模和分析所产生的问题，这些问题可以在许多科学学科中找到，包括生命科学、环境科学、社会科学、工业和经济学。这些混乱数据的一个共同特征是，它们都违反了一些传统的模型假设，否则这些假设会在更标准的设置下被采用。对杂乱数据的精确建模可以消除不相关的信息，更好地理解潜在的机制；最终有利于预测和决策。尽管在过去的半个世纪里，复杂的统计方法和优雅的数学理论在发展方面取得了巨大的进步，但从实践和理论的角度来看，在混乱数据的建模和分析方面仍有许多重要的问题有待解决。本研究调查了几个这样的问题。虽然这些模型和方法是针对制药工业和环境科学中的具体问题量身定制的，但这些发展不仅将解决特定类型的问题，而且还将刺激新的研究以开发更通用的方法。

----------------------------------------------------------------------- Proposal Number: DMS 9803273 PI: Minge Xie Institution: Rutgers University Project: Messy Data Modeling and Related Topics Abstract: The main objective of this research is to investigate a number of problems arising from dealing with messy data sets that occurred in many disciplines of science. The data studied in this research violate conventional assumptions, such as independence, homogeneity, among others, which are otherwise adopted under more standard settings. Six specific topics are presented; each corresponds to at least one violation of conventional assumptions, and a messy data set may have one or more of these types of violations. According to their origins from two different motivating data sets, two of the six topics address problems on group testing scheme (Dorfman, 1943) and its variants, including issues on modeling false negatives and relaxing the assumption of independence on individuals. The rest of the topics investigate practical and theoretical issues related to modeling batch correlated regression data and develop new models and methods for heterogeneous observations. These developments will not only solve the specific type of problems, but also stimulate new researches to develop more general methodologies. This research develops statistical methodologies, models, and related theories to address issues arising from modeling and analysis of messy data, which can be found in many disciplines of the sciences, including life science, environmental science, social science, industry and economics. A common feature of these messy data is that they all violate some conventional model assumptions, which otherwise are adopted under more standard settings. Accurate modeling of messy data can eliminate irrelevant information and provide better understanding of underlying mechanisms; ultimately benefiting prediction and decision making. Although tremendous progress has been made in development of both sophisticated statistical methodologies and elegant mathematical theories in the past half century, many important problems in modeling and analysis of messy data have yet to be tackled, both from practical and theoretical viewpoints. This research investigates several such problems. Although the models and methodologies are tailored to specific problems in the Pharmaceutical industry and environmental science, these developments will not only solve the specific type of problems, but also stimulate new researches to develop more general methodologies.