Messy Data Modeling and Related Topics

-----------------------------------------------------------------------提案编号：DMS 9803273 PI：谢明明机构：罗格斯大学项目：杂乱数据建模及相关主题摘要：本研究的主要目的是调查在处理许多科学学科中出现的杂乱无章的数据集时出现了一些问题。这项研究中研究的数据违反了传统的假设，如独立性、同质性等，这些假设在其他更标准的环境下被采用。提出了六个特定的主题；每个主题对应于至少一个对传统假设的违反，而一个混乱的数据集可能有一个或多个这样的违反类型。根据它们来自两个不同的激励数据集，六个主题中的两个涉及团体测试方案(Dorfman，1943)及其变体的问题，包括建立假否定模型和放松对个人独立性的假设。其余的主题研究与批量相关回归数据建模相关的实践和理论问题，并为异质观测开发新的模型和方法。这些发展不仅将解决特定类型的问题，还将激励新的研究开发更一般的方法论。这项研究开发了统计方法、模型和相关理论，以解决在建模和分析杂乱数据时出现的问题，这些杂乱数据可以在许多科学学科中找到，包括生命科学、环境科学、社会科学、工业和经济学。这些混乱数据的一个共同特征是，它们都违反了一些传统的模型假设，否则这些假设就会在更标准的设置下被采用。对杂乱数据的准确建模可以消除不相关的信息，并提供对潜在机制的更好理解；最终有利于预测和决策。尽管在过去的半个世纪里，复杂的统计方法和优雅的数学理论的发展都取得了巨大的进步，但从实践和理论的角度来看，混乱数据的建模和分析中的许多重要问题仍有待解决。这项研究调查了几个这样的问题。虽然这些模型和方法是为制药工业和环境科学中的特定问题量身定做的，但这些发展不仅将解决特定类型的问题，而且还将激励新的研究开发更一般的方法。

----------------------------------------------------------------------- Proposal Number: DMS 9803273 PI: Minge Xie Institution: Rutgers University Project: Messy Data Modeling and Related Topics Abstract: The main objective of this research is to investigate a number of problems arising from dealing with messy data sets that occurred in many disciplines of science. The data studied in this research violate conventional assumptions, such as independence, homogeneity, among others, which are otherwise adopted under more standard settings. Six specific topics are presented; each corresponds to at least one violation of conventional assumptions, and a messy data set may have one or more of these types of violations. According to their origins from two different motivating data sets, two of the six topics address problems on group testing scheme (Dorfman, 1943) and its variants, including issues on modeling false negatives and relaxing the assumption of independence on individuals. The rest of the topics investigate practical and theoretical issues related to modeling batch correlated regression data and develop new models and methods for heterogeneous observations. These developments will not only solve the specific type of problems, but also stimulate new researches to develop more general methodologies. This research develops statistical methodologies, models, and related theories to address issues arising from modeling and analysis of messy data, which can be found in many disciplines of the sciences, including life science, environmental science, social science, industry and economics. A common feature of these messy data is that they all violate some conventional model assumptions, which otherwise are adopted under more standard settings. Accurate modeling of messy data can eliminate irrelevant information and provide better understanding of underlying mechanisms; ultimately benefiting prediction and decision making. Although tremendous progress has been made in development of both sophisticated statistical methodologies and elegant mathematical theories in the past half century, many important problems in modeling and analysis of messy data have yet to be tackled, both from practical and theoretical viewpoints. This research investigates several such problems. Although the models and methodologies are tailored to specific problems in the Pharmaceutical industry and environmental science, these developments will not only solve the specific type of problems, but also stimulate new researches to develop more general methodologies.