Topics in Predictive and Descriptive Data Mining

摘要PI：Jerry FriedmanDMS-0204029该提案寻求对预测和描述性数据挖掘（DM）研究的支持。决策树方法是最流行的预测性DM工具。根据这项补助金的研究将探讨如何克服其最严重的限制：严重过度拟合的分类（析因）预测变量的存在非常大的数值（因素）。聚类分析是描述性数据挖掘的常用工具。在大多数DM应用程序中，每次观察都会测量大量变量。通常，聚类（如果存在的话）只发生在所有测量变量的未知子集（通常很小）内。此外，个体聚类可以表示（可能重叠的）可变子集上的分组。我们的目标是确定聚类组以及每个优先聚类的特定变量子集。传统的聚类算法不适合这项任务。该基金的研究将探索解决这一问题的新方法，特别是在有大量可测量变量的情况下。数据挖掘用于发现数据中的模式和关系，重点是非常大的数据库。它对商业、工业、科学、医学以及最近的国土安全产生了重大影响。数据挖掘活动分为两种类型：预测性和描述性。预测性DM涉及使用来自系统的过去观测数据来构建该系统的数学模型。该模型用于预测系统的某些未来未知属性（属性或变量），给定未来将已知的其他属性。描述性数据挖掘试图构建紧凑的、可解释的数据摘要，以理解模式和关系，而不关注特定属性的预测。本研究将探讨新的方法，以增加权力的描述性和预测DM的问题，他们一直是传统上薄弱。

AbstractPI: Jerry FriedmanDMS-0204029This proposal seeks support for research in predictive and descriptive data mining (DM). Decision tree methods are the most popular predictive DM tools. Research under this grant will investigate ways to overcome their most serious limitation: severe over fitting in the presence of categorical (factorial) predictor variables with very large numbers of values (factors). Cluster analysis is often used as a tool in descriptive DM. In most DM applications a large number of variables are measured on each observation. Usually clustering, if it exists, occurs only within (often small) unknown subsets of all the measured variables. Moreover, individual clusters may represent groupings on (possibly overlapping) variable subsets. The goal is to identify the clustered groups as well as the particular variable subsets on which each one preferentially clusters. Traditional clustering algorithms are not well suited for this task. Research under this grant will investigate new approaches for solving this problem, especially in situations where there are a very large number of measured variables.Data mining is used to discover patterns and relationships in data, with an emphasis on very large data bases. It has had a major impact in business, industry, science, medicine, and most recently homeland security. Data mining activities divide into two types: predictive and descriptive. Predictive DM involves using past observational data from a system to build a mathematical model of that system. The model is used to predict some future unknown property (attribute or variable) of the system, given other properties that will be known in the future. Descriptive DM seeks to construct compact, interpretable summaries of the data in order to understand patterns and relationships, without focusing on the prediction of particular attributes. This research will investigate new methodologies for increasing the power of both descriptive and predictive DM in problems for which they have been traditionally weak.