CAREER: Data Quality Management through Statistical Quality Control and Data Mining

由于大量数据的出现，数据质量问题变得非常重要。许多商业和工业应用程序严重依赖于存储在不同数据库和数据仓库中的信息的质量。本研究的目标是开发一种系统的数据质量分析和改进方法，以实现不完美信息环境下的稳健决策。该项目为数据质量评估和评价制定了一个统一的框架，提供了通过信息制作和管理提高数据质量的切实可行的解决方案，并通过维护一个网站来传播研究成果，以提高学术界和工业界专业人员对信息质量的认识。该方法包括开发贝叶斯网络模型，以捕捉数据质量指标之间的相互关系，应用统计抽样方案和数据挖掘方法进行数据质量评估，并推广统计技术的根本原因识别和数据质量改进。该技术进行了评估和验证，使用合成的例子和现实生活中的情况下，从电信和信息技术（IT）行业。预计该项目的成果将是通用的，并为数据质量管理提供具体的基础，可适用于不同的数据密集型应用程序。该项目将对信息质量管理的先进理论和方法、增强决策以及创建数据质量保证研究人员和从业人员队伍产生广泛影响。从这一研究项目中获得的知识和成果将通过互联网（http：//www.stevens.edu/engineering/research/projects/DataQuality.html）、会议、讲习班和各级课程广泛传播。

Data quality problem is of great importance due to the emergence of large volumes of data. Many business and industrial applications critically rely on the quality of information stored in diverse databases and data warehouses. The goal of this research project is to develop a systematic methodology of data quality analysis and improvement to achieve robust decision making under imperfect information environments. The project develops a unified framework for data quality assessment and evaluation, deliveries practical solutions to improve data quality through information production and management, and disseminates research findings by maintaining a website to increase the awareness of information quality among academic and industrial professionals. The approach consists of developing Bayesian network models to capture inter-relationships between data quality metrics, applying statistical sampling schemes and data mining methods for data quality assessment, and generalizing statistical techniques for root cause identification and data quality improvement. The techniques are evaluated and validated using synthetic examples and real-life cases from telecommunication and information technologies (IT) industries. The outcomes of the project are expected to be generic and provide a concrete basis of data quality management that can be applied to different data-intensive applications. The project will have broad impacts on advanced theory and methodology of information quality management, enhanced decision making, and the creation of a workforce of data quality assurance researchers and practitioners. Knowledge gained and results obtained from this research project will be broadly disseminated via Internet (http://www.stevens.edu/engineering/seem/Research/projects/DataQuality.html), in conferences, workshops, and various levels of courses.