Data Smells: Categories, Causes and Consequences, and Detection of Suspicious Data in AI-based Systems
Data Smells: Categories, Causes and Consequences, and Detection of Suspicious Data in AI-based Systems
复制标题
数据异味：类别、原因和后果以及基于人工智能的系统中可疑数据的检测
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Rudolf Ramler
中科院分区：
文献类型：
--
作者：
Harald Foidl;M. Felderer;Rudolf Ramler
高数据质量是当今基于AI的系统的基础。然而，尽管几十年来数据质量一直是研究的对象，但显然缺乏对潜在数据质量问题的研究（例如，模糊的、无关的值）。这类问题本质上是潜在的，因此往往不明显。然而，它们可能与基于AI的系统中未来问题的风险增加有关（例如，技术债务、数据引起的故障）。作为软件工程中代码气味的对应物，我们将此类问题称为数据气味。本文对数据气味进行了概念化，并详细阐述了它们的原因、后果、检测以及在基于AI的系统中的使用。此外，36个数据气味的目录分为三类（即，可信性气味、可理解性气味、一致性气味）。此外，本文还概述了检测数据气味的工具支持，并展示了在240多个真实数据集上进行初始气味检测的结果。
High data quality is fundamental for today’s AI-based systems. However, although data quality has been an object of research for decades, there is a clear lack of research on potential data quality issues (e.g., ambiguous, extraneous values). These kinds of issues are latent in nature and thus often not obvious. Nevertheless, they can be associated with an increased risk of future problems in AI-based systems (e.g., technical debt, data-induced faults). As a counterpart to code smells in software engineering, we refer to such issues as Data Smells. This article conceptualizes data smells and elaborates on their causes, consequences, detection, and use in the context of AI-based systems. In addition, a catalogue of 36 data smells divided into three categories (i.e., Believability Smells, Understandability Smells, Consistency Smells) is presented. Moreover, the article outlines tool support for detecting data smells and presents the result of an initial smell detection on more than 240 real-world datasets.
DOI：
10.1109/ase51524.2021.9678520
发表时间：
2021-11
期刊：
2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)
影响因子：
--
作者：
Chenyang Yang;Shurui Zhou;Jin L. C. Guo;Christian Kästner
通讯作者：
Chenyang Yang;Shurui Zhou;Jin L. C. Guo;Christian Kästner