Improving Prediction Efficacy through Abnormality Detection and Data Preprocessing.
Improving Prediction Efficacy through Abnormality Detection and Data Preprocessing.
复制标题
通过异常检测和数据预处理提高预测效率。
DOI：
10.1109/access.2019.2930257
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
Wang,Naisyin
中科院分区：
文献类型：
--
作者：
Tu,Chun-Chen;Chen,Pin-Yu;Wang,Naisyin
异常测试数据如果处理不当会严重降低模型性能。在本文中，我们提出了一个预处理系统来处理不同类型的常见异常测试数据。该系统由异常数据检测器和异常数据校正器组成。异常数据检测器负责对输入数据的类型进行分类。基于数据类型，异常数据修正器将采取不同的动作来修正测试数据。然后，用户可以将其首选的预测方法应用于校正后的测试数据。具体而言，损坏和对抗图像被用作异常数据的示例。我们表明，损坏的数据可以通过高斯局部线性映射方法进行重建，并且可以通过使用最近邻作为替代来提高对抗样本的预测性能。我们比较了现有的和公认的替代品的异常数据检测器和校正。这些方法是单独发布的，并没有把两个组件放在一起作为一个预处理系统。数值结果表明，我们提出的组件，独立，是有竞争力的。所提出的系统是一种通用的方法，可以应用于不同的下游预测模型。我们使用三个现有的预测方法来说明所提出的系统的一般用途和提高预测效率的能力。
Abnormal testing data can severely reduce model performance if not processed properly. In this paper, we propose a preprocessing system to handle different types of commonly seen abnormal testing data. The system consists of an aberrant data detector and an aberrant data corrector. The aberrant data detector is responsible for classifying the type of incoming data. Based on the data type, the aberrant data corrector will take different actions to amend testing data. Users can then apply their preferred prediction methods on the corrected testing data. Specifically, corrupted and adversarial images are used as examples of abnormal data. We show that corrupted data can be reconstructed through a Gaussian locally linear mappings method, and the prediction performance of adversarial samples can be improved by using the nearest neighbors as a surrogate. We compare the proposed aberrant data detector and corrector with existing and well-recognized alternatives. These approaches are published individually and do not put two components together as a pre-processing system. The numerical outcomes show that our proposed components, standing alone, are competitive. The proposed system is a generic method that can be applied to different downstream predictive models. We use three existing prediction methods to illustrate the general usage of the proposed system and its capability of improving prediction efficacy.