Picket: guarding against corrupted data in tabular data during learning and inference
Picket: guarding against corrupted data in tabular data during learning and inference
复制标题
DOI：
10.1007/s00778-021-00699-w
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Zifan Liu;Zhechun Zhou;Theodoros Rekatsinas
中科院分区：
文献类型：
--
作者：
Zifan Liu;Zhechun Zhou;Theodoros Rekatsinas
数据损坏是现代机器学习部署的一个障碍。损坏的数据可能会严重影响学习模型，也可能导致无效的推断。Picket是一个简单的框架，可以在表格数据上训练和部署机器学习模型时防止数据损坏。在训练阶段，Picket从训练数据中识别并删除损坏的数据点，以避免获得有偏见的模型。对于部署阶段，Picket以在线方式标记损坏的查询指向一个经过训练的机器学习模型，由于噪声，该模型将导致不正确的预测。为了检测损坏的数据，Picket使用了一种用于混合类型表格数据的自监督深度学习模型，我们称之为PicketNet。为了最大限度地减少部署负担，学习PicketNet模型不需要任何人工标记的数据。Picket被设计为一个插件，可以增加任何机器学习管道的鲁棒性。我们在各种现实数据上评估Picket，考虑不同的腐败模型，包括训练和测试期间的系统性和对抗性噪声。我们表明，Picket在从SVM到神经网络的各种模型的训练和部署过程中始终能够防止数据损坏，击败了从数据质量验证模型到强大的离群值检测模型的各种竞争方法。
Data corruption is an impediment to modern machine learning deployments. Corrupted data can severely bias the learned model and can also lead to invalid inferences. We present, Picket, a simple framework to safeguard against data corruptions during both training and deployment of machine learning models over tabular data. For the training stage, Picket identifies and removes corrupted data points from the training data to avoid obtaining a biased model. For the deployment stage, Picket flags, in an online manner, corrupted query points to a trained machine learning model that due to noise will result in incorrect predictions. To detect corrupted data, Picket uses a self-supervised deep learning model for mixed-type tabular data, which we call PicketNet. To minimize the burden of deployment, learning a PicketNet model does not require any human-labeled data. Picket is designed as a plugin that can increase the robustness of any machine learning pipeline. We evaluate Picket on a diverse array of real-world data considering different corruption models that include systematic and adversarial noise during both training and testing. We show that Picket consistently safeguards against corrupted data during both training and deployment of various models ranging from SVMs to neural networks, beating a diverse array of competing methods that span from data quality validation models to robust outlier detection models.