Data Lifecycle Challenges in Production Machine Learning: A Survey
Data Lifecycle Challenges in Production Machine Learning: A Survey
复制标题
DOI：
10.1145/3299887.3299891
复制
发表时间：
2018-06-01
期刊：
影响因子：
1.1
通讯作者：
Zinkevich, Martin
中科院分区：
文献类型：
--
作者：
Polyzotis, Neoklis;Roy, Sudip;Zinkevich, Martin
机器学习已成为从数据中收集知识并解决各种计算艰巨任务的重要工具。但是，机器学习模型的准确性与训练的数据深深相关。设计和构建可靠的过程和工具，使得更容易分析，验证和转换数据，这些数据被供大型机器学习系统带来数据管理挑战。从我们为生产机器学习平台开发以数据为中心的基础架构的经验中在Google，我们总结了我们遇到的一些有趣的研究挑战，并调查了数据管理和机器学习社区的一些相关文献。具体来说，我们探讨了重点数据理解，数据验证和清洁以及数据准备的三个主要领域的挑战。在这些领域的每个领域中，我们都试图探讨如何根据模型的生命周期中的位置对解决方案施加不同的限制，这些问题是遇到问题并遇到的。
Machine learning has become an essential tool for gleaning knowledge from data and tackling a diverse set of computationally hard tasks. However, the accuracy of a machine learned model is deeply tied to the data that it is trained on. Designing and building robust processes and tools that make it easier to analyze, validate, and transform data that is fed into large-scale machine learning systems poses data management challenges.Drawn from our experience in developing data-centric infrastructure for a production machine learning platform at Google, we summarize some of the interesting research challenges that we encountered, and survey some of the relevant literature from the data management and machine learning communities. Specifically, we explore challenges in three main areas of focus data understanding, data validation and cleaning, and data preparation. In each of these areas, we try to explore how different constraints are imposed on the solutions depending on where in the lifecycle of a model the problems are encountered and who encounters them.