面向分布式IoT数据的流式清洗关键技术研究

随着信息通信技术的突飞猛进，物联网已经成为互联网的新发展，然而IoT数据在数据产生、收集、传递和解析过程中总是伴随着数据质量问题，为下游应用造成了很大的困扰。现有工作在集中式存储的时间序列数据、传统关系型数据的清洗方面进行了深入的研究，但是这些成果并没有充分考虑IoT数据具有的分布式，时间、空间、多维属性间相关联、要求高时效性的特点。因此本项目旨在研究面向分布式IoT数据的流式清洗关键技术，主要包括分布式环境下对IoT数据进行流式数据填充、异常检测、数据修复和相应的原型系统体系结构以及性能分析等方面。对于IoT数据的具体特点，本项目有针对性地提供有效实时的清洗技术。本项目的研究成果将为IoT数据的各类应用提供良好的数据质量保证，提升应用效果，具有较高的研究意义和应用价值。

With the rapid development of information and communication techniques, the Internet of Things (IoT) has become a new trend of the Internet. However, IoT data are often accompanied by data quality issues, which occur during data generation, collection, transmission, and parsing, causing great troubles for downstream applications. Existing work has studied the cleaning of centralized time series data and traditional relational data to a deep extend, but these results do not fully consider the distributed nature of IoT data as well as other characteristics such as the correlation between time, space and multi-dimensional attributes and the high requirements of the processing time..This project aims to study the key techniques of streaming cleaning for distributed IoT data, mainly including streaming data imputation, anomaly detection, data repair, and corresponding prototype system architecture, and performance analysis of IoT data in a distributed environment. For these specific characteristics of IoT data, it proposes effective and real-time cleaning techniques. The achievement of this project can provide high data quality assurance for various applications of IoT data and improve application effects, so it has high research significance and application value.