Learning Distributed Patterns from Multimodal Streaming Data

人类的认知依赖于使用感觉系统学习、关联和识别数据模式，有效地存储这些模式，并将它们与新感知的数据进行匹配，以进行识别和决策。模式识别已经探索了几十年，但随着数以千计的互联数字设备开始生成诸如音频、视频、信号和文本等不同形式的TB级的流数据，新的挑战已经演变。这些数据必须实时并行摄取、处理和关联，以实现先进的数字感知和识别，从而创造机器智能。大公司和互联网云服务提供商今天面临着处理数据爆炸问题的严峻挑战。我们需要一个系统，能够并行地从多种类型的流数据中学习有用的信息，用于高级认知，存储知识，并在必要时再现数据流，同时丢弃不重要的数据以优化存储。这项拟议的研究将开发一个具有新型深度学习模型的实时流数据处理框架，以学习和关联分布式数据模式，以便为患者监控应用程序实现机器认知。该研究的创新之处在于：1)开发深度学习模型和流数据处理框架，从多模物联网、文本和视频数据中提取规则、不规则和异常模式，同时解决学习模式中称为概念漂移的变化；2)创建通用模式表示和配置文件技术，将数据模式存储为周期性事件日志中的数据配置文件，而不是海量原始数据，从而优化存储消耗并促进机器学习模型的开发，以从存储的配置文件中再现数据流；以及3)开发模式关联模型，将物联网、文本和视频数据模式关联起来，以提高机器认知。拟议的技术将作为开发各种智能认知系统的支柱，如患者监测和辅助系统，以确保加拿大人的福祉和提高生活质量，从而为加拿大经济做出贡献。

Human cognition depends on learning, correlating and recognizing data patterns using the sensory system, storing these patterns efficiently and matching them against newly perceived data for recognition and decision making. Pattern recognition has been explored for several decades but new challenges have evolved as thousands of connected digital devices have started generating terabytes of streaming data of different modalities such as audio, video, signal, and text. This data must be ingested, processed and correlated in parallel in real time for advanced digital perception and recognition to create machine intelligence. Large companies and internet cloud service providers are facing a serious challenge today to handle the data explosion problem. We need a system that can learn useful information from multiple types of streaming data in parallel for advanced cognition, store the knowledge, and reproduce the data streams as necessary while discarding unimportant data to optimize storage. The proposed research will develop a real time streaming data processing framework with novel deep learning models to learn and correlate distributed data patterns to enable machine cognition for a patient monitoring application. The novelty of the proposed research lies in 1) developing deep learning models and streaming data processing frameworks to extract regular, irregular and abnormal patterns from multimodal IoT, text and video data while addressing changes in learned patterns known as concept drifts, 2) creating a universal pattern representation and profiling technology to store data patterns as data profiles in a periodic event log rather than massive raw data, thus optimizing storage consumption and facilitating the development of machine learning models to reproduce data streams from the stored profiles, and 3) developing a pattern association model to correlate IoT, text and video data patterns for improved machine cognition. The proposed technology will serve as the backbone for developing a variety of intelligent cognitive systems such as patient monitoring and assistive systems to ensure well being and improve the quality of life of Canadians and thus contribute to Canada's economy.