Learning Distributed Patterns from Multimodal Streaming Data

人类认知依赖于使用感觉系统学习、关联和识别数据模式，有效地存储这些模式，并将其与新感知的数据进行匹配，以进行识别和决策。模式识别已经探索了几十年，但随着数千个连接的数字设备开始生成TB级的不同模态（如音频、视频、信号和文本）的流数据，新的挑战也随之而来。这些数据必须在真实的时间内并行摄取、处理和关联，以实现先进的数字感知和识别，从而创造机器智能。如今，大公司和互联网云服务提供商在处理数据爆炸问题方面面临着严峻的挑战。我们需要一个系统，可以从多种类型的流数据中并行学习有用的信息，以实现高级认知，存储知识，并在必要时重现数据流，同时丢弃不重要的数据以优化存储。拟议的研究将开发一个具有新型深度学习模型的真实的实时流数据处理框架，以学习和关联分布式数据模式，从而实现患者监测应用的机器认知。拟议研究的新奇在于：1）开发深度学习模型和流数据处理框架，以从多模态物联网，文本和视频数据中提取规则，不规则和异常模式，同时解决被称为概念漂移的学习模式的变化，2）创建通用模式表示和分析技术，以将数据模式存储为周期性事件日志中的数据配置文件，而不是大量原始数据，从而优化存储消耗并促进机器学习模型的开发以从存储的简档再现数据流，以及3）开发模式关联模型以关联IoT、文本和视频数据模式以用于改进的机器认知。该技术将成为开发各种智能认知系统的支柱，如患者监测和辅助系统，以确保加拿大人的福祉和提高生活质量，从而为加拿大经济做出贡献。

Human cognition depends on learning, correlating and recognizing data patterns using the sensory system, storing these patterns efficiently and matching them against newly perceived data for recognition and decision making. Pattern recognition has been explored for several decades but new challenges have evolved as thousands of connected digital devices have started generating terabytes of streaming data of different modalities such as audio, video, signal, and text. This data must be ingested, processed and correlated in parallel in real time for advanced digital perception and recognition to create machine intelligence. Large companies and internet cloud service providers are facing a serious challenge today to handle the data explosion problem. We need a system that can learn useful information from multiple types of streaming data in parallel for advanced cognition, store the knowledge, and reproduce the data streams as necessary while discarding unimportant data to optimize storage. The proposed research will develop a real time streaming data processing framework with novel deep learning models to learn and correlate distributed data patterns to enable machine cognition for a patient monitoring application. The novelty of the proposed research lies in 1) developing deep learning models and streaming data processing frameworks to extract regular, irregular and abnormal patterns from multimodal IoT, text and video data while addressing changes in learned patterns known as concept drifts, 2) creating a universal pattern representation and profiling technology to store data patterns as data profiles in a periodic event log rather than massive raw data, thus optimizing storage consumption and facilitating the development of machine learning models to reproduce data streams from the stored profiles, and 3) developing a pattern association model to correlate IoT, text and video data patterns for improved machine cognition. The proposed technology will serve as the backbone for developing a variety of intelligent cognitive systems such as patient monitoring and assistive systems to ensure well being and improve the quality of life of Canadians and thus contribute to Canada's economy.