Learning Distributed Patterns from Multimodal Streaming Data

点击翻译按钮获取中文摘要

Human cognition depends on learning, correlating and recognizing data patterns using the sensory system, storing these patterns efficiently and matching them against newly perceived data for recognition and decision making. Pattern recognition has been explored for several decades but new challenges have evolved as thousands of connected digital devices have started generating terabytes of streaming data of different modalities such as audio, video, signal, and text. This data must be ingested, processed and correlated in parallel in real time for advanced digital perception and recognition to create machine intelligence. Large companies and internet cloud service providers are facing a serious challenge today to handle the data explosion problem. We need a system that can learn useful information from multiple types of streaming data in parallel for advanced cognition, store the knowledge, and reproduce the data streams as necessary while discarding unimportant data to optimize storage. The proposed research will develop a real time streaming data processing framework with novel deep learning models to learn and correlate distributed data patterns to enable machine cognition for a patient monitoring application. The novelty of the proposed research lies in 1) developing deep learning models and streaming data processing frameworks to extract regular, irregular and abnormal patterns from multimodal IoT, text and video data while addressing changes in learned patterns known as concept drifts, 2) creating a universal pattern representation and profiling technology to store data patterns as data profiles in a periodic event log rather than massive raw data, thus optimizing storage consumption and facilitating the development of machine learning models to reproduce data streams from the stored profiles, and 3) developing a pattern association model to correlate IoT, text and video data patterns for improved machine cognition. The proposed technology will serve as the backbone for developing a variety of intelligent cognitive systems such as patient monitoring and assistive systems to ensure well being and improve the quality of life of Canadians and thus contribute to Canada's economy.