Mining High-Dimensional Event Sequences for Predictive Modelling

点击翻译按钮获取中文摘要

The internet continuously produces large flows of data and form events sequences which convey knowledge about individuals’ profiles, communities, opinions, influences, intentions, and trends. Similarly, in areas such as healthcare, business, finance, defense, event sequences are generated conveying also a great deal of knowledge that can be used for social benefits, business intelligence, public security and national defense. Mining massive and complex sequence data for predictive analytics is the main purpose of this program. The data types addressed are multisource, characterized by heterogeneity, significant noise and missing values, high-dimensional and strong interrelations between their attributes. The long-term goal of this research program is to build and validate novel mathematical frameworks for mining complex event sequences and developing predictive models. Such frameworks will be based on solid statistical theories to produce easily interpretable knowledge, to deal with a variety of sequence types, and to be efficient enough to deal with large and complex data. The new frameworks will distinguish themselves from the conventional models by 1) their optimized use of historical data for model building; 2) their identification of relevant variables for analytics models; 3) their discovery and use of relational patterns such as structural and causal relations; 4) their plan-library building and plan/activity recognition. This program will be carried out by accomplishing a number of interrelated projects. 1) Develop efficient algorithms for clustering very high-dimensional data for modeling and discovering semantic behaviour from sequences; 2) Develop efficient algorithms for mining sequence events and cluster trajectories with a latent representation to reduce the dimensionality and facilitate tracing. We will develop also new measures to deal with concept drift; 3) Develop efficient algorithms for classifying “big” sequence data, identifying relevant subspaces for each class and incorporating drift detection; 4) Develop new algorithms for discovering patterns and relations by variable-order Markov chains and sparse Markov techniques to optimize discovery of sequential information. 5) Design new algorithms for building plans of actions with the help of probabilistic graphic models. 6) Develop new CRF-based algorithms to anticipate actions for early identification of the goals/plans. We will also investigate the use of patterns of actions to build a more effective Cox proportional hazards model for survival analysis. An integrated platform to support the predictive modelling will be built on the supercomputer Mammouth, one of the most powerful computing machines in Canada, at the Université de Sherbrooke. This platform will serve as a test-bed not only for validating our methods but also for developing real-world applications.