Learning in the presence of change: challenges and algorithms for data stream mining

点击翻译按钮获取中文摘要

Data streams as produced by sensor networks, customer click streams, scientific data are ubiquitous in our society, and extracting knowledge from these fast-evolving repositories is one of the most significant challenges that we face today. The principal objective of the proposed research is to develop novel algorithms for learning in such non-stationary environments. Specifically, the proposed work falls within the area of adaptive machine learning and focuses on the development of pro-active, incremental algorithms for mining multiple concepts in fast-evolving data streams. Application areas of this work are numerous and are related to the Internet of Things (IoT), including personalized routing in urban planning for smarter cities, intrusion detection in computer networking, fraud detection in financial institutions and the insurance industry, and social media analysis. Our aim is to design resource-aware algorithms that construct accurate, reliable models that adapt seamlessly to the ebb and flow of data streams. In recent years research has focused on the development of adaptive algorithms capable of learning from evolving data streams. Typically, these algorithms learn one instance at a time and in that manner can detect, and adapt to, changes. However, there are vital research questions that remain unanswered. New solutions are needed to learn from incomplete data and to handle multiple, concurrent heterogeneous changes in data distributions. Methods that recognise the current relevant subset of features are lacking. In a fraud-detection application, for instance, techniques are needed to detect when a person's age ceases to be a relevant indicator. It is still not clear how to build accurate models against infrequently appearing concepts. This class imbalance problem remains an open challenge, especially in environments such as fault detection in engineering systems. Machine learning from dynamically evolving repositories still presents our research community with many daunting challenges. To address this demand, the core of this proposed research will focus on five interrelated research questions, related to a) the velocity and volume of data arrival, b) the handling of heterogeneous changes in data distributions, c) the detection of emerging concepts and features, d) learning in the presence of highly skewed data distributions, and e) the modeling of imperfect and incomplete data.