Projected and semi-supervised clustering for high-dimensional data

点击翻译按钮获取中文摘要

Clustering is one of the major unsupervised data mining methods facing severe challenges when applied to today's high-dimensional data sets, which are collected on a large scale by automatic equipment (e.g. microarray chips, sensors, logging devices). The sparsity of the data, the small variance of distances in the full-dimensional space, and the inclusion of a large number of "irrelevant" or "random" dimensions make it typically impossible to detect a meaningful clustering structure using standard clustering algorithms with common full-dimensional (dis-)similarity measures. Meaningful structure can rather be detected by either considering lower-dimensional subspaces, or, by taking into account background knowledge if it is available (often as "must-link" or "cannot-link" constraints for a small subset of data points) to guide an algorithm to a certain clustering structure that is consistent with this information - overriding to some extent the information derived in the full-dimensional space. Main objectives of the proposed research program: 1) Advancement of the theoretical understanding of clustering methods applied to today's very high-dimensional data sets, particularly for the following relatively recent approaches "projected (or subspace-) clustering" and "semi-supervised clustering". 2) Development of novel and improved algorithms for projected and semi-supervised clustering, overcoming some of their current limitations, extending their applicability, and also combining the concepts of both for a wider range of application areas where such clustering methods can be useful. 3) Demonstration of the usefulness of the proposed methods on some real world data sets including gene expression data, text data, and medical image plus clinical data.