High dimensional data clustering and pattern recognition

点击翻译按钮获取中文摘要

Exploratory statistics plays a key role in fields such as bioinformatics, medical imaging and pharmaceutics. Data derived from these fields are complex, large and high-dimensional. Thus most classical statistical methods are inadequate for their analysis. Kernel-based methods often yield good results. However, they have two drawbacks: lack of inference, and too much generality. The former drawback is related to common statistical issues such as model selection, outlier detection, and multiple testing. The latter is related to their being general-purpose methods, and thus they cannot accommodate for specific data point or variable relations. Our long term goal is to develop sound statistical methodology for the exploratory analysis of high-dimensional large size data sets that take into account the two drawbacks mentioned above. We aim at developing models: (A) for finding intrinsic data structure (clustering), patterns (e.g. variable selection), and anomalies (multiple outliers); that at the same time (B) are amenable  for statistical inference, and (C) take into account structural constraints in the data. Short term tasks targeted are: (i) clustering of high-dimensional data using Potts model-like methods; (ii) estimating the clusters and the number of clusters by consensus; (iii) variable selection within clustering; (iv) biclustering; and (v) uncovering strong associations between variables. To account for (B), and to profit from the good performance of kernel-based methods, we seek Bayesian models, or probability-based models that entail non-parametric kernel density estimators and efficient use of sampling mechanisms (e.g. MCMC) to generate samples of the quantities of interest. Some applications guiding this research are discovering groups of genes associated to certain diseases or tumors, finding the role of proteins through association with those whose functions are known, detecting relevant association between drugs and adverse reactions, and discovering patterns in brain responses.