High dimensional data clustering and pattern recognition

探索性统计在生物信息学、医学成像和药剂学等领域发挥着关键作用。从这些领域获得的数据是复杂、庞大和高维的。因此，大多数经典的统计方法对它们的分析是不够的。基于核的方法通常产生良好的结果。然而，它们有两个缺点：缺乏推理和过于一般化。前者的缺点与常见的统计问题有关，如模型选择、离群值检测和多重测试。后者与它们是通用方法有关，因此它们不能适应特定的数据点或变量关系。我们的长期目标是为考虑到上面提到的两个缺点的高维大数据集的探索性分析开发可靠的统计方法。我们的目标是开发模型：(A)寻找内在的数据结构（聚类），模式（例如变量选择）和异常（多个异常值）；同时(B)可以进行统计推断，(C)考虑到数据中的结构约束。短期目标任务是：(i)使用Potts类模型方法对高维数据进行聚类；（ii）以协商一致方式估计集群和集群数量；（iii）聚类内变量选择；(四)biclustering;(v)揭示变量之间的强关联。为了解释(B)，并从基于核的方法的良好性能中获利，我们寻求贝叶斯模型，或基于概率的模型，这些模型需要非参数核密度估计器和有效使用采样机制（例如MCMC）来生成感兴趣数量的样本。指导这项研究的一些应用是发现与某些疾病或肿瘤相关的基因群，通过与已知功能的蛋白质的关联发现蛋白质的作用，检测药物和不良反应之间的相关关联，以及发现大脑反应的模式。

Exploratory statistics plays a key role in fields such as bioinformatics, medical imaging and pharmaceutics. Data derived from these fields are complex, large and high-dimensional. Thus most classical statistical methods are inadequate for their analysis. Kernel-based methods often yield good results. However, they have two drawbacks: lack of inference, and too much generality. The former drawback is related to common statistical issues such as model selection, outlier detection, and multiple testing. The latter is related to their being general-purpose methods, and thus they cannot accommodate for specific data point or variable relations. Our long term goal is to develop sound statistical methodology for the exploratory analysis of high-dimensional large size data sets that take into account the two drawbacks mentioned above. We aim at developing models: (A) for finding intrinsic data structure (clustering), patterns (e.g. variable selection), and anomalies (multiple outliers); that at the same time (B) are amenable  for statistical inference, and (C) take into account structural constraints in the data. Short term tasks targeted are: (i) clustering of high-dimensional data using Potts model-like methods; (ii) estimating the clusters and the number of clusters by consensus; (iii) variable selection within clustering; (iv) biclustering; and (v) uncovering strong associations between variables. To account for (B), and to profit from the good performance of kernel-based methods, we seek Bayesian models, or probability-based models that entail non-parametric kernel density estimators and efficient use of sampling mechanisms (e.g. MCMC) to generate samples of the quantities of interest. Some applications guiding this research are discovering groups of genes associated to certain diseases or tumors, finding the role of proteins through association with those whose functions are known, detecting relevant association between drugs and adverse reactions, and discovering patterns in brain responses.