Multivariate consensus trees: tree-based clustering and profiling for mixed data types
Multivariate consensus trees: tree-based clustering and profiling for mixed data types
复制标题
多元共识树：混合数据类型的基于树的聚类和分析
DOI：
--
复制
发表时间：
2006
期刊：
影响因子：
--
通讯作者：
Timothy Hancock
中科院分区：
文献类型：
--
作者：
Timothy Hancock
多变量分析的目的是在响应数据集中找到由
Multivariate profiling aims to find groups in a response dataset that are described by
relationships with another. Profiling is not predicting each variable within the
response set, but finding stable relationships between the two datasets that define
common groups. Profiling styles of analysis arise commonly within the context of
survey, experimental design and diagnosis type of studies. These studies produce
complex multivariate datasets that contain mixed variables often with missing values
that require analysis with a flexible, stable statistical technique.
The profiling model under consideration within this thesis is a Classification and
Regression Tree (CART). A standard CART model finds groups within a univariate
response by building a decision tree from a set of predictor variables. The flexible
structure of a CART model allow it to be used for either discriminate or regression
analysis whilst also catering for mixed types within the predictor set.
The goal of this thesis to develop methods that extend CART for a multivariate
response dataset involving mixed data types. Multivariate regression for CART
(MRT) has recently been shown to be a powerful profiling and clustering tool.
However the same successes in extending CART for multivariate classification and
multivariate mixed type analysis is yet to be realised. To begin with thesis explores
simple extensions to CART for multivariate mixed type analysis. These are binary
substitution of categorical variables within the response set and partitioning of a
distance matrix using Db-MRT. These techniques use already existing extensions to
CART methods and are used as comparison methods to gauge the performance of the
ensemble and consensus approaches that are the focus of this thesis.
Ensemble models using CART, such as random forests and treeboost, not only
improve the overall accuracy of the model predictions but also introduce an ensemble
proximity matrix as a measure of similarity between observations of the response set.
In this thesis, through MRT, extensions to both random forests and treeboost are
developed such that they predict a multivariate response. Furthermore, by binary
substitution of the categorical variables within the response set these multivariate
ensemble techniques are further extended to mixed type profiling. A result of this
extension is that the ensemble proximity matrix now describes the groups found
within the multivariate response. In this way multivariate tree-base ensembles can be
interpreted as a cluster ensemble method, where the ensemble proximity matrices can
be seen as cluster ensemble consensus matrices. In this thesis these proximity
matrices are found to be powerful visualisation tools providing improved resolution of
group structure found by a multivariate ensemble method. More so, as in cluster
ensembles using these matrices as an input in to a clustering method improves the
accuracy of the groups found.
The main work of this thesis is the development of the Multivariate Consensus
Tree (MCT) framework for mixed type profiling. Motivating the MCT approach
is the need to further understand which variables relate to the groups observed within
the proximity matrix. To do this MCTs describe three methods to intelligently
combine the ensemble proximity matrices of individual responses into one overall
consensus matrix. This consensus matrix is a summary of the overall group structure
within each individual proximity matrix. As MCTs work solely with proximity
matrices they are independent of the data types within the variables of the response
set. Furthermore as each response variable is explicitly predicted it is possible to
assess the quality of each proximity matrix in terms of predictive accuracy of the
corresponding ensemble.
The MCT consensus matrix is a visualisation tool for the groups present within both
the response and predictor datasets. As a consensus matrix is a similarity matrix this
thesis proposes five new splitting criteria for tree-based models that search for
decision rules within variables of the predictor set that partition the consensus matrix
into the observed groups. This tree provides a logical decision path that predicts each
group. As the groups within the response are now defined by their relationships
within the predictor set, the MCT profiling is complete. This thesis proposes two
algorithms for building an MCT; global MCTs and local MCTs. Global MCTs
construct an overall consensus matrix spanning all observations, and recursively
partition on this matrix to build the tree. Local MCTs build a new consensus matrix
at each terminal node to evaluate each new split.
As MCTs have the proximity matrices to summarise the group structure within each
response variable methods to identify important subgroups within these variables are
also proposed. This search for subgroups within the response can be done on two
levels. Firstly to identify subgroups of response variables for overall analysis; and
secondly to identify subsets of response variables within any specific group found by
the MCT. By finding subsets of response variables that relate to specific group
structure the understanding of structure within the dataset is greatly improved.
This thesis shows tree-based methods for profiling, in particular MCTs, to be a
powerful tool for mixed type analysis. Firstly, the visualisation of the tree, combined
with the proximity matrices, provide a unique view of the groups found and allow for
their easy interpretation within the context of the analysis. Secondly, MCTs are
shown to accurately estimate the number of groups and provide measures on their
stability and accuracy. Furthermore, MCTs are found to be resistant to noise
variables within the analysis. Finally they provide methods to find subgroups within
the response variables and to identify unimportant variables from the analysis.
Throughout this thesis these tree-based methods are compared with standard
clustering techniques to provide an accurate benchmark for their performance.