Robust mixture of experts modeling using the t distribution
Robust mixture of experts modeling using the t distribution
复制标题
DOI：
10.1016/j.neunet.2016.03.002
复制
发表时间：
2016-07
期刊：
影响因子：
--
通讯作者：
Faicel Chamroukhi
中科院分区：
文献类型：
--
作者：
Faicel Chamroukhi
摘要专家混合模型（莫伊）是一个流行的模型框架，用于对数据的异质性进行回归、分类和聚类。对于连续数据的回归和聚类分析，莫伊通常使用遵循高斯分布的正态专家。然而，对于包含一组或多组具有重尾或非典型观测值的观测值的数据集，使用正常专家是不合适的，并且可能不适当地影响莫伊模型的拟合。我们介绍了一个强大的莫伊建模使用t分布。建议的t莫伊（TMoE）处理这些问题的重尾和噪声数据。我们开发了一个专用的期望最大化（EM）算法来估计所提出的模型的参数，通过单调最大化所观察到的数据对数似然。我们描述了如何提出的模型可以用于预测和基于模型的聚类回归数据。仿真数据的数值实验表明，该模型在非线性回归函数建模和基于模型的聚类方面具有较好的有效性和鲁棒性。然后，将其应用于真实世界的音调感知数据进行音乐数据分析，并将其应用于温度异常数据进行气候变化数据分析。所得到的结果表明TMoE模型在实际应用中的实用性。
Abstract Mixture of Experts (MoE) is a popular framework for modeling heterogeneity in data for regression, classification, and clustering. For regression and cluster analyses of continuous data, MoE usually uses normal experts following the Gaussian distribution. However, for a set of data containing a group or groups of observations with heavy tails or atypical observations, the use of normal experts is unsuitable and can unduly affect the fit of the MoE model. We introduce a robust MoE modeling using the t distribution. The proposed t MoE (TMoE) deals with these issues regarding heavy-tailed and noisy data. We develop a dedicated expectation–maximization (EM) algorithm to estimate the parameters of the proposed model by monotonically maximizing the observed data log-likelihood. We describe how the presented model can be used in prediction and in model-based clustering of regression data. The proposed model is validated on numerical experiments carried out on simulated data, which show the effectiveness and the robustness of the proposed model in terms of modeling non-linear regression functions as well as in model-based clustering. Then, it is applied to the real-world data of tone perception for musical data analysis, and the one of temperature anomalies for the analysis of climate change data. The obtained results show the usefulness of the TMoE model for practical applications.