GAMLS: a generalized framework for associative modular learning systems
GAMLS: a generalized framework for associative modular learning systems
复制标题
GAMLS：关联模块化学习系统的通用框架
DOI：
10.1117/12.342865
复制
发表时间：
1999
期刊：
影响因子：
--
通讯作者：
Joydeep Ghosh
中科院分区：
文献类型：
--
作者：
Shailesh Kumar;Joydeep Ghosh
学习大量简单的局部概念比学习单个全局概念更快更容易。受这种分而治之原则的启发，计算智能社区提出了许多模块化学习方法。在模块化学习中，分类/回归/聚类问题首先被分解为许多更简单的子问题，为每个子问题学习一个模块，最后通过合适的组合方法将其结果整合。专家和聚类的混合是在这个范例中可以描述的两种技术。在本文中，我们提出了一个广义联想模块化学习系统（GAMLS）的广泛框架。通过将每个训练模式与每个模块进行软关联来引入模块化。使用确定性退火迭代地解决学习模块参数和学习关联的耦合问题。从一个只有一个模块的高温开始，GAMLS框架通过系统的生长和修剪技术自动进化所需的模块数量。每个阶段开始时，将前一阶段中的每个模块分成两个，更新这些新模块，然后修剪和合并任何冗余模块。相变是由温度衰减引起的。一些现有的模块化学习问题，无监督（聚类，混合模型密度，主成分的混合）和监督（专家的混合，径向基函数网络），可以有效地解决在GAMLS。使用专家的混合物的聚类和回归的案例研究提供了一些数据集显示的效力，在不断发展的正确数量的模块，诱导模块之间的可解释的本地化和鲁棒性的解决方案的GAMLS框架。更重要的是，这个框架为理解和表征模块化学习方法提供了一个统一的视角。
Learning a large number of simple local concepts is both faster and easier than learning a single global concept. Inspired by this principle of divide and conquer, a number of modular learning approaches have been proposed by the computational intelligence community. In modular learning, the classification/regression/clustering problem is first decomposed into a number of simpler subproblems, a module is learned for each of these subproblems, and finally their results are integrated by a suitable combining method. Mixtures of experts and clustering are two of the techniques that are describable in this paradigm. In this paper we present a broad framework for Generalized Associative Modular Learning Systems (GAMLS). Modularity is introduced through soft association of each training pattern with every module. The coupled problems of learning the module parameters and learning associations are solved iteratively using deterministic annealing. Starting at a high temperature with only one module, GAMLS framework automatically evolves the required number of modules through a systematic growing and pruning technique. Each phase begins by splitting every module in the previous phase into two, updating these new modules and then pruning and merging any redundant modules. A phase transition is induced by temperature decay. A number of existing modular learning problems, both unsupervised (clustering, mixture model density, mixture of principal components) and supervised (mixture of experts, radial basis function networks), can be effectively tackled in GAMLS. Case studies for clustering and regression using mixture of experts are provided for a number of datasets showing the efficacy of the GAMLS framework in evolving the right number of modules, inducing interpretable localizations among modules and robustness of the solution obtained. More importantly, this framework provides a unifying view for understanding and characterizing modular learning methods.