A Novel Deep Learning Model by Stacking Conditional Restricted Boltzmann Machine and Deep Neural Network
A Novel Deep Learning Model by Stacking Conditional Restricted Boltzmann Machine and Deep Neural Network
复制标题
DOI：
10.1145/3394486.3403184
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Tianyu Kang;Ping Chen;John Quackenbush;Wei Ding
中科院分区：
文献类型：
--
作者：
Tianyu Kang;Ping Chen;John Quackenbush;Wei Ding
现实世界的系统经常表现出由其子单元之间的相互作用产生的复杂动态。在机器学习和数据挖掘中，这些相互作用通常被表述为系统变量之间的依赖性和相关性。与处理空间相关特征的卷积神经网络和处理时间相关特征的循环神经网络类似，在本文中，我们提出了一种新颖的深度学习模型，通过堆叠条件限制玻尔兹曼机和深度神经网络（CRBM-DNN）来处理功能交互特征。变量及其依赖关系被组织成二分图，该二分图进一步转换为以领域知识为条件的受限玻尔兹曼机。我们将此 CRBM 和 DNN 集成到一个受总体成本函数约束的深度学习模型中。 CRBM-DNN 可以解决监督和无监督学习问题。与相同大小的常规神经网络相比，CRBM-DNN 的参数较少，因此需要较少的训练样本。我们使用几个具有挑战性的现实世界数据集对大量监督学习和无监督学习方法进行了广泛的比较研究，并取得了显着的优越性能。
A real-world system often exhibits complex dynamics arising from interaction among its subunits. In machine learning and data mining, these interactions are usually formulated as dependency and correlation among system variables. Similar to Convolution Neural Network dealing with spatially correlated features and Recurrent Neural Network with temporally correlated features, in this paper we present a novel deep learning model to tackle functionally interactive features by stacking a Conditional Restricted Boltzmann Machine and a Deep Neural Network (CRBM-DNN). Variables with their dependency relationships are organized into a bipartite graph, which is further converted into a Restricted Boltzmann Machine conditioned by domain knowledge. We integrate this CRBM and a DNN into one deep learning model constrained by one overall cost function. CRBM-DNN can solve both supervised and unsupervised learning problems. Compared to a regular neural network of the same size, CRBM-DNN has fewer parameters so they require fewer training samples. We perform extensive comparative studies with a large number of supervised learning and unsupervised learning methods using several challenging real-world datasets, and achieve significant superior performance.