Sparse Deep Learning: A New Framework Immune to Local Traps and Miscalibration
Sparse Deep Learning: A New Framework Immune to Local Traps and Miscalibration
复制标题
DOI：
--
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Y. Sun;Wenjun Xiong;F. Liang
中科院分区：
文献类型：
--
作者：
Y. Sun;Wenjun Xiong;F. Liang
深度学习推动了人工智能(AI)最近的成功。然而，深度神经网络作为深度学习的基本模型，存在局部陷阱和误校等问题。在本文中，我们提供了一种新的稀疏深度学习框架，它以连贯的方式解决了上述问题。特别是为稀疏深度学习奠定了理论基础，提出了学习稀疏神经网络的先验退火法。前者成功地将稀疏深度神经网络引入统计建模框架，使预测不确定性得以正确量化。后者可以被渐近保证收敛到全局最优值，从而使得下游统计推断的有效性。数值结果表明，与已有方法相比，该方法具有一定的优越性。
Deep learning has powered recent successes of artificial intelligence (AI). However, the deep neural network, as the basic model of deep learning, has suffered from issues such as local traps and miscalibration. In this paper, we provide a new framework for sparse deep learning, which has the above issues addressed in a coherent way. In particular, we lay down a theoretical foundation for sparse deep learning and propose prior annealing algorithms for learning sparse neural networks. The former has successfully tamed the sparse deep neural network into the framework of statistical modeling, enabling prediction uncertainty correctly quantified. The latter can be asymptotically guaranteed to converge to the global optimum, enabling the validity of the down-stream statistical inference. Numerical result indicates the superiority of the proposed method compared to the existing ones.