Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity
Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity
复制标题
DOI：
10.48550/arxiv.2207.00751
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Jianyi Yang;Shaolei Ren
中科院分区：
文献类型：
--
作者：
Jianyi Yang;Shaolei Ren
通过将领域知识与标记样本相结合，信息机器学习已经出现，以提高广泛应用的学习性能。尽管如此，严格的理解注入领域知识的作用一直探索不足。在本文中，我们考虑了一个信息深度神经网络（DNN），其训练目标函数中集成了过参数化和领域知识，并研究了领域知识如何以及为什么有利于性能。具体地说，我们定量地证明了领域知识在知情学习中的两个好处-正规化基于标签的监督和补充标记的样本-并揭示了在人口风险的约束下标签和知识的不确定性之间的权衡。在理论分析的基础上，提出了一种广义的知情训练目标，以更好地利用知识的优势，平衡标签和知识的一致性，并通过种群风险界进行了验证。我们对采样复杂性的分析揭示了如何选择知情学习的超参数，并进一步证明了知识知情学习的优势。
By integrating domain knowledge with labeled samples, informed machine learning has been emerging to improve the learning performance for a wide range of applications. Nonetheless, rigorous understanding of the role of injected domain knowledge has been under-explored. In this paper, we consider an informed deep neural network (DNN) with over-parameterization and domain knowledge integrated into its training objective function, and study how and why domain knowledge benefits the performance. Concretely, we quantitatively demonstrate the two benefits of domain knowledge in informed learning - regularizing the label-based supervision and supplementing the labeled samples - and reveal the trade-off between label and knowledge imperfectness in the bound of the population risk. Based on the theoretical analysis, we propose a generalized informed training objective to better exploit the benefits of knowledge and balance the label and knowledge imperfectness, which is validated by the population risk bound. Our analysis on sampling complexity sheds lights on how to choose the hyper-parameters for informed learning, and further justifies the advantages of knowledge informed learning.