Statistical theory for Gaussian process function approximation based on theory of image restoration and its application

在图像复原和函数逼近的框架下，通常假定加性噪声彼此不相关。我们提出使用贝叶斯推理来恢复空间相关噪声条件下的降噪图像。我们得到了恢复图像的期望值，并得到了最优的超参数。我们讨论了传统的空间不相关噪声模型是否能够处理空间相关噪声，发现它不能。在此基础上，讨论了基于最大边际似然方法的超参数估计，发现求最大值的迭代算法是不收敛的。我们认为原因在于模型中的奇异性。因此，我们集中研究了具有奇异结构的分层模型的参数估计问题。利用两层神经网络，研究了在不同学习条件下，奇异性对标准梯度学习和自然梯度学习动力学的影响。在标准的梯度学习中，我们发现了一个准平台现象，在某些情况下，它比众所周知的平台更严重。当最优点在奇点附近时，准平台点和平台点的缓慢收敛线索变得非常严重。然而，在自然梯度学习中，没有观察到准平台和平台，收敛速度几乎不受奇异性的影响。

It had been usually assumed that the additive noise does not correlated with each other in the frameworks of the image restoration and function approximation. We instigated the use of the Baysian inference to restore noise-degraded images under conditions of spatially correlated noise. We obtained the expected value of a restored image and obtained the optimal hyper-parameters. We discussed whether the conventional spatially uncorrelated noise model could cope with the spatially correlated noise or not, and found it could not. Furthermore, we discussed the hyper-parameter estimation based on the maximum marginalized likelihood method, and found an iterative algorithm for obtaining the maximum can not be converged. We thought the reason is singularity in the model. Thus, we concentrated the parameter estimation for hierarchical models with singular structure. Using two-layer neural networks, we investigate influences of singularities on dynamics of standard gradient learning and natural gradient learning under various learning conditions. In the standard gradient learning, we found a quasi-plateau phenomenon, which is severer than the well known plateau in some cases. The slow convergence clue to the quasi-plateau and plateau becomes extremely serious when an optimal point is in a neighborhood of a singularity. In the natural gradient learning, however, the quasi-plateau and plateau are not observed and convergence speed is hardly affected by singularity.