Statistical theory for Gaussian process function approximation based on theory of image restoration and its application

在图像恢复和函数逼近的框架中，通常假定加性噪声是互不相关的。我们提倡使用贝叶斯推理来恢复空间相关噪声条件下的噪声退化图像。得到了恢复图像的期望值，并得到了最优超参数。讨论了传统的空间不相关噪声模型能否处理空间相关噪声，发现它不能处理空间不相关噪声。进一步讨论了基于极大边缘似然法的超参数估计，发现求极大值的迭代算法不收敛。我们认为原因是模型中的奇点。因此，我们集中了具有奇异结构的层次模型的参数估计。利用两层神经网络，研究了在不同学习条件下，奇异性对标准梯度学习和自然梯度学习动态特性的影响。在标准梯度学习中，我们发现了一种准平台现象，在某些情况下，这种现象比已知的平台更为严重。当最优点位于奇异点附近时，拟平台和平台的缓慢收敛线索变得非常严重。而在自然梯度学习中，不存在拟平台和平台，收敛速度几乎不受奇点的影响。

It had been usually assumed that the additive noise does not correlated with each other in the frameworks of the image restoration and function approximation. We instigated the use of the Baysian inference to restore noise-degraded images under conditions of spatially correlated noise. We obtained the expected value of a restored image and obtained the optimal hyper-parameters. We discussed whether the conventional spatially uncorrelated noise model could cope with the spatially correlated noise or not, and found it could not. Furthermore, we discussed the hyper-parameter estimation based on the maximum marginalized likelihood method, and found an iterative algorithm for obtaining the maximum can not be converged. We thought the reason is singularity in the model. Thus, we concentrated the parameter estimation for hierarchical models with singular structure. Using two-layer neural networks, we investigate influences of singularities on dynamics of standard gradient learning and natural gradient learning under various learning conditions. In the standard gradient learning, we found a quasi-plateau phenomenon, which is severer than the well known plateau in some cases. The slow convergence clue to the quasi-plateau and plateau becomes extremely serious when an optimal point is in a neighborhood of a singularity. In the natural gradient learning, however, the quasi-plateau and plateau are not observed and convergence speed is hardly affected by singularity.