Robust modal regression with direct gradient approximation of modal regression risk
Robust modal regression with direct gradient approximation of modal regression risk
复制标题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Hiroaki Sasaki;Tomoya Sakai;T. Kanamori
中科院分区：
文献类型：
--
作者：
Hiroaki Sasaki;Tomoya Sakai;T. Kanamori
模态回归旨在估计全局模态（即，全局最大值）的条件密度函数的输出变量给定的输入变量，并导致回归方法对广泛的噪声鲁棒。典型的模态回归方法采用两步方法，首先近似模态回归风险（MRR），然后用某种梯度方法最大化近似的MRR。然而，这种两步方法在基于梯度的最大化方法中可能是次优的，因为良好的MRR近似器不一定给出MRR的良好梯度近似器。在本文中，我们采取了一种新的方法，直接逼近的梯度MRR的模态回归。基于直接方法，我们首先提出了一种再生核的模态回归方法，其中一个新的更新规则来估计的条件模式是基于固定点方法。然后，推导出的更新规则的理论研究。此外，由于我们的直接方法与最近复杂的随机梯度方法（例如，Adam），提出了另一种基于神经网络的模态回归方法。最后，所提出的方法的上级性能表现在各种人工和基准数据集。
Modal regression is aimed at estimating the global mode (i.e., global maximum) of the conditional density function of the output variable given input variables, and has led to regression methods robust against a wide-range of noises. A typical approach for modal regression takes a two-step approach of firstly approximating the modal regression risk (MRR) and of secondly maximizing the approximated MRR with some gradient method. However, this two-step approach can be suboptimal in gradient-based maximization methods because a good MRR approximator does not necessarily give a good gradient approximator of MRR. In this paper, we take a novel approach of directly approximating the gradient of MRR in modal regression. Based on the direct approach, we first propose a modal regression method with reproducing kernels where a new update rule to estimate the conditional mode is derived based on a fixed-point method. Then, the derived update rule is theoretically investigated. Furthermore, since our direct approach is compatible with recent sophisticated stochastic gradient methods (e.g., Adam), another modal regression method is also proposed based on neural networks. Finally, the superior performance of the proposed methods is demonstrated on various artificial and benchmark datasets.