Nearest neighbor regression in the presence of bad hubs
Nearest neighbor regression in the presence of bad hubs
复制标题
DOI：
10.1016/j.knosys.2015.06.010
复制
发表时间：
2015-09-01
影响因子：
8.8
通讯作者：
Nagy, Gabor
中科院分区：
文献类型：
--
作者：
Buza, Krisztian;Nanopoulos, Alexandros;Nagy, Gabor
回归预测是机器学习中最重要的任务之一，在金融、医学、社会科学和自然科学等领域有着广泛的应用。由于它的简单性、理论性能保证和实际应用的成功，最流行的回归技术之一是k近邻回归。然而，k近邻方法会受到坏中心的存在的影响，这是最近观察到的一种现象，根据这种现象，一些实例与令人惊讶的许多其他实例相似，并且对整体预测性能具有不利影响。本文首次从回归的角度对不良枢纽进行研究。我们提出了具有Hubness感知的最近邻回归方案。我们在来自不同领域的公开可用的真实世界数据集上评估我们的方法。结果表明，所提出的方法优于其他各种回归方法，如KNN回归、回归树和神经网络。我们还评估了在存在标签噪声的情况下提出的方法，因为从现实应用的角度来看，对噪声的容忍度是最相关的方面之一。特别是，我们在传统的高斯标签噪声和最近提出的Hubness比例随机标签噪声的改进版本的假设下进行了实验。(C)2015爱思唯尔B.V.保留所有权利。
Prediction on a numeric scale, i.e., regression, is one of the most prominent machine learning tasks with various applications in finance, medicine, social and natural sciences. Due to its simplicity, theoretical performance guarantees and successful real-world applications, one of the most popular regression techniques is the k nearest neighbor regression. However, k nearest neighbor approaches are affected by the presence of bad hubs, a recently observed phenomenon according to which some of the instances are similar to surprisingly many other instances and have a detrimental effect on the overall prediction performance. This paper is the first to study bad hubs in context of regression. We propose hubness-aware nearest neighbor regression schemes. We evaluate our approaches on publicly available real-world data-sets from various domains. Our results show that the proposed approaches outperform various other regressions schemes such as kNN regression, regression trees and neural networks. We also evaluate the proposed approaches in the presence of label noise because tolerance to noise is one of the most relevant aspects from the point of view of real-world applications. In particular, we perform experiments under the assumption of conventional Gaussian label noise and an adapted version of the recently proposed hubness-proportional random label noise. (C) 2015 Elsevier B.V. All rights reserved.