权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CIF: SMALL: kNN methods for functional estimation and machine learning

CIF：SMALL：用于功能估计和机器学习的 kNN 方法

基本信息

批准号：
2112504
负责人：
Lifeng Lai
金额：
$ 50万
依托单位：
University of California-Davis
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2112504&HistoricalAwards=false
关键词：
CIF SMALL kNN methods functional

项目摘要

K Nearest Neighbor (kNN) methods are a class of nonparametric statistical methods. Compared with other methods, kNN methods have several advantages. In particular, kNN methods can automatically adapt to any continuous underlying functions without relying on any specific models. In addition, kNN methods are simple to use and do not require too much parameter tuning. Furthermore, kNN methods have achieved excellent empirical results. Due to these advantages, kNN methods are widely used in a large variety of statistical problems, including functional-estimation and machine-learning problems. However, the understanding of theoretical properties of kNN methods for these applications is incomplete. As the result, there is a pressing need to investigate the theoretical properties of kNN methods. By addressing the main sources of estimation errors identified by these investigations, one can design improved kNN methods that have better performance.In this project, the investigator is investigating: 1) theoretical properties of kNN methods in functional estimation and machine learning problems; and 2) the design of improved kNN algorithms with better performance for these applications. Despite many existing studies, several theoretical problems still need further investigation. In particular: 1) The theoretical convergence rates of kNN methods for functional estimations, classification and regression, etc., are still not fully established; 2) For many applications of practical interests, it is not clear under what conditions this type of methods is optimal; 3) Most of the existing analysis of kNN methods rely on availability of independent and identically distributed training data, while in certain applications (such as those involving Markov chains) the available data are dependent; 4) While there are many applications and analysis of kNN methods for supervised learning, the applications and analysis of kNN methods for reinforcement learning etc. are limited. To address these challenges, this project is focusing on two interconnected thrusts. In the first thrust, the project is investigating the application of kNN methods in the estimation of information-theoretic functionals, including entropy, mutual information, Kullback-Leibler divergence, directed information, etc. In the second thrust, the project is designing and analyzing kNN based algorithms for machine learning problems, including supervised learning, nonconvex optimization and reinforcement learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

K近邻方法（kNN）是一类非参数统计方法。与其他方法相比，kNN方法具有许多优点。特别是，kNN方法可以自动适应任何连续的底层函数，而不依赖于任何特定的模型。此外，kNN方法使用简单，不需要太多的参数调整。此外，kNN方法取得了良好的实证结果。由于这些优点，kNN方法被广泛用于各种统计问题，包括函数估计和机器学习问题。然而，这些应用的kNN方法的理论特性的理解是不完整的。因此，迫切需要研究kNN方法的理论特性。通过解决这些调查确定的估计误差的主要来源，可以设计改进的kNN方法，具有更好的performance.In本项目中，研究者正在调查：1）kNN方法在函数估计和机器学习问题的理论特性;和2）改进的kNN算法的设计，这些应用程序具有更好的性能。尽管已有许多研究，但仍有一些理论问题需要进一步研究。特别是：1）kNN方法用于函数估计、分类和回归等的理论收敛速度，仍然没有完全建立; 2）对于许多实际应用，不清楚在什么条件下这种类型的方法是最优的; 3）大多数现有的kNN方法的分析依赖于独立和同分布的训练数据的可用性，而在某些应用中，（如涉及马尔可夫链的那些）可用的数据是依赖的; 4）虽然kNN方法在监督学习中有很多应用和分析，但kNN方法在强化学习等方面的应用和分析有限。为了应对这些挑战，该项目侧重于两个相互关联的重点。在第一个推力中，该项目正在研究kNN方法在信息论泛函估计中的应用，包括熵，互信息，Kullback-Leibler散度，有向信息等。在第二个推力中，该项目正在设计和分析基于kNN的机器学习问题算法，包括监督学习，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。