面向大规模机器学习问题的立方正则化算法研究
结题报告
批准号:
12001367
项目类别:
青年科学基金项目
资助金额:
24.0 万元
负责人:
王浩
依托单位:
学科分类:
连续优化
结题年份:
2023
批准年份:
2020
项目状态:
已结题
项目参与者:
王浩
国基评审专家1V1指导 中标率高出同行96.8%
结合最新热点,提供专业选题建议
深度指导申报书撰写,确保创新可行
指导项目中标800+,快速提高中标率
客服二维码
微信扫码咨询
中文摘要
机器学习问题以其规模巨大、高度非凸等特点,给科学计算带来了巨大的挑战,同时也倒逼优化理论的创新。立方正则化算法是在牛顿类算法的基础上催生出的能够有效地利用更高阶信息逃离鞍点的一类高效求解算法。本项目以降低计算成本、改善模型解的适定性、设计快速高效的求解算法,以满足机器学习问题的实际需求为出发点,利用随机逼近理论和分布式计算框架,构建带有二阶随机逼近的分布式正则化算法,并对算法进行收敛性分析,以达到计算成本和求解质量之间的平衡,克服目前已有的立方正则化算法不能有效求解大规模非凸问题的缺陷。进一步地,项目拟引进分布式共识机制,以期将现有的理论结果推广到带约束的大规模非凸优化问题中。最后,利用机器学习的实际应用数据集对算法理论进行实证研究,检验模型的合理性及算法的有效性。本项目的研究成果有助于提高我国在机器学习应用领域以及大规模非凸优化方面的研究水平,提供全新的研究视角和理论支撑。
英文摘要
Machine learning problems have become a challenging task for scientific computing due to its nature of large-scale and severely nonconvexity, yet also created great need for the improvement on optimization theory and methods. Cubic regularization algorithms are a class of novel and efficient machine learning algorithms, which use higher-order derivative information compared with Newton methods and can escape from the saddle points. The primary focus of this project is to alleviate the computational burden brought by large-scale machine learning problems, improve the generalization of the solution, and design rapid and efficient optimization algorithms for solving practical machine learning problems. To achieve this, we plan to construct second-order stochastic and/or distributed cubic regularization algorithms using stochastic approximation theory and the framework of distributed methods. Furthermore, we aim to provide solid convergence analysis to investigate the balance between computational cost and the quality of the solution found by the proposed algorithms. Our purpose of this project is to extend the practicability of the current cubic regularization to large-scale nonconvex machine learning problems. Moreover, this project also investigates the possibility of extend the current distributed algorithmic framework to constrained large-scale nonconvex problems. Our proposed methods will be tested on the real machine learning problems, which can provide adequate proof of the practicability and efficiency of our proposed methods.
期刊论文列表
专著列表
科研奖励列表
会议论文列表
专利列表
DOI:--
发表时间:2021-01
期刊:J. Mach. Learn. Res.
影响因子:--
作者:Xiangyu Yang;Jiashan Wang;Hao Wang
通讯作者:Xiangyu Yang;Jiashan Wang;Hao Wang
DOI:10.1007/s11590-022-01907-4
发表时间:2020-07
期刊:Optim. Lett.
影响因子:--
作者:Hao Wang-;Hao Zeng;Jiashan Wang
通讯作者:Hao Wang-;Hao Zeng;Jiashan Wang
DOI:10.1007/s10589-022-00416-5
发表时间:2021-01
期刊:Comput. Optim. Appl.
影响因子:--
作者:Hongya Wang;Hao Zeng;Jiashan Wang
通讯作者:Hongya Wang;Hao Zeng;Jiashan Wang
DOI:10.1109/tcns.2021.3070663
发表时间:2020-11
期刊:IEEE Transactions on Control of Network Systems
影响因子:4.2
作者:Hejie Wei;Zhihai Qu;Xuyang Wu;Hao Wang-;Jie Lu
通讯作者:Hejie Wei;Zhihai Qu;Xuyang Wu;Hao Wang-;Jie Lu
Nonconvex and Nonsmooth Sparse Optimization via Adaptively Iterative Reweighted Methods
通过自适应迭代重加权方法进行非凸和非光滑稀疏优化
DOI:10.1007/s10898-021-01093-0
发表时间:2018-10
期刊:Journal of Global Optimization
影响因子:1.8
作者:Wang Hao;Zhang Fan;Shi Yuanming;Hu Yaohua
通讯作者:Hu Yaohua
大规模非凸正则化机器学习的高性能算法研究
  • 批准号:
    21ZR1442800
  • 项目类别:
    省市级项目
  • 资助金额:
    0.0万元
  • 批准年份:
    2021
  • 负责人:
    王浩
  • 依托单位:
国内基金
海外基金