Optimization and game dynamics in machine learning

机器学习中的优化和博弈动力学

• 批准号: RGPIN-2019-06512
• 负责人: Mitliagkas, Ioannis
• 金额: $ 1.68万
• 依托单位: Université de Montréal
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=739483
• 关键词: Optimization; game; dynamics; machine; learning

This proposal focuses on the investigation of dynamics and acceleration in the context of numerical methods for optimization and games in ML. There is vast literature on optimization and specifically on optimization for machine learning (ML). However, there are a number of fundamental questions that are still unanswered. Those unanswered questions surprisingly include fundamental properties of widely used methods, the interaction of acceleration and stochasticity, the dynamics of games, and the connection between acceleration, adaptivity and generalization. A consequence of this lack of understanding is that we do not fully benefit from existing methods, and further ignore the possibility of improved methods. I am proposing fundamental research on optimization and game dynamics, as they pertain to modern challenges in machine learning and deep learning. The work includes a deeper study of classic momentum dynamics, emphasizing on some previously unknown and very curious properties; a simple approach to accelerated stochastic optimization; the study of acceleration and stochasticity on game dynamics; and an investigation of the generalization abilities of adaptive optimization methods with emphases on the lens of dynamics.

该提案重点研究机器学习优化和博弈数值方法背景下的动力学和加速度。有大量关于优化，特别是机器学习 (ML) 优化的文献。然而，仍有许多基本问题尚未得到解答。令人惊讶的是，这些悬而未决的问题包括广泛使用的方法的基本属性、加速和随机性的相互作用、游戏的动态以及加速、适应性和泛化之间的联系。缺乏理解的后果是我们没有充分受益于现有方法，并进一步忽视了改进方法的可能性。我提议对优化和游戏动力学进行基础研究，因为它们涉及机器学习和深度学习的现代挑战。这项工作包括对经典动量动力学的深入研究，强调一些以前未知且非常好奇的特性；加速随机优化的简单方法；博弈动力学的加速度和随机性研究；并研究了自适应优化方法的泛化能力，重点是动力学视角。