Derivative-Free Optimization: Algorithmic Developments, Software Design, Applications, and Machine Learning

无导数优化：算法开发、软件设计、应用程序和机器学习

• 批准号: RGPIN-2018-05286
• 负责人: LeDigabel, Sébastien
• 金额: $ 3.13万
• 依托单位: École Polytechnique de Montréal
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=759419
• 关键词: Derivative; Free; Optimization; Algorithmic; Developments

My research focuses on derivative-free optimization (DFO), which is essential in many engineering applications. More precisely, I am concerned with blackbox optimization, which occurs when the objective(s) and constraints of an engineering optimization problem are obtained by a computer code seen as a blackbox. Such blackboxes may be expensive to evaluate, may be contaminated with noise, and sometimes fail to return a value. In this context, I am considering the mesh adaptive direct search algorithm (MADS). This proposal is the continuation of my previous NSERC Discovery Grant and describes a five-year research program based on extensions of MADS and the links between DFO and machine learning (ML). It is divided into 12 projects that are well-suited for students, who will develop advanced skills in both optimization and ML.The program will address algorithm design and analysis, as well as software development and applications to various fields.The algorithmic projects will develop new tools that will enhance the solution of blackbox optimization problems. They include the introduction of new surrogate techniques based on ML; sensitivity analyses to identify and scale the most important variables; robust optimization methods for constrained noisy problems; and a new multiobjective algorithm based on the use of different measures of the quality of an approximate Pareto front. Another project involves ML to tune the MADS parameters. The new algorithmic features will be mathematically analyzed to prove convergence.A major part of my research is the design and dissemination of free software. In particular, the state-of-the-art NOMAD package that I have developed since 2008 is freely available under the LGPL licence at www.gerad.ca/nomad. NOMAD is in constant evolution, and the algorithmic developments will be integrated into the package by the students in charge of the projects with the help of two research associates funded from other sources. The proposed research will also provide new versions of the sgtelib library of surrogates, and two application codes for benchmarking purposes in the DFO community.The last component of my research concerns real engineering applications. For example, in the past, I worked on the optimization of alloy design, aircraft design, and energy. In this proposal, we will use applications in material science to test new surrogates; develop robust solutions for noisy mechanical engineering problems; consider electrical engineering problems with two or three objectives; develop a solar farm simulator that will include most of the typical blackbox-problem characteristics. Finally, we will apply DFO techniques to the optimization of the hyperparameters of deep neural networks.This proposal contributes to the training of 9 graduate and undergraduate students who will receive a multidisciplinary formation with fundamental and applied aspects.

我的研究重点是无导数优化（DFO），这在许多工程应用中是必不可少的。更准确地说，我关注的是黑盒优化，当工程优化问题的目标和约束由被视为黑盒的计算机代码获得时，就会发生这种情况。这种黑盒的计算成本可能很高，可能受到噪声的污染，有时无法返回值。在这种情况下，我正在考虑网格自适应直接搜索算法（MADS）。该提案是我之前的NSERC发现资助的延续，并描述了一个基于MADS扩展和DFO与机器学习（ML）之间联系的五年研究计划。它分为12个项目，非常适合学生，他们将发展优化和机器学习的高级技能，该计划将解决算法设计和分析，以及软件开发和应用到各个领域。算法项目将开发新的工具，以增强黑盒优化问题的解决方案。它们包括引入基于ML的新代理技术；敏感性分析，以确定和衡量最重要的变量；约束噪声问题的鲁棒优化方法以及一种新的多目标算法，该算法基于使用不同的近似帕累托前沿质量度量。另一个项目涉及ML来调优MADS参数。新的算法特征将被数学分析以证明收敛性。我研究的一个主要部分是自由软件的设计和传播。特别是，我自2008年以来开发的最先进的NOMAD包可以在www.gerad.ca/nomad的LGPL许可下免费获得。NOMAD正在不断发展，算法的发展将由负责项目的学生在其他来源资助的两名研究助理的帮助下整合到软件包中。拟议的研究还将提供sgtelib代理库的新版本，以及用于DFO社区基准测试的两个应用程序代码。我研究的最后一部分涉及实际工程应用。例如，在过去的工作中，我做过合金设计、飞机设计和能源的优化。在本提案中，我们将利用材料科学的应用来测试新的替代品；为嘈杂的机械工程问题提供可靠的解决方案；考虑有两个或三个目标的电气工程问题；开发一个太阳能农场模拟器，它将包括大多数典型的黑箱问题特征。最后，我们将DFO技术应用于深度神经网络的超参数优化。该提案有助于培养9名研究生和本科生，他们将接受基础和应用方面的多学科培训。