Derivative-Free Optimization: Algorithmic Developments, Software Design, Applications, and Machine Learning

点击翻译按钮获取中文摘要

My research focuses on derivative-free optimization (DFO), which is essential in many engineering applications. More precisely, I am concerned with blackbox optimization, which occurs when the objective(s) and constraints of an engineering optimization problem are obtained by a computer code seen as a blackbox. Such blackboxes may be expensive to evaluate, may be contaminated with noise, and sometimes fail to return a value. In this context, I am considering the mesh adaptive direct search algorithm (MADS). This proposal is the continuation of my previous NSERC Discovery Grant and describes a five-year research program based on extensions of MADS and the links between DFO and machine learning (ML). It is divided into 12 projects that are well-suited for students, who will develop advanced skills in both optimization and ML. The program will address algorithm design and analysis, as well as software development and applications to various fields. The algorithmic projects will develop new tools that will enhance the solution of blackbox optimization problems. They include the introduction of new surrogate techniques based on ML; sensitivity analyses to identify and scale the most important variables; robust optimization methods for constrained noisy problems; and a new multiobjective algorithm based on the use of different measures of the quality of an approximate Pareto front. Another project involves ML to tune the MADS parameters. The new algorithmic features will be mathematically analyzed to prove convergence. A major part of my research is the design and dissemination of free software. In particular, the state-of-the-art NOMAD package that I have developed since 2008 is freely available under the LGPL licence at www.gerad.ca/nomad. NOMAD is in constant evolution, and the algorithmic developments will be integrated into the package by the students in charge of the projects with the help of two research associates funded from other sources. The proposed research will also provide new versions of the sgtelib library of surrogates, and two application codes for benchmarking purposes in the DFO community. The last component of my research concerns real engineering applications. For example, in the past, I worked on the optimization of alloy design, aircraft design, and energy. In this proposal, we will use applications in material science to test new surrogates; develop robust solutions for noisy mechanical engineering problems; consider electrical engineering problems with two or three objectives; develop a solar farm simulator that will include most of the typical blackbox-problem characteristics. Finally, we will apply DFO techniques to the optimization of the hyperparameters of deep neural networks. This proposal contributes to the training of 9 graduate and undergraduate students who will receive a multidisciplinary formation with fundamental and applied aspects.