Derivative-free and blackbox optimization for engineering problems

这个研究项目是我以前的工作的延续，由NSERC资助的网格自适应直接搜索算法（MADS）的发展衍生自由和黑盒优化问题。 在许多工程优化问题中，目标函数和约束函数是非光滑的，并且通过计算机模拟来评估。这些代码的输出通常精度有限，受到数字噪音的污染，评估成本高昂，有时代码无法返回值。 在开发模型以提高直接搜索优化方法的效率方面已经有大量的工作。 二次模型、高斯过程、克里金插值以及径向基函数都是这样的例子。 还开展了工作，将能力扩大到整数、分类和周期变量，有效处理约束条件，解决双目标和多目标问题。 在本提案中，我们计划以多种方式继续开发MADS方法。 当前的MADS算法在称为网格的变量空间的离散化上生成试验点。 几个工程师已经应用MADS的实际问题抱怨说，网格尺寸参数更新的方式的后果导致修改的变量值低于合理的粒度。 我们计划改变网格尺寸参数的更新方式，这是MADS算法的核心，也是非光滑收敛分析的基石。 一些约束返回一个二进制值，指示是否满足，例如，指示模拟是否失败的标志。 这种二元约束难以使用上述建模技术进行建模。 我们计划从机器学习社区导入分类方法和数据挖掘工具来对其进行建模。 提出了求解双目标黑箱约束优化问题的BiMADS算法。 对于这样的问题，BiMADS返回一个有限的权衡解决方案列表，相对于两个相互冲突的目标。 我们计划再次使用分类工具来分析权衡解决方案列表的性质，并创建代表性的解决方案。 MADS使用代理作为真实模拟的廉价替代品。 MADS在决定在哪里启动真正的模拟之前，通过对代理进行多次采样来使用它们。 我们的数值实验系统地表明，代理人相当有助于找到更好的解决方案。 我们计划设计一个通用的框架来处理具有参数化精度的代理，并建立额外的模型来动态估计真函数和代理函数之间的差异。 本提案中列出的所有项目都将使用非光滑微积分进行严格分析。

This research project is the continuation of my prior work funded by NSERC on the development of the Mesh Adaptive Direct Search algorithm (MADS) for derivative-free and blackbox optimization problems.  In many engineering optimization problems, the objective and constraint functions are nonsmooth and evaluated through a computer simulation. The outputs of these codes are often of limited precision, contaminated with numerical noise, expensive to evaluate, and sometimes the code fails to return a value.  There has been an important amount of work in developing models to improve the efficiency of direct search optimization methods.  Quadratic models, Gaussian processes, kriging interpolation as well as radial basis functions are such examples.  There has also been work on expanding the capabilities to integer, categorical and periodic variables, on efficient treatment of constraints, on bi- and multi objective problems.  In the present proposal we plan to pursue the development of MADS methods in many ways. The current MADS algorithm generates trial points on a discretization of the space of variables called the mesh.  Several engineers that have applied MADS to practical problems complained that a consequence of the way that the mesh size parameter is updated leads to modifications in the values of the variables below a reasonable granularity.  We plan to alter the way that the mesh size parameter is updated, which is at the heart of the MADS algorithm, and the cornerstone of the nonsmooth convergence analysis.   Some constraints return a binary value indicating if it is satisfied or not, e.g., a flag indicating if the simulation failed or not.  Such a binary constraint is difficult to model using the above-mentioned modelling techniques.  We plan to import classification methods and data mining tools from the machine learning community to model them.  We proposed the BiMADS algorithm for biobjective blackbox constrained optimization.  For such problems, BiMADS returns a finite list of trade-off solutions, with respect to two conflicting objectives.  We plan to once again use classification tools to analyze the nature of the list of trade-off solutions, and to create representative solutions. Surrogates are used by MADS as an inexpensive substitute of the true simulation.  MADS uses them by sampling the surrogate several times before deciding where to launch the true simulation.  Our numerical experiments systematically show that surrogates considerably help in finding better solutions.  We plan to devise a generic framework to handle surrogates with parametrizable precision, and to build additional models to dynamically estimate the difference between the true and surrogate functions.    All the projects outlined in this proposal will be rigorously analyzed using nonsmooth calculus.