Evolutionary Computing: Constraints, Surrogate Models, and Noisy Gradients

黑盒优化是解决优化问题的任务，其中查询目标函数的值是了解问题的唯一方法。黑箱问题发生在许多领域，例如，在需要运行模拟或构建原型以评估解决方案质量的情况下。目标函数不能解析地指定，不能做出关于连续性或平滑性的假设，并且函数值的观测可能是噪声。梯度近似有时可以通过有限差分获得，但可能无法证明获得它们所产生的成本是合理的。 进化算法是一种求解黑箱优化问题的随机算法。该研究计划的目标是在三个方面促进EA的发展：约束优化，代理模型辅助优化和基于噪声梯度的进化搜索。在所有情况下，设计将通过系统地研究可扩展单元测试问题的算法行为来了解。在发展中的一个主要问题将是保存理想的不变性。拟议的工作将有利于用户的EA，因为它将导致更有能力的算法黑盒优化。它将大大扩大环境影响评估可以有益地适用的问题的范围。基于噪声梯度的进化搜索策略在机器学习中具有潜在的广泛应用，其中通常用于训练神经网络的随机梯度下降的变体不具有期望的不变性，并且通常需要仔细调整参数才能成功。这项工作还将产生几个HQP与上级解决问题的技能和独特的专业知识，在现代随机黑箱优化技术的开发和应用，以及在机器学习和实验的设计和分析方面。

Black-box optimization is the task of solving optimization problems where querying the value of the objective function is the only way of learning about the problem. Black-box problems occur in many areas, as for example in cases where simulations need to be run or prototypes be built in order to assess the quality of a solution. The objective function cannot be specified analytically, no assumptions regarding continuity or smoothness can be made, and observations of function values may be noisy. Gradient approximations can sometimes be obtained through finite differencing, but may not justify the cost incurred in obtaining them. Evolutionary algorithms (EAs) are stochastic algorithms for black-box optimization. The objective of this program of research is to contribute to the development of EAs in three areas: constrained optimization, surrogate model assisted optimization, and evolutionary search based on noisy gradients. In all cases, designs will be informed by systematically studying algorithm behaviour on scalable unit test problems. A primary concern in the development will be the preservation of desirable invariance properties. The proposed work will benefit users of EAs in that it will result in more capable algorithms for black-box optimization. It will significantly expand the range of problems that EAs can beneficially be applied to. Evolutionary search strategies based on noisy gradients have potentially wide ranging applications in machine learning, where variants of stochastic gradient descent that are commonly used for training neural networks do not possess desirable invariance properties and often require the careful tuning of parameters in order to be successful. The work will also result in several HQP with superior problem solving skills and unique expertise in the development and application of modern stochastic black-box optimization techniques as well as in aspects of machine learning and the design and analysis of experiments.