Actor-Critic-Like Stochastic Adaptive Search Algorithms for Simulation Optimization

在工程设计、制造和医疗保健等应用中出现的许多系统都需要使用仿真优化技术来提高其性能。然而，尽管近年来取得了重大进展，仿真优化仍然是一个具有许多理论和实践挑战的领域。本研究项目旨在通过研究一种新的方法来扩展该领域的现有知识，该方法将强化学习（人工智能的一个子领域）的理论和工具集成到一类称为基于模型的方法的自适应搜索算法中，以解决模拟优化问题。由于这些方法的通用性，所产生的技术将在广泛的工业和科学领域具有广泛的适用性。特别是，通过与电力工程师的合作，开发的算法将被测试并应用于电力系统中的电压控制问题，这可能使公用事业公司和能源消费者受益。该研究计划将与工程专业学生的教育和培训紧密结合，将新的发展纳入研究员教授的研究生课程，并招募女性和代表性不足的少数民族学生参加该项目。这将通过探索基于模型的方法和基于策略梯度的学习算法之间的联系来实现。具体来说，研究人员将研究如何在强化学习框架中使用Actor-Critic算法的见解，以有效地减少基于模型的方法的采样方差。如果成功，该方法将在基于模型的优化设置中集成函数逼近技术，以提供具有低方差性能估计的算法，从而搜索改进的解决方案。这项研究可能会改变这些算法的实现和应用的方式，导致更快，更有效的算法来解决广泛的优化问题，特别是在需要昂贵的功能评估或性能估计模拟的设置。

Many systems arising in applications from engineering design, manufacturing, and health care require the use of simulation optimization techniques to improve their performance. However, despite significant progress in recent years, simulation optimization remains an area with many theoretical and practical challenges. This research project aims to expand the current knowledge in this field by investigating a novel approach that integrates theories and tools from reinforcement learning (a subarea of artificial intelligence) within a class of adaptive search algorithms called the model-based methods to solve simulation optimization problems. Because of the generality of these methodologies, the resulting techniques will have broad applicability in a wide array of industry and science sectors. In particular, through collaboration with power engineers, the developed algorithms will be tested and applied to voltage control problems in electric power systems, potentially benefiting both utility companies and energy consumers. The research plan will be closely integrated with the education and training of students in engineering by incorporating new developments into the graduate courses the investigator teaches and recruiting female and underrepresented minority students to the project.The goal of this research is to advance theoretical underpinnings of new model-based algorithms that can be orders of magnitude more efficient than the state-of-the-art. This will be accomplished by exploring the connections between model-based methods and policy gradient-based reinforcement-learning algorithms. Specifically, the investigator will examine how to use the insights from actor-critic algorithms in the reinforcement learning framework to effectively reduce the sampling variance of model-based methods. If successful, the approach will integrate function approximation techniques within a model-based optimization setting to provide algorithms with low-variance performance estimates in searching for improved solutions. This research may change the manner in which these algorithms are implemented and applied, leading to faster and more efficient algorithms for solving a broad class of optimization problems, especially in settings that require expensive function evaluations or simulations for performance estimation.