Online Improvement of Condition-Based Maintenance Policy via Monte Carlo Tree Search
Online Improvement of Condition-Based Maintenance Policy via Monte Carlo Tree Search
复制标题
DOI：
10.1109/tase.2021.3088603
复制
发表时间：
2021-06
影响因子：
5.6
通讯作者：
Michael Hoffman;Eunhye Song;Michael P. Brundage;S. Kumara
中科院分区：
文献类型：
--
作者：
Michael Hoffman;Eunhye Song;Michael P. Brundage;S. Kumara
在制造系统中，经常会出现维护需求超过维护资源能力的情况。这就导致了在竞争这些资源的机器之间分配有限资源的问题。这个维护调度问题可以表述为一个马尔可夫决策过程（MDP），其目标是在给定当前系统状态的情况下找到最优的动态维护措施。然而，随着系统变得更加复杂，求解MDP会受到维度灾难的影响。为了克服这个问题，我们提出了一种两阶段方法，首先使用遗传算法（GA）优化一个基于静态状态的维护（CBM）策略，然后通过蒙特卡洛树搜索（MCTS）在线改进该策略。静态策略通过允许我们忽略退化程度不足的机器，显著减少了在线问题的状态空间。此外，我们制定MCTS以寻求一个能使系统长期产量最大化的维护计划，从而调和维护和生产目标之间的冲突。我们证明，由此产生的在线策略比遗传算法找到的静态CBM策略有所改进。
给从业者的提示——本文提出了一种在维护资源竞争频繁的情况下，对复杂制造系统进行维护调度的方法。我们使用一种基于状态的维护策略，该策略根据机器的当前健康状况规定维护措施。然而，当多台机器需要维护时，维护技术人员必须在多个相互竞争的任务之间做出选择。虽然一种常见的方法是建立规定维护任务应如何排序的规则，比如先进先出规则，但这项工作的目标是实时改进静态策略。我们通过策略性地评估维护措施的顺序，并模拟许多“如果……会怎样”的情景，以了解系统在未来的表现来实现这一目标。所提出方法的实施依赖于构建目标系统的仿真模型。这个模型能够获取物理系统的当前状态，包括机器的退化状态、维护资源的可用性以及系统中各个缓冲区的零件分布。我们展示了几个仿真实验，证明了我们的方法对系统性能的改进。未来的工作将致力于通过在线学习提高维护优先级的效率，并更准确地识别能使这些方法产生最大效益的制造系统配置。
Often in manufacturing systems, scenarios arise where the demand for maintenance exceeds the capacity of maintenance resources. This results in the problem of allocating the limited resources among machines competing for them. This maintenance scheduling problem can be formulated as a Markov decision process (MDP) with the goal of finding the optimal dynamic maintenance action given the current system state. However, as the system becomes more complex, solving an MDP suffers from the curse of dimensionality. To overcome this issue, we propose a two-stage approach that first optimizes a static condition-based maintenance (CBM) policy using a genetic algorithm (GA) and then improves the policy online via Monte Carlo tree search (MCTS). The static policy significantly reduces the state space of the online problem by allowing us to ignore machines that are not sufficiently degraded. Furthermore, we formulate MCTS to seek a maintenance schedule that maximizes the long-term production volume of the system to reconcile the conflict between maintenance and production objectives. We demonstrate that the resulting online policy is an improvement over the static CBM policy found by GA. Note to Practitioners—This article proposes a method of scheduling maintenance in complex manufacturing systems in scenarios where there is frequent competition for maintenance resources. We use a condition-based maintenance policy that prescribes maintenance actions based on a machine’s current health. However, when several machines are due for maintenance, a maintenance technician must choose between multiple competing jobs. While a common approach is to establish rules that dictate how maintenance jobs should be prioritized, such as the first-in, first-out rule, the goal of this work is to improve upon static policies in real time. We do this by strategically evaluating sequences of maintenance actions and playing out many “what–if” scenarios to see how the system will behave in the future. Implementation of the proposed method relies on the construction of a simulation model of the target system. This model is capable of retrieving the current state of the physical system, including the degradation state of machines, the availability of maintenance resources, and the distribution of parts throughout buffers in the system. We present several simulation experiments that demonstrate the improvement in system performance that our approach provides. Future work will aim to improve the efficiency of maintenance prioritization through online learning as well as more accurately identify manufacturing system configurations that will yield the greatest benefit of these methods.