Collaborative Research: Continuous-State Reinforcement Learning for Remanufacturing

该奖项将通过开发新的强化学习（人工智能的一个子领域）方法来解决再制造行业中出现的库存控制问题，为国家繁荣和美国制造业竞争力做出贡献。再制造是一种产品管理制造过程，旨在减少传统制造的能源消耗和碳足迹。有效的生产/库存管理是再制造行业成功的关键因素。然而，此类问题的复杂性和再制造过程的不确定性使得传统的生产计划方法难以适用。由此产生的算法和工具将使用从行业收集的真实数据进行全面测试，预计将大大节省原材料和能源资源，从而产生具有工业利益的实用管理政策。该项目将包括研究生和本科生的研究，并将案例研究纳入不同机构的高级课程中。这项研究将基于强化学习和仿真优化领域技术的融合。通过对仿真优化中最先进的方差缩减和函数逼近技术的新颖适应，pi将研究一类新的学习技术，特别是针对再制造决策问题。其中包括用于求解连续状态半马尔可夫决策过程的经典q学习的扩展，以及克服现有方法局部收敛性的更一般的无梯度行为关键型算法。将研究所开发的算法的理论特性，如收敛性和性能一致性，然后在基于真实世界数据的再制造仿真模型上进行评估和验证，以研究其实际影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award will contribute to the national prosperity and U.S. manufacturing competitiveness by developing new reinforcement learning (a subfield of artificial intelligence) methods to address inventory-control problems arising in remanufacturing industry. Remanufacturing is a product-management manufacturing process that aims to reduce the energy consumption and carbon footprint of traditional manufacturing. Effective production/inventory management to match the supply with the demand is a key element to the success of remanufacturing industry. However, the complexity of such problems and the uncertainties involved in the remanufacturing process make the conventional production planning methods difficult to apply. The resulting algorithms and tools will be fully tested using real-world data collected from the industry and are expected to achieve significant savings in raw materials and energy resources, leading to practical management policies of industrial interest. The PIs will involve both graduate and undergraduate students in this research and incorporate case studies into the advanced courses taught at different institutions.This research will be based on a fusion of techniques from reinforcement learning and the field of simulation optimization. Through novel adaptations of the-state-of-the-art variance reduction and function approximation techniques from simulation optimization, the PIs will investigate a new class of learning techniques especially tailored to remanufacturing decision-making problems. These include an extension of classical Q-learning for solving continuous-state semi-Markov decision processes and more general gradient-free actor-critic-like algorithms that overcome the local convergence of existing approaches. The algorithms developed will be studied for their theoretical properties such as convergence and performance consistency, and then assessed and validated on remanufacturing simulation models built on real-world data to investigate their practical impact.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.