Collaborative Research: Continuous-State Reinforcement Learning for Remanufacturing

该奖项将通过开发新的强化学习(人工智能的一个子领域)方法来解决再制造业中出现的库存控制问题，从而为国家繁荣和美国制造业竞争力做出贡献。再制造是一种产品管理制造过程，旨在减少传统制造的能源消耗和碳足迹。有效的生产/库存管理，使供需相匹配是再制造行业成功的关键因素。然而，这些问题的复杂性和再制造过程中涉及的不确定性使得传统的生产计划方法难以应用。由此产生的算法和工具将使用从行业收集的真实世界数据进行全面测试，预计将在原材料和能源资源方面实现显著节省，从而制定符合行业利益的实际管理政策。PIS将包括研究生和本科生参与这项研究，并将案例研究纳入到不同机构教授的高级课程中。这项研究将基于强化学习和模拟优化领域的技术的融合。通过对模拟优化中最先进的方差减少和函数逼近技术的新颖改编，PI将研究一种专门针对再制造决策问题的新的学习技术。其中包括用于求解连续状态半马尔可夫决策过程的经典Q-学习的扩展，以及克服现有方法的局部收敛的更一般的无梯度演员-批评者类算法。将研究开发的算法的理论属性，如收敛和性能一致性，然后在基于真实世界数据的再制造模拟模型上进行评估和验证，以调查其实际影响。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This award will contribute to the national prosperity and U.S. manufacturing competitiveness by developing new reinforcement learning (a subfield of artificial intelligence) methods to address inventory-control problems arising in remanufacturing industry. Remanufacturing is a product-management manufacturing process that aims to reduce the energy consumption and carbon footprint of traditional manufacturing. Effective production/inventory management to match the supply with the demand is a key element to the success of remanufacturing industry. However, the complexity of such problems and the uncertainties involved in the remanufacturing process make the conventional production planning methods difficult to apply. The resulting algorithms and tools will be fully tested using real-world data collected from the industry and are expected to achieve significant savings in raw materials and energy resources, leading to practical management policies of industrial interest. The PIs will involve both graduate and undergraduate students in this research and incorporate case studies into the advanced courses taught at different institutions.This research will be based on a fusion of techniques from reinforcement learning and the field of simulation optimization. Through novel adaptations of the-state-of-the-art variance reduction and function approximation techniques from simulation optimization, the PIs will investigate a new class of learning techniques especially tailored to remanufacturing decision-making problems. These include an extension of classical Q-learning for solving continuous-state semi-Markov decision processes and more general gradient-free actor-critic-like algorithms that overcome the local convergence of existing approaches. The algorithms developed will be studied for their theoretical properties such as convergence and performance consistency, and then assessed and validated on remanufacturing simulation models built on real-world data to investigate their practical impact.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.