Collaborative Research: Continuous-State Reinforcement Learning for Remanufacturing

协作研究：再制造的连续状态强化学习

• 批准号: 2027527
• 负责人: Jiaqiao Hu
• 金额: $ 24.37万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2027527&HistoricalAwards=false
• 关键词: Collaborative; Research; Continuous; State; Reinforcement

This award will contribute to the national prosperity and U.S. manufacturing competitiveness by developing new reinforcement learning (a subfield of artificial intelligence) methods to address inventory-control problems arising in remanufacturing industry. Remanufacturing is a product-management manufacturing process that aims to reduce the energy consumption and carbon footprint of traditional manufacturing. Effective production/inventory management to match the supply with the demand is a key element to the success of remanufacturing industry. However, the complexity of such problems and the uncertainties involved in the remanufacturing process make the conventional production planning methods difficult to apply. The resulting algorithms and tools will be fully tested using real-world data collected from the industry and are expected to achieve significant savings in raw materials and energy resources, leading to practical management policies of industrial interest. The PIs will involve both graduate and undergraduate students in this research and incorporate case studies into the advanced courses taught at different institutions.This research will be based on a fusion of techniques from reinforcement learning and the field of simulation optimization. Through novel adaptations of the-state-of-the-art variance reduction and function approximation techniques from simulation optimization, the PIs will investigate a new class of learning techniques especially tailored to remanufacturing decision-making problems. These include an extension of classical Q-learning for solving continuous-state semi-Markov decision processes and more general gradient-free actor-critic-like algorithms that overcome the local convergence of existing approaches. The algorithms developed will be studied for their theoretical properties such as convergence and performance consistency, and then assessed and validated on remanufacturing simulation models built on real-world data to investigate their practical impact.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项将通过开发新的强化学习（人工智能的一个子领域）方法来解决再制造行业中出现的库存控制问题，为国家繁荣和美国制造业竞争力做出贡献。再制造是一种产品管理制造过程，旨在减少传统制造业的能源消耗和碳足迹。有效的生产/库存管理以匹配供应与需求是再制造行业成功的关键因素。然而，这些问题的复杂性和再制造过程中所涉及的不确定性使得传统的生产计划方法难以应用。 由此产生的算法和工具将使用从行业中收集的真实数据进行全面测试，预计将大幅节省原材料和能源，从而制定出符合行业利益的实用管理政策。 PI将让研究生和本科生参与这项研究，并将案例研究纳入不同机构教授的高级课程中。这项研究将基于强化学习和仿真优化领域的融合技术。通过对仿真优化中最先进的方差减小和函数逼近技术的新适应，PI将研究一类新的学习技术，特别是针对再制造决策问题。这些包括经典Q学习的扩展，用于解决连续状态半马尔可夫决策过程和更一般的无梯度的演员评论家算法，克服了现有方法的局部收敛。开发的算法将被研究其理论属性，例如收敛性和性能一致性，然后在基于真实数据构建的再制造仿真模型上进行评估和验证，以调查其实际影响。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。