CPS: Small: Informed Contextual Bandits to Support Decision-Making for Intelligent CPS

该NSF项目旨在为知情上下文多武装土匪（icmab）开发一种新的计算框架，该框架将能够在复杂的时变环境中稳健地运行。该项目将为CPS设计智能决策代理的方式带来革命性的变化，特别是那些利用多武装土匪变体的智能决策代理。该项目的智力价值包括：1)设计新颖的知情上下文强盗，维持其外部世界/环境的生成模型；2)设计基于神经进化的神经架构搜索处理，以在线方式自动设计这些生成模型；3)提供识别和处理腐败上下文和奖励信息的机制。IV)促进一个过程，使代理能够通过查询其内部生成模型来生成长期预测。该项目的更广泛影响包括：I)通过iCMAB框架推进智能CPS， II)提供易于与许多智能CPS/操作集成的决策模块和过程，以及III)在机器学习和自然启发计算领域做出重要贡献，特别是基于人工神经网络（ann）的智能代理的自动化设计。智能CPS目前面临的挑战，例如用于传感器验证和活动决策等任务的挑战，包括需要在面对噪声时稳健运行，这些噪声来自诸如损坏，碎片化和不确定的奖励值和上下文/状态信息，以及必须实时和持续地适应和做出预测。我们建议的iCMAB架构将使CPS能够通过以下方式解决这些问题：1)以在线方式共同发展奖励预测模型和情境生成世界模型；2)提供对奖励和情境信号预测的信心衡量；3)利用不断发展的循环神经网络（ernn）和大脑启发的神经系统/机制来预测流数据设置中的情境和奖励信息，同时减轻灾难性遗忘。这些更新的信息将作为由上下文强盗驱动的CPS的输入，使他们能够采取更明智的行动。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This NSF project aims to develop a novel computational framework for informed contextual multi-armed bandits (iCMABs) that will be capable of robustly operating in complex, time-varying environments. The project will bring transformative change to the way that intelligent decision-making agents are designed for CPS, specifically those that utilize variants of multi-armed bandits. The intellectual merits of the project include: I) designing novel informed contextual bandits that maintain a generative model of their external world/environment, II) designing neural architecture search processed based on neuro-evolution to automatically design these generative models in an online manner, III) providing mechanisms to identify and address corrupt contextual and reward information, and IV) facilitating a process that enables the agent to generate predictions over longer-term horizons by querying its internal generative model. The broader impacts of the project include: I) advancing intelligent CPS through the iCMAB framework, II) providing decision-making modules and processes that readily integrate with many intelligent CPS/operations, and III) making important contributions to the field of machine learning and nature-inspired computing, specifically the automated design of intelligent agents based on artificial neural networks (ANNs). Current challenges faced by intelligent CPS, such as those used for tasks such as sensor validation and activity decisions, include being required to robustly operate in the face of noise, coming from things such as corrupted, fragmented, and uncertain reward values and context/state information, as well as having to adapt and make predictions in real-time and continually. Our proposed iCMAB framework will enable CPS to tackle these problems by: I) jointly evolving, in an online fashion, a reward forecasting model and a generative world model of contexts, II) providing a measure of confidence in predictions of both reward and context signals, and III) utilizing evolving recurrent neural networks (eRNNs) and brain-inspired neural systems/mechanisms to predict both contextual and reward information in the streaming data setting while mitigating catastrophic forgetting. This updated information will serve as input to CPS that are driven by contextual-bandits, enabling them to take more informed actions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.