CPS: Small: Informed Contextual Bandits to Support Decision-Making for Intelligent CPS

该NSF项目旨在为知情的上下文多武装匪徒（iCMAB）开发一种新的计算框架，该框架将能够在复杂，时变的环境中稳健地运行。该项目将为CPS设计智能决策代理的方式带来革命性的变化，特别是那些利用多武装匪徒变体的智能决策代理。该项目的知识价值包括：I）设计保持其外部世界/环境的生成模型的新颖的知情的上下文强盗，II）设计基于神经进化处理的神经架构搜索，以在线方式自动设计这些生成模型，III）提供识别和解决损坏的上下文和奖励信息的机制，以及IV）促进使代理能够通过查询其内部生成模型来生成长期范围内的预测的过程。该项目的更广泛影响包括：I）通过iCMAB框架推进智能CPS，II）提供易于与许多智能CPS/操作集成的决策模块和流程，以及III）为机器学习和自然启发计算领域做出重要贡献，特别是基于人工神经网络（ANN）的智能代理的自动化设计。智能CPS目前面临的挑战，例如用于传感器验证和活动决策等任务的CPS，包括需要在噪声面前稳健地操作，这些噪声来自于损坏，碎片化和不确定的奖励值和上下文/状态信息，以及必须实时和连续地适应和进行预测。我们建议的综合管理及资讯基建架构，可让CPS透过以下方法解决这些问题：I）以在线方式联合地演化奖励预测模型和情境的生成世界模型，II）提供对奖励和情境信号两者的预测的置信度的度量，和III）利用进化的递归神经网络（eRNN）和大脑启发的神经系统/预测流数据设置中的上下文和奖励信息，同时减轻灾难性遗忘的机制。这一更新的信息将作为CPS的输入，由上下文驱动的强盗，使他们能够采取更明智的行动。这一奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

This NSF project aims to develop a novel computational framework for informed contextual multi-armed bandits (iCMABs) that will be capable of robustly operating in complex, time-varying environments. The project will bring transformative change to the way that intelligent decision-making agents are designed for CPS, specifically those that utilize variants of multi-armed bandits. The intellectual merits of the project include: I) designing novel informed contextual bandits that maintain a generative model of their external world/environment, II) designing neural architecture search processed based on neuro-evolution to automatically design these generative models in an online manner, III) providing mechanisms to identify and address corrupt contextual and reward information, and IV) facilitating a process that enables the agent to generate predictions over longer-term horizons by querying its internal generative model. The broader impacts of the project include: I) advancing intelligent CPS through the iCMAB framework, II) providing decision-making modules and processes that readily integrate with many intelligent CPS/operations, and III) making important contributions to the field of machine learning and nature-inspired computing, specifically the automated design of intelligent agents based on artificial neural networks (ANNs). Current challenges faced by intelligent CPS, such as those used for tasks such as sensor validation and activity decisions, include being required to robustly operate in the face of noise, coming from things such as corrupted, fragmented, and uncertain reward values and context/state information, as well as having to adapt and make predictions in real-time and continually. Our proposed iCMAB framework will enable CPS to tackle these problems by: I) jointly evolving, in an online fashion, a reward forecasting model and a generative world model of contexts, II) providing a measure of confidence in predictions of both reward and context signals, and III) utilizing evolving recurrent neural networks (eRNNs) and brain-inspired neural systems/mechanisms to predict both contextual and reward information in the streaming data setting while mitigating catastrophic forgetting. This updated information will serve as input to CPS that are driven by contextual-bandits, enabling them to take more informed actions.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.