CIF: Small: Inverse Reinforcement Learning for Cognitive Sensing

CIF：小：认知感知的逆强化学习

• 批准号: 2312198
• 负责人: Vikram Krishnamurthy
• 金额: $ 60万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2312198&HistoricalAwards=false
• 关键词: CIF; Small; Inverse; Reinforcement; Learning

Cognitive sensing systems such as adaptive radars choose their decisions by maximizing a utility function subject to sensing constraints. This project studies the question: By observing the decisions of a cognitive sensing system, how can an adversary estimate the sensor's utility function and therefore predict its future decisions? Such inverse reinforcement learning problems arise in numerous defense and civilian sensing systems. Using ideas from microeconomics, machine learning and optimization, this project investigates how to design algorithms to achieve inverse reinforcement learning. The project also investigates how to design a covert sensing system, such as a meta-cognitive radar that hides its utility and, therefore, its plan from an adversary. The research will lead to new methods for inverse reinforcement learning with performance guarantees for sensing and the design of covert sensing systems that hide their cognition. The project will support the cross-disciplinary development of a diverse cohort of PhD and undergraduate students at Cornell University and also contribute to the STEM education of high school students from rural New York state.This project draws from micro-economics, machine learning and stochastic optimization to study adversarial signal processing problems in cognitive sensing. The technical aims of this project fall under three interrelated themes. The first theme investigates revealed preference methods to detect the presence of cognitive sensors. The research studies how to detect statistically the presence of a cognitive sensing system and how to interrogate a sensor to detect if it is cognitive. The second theme investigates inverse Bayesian sequential detection: given the decisions of an optimal sequential detector, how to estimate its parameters such as misclassification costs and false alarm penalty? The third theme investigates inverse stochastic gradient algorithms: given real-time noisy estimates from a stochastic gradient algorithm, how to estimate the expected utility function that it is optimizing? The research in this theme studies adaptive inverse reinforcement learning while the cognitive sensor is learning to optimize its strategy. This project transcends classical statistical signal processing (estimation/detection) to address the deeper issue of how to infer strategy from sensing. The outcome of this research will be novel algorithms for strategy identification with provable performance guarantees that are broadly applicable to detect complex sensing systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

认知传感系统，如自适应雷达选择他们的决定，最大化的效用函数受到传感约束。本项目研究的问题：通过观察认知传感系统的决策，对手如何估计传感器的效用函数，从而预测其未来的决策？这种反向强化学习问题出现在许多国防和民用传感系统中。利用微观经济学，机器学习和优化的思想，该项目研究如何设计算法来实现反向强化学习。该项目还研究了如何设计一个隐蔽的传感系统，例如一个元认知雷达，隐藏其效用，因此，它的计划从对手。这项研究将为反向强化学习带来新的方法，同时保证传感的性能，并设计隐藏其认知的隐蔽传感系统。该项目将支持康奈尔大学博士生和本科生的跨学科发展，并为纽约州农村高中生的STEM教育做出贡献。该项目从微观经济学，机器学习和随机优化出发，研究认知传感中的对抗信号处理问题。 该项目的技术目标属于三个相互关联的主题。第一个主题调查发现偏好的方法来检测认知传感器的存在。 该研究研究如何统计检测认知传感系统的存在，以及如何询问传感器以检测它是否是认知的。第二个主题研究逆贝叶斯序贯检测：给定最优序贯检测器的决策，如何估计其参数，如误分类成本和虚警惩罚？第三个主题研究逆随机梯度算法：给定随机梯度算法的实时噪声估计，如何估计它正在优化的期望效用函数？本课题研究了认知传感器在学习优化策略时的自适应逆强化学习。该项目超越了经典的统计信号处理（估计/检测），以解决如何从感知中推断策略的更深层次问题。这项研究的成果将是新的算法，战略识别与可证明的性能保证，广泛适用于检测复杂的传感systems.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。