CIF: Small: Inverse Reinforcement Learning for Cognitive Sensing

认知传感系统，如自适应雷达，通过最大化受传感约束的效用函数来选择决策。本项目研究的问题是：通过观察认知感知系统的决策，对手如何估计传感器的效用函数，从而预测其未来的决策？这种逆强化学习问题出现在许多国防和民用传感系统中。利用微观经济学、机器学习和优化的思想，该项目研究如何设计算法来实现逆强化学习。该项目还研究了如何设计一个隐蔽的传感系统，比如一个隐藏其效用的元认知雷达，从而对对手隐藏其计划。该研究将为具有传感性能保证的逆强化学习和隐藏其认知的隐蔽传感系统的设计带来新的方法。该项目将支持康奈尔大学多样化的博士和本科生的跨学科发展，并为纽约州农村高中生的STEM教育做出贡献。本项目从微观经济学、机器学习和随机优化等方面研究认知感知中的对抗性信号处理问题。这个项目的技术目标分为三个相互关联的主题。第一个主题调查揭示的偏好方法来检测认知传感器的存在。该研究研究了如何统计检测认知感知系统的存在，以及如何询问传感器以检测它是否具有认知。第二个主题研究了逆贝叶斯序列检测：给定最优序列检测器的决策，如何估计其参数，如误分类成本和误报惩罚？第三个主题研究逆随机梯度算法：给定随机梯度算法的实时噪声估计，如何估计它正在优化的期望效用函数？本课题研究的是自适应逆强化学习，而认知传感器正在学习优化策略。该项目超越了经典的统计信号处理（估计/检测），以解决如何从感知推断策略的更深层次问题。这项研究的结果将是具有可证明的性能保证的策略识别新算法，广泛适用于检测复杂的传感系统。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Cognitive sensing systems such as adaptive radars choose their decisions by maximizing a utility function subject to sensing constraints. This project studies the question: By observing the decisions of a cognitive sensing system, how can an adversary estimate the sensor's utility function and therefore predict its future decisions? Such inverse reinforcement learning problems arise in numerous defense and civilian sensing systems. Using ideas from microeconomics, machine learning and optimization, this project investigates how to design algorithms to achieve inverse reinforcement learning. The project also investigates how to design a covert sensing system, such as a meta-cognitive radar that hides its utility and, therefore, its plan from an adversary. The research will lead to new methods for inverse reinforcement learning with performance guarantees for sensing and the design of covert sensing systems that hide their cognition. The project will support the cross-disciplinary development of a diverse cohort of PhD and undergraduate students at Cornell University and also contribute to the STEM education of high school students from rural New York state.This project draws from micro-economics, machine learning and stochastic optimization to study adversarial signal processing problems in cognitive sensing. The technical aims of this project fall under three interrelated themes. The first theme investigates revealed preference methods to detect the presence of cognitive sensors. The research studies how to detect statistically the presence of a cognitive sensing system and how to interrogate a sensor to detect if it is cognitive. The second theme investigates inverse Bayesian sequential detection: given the decisions of an optimal sequential detector, how to estimate its parameters such as misclassification costs and false alarm penalty? The third theme investigates inverse stochastic gradient algorithms: given real-time noisy estimates from a stochastic gradient algorithm, how to estimate the expected utility function that it is optimizing? The research in this theme studies adaptive inverse reinforcement learning while the cognitive sensor is learning to optimize its strategy. This project transcends classical statistical signal processing (estimation/detection) to address the deeper issue of how to infer strategy from sensing. The outcome of this research will be novel algorithms for strategy identification with provable performance guarantees that are broadly applicable to detect complex sensing systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.