Collaborative Research: CPS: Medium: Spatio-Temporal Logics for Analyzing and Querying Perception Systems

合作研究：CPS：媒介：用于分析和查询感知系统的时空逻辑

• 批准号: 2038666
• 负责人: Yezhou Yang
• 金额: $ 79.99万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-01-01 至 2024-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2038666&HistoricalAwards=false
• 关键词: Collaborative; Research; CPS; Medium; Spatio

The goals of Automated Driving Systems (ADS) and Advanced Driver Assistance Systems (ADAS) include reduction in accidental deaths, enhanced mobility for differently abled people, and an overall improvement in the quality of life for the general public. Such systems typically operate in open and highly uncertain environments for which robust perception systems are essential. However, despite the tremendous theoretical and experimental progress in computer vision, machine learning, and sensor fusion, the form and conditions under which guarantees should be provided for perception components is still unclear. The state-of-the-art is to perform scenario-based evaluation of data against ground truth values, but this has only limited impact. The lack of formal metrics to analyze the quality of perception systems has already led to several catastrophic incidents and a plateau in ADS/ADAS development. This project develops formal languages for specifying and evaluating the quality and robustness of perception sub-systems within ADS and ADAS applications. To enable broader dissemination of this technology, the project develops graduate and undergraduate curricula to train engineers in the use of such methods, and new educational modules to explain the challenges in developing safe and robust ADS for outreach and public engagement activities. To broaden participation in computing, the investigators target the inclusion of undergraduate women in research and development phases through summer internships.The formal language developed in this project is based on a new spatio-temporal logic pioneered by the investigators. This logic allows one to simultaneously perform temporal reasoning about streaming perception data, and spatially reason about objects both within a single frame of the data and across frames. The project also develops quantitative semantics for this logic, which provides the user with quantifiable quality metrics for perception sub-systems. These semantics enable comparisons between different perception systems and architectures. Crucially, the formal language facilitates the process of abstracting away implementation details, which in turn allows system designers and regulators to specify assumptions and guarantees for system performance at a higher-level of abstraction. An interesting benefit of this formal language is that it enables querying of databases with perception data for specific driving scenarios without the need for the highly manual process of creating ground truth annotations. Such a formal language currently does not exist, and this is a huge impediment to building a thriving marketplace for perception components used in safety-critical systems. This framework sets the foundation for a requirements language between suppliers of perception components and automotive companies. The open source and publicly available software tools developed in this project will assist with testing of perception systems by engineers and governmental agencies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自动驾驶系统（ADS）和高级驾驶员援助系统（ADA）的目标包括减少意外死亡，增加了不同吹入的人的流动性以及对公众生活质量的总体改善。这样的系统通常在开放且高度不确定的环境中运行，强大的感知系统至关重要。然而，尽管计算机视觉，机器学习和传感器融合方面取得了巨大的理论和实验进步，但仍不清楚应为感知组件提供保证的形式和条件。最先进的是针对地面真实价值对基于方案的数据进行评估，但这只有有限的影响。缺乏分析感知系统质量的正式指标已经导致了几起灾难性事件，并且在ADS/ADAS开发方面存在高原。该项目开发了用于指定和评估ADS和ADAS应用程序中感知子系统的质量和鲁棒性的正式语言。 为了更广泛地传播这项技术，该项目开发了研究生和本科课程，以培训工程师使用这种方法，以及新的教育模块，以解释开发安全和强大的广告范围，以进行外展和公众参与活动。为了扩大对计算的参与，研究人员将本科妇女通过暑期实习纳入研究和发展阶段。该项目中开发的正式语言基于研究人员开创的新时空逻辑。这种逻辑允许人们同时执行有关流知觉数据的时间推理，并在空间上进行有关对象在数据的单个帧和跨帧中的原因。 该项目还为此逻辑开发了定量语义，该语义为用户提供了可量化的质量指标，以了解感知子系统。这些语义可以在不同的感知系统和体系结构之间进行比较。至关重要的是，正式语言有助于抽象消除实施细节的过程，这又使系统设计人员和监管机构可以在更高级别的抽象中指定系统性能的假设和保证。这种形式语言的一个有趣的好处是，它可以对具有感知数据的数据库进行查询，以解决特定驾驶方案，而无需进行高度手动的过程来创建地面真相注释。目前不存在这样的正式语言，这是建立蓬勃发展的市场的巨大障碍，以构成安全至关重要系统中使用的感知组成部分。该框架为感知组件和汽车公司的供应商之间的需求语言奠定了基础。该项目开发的开源和公开软件工具将有助于测试工程师和政府机构的感知系统。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的评估评估标准来通过评估来支持的。

项目成果

Targeted Attack on Deep RL-based Autonomous Driving with Learned Visual Patterns

• DOI: 10.1109/icra46639.2022.9811574
• 发表时间: 2021-09
• 期刊: 2022 International Conference on Robotics and Automation (ICRA)
• 作者: Prasanth Buddareddygari;Travis Zhang;Yezhou Yang;Yi Ren-

PerceMon: Online Monitoring for Perception Systems

• DOI: 10.1007/978-3-030-88494-9_18
• 发表时间: 2021-08
• 期刊: ArXiv
• 作者: Anand Balakrishnan;Jyotirmoy V. Deshmukh;Bardh Hoxha;Tomoya Yamaguchi;Georgios Fainekos

Covariate Shift Detection via Domain Interpolation Sensitivity

• 发表时间: 2022
• 作者: Tejas Gokhale;Joshua Forster Feinglass

PyFoReL: A Domain-Specific Language for Formal Requirements in Temporal Logic

PyFoReL：用于时态逻辑中形式要求的特定于领域的语言

• DOI: 10.1109/re54965.2022.00037
• 发表时间: 2022
• 期刊: 2022 IEEE 30th International Requirements Engineering Conference (RE
• 作者: Anderson, Jacob;Hekmatnejad, Mohammad;Fainekos, Georgios
• 通讯作者: Fainekos, Georgios

Towards Addressing the Misalignment of Object Proposal Evaluation for Vision-Language Tasks via Semantic Grounding

• DOI: 10.1109/wacv57701.2024.00434
• 发表时间: 2023-09
• 期刊: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
• 作者: Joshua Forster Feinglass;Yezhou Yang