CAREER: Modeling Situated Intention during Nondeterministic Pedestrian-Vehicle Interactions through Explainable Compositional Learning of Naturalistic Driving Data

职业：通过自然驾驶数据的可解释组合学习，对非确定性行人-车辆交互过程中的情境意图进行建模

• 批准号: 2145565
• 负责人: Renran Tian
• 金额: $ 59.81万
• 依托单位: Indiana University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145565&HistoricalAwards=false
• 关键词: CAREER; Modeling; Situated; Intention; during

With the fast progress of artificial intelligence, vehicles with higher levels of automation are entering daily life. Automated driving technologies are expected to improve traffic safety, promote travel efficiency, protect the environment, reduce mobility barriers for older generations or people with disabilities, and thus deliver overall societal benefits. However, existing algorithms embedded in autonomous vehicles still face fundamental challenges in recognizing the quickly changing intentions of pedestrians moving on the road and sidewalks, making it hard to predict their behavior and plan vehicle motions. Such limitations impede the implementation of fully autonomous and safe cars in city environments and create additional risks for pedestrians and other road users. This project focuses on developing novel techniques to model and predict the complex and changing intentions of pedestrians. By learning the thinking process of drivers and their driving responses, an algorithm will be created to equip the automated cars with similar capabilities to interact with pedestrians and other road users smoothly and safely. The research process will include naturalistic driving data collection, subject experiments, knowledge modeling, and learning algorithm development. The developed algorithm will be evaluated in an immersive virtual environment. The project also includes activities to promote user-centered design in engineering education, foster the awareness of biases and ethical issues related to artificial intelligence technologies, and increase the participation of underrepresented communities in Science, Technology, Engineering, and Mathematics. This project surmounts limitations of current pedestrian behavior prediction to achieving mutual intelligibility between autonomous vehicles and pedestrians. Principally, unlike the traditional static view of pedestrian intention at a critical moment, this research investigates the relationship between non-verbal actions and intention changes of pedestrians moment-to-moment in dynamic (changing) and interactive situations. The project collects temporal video segments and human reasoning descriptions simultaneously through event-segmentation-based video experiments. Then, it develops a compositional learning method to learn and combine language features with visual features. This method can avoid the rigid structure of expert-selected feature space by creating collective features from ordinary drivers, and the three-level explainability of the learning model can justify model outputs from input features. Finally, the developed intention prediction model will be evaluated through subject experiments in a virtual interactive pedestrian simulator. The research findings will be shared through industrial collaborators and conferences, and a pedestrian behavior benchmark dataset will be disseminated to the public. The research results will be included in engineering education to promote design approaches that take into account the users’ feelings, values, and overall mental state (empathic design). These educational activities will include courses at the investigator’s university, the autonomous driving research community, and industry. They will increase the awareness of critical human-centered AI issues like biases, trust, and social intelligence in design of AI.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着人工智能的快速发展，自动化程度更高的车辆正在进入日常生活。自动驾驶技术有望改善交通安全，提高出行效率，保护环境，减少老年人或残疾人的移动障碍，从而带来整体社会效益。然而，嵌入自动驾驶汽车的现有算法在识别道路和人行道上行人快速变化的意图方面仍然面临根本挑战，这使得很难预测他们的行为和规划车辆运动。这些限制阻碍了在城市环境中实现完全自主和安全的汽车，并为行人和其他道路使用者带来额外的风险。该项目的重点是开发新的技术来模拟和预测行人复杂和不断变化的意图。通过学习驾驶员的思维过程及其驾驶反应，将创建一种算法，为自动汽车配备类似的功能，使其能够顺利安全地与行人和其他道路使用者互动。研究过程将包括自然驾驶数据收集，主题实验，知识建模和学习算法开发。开发的算法将在沉浸式虚拟环境中进行评估。该项目还包括在工程教育中促进以用户为中心的设计的活动，促进对与人工智能技术相关的偏见和道德问题的认识，并增加科学，技术，工程和数学中代表性不足的社区的参与。该项目克服了当前行人行为预测的局限性，实现了自动驾驶汽车和行人之间的相互理解。与传统的行人关键时刻意图静态研究不同，本研究主要探讨动态（变化）交互情境下行人非言语行为与意图变化之间的关系。该项目通过基于事件分割的视频实验同时收集时间视频片段和人类推理描述。然后，它开发了一种组合学习方法来学习和联合收割机的语言特征和视觉特征相结合。该方法可以避免专家选择的特征空间的刚性结构，通过创建普通驱动程序的集体特征，和三个层次的可解释性的学习模型可以证明模型输出从输入特征。最后，开发的意图预测模型将通过虚拟交互式行人模拟器中的主题实验进行评估。研究结果将通过行业合作者和会议分享，行人行为基准数据集将向公众传播。研究结果将被纳入工程教育，以促进考虑到用户的感受，价值观和整体心理状态的设计方法（移情设计）。 这些教育活动将包括在研究者所在大学、自动驾驶研究社区和工业界开设的课程。 该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Flexible and scalable annotation tool to develop scene understanding datasets

• DOI: 10.1145/3546930.3547499
• 发表时间: 2022-06
• 期刊: Proceedings of the Workshop on Human-In-the-Loop Data Analytics
• 作者: Md. Fazle Elahi Khan;Renran Tian;Xiao Luo

TrEP: Transformer-Based Evidential Prediction for Pedestrian Intention with Uncertainty

• DOI: 10.1609/aaai.v37i3.25463
• 发表时间: 2023-06
• 作者: Zhengming Zhang;Renran Tian;Zhengming Ding

Attention-Based Interrelation Modeling for Explainable Automated Driving

• DOI: 10.1109/tiv.2022.3229682
• 发表时间: 2023-02
• 期刊: IEEE Transactions on Intelligent Vehicles
• 影响因子: 8.2
• 作者: Zhengming Zhang;Renran Tian;Rini Sherony;Joshua E. Domeyer;Zhengming Ding