Collaborative Research: RI: Medium: Learning Joint Crowd-Space Embeddings for Cross-Modal Crowd Behavior Prediction

合作研究：RI：Medium：学习联合人群空间嵌入以进行跨模式人群行为预测

• 批准号: 1955404
• 负责人: Mubbasir Kapadia
• 金额: $ 83.67万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955404&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Medium; Learning

Many societal activities, including air transport, disaster remediation, social events such as concerts and sports, require efficient and effective methodologies for monitoring, understanding, and reacting to behaviors of large concentrations of people, the crowds, that give rise to those events. Simultaneously, the type and evolution of those behaviors are intimately tied to the form and function of the environments where they occur. As crowds increase in size or change their actions in response to intrinsic or extrinsic factors, it is critical for the built environments, including their future designs, to adapt to those changes. Present-day technological tools aim to analyze and predict the link between crowds and environments. However, they rely on rigid, hand-tuned, computationally costly simulation models, severely limiting their practical utility. This project seeks to bridge this gap by devising a novel way of modeling the inherent relationship between the structure and semantics of complex environments, and the presence and behavior of its human occupants, from small groups to dense crowds. The main goal is to predict crowd behavior accurately, from microscopic motion to aggregate crowd dynamics, in novel, never-before-seen environment configurations using Neuro-Cognitive Modeling of Environments and Humans (NUCLEUM) to replace the computationally expensive yet often mismatched-with-reality physical simulations. To accomplish this goal, this project collaboratively seeks to tackle the problem of predicting crowd behavior in complex environments by learning data-driven models that will seamlessly "translate" between different representations of crowds and their environments. Specifically, this project has three main research thrusts: (Thrust 1) Learning a Joint Crowd-Space Representation. The project will develop a novel multi-concept transfer learning framework to enable coupled learning across three highly heterogeneous concepts: (a) environment layouts (e.g., floor plans), (b) macroscopic crowd properties (e.g., flow), and (c) microscopic crowd trajectories. Once learned, the framework will enable predictions of flow patterns of a crowd, directly from the layout of an environment and vice versa. (Thrust 2) A Hybrid Multi-modal Corpus of Environment Contexts and Crowd Movement. This project will create a novel hybrid multi-modal corpus of environmental contexts and crowd behavior, which will leverage data from field observations, controlled laboratory experiments, crowd simulations, and multi-user virtual reality platforms. This corpus will allow training models that generalize across the space of environment and crowd conditions. (Thrust 3) Model Evaluation, Applications, and Use Cases. Trained models' robustness will be evaluated in terms of their ability to produce valid crowd trajectories, which are statistically similar to ground truth observations while generalizing to the new, unseen crowd, and environmental contexts. This project will subsequently apply the trained models in a variety of application contexts on real-world built and yet-to-be-built environments to predict crowd behavior in unseen environments, identify vulnerabilities in environments, and reconfigure environment designs to improve crowd behavior.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

许多社会活动，包括航空运输、灾害补救、音乐会和体育等社交活动，都需要高效有效的方法来监测、理解和应对引起这些事件的大量人群的行为。同时，这些行为的类型和演变与它们发生的环境的形式和功能密切相关。随着人群规模的增加或对内在或外在因素的反应而改变他们的行为，建造环境，包括他们未来的设计，适应这些变化是至关重要的。当今的技术工具旨在分析和预测人群与环境之间的联系。然而，它们依赖于僵化的、手动调整的、计算成本高昂的模拟模型，严重限制了它们的实用价值。这个项目试图通过设计一种新的方法来模拟复杂环境的结构和语义与其居住者的存在和行为之间的内在关系，从小群体到密集人群，从而弥合这一差距。主要目标是使用环境和人的神经认知建模(NUCLEUM)在新颖的、前所未见的环境配置中准确地预测人群行为，从微观运动到聚合人群动态，以取代计算昂贵但往往与现实不匹配的物理模拟。为了实现这一目标，该项目合作寻求通过学习数据驱动的模型来解决预测复杂环境中的人群行为的问题，这些模型将在人群及其环境的不同表示之间无缝地“转换”。具体地说，这个项目有三个主要的研究主旨：(推力1)学习联合人群空间表征。该项目将开发一个新的多概念迁移学习框架，以实现三个高度不同的概念之间的耦合学习：(A)环境布局(例如，平面图)，(B)宏观人群属性(例如，流动)，以及(C)微观人群轨迹。一旦学习，该框架将能够直接根据环境的布局预测人群的流动模式，反之亦然。(推力2)环境背景和人群运动的混合多模式语料库。该项目将创建环境背景和人群行为的新型混合多模式语料库，该语料库将利用来自现场观察、受控实验室实验、人群模拟和多用户虚拟现实平台的数据。该语料库将允许在环境和人群条件的空间中泛化培训模型。(主旨3)模型评估、应用程序和用例。训练模型的稳健性将根据它们产生有效人群轨迹的能力进行评估，这些轨迹在统计上类似于地面真实观测，同时概括到新的、看不见的人群和环境背景。该项目随后将在现实世界已建成和尚未建成的环境中的各种应用环境中应用训练的模型，以预测看不见的环境中的人群行为，识别环境中的漏洞，并重新配置环境设计以改善人群行为。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Towards Image-to-Video Translation: A Structure-Aware Approach via Multi-stage Generative Adversarial Networks

• DOI: 10.1007/s11263-020-01328-9
• 发表时间: 2020-04
• 期刊: International Journal of Computer Vision
• 影响因子: 19.5
• 作者: Long Zhao;Xi Peng;Yu Tian;M. Kapadia;Dimitris N. Metaxas

COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality

• DOI: 10.1007/978-3-031-19833-5_15
• 发表时间: 2021-12
• 作者: Honglu Zhou;Asim Kadav;Aviv Shamsian;Shijie Geng;Farley Lai;Long Zhao;Tingxi Liu;M. Kapadia

Harnessing Fourier Isovists and Geodesic Interaction for Long-Term Crowd Flow Prediction

利用傅里叶等量线和测地线相互作用进行长期人群流量预测

• DOI: 10.24963/ijcai.2022/185
• 发表时间: 2022
• 期刊: Thirty-First International Joint Conference on Artificial Intelligence (IJCAI
• 作者: Sohn, Samuel S.;Moon, Seonghyeon;Zhou, Honglu;Lee, Mihee;Yoon, Sejong;Pavlovic, Vladimir;Kapadia, Mubbasir
• 通讯作者: Kapadia, Mubbasir

A Social Distancing Index: Evaluating Navigational Policies on Human Proximity using Crowd Simulations

社交距离指数：使用人群模拟评估人类接近度的导航政策

• DOI: 10.1145/3424636.3426905
• 发表时间: 2020
• 期刊: ACM SIGGRAPH Motion Interaction and Games
• 作者: Usman, Muhammad;Lee, Tien-Chi;Moghe, Ryhan;Zhang, Xun;Faloutsos, Petros;Kapadia, Mubbasir
• 通讯作者: Kapadia, Mubbasir

The interaction between map complexity and crowd movement on navigation decisions in virtual reality

虚拟现实中地图复杂性和人群运动对导航决策的相互作用

• DOI: 10.1098/rsos.191523
• 发表时间: 2020
• 期刊: Royal Society Open Science
• 影响因子: 3.5
• 作者: Zhao, Hantao;Thrash, Tyler;Grossrieder, Armin;Kapadia, Mubbasir;Moussaïd, Mehdi;Hölscher, Christoph;Schinazi, Victor R.
• 通讯作者: Schinazi, Victor R.