Collaborative Research: RI: Medium: Learning Joint Crowd-Space Embeddings for Cross-Modal Crowd Behavior Prediction

许多社会活动，包括航空运输、灾难补救、音乐会和体育等社会活动，都需要高效和有效的方法来监测、理解和应对导致这些事件的大量人群的行为。同时，这些行为的类型和进化与它们发生的环境的形式和功能密切相关。随着人群规模的增加或对内在或外在因素的反应而改变他们的行为，建筑环境，包括他们未来的设计，适应这些变化是至关重要的。当今的技术工具旨在分析和预测人群与环境之间的联系。然而，它们依赖于严格的、手工调整的、计算成本很高的仿真模型，严重限制了它们的实际应用。该项目旨在通过设计一种新颖的方式来模拟复杂环境的结构和语义之间的内在关系，以及从小团体到密集人群的人类居住者的存在和行为，从而弥合这一差距。主要目标是准确预测人群行为，从微观运动到聚集人群动态，在新的，从未见过的环境配置中使用环境和人类的神经认知建模（NUCLEUM）来取代计算昂贵但经常与现实不匹配的物理模拟。为了实现这一目标，该项目寻求通过学习数据驱动模型来解决在复杂环境中预测人群行为的问题，这些模型将在人群及其环境的不同表示之间无缝地“转换”。具体来说，这个项目有三个主要的研究重点：（重点1）学习一个联合的人群空间表示。该项目将开发一种新的多概念迁移学习框架，以实现三个高度异构概念的耦合学习：(a)环境布局（例如，楼层平面图），(b)宏观人群属性（例如，流量）和(c)微观人群轨迹。一旦学习，该框架将能够预测人群的流动模式，直接从环境的布局，反之亦然。（主旨2）环境语境和人群运动的混合多模态语料库。该项目将创建一个新的混合多模态语料库，包括环境背景和人群行为，它将利用来自实地观察、受控实验室实验、人群模拟和多用户虚拟现实平台的数据。这个语料库将允许训练模型在环境和人群条件的空间中进行泛化。（重点3）模型评估、应用程序和用例。经过训练的模型的稳健性将根据其产生有效人群轨迹的能力进行评估，这些轨迹在统计上与地面真实观察相似，同时推广到新的、看不见的人群和环境背景。随后，该项目将在各种应用环境中应用训练好的模型，在真实世界已建成和尚未建成的环境中预测未知环境中的人群行为，识别环境中的漏洞，并重新配置环境设计以改善人群行为。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Many societal activities, including air transport, disaster remediation, social events such as concerts and sports, require efficient and effective methodologies for monitoring, understanding, and reacting to behaviors of large concentrations of people, the crowds, that give rise to those events. Simultaneously, the type and evolution of those behaviors are intimately tied to the form and function of the environments where they occur. As crowds increase in size or change their actions in response to intrinsic or extrinsic factors, it is critical for the built environments, including their future designs, to adapt to those changes. Present-day technological tools aim to analyze and predict the link between crowds and environments. However, they rely on rigid, hand-tuned, computationally costly simulation models, severely limiting their practical utility. This project seeks to bridge this gap by devising a novel way of modeling the inherent relationship between the structure and semantics of complex environments, and the presence and behavior of its human occupants, from small groups to dense crowds. The main goal is to predict crowd behavior accurately, from microscopic motion to aggregate crowd dynamics, in novel, never-before-seen environment configurations using Neuro-Cognitive Modeling of Environments and Humans (NUCLEUM) to replace the computationally expensive yet often mismatched-with-reality physical simulations. To accomplish this goal, this project collaboratively seeks to tackle the problem of predicting crowd behavior in complex environments by learning data-driven models that will seamlessly "translate" between different representations of crowds and their environments. Specifically, this project has three main research thrusts: (Thrust 1) Learning a Joint Crowd-Space Representation. The project will develop a novel multi-concept transfer learning framework to enable coupled learning across three highly heterogeneous concepts: (a) environment layouts (e.g., floor plans), (b) macroscopic crowd properties (e.g., flow), and (c) microscopic crowd trajectories. Once learned, the framework will enable predictions of flow patterns of a crowd, directly from the layout of an environment and vice versa. (Thrust 2) A Hybrid Multi-modal Corpus of Environment Contexts and Crowd Movement. This project will create a novel hybrid multi-modal corpus of environmental contexts and crowd behavior, which will leverage data from field observations, controlled laboratory experiments, crowd simulations, and multi-user virtual reality platforms. This corpus will allow training models that generalize across the space of environment and crowd conditions. (Thrust 3) Model Evaluation, Applications, and Use Cases. Trained models' robustness will be evaluated in terms of their ability to produce valid crowd trajectories, which are statistically similar to ground truth observations while generalizing to the new, unseen crowd, and environmental contexts. This project will subsequently apply the trained models in a variety of application contexts on real-world built and yet-to-be-built environments to predict crowd behavior in unseen environments, identify vulnerabilities in environments, and reconfigure environment designs to improve crowd behavior.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.