Collaborative Research: RI: Medium: Learning Joint Crowd-Space Embeddings for Cross-Modal Crowd Behavior Prediction

许多社会活动，包括航空运输、灾难补救、诸如音乐会和体育运动之类的社会事件，需要高效和有效的方法来监测、理解和对引起这些事件的大量人群的行为做出反应。同时，这些行为的类型和演变与它们发生的环境的形式和功能密切相关。随着人群规模的增加或改变他们的行动，以应对内在或外在因素，这是至关重要的建成环境，包括其未来的设计，以适应这些变化。当今的技术工具旨在分析和预测人群与环境之间的联系。然而，它们依赖于刚性的、手动调整的、计算成本高的仿真模型，严重限制了它们的实际效用。该项目旨在通过设计一种新颖的方式来建模复杂环境的结构和语义之间的内在关系，以及其人类居住者的存在和行为，从小团体到密集的人群，来弥合这一差距。其主要目标是准确预测人群行为，从微观运动到聚合人群动态，在新的，从未见过的环境配置中使用环境和人类的神经认知建模（NUCLEUM）来取代计算昂贵但经常与现实不匹配的物理模拟。为了实现这一目标，该项目通过学习数据驱动的模型来解决复杂环境中预测人群行为的问题，这些模型将在人群及其环境的不同表示之间进行无缝“翻译”。具体而言，该项目有三个主要研究方向：（方向1）学习联合人群空间表示。该项目将开发一个新的多概念迁移学习框架，以实现跨三个高度异构概念的耦合学习：（a）环境布局（例如，楼层平面图），（B）宏观人群特性（例如，流动），和（c）微观人群轨迹。一旦学会，该框架将能够直接从环境的布局预测人群的流动模式，反之亦然。（主旨2）环境语境和人群运动的混合多模态语料库。该项目将创建一个新的环境背景和人群行为的混合多模态语料库，该语料库将利用来自实地观察、受控实验室实验、人群模拟和多用户虚拟现实平台的数据。该语料库将允许训练模型在环境和人群条件的空间中泛化。（重点3）模型评估、应用和用例。训练模型的鲁棒性将根据其产生有效人群轨迹的能力进行评估，这些轨迹在统计上类似于地面实况观察，同时推广到新的、看不见的人群和环境背景。该项目随后将在真实世界构建和尚未构建的环境中的各种应用环境中应用经过训练的模型，以预测看不见的环境中的人群行为，识别环境中的漏洞，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查进行评估，被认为值得支持的搜索.

Many societal activities, including air transport, disaster remediation, social events such as concerts and sports, require efficient and effective methodologies for monitoring, understanding, and reacting to behaviors of large concentrations of people, the crowds, that give rise to those events. Simultaneously, the type and evolution of those behaviors are intimately tied to the form and function of the environments where they occur. As crowds increase in size or change their actions in response to intrinsic or extrinsic factors, it is critical for the built environments, including their future designs, to adapt to those changes. Present-day technological tools aim to analyze and predict the link between crowds and environments. However, they rely on rigid, hand-tuned, computationally costly simulation models, severely limiting their practical utility. This project seeks to bridge this gap by devising a novel way of modeling the inherent relationship between the structure and semantics of complex environments, and the presence and behavior of its human occupants, from small groups to dense crowds. The main goal is to predict crowd behavior accurately, from microscopic motion to aggregate crowd dynamics, in novel, never-before-seen environment configurations using Neuro-Cognitive Modeling of Environments and Humans (NUCLEUM) to replace the computationally expensive yet often mismatched-with-reality physical simulations. To accomplish this goal, this project collaboratively seeks to tackle the problem of predicting crowd behavior in complex environments by learning data-driven models that will seamlessly "translate" between different representations of crowds and their environments. Specifically, this project has three main research thrusts: (Thrust 1) Learning a Joint Crowd-Space Representation. The project will develop a novel multi-concept transfer learning framework to enable coupled learning across three highly heterogeneous concepts: (a) environment layouts (e.g., floor plans), (b) macroscopic crowd properties (e.g., flow), and (c) microscopic crowd trajectories. Once learned, the framework will enable predictions of flow patterns of a crowd, directly from the layout of an environment and vice versa. (Thrust 2) A Hybrid Multi-modal Corpus of Environment Contexts and Crowd Movement. This project will create a novel hybrid multi-modal corpus of environmental contexts and crowd behavior, which will leverage data from field observations, controlled laboratory experiments, crowd simulations, and multi-user virtual reality platforms. This corpus will allow training models that generalize across the space of environment and crowd conditions. (Thrust 3) Model Evaluation, Applications, and Use Cases. Trained models' robustness will be evaluated in terms of their ability to produce valid crowd trajectories, which are statistically similar to ground truth observations while generalizing to the new, unseen crowd, and environmental contexts. This project will subsequently apply the trained models in a variety of application contexts on real-world built and yet-to-be-built environments to predict crowd behavior in unseen environments, identify vulnerabilities in environments, and reconfigure environment designs to improve crowd behavior.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.