CIF: Small: Collaborative Research: Geometry-aware and data-adaptive signal processing for resource constrained activity analysis

利用多模态传感和自动推理技术表征人类活动的任务在许多应用中都是至关重要的。当使用传统的音频和视频传感器以及新兴的传感器（如深度相机、方向传感器和智能个人设备）观察人类活动时，会产生难以分析的复杂、高维时空特征。困难的来源有很多：高数据吞吐量，传感器配准的需求，与活动相关的可变执行速率，传感中的变化，由于环境和参与者的物理限制而导致的特征空间的非欧几里得性质，以及由于遮挡导致的数据损坏。目前的技术，包括欧几里得表示和多变量分析，在描述人类活动方面存在不足，因为它们不能处理非欧几里得结构，也不能获得姿态和执行速度的不变性。本研究使用跨越微分几何、统计学和信号近似理论的工具来开发表征人类活动的新框架。这些基本工具带来了适用于广泛的传统和新兴传感器的综合解决方案。该方法的突出方面是：1)几何感知，包括经典欧几里得和非欧几里得特征空间；2)传感器放置和执行速度的不变性紧密集成到表示中；3)数据自适应导致人类活动的低比特率表示，以减少通信和低计算场景。这项研究的应用包括在资源有限的环境中使用现成的传感器监测人类活动，例如在家庭和移动设备上。

The task of characterizing human activities using multimodal sensing and automated inference techniques is critically important in many applications. Human activities when observed using traditional audio and video sensors, as well as novel emerging sensors such as depth cameras, orientation sensors, and smart personal devices, result in complex, high-dimensional spatiotemporal signatures that are difficult to analyze. The sources of difficulty are many: high data throughput, the need for sensor registration, variable execution rates associated with activities, pose variability in sensing, the non-Euclidean nature of feature spaces due to physical constraints on environments and actors, and data corruption due to occlusions. The current techniques, involving Euclidean representations and multivariate analyses, fall short in characterizing human activities, as they do not handle non-Euclidean structures nor obtain invariances to pose and execution rates. This research uses tools spanning differential geometry, statistics and signal approximation theory to develop novel frameworks for characterizing human activities. These fundamental tools lead to comprehensive solutions that are applicable to a broad swath of traditional and emerging sensors. The salient aspects of this approach are: 1) geometry awareness, encompassing both classical Euclidean as well as non-Euclidean feature spaces, 2) invariance to sensor placement and execution rate tightly integrated into the representation, and 3) data adaptivity leading to low bitrate representation of human activities for reduced communication and low computational scenarios. Applications of this research include monitoring of human activities using off-the-shelf sensors in resource-constrained environments, such as at homes and on mobile devices.