CIF: Small: Collaborative Research: Geometry-aware and data-adaptive signal processing for resource constrained activity analysis

使用多模式传感和自动推理技术来表征人类活动的任务在许多应用中是至关重要的。当使用传统的音频和视频传感器以及新出现的传感器(如深度相机、方向传感器和智能个人设备)观察人类活动时，会产生难以分析的复杂的高维时空特征。困难的来源有很多：高数据吞吐量、传感器注册的需要、与活动相关的可变执行率、传感中的姿势可变性、由于环境和行为者的物理限制而导致的特征空间的非欧几里得性质以及由于遮挡而导致的数据损坏。目前的技术，包括欧几里得表示和多变量分析，在描述人类活动方面存在不足，因为它们不处理非欧几里德结构，也不能获得姿势和执行率的不变性。这项研究使用跨越微分几何、统计学和信号近似理论的工具来开发描述人类活动的新框架。这些基本工具带来了全面的解决方案，适用于广泛的传统和新兴传感器。该方法的显著特点是：1)几何感知，既包括经典的欧几里德特征空间，也包括非欧几里德特征空间；2)传感器位置和执行速度的不变性紧密集成到表示中；3)数据自适应导致人类活动的低比特率表示，以减少通信和低计算场景。这项研究的应用包括在资源有限的环境中使用现成的传感器监测人类活动，例如在家里和移动设备上。

The task of characterizing human activities using multimodal sensing and automated inference techniques is critically important in many applications. Human activities when observed using traditional audio and video sensors, as well as novel emerging sensors such as depth cameras, orientation sensors, and smart personal devices, result in complex, high-dimensional spatiotemporal signatures that are difficult to analyze. The sources of difficulty are many: high data throughput, the need for sensor registration, variable execution rates associated with activities, pose variability in sensing, the non-Euclidean nature of feature spaces due to physical constraints on environments and actors, and data corruption due to occlusions. The current techniques, involving Euclidean representations and multivariate analyses, fall short in characterizing human activities, as they do not handle non-Euclidean structures nor obtain invariances to pose and execution rates. This research uses tools spanning differential geometry, statistics and signal approximation theory to develop novel frameworks for characterizing human activities. These fundamental tools lead to comprehensive solutions that are applicable to a broad swath of traditional and emerging sensors. The salient aspects of this approach are: 1) geometry awareness, encompassing both classical Euclidean as well as non-Euclidean feature spaces, 2) invariance to sensor placement and execution rate tightly integrated into the representation, and 3) data adaptivity leading to low bitrate representation of human activities for reduced communication and low computational scenarios. Applications of this research include monitoring of human activities using off-the-shelf sensors in resource-constrained environments, such as at homes and on mobile devices.