EAGER: Urban Sensing of Pedestrians through Integrated, Cost-Effective, and Scalable Audio Sensor Networks

EARLY概念探索性研究资助（EAGER）项目将调查麦克风在估计行人流量方面的有用性。最近，随着人们对主动移动性和步行性的兴趣越来越大，一些城市正在试验各种技术来感知人。行人流量估计主要基于视频数据分析或红外传感器，如果基于麦克风的传感器被认为同样有效，则可以扩大规模。声学传感器技术，虽然开始被部署用于噪声污染测量和城市噪声源的分类，但尚未在行人感测方面进行探索，尽管其在成本和功率要求方面具有相当大的优势。目前基于麦克风的传感器未得到充分利用的主要原因是分析来自众多不同来源的音频信号的挑战性任务。为了应对这一迄今尚未解决的挑战，为高度复杂的音乐音频信号开发的技术将适应城市传感环境。此外，一个新的数据集，包括带有行人计数注释的音频记录，将被策划和发布，以促进这一领域的未来研究。该项目还将展示如何将从音频传感中提取的数据用于小城市地区的行人流量估计。该项目将在拥挤的校园环境中使用一系列现成的硬件组件进行实验，以研究音频技术在感知人群方面的应用范围，并评估可扩展性的可能性。由于高噪声水平和潜在的低水平行人声音，这个问题特别具有挑战性，我们推测，音频分类中最先进的方法可能不足以很好地解决行人计数估计的问题。受最近通过多任务学习和监督潜在空间正则化学习的结构化音乐表示的工作的启发，应用了一种新的实验正则化方法来进行音频数据的表示学习。这种正则化的自监督学习方法支持基于特征距离构建潜在空间表示。额外的正则化损失项是从当前音频表示结构中与任务相关的强大预训练特征的距离导出的。针对每对训练数据点计算的这种损失可以在没有数据注释的情况下计算，因为它仅基于训练数据点之间的特征距离。通过正则化功能，可以隐式地传递领域知识，从而改善网络的归纳偏差。该奖项反映了NSF的法定使命，通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This EArly-concept Grant for Exploratory Research (EAGER) project will investigate the usefulness of microphones for estimating pedestrian traffic. Recently, with the growing interest in active mobility and walkability, several cities are experimenting with various technologies to sense people. Pedestrian traffic estimation, which has been mostly based on video data analysis or infrared sensors, can be scaled up if microphone-based sensors are deemed equally effective. Acoustic sensor technology, while starting to be deployed for noise pollution measurement and the classification of urban noise sources, has not been explored in the pedestrian sensing context despite its considerable advantages in cost and power requirements. The main reason for the current underutilization of microphone-based sensors is the challenging task of analyzing the audio signal from a multitude of different sources. To address this hitherto unsolved challenge, technology developed for highly complex music audio signals will be adapted to the urban sensing context. Furthermore, a novel dataset comprising audio recordings with pedestrian count annotations will be curated and released to facilitate future research in this area. The project will also demonstrate how the data extracted from audio sensing can be used for pedestrian flow estimation in a small urban area.The project will experiment with a range of off-the-shelf hardware components in a pedestrian-heavy campus environment to investigate how far audio technology can be pushed to sense people, and to assess the possibilities for scalability. As the problem is particularly challenging due to high noise level and potentially low-level pedestrian sound, we speculate that state-of-the-art approaches in audio classification might not be powerful enough to solve the problem of pedestrian count estimation sufficiently well. Inspired by recent work with structured music representations learned through multi-task learning and supervised latent space regularization, a novel experimental regularization approach to representation learning for audio data is applied. This self-supervised learning approach to regularization supports structuring the latent space representation based on feature distances. The additional regularization loss term is derived from the distances of powerful task-relevant pre-trained features in current audio representation structures. This loss, computed for each pair of training data points, can be computed without data annotations as it is based solely on the feature distances between training data points. It enables implicitly imparting domain knowledge through the regularizing feature and thus improves the inductive bias of the network.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.