CPS: Small: Real-time spatial audio on the Internet of Things

随着物联网的激增，许多联网设备充斥着家庭、办公室和公共场所，通常配有扬声器来进行听觉响应。这个项目力求使用这样的扬声器来感知地将听觉声音放置在任意的虚拟空间中。为了实现这一目标，本项目研究了听觉相声消除的网络软件系统。该研究项目研究了以下方面的实际基础问题：(i)用于校准和利用声音传播模型的视觉跟踪驱动反馈系统的设计，（ii）在空间音频系统上优化空间化声音的设计，以及（iii）网络系统基础设施的设计，以便从计算异构的各种物联网设备中生成空间化声音。为了解决这些问题，该项目设计并描述了一系列系统方法，通过增强现实技术和接口调用声学模型的人在环校准，以及利用基于云的虚拟音频传播模拟进行实时决策。该项目通过对原型测试平台的用户研究来评估设计和方法，原型测试平台包括仿真框架、原型无线音频系统以及与现有物联网设备和环境的集成。该研究通过实现基于音频的空间制导系统和空间个人辅助，为物联网提供了变革性的能力。这样的系统将有利于视力受损和记忆受损的人的辅助技术。空间指南还可协助第一响应者引导应急人员前往在陌生环境中被空间跟踪的个人。最后，空间音频系统可以极大地增强商业物联网的能力，提供放置响应声音的能力，允许用户定位放错位置的物体，响应空间定位的虚拟助手，并提供来自无扬声器设备的听觉反馈，例如书籍，电源插座，运动器材等。为了促进更广泛的应用，该项目将发现集成到定制的开源软件框架中。为了扩大使用计算机的机会，该项目开展了外展活动，包括将研究纳入高中夏令营、大学课程以及基于网络和/或增强现实的音频传播模型演示。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

As the Internet of Things proliferates, many networked devices fill the home, the office, and public spaces, often with speakers for auditory response. This project strives to use such speakers to perceptually place auditory sounds in arbitrary virtual spaces. Towards this goal, the project studies networked software systems for auditory crosstalk cancellation. The research project investigates practical foundational issues regarding: (i) the design of a visual tracking-driven feedback system for calibration and utilization of sound propagation models, (ii) the design of sounds that optimally spatialize on the spatial audio system, and (iii) the design of a networked system infrastructure to generate spatialized sounds from a computationally heterogeneous variety of Internet of Things devices. To approach such issues, the project designs and characterizes a family of system methods that invoke human-in-the-loop calibration of acoustical models through augmented reality technologies and interfaces, as well as leveraging cloud-based virtual audio propagation simulations for real-time decision-making. The project evaluates designs and methods through user studies on a prototype testbed that includes simulation frameworks, prototyped wireless audio systems and integration with existing Internet of Things devices and environments.The research offers transformative capabilities to the Internet of Things by enabling audio-based spatial guidance systems and spatial personal assistance. Such systems would be beneficial in assistive technologies for visually impaired and memory impaired individuals. The spatial guidance could also assist first responders in navigating emergency personnel towards spatially tracked individuals in foreign environments. Finally, spatial audio systems could greatly enhance commercial Internet of Things capabilities, offering the ability to place responsive sounds to allow users to e.g., locate misplaced objects, respond to spatially positioned virtual assistants, and provide auditory feedback from speaker-less devices, e.g., books, electrical outlets, exercise equipment, etc. To promote broader utility, the project integrates findings in custom open source software frameworks. Towards broadening access to computing, the project exercises outreach activities, including integrating the research into a high school summer camp, university curricula, and web-based and/or augmented reality demonstrations of audio propagation models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.