CAREER: Acoustic Ambient Computing: Algorithms, Architectures, and Prototypes

声学在连接人类和计算机基础设施方面发挥着至关重要的作用。语音助手创造了一种与物联网设备交互的自然方式；具有传感和辅助功能的智能耳机正在兴起；声学增强现实即将到来。该提案通过在日常环境中嵌入的低功耗、无电池的计算平台上实现先进的声学传感和推理，创造了一种称为声学环境计算的范式，从而设想了该计算领域的新前沿。该范式旨在通过环境、基础设施嵌入式传感来理解人类活动、动作、手势和行为。目标应用包括数据驱动的健康技术和医院的身体恢复跟踪、车间的沉浸式辅助、办公室和协作工作场所的积极行为强化，甚至是公园或公交车站等开放空间的集成。本研究的重点是开发理论构建模块、算法和硬件软件原型，以实现低成本、可扩展和可持续的解决方案，以声音作为无处不在的感知方式来感知人类活动和互动。该项目的工作分为三个研究重点：(a)第一个重点使人们能够在墙壁、地板和桌面等日常表面附近感知大规模的手势和运动。它是通过一种新颖的编码信号投影方法实现的，该方法利用了物理层的无源微结构和自适应间歇计算架构。(b)第二次推力放大到微米级运动和微手势检测。本研究模块探索了新的跨光谱传感和定位技术，以及一种轻量级的基于神经注意的选择性分辨率处理方法。(c)最后的研究重点是将物理环境的概念引入到室内和室外运动、手势和微活动中的声音和语音交互中。该项目将建立一个非可穿戴模式的关键组成部分，在日常环境中交织传感和推理，使计算服务可以毫不费力地融入人们的日常生活。开发的平台和原型将在交互式家庭自动化、辅助和医疗保健、协作工作空间和家庭安全方面促进一系列新的应用。它将有助于创建改造现有建筑的框架，并增加新的室内和室外环境，将它们变成交互式智能空间。本研究将探讨嵌入式智能、无线传感、低功耗网络系统和人机交互的相互作用。从这个项目中获得的见解还将导致新的学习模块，用于向学生教授无设备计算的基础知识。此外，本项目将向本科生和K-12学生介绍低功耗嵌入式智能的概念。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Acoustics play a crucial role in connecting humans to computing infrastructure. Voice assistants have created a natural means of interacting with IoT devices; smart earphones are emerging with sensing and assistive skills; and acoustic augmented reality is around the corner. This proposal envisions a new frontier in this computing space by enabling advanced acoustic sensing and inference on low-power, battery-free computing platforms embedded in everyday environments — creating a paradigm called acoustic ambient computing. The paradigm aims to comprehend human activities, motions, gestures, and behavior through ambient, infrastructure-embedded sensing. Target applications include data-driven health technologies and physical recovery tracking in hospitals, immersive assistance at workshops, positive behavioral reinforcement in offices and collaborative workplaces, and even integration in open spaces such as parks or bus stops. This research focuses on developing theoretical building blocks, algorithms, and hardware-software prototypes to realize low-cost, scalable, and sustainable solutions for sensing human activities and interactions using sound as a ubiquitous sensing modality. The project efforts are organized as three research thrusts: (a) The first thrust enables sensing of large-scale gestures and movements near everyday surfaces, like walls, floors, and tabletops. It is realized through a novel coded signal projection method using passive micro-structures at the physical layer and an adaptive intermittent computing architecture. (b) The second thrust zooms into micro-meter level motion and micro gesture detection. This research module explores novel cross-spectrum sensing and localization techniques and a lightweight neural attention-based selective resolution approach for processing. (c) The final research thrust introduces the notion of physical context to voice and speech interactions across indoor and outdoor movements, gestures, and micro-activities. This project will establish the critical components of a wearable-free paradigm to interweave sensing and inference within everyday environments so that computing services can be effortlessly blended into people’s daily lives. The developed platforms and prototypes will foster a range of new applications in interactive home automation, assistance and healthcare, collaborative workspaces, and home security. It will help create frameworks for retrofitting existing buildings and augmenting new indoor and outdoor environments to turn them into interactive smart spaces. The proposed research examines the interplay of embedded intelligence, wireless sensing, low-power networking systems, and human-computer interaction. Insights from this project will also lead to new learning modules used to teach the fundamentals of device-free computing to students. Moreover, this project will introduce the concepts of low-power embedded intelligence to undergraduate and K-12 students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.