Scene-Aware Audio Rendering via Deep Acoustic Analysis
Scene-Aware Audio Rendering via Deep Acoustic Analysis
复制标题
DOI：
10.1109/tvcg.2020.2973058
复制
发表时间：
2020-05-01
影响因子：
5.2
通讯作者：
Manocha, Dinesh
中科院分区：
文献类型：
--
作者：
Tang, Zhenyu;Bryan, Nicholas J.;Manocha, Dinesh
我们提出了一种新的方法来捕获使用商品设备的真实世界的房间的声学特性，并使用捕获的特性来生成类似的声源与虚拟模型。给定捕获的音频和真实世界房间的近似几何模型，我们提出了一种新的基于学习的方法来估计其声学材料特性。我们的方法基于深度神经网络，可以从录制的音频中估计混响时间和房间的均衡。这些估计被用来计算材料属性相关的房间混响使用一种新的材料优化目标。我们使用估计的声学材料特性的音频渲染使用交互式几何声音传播和突出的性能在许多现实世界的情况下。我们还进行了用户研究，以评估录制的声音和我们渲染的音频之间的感知相似性。
We present a new method to capture the acoustic characteristics of real-world rooms using commodity devices, and use the captured characteristics to generate similar sounding sources with virtual models. Given the captured audio and an approximate geometric model of a real-world room, we present a novel learning-based method to estimate its acoustic material properties. Our approach is based on deep neural networks that estimate the reverberation time and equalization of the room from recorded audio. These estimates are used to compute material properties related to room reverberation using a novel material optimization objective. We use the estimated acoustic material characteristics for audio rendering using interactive geometric sound propagation and highlight the performance on many real-world scenarios. We also perform a user study to evaluate the perceptual similarity between the recorded sounds and our rendered audio.