I-Corps: Creating Immersive 3D Audio

人类（和其他动物）从听觉输入感知空间的过程是复杂的。 在过去的几十年里，世界各地的研究人员（包括参与当前提案的人员）已经阐明了人类感知声音的机制，并能够开发用于创建沉浸式音频的算法。其中包括来自游戏、混合音乐和现场活动捕捉的音频。 由于可听声音的波长与环境物体和收听者身体上的特征的大小相当，因此收听者感知到的声音通过散射这些物体而被“着色”。理解和有效地逼近分别表征这些散射过程的房间脉冲响应和头部相关传递函数（HRTF），一直是PI在过去十年研究的主要贡献。作为每一个听众？的身体特征有不同的形状和大小，HRTF显示出相当大的人际差异。PI开发了非常快速的技术，用于测量HRTF，用于基于收听者的身体和耳朵测量来表征HRTF，以及用于使用HRTF和房间响应来快速创建沉浸式声音。此外，单个麦克风记录丢失了声音信号中的空间信息。为了在听觉环境捕获期间保留空间信息，PI开发了新颖的球形麦克风阵列技术。捕获的声音在平面波基函数方面的分解允许在捕获的声音中容易地并入HRTF。今天的消费者在他们的移动的设备上享受音乐、游戏和其他媒体;大多数人满足于耳机产生的平淡声音。目前的耳机声音根本无法产生高端音乐系统、电影院或现场活动所提供的迷人的声音体验。凭借在之前NSF支持下开发的技术，该团队允许娱乐内容创作者通过耳机实现无与伦比的声音呈现。结合团队的声音捕捉硬件，可以实现现场活动的沉浸式再现。去年，智能手机的销售量在历史上首次超过了PC;因此，通过耳机消费的内容数量正在大幅增加。我们的技术将使公司能够使耳机收听更加身临其境，并接触到不断增长的消费者群体。有几个行业生产和消费音频产品。其中包括移动终端/游戏/ PC硬件制造商、娱乐内容制作商和内容交付网络。他们都将受到这项技术的积极影响。由于巨大的市场和团队的力量，他们预计可以围绕这项技术建立一个大型和成功的业务。如果成功，这一努力将有助于美国在移动的技术、游戏、娱乐和沉浸式模拟领域的持续领导地位。

The process by which humans (and other animals) perceive space from auditory input is complex. Over the past decades, researchers worldwide (including those involved in the current proposal) have elucidated the mechanisms by which humans perceive sound and are able to develop algorithms for the creation of immersive audio. These include audio from games, from the mixing of music, and from the capture of live events. Because the wavelengths of audible sound are comparable to the sizes of environmental objects and of features on the listeners' bodies, the sound perceived by the listener is "colored" by scattering off these objects. Understanding and efficiently approximating the room impulse response and the Head Related Transfer Function (HRTF), which respectively characterize these scattering processes, has been a major contribution of the PI's research over the past decade. As each listener?s physical body features have different shapes and sizes, the HRTF shows considerable inter-personal variation. The PI has developed extremely rapid techniques for measuring HRTFs, for characterizing HRTFs based on the body and ear measurements of listeners, and for rapid creation of immersive sound using HRTFs and room responses. Further, single microphone recordings lose the spatial information in the sound signal. To retain the spatial information during auditory environment capture, the PI has developed novel spherical microphone array technology. A decomposition of the captured sound in terms of plane-wave basis functions allows the easy incorporation of HRTFs in the captured sound. Today's consumers enjoy music, games, and other media on their mobile devices; most settle for lackluster sound produced over headphones. Current headphone sound is simply unable to produce the engaging sound experience that high-end music systems, movie theaters, or live events deliver. With the technologies developed under previous NSF support, the team has allowed entertainment content creators to achieve unmatched realism in sound presentation over headphones. Combined with the team's sound capture hardware, immersive reproduction of live events can be achieved. In the last year, the number of smartphone sales outpaced that of PCs for the first time in history; consequentially, the amount of content consumed over headphones is vastly increasing. Our technology will allow companies to make headphone listening more immersive and reach that growing consumer base. Several industries produce and consume audio products. These include mobile device / gaming / PC hardware manufacturers, entertainment content producers, and content delivery networks. They will all be positively impacted by the technology. Because of the large market, and the strength of the team, they anticipate that a large and successful business can be built around this technology. If successful, this effort will contribute to ongoing U.S. leadership in the fields of mobile technologies, gaming, entertainment, and immersive simulation.