I-Corps: Creating Immersive 3D Audio

点击翻译按钮获取中文摘要

The process by which humans (and other animals) perceive space from auditory input is complex. Over the past decades, researchers worldwide (including those involved in the current proposal) have elucidated the mechanisms by which humans perceive sound and are able to develop algorithms for the creation of immersive audio. These include audio from games, from the mixing of music, and from the capture of live events. Because the wavelengths of audible sound are comparable to the sizes of environmental objects and of features on the listeners' bodies, the sound perceived by the listener is "colored" by scattering off these objects. Understanding and efficiently approximating the room impulse response and the Head Related Transfer Function (HRTF), which respectively characterize these scattering processes, has been a major contribution of the PI's research over the past decade. As each listener?s physical body features have different shapes and sizes, the HRTF shows considerable inter-personal variation. The PI has developed extremely rapid techniques for measuring HRTFs, for characterizing HRTFs based on the body and ear measurements of listeners, and for rapid creation of immersive sound using HRTFs and room responses. Further, single microphone recordings lose the spatial information in the sound signal. To retain the spatial information during auditory environment capture, the PI has developed novel spherical microphone array technology. A decomposition of the captured sound in terms of plane-wave basis functions allows the easy incorporation of HRTFs in the captured sound. Today's consumers enjoy music, games, and other media on their mobile devices; most settle for lackluster sound produced over headphones. Current headphone sound is simply unable to produce the engaging sound experience that high-end music systems, movie theaters, or live events deliver. With the technologies developed under previous NSF support, the team has allowed entertainment content creators to achieve unmatched realism in sound presentation over headphones. Combined with the team's sound capture hardware, immersive reproduction of live events can be achieved. In the last year, the number of smartphone sales outpaced that of PCs for the first time in history; consequentially, the amount of content consumed over headphones is vastly increasing. Our technology will allow companies to make headphone listening more immersive and reach that growing consumer base. Several industries produce and consume audio products. These include mobile device / gaming / PC hardware manufacturers, entertainment content producers, and content delivery networks. They will all be positively impacted by the technology. Because of the large market, and the strength of the team, they anticipate that a large and successful business can be built around this technology. If successful, this effort will contribute to ongoing U.S. leadership in the fields of mobile technologies, gaming, entertainment, and immersive simulation.