Spatial Sound Generation and Perceptual-Based Rendering for Virtual Environments and Serious Games

点击翻译按钮获取中文摘要

Spatial sound within a virtual environment such as a virtual simulation, serious game, or video game, allows users to perceive the position of a sound source at an arbitrary position in three-dimensional space and when properly reproduced, it can deliver a very life-like sense of being remotely immersed in the presence of people, musical instruments, and environmental sounds. Spatial sound can add a new layer of realism, contributing to a greater user experience and improved task performance, while conveying information that would otherwise be difficult to convey using other modalities (e.g., vision). However, spatial sound is often overlooked in immersive virtual environments where traditionally, emphasis has been placed on the visual scene (i.e., the “computer graphics”) instead. That being said, spatial sound generation for dynamic, interactive, and immersive virtual environments is difficult and computationally expensive except for trivial environments which are of limited use. Within the scope of the proposed research program, a virtual sound rendering engine (VSRE) that will allow for the incorporation of spatial sound within an interactive, dynamic virtual environments in a simple and efficient manner will be developed. The VSRE will be rigorously validated through a series of quantitative and qualitative techniques and the validation results will be made available to the research community thus allowing others to compare the VSRE with their own spatial sound systems. The VSRE will also be developed to allow for the simple manipulation of the acoustic environment and audio parameters (sound quality). It will then be incorporated into an existing serious game thus forming a “research test-bed” which will be used to test what effect varying the acoustic environment and audio parameters may have on immersion, engagement, and visual fidelity perception within a virtual environment via a series of experiments that will be conducted with human participants. Finally, a method will be developed that will allow various aspects of a virtual environment including levels of audio and visual fidelity, and background sound, to be tailored specifically to the user via a brief calibration process that the user will complete prior to using the virtual environment. This will result in a more engaging user experience. The proposed research program will lead to i) the development of a novel and unique virtual sound rendering engine that will allow for the inclusion of spatial sound within virtual environments at interactive rates, ii) a greater understanding of the role that spatial sound and its interaction with the visual cues play in interactive virtual environments and how spatial sound can be used to overcome potential limitations inherent in these environments, iii) a novel calibration method that will tailor the virtual environment specifically to each user, and iv)a set of “best practices” (guidelines) for designers/developers of multi-modal virtual environments (particularly virtual simulations, and serious games), to maximize engagement, immersion and overall user experience within such environments. Collectively, the proposed research program will result in more effective virtual simulations and serious games ultimately leading to improved training of a specific workforce (surgeons/healthcare professionals, police officers, etc.) in a safe, efficient, and cost-effective manner, ensuring they are better prepared to carry out their intended tasks.