Spatial Sound Generation and Perceptual-Based Rendering for Virtual Environments and Serious Games

虚拟环境和严肃游戏的空间声音生成和基于感知的渲染

• 批准号: RGPIN-2014-05640
• 负责人: Kapralos, Bill
• 金额: $ 1.46万
• 依托单位: University of Ontario Institute of Technology
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2014
• 资助国家: 加拿大
• 起止时间: 2014-01-01 至 2015-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=567295
• 关键词: Spatial; Sound; Generation; Perceptual; Based

Spatial sound within a virtual environment such as a virtual simulation, serious game, or video game, allows users to perceive the position of a sound source at an arbitrary position in three-dimensional space and when properly reproduced, it can deliver a very life-like sense of being remotely immersed in the presence of people, musical instruments, and environmental sounds. Spatial sound can add a new layer of realism, contributing to a greater user experience and improved task performance, while conveying information that would otherwise be difficult to convey using other modalities (e.g., vision). However, spatial sound is often overlooked in immersive virtual environments where traditionally, emphasis has been placed on the visual scene (i.e., the “computer graphics”) instead. That being said, spatial sound generation for dynamic, interactive, and immersive virtual environments is difficult and computationally expensive except for trivial environments which are of limited use. Within the scope of the proposed research program, a virtual sound rendering engine (VSRE) that will allow for the incorporation of spatial sound within an interactive, dynamic virtual environments in a simple and efficient manner will be developed. The VSRE will be rigorously validated through a series of quantitative and qualitative techniques and the validation results will be made available to the research community thus allowing others to compare the VSRE with their own spatial sound systems. The VSRE will also be developed to allow for the simple manipulation of the acoustic environment and audio parameters (sound quality). It will then be incorporated into an existing serious game thus forming a “research test-bed” which will be used to test what effect varying the acoustic environment and audio parameters may have on immersion, engagement, and visual fidelity perception within a virtual environment via a series of experiments that will be conducted with human participants. Finally, a method will be developed that will allow various aspects of a virtual environment including levels of audio and visual fidelity, and background sound, to be tailored specifically to the user via a brief calibration process that the user will complete prior to using the virtual environment. This will result in a more engaging user experience. The proposed research program will lead to i) the development of a novel and unique virtual sound rendering engine that will allow for the inclusion of spatial sound within virtual environments at interactive rates, ii) a greater understanding of the role that spatial sound and its interaction with the visual cues play in interactive virtual environments and how spatial sound can be used to overcome potential limitations inherent in these environments, iii) a novel calibration method that will tailor the virtual environment specifically to each user, and iv)a set of “best practices” (guidelines) for designers/developers of multi-modal virtual environments (particularly virtual simulations, and serious games), to maximize engagement, immersion and overall user experience within such environments. Collectively, the proposed research program will result in more effective virtual simulations and serious games ultimately leading to improved training of a specific workforce (surgeons/healthcare professionals, police officers, etc.) in a safe, efficient, and cost-effective manner, ensuring they are better prepared to carry out their intended tasks.

诸如虚拟模拟、严肃游戏或视频游戏之类的虚拟环境中的空间声音允许用户感知声源在三维空间中任意位置的位置，并且当适当地再现时，它可以提供一种非常逼真的感觉，即远程沉浸在人、乐器和环境声音的存在中。空间声音可以增加一层新的现实感，有助于更好的用户体验和改进的任务性能，同时传达使用其他方式(如视觉)难以传达的信息。然而，在沉浸式虚拟环境中，空间声音往往被忽视，传统上，空间声音的重点放在视觉场景(即计算机图形学)上。尽管如此，动态、交互和沉浸式虚拟环境的空间声音生成是困难的，并且计算成本很高，除了使用有限的琐碎环境。在拟议研究方案的范围内，将开发一个虚拟声音再现引擎，它将允许以简单和有效的方式将空间声音纳入互动的动态虚拟环境。VSRE将通过一系列定量和定性技术进行严格验证，验证结果将提供给研究界，从而允许其他人将VSRE与他们自己的空间音响系统进行比较。还将开发VSRE，以允许对声学环境和音频参数(音质)进行简单操作。然后，它将被整合到现有的严肃游戏中，从而形成一个“研究试验台”，用于通过与人类参与者进行的一系列实验，测试改变声学环境和音频参数对虚拟环境中的沉浸感、参与度和视觉保真度感知的影响。最后，将开发一种方法，该方法将允许通过用户将在使用虚拟环境之前完成的简短校准过程来为用户专门定制虚拟环境的各个方面，包括音频和视觉保真度级别以及背景声音。这将带来更吸引人的用户体验。所提议的研究计划将导致i)开发新颖且独特的虚拟声音呈现引擎，其将允许以交互速率在虚拟环境内包括空间声音，ii)更好地理解空间声音及其与视觉线索的交互在交互式虚拟环境中所起的作用，以及如何使用空间声音来克服这些环境中固有的潜在限制，iii)将专门为每个用户定制虚拟环境的新的校准方法，以及iv)为多模式虚拟环境(尤其是虚拟模拟和严肃游戏)的设计者/开发者提供的一组最佳实践(指南)，以最大化参与度，在这样的环境中沉浸和整体用户体验。总的来说，拟议的研究计划将产生更有效的虚拟模拟和严肃的游戏，最终改善对特定劳动力(外科医生/医疗保健专业人员、警察等)的培训。以安全、高效和成本效益的方式，确保他们为执行预期任务做好更好的准备。