Multisensory immersive media and next-generation applications

多感官沉浸式媒体和下一代应用

• 批准号: RTI-2021-00298
• 负责人: Falk, Tiago
• 金额: $ 6.93万
• 依托单位: Institut national de la recherche scientifique
• 依托单位国家: 加拿大
• 项目类别: Research Tools and Instruments
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=718072
• 关键词: Multisensory; immersive; media; next; generation

Immersive communication aims to enable natural experiences and interactions with remote people and environments in ways that blur the line between real and virtual environments. Today, our need for more effective ways to communicate with remote people has never been greater. Existing immersive technologies and applications, however, rely only on two senses: vision and hearing. To enable true immersive communications, we need to involve other senses, such as olfaction, touch, and thermo(re)ception. Imagine yourself watching a movie where you feel the heat from an explosion; a videogame where you can feel the wind in your face as you are driving; TV ads where you can smell the food; multisensory virtual therapy for individuals with anxiety; or a videoconference, where you can "touch" the person on the other side of the call. In fact, Ericsson's ConsumerLab suggests that upcoming technological advances will move us from a screen-based world to a sense-based world by 2030. It is envisioned that the Internet of Things will become the Internet of Senses where mind, touch, smells, taste, sight and sound will all come together with the help of emerging 5G networks and Tactile Internet innovations. Canada, despite being known worldwide for innovations in virtual reality (VR), falls far behind Europe, for example, when it comes to multisensory immersive media research. This proposal aims to fill this gap. We request funds to purchase a multisensory immersive virtual reality booth - the first in Canada for research purposes. It is portable, wheelchair accessible, and comprises a programmable interface in which scents, hot/cold air, arm gestures, haptic feedback, immersive audio, and VR content can be presented for research purposes. This fundamental component will enable a new line of research in Canada, that of multisensory immersive media. When coupled with our award-winning instrumented virtual reality headset and user-aware human factor metrics for quality-of-experience monitoring, applications across multiple verticals will be developed that are optimized for each user. Within the healthcare sector, applications such as multisensory anxiety therapeutics and Alzheimer's disease diagnosis will be explored. For defense, it will enable next-generation training of soldiers and pilots in high-risk situations. For the entertainment sector, new paradigms in gaming, gamer experience, and remote tourism will be enabled. The requested equipment will not only open up a new research frontier for Canada, but will also expose existing and incoming highly qualified personnel to a highly interdisciplinary topic, thus providing young researchers with invaluable cross-disciplinary expertise in domains that are highly sought-after in the marketplace. Represenatative examples include human factors, immersive media, signal processing, serious gaming, neuropsychology, healthcare 4.0, information fusion, big data, and machine learning, to name a few.

身临其境的通信旨在以模糊真实环境和虚拟环境之间的界限的方式，实现与远程人员和环境的自然体验和互动。今天，我们从未像现在这样需要更有效的方式与偏远地区的人们交流。然而，现有的沉浸式技术和应用只依赖两种感官：视觉和听觉。要实现真正的身临其境的交流，我们需要涉及其他感官，如嗅觉、触觉和温度(接收)。想象一下，你正在看一部电影，其中你感受到了爆炸产生的热量；一款视频游戏，你可以在开车时感觉到风吹过你的脸；电视广告中，你可以闻到食物的味道；多感官虚拟疗法，为焦虑的人提供了一种治疗；或者一场视频会议，你可以在视频会议上“触摸”电话另一端的人。事实上，爱立信的消费者实验室表明，到2030年，即将到来的技术进步将把我们从一个基于屏幕的世界带入一个基于感觉的世界。据设想，在新兴的5G网络和触觉互联网创新的帮助下，物联网将成为思想、触觉、嗅觉、味觉、视觉和声音都将汇聚在一起的感官互联网。 尽管加拿大在虚拟现实(VR)方面的创新闻名于世，但在多感官沉浸式媒体研究方面，加拿大远远落后于欧洲。这项提议旨在填补这一空白。我们申请资金购买一个多感官沉浸式虚拟现实展位-这是加拿大第一个用于研究目的的展位。它是便携的，轮椅可接近的，并包括一个可编程的界面，在其中可以展示气味、热/冷空气、手臂手势、触觉反馈、身临其境的音频和VR内容，用于研究目的。这一基本组成部分将在加拿大实现一条新的研究路线，即多感官沉浸式媒体。当与我们屡获殊荣的仪表式虚拟现实耳机和用于体验质量监控的用户感知的人为因素指标相结合时，将开发跨多个垂直领域的应用程序，这些应用程序将为每个用户进行优化。在医疗保健部门，将探索多感觉焦虑疗法和阿尔茨海默病诊断等应用。在防御方面，它将使下一代士兵和飞行员能够在高风险情况下接受培训。对于娱乐业，游戏、玩家体验和远程旅游方面的新模式将被启用。所要求的设备不仅将为加拿大开辟一个新的研究前沿，还将使现有和即将到来的高素质人员接触一个高度跨学科的主题，从而为年轻研究人员在市场上非常受欢迎的领域提供宝贵的跨学科专业知识。代表性的例子包括人为因素、沉浸式媒体、信号处理、严肃游戏、神经心理学、医疗保健4.0、信息融合、大数据和机器学习，仅举几例。