脳機能の特性を活用したクロスモーダルコンテンツ

研究人员通过针对特定场景的跨模态内容的制作和评估，构建了一个原型系统，重点关注人脑功能的特点。在联觉跨模态表征范围内，采用多种方法开发跨模态内容。作为具体的内容制作，创建了利用身体形象的感应来制作用于安全教育的虚拟现实（VR）内容。为了检验实际应用的可能性，进行了一项调查，并根据结果提出了针对各种认知状态的内容开发。作为涉及在虚拟空间中获取参与者的身体图像的认知体验的评估方法，使用脑功能测量技术近红外光谱（NIRS）来检验是否可以进行客观评估。结果表明，可以通过观察特定的大脑区域来评估身体意象的感应。还检查了通过大脑功能测量获得的数据的分析方法，研究人员能够通过趋势分析消除噪音。在关于身体意象感应的检查中，研究人员表明，通过应用视觉和触觉跨模态刺激，可以感知到与所呈现的不同的触觉运动错觉。研究人员评估了这种错觉，并发现了主观评估与 NIRS 数据结果之间的关系。此外，研究人员还检查了这种联觉现象，这种联觉现象导致了参与者实际上并未呈现出的触觉。NIRS 数据的结果表明存在联觉体验。从这项研究结果中，研究人员获得了许多具体且有用的发现。这些成果将有助于核心技术的发展，从而促进新型媒体内容的未来发展。

The researchers have constructed a prototype system through production and evaluation of cross-modal contents for specific situations, focusing on the characteristics of human brain function. Multiple approaches were used in developing cross-modal contents within the range of synesthetic cross-modal representation.As specific content production, production of virtual reality (VR) content for safety education utilizing the induction of body image was created. A survey was performed to examine the possibility of practical application, and content development for various cognitive statuses was suggested from the results.As the evaluation method for cognitive experience that involves obtaining the participant's body image in a virtual space, a brain function measurement technique, near infrared spectroscopy (NIRS), was used to examine whether objective evaluation is possible. The results indicated that the induction of body image could be evaluated by observing specific brain regions. The method of analysis of data obtained through brain function measurement was also examined, and the researchers were able to remove noise through trend analysis.During examination regarding induction of body image, the researchers showed that by applying visual and tactile cross-modal stimulation, an illusion of haptic movement is perceived, which is different from what is presented. The researchers evaluated this illusion, and found a relationship between subjective evaluation and NIRS data results. In addition, the researchers examined this synesthetic phenomenon causing a sense of touch that is not actually presented to the participant The results from NIRS data indicated the presence of a synesthetic experience. From the results of this research, the researchers obtained many specific and useful findings. These results will contribute to the development of core technology that may facilitate the future development of new types of media content.