Brain Dynamics in Cyber-Physical Systems as Measure of User Presence

网络物理系统中的大脑动力学作为用户存在的测量

• 批准号: 462163815
• 负责人: Professor Dr. Klaus Gramann
• 金额: --
• 依托单位: Fachgebiet Biopsychologie und Neuroergonomie
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/462163815?language=en
• 关键词: Brain; Dynamics; Cyber; Physical; Systems

Cyber-physical systems (CPS) allow for controlling or monitoring physical processes utilizing computer-based algorithms. CPS are transforming work places as well as daily interactions with technical systems. Virtual environments (VEs) are one example of advanced CPS that situate users in diverse environments including human-machine interaction settings that require dynamic and precise interaction with objects or other agents. Precise interactions in CPS, like in the real world, depend on the users’ presence as well as their ability to deal with uncertainties of dynamic changes in the environment. Only when users behave in VEs as they would do in the real world, they experience presence, the subjective feeling of being in the virtual place even though they are physically located in another place. The users’ ability to precisely interact with a technical system requires near-natural sensory feedback to reduce uncertainty regarding the action outcome in complex and dynamic environments by means of predictive processing. Until now, presence in VE applications was assessed using self-report questionnaires. This method interrupts the feeling of presence and is prone to problems associated with introspection. Alternatively, peripheral physiological measures are used that require emotional stimulation to trigger a measurable response while providing only limited temporal resolution (in the range of seconds) about the ongoing cognitive and affective processes. After decades of research, no standardized and objective presence metric has been identified to assess user presence without interrupting the ongoing VE experience. We aim to overcome existing limitations of presence metrics by investigating brain dynamics in combination with peripheral physiology accompanying multisensory integration during naturalistic interaction with a virtual system. We propose to develop a new method to unobtrusively and continuously assess the user’s experience of presence using a Mobile Brain/Body Imaging (MoBI) approach. We will use electroencephalography (EEG) synchronized with electrocardiography (ECG) and measures of electrodermal activity (EDA) plus motion capture and virtual reality technology while using data-driven analyses approaches to develop a multimodal presence metric. We will investigate event-related physiological activity related to deviations of predicted and real interaction outcomes and how these correspond to established questionnaire metrics of presence. We will further investigate how noise in different sensory systems influences presence experience and how multisensory integration of visual and vibrotactile feedback during a precision task in VE is realized on a neuronal level. The results of this project aim at establishing a new presence assessment that will allow us to continuously assess user presence in CPS without disrupting the interaction itself while quantifying the importance of different sensory modalities for experiencing presence.

网络物理系统（CPS）允许利用基于计算机的算法控制或监视物理过程。CPS正在改变工作场所以及与技术系统的日常互动。虚拟环境（ve）是高级CPS的一个例子，它将用户置于不同的环境中，包括需要与对象或其他代理进行动态和精确交互的人机交互设置。与现实世界一样，CPS中的精确交互取决于用户的存在以及他们处理环境动态变化的不确定性的能力。只有当用户在虚拟世界中表现得像在现实世界中一样时，他们才会体验到存在感，即即使他们的物理位置在另一个地方，但仍能感受到置身于虚拟世界的主观感觉。用户精确地与技术系统交互的能力需要近乎自然的感官反馈，以减少在复杂和动态环境中通过预测处理的行动结果的不确定性。到目前为止，VE应用程序的存在是通过自我报告问卷来评估的。这种方法打断了存在感，并且容易出现与自省有关的问题。另外，外围生理测量需要情绪刺激来触发可测量的反应，而对正在进行的认知和情感过程只提供有限的时间分辨率（在秒的范围内）。经过几十年的研究，目前还没有确定一个标准化的、客观的存在度指标来评估用户的存在度，同时又不会中断VE的持续体验。我们的目标是克服存在度量的现有局限性，通过研究大脑动力学与周边生理学相结合，在与虚拟系统的自然互动过程中伴有多感觉整合。我们建议开发一种新方法，使用移动脑/体成像（MoBI）方法来不引人注目地持续评估用户的存在体验。我们将使用与心电图（ECG）同步的脑电图（EEG）和皮电活动（EDA）测量以及动作捕捉和虚拟现实技术，同时使用数据驱动的分析方法来开发多模态存在度量。我们将研究与预测和实际交互结果偏差相关的事件相关生理活动，以及这些偏差如何与既定的问卷存在度量相对应。我们将进一步研究不同感觉系统中的噪声如何影响在场体验，以及视觉和振动触觉反馈在VE精确任务中的多感觉整合如何在神经元水平上实现。该项目的结果旨在建立一种新的存在评估，使我们能够在不破坏交互本身的情况下持续评估CPS中的用户存在，同时量化体验存在的不同感官模式的重要性。