Brain Dynamics in Cyber-Physical Systems as Measure of User Presence

网络物理系统中的大脑动力学作为用户存在的测量

• 批准号: 462163815
• 负责人: Professor Dr. Klaus Gramann
• 金额: --
• 依托单位: Fachgebiet Biopsychologie und Neuroergonomie
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/462163815?language=en
• 关键词: Brain; Dynamics; Cyber; Physical; Systems

Cyber-physical systems (CPS) allow for controlling or monitoring physical processes utilizing computer-based algorithms. CPS are transforming work places as well as daily interactions with technical systems. Virtual environments (VEs) are one example of advanced CPS that situate users in diverse environments including human-machine interaction settings that require dynamic and precise interaction with objects or other agents. Precise interactions in CPS, like in the real world, depend on the users’ presence as well as their ability to deal with uncertainties of dynamic changes in the environment. Only when users behave in VEs as they would do in the real world, they experience presence, the subjective feeling of being in the virtual place even though they are physically located in another place. The users’ ability to precisely interact with a technical system requires near-natural sensory feedback to reduce uncertainty regarding the action outcome in complex and dynamic environments by means of predictive processing. Until now, presence in VE applications was assessed using self-report questionnaires. This method interrupts the feeling of presence and is prone to problems associated with introspection. Alternatively, peripheral physiological measures are used that require emotional stimulation to trigger a measurable response while providing only limited temporal resolution (in the range of seconds) about the ongoing cognitive and affective processes. After decades of research, no standardized and objective presence metric has been identified to assess user presence without interrupting the ongoing VE experience. We aim to overcome existing limitations of presence metrics by investigating brain dynamics in combination with peripheral physiology accompanying multisensory integration during naturalistic interaction with a virtual system. We propose to develop a new method to unobtrusively and continuously assess the user’s experience of presence using a Mobile Brain/Body Imaging (MoBI) approach. We will use electroencephalography (EEG) synchronized with electrocardiography (ECG) and measures of electrodermal activity (EDA) plus motion capture and virtual reality technology while using data-driven analyses approaches to develop a multimodal presence metric. We will investigate event-related physiological activity related to deviations of predicted and real interaction outcomes and how these correspond to established questionnaire metrics of presence. We will further investigate how noise in different sensory systems influences presence experience and how multisensory integration of visual and vibrotactile feedback during a precision task in VE is realized on a neuronal level. The results of this project aim at establishing a new presence assessment that will allow us to continuously assess user presence in CPS without disrupting the interaction itself while quantifying the importance of different sensory modalities for experiencing presence.

信息物理系统（CPS）允许利用基于计算机的算法来控制或监测物理过程。CPS正在改变工作场所以及与技术系统的日常互动。虚拟环境（VE）是高级CPS的一个示例，其在不同的环境（包括需要与对象或其他代理进行动态和精确交互的人机交互设置）中为用户提供服务。CPS中的精确交互，就像在真实的世界中一样，取决于用户的存在以及他们处理环境中动态变化的不确定性的能力。只有当用户在虚拟环境中的行为与他们在真实的世界中的行为一样时，他们才能体验到在场，即即使他们实际上位于另一个地方，也能体验到在虚拟场所的主观感觉。用户精确地与技术系统交互的能力需要接近自然的感觉反馈，以通过预测处理来减少关于复杂和动态环境中的动作结果的不确定性。到目前为止，存在于VE应用程序进行评估，使用自我报告问卷。这种方法中断了存在感，并且容易出现与内省相关的问题。可替代地，使用需要情绪刺激以触发可测量的反应的外周生理测量，同时仅提供关于正在进行的认知和情感过程的有限的时间分辨率（在秒的范围内）。经过几十年的研究，没有标准化和客观的存在度量已被确定为评估用户的存在，而不中断正在进行的VE体验。我们的目标是克服现有的存在指标的局限性，通过调查脑动力学与周边生理伴随多感官整合过程中的自然互动与虚拟系统相结合。我们建议开发一种新的方法，使用移动的大脑/身体成像（MoBI）的方法来不引人注目地和连续地评估用户的存在体验。我们将使用与心电图（ECG）同步的脑电图（EEG）和皮肤电活动（EDA）测量加上运动捕捉和虚拟现实技术，同时使用数据驱动的分析方法来开发多模态存在度量。我们将研究与预测和真实的交互结果的偏差相关的事件相关的生理活动，以及这些活动如何对应于既定的问卷调查指标。我们将进一步研究如何在不同的感觉系统的噪声影响存在的经验，以及如何在VE中的精确任务的视觉和振动触觉反馈的多感官整合是在神经元水平上实现的。该项目的结果旨在建立一个新的存在评估，这将使我们能够不断评估用户在CPS的存在，而不会破坏互动本身，同时量化的重要性，不同的感官形式体验存在。