CHS: Small: Collaborative Research: EEG-Guided Electrical Stimulation for Immersive Virtual Reality

CHS：小型：合作研究：脑电图引导的沉浸式虚拟现实电刺激

• 批准号: 1717654
• 负责人: Murat Akcakaya
• 金额: $ 35.7万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717654&HistoricalAwards=false
• 关键词: CHS; Small; Collaborative; Research; EEG

Spatial presence, in Virtual Reality (VR) terminology, refers to the perception (or illusion) of being physically present in a simulated environment. VR strives to create interactive environments that provide experiences of spatial presence through accurate delivery and perception of multimodal sensory stimuli. Research in VR spans fields ranging from neuroscience and medicine to gaming. While the computing and gaming industries have generated tremendous advances in hardware and software for graphics processing and 3D display technologies, VR systems still lack capabilities for providing users with haptic feedback (a sense of touch), which is crucial for generating truly immersive, real-world experiences. It is known that an increase in the feeling of spatial presence manifests itself in the form of increased brain activity. This research aims to achieve the control of haptic sensory stimulation adaptively, based on the changes in brain activity associated with perceptual responses elicited by sensory stimulation in VR environments. Project outcomes will include novel scientific discoveries and engineering enhancements that will make significant contributions to other areas of interest, such as prosthetic limbs, augmented reality, and telepresence applications. The project will help train a new generation of engineers skilled in addressing multidisciplinary challenges, while through outreach activities STEM careers will be promoted at the K-12 level.The research objective is to identify and analyze brain activity associated with the increased feeling of haptic spatial presence elicited by electro-tactile stimulation and measured through EEG, and to investigate closed-loop techniques to control electro-tactile stimulation for enhanced haptic presence in VR environments. Specifically, the project will: (1) develop an electrical haptic stimulation framework; (2) design analysis techniques to identify markers of haptic inputs in EEG; (3) establish control policies for adaptive electrical stimulation; and (4) evaluate and refine EEG-guided adaptive stimuli control framework in VR environments. In particular, the proposition to actuate haptic feedback through electrical stimulation is novel, while formulating design principles for model-based optimal EEG-guided closed-loop haptic feedback for immersive spatial presence is transformative. Additional innovative propositions to advance adaptive control under uncertainty and psychophysical investigations are unique; these present a potentially game-changing opportunity for VR system development and perhaps for general human-computer interaction.

在虚拟现实（VR）术语中，空间存在是指在模拟环境中物理存在的感知（或错觉）。 VR致力于创建交互式环境，通过准确传递和感知多模态感官刺激来提供空间存在体验。 VR的研究涵盖了从神经科学和医学到游戏的各个领域。 虽然计算和游戏行业在图形处理和3D显示技术的硬件和软件方面取得了巨大的进步，但VR系统仍然缺乏为用户提供触觉反馈（触觉）的能力，这对于生成真正沉浸式的真实世界体验至关重要。 众所周知，空间存在感的增加以大脑活动增加的形式表现出来。 本研究的目的是实现自适应的触觉感官刺激的控制，在VR环境中的感觉刺激引起的感知反应的大脑活动的变化的基础上。 项目成果将包括新的科学发现和工程改进，这将为其他感兴趣的领域做出重大贡献，如假肢，增强现实和远程呈现应用。 该项目将帮助培养新一代的工程师，他们有能力应对多学科的挑战，同时通过推广活动，STEM职业将在K-12水平得到推广。研究目标是识别和分析与触觉空间存在感增加相关的大脑活动，触觉空间存在感由电触觉刺激引起，并通过EEG测量，并研究闭环技术来控制电触觉刺激，以增强VR环境中的触觉存在。 具体而言，该项目将：（1）开发一个电触觉刺激框架;（2）设计分析技术，以识别EEG中触觉输入的标记;（3）建立自适应电刺激的控制策略;（4）评估和改进VR环境中EEG引导的自适应刺激控制框架。 特别是，通过电刺激致动触觉反馈的提议是新颖的，而制定基于模型的最佳EEG引导闭环触觉反馈的沉浸式空间存在的设计原则是变革性的。 在不确定性和心理物理学研究下推进自适应控制的其他创新主张是独一无二的;这些为VR系统开发以及可能的一般人机交互提供了潜在的改变游戏规则的机会。

项目成果

EEG-based trial-by-trial texture classification during active touch.

• DOI: 10.1038/s41598-020-77439-7
• 发表时间: 2020-11-27
• 期刊: Scientific reports
• 影响因子: 4.6
• 作者: Eldeeb S;Weber D;Ting J;Demir A;Erdogmus D;Akcakaya M
• 通讯作者: Akcakaya M

Optimal Modality Selection Using Information Transfer Rate for Event Related Potential Driven Brain Computer Interfaces

使用事件相关电位驱动脑机接口的信息传输率进行最佳模态选择

• 发表时间: 2020
• 期刊: ACM PETRA
• 作者: A. Kocanaogullari, M. Akcakaya

On Analysis of Active Querying for Recursive State Estimation

递归状态估计的主动查询分析

• DOI: 10.1109/lsp.2018.2823271
• 发表时间: 2018
• 期刊: IEEE Signal Processing Letters
• 影响因子: 3.9
• 作者: Kocanaogullari, Aziz;Erdogmus, Deniz;Akcakaya, Murat
• 通讯作者: Akcakaya, Murat

Dynamic System Identification For Guidance Of Stimulation Parameters In Haptic Simulation Environments

用于指导触觉模拟环境中的刺激参数的动态系统识别

• DOI: 10.1109/mlsp.2019.8918815
• 发表时间: 2019
• 期刊: 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP
• 作者: Demir, Andac;Eldeeb, Safaa;Akcakaya, Murat;Erdogmus, Deniz
• 通讯作者: Erdogmus, Deniz

Stopping Criterion Design for Recursive Bayesian Classification: Analysis and Decision Geometry

• DOI: 10.1109/tpami.2021.3075915
• 发表时间: 2020-07
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Aziz Koçanaoğulları;M. Akçakaya;Deniz Erdoğmuş