CHS: Small: Auditory and Haptic Based Brain-Computer Interfaces Using In-Ear Electrodes

CHS：小型：使用入耳式电极的基于听觉和触觉的脑机接口

• 批准号: 1718705
• 负责人: Melody Jackson
• 金额: $ 49.77万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1718705&HistoricalAwards=false
• 关键词: CHS; Small; Auditory; Haptic; Based

Brain Computer Interfaces (BCIs) are systems that allow people to control computers and other devices with brain signals alone. BCIs have the potential to improve the lives of many individuals with severe physical disabilities, by allowing them to communicate and control their environment without needing muscle movement or voice. After more than two decades of research, BCIs have become robust and reliable enough to consider them for mainstream applications, such as hands-free and voice-free control of devices. However, the most common BCIs require visual attention, which restricts their utility for people with visual impairments, or for mobile environments (such as driving) where diverting visual attention is dangerous or not possible. Intriguing alternatives to visual displays are auditory (sounds) or haptic (touch or sensations such as vibrations) based BCI interfaces, or multimodal BCI interfaces which are a combination of these. In addition to providing an alternate interface for people with visual impairments or for special purposes, nonvisual BCI control of devices could also be useful in everyday life, such as when responding to a text message in a movie theater without looking at a device screen or speaking. This project will extend the state of the art in BCIs by evolving the body of knowledge in auditory, haptic, and multimodal stimuli, which are relatively unexplored areas of the field. Additionally, the research will create and explore small, wearable in-ear electrodes to detect brain signals, and thus will contribute to the emerging field of mobile BCIs which will open possibilities for large numbers of mainstream users. Current BCI systems cover a wide and varying range of brain signals and recording technologies; this research focuses on "evoked-response" electroencephalograph (EEG) approaches, that is to say brain signals that are triggered due to a stimulus such as a sound, flashing light, or touch.To these ends, the project will study auditory and haptic cues to determine the best ways to map them to input choices. One of the biggest challenges with auditory and haptic interfaces is how to label a stimulus so it is meaningful to the BCI user. To address this problem, the project will explore novel methods of encoding the labeling and mapping of auditory and haptic stimuli into the stimuli themselves (in a manner analogous to how it is possible to label visual stimuli, such as when the target is a flashing letter so the user can determine the meaning of a cue by looking at it). The plan is to leverage research in sonification (which largely focuses on presenting data in auditory "displays") to devise techniques for encoding speech or patterned tones (such as Morse code) into audio cues in such a way that a user can simply listen to the cue to determine its meaning. The team will also experiment with a variety of multimodal approaches (combining visual, auditory, and haptic cues in a single system) in order to achieve higher accuracy than is possible with a single stimulus mode alone. Finally, the project will evaluate the effectiveness of in-ear EEG electrodes in detecting brain responses to auditory and haptic stimuli, experimenting with electrode design, placement within the ear, and various filters and classifiers to improve the signal-to-noise ratio. The results of the experiments in alternative stimuli will be combined with the optimized wearable electrode system, to create the first hands-free, voice-free, vision-free interfaces for mainstream users.

脑机接口（BCI）是允许人们仅用大脑信号控制计算机和其他设备的系统。 BCI有可能改善许多严重身体残疾者的生活，使他们能够在不需要肌肉运动或声音的情况下交流和控制环境。 经过二十多年的研究，BCI已经变得足够强大和可靠，可以考虑将其用于主流应用，例如免提和语音控制设备。 然而，最常见的BCI需要视觉注意力，这限制了它们对于有视觉障碍的人或对于转移视觉注意力是危险的或不可能的移动的环境（例如驾驶）的效用。 视觉显示的有趣的替代方案是基于听觉（声音）或触觉（触摸或感觉，如振动）的BCI接口，或多模态BCI接口，它们是这些接口的组合。 除了为有视觉障碍的人或特殊用途提供替代界面外，设备的非视觉BCI控制在日常生活中也很有用，例如在电影院里不看设备屏幕或说话就回复短信。 该项目将通过发展听觉，触觉和多模态刺激方面的知识体系来扩展BCIs的最新技术水平，这些知识是该领域相对未开发的领域。 此外，该研究将创建和探索小型可穿戴耳内电极来检测大脑信号，从而为新兴的移动的BCI领域做出贡献，这将为大量主流用户提供可能性。目前的脑机接口系统涵盖了各种各样的大脑信号和记录技术;本研究的重点是“诱发反应”脑电图（EEG）方法，即由于声音、闪光或触摸等刺激而触发的大脑信号。为此，该项目将研究听觉和触觉线索，以确定将它们映射到输入选择的最佳方法。 听觉和触觉接口的最大挑战之一是如何标记刺激，使其对BCI用户有意义。 为了解决这个问题，该项目将探索将听觉和触觉刺激标记和映射到刺激本身的编码新方法（类似于标记视觉刺激的方式，例如当目标是一个闪烁的字母时，用户可以通过查看它来确定提示的含义）。 该计划是利用对发音的研究（主要集中在听觉“显示”中呈现数据）来设计将语音或模式音调（如莫尔斯码）编码成音频提示的技术，以便用户可以简单地听提示来确定其含义。 该团队还将尝试各种多模式方法（在单个系统中结合视觉，听觉和触觉提示），以实现比单独使用单一刺激模式更高的准确性。 最后，该项目将评估入耳式EEG电极在检测大脑对听觉和触觉刺激的反应方面的有效性，试验电极设计，耳内放置以及各种滤波器和分类器以提高信噪比。 替代刺激实验的结果将与优化的可穿戴电极系统相结合，为主流用户创造第一个免提，语音，视觉的界面。

项目成果

Design and Development of a Haptic BCI

触觉 BCI 的设计和开发

• 发表时间: 2021
• 期刊: Abstract Book of the 8th International Brain-Computer Interface Meeting
• 作者: Ganesan, Karthikeyan;Mahalingam, Karthiga;Jackson, Melody Moore
• 通讯作者: Jackson, Melody Moore