HCC: Collaborative Research: Continuous Control Brain-Computer Interfaces for Creative Expression

包括闭锁综合征和瘫痪在内的严重运动残疾影响着全世界数百万人的生活质量。PI先前在基于功能性近红外（fNIR）成像的脑机接口方面的工作显示出对恢复此类残疾人的沟通和环境控制的巨大希望。 目前，这些系统的典型控制接口是简单的离散选择范例，这已被证明是有效的，但在信息吞吐率的限制。 基于连续控制范例的创新控制接口，其动态地将脑信号水平映射到控制信号，尚未针对fNIR成像进行充分研究。 根据大脑信号可以有效地映射到连续控制的程度，将该特征添加到现有的离散控制可以显著增加基于fNIR的直接大脑接口的用户可以执行的任务范围（例如，位置选择或2-D绘图）。 在这项工作中，PI将探索创新的直接脑机接口，用于连续控制，并使用它们开发创造性表达的应用程序。 对于有严重运动障碍的人来说，创造性的表达可以提供一个情感的出口以及精神锻炼，以提高生活质量。创建视觉艺术所固有的任务（如绘制、着色和纹理）无法通过离散控件来完成。 因此，视觉艺术为研究基于fNIR的连续控制界面提供了一个理想的实验平台。 它还提供了一个吸引人和激励人的培训平台，这将提高用户控制直接大脑接口的能力。 为此，PI将研究用于连续和离散选择的非传统控制接口，例如轮子，表盘和仪表，以确定fNIR信号可以在多大程度上映射到连续控制。 PI将探索选择和控制艺术媒体（如画笔，颜色，纹理和形状）的连续方法，并研究连续大脑信号在多大程度上可以转化为视觉艺术手势（绘画，阴影，着色）。 一位患有ALS的专业国际知名艺术家的建议将指导用户对控制界面的要求。 将收集定量和定性的用户表现数据，并将用于比较视觉艺术范式与传统的离散选择练习的学习效果，以确定是否可以改善培训时间和性能。 项目成果将增加辅助技术和人机接口的知识体系。更广泛的影响：将皮层氧合信号转化为用户界面连续控制信号的方法将通过基本上“平滑”嘈杂的输入信号而成为辅助技术的主流应用。 这种发展可以应用于那些运动协调能力下降的人，包括老年人，幼儿和那些患有运动疾病如帕金森病的人。 主流用户可以从免提界面中受益，神经控制可以为创意过程提供更多维度。

Severe motor disabilities, including locked-in syndrome and paralysis, impact the quality of life for millions of people worldwide. The PIs' prior work in brain-computer interfaces based on functional near-infrared (fNIR) imaging has shown great promise for restoring communication and environmental control to people with such disabilities. Currently, typical control interfaces for these systems are simple discrete selection paradigms, which have proven to be effective but limited in information throughput rates. Innovative control interfaces based on continuous control paradigms, which dynamically map brain signal levels to control signals, have not been adequately studied for fNIR imaging. Depending upon the extent to which brain signals can be effectively mapped to continuous control, adding this feature to existing discrete control could significantly increase the range of tasks that can be performed by users of an fNIR-based direct brain interface (e.g., positional selection or 2-D drawing). In this work, the PIs will explore innovative direct brain-computer interfaces for continuous control and use them to develop applications for creative expression. For people with severe motor disabilities, creative expression can provide an emotional outlet as well as mental exercise to improve quality of life. The tasks inherent in creating visual art, such as drawing, coloring, and texturing, cannot be accomplished with discrete controls. Therefore, visual art provides an ideal experimental platform to study fNIR-based continuous control interfaces. It also provides an engaging and motivating platform for training that will improve users' abilities to control a direct brain interface. To these ends, the PIs will study non-traditional control interfaces for continuous and discrete selection such as wheels, dials, and gauges, to determine to what extent fNIR signals can be mapped to continuous control. The PIs will explore continuous methods for selection and control of art media such as brushes, colors, textures, and shapes, and investigate to what extent continuous brain signals can be translated into visual art gestures (drawing, shading, coloring). The advice of a professional, internationally-known artist who has ALS will guide the user requirements of the control interfaces. Quantitative and qualitative user performance data will be collected, and will among other things be used to compare learning effects with a visual art paradigm against traditional, discrete selection exercises to determine if training time and performance can be improved. Project outcomes will add to the body of knowledge for assistive technology and human-computer interfaces.Broader Impacts: Methods for translating cortical oxygenation signals into continuous control signals for user interfaces will have mainstream applications for assistive technologies by essentially "smoothing" noisy input signals. Such developments could be applied for use by those with reduced motor coordination, including the elderly, young children, and those with motor diseases such as Parkinson's disease. Mainstream users may benefit from a hands-free interface, and neural control could provide added dimensions to the creative process.