HCC: Collaborative Research: Continuous Control Brain-Computer Interfaces for Creative Expression

严重的运动障碍，包括闭锁综合症和瘫痪，影响着全世界数百万人的生活质量。 PI 之前在基于功能性近红外 (fNIR) 成像的脑机接口方面的工作显示出恢复此类残疾人的沟通和环境控制的巨大希望。 目前，这些系统的典型控制接口是简单的离散选择范例，已被证明是有效的，但信息吞吐率有限。 基于连续控制范式的创新控制接口可动态地将大脑信号水平映射到控制信号，但尚未针对 fNIR 成像进行充分研究。 根据大脑信号可以有效映射到连续控制的程度，将此功能添加到现有的离散控制中可以显着增加基于 fNIR 的直接大脑接口用户可以执行的任务范围（例如位置选择或 2D 绘图）。 在这项工作中，PI 将探索用于连续控制的创新直接脑机接口，并利用它们开发用于创意表达的应用程序。 对于患有严重运动障碍的人来说，创造性表达可以提供情感宣泄以及心理锻炼，从而提高生活质量。视觉艺术创作中固有的任务，例如绘图、着色和纹理，无法通过离散控制来完成。 因此，视觉艺术为研究基于 fNIR 的连续控制接口提供了理想的实验平台。 它还提供了一个引人入胜且激励人心的培训平台，可提高用户控制直接大脑接口的能力。 为此，PI 将研究用于连续和离散选择的非传统控制接口，例如轮子、刻度盘和仪表，以确定 fNIR 信号可以在多大程度上映射到连续控制。 PI 将探索选择和控制画笔、颜色、纹理和形状等艺术媒体的连续方法，并研究连续的大脑信号在多大程度上可以转化为视觉艺术手势（绘画、阴影、着色）。 患有 ALS 的专业国际知名艺术家的建议将指导用户对控制界面的要求。 将收集定量和定性的用户表现数据，并将用于将视觉艺术范式的学习效果与传统的离散选择练习进行比较，以确定是否可以改进训练时间和表现。 项目成果将丰富辅助技术和人机界面的知识体系。 更广泛的影响：将皮质氧合信号转换为用户界面的连续控制信号的方法将通过本质上“平滑”噪声输入信号而成为辅助技术的主流应用。 这种开发成果可适用于运动协调能力较差的人，包括老年人、幼儿和患有帕金森病等运动疾病的人。 主流用户可能会受益于免提界面，而神经控制可以为创作过程提供更多维度。

Severe motor disabilities, including locked-in syndrome and paralysis, impact the quality of life for millions of people worldwide. The PIs' prior work in brain-computer interfaces based on functional near-infrared (fNIR) imaging has shown great promise for restoring communication and environmental control to people with such disabilities. Currently, typical control interfaces for these systems are simple discrete selection paradigms, which have proven to be effective but limited in information throughput rates. Innovative control interfaces based on continuous control paradigms, which dynamically map brain signal levels to control signals, have not been adequately studied for fNIR imaging. Depending upon the extent to which brain signals can be effectively mapped to continuous control, adding this feature to existing discrete control could significantly increase the range of tasks that can be performed by users of an fNIR-based direct brain interface (e.g., positional selection or 2-D drawing). In this work, the PIs will explore innovative direct brain-computer interfaces for continuous control and use them to develop applications for creative expression. For people with severe motor disabilities, creative expression can provide an emotional outlet as well as mental exercise to improve quality of life. The tasks inherent in creating visual art, such as drawing, coloring, and texturing, cannot be accomplished with discrete controls. Therefore, visual art provides an ideal experimental platform to study fNIR-based continuous control interfaces. It also provides an engaging and motivating platform for training that will improve users' abilities to control a direct brain interface. To these ends, the PIs will study non-traditional control interfaces for continuous and discrete selection such as wheels, dials, and gauges, to determine to what extent fNIR signals can be mapped to continuous control. The PIs will explore continuous methods for selection and control of art media such as brushes, colors, textures, and shapes, and investigate to what extent continuous brain signals can be translated into visual art gestures (drawing, shading, coloring). The advice of a professional, internationally-known artist who has ALS will guide the user requirements of the control interfaces. Quantitative and qualitative user performance data will be collected, and will among other things be used to compare learning effects with a visual art paradigm against traditional, discrete selection exercises to determine if training time and performance can be improved. Project outcomes will add to the body of knowledge for assistive technology and human-computer interfaces.Broader Impacts: Methods for translating cortical oxygenation signals into continuous control signals for user interfaces will have mainstream applications for assistive technologies by essentially "smoothing" noisy input signals. Such developments could be applied for use by those with reduced motor coordination, including the elderly, young children, and those with motor diseases such as Parkinson's disease. Mainstream users may benefit from a hands-free interface, and neural control could provide added dimensions to the creative process.