HCC: Collaborative Research: Continuous Control Brain-Computer Interfaces for Creative Expression

严重的运动残疾，包括闭锁综合征和瘫痪，影响着全世界数百万人的生活质量。pi先前在基于功能性近红外（fNIR）成像的脑机接口方面的工作显示了为这些残疾人恢复通信和环境控制的巨大希望。目前，这些系统的典型控制接口是简单的离散选择范例，已被证明是有效的，但在信息吞吐率方面受到限制。基于连续控制范式的创新控制接口，将大脑信号水平动态映射到控制信号，尚未充分研究用于近红外成像。根据大脑信号可以有效映射到连续控制的程度，将这一功能添加到现有的离散控制中可以显著增加基于fnir的直接脑接口用户可以执行的任务范围（例如，位置选择或二维绘图）。在这项工作中，pi将探索创新的直接脑机接口，用于持续控制，并利用它们开发创造性表达的应用程序。对于有严重运动障碍的人来说，创造性的表达可以提供一种情感出口和精神锻炼，以提高生活质量。创造视觉艺术所固有的任务，如绘画、着色和纹理，不能通过离散的控制来完成。因此，视觉艺术为研究基于fnir的连续控制接口提供了理想的实验平台。它还提供了一个有吸引力和激励的培训平台，将提高用户控制直接大脑接口的能力。为此，pi将研究用于连续和离散选择的非传统控制接口，如车轮、表盘和仪表，以确定在多大程度上fNIR信号可以映射到连续控制。pi将探索画笔、颜色、纹理、形状等艺术媒介的连续选择和控制方法，并研究连续的大脑信号在多大程度上可以转化为视觉艺术手势（绘画、阴影、着色）。一位国际知名的ALS专业艺术家的建议将指导用户对控制界面的要求。将收集定量和定性的用户表现数据，并将除其他事项外，用于将视觉艺术范例的学习效果与传统的离散选择练习进行比较，以确定是否可以改进训练时间和表现。项目成果将增加辅助技术和人机界面的知识体系。更广泛的影响：将皮质氧合信号转换为用户界面的连续控制信号的方法将通过本质上“平滑”噪声输入信号而成为辅助技术的主流应用。这一进展可以应用于那些运动协调能力下降的人，包括老年人、幼儿和患有帕金森病等运动疾病的人。主流用户可能会从免提界面中受益，神经控制可以为创作过程提供额外的维度。

Severe motor disabilities, including locked-in syndrome and paralysis, impact the quality of life for millions of people worldwide. The PIs' prior work in brain-computer interfaces based on functional near-infrared (fNIR) imaging has shown great promise for restoring communication and environmental control to people with such disabilities. Currently, typical control interfaces for these systems are simple discrete selection paradigms, which have proven to be effective but limited in information throughput rates. Innovative control interfaces based on continuous control paradigms, which dynamically map brain signal levels to control signals, have not been adequately studied for fNIR imaging. Depending upon the extent to which brain signals can be effectively mapped to continuous control, adding this feature to existing discrete control could significantly increase the range of tasks that can be performed by users of an fNIR-based direct brain interface (e.g., positional selection or 2-D drawing). In this work, the PIs will explore innovative direct brain-computer interfaces for continuous control and use them to develop applications for creative expression. For people with severe motor disabilities, creative expression can provide an emotional outlet as well as mental exercise to improve quality of life. The tasks inherent in creating visual art, such as drawing, coloring, and texturing, cannot be accomplished with discrete controls. Therefore, visual art provides an ideal experimental platform to study fNIR-based continuous control interfaces. It also provides an engaging and motivating platform for training that will improve users' abilities to control a direct brain interface. To these ends, the PIs will study non-traditional control interfaces for continuous and discrete selection such as wheels, dials, and gauges, to determine to what extent fNIR signals can be mapped to continuous control. The PIs will explore continuous methods for selection and control of art media such as brushes, colors, textures, and shapes, and investigate to what extent continuous brain signals can be translated into visual art gestures (drawing, shading, coloring). The advice of a professional, internationally-known artist who has ALS will guide the user requirements of the control interfaces. Quantitative and qualitative user performance data will be collected, and will among other things be used to compare learning effects with a visual art paradigm against traditional, discrete selection exercises to determine if training time and performance can be improved. Project outcomes will add to the body of knowledge for assistive technology and human-computer interfaces.Broader Impacts: Methods for translating cortical oxygenation signals into continuous control signals for user interfaces will have mainstream applications for assistive technologies by essentially "smoothing" noisy input signals. Such developments could be applied for use by those with reduced motor coordination, including the elderly, young children, and those with motor diseases such as Parkinson's disease. Mainstream users may benefit from a hands-free interface, and neural control could provide added dimensions to the creative process.