HCC: Human-Computer Interaction and Brain Measurement Using fNIR Spectroscopy

在脑机接口（BCI）的一般领域有很多先前的研究，其主要目的是帮助有严重运动障碍的人通过将他们的大脑活动转化为特定的设备控制信号来与他们的环境进行互动。完全瘫痪的用户（“锁定”）或缺乏肌肉控制的用户（例如脑瘫患者或中风患者）可以使用脑机接口回答简单的问题，控制他们的环境，以及进行文字处理。但是，迄今为止为脑机接口尝试的各种脑成像技术都存在严重的缺陷，因此，通过这些系统进行通信目前既耗时又费力。开放的研究挑战涉及脑机接口的准确性（系统经常误解用户的意图），以及此类系统的信息传输速率（在现实世界中使用通常太慢）。在该项目中，pi将利用他们在设计、实施和评估非命令、自适应用户界面（例如，基于眼动）方面的经验，通过采用功能近红外（fNIR）光谱和机器学习相结合的新兴技术来测量大脑活动，以分析人机交互中的用户数据，从而推进脑机接口。虽然fNIR技术仍处于起步阶段，但pi希望将其用作自适应界面的实时输入，从而开辟新天地。pi认为，这种结合也将导致新的，更客观的方法来评估下一代HCI的交互风格。pi的方法是一种自然的延伸，并将利用他们之前与基于现实的交互概念相关的工作，它关注的是新的交互风格利用用户来自现实世界的预先存在的技能和期望的方式，而不是训练有素的计算机技能，同时有助于区分专注于接口相关或语法方面的精神努力与专注于底层任务或语义方面的精神努力。更广泛的影响：该项目将开发对运动障碍用户立即有益的技术。此外，这项工作将开辟一个新的HCI研究领域（即，与能够实时动态适应用户认知负荷的界面有关），并将在总体上推进交互风格的理论和评估。

There has been much previous research in the general area of brain-computer interfaces (BCI), which has primarily aimed to help people with severe motor disabilities interact with their environment by translating their brain activity into specific device control signals. Users who are completely paralyzed ("locked-in") or those who lack muscle control (e.g., people with cerebral palsy or stroke victims) can use BCIs to answer simple questions, to control their environment, and for word processing. But the variety of brain imaging techniques that have been tried to date for BCIs have all suffered from serious drawbacks, so that communicating through such systems is currently time consuming and mentally demanding. Open research challenges concern the accuracy of BCIs (systems often misinterpret a user's intentions), and the information transfer rate of such systems (which is often too slow for use in real world settings). In this project, the PIs will build on their experience in designing, implementing, and evaluating non-command, adaptive user interfaces (e.g., based on eye movement), to advance BCI by bringing to bear emerging technology for measuring brain activity using functional near infrared (fNIR) spectroscopy coupled with machine learning to analyze user data in human-computer interaction. While fNIR technology is still in its infancy, the PIs expect its use as a real-time input to an adaptive interface to break new ground. The PIs believes that this combination will also lead to new, more objective methods for evaluating next generation interaction styles for HCI in general. The PIs' approach is a natural extension of and will exploit their prior work relating to the concept of reality-based interaction, which focuses on the ways that new interaction styles exploit the user's pre-existing skills and expectations from the real world more than trained computer skills, while at the same time helping to differentiate mental effort devoted to interface-related or syntactic aspects from that devoted to the underlying task or semantic aspects.Broader Impacts: This project will develop technologies of immediate benefit to users with motor disabilities. Additionally, the work will open up a new area of HCI research (namely, relating to interfaces that can dynamically adapt in real time to the user's cognitive load), and will advance the theory and evaluation of interaction styles in general.