Evaluating the Human-Robot Interaction (HRI) during tele-operation

本博士的目的是研究人机交互（HRI），并评估操作员在HRI任务和遥操作期间的心理工作量和努力（见图1遥操作设置）。除了使用调查来获取这些数据的传统方法之外，该项目将重点关注使用功能性近红外光谱（fNIRS）脑机接口来使用参与者的大脑和生理数据。fNIRS是一种新兴的神经成像技术，它提供了一种无创、便携和低成本的监测大脑活动的方法（见图2）。fNIRS基于使用近红外光谱来测量大脑活动，并已被用于评估各种任务，包括远程操作车辆[5]，心算[6]，n-back任务[5,7]，以及其他复杂的认知任务，如视频游戏[4,5,8]。在许多应用中，远程操作机器人执行任务仍然是唯一可能的解决方案。在为安全关键、高后果环境领域（例如核或手术）设计机器人时，高度保守的行业尚未充分信任自主机器人方法，这些行业需要人类参与其中。因此，可以合理地假设，通过在自主/智能系统和人类操作员之间安全地共享控制，为操作员创建的交互和界面具有可接受/减少的工作量。至关重要的是，作业者要有足够的备用能力来应对突发事件。我们将利用林肯大学在过去/正在进行的远程操作项目中开发的不同共享控制技术（例如使用触觉引导和/或口头共享机器人操纵器的控制[1-3]）。这项工作的目的是整合新的评估技术，用于评估操作员的心理工作量和努力，基于生理和大脑数据，在HRI和远程操作领域，以补充现有的技术。远程操作被称为从臂（slave-arm， SA）的机械臂，即使是对熟练的人类操作员来说，也会带来很高的认知负荷和精神负荷，因此会导致严重的疲劳和性能的进行性退化[1-3]。目前捕捉操作员心理工作量的技术包括要求他们主观评估和自我报告他们的工作量水平，使用问卷调查，如NASA-TLX和瞬时自我评估（ISA）。在评估更复杂的系统和任务时，主观度量变得非常重要，在这些系统和任务中，基于性能的度量变得非常难以捕捉[9]。尽管它们对这些系统的评估至关重要，但在使用它们时也存在某些不可忽视的限制。首先，主观测量依赖于参与者在整个任务过程中判断和报告状态的能力。这不仅需要操作员付出额外的努力，还需要技能和潜在的培训。其次，主观测量，如果实时使用有可能中断和负面影响性能；如果在任务后使用，它们依赖于操作者回忆过去特定时刻发生的事情的能力。使用近红外光谱（fNIRS）对大脑活动进行直接的生理测量（图2），为捕捉和评估操作人员的心理工作量提供了机会，同时克服了上述主观技术和性能指标的局限性。然而，为了了解在HRI[9]期间应用生理技术时的考虑因素，需要进行新的研究。

The aim of this PhDis to study Human Robot Interactions (HRI's) and assess the operators' mental workload and effort during HRI tasks and teleoperation (See Fig. 1 for the teleoperation setup). Beyond traditional approaches of using surveys to capture this data, this project will focus on using participants' brain and physiological data using the functional Near Infrared Spectroscopy (fNIRS) Brain Computer Interface. fNIRS is an emerging neuroimaging technique that offers a non-invasive, portable and low-cost method of monitoring brain activity (See Fig. 2 ). fNIRS is based on the use of near infrared spectroscopy to measure brain activity, and has been used to evaluate various tasks, including remotely operating vehicles [5], mental arithmetic [6], n-back tasks [5, 7], and other complex cognition tasks such as video games [4, 5, 8].Tele-operating a robot to perform a task remotely is still the only possible solution in many applications. When designing robots for the safety-critical, high-consequence environments domain (e.g. nuclear or surgery), autonomous robotics methods are not yet sufficiently trusted by highly conservative industries, which demand a human in the loop. Therefore, it is reasonable to assume that the interaction and interface created for the operators are of an acceptable/reduced work load by safely sharing the control between autonomous/intelligent system and huma operator. It is critical that the operator has the spare capacity to be able to respond in the case of the unexpected events. We will exploit different shared control techniques developed at the University of Lincoln in the past/ongoing teleoperation projects (e.g. using haptic-guidance and/or Verbally Sharing Control of robotic manipulators[1-3]). The aim of this work is to integrate novel evaluation techniques used to assess operators' mental workload and effort based on physiological and brain data, in the area of HRI and teleoperation, to complement existing techniques. Tele-operating a robotic manipulator, called slave-arm (SA) imposes a high cognitive load and mental workload even on expert human operators and, consequently, results in severe fatigue and progressive degeneration in performance [1-3].Current techniques of capturing operators' mental workload involve asking them to subjectively assess and self-report their levels of workloads using questionnaires such as NASA-TLX and Instantaneous Self-Assessment (ISA). The subjective measures become highly important when it comes to evaluating more complex systems and tasks, where performance-based measures become highly difficult to capture [9]. Even though they are critical for evaluation of these systems, there are certain limitations that cannot be overlooked when using them. Firstly, subjective measures rely on the participants' ability to judge and report the state throughout the task. This requires not only extra effort from the operator [10], but also skill and potential training. Secondly, subjective measures, if used in real-time have the potential to interrupt and negatively affect performance; if used post-task, they rely on the operators' ability to recall what happened during certain moments in the past. Direct physiological measures of brain activity using fNIRS (Fig. 2) offer the opportunity to capture and assess operators' mental workload whilst overcoming the above-mentioned limitations with subjective techniques and performance metrics. However, new research is needed in order to understand the considerations when applying physiological techniques during HRI [9].