Evaluating the Human-Robot Interaction (HRI) during tele-operation

该博士学位的目的是研究人类机器人交互作用(HRI‘s)，并评估操作员在HRI任务和遥操作(遥操作设置见图1)过程中的精神工作量和努力程度。除了使用调查来获取这些数据的传统方法之外，该项目还将重点使用功能性近红外光谱(FNIRS)脑计算机接口来使用参与者的大脑和生理数据。FNIRS是一种新兴的神经成像技术，它提供了一种非侵入性、便携和低成本的监测大脑活动的方法(见图2)。FNIRS基于使用近红外光谱来测量大脑活动，并已被用于评估各种任务，包括远程操作车辆[5]、心算[6]、n-back任务[5，7]以及其他复杂的认知任务，如视频游戏[4，5，8]。在许多应用中，远程操作机器人来执行任务仍然是唯一可能的解决方案。在为安全关键、后果严重的环境领域(例如核或外科手术)设计机器人时，自主机器人方法还没有得到高度保守的行业的充分信任，因为这些行业需要有人参与。因此，通过在自主/智能系统和HUMA操作员之间安全地共享控制，可以合理地假设为操作员创建的交互和界面具有可接受/减少的工作负荷。至关重要的是，操作员拥有备用容量，以便能够在发生意外事件的情况下做出反应。我们将利用林肯大学在过去/正在进行的远程操作项目中开发的不同共享控制技术(例如，使用机器人的触觉引导和/或口头共享控制[1-3])。这项工作的目的是整合新的评估技术，用于在HRI和遥操作领域基于生理和大脑数据评估操作员的脑力劳动和努力，以补充现有技术。遥操作被称为从臂(SA)的机械手，即使对专业的人类操作员也施加了高的认知负荷和脑力劳动，从而导致严重的疲劳和渐进性的性能退化[1-3]。目前捕捉操作员精神负荷的技术包括要求他们使用诸如NASA-TLX和瞬时自我评估(ISA)的问卷来主观地评估和自我报告他们的工作负荷水平。当涉及到评估更复杂的系统和任务时，主观测量变得非常重要，其中基于性能的测量变得非常难以获取[9]。尽管它们对评估这些系统至关重要，但在使用它们时也有一些不能忽视的限制。首先，主观测量依赖于参与者在整个任务过程中判断和报告状态的能力。这不仅需要操作员付出额外的努力，还需要技能和潜在的培训。其次，主观测量，如果在实时使用，可能会中断并对绩效产生负面影响；如果在任务后使用，它们依赖于操作员回忆过去特定时刻发生的事情的能力。使用fNIRS(图2)对大脑活动的直接生理测量提供了捕获和评估操作员的精神工作负荷的机会，同时克服了主观技术和性能指标的上述限制。然而，需要新的研究来理解在HRI期间应用生理学技术时的考虑因素[9]。

The aim of this PhDis to study Human Robot Interactions (HRI's) and assess the operators' mental workload and effort during HRI tasks and teleoperation (See Fig. 1 for the teleoperation setup). Beyond traditional approaches of using surveys to capture this data, this project will focus on using participants' brain and physiological data using the functional Near Infrared Spectroscopy (fNIRS) Brain Computer Interface. fNIRS is an emerging neuroimaging technique that offers a non-invasive, portable and low-cost method of monitoring brain activity (See Fig. 2 ). fNIRS is based on the use of near infrared spectroscopy to measure brain activity, and has been used to evaluate various tasks, including remotely operating vehicles [5], mental arithmetic [6], n-back tasks [5, 7], and other complex cognition tasks such as video games [4, 5, 8].Tele-operating a robot to perform a task remotely is still the only possible solution in many applications. When designing robots for the safety-critical, high-consequence environments domain (e.g. nuclear or surgery), autonomous robotics methods are not yet sufficiently trusted by highly conservative industries, which demand a human in the loop. Therefore, it is reasonable to assume that the interaction and interface created for the operators are of an acceptable/reduced work load by safely sharing the control between autonomous/intelligent system and huma operator. It is critical that the operator has the spare capacity to be able to respond in the case of the unexpected events. We will exploit different shared control techniques developed at the University of Lincoln in the past/ongoing teleoperation projects (e.g. using haptic-guidance and/or Verbally Sharing Control of robotic manipulators[1-3]). The aim of this work is to integrate novel evaluation techniques used to assess operators' mental workload and effort based on physiological and brain data, in the area of HRI and teleoperation, to complement existing techniques. Tele-operating a robotic manipulator, called slave-arm (SA) imposes a high cognitive load and mental workload even on expert human operators and, consequently, results in severe fatigue and progressive degeneration in performance [1-3].Current techniques of capturing operators' mental workload involve asking them to subjectively assess and self-report their levels of workloads using questionnaires such as NASA-TLX and Instantaneous Self-Assessment (ISA). The subjective measures become highly important when it comes to evaluating more complex systems and tasks, where performance-based measures become highly difficult to capture [9]. Even though they are critical for evaluation of these systems, there are certain limitations that cannot be overlooked when using them. Firstly, subjective measures rely on the participants' ability to judge and report the state throughout the task. This requires not only extra effort from the operator [10], but also skill and potential training. Secondly, subjective measures, if used in real-time have the potential to interrupt and negatively affect performance; if used post-task, they rely on the operators' ability to recall what happened during certain moments in the past. Direct physiological measures of brain activity using fNIRS (Fig. 2) offer the opportunity to capture and assess operators' mental workload whilst overcoming the above-mentioned limitations with subjective techniques and performance metrics. However, new research is needed in order to understand the considerations when applying physiological techniques during HRI [9].