Offline Evaluation Matters: Investigation of the Influence of Offline Performance of EMG-Based Neural-Machine Interfaces on User Adaptation, Cognitive Load, and Physical Efforts in a Real-Time Application
Offline Evaluation Matters: Investigation of the Influence of Offline Performance of EMG-Based Neural-Machine Interfaces on User Adaptation, Cognitive Load, and Physical Efforts in a Real-Time Application
复制标题
DOI：
10.1109/tnsre.2023.3297448
复制
发表时间：
2023-07
影响因子：
4.9
通讯作者：
Robert M. Hinson;Joseph Berman;I-Chieh Lee;William G. Filer;H. Huang
中科院分区：
文献类型：
--
作者：
Robert M. Hinson;Joseph Berman;I-Chieh Lee;William G. Filer;H. Huang
基于EMG的神经-机器接口（NMIs）的离线评估对于其实时应用的价值一直存在争议。通常，在研究了离线EMG解码准确度/误差与NMI用户的实时任务性能的相关性之后得出结论，而没有进一步考虑其他重要的人类性能指标，例如适应率、认知负荷和体力。为了填补这一空白，本研究旨在调查基于EMG的NMI的离线解码准确性与实时NMI使用中的用户适应、认知负荷和体力之间的关系。12名非残疾受试者参加了本研究。对于每个受试者，我们建立了三个EMG解码器，在预测连续的手部和手腕运动时产生不同的离线准确度（低、中和高）。然后，受试者使用每个EMG解码器在真实的时间内执行虚拟手部姿势匹配任务，并作为评估试验。结果表明，高水平的离线性能解码器产生最快的适应率和最高的姿态匹配完成率，在最少的肌肉努力，在用户在线测试。一个次要的任务增加了认知负荷，降低了实时虚拟任务的竞争率，所有的解码器，然而，解码器具有高离线准确性仍然产生了最高的任务完成率。这些结果意味着基于EMG的NMI的离线性能为用户利用它们的能力提供了重要的见解，并且应该在新型NMI算法的研究和开发中发挥重要作用。
There has been controversy about the value of offline evaluation of EMG-based neural-machine interfaces (NMIs) for their real-time application. Often, conclusions have been drawn after studying the correlation of the offline EMG decoding accuracy/error with the NMI user’s real-time task performance without further considering other important human performance metrics such as adaptation rate, cognitive load, and physical effort. To fill this gap, this study aimed to investigate the relationship between the offline decoding accuracy of EMG-based NMIs and user adaptation, cognitive load, and physical effort in real-time NMI use. Twelve non-disabled subjects participated in this study. For each subject, we established three EMG decoders that yielded different offline accuracy (low, moderate, and high) in predicting continuous hand and wrist motions. The subject then used each EMG decoder to perform a virtual hand posture matching task in real time with and without a secondary task as the evaluation trials. Results showed that the high-level offline performance decoders yield the fastest adaptation rate and highest posture matching completion rate with the least muscle effort in users during online testing. A secondary task increased the cognitive load and reduced real-time virtual task competition rate for all the decoders; however, the decoder with high offline accuracy still produced the highest task completion rate. These results imply that the offline performance of EMG-based NMIs provide important insight to users’ abilities to utilize them and should play an important role in research and development of novel NMI algorithms.