Fusion of Human Gaze and Machine Vision for Predicting Intended Locomotion Mode
Fusion of Human Gaze and Machine Vision for Predicting Intended Locomotion Mode
复制标题
人类凝视和机器视觉的融合用于预测预期的运动模式
DOI：
10.1109/tnsre.2022.3168796
复制
发表时间：
2022
影响因子：
4.9
通讯作者：
Huang, He
中科院分区：
文献类型：
--
作者：
Li, Minhan;Zhong, Boxuan;Lobaton, Edgar;Huang, He
预测用户的预期运动模式是可穿戴机器人控制的关键，以帮助用户在不断变化的地形上行走时实现无缝过渡。虽然机器视觉最近已被证明是一个很有前途的工具，在确定即将到来的地形在旅行路径，现有的方法是有限的环境感知，而不是人类的意图识别，这是必不可少的协调可穿戴机器人操作。因此，在这项研究中，我们的目标是开发一种新的系统，融合人类的目光（代表用户的意图）和机器视觉（捕捉环境信息），以准确预测用户的运动模式。该系统具有多模态视觉信息，能够识别用户在多地形复杂场景中的运动意图。此外，基于动态时间规整算法，融合策略的发展，以调整从各个模态的时间预测，同时产生灵活的决定的运动模式转换的时间可穿戴机器人控制。使用从五名参与者收集的实验数据对系统性能进行了验证，显示出高准确性（平均超过96%）的意图识别和可靠的运动过渡决策，可调提前时间。这一结果表明了融合人类视线和机器视觉的下肢可穿戴机器人运动意图识别的潜力。
Predicting the user’s intended locomotion mode is critical for wearable robot control to assist the user’s seamless transitions when walking on changing terrains. Although machine vision has recently proven to be a promising tool in identifying upcoming terrains in the travel path, existing approaches are limited to environment perception rather than human intent recognition that is essential for coordinated wearable robot operation. Hence, in this study, we aim to develop a novel system that fuses the human gaze (representing user intent) and machine vision (capturing environmental information) for accurate prediction of the user’s locomotion mode. The system possesses multimodal visual information and recognizes user’s locomotion intent in a complex scene, where multiple terrains are present. Additionally, based on the dynamic time warping algorithm, a fusion strategy was developed to align temporal predictions from individual modalities while producing flexible decisions on the timing of locomotion mode transition for wearable robot control. System performance was validated using experimental data collected from five participants, showing high accuracy (over 96% in average) of intent recognition and reliable decision-making on locomotion transition with adjustable lead time. The promising results demonstrate the potential of fusing human gaze and machine vision for locomotion intent recognition of lower limb wearable robots.