CAREER: Goal-Guided Self-Reflective Control Interface in Teleoperation

职业：远程操作中的目标引导自反射控制界面

• 批准号: 1652454
• 负责人: Xiaoli Zhang
• 金额: $ 52.07万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-06-01 至 2023-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1652454&HistoricalAwards=false
• 关键词: CAREER; Goal; Guided; Self; Reflective

The focus of this project is on human-robot interactions (HRI) during teleoperation, for both the fundamental research and the educational activities. The research derives from the observation that object grasping and manipulation using conventional teleoperation approaches places a significant control burden on the human operator and reduces task performance. This is because indirect manipulation of an object through a robot hand may cause inaccurate or undesired robot motion, while the limited control inputs available to the operator (e.g., joystick, data glove) make direct kinematic mapping challenging for complex object manipulation tasks. So the human operator must mentally and physically transform (e.g., rotate, translate, scale, deform) the desired robot actions to required inputs at the interface; these transformations significantly increase control difficulty. The primary research goal of this project is to develop a novel goal-guided self-reflective control interface (GSRCI), which will enable the robot to understand the operator's high-level objective during an object-grasping operation and to conform to task constraints in order to reduce control difficulties and ensure the success of subsequent manipulation. The primary educational activity derives from the common deficiency of distance learning programs supported by existing teleconferencing technologies, namely that they offer limited or no opportunities for hands-on learning. To address this problem, an interactive distance learning system (IDLS) will be developed that immerses remote students in the classroom environment through student tele-controlling of a robot's arms and hands for object manipulation and/or interaction with other classmates, thereby enabling remote users to feel present in the classroom and engaged in class activities. The task modeling technology as well as the goal-guided control interface technology from the research work will be coupled to develop an easy and intuitive teleoperation-based distance learning system for K-12 students. The GSRCI represents transformative technology with the potential to provide a new paradigm of HRI that will significantly improve the power and quality of teleoperation and broadly impact applications related to diverse domains including assistance for the elderly and disabled, minimally invasive surgery, space and underwater explorations, military reconnaissance, nuclear servicing, and urban search and rescue. The IDLS will foster engagement for remote students and allow them to successfully participate in STEM activities, offering disadvantaged groups the potential to learn regardless of their ability to physically attend a class setting.To achieve these goals, a cognitive interface will be created that enables a robot to flexibly reproduce actions that accommodate the operator's motion inputs as well as autonomously regulate these actions in a self-reflective manner to compensate task constraints that facilitate subsequent manipulations. A novel goal-achievement indicator will predict the level of goal accomplishment for the planned action. Additionally, a goal-guided remedial planner will regulate this action to accomplish the goal by relaxing the constraint bound of the operator's motion inputs using an adaptive local search strategy. New models for human and robot task-based grasp behaviors will also be developed, which will provide a knowledge base to infer human goals in a task and to conduct goal-guided robot-grasp planning. To link quantified task constraints with symbolic tasks, account for uncertainty in task modeling, and allow task reasoning from partially observed data, a directed probabilistic Bayesian model will encode the statistical dependence among the task goal, object attributes, actions, and task constraints.

该项目的重点是遥操作过程中的人-机器人交互(HRI)，用于基础研究和教育活动。这项研究源于这样的观察，即使用传统的遥操作方法抓取和操纵对象会给人类操作员带来巨大的控制负担，并降低任务绩效。这是因为通过机械手间接操纵对象可能导致不准确或不期望的机器人运动，而操作员可用的有限的控制输入(例如，操纵杆、数据手套)使得直接运动学映射对于复杂的对象操纵任务具有挑战性。因此，人类操作员必须在精神上和物理上将所需的机器人动作转换(例如，旋转、平移、缩放、变形)为界面上所需的输入；这些转换显著增加了控制难度。本项目的主要研究目标是开发一种新颖的目标导引的自反射控制接口(GSRCI)，使机器人能够在抓取对象的操作过程中理解操作员的高层目标，并遵守任务约束，以降低控制难度，确保后续操作的成功。初级教育活动源于现有电话会议技术支持的远程学习方案的普遍不足，即它们提供的动手学习机会有限或没有机会。为了解决这一问题，将开发一种交互式远程学习系统(IDLS)，通过学生遥控机器人的手臂和手来操纵物体和/或与其他同学互动，使远程学生沉浸在课堂环境中，从而使远程用户能够感觉到身处课堂并参与课堂活动。将研究工作中的任务建模技术以及目标引导控制接口技术结合起来，为K-12学生开发一个简单直观的基于远程操作的远程学习系统。GSRCI代表着变革性的技术，有可能提供一种新的HRI范式，显著提高远程操作的能力和质量，并广泛影响与不同领域相关的应用，包括援助老年人和残疾人、微创手术、空间和水下探索、军事侦察、核服务以及城市搜索和救援。IDL将培养远程学生的参与度，并允许他们成功地参与STEM活动，为弱势群体提供学习的潜力，无论他们是否有能力实际参加课堂设置。为了实现这些目标，将创建一个认知界面，使机器人能够灵活地复制适应操作员的运动输入的动作，并以自我反思的方式自主调节这些动作，以补偿促进后续操作的任务限制。一种新的目标实现指标将预测计划行动的目标实现水平。此外，目标引导的治疗规划器将通过使用自适应局部搜索策略放松操作员的运动输入的约束范围来调节该动作以实现目标。还将开发基于人类和机器人基于任务的抓取行为的新模型，这将提供一个知识库，以推断人类在任务中的目标，并进行目标引导的机器人抓取规划。为了将量化的任务约束与符号任务联系起来，考虑任务建模中的不确定性，并允许根据部分观测数据进行任务推理，有向概率贝叶斯模型将对任务目标、对象属性、动作和任务约束之间的统计相关性进行编码。

项目成果

WE-Filter: Adaptive Acceptance Criteria for Filter-based Shared Autonomy

WE-Filter：基于过滤器的共享自治的自适应验收标准

• DOI: 10.1109/icra48891.2023.10161228
• 发表时间: 2023
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Bowman, Michael;Zhang, Xiaoli
• 通讯作者: Zhang, Xiaoli

Self-Reflective Learning Strategy for Persistent Autonomy of Aerial Manipulators

空中机械手持续自主的自我反思学习策略

• DOI: 10.1115/dscc2019-9086
• 发表时间: 2019
• 期刊: ASME 2019 Dynamic Systems and Control Conference
• 作者: Zhou, Xu;Zhang, Jiucai;Zhang, Xiaoli
• 通讯作者: Zhang, Xiaoli

Learn and Transfer Knowledge of Preferred Assistance Strategies in Semi-Autonomous Telemanipulation

• DOI: 10.1007/s10846-022-01596-2
• 发表时间: 2020-03
• 期刊: Journal of Intelligent & Robotic Systems
• 影响因子: 3.3
• 作者: Lingfeng Tao;Michael Bowman;Xu Zhou;Jiucai Zhang;Xiaoli Zhang

Intent-Uncertainty-Aware Grasp Planning for Robust Robot Assistance in Telemanipulation

远程操作中鲁棒机器人辅助的意图不确定性感知抓取规划

• DOI: 10.1109/icra.2019.8793819
• 发表时间: 2019
• 期刊: 2019 International Conference on Robotics and Automation
• 作者: Bowman, Michael;Li, Songpo;Zhang, Xiaoli
• 通讯作者: Zhang, Xiaoli

Robust Motion Mapping Between Human and Humanoids Using CycleAutoencoder

• DOI: 10.1109/robio54168.2021.9739345
• 发表时间: 2021-12
• 期刊: 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO)
• 作者: Matthew Stanley;Lingfeng Tao;Xiaoli Zhang