CAREER: Goal-Guided Self-Reflective Control Interface in Teleoperation

本课题的研究重点是远程操作过程中人机交互的基础研究和教学活动。该研究源于观察到，使用传统的远程操作方法抓取和操纵物体给操作员带来了巨大的控制负担，降低了任务性能。这是因为通过机器人手间接操作物体可能会导致不准确或不希望的机器人运动，而操作者可用的有限控制输入（例如，操纵杆，数据手套）使得直接运动学映射对复杂的物体操作任务具有挑战性。因此，人类操作员必须在精神上和物理上转换（例如，旋转，翻译，缩放，变形）所需的机器人动作到界面上所需的输入；这些转换显著增加了控制难度。本项目的主要研究目标是开发一种新的目标引导自反射控制接口（GSRCI），使机器人在抓取物体操作过程中能够理解操作者的高层目标，并符合任务约束，从而降低控制难度，保证后续操作的成功。主要的教育活动源于现有远程会议技术支持的远程学习计划的共同缺陷，即它们提供的实践学习机会有限或根本没有。为了解决这一问题，将开发一种交互式远程学习系统（IDLS），通过学生远程控制机器人的手臂和手来操纵物体和/或与其他同学互动，使远程学生沉浸在课堂环境中，从而使远程用户感觉身在课堂并参与课堂活动。研究工作中的任务建模技术和目标导向控制接口技术将结合起来，为K-12学生开发一个简单直观的基于远程操作的远程学习系统。GSRCI代表了变革性技术，具有提供远程操作新范式的潜力，将显著提高远程操作的能力和质量，并广泛影响与不同领域相关的应用，包括老年人和残疾人援助、微创手术、空间和水下探索、军事侦察、核服务和城市搜索和救援。IDLS将促进远程学生的参与，使他们能够成功参与STEM活动，为弱势群体提供学习的潜力，而不管他们是否有能力参加课堂设置。为了实现这些目标，将创建一个认知界面，使机器人能够灵活地再现适应操作员运动输入的动作，并以自我反射的方式自主调节这些动作，以补偿促进后续操作的任务约束。一种新的目标实现指标将预测计划行动的目标完成程度。此外，目标导向的补救计划器将通过使用自适应局部搜索策略放松操作员运动输入的约束界限来调节该动作以实现目标。人类和机器人基于任务的抓取行为的新模型也将被开发，这将为在任务中推断人类目标和进行目标引导的机器人抓取规划提供知识库。为了将量化任务约束与符号任务联系起来，考虑任务建模中的不确定性，并允许从部分观察数据进行任务推理，有向概率贝叶斯模型将对任务目标、对象属性、动作和任务约束之间的统计依赖性进行编码。

The focus of this project is on human-robot interactions (HRI) during teleoperation, for both the fundamental research and the educational activities. The research derives from the observation that object grasping and manipulation using conventional teleoperation approaches places a significant control burden on the human operator and reduces task performance. This is because indirect manipulation of an object through a robot hand may cause inaccurate or undesired robot motion, while the limited control inputs available to the operator (e.g., joystick, data glove) make direct kinematic mapping challenging for complex object manipulation tasks. So the human operator must mentally and physically transform (e.g., rotate, translate, scale, deform) the desired robot actions to required inputs at the interface; these transformations significantly increase control difficulty. The primary research goal of this project is to develop a novel goal-guided self-reflective control interface (GSRCI), which will enable the robot to understand the operator's high-level objective during an object-grasping operation and to conform to task constraints in order to reduce control difficulties and ensure the success of subsequent manipulation. The primary educational activity derives from the common deficiency of distance learning programs supported by existing teleconferencing technologies, namely that they offer limited or no opportunities for hands-on learning. To address this problem, an interactive distance learning system (IDLS) will be developed that immerses remote students in the classroom environment through student tele-controlling of a robot's arms and hands for object manipulation and/or interaction with other classmates, thereby enabling remote users to feel present in the classroom and engaged in class activities. The task modeling technology as well as the goal-guided control interface technology from the research work will be coupled to develop an easy and intuitive teleoperation-based distance learning system for K-12 students. The GSRCI represents transformative technology with the potential to provide a new paradigm of HRI that will significantly improve the power and quality of teleoperation and broadly impact applications related to diverse domains including assistance for the elderly and disabled, minimally invasive surgery, space and underwater explorations, military reconnaissance, nuclear servicing, and urban search and rescue. The IDLS will foster engagement for remote students and allow them to successfully participate in STEM activities, offering disadvantaged groups the potential to learn regardless of their ability to physically attend a class setting.To achieve these goals, a cognitive interface will be created that enables a robot to flexibly reproduce actions that accommodate the operator's motion inputs as well as autonomously regulate these actions in a self-reflective manner to compensate task constraints that facilitate subsequent manipulations. A novel goal-achievement indicator will predict the level of goal accomplishment for the planned action. Additionally, a goal-guided remedial planner will regulate this action to accomplish the goal by relaxing the constraint bound of the operator's motion inputs using an adaptive local search strategy. New models for human and robot task-based grasp behaviors will also be developed, which will provide a knowledge base to infer human goals in a task and to conduct goal-guided robot-grasp planning. To link quantified task constraints with symbolic tasks, account for uncertainty in task modeling, and allow task reasoning from partially observed data, a directed probabilistic Bayesian model will encode the statistical dependence among the task goal, object attributes, actions, and task constraints.