A Framework for Semi-Autonomous In-Hand Telemanipulation

半自主手持遥控框架

• 批准号: 2114464
• 负责人: Xiaoli Zhang
• 金额: $ 47.34万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2114464&HistoricalAwards=false
• 关键词: Framework; Semi; Autonomous; Hand; Telemanipulation

This award supports research to develop and test a control framework for stable in-hand telemanipulation, using a physic-informed machine learning approach that can adapt the telerobotic controller to physiological limitations of individual users. In daily life, in-hand manipulation is needed to complete seemingly simple tasks such as adjusting the orientation of a power adaptor for plugging in to an outlet, as well as to complete complex skilled tasks such as manipulating a scalpel during a surgical procedure. The long-term objective of the line of research initiated with this project is to enable dexterous in-hand manipulation in tele-operation scenarios in which the robot can understand how to track real-time changes in human finger motions and use that information to actively ensure the stability of an in-hand object while it is being manipulated. This project promotes the progress of science and advances the national health, prosperity, and welfare by developing and testing novel methods implementing stable shared human and machine control of hand-held objects in tele-operation scenarios, such as tele-surgery, healthcare assistive robotics, and remote search and rescue. The project will also support outreach activities through an existing K-12 program at the Colorado School of Mines, at a local community college, and at STEM camp for girls.This project seeks to solve challenges associated with in-hand telemanipulation through the development and testing of a control framework that utilizes physic-informed hierarchical machine learning approaches to adapt the telerobot to the physiological limitations of the individual users. There are three research objectives: 1) to use physics-informed metrics to guide the robot control policy to generate stable grasp configurations; 2) to optimize interpretation of signals derived from the human hand motion tracking system using a hierarchical learning model; and 3) to personalize shared control of the manipulation task between the human and the robot through active machine learning guided by the user's corrective adjustments in real-time. The project team will use a commercially available robotic hand system with the semi-autonomous framework to conduct human-subject-involved evaluation. Subjects will perform tele-manipulation tasks of increasing complexity, ranging from the relative simplicity of a jar opening task to a more complex case requiring in-hand changes of tool position and orientation. The work promises to advance the science and engineering of human-robot cooperation through a novel semi-autonomous control framework that can support complex in-hand object manipulation for teleoperation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项支持开发和测试稳定的手部远程操纵控制框架的研究，该框架使用一种物理信息机器学习方法，可以使远程机器人控制器适应个人用户的生理限制。在日常生活中，需要手部操纵来完成看似简单的任务，如调整电源适配器的方向以插入插座，以及完成复杂的熟练任务，如在手术过程中操纵手术刀。该项目启动的研究领域的长期目标是在遥操作场景中实现灵巧的手部操作，在远程操作场景中，机器人可以了解如何跟踪人类手指运动的实时变化，并使用这些信息来积极确保手部对象在被操作时的稳定性。该项目通过开发和测试在远程操作场景中对手持对象实施稳定的共享人机控制的新方法，促进了科学的进步，促进了国家的健康、繁荣和福利，例如远程手术、医疗辅助机器人和远程搜救。该项目还将通过科罗拉多矿业学院、当地社区大学和STEM女孩夏令营现有的K-12项目来支持外联活动。该项目寻求通过开发和测试控制框架来解决与手部远程操纵相关的挑战，该框架利用物理知识分层机器学习方法使远程机器人适应个人用户的生理限制。研究的目标有三个：1)使用物理信息度量来指导机器人控制策略以生成稳定的抓取配置；2)使用分层学习模型来优化对来自人类手部运动跟踪系统的信号的解释；3)通过在用户实时校正调整的指导下通过主动机器学习来个性化对人和机器人之间的操作任务的共享控制。项目组将使用具有半自主框架的商用机械手系统进行涉及人类主体的评价。受试者将执行越来越复杂的遥操作任务，从相对简单的罐子打开任务到需要手动改变工具位置和方向的更复杂的情况。这项工作承诺通过一种新颖的半自主控制框架来推进人-机器人合作的科学和工程技术，该框架可以支持远程操作的复杂手持对象操作。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Robust Motion Mapping Between Human and Humanoids Using CycleAutoencoder

• DOI: 10.1109/robio54168.2021.9739345
• 发表时间: 2021-12
• 期刊: 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO)
• 作者: Matthew Stanley;Lingfeng Tao;Xiaoli Zhang

WE-Filter: Adaptive Acceptance Criteria for Filter-based Shared Autonomy

WE-Filter：基于过滤器的共享自治的自适应验收标准

• DOI: 10.1109/icra48891.2023.10161228
• 发表时间: 2023
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Bowman, Michael;Zhang, Xiaoli
• 通讯作者: Zhang, Xiaoli

Learn and Transfer Knowledge of Preferred Assistance Strategies in Semi-Autonomous Telemanipulation

• DOI: 10.1007/s10846-022-01596-2
• 发表时间: 2020-03
• 期刊: Journal of Intelligent & Robotic Systems
• 影响因子: 3.3
• 作者: Lingfeng Tao;Michael Bowman;Xu Zhou;Jiucai Zhang;Xiaoli Zhang

Learn Task First or Learn Human Partner First: A Hierarchical Task Decomposition Method for Human-Robot Cooperation

• DOI: 10.1109/smc52423.2021.9659041
• 发表时间: 2020-03
• 期刊: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
• 作者: Lingfeng Tao;Michael Bowman;Jiucai Zhang;Xiaoli Zhang

A Multi-Agent Approach for Adaptive Finger Cooperation in Learning-based In-Hand Manipulation

• DOI: 10.1109/icra48891.2023.10160909
• 发表时间: 2022-10
• 期刊: 2023 IEEE International Conference on Robotics and Automation (ICRA)
• 作者: Lingfeng Tao;Jiucai Zhang;Michael Bowman;Xiaoli Zhang