A Framework for Semi-Autonomous In-Hand Telemanipulation

该奖项支持研究开发和测试稳定的手持遥控控制框架，使用物理信息机器学习方法，可以使遥控控制器适应个体用户的生理限制。在日常生活中，手操作需要完成看似简单的任务，如调整电源适配器的方向以插入插座，以及完成复杂的技术任务，如在手术过程中操作手术刀。该项目启动的研究路线的长期目标是在远程操作场景中实现灵巧的手持操作，机器人可以理解如何跟踪人类手指运动的实时变化，并利用该信息主动确保手持物体在被操纵时的稳定性。本项目通过开发和测试在远程手术、医疗辅助机器人、远程搜索和救援等远程操作场景中实现手持物体稳定的人机共享控制的新方法，促进科学进步，促进国家健康、繁荣和福利。该项目还将通过科罗拉多矿业学院（Colorado School of Mines）现有的K-12课程、当地一所社区大学和女童STEM训练营支持外展活动。该项目旨在通过开发和测试控制框架来解决与手持远程操作相关的挑战，该框架利用物理信息分层机器学习方法使远程机器人适应个体用户的生理限制。主要有三个研究目标：1)利用物理信息指标指导机器人控制策略生成稳定的抓取构型；2)利用层次学习模型对手部运动跟踪系统产生的信号进行优化解释；3)在用户实时纠错调整的引导下，通过主动机器学习实现人与机器人对操作任务的个性化共享控制。项目团队将使用市售的带有半自主框架的机械手系统来进行涉及人类受试者的评估。受试者将执行越来越复杂的远程操作任务，从相对简单的开罐任务到需要在手改变工具位置和方向的更复杂的情况。这项工作有望通过一种新的半自主控制框架来推进人机合作的科学和工程，该框架可以支持远程操作中复杂的手持物体操作。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award supports research to develop and test a control framework for stable in-hand telemanipulation, using a physic-informed machine learning approach that can adapt the telerobotic controller to physiological limitations of individual users. In daily life, in-hand manipulation is needed to complete seemingly simple tasks such as adjusting the orientation of a power adaptor for plugging in to an outlet, as well as to complete complex skilled tasks such as manipulating a scalpel during a surgical procedure. The long-term objective of the line of research initiated with this project is to enable dexterous in-hand manipulation in tele-operation scenarios in which the robot can understand how to track real-time changes in human finger motions and use that information to actively ensure the stability of an in-hand object while it is being manipulated. This project promotes the progress of science and advances the national health, prosperity, and welfare by developing and testing novel methods implementing stable shared human and machine control of hand-held objects in tele-operation scenarios, such as tele-surgery, healthcare assistive robotics, and remote search and rescue. The project will also support outreach activities through an existing K-12 program at the Colorado School of Mines, at a local community college, and at STEM camp for girls.This project seeks to solve challenges associated with in-hand telemanipulation through the development and testing of a control framework that utilizes physic-informed hierarchical machine learning approaches to adapt the telerobot to the physiological limitations of the individual users. There are three research objectives: 1) to use physics-informed metrics to guide the robot control policy to generate stable grasp configurations; 2) to optimize interpretation of signals derived from the human hand motion tracking system using a hierarchical learning model; and 3) to personalize shared control of the manipulation task between the human and the robot through active machine learning guided by the user's corrective adjustments in real-time. The project team will use a commercially available robotic hand system with the semi-autonomous framework to conduct human-subject-involved evaluation. Subjects will perform tele-manipulation tasks of increasing complexity, ranging from the relative simplicity of a jar opening task to a more complex case requiring in-hand changes of tool position and orientation. The work promises to advance the science and engineering of human-robot cooperation through a novel semi-autonomous control framework that can support complex in-hand object manipulation for teleoperation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.