CAREER: Context-Aware Task-Oriented Dexterous Robotic Manipulation

随着现代工业的发展，有效地操纵各种尺寸、形状或材料的陌生物体成为制造业、服务业和零售业自动化的瓶颈问题。这个教师早期职业发展（CAREER）项目将建立创新技术，使机器人能够操纵不熟悉的物体完成困难的任务，例如，货物包装、钻孔、废物分类和小部件组装。这些技术的优点是能够根据任务要求和感知的对象属性（包括形状和材料）来操纵对象。该项目具有很大的潜力，通过提高生产力，机器人编程，简化机器人规划和降低技术边界，使众多行业受益。该项目将支持国家战略，将制造业带回美国，同时使许多行业受益并提高其经济竞争力。该项目将建立一个整体框架的上下文感知面向任务的操作（CATOM），以解决三个基石的灵巧操作的挑战：对象启示感知，灵巧建模，操作规划和学习。对象示能表示是指与对象的物理属性（如形状、质量和摩擦力）相匹配的动作。对象示能表示是从由多模态传感器测量的对象特性获得的。为此，将设计一个知识驱动的模型，融合异质传感和人类经验。对象可供性将由每个抓取分类下的潜在联系人表示。基于拓扑的建模方法将用于将潜在的手部姿势与接触对齐，以进行适当的抓握。基于拓扑的建模，一个复杂的任务将被表示为一个时空序列的手的拓扑结构和操作。将实施混合学习和规划机制，以部署手部拓扑并在上下文约束下执行动作。对这些感知、规划和学习方法的研究将推进用于复杂任务的灵巧机器人操作的知识。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

With the advancement of modern industries, effective manipulation of unfamiliar objects in an assortment of dimensions, shapes, or materials becomes the bottleneck problem in the automation of manufacturing, services, and retail trade. This Faculty Early Career Development (CAREER) project will build innovative technologies to enable a robot to manipulate unfamiliar objects for difficult tasks, e.g., goods packing, drilling, waste sorting, and small part assembly. The advantage of the technologies is the capability of manipulating an object in accordance with the task requirements and the perceived object properties including shapes and materials. This project has great potential to benefit numerous industries by improving productivity, deskilling robot programming, simplifying robot planning, and lowering the technical boundaries. This project will support the national strategy in bringing manufacturing back to the US while benefiting many industries and increasing their economic competitiveness. The project will establish a holistic framework of Context-Aware Task-Oriented Manipulation (CATOM) to address three cornerstone challenges in dexterous manipulation: object affordance perception, dexterity modeling, and manipulation planning and learning. An object affordance refers to actions that match with the physical properties of an object, such as shapes, mass, and friction. The object affordances are obtained from object characteristics measured by multimodal sensors. For this purpose, a knowledge-driven model will be designed, which fuses heterogenous sensing and incorporates human experience. The object affordances will be represented by potential contacts under each grasp taxonomy. A topology-based modeling method will be used to align potential hand postures with the contacts for a proper grasp. With the topology-based modeling, a complex task will be represented as a spatial-temporal sequence of hand topologies and operations. A hybrid learning and planning mechanism will be implemented to deploy hand topologies and perform actions under contextual constraints. The research of these perception, planning, and learning methodologies will advance the knowledge in dexterous robotic manipulation for complex tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.