CAREER: Context-Aware Task-Oriented Dexterous Robotic Manipulation

随着现代工业的发展，对各种尺寸、形状或材料的陌生物体的有效操作成为制造业、服务业和零售业自动化的瓶颈问题。这个学院早期职业发展（Career）项目将建立创新技术，使机器人能够操纵不熟悉的物体来完成困难的任务，例如货物包装、钻孔、废物分类和小部件组装。这些技术的优势在于能够根据任务要求和感知到的物体属性（包括形状和材料）操纵物体。该项目具有巨大的潜力，通过提高生产率、减少机器人编程技能、简化机器人规划和降低技术界限，使许多行业受益。该项目将支持将制造业带回美国的国家战略，同时使许多行业受益并提高其经济竞争力。该项目将建立一个上下文感知任务导向操作（CATOM）的整体框架，以解决灵巧操作中的三个基石挑战：对象可视性感知、灵巧建模以及操作规划和学习。对象赋能指的是与对象的物理属性（如形状、质量和摩擦）相匹配的动作。通过多模态传感器测量的物体特征，获得物体的可视性。为此，将设计一个知识驱动的模型，该模型融合了异质感知并结合了人类经验。对象的可视性将由每个把握分类法下的潜在接触表示。基于拓扑的建模方法将用于对齐潜在的手部姿势与接触，以获得正确的抓取。在基于拓扑的建模中，一个复杂的任务将被表示为一个手部拓扑和操作的时空序列。将实现混合学习和规划机制来部署手部拓扑并在上下文约束下执行操作。这些感知、规划和学习方法的研究将促进复杂任务中灵巧机器人操作的知识。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

With the advancement of modern industries, effective manipulation of unfamiliar objects in an assortment of dimensions, shapes, or materials becomes the bottleneck problem in the automation of manufacturing, services, and retail trade. This Faculty Early Career Development (CAREER) project will build innovative technologies to enable a robot to manipulate unfamiliar objects for difficult tasks, e.g., goods packing, drilling, waste sorting, and small part assembly. The advantage of the technologies is the capability of manipulating an object in accordance with the task requirements and the perceived object properties including shapes and materials. This project has great potential to benefit numerous industries by improving productivity, deskilling robot programming, simplifying robot planning, and lowering the technical boundaries. This project will support the national strategy in bringing manufacturing back to the US while benefiting many industries and increasing their economic competitiveness. The project will establish a holistic framework of Context-Aware Task-Oriented Manipulation (CATOM) to address three cornerstone challenges in dexterous manipulation: object affordance perception, dexterity modeling, and manipulation planning and learning. An object affordance refers to actions that match with the physical properties of an object, such as shapes, mass, and friction. The object affordances are obtained from object characteristics measured by multimodal sensors. For this purpose, a knowledge-driven model will be designed, which fuses heterogenous sensing and incorporates human experience. The object affordances will be represented by potential contacts under each grasp taxonomy. A topology-based modeling method will be used to align potential hand postures with the contacts for a proper grasp. With the topology-based modeling, a complex task will be represented as a spatial-temporal sequence of hand topologies and operations. A hybrid learning and planning mechanism will be implemented to deploy hand topologies and perform actions under contextual constraints. The research of these perception, planning, and learning methodologies will advance the knowledge in dexterous robotic manipulation for complex tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.