NRI: FND: Collaborative Research: DeepSoRo: High-dimensional Proprioceptive and Tactile Sensing and Modeling for Soft Grippers

NRI：FND：合作研究：DeepSoRo：软抓手的高维本体感受和触觉感知与建模

• 批准号: 2348839
• 负责人: Wenzhen Yuan
• 金额: $ 35万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2348839&HistoricalAwards=false
• 关键词: NRI; FND; Collaborative; Research; DeepSoRo

This National Robotics Initiative 2.0 award supports fundamental research on fast, high-dimensional, and scalable sensing and modeling methods for soft grippers. The research will create soft grippers with significantly improved ability to handle objects in complicated environments. Soft grippers are constructed from flexible and soft materials that passively adapt to external forces, making them intrinsically safe for collaborating with humans and for handling delicate objects such as fruits and vegetables. Soft materials deform easily in response to applied forces, making them promising candidates for self-sensing. This project harnesses that promise, using embedded cameras and sophisticated algorithms to translate complex images into quantitative configuration and contact force information. Self-sensing enables soft grippers that are not limited to a preset passive response but can actively modify their operation according to their status. The active soft grippers arising from this project will find application in fields such as food industries, agriculture, assisted living for senior citizens or people with disabilities, increasing productivity and improving the quality of human life. The project follows a convergent research approach involving robotics and artificial intelligence, culminating in formal and informal learning activities to broaden the participation of underrepresented groups in engineering. This award supports the development of DeepSoRo as a new framework of integrated proprioceptive and tactile sensing using embedded cameras to provide high-dimensional sensory input, and advanced deep learning models of the gripper’s full-body kinematics and dynamics. This framework will overcome the key limitations of existing soft grippers in modeling and sensing of their own states, including the over-simplified low-resolution representation, low-speed, and difficulty in scalability and adaptability to various gripper designs. To unleash the full potential of soft grippers, several scientific boundaries must be pushed, ensuring more holistic situational awareness of those grippers to perform dexterous and safe manipulations in complex environments. This research will fill critical knowledge gaps in soft robot sensing, sensor design, and deep learning, to realize the online shape estimation and feedback control of soft grippers, especially when the grippers are in contact with external objects. This interdisciplinary research program will unfold along three directions: high dimensional shape modeling in a latent space, joint proprioceptive and tactile sensing, and sensor design and integration in hardware prototypes. Theoretical advancements will proceed alongside with experimental research toward demonstrating the potential of DeepSoRo to accurately and efficiently model and sense soft grippers in real-world settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这个国家机器人倡议2.0奖支持基础研究的快速，高维，可扩展的传感和建模方法的软抓手。这项研究将创造出在复杂环境中处理物体的能力显著提高的软夹持器。软夹持器由柔性和柔软的材料制成，可被动适应外力，使其在与人类合作和处理水果和蔬菜等微妙物体时具有本质的安全性。软材料在受到外力时很容易变形，这使它们成为自感知的理想候选者。该项目利用这一承诺，使用嵌入式摄像头和复杂的算法将复杂的图像转化为定量配置和接触力信息。自感知使软夹持器不限于预设的被动响应，而是可以根据其状态主动修改其操作。该项目产生的主动式软夹将应用于食品工业、农业、老年人或残疾人的辅助生活等领域，提高生产力，改善人类生活质量。该项目遵循融合的研究方法，涉及机器人技术和人工智能，最终在正式和非正式的学习活动，以扩大在工程中代表性不足的群体的参与。该奖项支持DeepSoRo的开发，将其作为一个集成本体感受和触觉传感的新框架，使用嵌入式摄像头提供高维感觉输入，以及抓取器全身运动学和动力学的高级深度学习模型。该框架将克服现有的软夹持器在建模和感测其自身状态方面的关键限制，包括过度简化的低分辨率表示、低速度以及难以扩展和适应各种夹持器设计。为了充分发挥软抓手的潜力，必须突破几个科学界限，确保对这些抓手进行更全面的态势感知，以便在复杂环境中进行灵巧、安全的操作。该研究将填补软机器人传感、传感器设计和深度学习方面的关键知识空白，实现软抓取器的在线形状估计和反馈控制，特别是当抓取器与外部物体接触时。这项跨学科的研究计划将沿着沿着三个方向展开：潜在空间中的高维形状建模，关节本体感受和触觉传感，以及硬件原型中的传感器设计和集成。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Toward Zero-Shot Sim-to-Real Transfer Learning for Pneumatic Soft Robot 3D Proprioceptive Sensing

面向气动软机器人 3D 本体感知的零样本模拟到真实迁移学习

• DOI: 10.1109/icra48891.2023.10160384
• 发表时间: 2023
• 期刊: IEEE
• 作者: Yoo, Uksang;Zhao, Hanwen;Altamirano, Alvaro;Yuan, Wenzhen;Feng, Chen
• 通讯作者: Feng, Chen