NRI: FND: Collaborative Research: DeepSoRo: High-dimensional Proprioceptive and Tactile Sensing and Modeling for Soft Grippers

这个国家机器人倡议2.0奖支持基础研究的快速，高维，可扩展的传感和建模方法的软抓手。这项研究将创造出在复杂环境中处理物体的能力显著提高的软夹持器。软夹持器由柔性和柔软的材料制成，可被动适应外力，使其在与人类合作和处理水果和蔬菜等微妙物体时具有本质的安全性。软材料在受到外力时很容易变形，这使它们成为自感知的理想候选者。该项目利用这一承诺，使用嵌入式摄像头和复杂的算法将复杂的图像转化为定量配置和接触力信息。自感知使软夹持器不限于预设的被动响应，而是可以根据其状态主动修改其操作。该项目产生的主动式软夹将应用于食品工业、农业、老年人或残疾人的辅助生活等领域，提高生产力，改善人类生活质量。该项目遵循融合的研究方法，涉及机器人技术和人工智能，最终在正式和非正式的学习活动，以扩大在工程中代表性不足的群体的参与。该奖项支持DeepSoRo的开发，将其作为一个集成本体感受和触觉传感的新框架，使用嵌入式摄像头提供高维感觉输入，以及抓取器全身运动学和动力学的高级深度学习模型。该框架将克服现有软抓手在建模和感知自身状态方面的关键限制，包括过度简化的低分辨率表示、低速以及对各种抓手设计的可扩展性和适应性困难。为了充分发挥软夹持器的潜力，必须突破几个科学界限，确保对这些夹持器有更全面的态势感知，以便在复杂环境中进行灵巧和安全的操作。该研究将填补软机器人传感、传感器设计和深度学习方面的关键知识空白，实现软抓取器的在线形状估计和反馈控制，特别是当抓取器与外部物体接触时。这项跨学科的研究计划将沿着沿着三个方向展开：潜在空间中的高维形状建模，关节本体感受和触觉传感，以及硬件原型中的传感器设计和集成。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

This National Robotics Initiative 2.0 award supports fundamental research on fast, high-dimensional, and scalable sensing and modeling methods for soft grippers. The research will create soft grippers with significantly improved ability to handle objects in complicated environments. Soft grippers are constructed from flexible and soft materials that passively adapt to external forces, making them intrinsically safe for collaborating with humans and for handling delicate objects such as fruits and vegetables. Soft materials deform easily in response to applied forces, making them promising candidates for self-sensing. This project harnesses that promise, using embedded cameras and sophisticated algorithms to translate complex images into quantitative configuration and contact force information. Self-sensing enables soft grippers that are not limited to a preset passive response but can actively modify their operation according to their status. The active soft grippers arising from this project will find application in fields such as food industries, agriculture, assisted living for senior citizens or people with disabilities, increasing productivity and improving the quality of human life. The project follows a convergent research approach involving robotics and artificial intelligence, culminating in formal and informal learning activities to broaden the participation of underrepresented groups in engineering. This award supports the development of DeepSoRo as a new framework of integrated proprioceptive and tactile sensing using embedded cameras to provide high-dimensional sensory input, and advanced deep learning models of the gripper’s full-body kinematics and dynamics. This framework will overcome the key limitations of existing soft grippers in modeling and sensing of their own states, including the over-simplified low-resolution representation, low-speed, and difficulty in scalability and adaptability to various gripper designs. To unleash the full potential of soft grippers, several scientific boundaries must be pushed, ensuring more holistic situational awareness of those grippers to perform dexterous and safe manipulations in complex environments. This research will fill critical knowledge gaps in soft robot sensing, sensor design, and deep learning, to realize the online shape estimation and feedback control of soft grippers, especially when the grippers are in contact with external objects. This interdisciplinary research program will unfold along three directions: high dimensional shape modeling in a latent space, joint proprioceptive and tactile sensing, and sensor design and integration in hardware prototypes. Theoretical advancements will proceed alongside with experimental research toward demonstrating the potential of DeepSoRo to accurately and efficiently model and sense soft grippers in real-world settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.