NRI: FND: Collaborative Research: DeepSoRo: High-dimensional Proprioceptive and Tactile Sensing and Modeling for Soft Grippers

这一国家机器人计划2.0奖支持软抓手的快速、高维和可扩展的传感和建模方法的基础研究。这项研究将创造出在复杂环境中处理物体的能力显著提高的柔软抓手。柔软的抓手是由柔软的材料制成的，这些材料被动地适应外力，使它们在与人类合作以及处理水果和蔬菜等精致物体时本质上是安全的。软材料很容易在外力的作用下变形，这使得它们很有希望成为自我检测的候选材料。该项目利用了这一承诺，使用嵌入式摄像头和复杂的算法将复杂的图像转换为定量的配置和接触力信息。自感知使柔软的抓取器不仅限于预设的被动响应，而且可以根据其状态主动修改其操作。由该项目产生的主动软抓手将在食品工业、农业、老年人或残疾人的辅助生活、提高生产力和改善人类生活质量等领域得到应用。该项目采用了一种涉及机器人学和人工智能的融合研究方法，最终以正式和非正式的学习活动来扩大未被充分代表的群体对工程学的参与。该奖项支持DeepSoRo作为集成本体感觉和触觉感知的新框架的开发，该框架使用嵌入式摄像头来提供高维感觉输入，并支持对握手的全身运动学和动力学的高级深度学习模型。该框架将克服现有软抓取器在建模和感知自身状态方面的关键局限性，包括表示过于简化、分辨率低、速度慢以及难以扩展和适应各种抓取器设计。为了充分释放软抓手的潜力，必须突破几个科学界限，确保这些抓手更全面地了解情况，在复杂的环境中执行灵活和安全的操作。本研究将填补软机器人传感、传感器设计、深度学习等方面的关键知识空白，实现软抓手的在线形状估计和反馈控制，特别是在手爪与外部物体接触的情况下。这一跨学科研究计划将沿着三个方向展开：潜在空间中的高维形状建模、关节本体感知和触觉感知，以及传感器设计和硬件原型集成。理论进步将与实验研究一起进行，以展示DeepSoRo在真实世界环境中准确和高效地模拟和感知软抓手的潜力。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This National Robotics Initiative 2.0 award supports fundamental research on fast, high-dimensional, and scalable sensing and modeling methods for soft grippers. The research will create soft grippers with significantly improved ability to handle objects in complicated environments. Soft grippers are constructed from flexible and soft materials that passively adapt to external forces, making them intrinsically safe for collaborating with humans and for handling delicate objects such as fruits and vegetables. Soft materials deform easily in response to applied forces, making them promising candidates for self-sensing. This project harnesses that promise, using embedded cameras and sophisticated algorithms to translate complex images into quantitative configuration and contact force information. Self-sensing enables soft grippers that are not limited to a preset passive response but can actively modify their operation according to their status. The active soft grippers arising from this project will find application in fields such as food industries, agriculture, assisted living for senior citizens or people with disabilities, increasing productivity and improving the quality of human life. The project follows a convergent research approach involving robotics and artificial intelligence, culminating in formal and informal learning activities to broaden the participation of underrepresented groups in engineering. This award supports the development of DeepSoRo as a new framework of integrated proprioceptive and tactile sensing using embedded cameras to provide high-dimensional sensory input, and advanced deep learning models of the gripper’s full-body kinematics and dynamics. This framework will overcome the key limitations of existing soft grippers in modeling and sensing of their own states, including the over-simplified low-resolution representation, low-speed, and difficulty in scalability and adaptability to various gripper designs. To unleash the full potential of soft grippers, several scientific boundaries must be pushed, ensuring more holistic situational awareness of those grippers to perform dexterous and safe manipulations in complex environments. This research will fill critical knowledge gaps in soft robot sensing, sensor design, and deep learning, to realize the online shape estimation and feedback control of soft grippers, especially when the grippers are in contact with external objects. This interdisciplinary research program will unfold along three directions: high dimensional shape modeling in a latent space, joint proprioceptive and tactile sensing, and sensor design and integration in hardware prototypes. Theoretical advancements will proceed alongside with experimental research toward demonstrating the potential of DeepSoRo to accurately and efficiently model and sense soft grippers in real-world settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.