NRI: FND: Collaborative Research: DeepSoRo: High-dimensional Proprioceptive and Tactile Sensing and Modeling for Soft Grippers

这项国家机器人计划2.0奖支持对软抓取器的快速、高维和可扩展传感和建模方法的基础研究。这项研究将创造出柔软的抓手，其在复杂环境中处理物体的能力将得到显著提高。软爪由柔性和柔软的材料构成，被动地适应外力，使它们在与人类合作和处理水果和蔬菜等脆弱物体时具有本质上的安全性。软质材料在外力作用下容易变形，这使它们成为自我传感的有希望的候选材料。该项目利用了这一前景，使用嵌入式摄像头和复杂的算法将复杂的图像转换为定量配置和接触力信息。自传感使软爪不局限于预设的被动响应，而是可以根据自己的状态主动修改其操作。该项目产生的主动软爪将在食品工业、农业、老年人或残疾人辅助生活等领域得到应用，提高生产力，改善人类生活质量。该项目采用融合的研究方法，涉及机器人和人工智能，最终通过正式和非正式的学习活动来扩大工程中代表性不足群体的参与。该奖项支持DeepSoRo作为集成本体感觉和触觉传感的新框架的开发，该框架使用嵌入式摄像头提供高维感官输入，并提供抓手全身运动学和动力学的先进深度学习模型。该框架将克服现有软抓取器在建模和感知自身状态方面的主要限制，包括过度简化的低分辨率表示、低速度、难以扩展和适应各种抓取器设计。为了释放软抓取器的全部潜力，必须突破几个科学界限，确保这些抓取器更全面的态势感知，在复杂的环境中执行灵巧和安全的操作。本研究将填补软机器人传感、传感器设计和深度学习领域的关键知识空白，实现软抓取手的在线形状估计和反馈控制，特别是当抓取手与外界物体接触时。这个跨学科的研究项目将沿着三个方向展开：潜伏空间的高维形状建模，关节本体感觉和触觉传感，传感器设计和硬件原型集成。理论进步将与实验研究一起进行，以证明DeepSoRo在现实环境中准确有效地建模和感知软抓取器的潜力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This National Robotics Initiative 2.0 award supports fundamental research on fast, high-dimensional, and scalable sensing and modeling methods for soft grippers. The research will create soft grippers with significantly improved ability to handle objects in complicated environments. Soft grippers are constructed from flexible and soft materials that passively adapt to external forces, making them intrinsically safe for collaborating with humans and for handling delicate objects such as fruits and vegetables. Soft materials deform easily in response to applied forces, making them promising candidates for self-sensing. This project harnesses that promise, using embedded cameras and sophisticated algorithms to translate complex images into quantitative configuration and contact force information. Self-sensing enables soft grippers that are not limited to a preset passive response but can actively modify their operation according to their status. The active soft grippers arising from this project will find application in fields such as food industries, agriculture, assisted living for senior citizens or people with disabilities, increasing productivity and improving the quality of human life. The project follows a convergent research approach involving robotics and artificial intelligence, culminating in formal and informal learning activities to broaden the participation of underrepresented groups in engineering. This award supports the development of DeepSoRo as a new framework of integrated proprioceptive and tactile sensing using embedded cameras to provide high-dimensional sensory input, and advanced deep learning models of the gripper’s full-body kinematics and dynamics. This framework will overcome the key limitations of existing soft grippers in modeling and sensing of their own states, including the over-simplified low-resolution representation, low-speed, and difficulty in scalability and adaptability to various gripper designs. To unleash the full potential of soft grippers, several scientific boundaries must be pushed, ensuring more holistic situational awareness of those grippers to perform dexterous and safe manipulations in complex environments. This research will fill critical knowledge gaps in soft robot sensing, sensor design, and deep learning, to realize the online shape estimation and feedback control of soft grippers, especially when the grippers are in contact with external objects. This interdisciplinary research program will unfold along three directions: high dimensional shape modeling in a latent space, joint proprioceptive and tactile sensing, and sensor design and integration in hardware prototypes. Theoretical advancements will proceed alongside with experimental research toward demonstrating the potential of DeepSoRo to accurately and efficiently model and sense soft grippers in real-world settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.