NRI: FND: Optoacoustic Material and Structure Pretouch Sensing at Robot Fingertip

NRI：FND：机器人指尖的光声材料和结构预触摸传感

• 批准号: 1925037
• 负责人: Dezhen Song
• 金额: $ 75万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-01-01 至 2024-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1925037&HistoricalAwards=false
• 关键词: NRI; FND; Optoacoustic; Material; Structure

When robots move from factory floors to a wider service market, it is imperative to enable robots to grasp objects with no prior knowledge. Contactless detection of object: material type; shape; and close-to-surface interior structure, can provide vital information such as friction coefficient and applicable grasping force for planning for successful grasps. Unfortunately, no existing sensors can achieve this. Imaging and ranging devices such as cameras or lidars can neither see through surface nor distinguish material type. Tactile sensing requires physical contacts between the robot finger and the object surface which may risk damaging the object or changing the position of the object. Either case may lead to a grasping failure. Microelectromechanical systems and robot perception experts will develop systems and algorithms to create a new type of miniature fingertip-mounted sensor that can detect and map object material type, shape, and close-to-surface interior structure without physical contact. The project will benefit a wide range of robotic applications that require grasping and manipulation such as manufacturing, service robots, search & rescue, etc.Building on the working principle of optoacoustic effect which refers to the formation of acoustic waves following light absorption in a solid material, investigators propose to send modulated laser pulse signals to probe material type and structure based on the acoustic spectrum, time-of-flight, and intensity analyses of the received ultrasound signals. The proposed sensor will be enabled by new and efficient material recognition and surface/interior structure mapping algorithms so that the recommended grasping points and force range will be available before robot fingers are closed. The integrated new research and educational effort is named as the Optoacoustic Material And Structure Sensor (OMASS) project which focuses three main tasks, 1) Development of OMASS devices: an iterative study on design, fabrication, packaging, calibration, testing, and device control, 2) Pretouch perception algorithms to enable the core functions of OMASS devices: material type recognition, surface shape & interior structure mapping, and grasping point planning, and 3) Building a material database with raw signals and signatures for common household items. The OMASS project will share development and educational efforts via journal and conference publications, seminars, research experience for undergraduates and teachers, open-house activities, and the Internet to scientists, students, underrepresented groups, and the public worldwide. The OMASS project will demonstrate the state-of-the-art robotics to the public. The research team will distribute hardware designs, source codes (e.g. ROS stacks), application programming interfaces, experimental data, and documentation via the project website so that other groups can learn from the project team's experience.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当机器人从工厂车间走向更广泛的服务市场时，必须使机器人能够在没有先验知识的情况下抓取物体。物体的非接触式检测：材料类型;形状;和接近表面的内部结构，可以提供重要的信息，如摩擦系数和适用的抓取力，以规划成功的抓取。不幸的是，现有的传感器无法实现这一点。成像和测距设备，如相机或激光雷达，既不能穿透表面，也不能区分材料类型。触觉感测需要机器人手指和物体表面之间的物理接触，这可能有损坏物体或改变物体位置的风险。任何一种情况都可能导致抓取失败。 微机电系统和机器人感知专家将开发系统和算法，以创建一种新型的微型指尖安装传感器，可以在没有物理接触的情况下检测和映射物体材料类型，形状和接近表面的内部结构。该项目将有利于广泛的机器人应用，需要掌握和操作，如制造，服务机器人，搜索救援等。光声效应的工作原理，是指在固体材料中的光吸收后形成声波的基础上，研究人员建议发送调制激光脉冲信号探测材料类型和结构的基础上的声学频谱，飞行时间，和强度分析接收到的超声波信号。所提出的传感器将通过新的和有效的材料识别和表面/内部结构映射算法来实现，以便在机器人手指闭合之前提供推荐的抓取点和力范围。集成的新研究和教育工作被命名为光声材料和结构传感器（OMASS）项目，该项目侧重于三个主要任务，1）OMASS设备的开发：设计，制造，包装，校准，测试和设备控制的迭代研究，2）Pretouch感知算法，以实现OMASS设备的核心功能：材料类型识别，表面形状&内部结构映射，和抓取点规划，以及3）建立具有常见家庭物品的原始信号和签名的材料数据库。OMASS项目将通过期刊和会议出版物、研讨会、本科生和教师的研究经验、开放日活动和互联网，向全世界的科学家、学生、代表性不足的群体和公众分享发展和教育工作。OMASS项目将向公众展示最先进的机器人技术。研究团队将通过项目网站发布硬件设计、源代码（如ROS堆栈）、应用程序编程接口、实验数据和文档，以便其他团队可以学习项目团队的经验。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Toward Robotic Weed Control: Detection of Nutsedge Weed in Bermudagrass Turf Using Inaccurate and Insufficient Training Data

• DOI: 10.1109/lra.2021.3098012
• 发表时间: 2021-06
• 期刊: IEEE Robotics and Automation Letters
• 影响因子: 5.2
• 作者: Shuangyun Xie;Chengsong Hu;M. Bagavathiannan;Dezhen Song

Device Design and System Integration of a Two-Axis Water-immersible Micro Scanning Mirror (WIMSM) to Enable Dual-modal Optical and Acoustic Communication and Ranging for Underwater Vehicles

两轴水浸式微扫描镜 (WIMSM) 的器件设计和系统集成，可实现水下航行器双模光声通信和测距

• DOI: 10.1109/icra48506.2021.9561953
• 发表时间: 2021
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Duan, Xiaoyu;Wang, Di;Song, Dezhen;Zou, Jun
• 通讯作者: Zou, Jun

The Third Generation (G3) Dual-Modal and Dual Sensing Mechanisms (DMDSM) Pretouch Sensor for Robotic Grasping

用于机器人抓取的第三代 (G3) 双模态和双传感机制 (DMDSM) 预触摸传感器

• DOI: 10.1109/icra48891.2023.10161337
• 发表时间: 2023
• 期刊: Proceedings of IEEE International Conference on Robotics and Automation
• 作者: Fang, Cheng;Li, Shuangliang;Wang, Di;Guo, Fengzhi;Song, Dezhen;Zou, Jun
• 通讯作者: Zou, Jun

Vision-Based Camera/Robot Pose Estimation Using Both Semantic and Geometric Features on LEGO Baseplates

• DOI: 10.1109/case56687.2023.10260305
• 发表时间: 2023-08
• 期刊: 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)
• 作者: Shu-Hao Yeh;Shuangyun Xie;Wei Yan;Dezhen Song

An optically-transparent transducer with a high-NA and wide-bandwidth for photoacoustic microscopy (PAM)

一种用于光声显微镜 (PAM) 的具有高数值孔径和宽带的光学透明传感器

• DOI: 10.1117/12.2576834
• 发表时间: 2021
• 期刊: Photons Plus Ultrasound: Imaging and Sensing 2021
• 作者: Fang, Cheng;Zou, Jun
• 通讯作者: Zou, Jun