NRI: INT: FIngers See Things Differently (FIST-D): A Robotic Explosive Ordnance Disposal (EOD) based on Augmented Tactile Imaging

爆炸物处理是危险性较高的职业之一，无论是在军事和执法部门遇到简易爆炸装置时，还是在自然灾害发生后必须迅速、安全和迅速处理暴露的爆炸材料时。 爆炸物处理技术人员应对危险情况时只穿防护服，这可能会减少爆炸的影响，但不能提供全面保护。 此外，爆炸物处理技术人员容易疲劳、热应激和灵活性降低。 这可能导致爆炸物处理技术人员有效和安全地处理威胁的能力下降。机器人可以极大地降低许多风险，帮助减少技术人员的暴露和目标时间，减少其他非常危险的情况的风险，并使技术人员更好地远离伤害。技术人员仍然需要远程操作机器人。爆炸物处理中的一个长期问题是这种机器人系统能够传达的感知信息有限。目前可用的机器人在很大程度上依赖于视觉和视频遥测，当简易爆炸装置被掩埋或隐藏时，这一点可能会严重受损。这项研究解决了这一科学挑战，通过开发一个多模态感知图像组成的触觉，光学和力的信息，使用遥控机器人。这样的遥控机器人将：1）以操作员易于解释的格式提供多模态图像; 2）使用机器学习原理从有限的观察中学习; 3）从纹理和权重签名中识别对象;以及4）推荐接近和探索对象的最佳策略。除了其主要应用之外，该提案中涉及的理论和技术还影响了其他领域，其中灵巧性和触觉反馈是成功完成任务的关键，例如远程手术。该项目的一个独特之处是一套双手工具提示，配备了多传感器设备，用于收集触觉，力和化学成分信息，用于目标表征和行动。随着探索的进行，物体轮廓（基于感知信息）从离散观测中构建，这些观测被积累并拟合到基于迁移学习和零/一次性学习的机器学习模型中。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Explosive ordnance disposal is among the more hazardous occupations whether in the military and law enforcement when confronted with improvised explosive devices (IEDs), or after natural disasters when exposed explosive materials must be quickly, safely, and expeditiously disposed. Explosive ordnance disposal (EOD) technicians respond to dangerous situations, wearing only protective suits, which may reduce the impact of a blast but do not provide complete protection. Moreover, EOD technicians are subject to fatigue, heat stress, and reduced dexterity. These can lead to a reduced ability by EOD technicians to deal with threats effectively and safely. Robots can greatly mitigate many of the risks, help reduce a technician's exposure and time-on-target, lessen the risk of otherwise very dangerous situations, and keep the technician better out of harm's way. Technician are still need albeit from afar by teleoperating the robots. A chronic problem in EOD is the limited perceptual information that such robotic systems can convey. Currently available robots depend heavily on vision and video telemetry, which can be drastically impaired when IEDs are buried or concealed. This research addresses this scientific challenge by developing a multi-modal perceptual image comprised of tactile, optical, and force information, using a telerobot. Such a telerobot will: 1) deliver a multimodal image in a format that is easily interpretable by the operator; 2) learn from limited observations using principles of machine learning; 3) recognize objects from texture and weight signatures; and 4) recommend best strategies to approach and explore the objects. In addition to its main application, the theories and technologies involved in this proposal impact other fields in which dexterity and tactile feedback is key for successful task completion, such as tele-surgery. A unique feature of this project is a set of bimanual tool tips, equipped with multi-sensory devices for collecting tactile, force and chemical composition information for target characterization and action. As the exploration takes place, object profiles (based on sensed information) are built from discrete observations, which are accumulated and fitted to machine learning models based on transfer learning and zero/one-shot learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.