NRI: A Wearable Robotic Object Manipulation Aid for the Visually Impaired

项目总结（见说明）： 拟议研究的目的是开发一种新的技术，为视障人士的可穿戴机器人对象操作援助（W-ROMA）。W-ROMA是一种手戴式辅助设备，可帮助视障人士有效地抓住物体。得益于板载计算机视觉方法，W-ROMA能够检测目标物体，确定手-物体未对准，并通过自然的人机界面向佩戴者传达手-物体对准所需的手部运动。W-ROMA将从两个方面为视障者的独立生活做出贡献：首先，它帮助视障者独立旅行，使他们能够识别可移动的障碍物并操纵障碍物通过。第二，协助视障人士有效地抓取物体作非航行用途。PI将让研究生、本科生和高中生参与该项目，并利用拟议的项目活动来招募和留住工程专业的学生。该提案的智力价值是开发新的计算机视觉和人机交互方法，支持视障人士在独立的日常生活中准确有效地抓取物体。这些方法包括：（1）新的实时对象识别方法;（2）创新的手-对象对准机制;（3）用于对象形状呈现的新的混合触觉显示系统;以及（4）计算高效的设备定位方法。所提出的解决方案可以封装在手持机器人设备中。W-ROMA将为视障人士提供新的协作机器人功能。PI已经对计算机视觉和触觉显示方法进行了概念验证研究，结果很有希望。更广泛的影响包括：（1）研究将对大型视障群体产生积极影响;（2）所提出的方法可以应用于其他小型机器人系统，这些系统在军事监视，执法和搜索和救援方面具有广泛的应用;以及（3）该项目将改善PI大学的研究基础设施，并为研究生和本科生未来在科学和工程领域的职业生涯提供培训。

PROJECT SUMMARY (See instructions): The objective of the proposed research is to develop new technology for a Wearable Robotic Object Manipulation Aid (W-ROMA) for the visually impaired. The W-ROMA is a hand-worn assistive device that provides assistance to a visually impaired individual in effectively grasping an object. Thanks to the onboard computer vision methods, the W-ROMA is capable of detecting a target object, determining the hand-object misalignment, and conveying to the wearer, via natural human-device interfaces, the desired hand motion for hand-object alignment. The W-ROMA will contribute to the independent lives of the visually impaired in twofold: First, it helps the visually impaired with independent travel by enabling them to identify a movable obstacle and manipulate the obstacle to make a passage. Second, it assists the visually impaired in effectively grasping an object for non-navigational purpose. The PIs will involve graduate, undergraduate and high school students in the project and use the proposed project activities to recruit and retain engineering students. The proposal's Intellectual Merit is the development of new computer vision and human-robot interaction methods that support accurate and effective object grasping for the visually impaired for their independent daily lives. These methods include: (1) a new real-time object recognition method; (2) an innovative hand-object alignment mechanism; (3) a novel hybrid tactile display system for object shape rendering; and (4) a computationally efficient device localization method. The proposed solutions can be encapsulated in a hand-worn robotic device. The W-ROMA will provide new co-robotic functions for the visually impaired. The PIs have performed proof of concept studies for the computer vision and tactile display methods and the results are promising. The broader impacts include: (1) the research will positively impact the large visually impaired community; (2) the proposed methods can be applied to other small robotic systems that have a wide range of applications in military surveillance, law enforcement, and search and rescue; and (3) the project will improve the research infrastructure of the PI's university and train graduate and undergraduate students for their future careers in science and engineering.