NRI: A Wearable Robotic Object Manipulation Aid for the Visually Impaired

项目总结(见说明)： 这项拟议的研究的目标是为视障人士开发一种可穿戴的机器人对象操纵辅助设备(W-ROMA)的新技术。W-Roma是一种手持辅助设备，可以帮助视障人士有效地抓住物体。多亏了车载计算机视觉方法，W-Roma能够检测目标对象，确定手-对象未对准，并通过自然的人-设备界面向佩戴者传达手-对象对齐所需的手运动。W-Roma将在两个方面为视障人士的独立生活做出贡献：首先，它帮助视障人士进行独立旅行，使他们能够识别可移动的障碍物并操纵障碍物通过。第二，它帮助视障人士有效地抓住非导航目的的物体。PIs将让研究生、本科生和高中生参与该项目，并利用拟议的项目活动来招募和留住工程专业的学生。该提案的智力价值在于开发了新的计算机视觉和人机交互方法，支持视障人士在独立的日常生活中准确有效地抓取物体。这些方法包括：(1)一种新的实时物体识别方法；(2)一种创新的手-物体对准机制；(3)一种用于物体形状绘制的新型混合触觉显示系统；(4)一种计算效率高的设备定位方法。建议的解决方案可以封装在手持机器人设备中。W-Roma将为视障人士提供新的联合机器人功能。PI已经对计算机视觉和触觉显示方法进行了概念验证研究，结果令人振奋。更广泛的影响包括：(1)研究将对大型视障社区产生积极影响；(2)建议的方法可应用于其他在军事监视、执法和搜救中有广泛应用的小型机器人系统；以及(3)该项目将改善PI大学的研究基础设施，并为他们未来的科学和工程职业生涯培养研究生和本科生。

PROJECT SUMMARY (See instructions): The objective of the proposed research is to develop new technology for a Wearable Robotic Object Manipulation Aid (W-ROMA) for the visually impaired. The W-ROMA is a hand-worn assistive device that provides assistance to a visually impaired individual in effectively grasping an object. Thanks to the onboard computer vision methods, the W-ROMA is capable of detecting a target object, determining the hand-object misalignment, and conveying to the wearer, via natural human-device interfaces, the desired hand motion for hand-object alignment. The W-ROMA will contribute to the independent lives of the visually impaired in twofold: First, it helps the visually impaired with independent travel by enabling them to identify a movable obstacle and manipulate the obstacle to make a passage. Second, it assists the visually impaired in effectively grasping an object for non-navigational purpose. The PIs will involve graduate, undergraduate and high school students in the project and use the proposed project activities to recruit and retain engineering students. The proposal's Intellectual Merit is the development of new computer vision and human-robot interaction methods that support accurate and effective object grasping for the visually impaired for their independent daily lives. These methods include: (1) a new real-time object recognition method; (2) an innovative hand-object alignment mechanism; (3) a novel hybrid tactile display system for object shape rendering; and (4) a computationally efficient device localization method. The proposed solutions can be encapsulated in a hand-worn robotic device. The W-ROMA will provide new co-robotic functions for the visually impaired. The PIs have performed proof of concept studies for the computer vision and tactile display methods and the results are promising. The broader impacts include: (1) the research will positively impact the large visually impaired community; (2) the proposed methods can be applied to other small robotic systems that have a wide range of applications in military surveillance, law enforcement, and search and rescue; and (3) the project will improve the research infrastructure of the PI's university and train graduate and undergraduate students for their future careers in science and engineering.