NRI: An Egocentric Computer Vision based Active Learning Co-Robot Wheelchair

描述(由申请人提供)： 该方案的目的是研究基于自我中心视觉的主动学习协同机器人轮椅系统的计算机视觉和机器学习的基本模型和算法，以改善手功能有限或根本没有手功能的老年人和残疾人的生活质量，并依赖轮椅进行行动。在这种协作-Robt系统中，轮椅用户戴着一副以自我为中心的相机眼镜，即相机捕捉到用户的视场。这个项目有助于减少患者对照顾者的依赖。它符合NINR的使命，即解决国家人口老龄化和医疗保健劳动力短缺带来的关键问题，并支持以患者为中心的研究，鼓励并使个人成为自己福祉的守护者。以自我为中心的相机有两个目的。一方面，通过基于视觉的运动感知，该系统可以捕捉用户独特的头部运动模式，以非侵入性的方式控制机器人轮椅。其次，它是协作机器人系统独特的环境感知视觉传感器，因为用户会自然地通过转移他们的注意力焦点来对环境做出反应，无论是有意识的还是下意识的。基于以自我为中心的视觉传感器和其他车载机器人传感器的输入，开发了在线学习库计算网络，不仅使机器人轮椅系统在不确定性太高而无法自主操作时能够主动向用户请求控制，而且便于机器人轮椅系统学习请求的用户控制。这样，闭环式协作机器人轮椅系统将进化，更有能力加班处理更复杂的环境。该项目的目标包括：1)开发一种方法来利用基于自我中心的计算机视觉的头部运动感知作为轮椅控制的替代方法；2)开发一种利用自我中心摄像头的视觉运动来检测类别独立的运动障碍物的方法；以及3)通过基于不确定性的主动在线学习来关闭主动学习协作机器人轮椅系统的回路。

DESCRIPTION (provided by applicant): The aim of this proposal is to conduct research on the foundational models and algorithms in computer vision and machine learning for an egocentric vision based active learning co-robot wheelchair system to improve the quality of life of elders and disabled who have limited hand functionality or no hand functionality at all, and rely on wheelchairs for mobility. In this co-robt system, the wheelchair users wear a pair of egocentric camera glasses, i.e., the camera is capturing the users' field-of-the-views. This project help reduce the patients' reliance on care-givers. It fits NINR's mission in addressing key issues raised by the Nation's aging population and shortages of healthcare workforces, and in supporting patient-focused research that encourage and enable individuals to become guardians of their own well-beings. The egocentric camera serves two purposes. On one hand, from vision based motion sensing, the system can capture unique head motion patterns of the users to control the robot wheelchair in a noninvasive way. Secondly, it serves as a unique environment aware vision sensor for the co-robot system as the user will naturally respond to the surroundings by turning their focus of attention, either consciously or subconsciously. Based on the inputs from the egocentric vision sensor and other on-board robotic sensors, an online learning reservoir computing network is exploited, which not only enables the robotic wheelchair system to actively solicit controls from the users when uncertainty is too high for autonomous operation, but also facilitates the robotic wheelchair system to learn from the solicited user controls. This way, the closed- loop co-robot wheelchair system will evolve and be more capable of handling more complicated environment overtime. The aims ofthe project include: 1) develop an method to harness egocentric computer vision-based sensing of head movements as an alternative method for wheelchair control; 2) develop a method leveraging visual motion from the egocentric camera for category independent moving obstacle detection; and 3) close the loop of the active learning co-robot wheelchair system through uncertainty based active online learning.