NRI: An Egocentric Computer Vision based Active Learning Co-Robot Wheelchair

描述（由申请人提供）： 该提案的目的是对计算机视觉和机器学习中的基础模型和算法进行研究，用于基于自我中心视觉的主动学习协作机器人轮椅系统，以改善手部功能有限或完全没有手部功能的老年人和残疾人的生活质量，并依赖轮椅进行移动。在该合作机器人系统中，轮椅使用者佩戴一副以自我为中心的相机眼镜，即，摄像机捕捉用户的视野。这个项目有助于减少病人对护理人员的依赖。它符合NINR的使命，即解决国家人口老龄化和医疗保健劳动力短缺所提出的关键问题，并支持以患者为中心的研究，鼓励和使个人成为自己福祉的监护人。 以自我为中心的相机有两个目的。一方面，基于视觉的运动传感，该系统可以捕捉独特的头部运动模式的用户控制机器人轮椅在一个非侵入性的方式。其次，它作为协作机器人系统的独特环境感知视觉传感器，因为用户将通过有意识或无意识地转移注意力来自然地对周围环境做出反应。基于来自自我中心视觉传感器和其他车载机器人传感器的输入，利用在线学习水库计算网络，这不仅使机器人轮椅系统能够在不确定性对于自主操作太高时主动地请求来自用户的控制，而且还便于机器人轮椅系统从请求的用户控制中学习。通过这种方式，闭环协作机器人轮椅系统将不断发展，并且随着时间的推移，更有能力处理更复杂的环境。 该项目的目标包括：1）开发一种方法，利用以自我为中心的计算机视觉为基础的头部运动传感作为轮椅控制的替代方法; 2）开发一种方法，利用视觉运动从以自我为中心的摄像机类别独立的移动障碍物检测;和3）关闭主动学习合作机器人轮椅系统的循环，通过基于不确定性的主动在线学习。

DESCRIPTION (provided by applicant): The aim of this proposal is to conduct research on the foundational models and algorithms in computer vision and machine learning for an egocentric vision based active learning co-robot wheelchair system to improve the quality of life of elders and disabled who have limited hand functionality or no hand functionality at all, and rely on wheelchairs for mobility. In this co-robt system, the wheelchair users wear a pair of egocentric camera glasses, i.e., the camera is capturing the users' field-of-the-views. This project help reduce the patients' reliance on care-givers. It fits NINR's mission in addressing key issues raised by the Nation's aging population and shortages of healthcare workforces, and in supporting patient-focused research that encourage and enable individuals to become guardians of their own well-beings. The egocentric camera serves two purposes. On one hand, from vision based motion sensing, the system can capture unique head motion patterns of the users to control the robot wheelchair in a noninvasive way. Secondly, it serves as a unique environment aware vision sensor for the co-robot system as the user will naturally respond to the surroundings by turning their focus of attention, either consciously or subconsciously. Based on the inputs from the egocentric vision sensor and other on-board robotic sensors, an online learning reservoir computing network is exploited, which not only enables the robotic wheelchair system to actively solicit controls from the users when uncertainty is too high for autonomous operation, but also facilitates the robotic wheelchair system to learn from the solicited user controls. This way, the closed- loop co-robot wheelchair system will evolve and be more capable of handling more complicated environment overtime. The aims ofthe project include: 1) develop an method to harness egocentric computer vision-based sensing of head movements as an alternative method for wheelchair control; 2) develop a method leveraging visual motion from the egocentric camera for category independent moving obstacle detection; and 3) close the loop of the active learning co-robot wheelchair system through uncertainty based active online learning.