NRI: An Egocentric Computer Vision based Active Learning Co-Robot Wheelchair

点击翻译按钮获取中文摘要

DESCRIPTION (provided by applicant): The aim of this proposal is to conduct research on the foundational models and algorithms in computer vision and machine learning for an egocentric vision based active learning co-robot wheelchair system to improve the quality of life of elders and disabled who have limited hand functionality or no hand functionality at all, and rely on wheelchairs for mobility. In this co-robt system, the wheelchair users wear a pair of egocentric camera glasses, i.e., the camera is capturing the users' field-of-the-views. This project help reduce the patients' reliance on care-givers. It fits NINR's mission in addressing key issues raised by the Nation's aging population and shortages of healthcare workforces, and in supporting patient-focused research that encourage and enable individuals to become guardians of their own well-beings. The egocentric camera serves two purposes. On one hand, from vision based motion sensing, the system can capture unique head motion patterns of the users to control the robot wheelchair in a noninvasive way. Secondly, it serves as a unique environment aware vision sensor for the co-robot system as the user will naturally respond to the surroundings by turning their focus of attention, either consciously or subconsciously. Based on the inputs from the egocentric vision sensor and other on-board robotic sensors, an online learning reservoir computing network is exploited, which not only enables the robotic wheelchair system to actively solicit controls from the users when uncertainty is too high for autonomous operation, but also facilitates the robotic wheelchair system to learn from the solicited user controls. This way, the closed- loop co-robot wheelchair system will evolve and be more capable of handling more complicated environment overtime. The aims ofthe project include: 1) develop an method to harness egocentric computer vision-based sensing of head movements as an alternative method for wheelchair control; 2) develop a method leveraging visual motion from the egocentric camera for category independent moving obstacle detection; and 3) close the loop of the active learning co-robot wheelchair system through uncertainty based active online learning.