Vision-based supervisory control and predictive display for uncalibrated tele-robotics

点击翻译按钮获取中文摘要

My research is in the intersection of computer vision, robotics and graphics. In particular I'm interested in how to use visual information and coordinate frames to encode and communicate spatial information, motion, and interactions. Among applications I work on are visual servoing for robot motion control and predictive display for operator visualization. Both of these combine into tele-robotics systems.         A key idea is to work with visual (camera) spaces instead of Euclidean world coordinates as in conventional engineering. Deictic motions and gestures of a human in a scene can be viewed by cameras and interpreted in camera space w.r.t. that scene. Similarly, in tele-robotics, a human operator is immersed in a visualization of a remote scene and can gesture instructions w.r.t. the shared visual space between the remote and local sites. Using sensory frames instead of a conventional world coordinate frame brings up new issues in task and trajectory planning. Furthermore, robots operating in human settings face more varying tasks than currently in manufacturing. Achieving full autonomy in such challenging situations is difficult, and may not always be desirable. Instead our thrust is to endow the robot with the ability to carry out motions and small subtasks semi-autonomously, while under the guidance of a human, so called supervised control. The motions and tasks are built bottom up, from simple motions, reaching and fine manipulation to more complex subroutines, e.g. pick-and-place. Tasks we can implement include a light bulb exchange subroutine, setting a dinner table and solving a child's shape sorter puzzle. The routines are currently built by scripting, but we plan to look into learning from observing a human. A general way of visually instructing a robot has many applications from the high end e.g. in surgical assistive robotics, where the human surgeon can't interrupt the surgery to use a conventional keyboard interface, to everyday service robotics e.g. for the handicapped and elderly.