Collaborative Research: A Novel User Interface for Operating an Assistive Robot Arm in Unstructured Environments

在这个项目中，PI将设计、开发和临床测试基于视觉的用户界面，以提高非结构化环境中安装在轮椅上的辅助机械臂的效率。目标人群是坐在轮椅上、上肢活动受限的人，例如被诊断为脑瘫、肌萎缩侧索硬化症、小儿麻痹症、多发性硬化症、脊髓损伤、肌肉营养不良以及影响上肢使用的类似疾病的患者。目标是让这些人能够在杂货店、起居室或办公室等各种非结构化环境中舒适而快速地独立工作。这个项目的创新之处在于，它将机器人的运动分为粗略和精细两个部分，而不是像目前的标准那样，将不自然的关节或笛卡尔运动分离出来。为了将大胆的愿景转化为现实，PI将：利用计算机视觉技术和循环中的人开发粗大运动的人-机器人界面；利用计算机图像处理算法实现实时稳健的特征识别器，该特征识别器能够向用户建议感兴趣的区域，使用计算机视觉技术根据颜色、深度和其他标准进行分割；通过融合视觉和自适应控制的几何思想来实现机器人末端执行器的精细运动，以便于挑选和放置任务；通过统一HRI、传感和控制算法来开发工作原型；并通过利用位于宾夕法尼亚州利哈伊谷的好牧人康复医院可用的人力资源，展示坐在轮椅上的上肢损伤患者的日常生活任务基准活动。更广泛的影响：功能更强的轮椅机器人的设计将是朝着恢复广大社会阶层的方向迈出的重大飞跃，否则他们只能有限地获得资源和机会。PI希望他们的方法与任何具有车载视觉的移动设备直接相关，在这种设备中，人们可以利用循环中的人，从而为辅助技术提供一种新的人-机器人交互模式。开发的交互方法将适用于广泛的接入设备，从单开关扫描到sip-and-puff再到操纵杆。此外，PI还期望本项目中展示的视觉与非线性控制的融合将促进计算机视觉和视觉伺服的理论和应用。

In this project the PIs will design, develop, and clinically test a vision-based user interface for enhancing the efficacy of wheelchair mounted assistive robot arms in unstructured environments. The target population is wheelchair bound individuals with limited upper extremity movement, such as patients diagnosed with Cerebral Palsy, ALS, Poliomyelitis, Multiple Sclerosis, Spinal Cord Injury, Muscular Dystrophy, and similar conditions that affect use of the upper limbs. The goal is to allow these people to function independently with comfort and speed in a variety of unstructured environments such as a grocery store, a living room, or an office. The innovation in this project that sets it apart from existing approaches is the segregation of robot motion into gross and fine components instead of unnatural joint or Cartesian motion as is currently the norm. To transform their bold vision into reality, the PIs will: develop a gross motion human-robot interface utilizing computer vision techniques and the human in the loop; utilize computer image processing algorithms for implementing a real-time robust feature identifier that is able to suggest to the user areas of interest, using computer vision techniques to segment by color, depth and other criteria; effect fine motion of the robot end-effector to facilitate pick-and-place tasks via fusion of geometric ideas from vision and adaptive control; develop a working prototype through unification of HRI, sensing, and control algorithms; and demonstrate benchmark activities of daily living tasks for wheelchair bound individuals with upper extremity impairments by tapping into the human resources available at Good Shepherd Rehabilitation Hospital located in Pennsylvania's Lehigh Valley. Broader Impacts: The design of an enhanced functionality wheelchair robot will be a major leap toward rehabilitation of a broad segment of society whose members otherwise have only limited access to resources and opportunity. The PIs expect their approach to be directly relevant to any mobile device with on-board vision and where one can take advantage of the human in the loop, and thus to provide a new model of human-robot interaction for assistive technology. Interaction methods developed will be adaptable to a wide range of access devices, ranging from single switch scanning to sip-and-puff to a joystick. Moreover, the PIs expect that the fusion of vision and nonlinear control demonstrated in this project will advance the theory and applicability of computer vision and visual servoing.