Collaborative Research: A Novel User Interface for Operating an Assistive Robot Arm in Unstructured Environments

在这个项目中，PI将设计，开发和临床测试基于视觉的用户界面，以提高轮椅安装辅助机器人手臂在非结构化环境中的功效。 目标人群是上肢运动受限的轮椅患者，如诊断为脑瘫、ALS、脊髓灰质炎、多发性硬化症、脊髓损伤、肌营养不良和影响上肢使用的类似疾病的患者。 我们的目标是让这些人在各种非结构化环境中（如杂货店、客厅或办公室）舒适、快速地独立工作。 该项目的创新之处在于将机器人运动分离为粗和细的组件，而不是目前规范的不自然的关节或笛卡尔运动。 为了将他们大胆的设想变成现实，PI将：利用计算机视觉技术和人在回路中开发一个整体运动人机界面;利用计算机图像处理算法实现一个实时鲁棒的特征识别器，该识别器能够向用户建议感兴趣的区域，使用计算机视觉技术按颜色、深度和其他标准进行分割;通过融合视觉和自适应控制的几何思想，实现机器人末端执行器的精细运动，以促进拾取和放置任务;通过统一HRI，传感和控制算法，开发工作原型;并利用以下人力资源，为上肢有障碍的轮椅使用者示范日常生活任务的基准活动：好牧人康复医院位于宾夕法尼亚州的利哈伊谷。 更广泛的影响：增强功能的轮椅机器人的设计将是一个重大的飞跃，走向社会的广泛部分，其成员，否则只有有限的资源和机会的康复。 PI希望他们的方法与任何具有车载视觉的移动终端直接相关，并且可以利用人类在回路中的优势，从而为辅助技术提供一种新的人机交互模型。 开发的交互方法将适用于各种各样的接入设备，从单开关扫描到啜吸到操纵杆。 此外，PI期望在这个项目中展示的视觉和非线性控制的融合将推进计算机视觉和视觉伺服的理论和应用。

In this project the PIs will design, develop, and clinically test a vision-based user interface for enhancing the efficacy of wheelchair mounted assistive robot arms in unstructured environments. The target population is wheelchair bound individuals with limited upper extremity movement, such as patients diagnosed with Cerebral Palsy, ALS, Poliomyelitis, Multiple Sclerosis, Spinal Cord Injury, Muscular Dystrophy, and similar conditions that affect use of the upper limbs. The goal is to allow these people to function independently with comfort and speed in a variety of unstructured environments such as a grocery store, a living room, or an office. The innovation in this project that sets it apart from existing approaches is the segregation of robot motion into gross and fine components instead of unnatural joint or Cartesian motion as is currently the norm. To transform their bold vision into reality, the PIs will: develop a gross motion human-robot interface utilizing computer vision techniques and the human in the loop; utilize computer image processing algorithms for implementing a real-time robust feature identifier that is able to suggest to the user areas of interest, using computer vision techniques to segment by color, depth and other criteria; effect fine motion of the robot end-effector to facilitate pick-and-place tasks via fusion of geometric ideas from vision and adaptive control; develop a working prototype through unification of HRI, sensing, and control algorithms; and demonstrate benchmark activities of daily living tasks for wheelchair bound individuals with upper extremity impairments by tapping into the human resources available at Good Shepherd Rehabilitation Hospital located in Pennsylvania's Lehigh Valley. Broader Impacts: The design of an enhanced functionality wheelchair robot will be a major leap toward rehabilitation of a broad segment of society whose members otherwise have only limited access to resources and opportunity. The PIs expect their approach to be directly relevant to any mobile device with on-board vision and where one can take advantage of the human in the loop, and thus to provide a new model of human-robot interaction for assistive technology. Interaction methods developed will be adaptable to a wide range of access devices, ranging from single switch scanning to sip-and-puff to a joystick. Moreover, the PIs expect that the fusion of vision and nonlinear control demonstrated in this project will advance the theory and applicability of computer vision and visual servoing.