Development of Vision-Guided Shared Control for Assistive Robotic Manipulators

使用电动移动设备的退伍军人，包括那些患有高位脊髓损伤（SCI）的退伍军人， 肌萎缩侧索硬化症（ALS）和多发性硬化症（MS）经常会出现严重的上肢 损伤此外，许多年老的退伍军人和偏瘫的退伍军人也可能患有功能性瘫痪。 失去上肢功能上肢损伤的管理和护理通常涉及一系列 辅助解决方案。然而，操作辅助的产品可用性和技术进步 远远落后于那些流动性。这些人中的许多人，尽管他们的独立流动性，无法达到 一杯水，做一顿简单的饭，拿起牙刷。他们仍然需要一个私人助理的帮助。 日常生活基本活动（ADL）的护理人员，包括接触和物体处理/操作。与 随着机器人技术的快速发展，辅助机器人操作器（ARM）成为一种可行的 解决方案，帮助上肢残疾的退伍军人完成日常任务， 对象处理和操纵。ARM通常配备有许多自由度（DOF），但用户 不能控制所有的自由度在同一时间与传统的操纵杆，并需要切换模式相当 通常是为了完成简单的操作任务，特别是当ARM接近目标并需要 以适当地对准以便于操作。因此，现有的ARM缺乏效率， 特别是在非结构化环境中。 本项目的目标是将联合收割机视觉引导共享（VGS）控制与两种类型的环境相结合 修改，以解决实际使用的ARM的有效性和效率。两类 环境修改包括使用商业或定制的自适应工具（例如，可以容纳的保持器 瓶子或罐子，以便ARM可以打开它），并将基准标记（类似于QR码）添加到对象或自适应 工具，使基于视觉的跟踪鲁棒性和可靠的现实世界的应用。建立在环境之上 修改后，VGS控制将允许用户通过将ARM移动到靠近标记对象来启动任何任务， 而ARM在检测到目标后接管精细操作。具体目标是：（1）发展， 实施，并使用基准标记和自适应工具对VGS控件进行台架测试;（2）评估新的 控制电动轮椅使用者谁将使用轮椅安装ARM完成一套 日常操作任务;（3）探索增强的ARM的潜在益处和局限性， 在为期一个月的试点家庭试验中，使用基准标记和自适应工具进行VGS控制。 我们期望通过一个更实用的方法来改善上肢残疾退伍军人的操作功能。 以及基于视觉的机器人控制和人机交互技术的可用实现。的 VA假肢和感觉辅助服务（PSAS）的使命是提供全面的支持， 退伍军人的健康和独立。拟议的工作显然支持这一使命， ARM在协助退伍军人完成日常操作任务和支持采用 这种先进的辅助技术

Veterans who use powered mobility devices including those with high-level spinal cord injury (SCI), amyotrophic lateral sclerosis (ALS), and multiple sclerosis (MS) often experience serious upper extremity impairments. In addition, many older Veterans and Veterans with hemiplegia may also suffer from functional loss in their upper extremities. Management and care of upper extremity impairments often involve a range of assistive solutions. However, product availability and technological advancement for manipulation assistance fall far behind those for mobility. Many of these individuals, despite their independent mobility, cannot reach for a glass of water, make a simple meal, and pick up a tooth brush. They still require assistance from a personal caregiver for essential activities of daily living (ADLs) involving reaching and object handling/manipulation. With the rapid advancement of robotics technology, assistive robotic manipulators (ARMs) emerge as a viable solution for assisting Veterans with upper extremity impairments to complete daily tasks involving reaching, object handling, and manipulation. ARMs are often equipped with many degrees of freedom (DOF), but users cannot control all of the DOFs at the same time with a conventional joystick, and need to switch modes quite often to complete even simple manipulation tasks, especially when an ARM gets close to the target and need to be aligned appropriately for manipulation. Thus existing ARMs suffer from the lack of efficiency and effectiveness especially in an unstructured environment. The goal of this project is to combine vision-guided shared (VGS) control with two types of environment modifications to address the effectiveness and efficiency of ARMs for real-world use. The two types of environment modifications include using commercial or custom adaptive tools (e.g., a holder that can hold a bottle or jar so an ARM can open it), and adding fiducial markers (similar to QR codes) to objects or adaptive tools to make vision-based tracking robust and reliable for real-world applications. Built upon the environment modifications, the VGS control will allow a user to initiate any task by moving an ARM close to a tagged object, and the ARM to take over fine manipulation upon detecting the target. The specific aims are to (1) to develop, implement, and bench-test the VGS control with fiducial markers and adaptive tools; (2) to evaluate the new control among powered wheelchair users who will use a wheelchair-mounted ARM to complete a set of everyday manipulation tasks; (3) to explore the potential benefits and limitations of ARMs enhanced by the VGS control with fiducial markers and adaptive tools in a one-month pilot home trial. We expect to improve manipulation functions of Veterans with upper limb impairments through a more practical and usable implementation of vision-based robotic control and human-robot interaction technologies. The mission of the VA Prosthetic & Sensory Aids Service (PSAS) is to provide comprehensive support to optimize the health and independence of the Veteran. The proposed work clearly supports this mission by maximizing the potential of ARMs in assisting Veterans with everyday manipulation tasks and supporting the adoption of this advanced assistive technology.