Project Title: Integrating computer vision with neural interfacing for semi-autonomous control of robotic limbs.

背景：人类的许多日常活动和任务都依赖于手，因此失去一只手可能会对他们与其他物体互动的方式产生很大的影响。假肢的使用可以在一定程度上改善截肢者的生活方式，起到部分替代失去的肢体的作用。大多数假肢系统依赖顺序和比例控制，这样用户就可以通过输入动作驱动假肢调整多个自由度[1]。因此，用户负责抓取的大部分步骤，从生物学上讲，抓取通过一系列阶段发生，从规划到执行，涉及到来自多个来源的感觉信息的整合[1]。目的：在这个项目中，我们将追求这样的概念，即包括视觉信息在内的感觉信息可以通过机器人系统本身来收集和输入，从而允许对假肢的半自主控制，使得用户需要执行更少的输入动作来以正确的对准方式抓取对象。因此，该项目将研究在假肢中使用共享控制来抓取物体的潜力。摄像头和可能的其他传感器，如加速计，将被放置在假体本身上，系统将从这些传感器和用户本身接收信息(例如肌电控制)。最终目标是将整个系统作为一个完整的嵌入式解决方案。研究步骤：除了围绕研究主题进行全面的文献综述外，我们还将选择要集成到假肢中的传感器以及它们的安装点。将实施计算机视觉和机器学习算法，以便根据对象和最合适的接触点等多种因素选择要使用的最佳抓取类型。除此之外，我们还将加强肌电(EMG)接口与假肢的结合。然后，计算机视觉、抓取选择系统将被整合到假肢中，以创建一个原型，涉及两个系统的完全嵌入、共享控制。最后，将进行研究，以了解与标准的多自由度假肢相比，完整的系统是否改善了个人的抓地力。

Background: Humans depend on their hands for a multitude of everyday activities and tasks, thus the loss of one's hand can have a large impact on the way in which they interact with other objects. Use of a prosthesis can enhance an amputee's lifestyle to some extent, acting as a partial substitute for the lost limb. Most prosthetic systems rely on sequential and proportional control, such that the user drives the prosthesis to adjust multiple degrees of freedom through input actions [1]. Thus, the user is responsible for most of the steps of grasping, which biologically takes place through a sequence of phases, from planning to execution, and involves the integration of sensory information from multiple sources [1].Aims: In this project we will pursue the notion that sensory information, including visual information, can be collected and input through the robotic system itself, allowing for semi-autonomous control of the prosthesis, such that the user needs to perform fewer input actions to grasp an object in the correct alignment. Thus, the project will study the potential for using shared control for grasping objects in prosthesis. Cameras and potentially other sensors, such as accelerometers, will be placed on the prosthetic itself, and the system will receive information both from these sensors and from the user themselves (e.g. myoelectric control). The end goal is to have the full system as a complete embedded solution.Research steps: Alongside performing a thorough literature review surrounding the research subject, we will choose the sensors to be incorporated into the prosthetic, as well as their mounting points. Computer vision and machine learning algorithms will be implemented in order to select the optimal grasp type to be used based on multiple factors, such as the object and the most appropriate contact points. Alongside this, we will be fortifying a robust electromyography (EMG) interface incorporated with a prosthesis. The computer vision, grasp-selective system will then be incorporated into the prosthesis to create a prototype involving the fully-embedded, shared control of the two systems. Finally, studies will be performed to understand whether or not the completed system improves an individual's grip compared to a standard multiple degree of freedom prosthesis.