Project Title: Integrating computer vision with neural interfacing for semi-autonomous control of robotic limbs.

背景：人类的许多日常活动和任务都依赖于他们的手，因此失去一只手会对他们与其他物体互动的方式产生很大影响。使用假肢可以在一定程度上改善被截肢者的生活方式，作为失去的肢体的部分替代品。大多数假肢系统依赖于顺序和比例控制，用户通过输入动作[1]来驱动假肢调整多个自由度。因此，使用者负责抓取的大部分步骤，这在生物学上是通过一系列阶段发生的，从计划到执行，并涉及来自多个来源的感官信息的整合。目的：在这个项目中，我们将追求的概念是，包括视觉信息在内的感官信息可以通过机器人系统本身收集和输入，从而实现对假肢的半自主控制，这样用户就需要执行更少的输入动作来以正确的方式抓住物体。因此，该项目将研究在假肢中使用共享控制来抓取物体的潜力。摄像头和其他传感器，如加速度计，将被放置在假肢本身，系统将从这些传感器和用户自己（如肌电控制）接收信息。最终目标是将整个系统作为一个完整的嵌入式解决方案。研究步骤：除了围绕研究主题进行彻底的文献综述外，我们还将选择要纳入假肢的传感器及其安装点。将实施计算机视觉和机器学习算法，以便根据多个因素（例如对象和最合适的接触点）选择要使用的最佳抓取类型。除此之外，我们将加强与假体结合的强健肌电图（EMG）界面。然后，计算机视觉、抓取选择系统将被整合到假肢中，以创建一个包含两个系统的完全嵌入式共享控制的原型。最后，研究将进行，以了解是否完成系统改善个人的抓地力相比，一个标准的多自由度假体。

Background: Humans depend on their hands for a multitude of everyday activities and tasks, thus the loss of one's hand can have a large impact on the way in which they interact with other objects. Use of a prosthesis can enhance an amputee's lifestyle to some extent, acting as a partial substitute for the lost limb. Most prosthetic systems rely on sequential and proportional control, such that the user drives the prosthesis to adjust multiple degrees of freedom through input actions [1]. Thus, the user is responsible for most of the steps of grasping, which biologically takes place through a sequence of phases, from planning to execution, and involves the integration of sensory information from multiple sources [1].Aims: In this project we will pursue the notion that sensory information, including visual information, can be collected and input through the robotic system itself, allowing for semi-autonomous control of the prosthesis, such that the user needs to perform fewer input actions to grasp an object in the correct alignment. Thus, the project will study the potential for using shared control for grasping objects in prosthesis. Cameras and potentially other sensors, such as accelerometers, will be placed on the prosthetic itself, and the system will receive information both from these sensors and from the user themselves (e.g. myoelectric control). The end goal is to have the full system as a complete embedded solution.Research steps: Alongside performing a thorough literature review surrounding the research subject, we will choose the sensors to be incorporated into the prosthetic, as well as their mounting points. Computer vision and machine learning algorithms will be implemented in order to select the optimal grasp type to be used based on multiple factors, such as the object and the most appropriate contact points. Alongside this, we will be fortifying a robust electromyography (EMG) interface incorporated with a prosthesis. The computer vision, grasp-selective system will then be incorporated into the prosthesis to create a prototype involving the fully-embedded, shared control of the two systems. Finally, studies will be performed to understand whether or not the completed system improves an individual's grip compared to a standard multiple degree of freedom prosthesis.