Collaborative proposal: A multimodal tactile sensor skin designed to reduce the cognitive burden on the user of a prosthetic hand

PI：桑托斯，Veronica J.和Posner，Jonathan D.建议编号：1264444 1264046智力优点：无论假肢手是一个简单的身体动力钩或先进的拟人设备，它只会是有用的，如果它提高了生活质量和直观的控制截肢者的愿望。如果假体对使用者造成太大的认知负担，则假体将被拒绝。一种方法是同时减少用户的认知负担并增强用户的功能，以专注于高级命令，而不是低级细节，由于不同的时间尺度和控制分辨率，考虑到人类和机器之间的“语言障碍”，低级细节可能令人沮丧地控制或甚至不可能控制。截肢者可以使用具有自主的局部反射算法的假肢，这些算法类似于在人类中观察到的短潜伏期抓握反射，甚至是对日常生活活动至关重要的基本行为构建模块。半自治系统获得其操作员信任的唯一途径是通过可靠的、依赖于上下文的性能。这种情境感知性能将需要关于假手与非结构化环境中的日常物体之间的有力交互的信息，而这些信息只能通过触摸来获得。大量和动态范围的触觉mechanoreceptors在人手（17000触觉传感器总数，2000在每个指尖）突出了丰富的多模态触觉反馈的重要性，把握和灵巧的操作。不幸的是，许多触觉传感器设计集中在检测法向力单独，这是必要的，但不足以可靠的人工把握。迫切需要的是一种多模态触觉传感器，它可以检测手指-物体相互作用的其他重要特征，如剪切力、振动和滑动方向。 该提案旨在通过检测，处理和利用创新的多模态触觉传感器皮肤来检测，处理和利用有关手指-物体交互的丰富实时信息，从而加强人工手可靠地执行自动化行为的能力。这种传感器系统是变革性的，因为它将减少截肢者的认知负担，并将为通过假手自动化复杂行为的范式转变进步提供基础，并通过感官反馈向用户提供有意识的触摸感知。该提案的长期研究目标是减少上肢假肢使用者的认知负担。研究目标1）设计、建模、制造和测试一种用于人工指尖的柔性、多模态触觉传感器皮肤系统;研究目标2）建立手指-物体交互和触觉传感器皮肤数据之间的函数关系，用于自主抓握控制算法;和研究目标3）将触觉传感器皮肤数据集成到抓握控制算法中，并评估减轻假肢使用者认知负担的有效性。拟议的转化研究可以增强用于非结构化，不安全，或限制进入的环境（假肢、康复、辅助、太空、水下、军事、救援、手术）。拟议的工作可在改善假肢和辅助装置最终用户的生活质量方面发挥关键作用。对假肢设备最终用户的具体好处包括：复杂假肢行为的自动化、丰富的人工感觉反馈以及用于监测用户安全性和舒适性的“智能承窝衬垫”。教育目标1）开发实际操作的教学模块，用于使用低成本材料向小学生教授传感器，教育目标2）利用传感器模块加强题为“传感器和控制”的本科生课程和题为“机器人”的研究生课程;和教育目标3）通过与传感器的开发，测试和应用相关的实习，促进跨学科的本科生研究机会。

PI: Santos, Veronica J. and Posner, Jonathan D.Proposal Number: 1264444 & 1264046Intellectual Merit: Whether a prosthetic hand is a simple body-powered hook or an advanced anthropomorphic device, it will only be useful and desirable to an amputee if it improves quality of life and is intuitive to control. A prosthesis will be rejected if it poses too great of a cognitive burden on the user. One way to simultaneously reduce the cognitive burden on the user and enhance the functionality of the user to focus on high-level commands as opposed to low-level details that may be frustrating to control or even impossible to control given the "language barrier" between human and machine because of different timescales and resolutions of control. Amputees could be empowered with prostheses having autonomous, local reflex algorithms akin to short latency grip reflexes observed in humans, and even suites of basic behavioral building blocks that are critical for activities of daily living. The only way for a semi-autonomous system to gain the trust of its operator is through reliable, context-dependent performance. Such context-aware performance will require information about forceful interactions between the prosthetic hand and everyday objects in unstructured environments that can only be obtained through touch. The great number and dynamic range of tactile mechanoreceptors in the human hand (17000 tactile sensors total, 2000 in each fingertip) highlight the importance of rich multimodal tactile feedback for grasp and dexterous manipulation. Unfortunately, many tactile sensor designs have focused on detection of normal forces alone, which are necessary but not sufficient for reliable artificial grasp. What is sorely needed is a multimodal tactile sensor that can detect additional important features of finger-object interactions such as shear force, vibration, and slip direction. This proposal aims to strengthen the ability of an artificial hand to perform automated behaviors reliably by detecting, processing, and utilizing rich, real-time information about finger-object interactions with an innovative multimodal tactile sensor skin. This sensor system is transformative because it will reduce the cognitive burden on an amputee and will provide a foundation for paradigm-shifting advancements for automating complex behaviors by artificial hands and providing a conscious perception of touch through sensory feedback to the user. The long-term research objective of this proposal is to reduce the cognitive burden on the user of an upper extremity prosthesis. The following contributions to artificial hand systems are proposed: Research Goal 1) Design, model, fabricate, and test a flexible, multimodal tactile sensor skin system for artificial fingertips using a multilayer microfluidic architecture; Research Goal 2) Establish functional relationships between finger-object interactions and tactile sensor skin data for use in autonomous grip control algorithms; and Research Goal 3) Integrate the tactile sensor skin data into grip control algorithms and evaluate effectiveness for reducing the cognitive burden on prosthesis users.Broader Impacts: The proposed translational research could enhance the functional capabilities of artificial, robotic manipulators intended for unstructured, unsafe, or limited-access environments (prosthetic, rehabilitative, assistive, space, underwater, military, rescue, surgery). The proposed work could play a critical role in improving the quality of life for end-users of prosthetic and assistive devices. Specific benefits to end-users of prosthetic devices include: automation of complex prosthesis behaviors, rich artificial sensory feedback, and "smart socket liners" for monitoring user safety and comfort.Contributions to elementary school, undergraduate, and graduate-level education are proposed: Education Goal 1) Develop hands-on instructional modules for teaching elementary school students about sensors using low-cost materials, and deploy them locally for the benefit of students underrepresented in science, technology, engineering, and mathematics fields; Education Goal 2) Enhance undergraduate-level course titled Sensors and Controls and graduate-level course titled "Robotics" with a sensors module; and Education Goal 3) Promote interdisciplinary undergraduate research opportunities via internships related to the development, testing, and application of sensors.