Collaborative proposal: A multimodal tactile sensor skin designed to reduce the cognitive burden on the user of a prosthetic hand

PI: Santos, Veronica J.和Posner， Jonathan d .提案编号：1264444 & 1264046智力优点：无论假肢是一个简单的身体驱动的钩子还是一个先进的拟人装置，只有当它能提高生活质量并易于控制时，它才会对截肢者有用和可取。如果假体给使用者带来太大的认知负担，就会被拒绝。一种同时减少用户认知负担和增强用户功能的方法是专注于高级命令，而不是低级细节，低级细节可能令人沮丧，甚至无法控制，因为人与机器之间存在“语言障碍”，因为控制的时间尺度和分辨率不同。截肢者可以被赋予具有自主的局部反射算法的假肢，类似于在人类身上观察到的短延迟握力反射，甚至是对日常生活活动至关重要的基本行为构建模块。半自治系统获得操作人员信任的唯一途径是通过可靠的、与上下文相关的性能。这种情境感知性能将需要关于假手与非结构化环境中日常物体之间的有力相互作用的信息，这些信息只能通过触摸获得。人手中触觉机械感受器的数量和动态范围（总共17000个触觉传感器，每个指尖2000个）突出了丰富的多模态触觉反馈对抓握和灵巧操作的重要性。不幸的是，许多触觉传感器的设计都只关注在法向力的检测，这对于可靠的人工抓取来说是必要的，但还不够。迫切需要的是一种多模态触觉传感器，它可以检测手指与物体相互作用的其他重要特征，如剪切力、振动和滑动方向。该方案旨在通过一种创新的多模态触觉传感器皮肤，检测、处理和利用手指与物体交互的丰富实时信息，从而增强人工手部可靠地执行自动化行为的能力。这种传感器系统具有变革性，因为它将减轻截肢者的认知负担，并将为人工手自动化复杂行为和通过感官反馈向用户提供有意识的触觉提供范式转变的基础。本课题的长期研究目标是减轻上肢义肢使用者的认知负担。研究目标1)利用多层微流体结构设计、建模、制造和测试一种柔性、多模态的人造指尖触觉传感器皮肤系统；研究目标2)建立手指-物体交互和触觉传感器皮肤数据之间的函数关系，用于自主抓握控制算法；研究目标3)将触觉传感器皮肤数据整合到握把控制算法中，评估减轻义肢使用者认知负担的有效性。更广泛的影响：拟议的转化研究可以增强用于非结构化、不安全或有限访问环境（假肢、康复、辅助、太空、水下、军事、救援、手术）的人工、机器人操纵器的功能。拟议的工作可能在改善假肢和辅助装置最终用户的生活质量方面发挥关键作用。假肢设备对最终用户的具体好处包括：复杂假肢行为的自动化，丰富的人工感官反馈，以及用于监测用户安全和舒适度的“智能插座衬垫”。建议对小学、本科和研究生教育的贡献：教育目标1)开发动手教学模块，用于教授小学生使用低成本材料的传感器，并在当地部署，以使科学、技术、工程和数学领域代表性不足的学生受益；教育目标2)加强本科阶段的“传感器与控制”课程和研究生阶段的“机器人”课程，增加传感器模块；3)通过与传感器的开发、测试和应用相关的实习，促进跨学科的本科生研究机会。

PI: Santos, Veronica J. and Posner, Jonathan D.Proposal Number: 1264444 & 1264046Intellectual Merit: Whether a prosthetic hand is a simple body-powered hook or an advanced anthropomorphic device, it will only be useful and desirable to an amputee if it improves quality of life and is intuitive to control. A prosthesis will be rejected if it poses too great of a cognitive burden on the user. One way to simultaneously reduce the cognitive burden on the user and enhance the functionality of the user to focus on high-level commands as opposed to low-level details that may be frustrating to control or even impossible to control given the "language barrier" between human and machine because of different timescales and resolutions of control. Amputees could be empowered with prostheses having autonomous, local reflex algorithms akin to short latency grip reflexes observed in humans, and even suites of basic behavioral building blocks that are critical for activities of daily living. The only way for a semi-autonomous system to gain the trust of its operator is through reliable, context-dependent performance. Such context-aware performance will require information about forceful interactions between the prosthetic hand and everyday objects in unstructured environments that can only be obtained through touch. The great number and dynamic range of tactile mechanoreceptors in the human hand (17000 tactile sensors total, 2000 in each fingertip) highlight the importance of rich multimodal tactile feedback for grasp and dexterous manipulation. Unfortunately, many tactile sensor designs have focused on detection of normal forces alone, which are necessary but not sufficient for reliable artificial grasp. What is sorely needed is a multimodal tactile sensor that can detect additional important features of finger-object interactions such as shear force, vibration, and slip direction. This proposal aims to strengthen the ability of an artificial hand to perform automated behaviors reliably by detecting, processing, and utilizing rich, real-time information about finger-object interactions with an innovative multimodal tactile sensor skin. This sensor system is transformative because it will reduce the cognitive burden on an amputee and will provide a foundation for paradigm-shifting advancements for automating complex behaviors by artificial hands and providing a conscious perception of touch through sensory feedback to the user. The long-term research objective of this proposal is to reduce the cognitive burden on the user of an upper extremity prosthesis. The following contributions to artificial hand systems are proposed: Research Goal 1) Design, model, fabricate, and test a flexible, multimodal tactile sensor skin system for artificial fingertips using a multilayer microfluidic architecture; Research Goal 2) Establish functional relationships between finger-object interactions and tactile sensor skin data for use in autonomous grip control algorithms; and Research Goal 3) Integrate the tactile sensor skin data into grip control algorithms and evaluate effectiveness for reducing the cognitive burden on prosthesis users.Broader Impacts: The proposed translational research could enhance the functional capabilities of artificial, robotic manipulators intended for unstructured, unsafe, or limited-access environments (prosthetic, rehabilitative, assistive, space, underwater, military, rescue, surgery). The proposed work could play a critical role in improving the quality of life for end-users of prosthetic and assistive devices. Specific benefits to end-users of prosthetic devices include: automation of complex prosthesis behaviors, rich artificial sensory feedback, and "smart socket liners" for monitoring user safety and comfort.Contributions to elementary school, undergraduate, and graduate-level education are proposed: Education Goal 1) Develop hands-on instructional modules for teaching elementary school students about sensors using low-cost materials, and deploy them locally for the benefit of students underrepresented in science, technology, engineering, and mathematics fields; Education Goal 2) Enhance undergraduate-level course titled Sensors and Controls and graduate-level course titled "Robotics" with a sensors module; and Education Goal 3) Promote interdisciplinary undergraduate research opportunities via internships related to the development, testing, and application of sensors.