CAREER: Closing the Loop of Human-Machine Interactions via Skin-Like Multimodal Haptic Interfaces

职业：通过类肤多模态触觉界面闭合人机交互循环

• 批准号: 2238363
• 负责人: Shanshan Yao
• 金额: $ 50万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-03-01 至 2028-02-29
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2238363&HistoricalAwards=false
• 关键词: CAREER; Closing; Loop; Human; Machine

Haptic perception is essential for all human actions, which helps humans explore, understand, and interact with the surrounding environment. When interacting with smart machines, such as robots, prosthetics, computers, or even virtual environments, such haptic feedback is crucial for efficient human-machine interactions. Remarkable progress has been made in developing skin-like sensors for users to control machines and for machines to learn about their surrounding environment. However, skin-compatible haptic feedback from machines to users is largely unexplored. This project seeks to develop haptic interfaces that can produce haptic cues compatible with the sensing capabilities and mechanical properties of the skin. The proposed skin-like multimodal haptic interface represents a significant opportunity to enable bilateral machine control. The resulting closed-loop human-machine interface is the enabling technology that can impact a wide range of applications, including robotics, prosthetics, teleoperation, smart manufacturing, entertainment, medical/military training, and virtual reality (VR)/augmented reality (AR). The haptic interface can also benefit individuals with hearing and visual impairments by encoding and representing information using haptic feedback. The integrated research and educational activities will inspire and educate a diverse group of students in the fields of haptic devices and nanotechnology. Through team-based research activities, workshops, open house activities, and research mentoring, the project will strengthen students’ skillset development and advance their pursuit of STEM careers. The educational and outreach activities will engage underrepresented groups and students of all ages.The research objective of this CAREER proposal is to explore the interplay between functional materials, structural design, as well as integration strategies to realize haptic devices that can match the functional and mechanical characteristics of the skin for enhanced haptic sensations. Three research thrusts are proposed: (1) High-performance soft actuators to generate mechano-haptic sensations. (2) Robust heating/cooling devices to generate thermo-haptic sensations. (3) Integration of both modalities for skin-like multimodal haptic interfaces. The educational objective of this CAREER proposal is to disseminate research discoveries in wearable haptics and nanomaterials to promote diversity in STEM fields, and to equip the next generation workforce with skillsets to advance these emerging fields. The PI will conduct the following educational activities: (1) Mentor a multidisciplinary Vertically Integrated Project (VIP) team to strengthen students’ soft skills in addition to technical skills. (2) Develop a workshop for Mater Teachers to enhance teachers’ STEM knowledge. These emerging technologies will then be communicated more broadly to students when teachers integrate them into the curriculum. (3) Host research open houses and provide research opportunities for a diverse group of students to stimulate interest and increase diversity. The proposed work will yield new knowledge about the rational design of haptic interfaces to realize efficient feedback sensations through the skin. Completion of the proposed research will lay the ground for closed-loop human-machine interfaces and pave the way for applications involving interactions between users and machines.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

触觉感知对于所有人类行为至关重要，它有助于人类探索、理解周围环境并与之互动。当与机器人、假肢、计算机甚至虚拟环境等智能机器交互时，这种触觉反馈对于高效的人机交互至关重要。在开发类皮肤传感器方面已经取得了显着进展，这些传感器可供用户控制机器和机器了解周围环境。然而，从机器到用户的皮肤兼容触觉反馈在很大程度上尚未被探索。该项目旨在开发触觉界面，能够产生与皮肤的传感能力和机械特性兼容的触觉提示。所提出的类皮肤多模式触觉界面代表了实现双边机器控制的重要机会。由此产生的闭环人机界面是一种使能技术，可以影响广泛的应用，包括机器人、假肢、远程操作、智能制造、娱乐、医疗/军事训练和虚拟现实（VR）/增强现实（AR）。通过使用触觉反馈编码和表示信息，触觉界面还可以使有听力和视觉障碍的个人受益。综合研究和教育活动将激励和教育触觉设备和纳米技术领域的多元化学生。通过基于团队的研究活动、研讨会、开放日活动和研究指导，该项目将加强学生的技能发展并促进他们对 STEM 职业的追求。教育和外展活动将吸引代表性不足的群体和各个年龄段的学生。这项职业提案的研究目标是探索功能材料、结构设计以及集成策略之间的相互作用，以实现能够匹配皮肤功能和机械特性的触觉设备，从而增强触觉。提出了三个研究重点：（1）产生机械触觉的高性能软执行器。 (2) 坚固的加热/冷却装置，产生热触觉。 (3) 两种模式的集成，用于类皮肤多模式触觉界面。该职业提案的教育目标是传播可穿戴触觉和纳米材料的研究发现，以促进 STEM 领域的多样性，并为下一代劳动力配备推动这些新兴领域发展的技能。 PI将开展以下教育活动：（1）指导多学科垂直整合项目（VIP）团队，以加强学生除技术技能外的软技能。 (2) 为教师举办研讨会，以提升教师的 STEM 知识。当教师将这些新兴技术融入课程时，它们将更广泛地传播给学生。 (3) 举办研究开放日，为多元化的学生群体提供研究机会，以激发兴趣并增加多样性。拟议的工作将产生有关触觉界面合理设计的新知识，以通过皮肤实现有效的反馈感觉。拟议研究的完成将为闭环人机界面奠定基础，并为涉及用户和机器之间交互的应用铺平道路。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Bio-Inspired Untethered Robot-Sensor Platform for Minimally Invasive Biomedical Sensing

• DOI: 10.1021/acsami.3c13425
• 发表时间: 2023-12-05
• 期刊: ACS APPLIED MATERIALS & INTERFACES
• 影响因子: 9.5
• 作者: Li，Yizong;Halwah，Amro;Yao，Shanshan
• 通讯作者: Yao，Shanshan

Decoding silent speech commands from articulatory movements through soft magnetic skin and machine learning

通过软磁皮肤和机器学习从发音运动解码无声语音命令

• DOI: 10.1039/d3mh01062g
• 发表时间: 2023
• 期刊: Materials Horizons
• 影响因子: 13.3
• 作者: Dong, Penghao;Li, Yizong;Chen, Si;Grafstein, Justin T.;Khan, Irfaan;Yao, Shanshan
• 通讯作者: Yao, Shanshan