CAREER: Closing the Loop of Human-Machine Interactions via Skin-Like Multimodal Haptic Interfaces

职业：通过类肤多模态触觉界面闭合人机交互循环

• 批准号: 2238363
• 负责人: Shanshan Yao
• 金额: $ 50万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-03-01 至 2028-02-29
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2238363&HistoricalAwards=false
• 关键词: CAREER; Closing; Loop; Human; Machine

Haptic perception is essential for all human actions, which helps humans explore, understand, and interact with the surrounding environment. When interacting with smart machines, such as robots, prosthetics, computers, or even virtual environments, such haptic feedback is crucial for efficient human-machine interactions. Remarkable progress has been made in developing skin-like sensors for users to control machines and for machines to learn about their surrounding environment. However, skin-compatible haptic feedback from machines to users is largely unexplored. This project seeks to develop haptic interfaces that can produce haptic cues compatible with the sensing capabilities and mechanical properties of the skin. The proposed skin-like multimodal haptic interface represents a significant opportunity to enable bilateral machine control. The resulting closed-loop human-machine interface is the enabling technology that can impact a wide range of applications, including robotics, prosthetics, teleoperation, smart manufacturing, entertainment, medical/military training, and virtual reality (VR)/augmented reality (AR). The haptic interface can also benefit individuals with hearing and visual impairments by encoding and representing information using haptic feedback. The integrated research and educational activities will inspire and educate a diverse group of students in the fields of haptic devices and nanotechnology. Through team-based research activities, workshops, open house activities, and research mentoring, the project will strengthen students’ skillset development and advance their pursuit of STEM careers. The educational and outreach activities will engage underrepresented groups and students of all ages.The research objective of this CAREER proposal is to explore the interplay between functional materials, structural design, as well as integration strategies to realize haptic devices that can match the functional and mechanical characteristics of the skin for enhanced haptic sensations. Three research thrusts are proposed: (1) High-performance soft actuators to generate mechano-haptic sensations. (2) Robust heating/cooling devices to generate thermo-haptic sensations. (3) Integration of both modalities for skin-like multimodal haptic interfaces. The educational objective of this CAREER proposal is to disseminate research discoveries in wearable haptics and nanomaterials to promote diversity in STEM fields, and to equip the next generation workforce with skillsets to advance these emerging fields. The PI will conduct the following educational activities: (1) Mentor a multidisciplinary Vertically Integrated Project (VIP) team to strengthen students’ soft skills in addition to technical skills. (2) Develop a workshop for Mater Teachers to enhance teachers’ STEM knowledge. These emerging technologies will then be communicated more broadly to students when teachers integrate them into the curriculum. (3) Host research open houses and provide research opportunities for a diverse group of students to stimulate interest and increase diversity. The proposed work will yield new knowledge about the rational design of haptic interfaces to realize efficient feedback sensations through the skin. Completion of the proposed research will lay the ground for closed-loop human-machine interfaces and pave the way for applications involving interactions between users and machines.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

触觉对于人类的所有行为都是必不可少的，它有助于人类探索、理解周围环境并与之互动。当与机器人、假肢、计算机甚至虚拟环境等智能机器互动时，这种触觉反馈对于高效的人机互动至关重要。在开发皮肤状传感器方面取得了显著进展，用户可以用来控制机器，机器也可以了解周围环境。然而，从机器到用户的皮肤兼容触觉反馈在很大程度上还没有被探索。该项目旨在开发能够产生与皮肤的感知能力和机械性能相兼容的触觉提示的触觉界面。拟议的皮肤状多模式触觉界面是实现双边机器控制的重要机会。由此产生的闭环人机界面是一种使能技术，可以影响广泛的应用，包括机器人、假肢、遥操作、智能制造、娱乐、医疗/军事训练和虚拟现实(VR)/增强现实(AR)。触觉界面还可以通过使用触觉反馈对信息进行编码和表示，从而使听力和视觉障碍的个人受益。综合研究和教育活动将在触觉设备和纳米技术领域启发和教育不同群体的学生。通过基于团队的研究活动、工作坊、开放参观活动和研究指导，该项目将加强学生的技能发展，促进他们对STEM职业生涯的追求。这项职业计划的研究目标是探索功能材料、结构设计以及整合策略之间的相互作用，以实现与皮肤的功能和机械特征相匹配的触觉设备，以增强触觉感觉。提出了三个研究方向：(1)产生机械触觉的高性能软驱动器。(2)坚固的加热/冷却设备，可产生热触觉。(3)两种模式的整合，形成类似皮肤的多模式触觉界面。这项职业计划的教育目标是传播可穿戴触觉和纳米材料方面的研究发现，以促进STEM领域的多样性，并为下一代劳动力配备技能，以推动这些新兴领域的发展。PI将开展以下教育活动：(1)指导一个多学科垂直整合项目(VIP)团队，以加强学生的软技能和技术技能。(2)为教师开办工作坊，提高教师的STEM知识。当教师将这些新兴技术整合到课程中时，这些新兴技术将被更广泛地传播给学生。(3)举办研究性开放日，为不同的学生群体提供研究机会，以激发兴趣，增加多样性。这项拟议的工作将为触觉界面的合理设计提供新的知识，以实现通过皮肤的高效反馈感觉。这项拟议研究的完成将为闭环人机界面奠定基础，并为涉及用户和机器之间交互的应用程序铺平道路。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Bio-Inspired Untethered Robot-Sensor Platform for Minimally Invasive Biomedical Sensing

• DOI: 10.1021/acsami.3c13425
• 发表时间: 2023-12-05
• 期刊: ACS APPLIED MATERIALS & INTERFACES
• 影响因子: 9.5
• 作者: Li，Yizong;Halwah，Amro;Yao，Shanshan
• 通讯作者: Yao，Shanshan

Decoding silent speech commands from articulatory movements through soft magnetic skin and machine learning

通过软磁皮肤和机器学习从发音运动解码无声语音命令

• DOI: 10.1039/d3mh01062g
• 发表时间: 2023
• 期刊: Materials Horizons
• 影响因子: 13.3
• 作者: Dong, Penghao;Li, Yizong;Chen, Si;Grafstein, Justin T.;Khan, Irfaan;Yao, Shanshan
• 通讯作者: Yao, Shanshan