HCC: Small: Assistive Social Situational Awareness Aids for Individuals with Disabilities

PI在这个项目中的目标是实现迈向下一代社交辅助工具的巨大飞跃，这种辅助工具丰富了视觉障碍个人的生活社交体验。社会互动是人类体验的核心组成部分。与他人有效沟通的能力是职业成功和个人成就感的基本要求。但非语言线索(包括韵律、环境属性、交际者的外表和他们的身体动作)在社会互动中所传达的信息中占了相当重要的一部分。因此，美国有130多万合法失明的人(全世界有3700万人)，他们在社交方面的经验有限。这种“社会残疾”往往使他们与社会环境隔绝。现有的辅助技术专注于导航、阅读文本和访问日常电器以及计算机和互联网等问题，而很少或根本没有致力于实时获取社交和行为线索的工作。为视障用户提供对非语言交流线索的实时访问在情感计算、人类交流工程、行为建模、机器学习、人机交互、多模式界面、可用性工程、多媒体计算和辅助技术设计和开发等相关领域提出了根本挑战。作为迈向实用和可行的社交辅助解决方案的第一步，PI将专注于设计和开发用于二元(一对一)交互的社会情景感知辅助原型，重点是基于头/脸的非语言线索。这项研究将通过以下具体目标完成：(1)设计和开发一个二元人际中介界面；(2)非语言交流线索的提取和理解；(3)用于传递高带宽社会行为数据的视觉-触觉替代；以及(4)在代表现实世界条件的二元互动场景中对社交辅助原型系统的评估。该项目利用了团队成员之间的智力协同效应，他们是以下领域的专家：以人为中心的多媒体计算、人机界面和机器智能(Panchanathan，计算机科学)；辅助技术设计和可用性工程(Hedgpeth，残疾资源中心)；以及人类沟通建模和社会行为分析(Ramirez，人类沟通)。这项工作将建立在该团队过去在开发辅助技术方面的成功基础上，这些技术已经为视障人士设计、原型、部署和测试。项目成果将通过亚利桑那州立大学残疾资源中心和亚利桑那州盲人和视觉障碍人士中心进行评估。广泛影响：该项目将开创为视觉障碍人士开发下一代社会辅助工具的先河，从而对他们的生活产生重大影响。为此目的进行的研究将促进各组成部分学科内部和各组成部分交汇处的计算思维的发展，这些学科是社会行为计算(通过引入计算分析和评估一般人类交流动态的新方法，特别是通过对二元互动中的社会行为的计算分析和评估)、人机界面(通过设计提供高带宽社会数据的新接口)、机器智能(通过研究引出不同层次互动语义的算法)和辅助技术/可用性(通过开发和评估社会辅助原型)。开发的概念和技术还将为其他残疾人士提供技术途径，如自闭症、痴呆症，以及(在最一般意义上)社会的很大一部分人。制定的方法将提供丰富的数据和信息，这些数据和信息将向公众公布，以促进和促进各组成部分学科的进一步研究。

The PI's goal in this project is to enable a quantum leap towards next-generation social assistive aids that enrich the lived social experiences of individuals with visual impairments. Social interaction is a central component of the human experience. The ability to communicate effectively with fellow individuals is a fundamental necessity for professional success as well as personal fulfillment. But nonverbal cues (including prosody, environment attributes, the appearance of communicators and their physical movements) account for a substantial and important part of the information conveyed during social interactions. As a consequence, the more than 1.3 million individuals in the United States (and 37 million worldwide) who are legally blind have only a limited experience of social interaction. This "social disability" often isolates them from their social environments. Existing assistive technologies are focused on problems such as navigation, reading text, and access to everyday appliances as well as to computers and the Internet, whereas little or no work has been devoted to real-time accessibility to social and behavioral cues. Providing real-time access to nonverbal communication cues to visually impaired users poses fundamental challenges in several related fields including affective computing, human communication engineering, behavioral modeling, machine learning, human-machine interaction, multimodal interfaces, usability engineering, multimedia computing, and assistive technology design and developmentAs a first step towards practical and viable social assistive solutions, the PI will focus in this project on the design and development of a social situational awareness assistive prototype for dyadic (one-on-one) interactions, with an emphasis on head/face-based nonverbal cues. The research will be accomplished through the following specific objectives: (1) Design and development of a dyadic interpersonal mediation interface; (2) Extraction and understanding of nonverbal communication cues; (3) Visuo-haptic sensory substitution for delivering high-bandwidth socio-behavioral data; and (4) Evaluation of the social assistive prototype system in dyadic interaction scenarios representing real-world conditions. The project draws upon intellectual synergies among the team members, who are experts in human-centered multimedia computing, human-computer interfaces and machine intelligence (Panchanathan, Computer Science); assistive technology design and usability engineering (Hedgpeth, Disability Resources Center); and human communication modeling and socio-behavioral analysis (Ramirez, Human Communication). The work will build on the team's past successes in developing assistive technologies that have been designed, prototyped, deployed and tested for individuals who are visually impaired. Project outcomes will be evaluated through the Arizona State University Disability Resource Center and the Arizona Center for the Blind and Visually Impaired.Broader Impacts: This project will pioneer the development of next-generation social assistive aids for individuals with visual impairments and thus will have a significant impact on their lives. The research to these ends will result in the advancement of computational thinking within and at the confluence of the component disciplines, namely socio-behavioral computing (through the introduction of novel methodologies for computational analysis and evaluation of human communication dynamics in general and social behavior in dyadic interactions, specifically), human-computer interfaces (through the design of novel interfaces that deliver high-bandwidth social data), machine intelligence (through the study of algorithms that elicit various levels of interaction semantics) and assistive technology/usability (through the development and evaluation of social assistive prototypes). The concepts and technologies developed will also provide pathways to technologies for individuals with other disabilities, such as autism, dementia, and (in the most general sense) a very large portion of society. The methodologies developed will provide a wealth of data and information that will be made publicly accessible to promote and catalyze further research in the component disciplines.