HCC: Intelligent Interfaces to Empower People with Disabilities to Participate in the Information Society

对于患有严重瘫痪的人来说，沟通的能力通常仅限于使用小的头部，手部或眼睛运动的是/否反应。 为这些人提供参与信息社会的能力（例如，通过浏览网页、发布信息或给朋友发电子邮件），将极大地增强他们的情感健康，因为这可以减轻他们因身体瘫痪而导致的沮丧情绪。 此前，PI已经开发了基于摄像头的界面，部分实现了这些目标。 在目前的项目中，PI将建立在她以前的成就基础上，同时在几个方面向前推进。 她将开发一个鼠标替代界面，通过多摄像头计算机视觉系统检测到的用户头部运动来计算指针坐标。 她将探索用于手势检测的上下文感知方法，该方法可以将用户的交流动作与非自愿运动或社交互动区分开来。 她将设计和实现一个新的工具，允许用户开发自己的基于信号量的通信接口。 她将构建一个应用程序中介器，作为几个独立软件组件之间的中间人。 她将为各种常见应用程序开发创新的辅助软件，包括文本输入，网页浏览和动画（在后一种情况下，它使用户只能以有限的精度控制鼠标光标来创建和与3D对象交互）。 更广泛的影响：项目成果将对严重残疾的成人和儿童及其朋友、家人和照顾者的生活质量产生直接和积极的影响。 将开发的软件将在特殊护理设施（包括医院、严重身体残疾儿童学校和多发性硬化症和ALS患者长期护理设施）传播，也将通过免费下载在互联网上提供。 这项工作还将通过开发采用3D模型的技术，利用多个平移/倾斜/变焦相机实时跟踪面部特征，推进计算机视觉的最新技术。

For individuals who suffer form extreme paralysis, the ability to communicate is often limited to yes/no responses using small head, hand, or eye movements. Providing these people with the ability to participate in the Information Society (for example, by browsing the Web, posting messages, or emailing friends), would greatly enhance their emotional well being by alleviating the frustration caused by having an active mind trapped in a paralyzed body. Previously, the PI has developed camera-based interfaces that have afforded partial attainment of such goals. In the current project, the PI will build upon her prior achievements and push forward on several fronts concurrently. She will develop a mouse-replacement interface that computes pointer coordinates from the user's head movements, which are detected by a multi-camera computer vision system. She will explore context-aware approaches for gesture detection that can distinguish a user's communicative motions from involuntary movements or social interactions. She will design and implement a novel tool that allows users to develop their own semaphore-based communication interfaces. She will build an application mediator that serves as an intercessor between several separate software components. And she will develop innovative assistive software for a variety of common applications including text entry, web browsing, and animation (in the latter case, which enables users capable only of controlling a mouse cursor with limited precision to create and interact with 3D objects). Broader Impacts: Project outcomes will have a direct and positive impact on the quality of life of adults and children with severe disabilities, as well as their friends, families and caregivers. The software to be developed will be disseminated at special care facilities (including a hospital, schools for children with severe physical disabilities, and a long-term care facility for people with Multiple Sclerosis and ALS), and will also be available on the internet via free download. This work will also advance the state of the art in computer vision, through the development of technology that employs 3D models for real-time tracking of facial features with multiple pan/tilt/zoom cameras.