HCC: Small: Head Activated Technology for Off-the-Shelf Mobile Devices

移动计算设备通过使用多点触摸表面、麦克风和摄像头等新的传感器技术，提供了越来越丰富的人机交互机会。这些进步为用户提供了更丰富的交互模式，并激发了新的、新颖的以人为中心的界面。在新的输入方式中，基于触摸的交互是最先进和应用最广泛的，而语音正在获得越来越多的关注。然而，即使最近在计算机视觉、深度和运动传感技术方面取得了突破，成像方式仍然是移动设备中最不发达的。然而，基于图像的传感(A)将为普通用户群体提供更丰富的互动机会，(B)对于发现基于触摸的界面非常难以使用的残疾人来说，具有特殊的前景。本项目致力于开发用于现成移动设备的头部激活技术(HAT-MD)。HAT-MD系统将采用下一代成像技术，该技术将包括在广泛的移动设备中。高级算法将检测和跟踪用户的头部和面部特征，并将特定的运动映射到特定的界面控件和应用程序操作。HAT-MD最初是针对身体残疾的个人，将(A)将台式计算机上经常使用的头部控制交互技术扩展到移动平台，以及(B)扩大可以使用的头部和面部动作集。项目交付成果包括：(A)HAT-MD算法：下一代成像传感器将用于检测、跟踪和3D重建。头部位置和面部特征将被检测、跟踪并映射到界面控件。计算将在移动设备和云计算服务之间分配。(B)HAT-MD应用程序：将开发头部激活界面，以应对肢体残疾人士在使用现代移动平台时面临的挑战。(C)HAT-MD评价：严格的人体对象研究将评价HAT-MD一般方法以及所开发的具体接口和应用程序的有效性和易用性。算法将以当前最先进的检测、跟踪和重建方法为基准。更广泛的影响：HAT-MD项目提供的更多可访问性将为人们与移动设备交互创造新的机会，特别是对于目前对此类设备的独立控制有限的肢体残疾人士。该项目还将包括残疾人在研究、开发和评估阶段的直接参与，以便将智力发展集中在一个真正有用的方向，以补充更广泛的影响。

Mobile computing devices offer increasingly rich human-computer interaction opportunities through the use of new sensor technology such as multi-touch surfaces, microphones, and cameras. These advances provide users with richer sets of interaction modalities and motivate new and novel human-centric interfaces. Among the new input modalities, touch-based interaction is the most advanced and widely deployed, while speech is gaining increased attention. However, even with recent breakthroughs in computer vision and depth and motion sensing technology, the imaging modality remains the least developed for mobile devices. And yet image-based sensing (a) would permit richer interaction opportunities for a general user population and (b) holds special promise for individuals with disabilities who find touch-based interfaces very challenging to use. This project focuses on the development of Head-Activated Technology for off-the- shelf Mobile Devices (HAT-MD). HAT-MD systems will employ the next-generation imaging technology that will be included in a wide array of mobile devices. Advanced algorithms will detect and track user head and facial features, and map specific movements to specific interface controls and application actions. HAT-MD is initially targeted towards individuals with physical disabilities, and will both (a) extend to mobile platforms the head control interaction techniques that are often employed on desktop computers and (b) broaden the set of head and face movements that can be employed.Project deliverables include: (a) HAT-MD Algorithms: Next-generation imaging sensors will be used for detection, tracking, and 3D reconstruction. Head position and facial features will be detected, tracked, and mapped to interface controls. Computation will be distributed between the mobile devices and cloud computing services. (b) HAT-MD Applications: Head activated interfaces will be developed to address the challenges that individuals with physical disabilities have when using contemporary mobile platforms. (c) HAT-MD Evaluations: Rigorous human subject studies will evaluate the effectiveness and ease-of-use of the general HAT-MD methodology as well as specific interface and applications that are developed. Algorithms will be benchmarked to current state-of-the-art detection, tracking, and reconstruction methods.Broader Impacts: The increased accessibility provided by the HAT-MD project will create new opportunities for people to interact with mobile devices, especially for individuals with physical disabilities who currently have limited independent control of such devices. The project will also feature direct involvement by individuals with disabilities at the research, development, and evaluation phases in order to focus the intellectual development in a truly useful direction that will complement the broader impact.