CSR: CHS: Large: Wearable Cognitive Assistance

这项研究探索了一种新型计算系统的深层技术挑战，该系统将可穿戴设备（如谷歌Glass）与基于云的处理集成在一起，以指导用户逐步完成复杂的任务。虽然很容易描述，但要使这一概念成为现实，必须克服计算机系统、计算机视觉和人机交互方面的许多挑战。人类的认知能力是一项了不起的实时处理能力。它涉及对多个传感器流输入的实时分析输出的综合。辅助系统通过计算密集型处理来增强人类的认知，这种处理反应迅速，符合人类认知工作流程的内部循环。就其最一般的形式而言，认知辅助是一个非常广泛和雄心勃勃的概念，可以应用于日常生活的几乎所有方面。作为一项开创性的努力，本研究更狭隘地关注于需要专业知识和/或技能的定义良好的任务的用户辅助，并且任务状态和任务相关动作完全可以由计算机视觉算法访问。这项研究分为四大重点。第一个推力将低级移动计算和云计算问题（如资源管理、网络延迟、放置、供应、可伸缩性和负载平衡）与其他任务的任务中心焦点分离开来。第二个重点是解决可穿戴认知辅助挑战所必需的计算机视觉研究。对于本研究中处理的各种任务，视觉是主要的感知方式，但验证实验将包括音频和位置等其他感知方式的概念验证。第三个重点是任务描述、跟踪、排序和用户指导。它的目标是创建一套可应用于广泛任务的可推广原则和工具。将任务协助与任务需求和用户能力相匹配将是这一推动力的组成部分。第四个推力涉及前三个推力的持续集成研究，并将其应用于对一系列日益复杂和困难的任务的端到端验证。这一推动力涉及与行业伙伴的密切合作。这项研究将通过在计算机系统（包括移动计算、云计算、虚拟机、操作系统、无线网络和传感器网络）、视觉技术（包括计算机视觉和机器学习）和人机交互的交叉领域产生科学见解、算法、系统设计、实现技术和实验验证来推进计算机科学。更广泛地说，社会将受益于可穿戴式认知应用，如医疗保健培训、工业故障排除和消费品组装等领域。从教育的角度来看，这项研究提供了许多独特的机会来训练研究生和本科生如何从广泛的跨学科观点来解决问题。

This research explores the deep technical challenges of a new class of computing systems that integrate a wearable device (such as Google Glass) with cloud-based processing to guide a user step by step through a complex task. Although easy to describe, many challenges in computer systems, computer vision and human-computer interaction must be overcome for this concept to become reality. Human cognition is a remarkable feat of real-time processing. It involves the synthesis of outputs from real-time analytics on multiple sensor stream inputs. An assistive system amplifies human cognition with compute-intensive processing that is so responsive that it fits into the inner loop of the human cognitive workflow. In its most general form, cognitive assistance is a very broad and ambitious concept that could be applied to virtually all facets of everyday life. As a pioneering effort, this research is more narrowly focused on user assistance for well-defined tasks that require specialized knowledge and/or skills, and for which task state and task-relevant actions are fully accessible to computer vision algorithms. The research is organized into four broad thrusts. The first thrust decouples and cleanly separates low-level mobile computing and cloud computing issues such as resource management, network latency, placement, provisioning, scalability, and load balancing from the task-centric foci of the other tasks. The second thrust focuses on the computer vision research necessary to address the challenges of wearable cognitive assistance. Vision is the dominant sensing modality for the kinds of tasks addressed in this research, but the validation experiments will include proof-of-concept use of other sensing modalities such as audio and location. The third thrust focuses on task description, tracking, sequencing and user guidance. Its goal is to create a set of generalizable principles and tools that can be applied to a wide range of tasks. Matching task assistance to task demands and user capabilities will be integral to this thrust. The fourth thrust involves continuous integration of research from the first three thrusts and applies it towards end-to-end validation on a series of tasks of increasing sophistication and difficulty. This thrust involves close collaboration with industry partners. This research will advance computer science by producing scientific insights, algorithms, system designs, implementation techniques, and experimental validations at the intersection of computer systems (including mobile computing, cloud computing, virtual machines, operating systems, wireless networking, and sensor networks), vision technologies (including computer vision and machine learning), and human-computer interaction. More broadly, society will benefit from wearable cognitive application in areas such as health care training, industrial troubleshooting and consumer product assembly. From an educational viewpoint, this research offers many unique opportunities to train graduate and undergraduate students on how to approach problems from a broad cross-disciplinary viewpoint.