CSR: CHS: Large: Wearable Cognitive Assistance

这项研究探讨了一类新的计算系统的深层技术挑战，这些系统将可穿戴设备（如谷歌眼镜）与基于云的处理集成在一起，以指导用户逐步完成复杂的任务。虽然很容易描述，但要使这一概念成为现实，必须克服计算机系统、计算机视觉和人机交互方面的许多挑战。人类的认知是实时处理的一项非凡壮举。它涉及对多个传感器流输入的实时分析的输出的合成。辅助系统通过计算密集型处理来放大人类认知，这种处理是如此敏感，以至于它适合人类认知工作流程的内部循环。认知辅助是一个非常广泛和雄心勃勃的概念，可以应用于日常生活的几乎所有方面。 作为一项开创性的努力，这项研究更狭隘地集中在用户协助定义明确的任务，需要专业知识和/或技能，任务状态和任务相关的行动是完全可以访问计算机视觉算法。 这项研究分为四个主要方面。第一个推力将低级移动的计算和云计算问题（如资源管理、网络延迟、布局、供应、可扩展性和负载平衡）与其他任务的以任务为中心的焦点分离开来。第二个重点是解决可穿戴认知辅助挑战所需的计算机视觉研究。视觉是本研究中所涉及的各种任务的主要感知方式，但验证实验将包括使用其他感知方式（如音频和位置）的概念验证。第三个重点是任务说明、跟踪、排序和用户指导。它的目标是创建一套可推广的原则和工具，可应用于广泛的任务。使任务援助与任务需求和用户能力相匹配将是这一努力的组成部分。第四个重点是持续整合前三个重点的研究，并将其应用于一系列日益复杂和困难的任务的端到端验证。这一目标涉及与行业合作伙伴的密切合作。这项研究将通过在计算机系统（包括移动的计算，云计算，虚拟机，操作系统，无线网络和传感器网络），视觉技术（包括计算机视觉和机器学习）和人机交互的交叉点产生科学见解，算法，系统设计，实现技术和实验验证来推进计算机科学。 更广泛地说，社会将受益于医疗保健培训、工业故障排除和消费品组装等领域的可穿戴认知应用。从教育的角度来看，这项研究提供了许多独特的机会，培养研究生和本科生如何从广泛的跨学科的观点来处理问题。

This research explores the deep technical challenges of a new class of computing systems that integrate a wearable device (such as Google Glass) with cloud-based processing to guide a user step by step through a complex task. Although easy to describe, many challenges in computer systems, computer vision and human-computer interaction must be overcome for this concept to become reality. Human cognition is a remarkable feat of real-time processing. It involves the synthesis of outputs from real-time analytics on multiple sensor stream inputs. An assistive system amplifies human cognition with compute-intensive processing that is so responsive that it fits into the inner loop of the human cognitive workflow. In its most general form, cognitive assistance is a very broad and ambitious concept that could be applied to virtually all facets of everyday life. As a pioneering effort, this research is more narrowly focused on user assistance for well-defined tasks that require specialized knowledge and/or skills, and for which task state and task-relevant actions are fully accessible to computer vision algorithms. The research is organized into four broad thrusts. The first thrust decouples and cleanly separates low-level mobile computing and cloud computing issues such as resource management, network latency, placement, provisioning, scalability, and load balancing from the task-centric foci of the other tasks. The second thrust focuses on the computer vision research necessary to address the challenges of wearable cognitive assistance. Vision is the dominant sensing modality for the kinds of tasks addressed in this research, but the validation experiments will include proof-of-concept use of other sensing modalities such as audio and location. The third thrust focuses on task description, tracking, sequencing and user guidance. Its goal is to create a set of generalizable principles and tools that can be applied to a wide range of tasks. Matching task assistance to task demands and user capabilities will be integral to this thrust. The fourth thrust involves continuous integration of research from the first three thrusts and applies it towards end-to-end validation on a series of tasks of increasing sophistication and difficulty. This thrust involves close collaboration with industry partners. This research will advance computer science by producing scientific insights, algorithms, system designs, implementation techniques, and experimental validations at the intersection of computer systems (including mobile computing, cloud computing, virtual machines, operating systems, wireless networking, and sensor networks), vision technologies (including computer vision and machine learning), and human-computer interaction. More broadly, society will benefit from wearable cognitive application in areas such as health care training, industrial troubleshooting and consumer product assembly. From an educational viewpoint, this research offers many unique opportunities to train graduate and undergraduate students on how to approach problems from a broad cross-disciplinary viewpoint.