CSR: CHS: Large: Wearable Cognitive Assistance

这项研究探索了一种新型计算系统的深层技术挑战，这种系统将可穿戴设备(如谷歌眼镜)与基于云的处理相结合，以引导用户逐步完成复杂的任务。虽然很容易描述，但要使这一概念成为现实，必须克服计算机系统、计算机视觉和人机交互方面的许多挑战。人类认知是实时处理的一项非凡壮举。它涉及对多个传感器流输入的实时分析的输出的合成。辅助系统通过计算密集型处理放大人类的认知，这种处理反应非常灵敏，适合人类认知工作流程的内环。在其最一般的形式下，认知辅助是一个非常广泛和雄心勃勃的概念，可以应用于日常生活的几乎所有方面。作为一项开创性的工作，这项研究更狭隘地关注用户对定义明确的任务的帮助，这些任务需要专门的知识和/或技能，并且计算机视觉算法完全可以访问这些任务的任务状态和与任务相关的动作。这项研究被组织成四大主题。第一个推力将低级别移动计算和云计算问题(如资源管理、网络延迟、放置、配置、可扩展性和负载平衡)与其他任务的以任务为中心的重点完全分开。第二个重点集中在应对可穿戴式认知辅助挑战所需的计算机视觉研究上。视觉是本研究中涉及的这类任务的主要感知方式，但验证实验将包括对其他感知方式的概念验证使用，如音频和位置。第三个重点是任务描述、跟踪、排序和用户指导。其目标是创建一套可广泛应用于各种任务的概括性原则和工具。将任务协助与任务需求和用户能力相匹配将是实现这一目标不可或缺的一部分。第四个推动力涉及持续整合前三个推力的研究，并将其应用于对一系列日益复杂和困难的任务进行端到端验证。这一努力涉及与行业合作伙伴的密切合作。这项研究将通过在计算机系统(包括移动计算、云计算、虚拟机、操作系统、无线网络和传感器网络)、视觉技术(包括计算机视觉和机器学习)和人机交互的交叉领域产生科学见解、算法、系统设计、实现技术和实验验证来推动计算机科学的发展。更广泛地说，在医疗培训、工业故障排除和消费品组装等领域，可穿戴式认知应用将使社会受益。从教育的角度来看，这项研究提供了许多独特的机会来训练研究生和本科生如何从广泛的跨学科观点来处理问题。

This research explores the deep technical challenges of a new class of computing systems that integrate a wearable device (such as Google Glass) with cloud-based processing to guide a user step by step through a complex task. Although easy to describe, many challenges in computer systems, computer vision and human-computer interaction must be overcome for this concept to become reality. Human cognition is a remarkable feat of real-time processing. It involves the synthesis of outputs from real-time analytics on multiple sensor stream inputs. An assistive system amplifies human cognition with compute-intensive processing that is so responsive that it fits into the inner loop of the human cognitive workflow. In its most general form, cognitive assistance is a very broad and ambitious concept that could be applied to virtually all facets of everyday life. As a pioneering effort, this research is more narrowly focused on user assistance for well-defined tasks that require specialized knowledge and/or skills, and for which task state and task-relevant actions are fully accessible to computer vision algorithms. The research is organized into four broad thrusts. The first thrust decouples and cleanly separates low-level mobile computing and cloud computing issues such as resource management, network latency, placement, provisioning, scalability, and load balancing from the task-centric foci of the other tasks. The second thrust focuses on the computer vision research necessary to address the challenges of wearable cognitive assistance. Vision is the dominant sensing modality for the kinds of tasks addressed in this research, but the validation experiments will include proof-of-concept use of other sensing modalities such as audio and location. The third thrust focuses on task description, tracking, sequencing and user guidance. Its goal is to create a set of generalizable principles and tools that can be applied to a wide range of tasks. Matching task assistance to task demands and user capabilities will be integral to this thrust. The fourth thrust involves continuous integration of research from the first three thrusts and applies it towards end-to-end validation on a series of tasks of increasing sophistication and difficulty. This thrust involves close collaboration with industry partners. This research will advance computer science by producing scientific insights, algorithms, system designs, implementation techniques, and experimental validations at the intersection of computer systems (including mobile computing, cloud computing, virtual machines, operating systems, wireless networking, and sensor networks), vision technologies (including computer vision and machine learning), and human-computer interaction. More broadly, society will benefit from wearable cognitive application in areas such as health care training, industrial troubleshooting and consumer product assembly. From an educational viewpoint, this research offers many unique opportunities to train graduate and undergraduate students on how to approach problems from a broad cross-disciplinary viewpoint.