CPS: Synergy: Collaborative Research: SensEye: An Architecture for Ubiquitous, Real-Time Visual Context Ssensing and Inference

对个人眼睛和视野的持续实时跟踪对于理解人类如何感知和与物理世界互动至关重要。这项工作通过设计一种创新的范例来推进网络物理系统的技术和工程，该范例涉及下一代计算眼镜，该眼镜与用户的手机交互，提供实时视觉情境感知和推理的能力。本研究整合了低功耗嵌入式系统、图像表示、图像处理和机器学习以及移动传感和推理等领域的新研究，以推动CPS应用中连续传感的发展。这项活动解决了几个基本的研究挑战，包括：1)设计新颖的、高度集成的计算眼镜，用于跟踪眼睛运动、用户的视野和头部运动模式，所有这些都是实时的；2)统一的压缩信号处理框架，优化感知和估计，同时使设备能够根据应用的需要重新定位以执行广泛的任务；3)设计新颖的实时视觉上下文感知系统，从压缩数据表示中提取感兴趣的高级上下文；4)一层智能层，它将从计算眼镜提取的上下文与从移动电话获得的上下文结合在一起，以提高能效和传感精度。这项技术可以彻底改变包括交通、医疗保健、行为科学和市场研究在内的一系列学科。持续监测个人的眼睛和视野可以检测到危险行为，如驾驶时昏昏欲睡，精神健康问题，如精神分裂症，成瘾行为和药物滥用，神经疾病进展，头部受伤，以及其他。该研究通过设计原型平台和实时传感器处理算法，并通过开源场所使这些系统可供更广泛的使用，为此类应用提供了基础。该项目的推广活动包括在面向高中生的科学博览会上展示该设备，以及将该平台整合到本科和研究生课程中。

Continuous real-time tracking of the eye and field-of-view of an individual is profoundly important to understanding how humans perceive and interact with the physical world. This work advances both the technology and engineering of cyber-physical systems by designing an innovative paradigm involving next-generation computational eyeglasses that interact with a user's mobile phone to provide the capability for real-time visual context sensing and inference. This research integrates novel research into low-power embedded systems, image representation, image processing and machine learning, and mobile sensing and inference, to advance the state-of-art in continuous sensing for CPS applications. The activity addresses several fundamental research challenges including: 1) design of novel, highly integrated, computational eyeglasses for tracking eye movements, the visual field of a user, and head movement patterns, all in real-time; 2) a unified compressive signal processing framework that optimizes sensing and estimation, while enabling re-targeting of the device to perform a broad range of tasks depending on the needs of an application; 3) design of a novel real-time visual context sensing system that extracts high-level contexts of interest from compressed data representations; and 4) a layer of intelligence that combines contexts extracted from the computational eyeglass together with contexts obtained from the mobile phone to improve energy-efficiency and sensing accuracy.This technology can revolutionize a range of disciplines including transportation, healthcare, behavioral science and market research. Continuous monitoring of the eye and field-of-view of an individual can enable detection of hazardous behaviors such as drowsiness while driving, mental health issues such as schizophrenia, addictive behavior and substance abuse, neurological disease progression, head injuries, and others. The research provides the foundations for such applications through the design of a prototype platform together with real-time sensor processing algorithms, and making these systems available through open source venues for broader use. Outreach for this project includes demonstrations of the device at science fairs for high-school students, and integration of the platform into undergraduate and graduate courses.