CPS: Synergy: Collaborative Research: SensEye: An Architecture for Ubiquitous, Real-Time Visual Context Sensing and Inference

持续实时跟踪个人的眼睛和视野对于理解人类如何感知和与物理世界互动非常重要。 这项工作通过设计一种创新的范式，涉及下一代计算眼镜，与用户的移动的手机进行交互，提供实时视觉上下文感知和推理的能力，从而推进了网络物理系统的技术和工程。本研究将新的研究整合到低功耗嵌入式系统、图像表示、图像处理和机器学习以及移动的传感和推理中，以推进CPS应用中连续传感的最新技术。该活动解决了几个基本的研究挑战，包括：1）设计新颖的，高度集成的，计算眼镜，用于跟踪眼睛运动，用户的视野和头部运动模式，所有这些都是实时的; 2）优化感测和估计的统一压缩信号处理框架，同时使得能够根据应用的需要来重定向设备以执行广泛的任务; 3）设计了一种新颖的实时视觉上下文感知系统，该系统从压缩数据表示中提取感兴趣的高级上下文;以及4）智能层，其将从计算流提取的上下文与从移动的电话获得的上下文结合在一起，以提高能量效率和感测精度。医疗保健、行为科学和市场研究。对个体的眼睛和视野的连续监测可以使得能够检测危险行为，例如驾驶时的困倦，精神健康问题，例如精神分裂症，成瘾行为和药物滥用，神经疾病进展，头部损伤等。该研究通过设计原型平台以及实时传感器处理算法为此类应用提供了基础，并通过开源场所使这些系统可用于更广泛的用途。该项目的推广包括在高中学生的科学博览会上展示该设备，以及将该平台整合到本科和研究生课程中。

Continuous real-time tracking of the eye and field-of-view of an individual is profoundly important to understanding how humans perceive and interact with the physical world. This work advances both the technology and engineering of cyber-physical systems by designing an innovative paradigm involving next-generation computational eyeglasses that interact with a user's mobile phone to provide the capability for real-time visual context sensing and inference. This research integrates novel research into low-power embedded systems, image representation, image processing and machine learning, and mobile sensing and inference, to advance the state-of-art in continuous sensing for CPS applications. The activity addresses several fundamental research challenges including: 1) design of novel, highly integrated, computational eyeglasses for tracking eye movements, the visual field of a user, and head movement patterns, all in real-time; 2) a unified compressive signal processing framework that optimizes sensing and estimation, while enabling re-targeting of the device to perform a broad range of tasks depending on the needs of an application; 3) design of a novel real-time visual context sensing system that extracts high-level contexts of interest from compressed data representations; and 4) a layer of intelligence that combines contexts extracted from the computational eyeglass together with contexts obtained from the mobile phone to improve energy-efficiency and sensing accuracy.This technology can revolutionize a range of disciplines including transportation, healthcare, behavioral science and market research. Continuous monitoring of the eye and field-of-view of an individual can enable detection of hazardous behaviors such as drowsiness while driving, mental health issues such as schizophrenia, addictive behavior and substance abuse, neurological disease progression, head injuries, and others. The research provides the foundations for such applications through the design of a prototype platform together with real-time sensor processing algorithms, and making these systems available through open source venues for broader use. Outreach for this project includes demonstrations of the device at science fairs for high-school students, and integration of the platform into undergraduate and graduate courses.