I-CORPS: First Person Visual Analytics

迫切需要对对治疗相关变化敏感的自闭症谱系障碍(ASD)患者的社会行为进行直接、连续和客观的测量。通过测量行为变化来跟踪进展对于确定治疗是否有效或需要修改至关重要。直接观察措施，如现场或基于视频的评分，耗费时间和资源，而且几乎不可能在诊所和家庭实施，因为那里提供的治疗越来越多。目前还没有标准的、客观的和可靠的关键社会行为测量方法，可以用于临床环境中评估ASD患者的治疗结果。该项目的目标是开发视觉分析工具，以实现从可穿戴摄像头捕获的视频中自动检测和量化感兴趣的社交行为，i-Corps团队将这项任务称为第一人称视觉(FPV)。在之前的工作中，该团队开发了一种方法，通过分析成年人戴的一副商用眼镜拍摄的视频，自动检测儿童与成年社交伙伴进行眼神接触的时刻。这副眼镜在鼻子上方的桥上嵌入了摄像头。通过识别和定位视频中儿童的头部和面部，并结合头部姿势分析包含眼睛的图像区域，所提出的算法可以确定儿童的凝视是否指向成人的眼睛。所提出的方法可以生成帧级别的眼神接触预测，并且还可以用于检测眼神接触事件，由此可以得出持续时间的度量。如果开发得当，拟议的测量方法可能使临床提供者能够扩大他们的干预努力，有可能改善数千名自闭症儿童及其家庭的生活质量。更广泛地说，从人体佩戴的摄像头(第一人称视频)捕获的视频的分析工具将提供新的机会来模拟和分析人类行为，创建个性化的视觉体验记录，并改进对广泛的心理和身体健康状况的治疗。该团队将在人力资源培训和产品营销领域进行更多的客户发现，有可能将我们分析的重点从凝视行为扩展到与这些背景相关的其他社交行为，如面部表情和手势。该团队还希望开发硬件的原型(宽视场可穿戴式相机系统)，并将我们团队开发的分析功能(用于自动检测眼神接触、对对象的注意力、凝视转移的算法)整合到系统中。拟议的原型还将包括将自动化分析的结果可视化并报告给用户的选项。

There is an urgent need for direct, continuous, and objective measures of social behavior in individuals with Autism Spectrum Disorder (ASD) that are sensitive to treatment-related change. Tracking progress by measuring changes in behavior is crucial to determine whether the treatment is working or needs to be modified. Direct observational measures, such as live or video-based scoring, are time and resource intensive, and all but impossible to implement in clinics and homes, where treatments are increasingly delivered. There are currently no standardized, objective, and reliable measures of key social behaviors that can be used in clinical settings to assess treatment outcomes in individuals with ASD. The goal of this project is to develop visual analytics tools to enable the automatic detection and quantification of social behaviors of interest from video captured by a wearable camera, a task this I-Corps team refers to as First Person Vision (FPV). In previous work, this team developed a method for automatically detecting moments when a child makes eye contact with an adult social partner by analyzing video recorded by a pair of commercially available glasses worn by the adult which had a camera embedded in the bridge over the nose. By identifying and localizing the child's head and face in the video and analyzing the image regions containing the eyes in conjunction with the head pose, the proposed algorithm can determine whether the child's gaze is directed towards the adult's eyes. The proposed approach can generate predictions for eye contact at the frame level, and can also be used to detect eye contact events from which measures of duration can be derived. If properly developed, the proposed measurement approach may enable clinical providers to scale their intervention efforts, with the potential of improving the quality of life for thousands of children with autism and their families. More broadly, analytics tools for video captured from body-worn cameras (first person video) will provide new opportunities to model and analyze human behavior, create personalized records of visual experiences, and improve the treatment of a broad range of mental and physical health conditions. This team will conduct additional customer discovery in the areas of human resources training and product marketing, with the potential of expanding the focus of our analytics from gaze behavior to other social behaviors relevant to these contexts, such as facial expressions and gestures. The team also expects to develop a prototype of the hardware (wearable camera system with wide field of view) and incorporate analytic capabilities developed by our group (algorithms for automated detection of eye contact, attention to objects, gaze shifts) into the system. the proposed prototype will also include options for visualizing and reporting the results of the automated analysis back to the user.