权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CRII: RI: RUI: Representations for multi-timescale scene dynamics in webcam video streams

CRII：RI：RUI：网络摄像头视频流中多时间尺度场景动态的表示

基本信息

批准号：
2105372
负责人：
Scott Wehrwein
金额：
$ 17.47万
依托单位：
Western Washington University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-06-01 至 2025-05-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2105372&HistoricalAwards=false
关键词：
CRII RI RUI Representations multi

项目摘要

Thousands of webcams positioned all over the world are constantly streaming live HD views of a wide variety of scenes. Thanks to their longevity, these streams capture many visually interesting phenomena that occur not only in real time but over longer scales of time. Although computer vision algorithms have been developed for video and scene understanding, the scale - both in data volume and in time - of these webcam streams require new approaches. This project will develop representations that facilitate techniques to automatically extract insights about short and long-term changes in fixed-view webcam scenes. These techniques can be applied to a variety of areas including security, surveillance, and a diverse array of monitoring use cases including climate science, ecology, and development. Undergraduate students at a teaching-oriented public Primarily Undergraduate Institution will gain valuable skills and experience by being directly involved in all aspects of the research.Existing video understanding and analysis techniques focus almost exclusively on real-time dynamics, such as human actions, while little attention has been paid to longer-term phenomena. The objective of this research is to develop representations for years-long video streams that facilitate analysis and understanding of phenomena at a wide range of time scales. To serve this purpose, the representations need to be compact, allow for full reconstruction of the input, and organize scene content by timescale. To achieve this, a dataset of video streams will be collected and scene-specific models will be trained to encode video frames into compact latent vectors. A novel regularization scheme will impose order in the latent space, arranging the representation of the scene's content by how quickly it varies. The research will enable new ways to detect, monitor, and understand phenomena that occur over weeks, months, and years. Existing video understanding applications such as anomaly detection, action recognition, and video prediction will also benefit by operating in a compact and organized self-supervised latent space. Finally, the research will further progress towards understanding how to encode temporal redundancy in videos and yield insights about general video understanding by helping to determine what is possible with the absence of camera motion and an abundance of observations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

遍布世界各地的成千上万的网络摄像头不断地直播各种场景的高清视图。由于它们的寿命长，这些流捕捉到许多视觉上有趣的现象，这些现象不仅发生在实时，而且发生在更长的时间尺度上。尽管计算机视觉算法已经被开发出来用于视频和场景的理解，但这些网络摄像头流的规模——无论是在数据量还是在时间上——都需要新的方法。该项目将开发表示，促进技术自动提取关于固定视图网络摄像头场景的短期和长期变化的见解。这些技术可以应用于各种领域，包括安全、监视和各种监测用例，包括气候科学、生态和发展。在以教学为导向的公立本科院校，本科生将通过直接参与研究的各个方面获得宝贵的技能和经验。现有的视频理解和分析技术几乎只关注实时动态，如人类行为，而很少关注长期现象。本研究的目的是为长达数年的视频流开发表示，以促进在大范围时间尺度上对现象的分析和理解。为了达到这个目的，表示需要紧凑，允许输入的完全重建，并按时间尺度组织场景内容。为了实现这一目标，将收集视频流数据集，并训练特定场景的模型，将视频帧编码为紧凑的潜在向量。一种新的正则化方案将在潜在空间中施加秩序，根据场景内容的变化速度来安排其表示。这项研究将提供新的方法来检测、监测和理解持续数周、数月和数年的现象。现有的视频理解应用，如异常检测、动作识别和视频预测，也将受益于在一个紧凑和有组织的自监督潜在空间中运行。最后，该研究将进一步了解如何在视频中编码时间冗余，并通过帮助确定在没有摄像机运动和大量观察的情况下可能发生的情况，从而产生关于一般视频理解的见解。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。