Visualizing the Passage of Time with Video Temporal Pyramids
Visualizing the Passage of Time with Video Temporal Pyramids
复制标题
DOI：
10.1109/tvcg.2022.3209454
复制
发表时间：
2022-08
影响因子：
5.2
通讯作者：
Melissa E. Swift;Wyatt Ayers;Sophie Pallanck;Scott Wehrwein
中科院分区：
文献类型：
--
作者：
Melissa E. Swift;Wyatt Ayers;Sophie Pallanck;Scott Wehrwein
通过几个月或几年的观察，我们能了解到什么？长时间录制的视频将在多个时间尺度上描绘有趣的现象，但识别和观看它们是一个挑战。视频太长，无法完整观看，有些东西太慢，无法实时体验，例如冰川退缩或从夏季逐渐转向秋季。时间流逝视频是总结长视频和可视化慢时间尺度的常用方法。然而，时移被限制到单个选择的时间频率，并且由于混叠而经常出现闪烁。此外，时移视频的长度直接与其时间分辨率相关，这需要在这两个方面之间进行权衡。在本文中，我们提出了视频时间金字塔，一种技术，解决了这些限制，并扩大了可视化的时间流逝的可能性。受计算机视觉中空间图像金字塔的启发，我们开发了一种在时域中构建视频金字塔的算法。视频时间金字塔的每个级别都可视化不同的时间尺度;例如，来自每月时间尺度的视频通常适合可视化季节变化，而来自一分钟时间尺度的视频最适合可视化日出或天空中云层的运动。为了帮助探索不同的金字塔级别，我们还提出了一个视频频谱图，以可视化整个金字塔的活动量，提供场景动态的整体概述，以及跨时间和时间尺度探索和发现现象的能力。为了展示我们的方法，我们从10个户外场景中构建了视频时间金字塔，每个场景包含数月或数年的数据。我们将视频时间金字塔层与天真的时间流逝进行比较，发现我们的金字塔可以无锯齿地查看长期变化。我们还表明，视频频谱图有利于探索和发现的现象，通过启用概述和细节为重点的观点。
What can we learn about a scene by watching it for months or years? A video recorded over a long timespan will depict interesting phenomena at multiple timescales, but identifying and viewing them presents a challenge. The video is too long to watch in full, and some things are too slow to experience in real-time, such as glacial retreat or the gradual shift from summer to fall. Timelapse videography is a common approach to summarizing long videos and visualizing slow timescales. However, a timelapse is limited to a single chosen temporal frequency, and often appears flickery due to aliasing. Also, the length of the timelapse video is directly tied to its temporal resolution, which necessitates tradeoffs between those two facets. In this paper, we propose Video Temporal Pyramids, a technique that addresses these limitations and expands the possibilities for visualizing the passage of time. Inspired by spatial image pyramids from computer vision, we developed an algorithm that builds video pyramids in the temporal domain. Each level of a Video Temporal Pyramid visualizes a different timescale; for instance, videos from the monthly timescale are usually good for visualizing seasonal changes, while videos from the one-minute timescale are best for visualizing sunrise or the movement of clouds across the sky. To help explore the different pyramid levels, we also propose a Video Spectrogram to visualize the amount of activity across the entire pyramid, providing a holistic overview of the scene dynamics and the ability to explore and discover phenomena across time and timescales. To demonstrate our approach, we have built Video Temporal Pyramids from ten outdoor scenes, each containing months or years of data. We compare Video Temporal Pyramid layers to naive timelapse and find that our pyramids enable alias-free viewing of longer-term changes. We also demonstrate that the Video Spectrogram facilitates exploration and discovery of phenomena across pyramid levels, by enabling both overview and detail-focused perspectives.