Analysis of Rain and Snow in Frequency Space
Analysis of Rain and Snow in Frequency Space
复制标题
DOI：
10.1007/s11263-008-0200-2
复制
发表时间：
2010-01-01
影响因子：
19.5
通讯作者：
Kanade, Takeo
中科院分区：
文献类型：
--
作者：
Barnum, Peter C.;Narasimhan, Srinivasa;Kanade, Takeo
雨雪等动态天气会导致视频中复杂的时空强度波动。这种波动可能会对依赖小图像特征进行跟踪、目标检测和识别的视觉系统造成不利影响。虽然这些影响在空间和时间上看起来是混沌的，但我们证明了动态天气在频率空间具有可预测的全球效应。为此，我们首先开发了图像空间中单个雨或雪条纹的形状和外观的模型。即使有一个准确的外观模型也很难检测到单个条纹，因此我们将条纹模型与雨雪的统计特征相结合，在频率空间创建了动态天气的整体效果模型。然后将该模型应用于视频，并首先在频率空间检测雨雪条纹，然后将检测结果传输到图像空间。一旦探测到，降雨或降雪量可以减少或增加。我们证明，我们的频率分析在去除动态天气和特征提取性能方面比以前的基于像素或基于面片的方法具有更高的准确性。我们还表明，与以前的技术不同，我们的方法对于场景和摄像机运动的视频都是有效的。
Dynamic weather such as rain and snow causes complex spatio-temporal intensity fluctuations in videos. Such fluctuations can adversely impact vision systems that rely on small image features for tracking, object detection and recognition. While these effects appear to be chaotic in space and time, we show that dynamic weather has a predictable global effect in frequency space. For this, we first develop a model of the shape and appearance of a single rain or snow streak in image space. Detecting individual streaks is difficult even with an accurate appearance model, so we combine the streak model with the statistical characteristics of rain and snow to create a model of the overall effect of dynamic weather in frequency space. Our model is then fit to a video and is used to detect rain or snow streaks first in frequency space, and the detection result is then transferred to image space. Once detected, the amount of rain or snow can be reduced or increased. We demonstrate that our frequency analysis allows for greater accuracy in the removal of dynamic weather and in the performance of feature extraction than previous pixel-based or patch-based methods. We also show that unlike previous techniques, our approach is effective for videos with both scene and camera motions.