Progressive Rain Removal via a Recurrent Convolutional Network for Real Rain Videos

Abstract

Rain removal in videos is a problem that has attracted tremendous interest of researchers within the field of deep learning. Although deep-learning-based rain removal methods outperform large number of conventional vision methods, some technical issues that need to be resolved remain. In this article, we propose a new deep learning method for video rain removal based on recurrent neural network (RNN) architecture. Pseudo groundtruth was generated from real rainy video sequence by temporal filtering for supervised learning. Instead of focusing on various shapes of rain streaks similar to conventional methods, in this article, we focused on the changing behaviors of rain streaks. To accomplish this, images of progressive rain streaks were generated from the real rain videos and are sequentially fed to the network in a decreasing rain order. Multiple images with different amounts of rain streaks were used as RNN inputs to more efficiently identify rain streaks and then remove them. Experimental results demonstrate that our method is suitable for a wide range of rainy images. Moreover, experiments performed on both real-world and synthetic images demonstrate that our proposed method can achieve competitive results in comparison with the benchmarked and conventional approaches for rain streak removal from images.