iccv iccv2013 iccv2013-351 iccv2013-351-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: David Eigen, Dilip Krishnan, Rob Fergus
Abstract: Photographs taken through a window are often compromised by dirt or rain present on the window surface. Common cases of this include pictures taken from inside a vehicle, or outdoor security cameras mounted inside a protective enclosure. At capture time, defocus can be used to remove the artifacts, but this relies on achieving a shallow depth-of-field and placement of the camera close to the window. Instead, we present a post-capture image processing solution that can remove localized rain and dirt artifacts from a single image. We collect a dataset of clean/corrupted image pairs which are then used to train a specialized form of convolutional neural network. This learns how to map corrupted image patches to clean ones, implicitly capturing the characteristic appearance of dirt and water droplets in natural images. Our models demonstrate effective removal of dirt and rain in outdoor test conditions.
[1] P. Barnum, S. Narasimhan, and K. Takeo. Analysis of rain and snow in frequency space. IJCV, 86(2):256–274, 2010. 2
[2] H. Burger, C. Schuler, and S. Harmeling. Image denoising: Can plain neural networks compete with BM3D? In CVPR, 2012. 2, 3, 4, 5
[3] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Image denoising with block-matching and 3D filtering. In Proc. SPIE Electronic Imaging, 2006. 1, 4
[4] B. Dong, H. Ji, J. Li, Z. Shen, and Y. Xu. Wavelet frame based blind image inpainting. Applied and Comp ’l Harmonic Analysis, 32(2):268–279, 2011. 2
[5] M. Elad and M. Aharon. Image denoising via learned dictionaries and sparse representation. In CVPR, 2006. 2
[6] C. Farabet, B. Martini, B. Corda, P. Akselrod, E. Culurciello, and Y. LeCun. NeuFlow: A runtime reconfigurable dataflow processor for vision. In IEEE Workshop on Embedded Computer Vision (ECV at CVPR), 2011. 8
[7] K. Garg and S. Nayar. Detection and removal of rain from videos. In CVPR, pages 528–535, 2004. 2 Bottom: Output of our algorithm.
[8] J. Gu, R. Ramamoorthi, P. Belhumeur, and S. Nayar. Dirty Glass: Rendering
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24] Contamination on Transparent Surfaces. In Eurographics, Jun 2007. 4 J. Gu, R. Ramamoorthi, P. Belhumeur, and S. Nayar. Removing Image Artifacts Due to Dirty Camera Lenses and Thin Occluders. SIGGRAPH Asia, Dec 2009. 2, 4 V. Jain and S. Seung. Natural image denoising with convolutional networks. In NIPS, 2008. 2 J. Jancsary, S. Nowozin, and C. Rother. Loss-specific training ofnon-parametric image restoration models: A new state of the art. In ECCV, 2012. 2 Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proc. IEEE, 86(1 1):2278–2324, Nov 1998. 2 A. Levin and B. Nadler. Natural image denoising: Optimality and inherent bounds. In CVPR, 201 1. 1 B. A. Olshausen and D. J. Field. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37(23):331 1–3325, 1997. 2 S. Paris and F. Durand. A fast approximation of the bilateral filter using a signal processing approach. In ECCV, pages IV: 568–580, 2006. 4 J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli. Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans Image Processing, 12(1 1): 1338–135 1, November 2003. 2 M. Roser and A. Geiger. Video-based raindrop detection for improved image registration. In ICCV Workshop on Video-Oriented Object and Event Classification, Kyoto, Japan, September 2009. 2 C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In CVPR, 1998. 4 R. G. Willson, M. W. Maimone, A. E. Johnson, and L. M. Scherr. An optical model for image artifacts produced by dust particles on lenses. In i-SAIRAS, volume 1, 2005. 2 J. Xie, L. Xu, and E. Chen. Image denoising and inpainting with deep neural networks. In NIPS, 2012. 2 S. Zhang and E. Salari. Image denosing using a neural network based non-linear filter in the wavelet domain. In ICASSP, 2005. 2 C. Zhou and S. Lin. Removal of image artifacts due to sensor dust. In CVPR, 2007. 2 S. C. Zhu and D. Mumford. Prior learning and gibbs reaction-diffusion. PAMI, 19(1 1): 1236–1250, 1997. 2 D. Zoran and Y. Weiss. From learning models ofnatural image patches to whole image restoration. In ICCV, 2011. 2 640