jmlr jmlr2010 jmlr2010-50 jmlr2010-50-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Valero Laparra, Juan Gutiérrez, Gustavo Camps-Valls, Jesús Malo
Abstract: A successful class of image denoising methods is based on Bayesian approaches working in wavelet representations. The performance of these methods improves when relations among the local frequency coefficients are explicitly included. However, in these techniques, analytical estimates can be obtained only for particular combinations of analytical models of signal and noise, thus precluding its straightforward extension to deal with other arbitrary noise sources. In this paper, we propose an alternative non-explicit way to take into account the relations among natural image wavelet coefficients for denoising: we use support vector regression (SVR) in the wavelet domain to enforce these relations in the estimated signal. Since relations among the coefficients are specific to the signal, the regularization property of SVR is exploited to remove the noise, which does not share this feature. The specific signal relations are encoded in an anisotropic kernel obtained from mutual information measures computed on a representative image database. In the proposed scheme, training considers minimizing the Kullback-Leibler divergence (KLD) between the estimated and actual probability functions (or histograms) of signal and noise in order to enforce similarity up to the higher (computationally estimable) order. Due to its non-parametric nature, the method can eventually cope with different noise sources without the need of an explicit re-formulation, as it is strictly necessary under parametric Bayesian formalisms. Results under several noise levels and noise sources show that: (1) the proposed method outperforms conventional wavelet methods that assume coefficient independence, (2) it is similar to state-of-the-art methods that do explicitly include these relations when the noise source is Gaussian, and (3) it gives better numerical and visual performance when more complex, realistic noise sources are considered. Therefore, the proposed machine learning approach can be seen as a mor
H.C. Andrews and B.R. Hunt. Digital Image Restoration. Prentice Hall, NY, 1977. M. Banham and A. Katsaggelos. Digital image restoration. IEEE Signal Processing Magazine, 14: 24–41, 1997. A. Barducci and I. Pippi. Analysis and rejection of systematic disturbances in hyperspectral remotely sensed images of the Earth. Applied Optics, 40:1464–1477, 2001. A. J. Bell and T. J. Sejnowski. The ‘independent components’ of natural scenes are edge filters. Vision Research, 37(23):3327–3338, 1997. M. Bertero, T.A. Poggio, and V. Torre. Ill-posed problems in early vision. Proceedings of the IEEE, 76(8):869–889, 1988. R. W. Buccigrossi and E. P. Simoncelli. Image compression via joint statistical characterization in the wavelet domain. IEEE Transactions on Image Processing, 8(12):1688–1701, 1999. P. J. Burt and E. H. Adelson The Laplacian Pyramid as a compact image code. IEEE Transactions on Communications, 31(4):532–540, 1983. G. Camps-Valls, E. Soria-Olivas, J. P´ rez-Ruixo, A. Art´ s-Rodr´guez, F. P´ rez-Cruz, and e e ı e A. Figueiras-Vidal. A profile-dependent kernel-based regression for cyclosporine concentration prediction. In Neural Information Processing Systems (NIPS) – Workshop on New Directions in Kernel-Based Learning Methods, Vancouver, Canada, December 2001. G. Camps-Valls, J. Guti´ rrez, G. G´ mez, and J. Malo. On the suitable domain for SVM training in e o image coding. Journal of Machine Learning Research, 9:49–66, 2008. A Chalimourda, B Sch¨ lkopf, and Alex J. Smola. Experimentally optimal ν in support vector o regression for different noise models and parameter settings. Neural Networks, 17(1):127–141, 2004. 899 ´ L APARRA , G UTI E RREZ , C AMPS -VALLS AND M ALO C. Chih Chang and Chih J. Lin. libSVM: A Library for Support Vector Machines, (http://www.csie.ntu.edu.tw/˜cjlin/libsvm/), 2001. H. Cheng, J.W. Tian, J. Liu, and Q.Z. Yu. Wavelet domain image denoising via SVR. IEE Electronics Letters, 40, 2004. V. Cherkassky. Practical selection of SVM parameters and noise estimation for SVM regression. Neural Networks, 17(1):113–126, 2004. V. Cherkassky and Y. Ma. Comparison of model selection for regression. Neural Comput., 15(7): 1691–1714, 2003. R.J. Clarke. Transform Coding of Images. Academic Press, New York, 1985. R. R. Coifman and D. L. Donoho. Translation-invariant de-noising. In A. Antoniadis and G. Oppenheim, editors, Wavelets and Statistics. Lecture Notes in Statistics, volume 103, pages 125–150. Springer, Berlin, Department of Statistics, 1995. T.M. Cover and J.A. Tomas. Elements of Information Theory. John Wiley & Sons, New York, 1991. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Image denoising by sparse 3D transformdomain collaborative filtering. IEEE Transactions on Image Processing, 16(8):2080–2095, 2007. D. L. Donoho and I. M. Johnstone. Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association, 90:1200–1224, 1995. D. Field. Relations between the Statistics of Natural Images and the Response Properties of Cortical Cells. Journal of the Optical Society of America A, 4(12):2379–2394, 1987. M. Figueiredo and R. Nowak. Wavelet-based image estimation: an empirical Bayes approach using Jeffrey’s noninformative prior. IEEE Transactions on Image Processing, 10(9):1322–1331, 2001. W. T. Freeman and E. H. Adelson. The design and use of steerable filters. IEEE Pattern Analysis and Machine Intelligence, 13(9):891–906, 1991. G. G´ mez, G. Camps-Valls, J. Guti´ rrez, and J. Malo. Perceptual adaptive insensitivity for support o e vector machine image coding. IEEE Transactions on Neural Networks, 16(6):1574–1581, 2005. B. Goossens, and Pizurica, A. and Philips, W. Removal of correlated noise by modeling the signal of interest in the wavelet domain. IEEE Transactions on Image Processing, 18(6):1153–1165, 2009. J. Guti´ rrez, F. Ferri, and J. Malo. Regularization operators for natural images based on nonlinear e perception models. IEEE Transactions on Image Processing, 15(1):189–200, 2006. T.-M. Huang and V. Kecman. Bias term b in SVMs again. In 12th European Symposium on Artificial Neural Network, ESANN 2004, pages 441–448, Bruges, Belgium, April 2004. A. Hyv¨ rinen. Sparse code shrinkage: denoising of nongaussian data by maximum likelihood a estimation. Neural Computation, 11(7):1739–1768, 1999. 900 I MAGE D ENOISING WITH K ERNELS BASED ON NATURAL I MAGE R ELATIONS A. Hyvarinen, J. Karhunen, and E. Oja. Independent Component Analysis. John Wiley & Sons, New York, 2001. T. Jebara, R. Kondor, and A. Howard. Probability product kernels. Journal of Machine Learning Research, 5:819–844, 2004. D. Kai Tick Chow and T. Lee. Image approximation and smoothing by support vector regression. In International Joint Conference on Neural Networks, IJCNN’01., volume 4, pages 2427–2432, Washington, DC, USA, 2001. V. Kecman, T. Huang, and M. Vogt. Iterative single data algorithm for training kernel machines from huge data sets: Theory. In Performance, Support Vector Machines: Theory and Applications, Springer-Verlag, Studies in Fuzziness and Soft Computing, p 255–274, 2004. C. Kervrann and J. Boulanger. Local adaptivity to variable smoothness for exemplar-based image denoising and representation. Intl Journal of Computer Vision, 16(2):349–366, Feb 2007. N. Kingsbury. Rotation-invariant local feature matching with complex wavelets. In Proc. European Conference on Signal Processing (EUSIPCO), Florence, Italy, 2006. J. T. Kwok and I. W. Tsang. Linear dependency between ε and the input noise in ε-Support Vector Regression. IEEE Transactions on Neural Networks, pages 544–553, May 2003. V. Laparra, J. Mu˜ oz-Mar´, and J. Malo. Divisive Normalization Image Quality Metric Revisited. n ı Accepted for publication in Journal of the Optical Society of America A, 2010. R. Michael Lewis and V. Torczon. A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. on Optimization, 12(4):1075–1089, 2002. J. Liu and P. Moulin. Information-theoretic analysis of interscale and intrascale dependencies between image wavelet coefficients. IEEE Transactions on Image Processing, 10:1647–1658, 2001. J. Malo, I. Epifanio, R. Navarro, and E. Simoncelli. Non-linear image representation for efficient perceptual coding. IEEE Transactions on Image Processing, 15(1):68–80, 2006. J. Malo and J. Guti´ rrez. V1 non-linear properties emerge from local-to-global non-linear ICA. e Network: Computation in Neural Systems, 17(1):85–102, 2006. J. Mercer. Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London, CCIX(A456):215–228, May 1905. P. Mouroulis, R. O. Green, and T. G. Chrien. Design of pushbroom imaging spectrometers for optimum recovery of spectroscopic and spatial information. Applied Optics, 39:2210–2220, 2000. A. Navia-V´ zquez, and F. P´ rez-Cruz, and A. Art´ s-Rodr´guez, and A. Figueiras-Vidal. Weighted a e e ı least squares training of support vector classifiers leading to compact and adaptive schemes IEEE Transactions on Neural Networks, 12:1047–1059, 2001. 901 ´ L APARRA , G UTI E RREZ , C AMPS -VALLS AND M ALO B. A. Olshausen and D. J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381:607–609, 1996. F. P´ rez-Cruz, A. Navia-V´ zquez, P. Alarc´ n-Diana, and A. Art´ s-Rodr´guez. An IRWLS procedure e a o e ı for SVR. In European Signal Processing Conference (EUSIPCO), Tampere, Finland, Sept 2000. J. Platt. Fast training of support vector machines using sequential minimal optimization. In B. Sch¨ lkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods — Supo port Vector Learning, pages 185–208, Cambridge, MA, 1999. MIT Press. J. Portilla and E. P. Simoncelli. A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal on Computer Vision, 40(1):49–71, 2000. J. Portilla, V. Strela, M. Wainwright, and E. Simoncelli. Image denoising using a scale mixture of Gaussians in the wavelet domain. IEEE Transactions on Image Processing, 12(11):1338–1351, 2003. L. Siwei and E. Simoncelli. Statistical modeling of images with fields of GSMs. In Proc. NIPS06’. MIT Press, May 2007. E. Simoncelli. Bayesian denoising of visual images in the wavelet domain. In Bayesian Inference in Wavelet Based Models, pages 291–308. Springer-Verlag, New York, 1999. E. Simoncelli and W. Freeman. The steerable pyramid: A flexible architecture for multi-scale derivative computation. In Proc 2nd IEEE Int’l Conference on Image Processing, 1995. E. P. Simoncelli. Statistical models for images: Compression, restoration and synthesis. In 31st Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, 1997. A. J. Smola and B. Sch¨ lkopf. A tutorial on support vector regression. Statistics and Computing, o 14:199–222, 2004. H. Stark and J. Woods. Probability, Random Processes, and Estimation Theory for Engineers. Prentice Hall, NJ, 1994. H. Takeda, S. Farsiu, and P. Milanfar. Kernel regression for image processing and reconstruction. IEEE Transactions on Image Processing, 16(2):349–366, Feb 2007. D. S. Taubman and M. W. Marcellin. JPEG2000: Image Compression Fundamentals, Standards and Practice. Kluwer Academic Publishers, Boston, 2001. V. Torczon. On the convergence of pattern search algorithms. SIAM J. on Optimization, 7(1):1–25, 1997. I. W. Tsang, J. T. Kwok, and P. Cheung. Core vector machines: Fast svm training on very large data sets. Journal of Machine Learning Research, 6:363–392, 2005. B. van Ginneken and A. Mendrik. Image denoising with k-nearest neighbor and support vector regression. In The 18th International Conference on Pattern Recognition, ICPR’06, volume 3, pages 603–606, Hong Kong, 2006. 902 I MAGE D ENOISING WITH K ERNELS BASED ON NATURAL I MAGE R ELATIONS S. V. N. Vishwanathan, Nicol N. Schraudolph, and Alex J. Smola. Step size adaptation in reproducing kernel hilbert space. Journal of Machine Learning Research, 7:1107–1133, 2006. Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. J. Weston, C. Leslie, E. Ie, D. Zhou, A. Elisseeff, and W. Stafford Noble. Semi-supervised protein classification using cluster kernels. Bioinformatics, 21(15):3241–3247, 2004. 903