iccv iccv2013 iccv2013-105 iccv2013-105-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Abstract: Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox & Malik [6], our approach, termed DeepFlow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable assetfor integration into an energy minimizationframework for optical flow estimation. DeepFlow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset [8].
[1] S. Baker and I. Matthews. Lucas-kanade 20 years on: A unifying framework. IJCV, 2004. 2, 6
[2] S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black, and R. Szeliski. A database and evaluation methodology for optical flow. IJCV, 2011. 1, 2, 6
[3] C. Barnes, E. Shechtman, D. B. Goldman, and A. Finkelstein. The generalized PatchMatch correspondence algo-
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14] rithm. In ECCV, 2010. 2 M. J. Black and P. Anandan. The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 1996. 1, 2 T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. High accuracy optical flow estimation based on a theory for warping. In ECCV, 2004. 2, 3, 6 T. Brox and J. Malik. Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. PAMI, 2011. 1, 2, 5, 6, 7, 8 A. Bruhn and J. Weickert. Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In ICCV, 2005. 2, 6 D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. In ECCV, 2012. 1, 2, 5, 6 A. Ecker and S. Ullman. A hierarchical non-parametric method for capturing non-rigid deformations. Image and Vision Computing, 2009. 2 A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The KITTI dataset. IJRR, 2013. 2, 6 L. Gorelick, M. Blank, E. Shechtman, M. Irani, and R. Basri. Actions as space-time shapes. IEEE Trans. PAMI, 2007. 1 D. Keysers, T. Deselaers, C. Gollan, and H. Ney. Deformation models for image recognition. IEEE Trans. PAMI, 2007. 2, 3 J. Kim, C. Liu, F. Sha, and K. Grauman. Deformable spatial pyramid matching for fast dense correspondences. In CVPR, 2013. 2 I. Laptev and P. Pérez. Retrieving actions in movies. In ICCV, 2007. 1
[15] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceedings of the IEEE, 1998. 2, 3, 4
[16] M. Leordeanu, A. Zanfir, and C. Sminchisescu. Locally affine sparse-to-dense matching for motion and occlusion estimation. In ICCV, 2013. 2, 7
[17] C. Liu, J. Yuen, and A. Torralba. SIFT flow: Dense correspondence across scenes and its applications. IEEE Trans. PAMI, 2011. 2
[18] J. Malik and P. Perona. Preattentive texture discrimination with early vision mechanisms. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 1990. 4
[19] P. Matikainen, M. Hebert, and R. Sukthankar. Trajectons: Action recognition through the motion analysis of tracked features. In ICCV Work., 2009. 1
[20] K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. V. Gool. A comparison of affine region detectors. IJCV, 2005. 2
[21] N. Papenberg, A. Bruhn, T. Brox, S. Didas, and J. Weickert. Highly accurate optic flow computation with theoretically justified warping. IJCV, 2006. 1, 2
[22] J. Revaud, P. Weinzaepfel, Z. Harchaoui, and C. Schmid. Deep matching and its application to large displacement optical flow. Technical report, 2013. 4, 5, 6
[23] F. Steinbrucker, T. Pock, and D. Cremers. Large displacement optical flow computation without warping. In ICCV, 2009. 1
[24] M. Stoll, S. Volz, and A. Bruhn. Adaptive integration of feature matches into variational optical flow methods. In ACCV, 2012. 6
[25] D. Sun, S. Roth, and M. J. Black. Secrets of optical flow
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34] estimation and their principles. In CVPR, 2010. 2, 5, 7, 8 R. Szeliski. Computer Vision: Algorithms and Applications. 2010. 1, 2, 3, 6 E. Tola, V. Lepetit, and P. Fua. A fast local descriptor for dense matching. CVPR, 2008. 2 S. Uchida and H. Sakoe. A monotonic and continuous twodimensional warping based on dynamic programming. In ICPR, 1998. 2, 3 C. Vogel, S. Roth, and K. Schindler. An evaluation of data costs for optical flow. In GCPR, 2013. 2, 8 H. Wang, A. Kläser, C. Schmid, and C.-L. Liu. Dense trajectories and motion boundary descriptors for action recognition. IJCV, 2013. 1 M. Werlberger, W. Trobin, T. Pock, A. Wedel, D. Cremers, and H. Bischof. Anisotropic Huber-L1 optical flow. In BMVC, 2009. 1, 2, 3 J. Wills, S. Agarwal, and S. Belongie. A feature-based approach for dense segmentation and estimation of large disparity motion. IJCV, 2006. 1, 2 L. Xu, J. Jia, and Y. Matsushita. Motion detail preserving optical flow estimation. IEEE Trans. PAMI, 2012. 1, 2, 7 H. Zimmer, A. Bruhn, and J. Weickert. Optic flow in harmony. IJCV, 2011. 5, 6 1392