cvpr cvpr2013 cvpr2013-333 cvpr2013-333-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Zihan Zhou, Hailin Jin, Yi Ma
Abstract: Recently, a new image deformation technique called content-preserving warping (CPW) has been successfully employed to produce the state-of-the-art video stabilization results in many challenging cases. The key insight of CPW is that the true image deformation due to viewpoint change can be well approximated by a carefully constructed warp using a set of sparsely constructed 3D points only. However, since CPW solely relies on the tracked feature points to guide the warping, it works poorly in large textureless regions, such as ground and building interiors. To overcome this limitation, in this paper we present a hybrid approach for novel view synthesis, observing that the textureless regions often correspond to large planar surfaces in the scene. Particularly, given a jittery video, we first segment each frame into piecewise planar regions as well as regions labeled as non-planar using Markov random fields. Then, a new warp is computed by estimating a single homography for regions belong to the same plane, while in- heriting results from CPW in the non-planar regions. We demonstrate how the segmentation information can be efficiently obtained and seamlessly integrated into the stabilization framework. Experimental results on a variety of real video sequences verify the effectiveness of our method.
[1] C. Baillard and A. Zisserman. Automatic reconstruction of piecewise planar models from multiple views. In CVPR, pages 2559–2565, 1999. 3
[2] A. Bartoli. A random sampling strategy for piecewise planar scene segmentation. Computer Vision and Image Under-
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13] standing, 105(1):42–59, 2007. 3 P. Bhat, C. L. Zitnick, N. Snavely, A. Agarwala, M. Agrawala, M. F. Cohen, B. Curless, and S. B. Kang. Using photographs to enhance videos of a static scene. In Rendering Techniques, pages 327–338, 2007. 1, 2 Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell., 23(1 1): 1222–1239, 2001. 4 C. Buehler, M. Bosse, and L. McMillan. Non-metric imagebased rendering for video stabilization. In CVPR (2), pages 609–614, 2001. 1, 2 B.-Y. Chen, K.-Y. Lee, W.-T. Huang, and J.-S. Lin. Capturing intention-based full-frame video stabilization. Comput. Graph. Forum, 27(7):1805–1814, 2008. 1, 2 A. W. Fitzgibbon, Y. Wexler, and A. Zisserman. Image-based rendering using image-based priors. International Journal of Computer Vision, 63(2): 141–151, 2005. 1, 2 Y. Furukawa, B. Curless, S. M. Seitz, and R. Szeliski. Manhattan-world stereo. In CVPR, pages 1422–1429, 2009. 3 D. Gallup, J.-M. Frahm, and M. Pollefeys. Piecewise planar and non-planar stereo for urban scene reconstruction. In CVPR, pages 1418–1425, 2010. 3 M. Gleicher and F. Liu. Re-cinematography: Improving the camerawork of casual video. TOMCCAP, 5(1), 2008. 1, 2 A. Goldstein and R. Fattal. Video stabilization using epipolar geometry. ACM Trans. Graph., 32(5), 2012. 3 M. Grundmann, V. Kwatra, D. Castro, and I. Essa. Effective calibration free rolling shutter removal. IEEE ICCP, 2012. 8 M. Grundmann, V. Kwatra, and I. A. Essa. Auto-directed video stabilization with robust l1 optimal camera paths. In
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22] CVPR, pages 225–232, 2011. 1, 2 T. Igarashi, T. Moscovich, and J. F. Hughes. As-rigidas-possible shape manipulation. ACM Trans. Graph., 24(3): 1134–1 141, 2005. 3 K.-Y. Lee, Y.-Y. Chuang, B.-Y. Chen, and M. Ouhyoung. Video stabilization using robust feature trajectories. In ICCV, pages 1397–1404, 2009. 1, 2 F. Liu, M. Gleicher, H. Jin, and A. Agarwala. Contentpreserving warps for 3d video stabilization. ACM Trans. Graph., 28(3), 2009. 1, 2, 3, 6, 7 F. Liu, M. Gleicher, J. Wang, H. Jin, and A. Agarwala. Subspace video stabilization. ACM Trans. Graph. , 30(1):4, 2011. 2 S. Liu, Y. Wang, L. Yuan, J. Bu, P. Tan, and J. Sun. Video stabilization with a depth camera. In CVPR, pages 89–95, 2012. 3 Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H.-Y. Shum. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell., 28(7): 1150–1 163, 2006. 1, 2, 8 B. Micus ´ık and J. Koseck´ a. Multi-view superpixel stereo in urban environments. International Journal of Computer Vision, 89(1): 106–1 19, 2010. 3 S. N. Sinha, D. Steedly, and R. Szeliski. Piecewise planar stereo for image-based rendering. In ICCV, pages 1881 1888, 2009. 3 R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. F. Tappen, and C. Rother. A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell., 30(6): 1068–1080, 2008. 5
[23] R. Toldo and A. Fusiello. Robust multiple structures estimation with j-linkage. In ECCV (1), pages 537–547, 2008. 4
[24] R. Toldo and A. Fusiello. Photo-consistent planar patches from unstructured cloud of points. In ECCV (5), pages 589– 602, 2010. 3
[25] J. Y. A. Wang and E. H. Adelson. Representing moving images with layers. IEEE Transactions on Image Processing, 3(5):625–638, 1994. 3
[26] T. Werner and A. Zisserman. New techniques for automated architectural reconstruction from photographs. In ECCV (2), pages 541–555, 2002. 3
[27] G. Zhang, X. Qin, W. Hua, T.-T. Wong, P.-A. Heng, and H. Bao. Robust metric reconstruction from challenging video sequences. In CVPR, 2007. 6
[28] Z. Zhou, H. Jin, and Y. Ma. Robust plane-based structure from motion. In CVPR, pages 1482–1489, 2012. 8
[29] M. Zuliani, C. S. Kenney, and B. S. Manjunath. The multiransac algorithm and its application to detect planar homographies. In ICIP (3), pages 153–156, 2005. 4 222333000644