iccv iccv2013 iccv2013-82 iccv2013-82-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Supreeth Achar, Stephen T. Nuske, Srinivasa G. Narasimhan
Abstract: Separating the direct and global components of radiance can aid shape recovery algorithms and can provide useful information about materials in a scene. Practical methods for finding the direct and global components use multiple images captured under varying illumination patterns and require the scene, light source and camera to remain stationary during the image acquisition process. In this paper, we develop a motion compensation method that relaxes this condition and allows direct-global separation to beperformed on video sequences of dynamic scenes captured by moving projector-camera systems. Key to our method is being able to register frames in a video sequence to each other in the presence of time varying, high frequency active illumination patterns. We compare our motion compensated method to alternatives such as single shot separation and frame interleaving as well as ground truth. We present results on challenging video sequences that include various types of motions and deformations in scenes that contain complex materials like fabric, skin, leaves and wax.
[1] T. Brox, N. Papenberg, and J. Weickert. High Accuracy Optical Flow Estimation Based on a Theory for Warping. European Conf. on Computer Vision, 4(May):25–36, 2004. 4
[2] A. Chambolle. An Algorithm for Total Variation Minimization and Applications. Journal of Mathematical Imaging and Vision, 20(1-2):89–97, 2004. 3
[3] V. Couture, N. Martin, and S. Roy. Unstructured light scanning to overcome interreflections. ICCV, 2011. 1, 3
[4] J. Gu, T. Kobayashi, M. Gupta, and S. K. Nayar. Multiplexed illumination for scene recovery in the presence of global illumination. Intl. Conf. on Computer Vision, 2011. 2
[5] M. Gupta, A. Agrawal, A. Veeraraghavan, and S. G. Narasimhan. A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus. IJCV, 102(1-3):33–55, 2012. 1
[6] M. Gupta, Y. Tian, S. G. Narasimhan, and L. Zhang. A Combined Theory of Defocused Illumination and Global Light Transport. IJCV, 98(2): 146–167, Oct. 2011. 2 11448866 Interleaving Result on Center Frame Direct Component Global Component ez3Fraems gn dWniwod Sii5semarF Sils F0amer1Input Pat erned FrameFul y Lit Frame Motion Compensated Result on Center Frame Direct Component Global Component Groundtruth Direct ComponentGlobal Component Figure 5: Comparison to Interleaving: The separation from interleaving and our algorithm as the number of frames in the sliding window is changed. The camera is panning across the scene. The first three rows show the results generated by both methods for various sliding window sizes (global images are shown at 2 times actual brightness). Our method makes more efficient use of images than interleaving because no frames are needed exclusively for tracking. Separations that resolve a given level of detail can be obtained with a smaller temporal sliding window than an interleaving approach. For instance, our method resolves the text on the tennis ball much earlier and more clearly than interleaving.
[7] S. W. Hasinoff and K. N. Kutulakos. Light-efficient photography. IEEE PAMI, 33(1 1):2203–14, Nov. 2011. 6
[8] H. Haussecker and D. Fleet. Computing optical flow with physical models of brightness variation. IEEE PAMI, 23(6), June 2001 . 2
[9] H. Kawasaki, R. Furukawa, R. Sagawa, and Y. Yagi. Dynamic scene shape reconstruction using a single structured light pattern. CVPR, 2008. 1
[10] S. Konig and S. Gumhold. Image-Based Motion Compensation for Structured Light Scanning of Dynamic Surfaces. Dynamic 3D Imaging Workshop, 2007. 1, 2
[11] B. Lamond and P. Debevec. Fast Image-based Separation of Diffuse and Specular Reflections. SIGGRAPH Sketches, 2007. 6
[12] S. P. Mallick, T. Zickler, P. N. Belhumeur, and D. J. Kriegman. Specularity Removal in Images and Videos : A PDE Approach. European Conf. on Computer Vision, 2006. 6
[13] Y. Mileva. Illumination-Robust Variational Optical Flow with Photometric Invariants. DAGM Conference on Pattern Recognition, pages 152–162, 2007. 2
[14] S. K. Nayar, G. Krishnan, M. D. Grossberg, and R. Raskar. Fast separation of direct and global components of a scene using high frequency illumination. ACM Transactions on Graphics, 25(3):935, July 2006. 1, 2, 4
[15] M. O’Toole, R. Raskar, and K. N. Kutulakos. Primaldual coding to probe light transport. ACM Transactions on Graphics, 3 1(4): 1–1 1, July 2012. 2
[16] R. Raskar, A. Agarwal, and J. Tumblin. Coded Exposure Photography : Motion Deblurring using Fluttered Shutter. ACM Transactions on Graphics, 25(3):795–804, 2006. 6
[17] V. Srinivasan, H. C. Liu, and M. Halioua. Automated phasemeasuring profilometry. Applied Optics, 24(2), 1985. 1
[18] F. Steinbr u¨cker and D. Cremers. Large Displacement Optical Flow Computation without Warping. ICCV, 2009. 2, 4
[19] Y. Taguchi, A. Agrawal, and O. Tuzel. Motion-Aware Structured Light Using Spatio-Temporal Decodable Patterns. ECCV, 2012. 2
[20] T. Weise, B. Leibe, and L. Van Gool. Fast 3D Scanning with Automatic Motion Compensation. CVPR, 2007. 1, 2
[21] L. Zhang, N. Snavely, B. Curless, and S. Seitz. Spacetime Faces: High-Resolution Capture for Modeling and Animation. SIGGRAPH, 2004. 2 11448877 Without Motion Compensation Direct Component Global Component Figure 6: Direct-Global Separation on Deformable Objects: With Motion Compensation Direct Component Global Component On the left are the global and direct components estimated without any motion compensation for faces changing expression, articulating hands and a plant moving in the breeze. Many details get blurred away like the hair, and the lines on the palm. Other motion artifacts are clearly visible on the fingers and leaves. The two columns on the right show the component estimates on the same frames using our motion compensation method. With motion compensation, many ofthe artifacts are corrected and a lot more ofthe original scene detail is recovered. 11448888