iccv iccv2013 iccv2013-363 iccv2013-363-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Olivier Saurer, Kevin Köser, Jean-Yves Bouguet, Marc Pollefeys
Abstract: A huge fraction of cameras used nowadays is based on CMOS sensors with a rolling shutter that exposes the image line by line. For dynamic scenes/cameras this introduces undesired effects like stretch, shear and wobble. It has been shown earlier that rotational shake induced rolling shutter effects in hand-held cell phone capture can be compensated based on an estimate of the camera rotation. In contrast, we analyse the case of significant camera motion, e.g. where a bypassing streetlevel capture vehicle uses a rolling shutter camera in a 3D reconstruction framework. The introduced error is depth dependent and cannot be compensated based on camera motion/rotation alone, invalidating also rectification for stereo camera systems. On top, significant lens distortion as often present in wide angle cameras intertwines with rolling shutter effects as it changes the time at which a certain 3D point is seen. We show that naive 3D reconstructions (assuming global shutter) will deliver biased geometry already for very mild assumptions on vehicle speed and resolution. We then develop rolling shutter dense multiview stereo algorithms that solve for time of exposure and depth at the same time, even in the presence of lens distortion and perform an evaluation on ground truth laser scan models as well as on real street-level data.
[1] J. B. Alexandre Karpenko, David E. Jacobs and M. Levoy. Digital video stabilization and rolling shutter correction using gyroscopes. In CSTR, 2011. 1, 2
[2] S. Baker, E. Bennett, S. B. Kang, and R. Szeliski. Removing rolling shutter wobble. In CVPR, 2010. 2
[3] D. Bradley, B. Atcheson, I. Ihrke, and W. Heidrich. Synchronization and rolling shutter compensation for consumer video camera arrays. In Int. Workshop on Projector-Camera Systems, 2009. 2
[4] D. C. Brown. Close-range camera calibration. Photogrammetric Engineering, 1971. 3
[5] D. Feldman, T. Pajdla, and D. Weinshall. On the epipolar geometry of the crossed-slits projection. In ICCV, 2003. 2
[6] P.-E. Forss ´en and E. Ringaby. Rectifying rolling shutter video from hand-held devices. In CVPR, 2010. 1, 2
[7] A. Fusiello, E. Trucco, and A. Verri. A compact algorithm for rectification of stereo pairs. Mach. Vis. Appl. , 12(1), 2000. 1
[8] D. Gallup, J.-M. Frahm, P. Mordohai, Q. Yang, and M. Pollefeys. Real-time plane-sweeping stereo with multiple sweeping directions. In CVPR, 2007. 6
[9] C. Geyer, M. Meingast, , and S. Sastry. Geometric models of rolling-shutter cameras. In OMNIVIS, 2005. 1, 2, 3
[10] M. Grundmann, V. Kwatra, D. Castro, and I. Essa. Effective calibration free rolling shutter removal. ICCP, 2012. 2
[11] R. Gupta and R. I. Hartley. Linear pushbroom cameras. PAMI, 1997. 2
[12] G. Hanning, N. Forsl ¨ow, P.-E. Forss ´en, E. Ringaby,
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22] D. T ¨ornqvist, and J. Callmer. Stabilizing cell phone video using inertial measurement sensors. In Sec. Int. Workshop on mobile Vision, 2011. 1, 2 J. Hedborg, P.-E. Forss ´en, M. Felsberg, and E. Ringaby. Rolling shutter bundle adjustment. In CVPR, 2012. 2, 6 H. Hirschmuller. Stereo processing by semiglobal matching and mutual information. PAMI, 2008. 6 A. Jordt-Sedlazeck, D. Jung, and R. Koch. Refractive plane sweep for underwater images. In GCPR 2013, 2013. 2 B. Klingner, D. Martin, and J. Roseborough. Street view structure-from-motion. In ICCV, 2013. 2, 6 C.-K. Liang, L.-W. Chang, and H. Chen. Analysis and compensation of rolling shutter effect. Trans. Img. Proc., 2008. 2 R. A. Newcombe and A. J. Davison. Live dense reconstruction with a single moving camera. In CVPR, 2010. 1 M. Pollefeys, D. Nist e´r, J. M. Frahm, A. Akbarzadeh, P. Mordohai, B. Clipp, C. Engels, D. Gallup, S. J. Kim, P. Merrell, C. Salmi, S. Sinha, B. Talton, L. Wang, Q. Yang, H. Stew e´nius, R. Yang, G. Welch, and H. Towles. Detailed real-time urban 3d reconstruction from video. IJCV, 2008. 1 E. Ringaby and P.-E. Forss ´en. Efficient video rectification and stabilisation for cell-phones. IJCV, 2012. 1, 2 R. Yang and M. Pollefeys. Multi-resolution real-time stereo on commodity graphics hardware. In CVPR, 2003. 2, 5, 6 A. Zomet, D. Feldman, S. Peleg, and D. Weinshall. Mosaicing new views: the crossed-slits projection. PAMI, 2003. 2 472