iccv iccv2013 iccv2013-402 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Bryan Klingner, David Martin, James Roseborough
Abstract: We describe a structure-from-motion framework that handles “generalized” cameras, such as moving rollingshutter cameras, and works at an unprecedented scale— billions of images covering millions of linear kilometers of roads—by exploiting a good relative pose prior along vehicle paths. We exhibit a planet-scale, appearanceaugmented point cloud constructed with our framework and demonstrate its practical use in correcting the pose of a street-level image collection.
Reference: text
sentIndex sentText sentNum sentScore
1 We exhibit a planet-scale, appearanceaugmented point cloud constructed with our framework and demonstrate its practical use in correcting the pose of a street-level image collection. [sent-5, score-0.283]
2 Introduction Google Street View has a repository of billions of 2D images captured with rolling-shutter camera rigs along vehicle trajectories. [sent-7, score-0.442]
3 Although we use GPS and inertial sensors to estimate the pose of this imagery, it still contains lowfrequency error due to challenging GPS environments in cities and elsewhere. [sent-8, score-0.248]
4 To improve the pose of these images, we have extended traditional Structure from Motion (SfM) techniques to construct a point-based model of the streetlevel world where each point carries both its geometric po- sition as well as its local appearance from several views (see Figure 1). [sent-9, score-0.292]
5 We use the appearance information from this model to find corresponding 3D points viewed from nearby images, and the geometric information to align the cameras that view them, thereby globally correcting the imagery’s pose: motion-from-structure-from-motion. [sent-10, score-0.243]
6 These sensors are trustworthy over short distances and they give us an accurate estimate ofthe vehicle’s motion during image capture as well as the relative pose of images nearby each other in the trajectory, as depicted in Figure 2. [sent-14, score-0.257]
7 In Sections 3 and 4, we present a generalized camera model that uses this local pose to handle rolling shutters in the Figure 1. [sent-15, score-0.719]
8 In Section 6, we descibe how we apply the resulting appearance-augmented SfM model to reduce the global pose error of our imagery by more than 85% in the densest urban environments in the world. [sent-21, score-0.281]
9 The vehicle’s path establishes good relative pose and natural connectivity for the panoramic imagery we capture. [sent-23, score-0.377]
10 Related work Perspective cameras and SfM for perspective cameras are well studied [ 14]. [sent-25, score-0.302]
11 Rigorous efforts to solve for generalized camera models exist: 8 solutions for 3 camera rays to intersect 3 known world points [21], 64 solutions for 6 corresponding camera ray pairs [25]. [sent-28, score-0.695]
12 The latter technique can be used with RANSAC for motion estimation between two generalized cameras, but the method is complex and ultimately unnecessary with a good relative pose prior, which we possess. [sent-29, score-0.309]
13 Rolling-shutter cameras are a particularly common type of generalized camera, and have a literature of their own. [sent-30, score-0.244]
14 Complementary work has been done to remove rolling shutter distortion from images and video created when the camera moves [18] [7] [9] [5] [13]. [sent-34, score-0.725]
15 Perhaps the most directly related work on bundle adjustment is that of Hedborg et al. [sent-35, score-0.276]
16 If the camera moves while the shutter rolls, different pixels in the image have different projection centers. [sent-58, score-0.485]
17 Moving, rolling-shutter cameras are an example of a generalized camera [ 12]. [sent-59, score-0.393]
18 Figure 4 shows how the rolling shutter complicates pixel projection for the 15 camera rig. [sent-62, score-0.801]
19 In our rigs, rosette intrinsics and rolling shutter timings are calibrated. [sent-63, score-0.938]
20 We require a generalized camera model for these moving rosettes of calibrated rolling-shutter cameras. [sent-64, score-0.355]
21 We may write the generalized camera model as a non- c 3m0≈ 3m Figure 4. [sent-76, score-0.242]
22 Right: the same rays when the rosette undergoes typical vehicle motion (30kph). [sent-78, score-0.581]
23 Each pixel has a different projection center, in this case spread over several meters of vehicle trajectory. [sent-79, score-0.303]
24 A rolling shutter exposure that starts at t1 and ends at t2. [sent-82, score-0.665]
25 The vehicle moves so each image column may have a different projection center. [sent-83, score-0.233]
26 Due to the rolling shutter, this transform is a function of time t: xim = im? [sent-87, score-0.734]
27 w(t) because the rosette is moving rigidly through the w? [sent-192, score-0.424]
28 w(t) represents the 6 DOF pose of the rosette in the world? [sent-195, score-0.513]
29 The rolling shutter model relates pixel coordinates and time as some function t(xim). [sent-197, score-0.6]
30 The image point xim yields an exposure time t(xim) from the rolling shutter, which in turn sets the rosette pose w? [sent-202, score-1.328]
31 im · xim (3) Projection from the world frame into the image, however, is not well defined for a generalized camera. [sent-206, score-0.706]
32 Some world points may be imaged multiple times, whereas other world points may not be imaged at all. [sent-207, score-0.234]
33 The worldto-image projection equation is therefore implicit in xim: xim = im? [sent-212, score-0.489]
34 w(t(xim)) · xw (4) In practice, if the speed of the camera in the world is slow relative to the speed of the rolling shutter across objects in the scene, which is generally the case for a vehicle-mounted camera that rotates slowly, the mapping is well behaved. [sent-215, score-1.124]
35 Instead, in the following sections, we show how the generalized camera model may be approximated effectively by local linearization at feature locations. [sent-217, score-0.273]
36 Triangulation A fundamental operation in bundle adjustment is triangulation from multiple views. [sent-220, score-0.387]
37 Given multiple image observations xˆikm (superscripts hereafter dropped for clarity) of an unknown 3D world point, we wish to find the world point xw that minimizes reprojection error: argxmwinv? [sent-221, score-0.372]
38 Because rolling shutters tend to be fast, t(xim) is a slowly changing function and therefore t(xim) ≈ t(ˆ xim). [sent-227, score-0.356]
39 Then we may avoid computing t(xim) entirely: t ≈he t projection of xˆw into each camera may be done at the a priori known exposure times t(ˆ xim) for each feature location xˆim. [sent-228, score-0.321]
40 This approximation degrades as xˆim and xim diverge, but, critically for optimization, the approximation is exact at 995555 Figure 7. [sent-383, score-0.406]
41 We establish a virtual, linear “feature camera” at each feature whose optical axis is the projection of the feature center through the generalized camera model. [sent-384, score-0.356]
42 Moreover, the approximation diverges slowly in practice: For our cameras, over a very large feature of diameter 100 pixels, lens distortion varies by a few percent and pose varies by ∼1 cm at 30 kph. [sent-386, score-0.293]
43 Feature cameras In addition to the simplification that t(xim) ≈ t(ˆ xim), we may further simplify triangulation by linearizing the (smooth) lens model at each feature location. [sent-389, score-0.374]
44 The monolithic, generalized camera model is thereby shattered into a constellation of simple, global-shutter, linear perspective feature cameras, depicted in Figure 7. [sent-390, score-0.314]
45 To project a world point into a feature camera, xf = f? [sent-392, score-0.257]
46 The feature frame “f” is centered on the feature point xˆim, so if xˆim is an exact view of xw, then xf is (0, 0). [sent-520, score-0.236]
47 Thus, the projection coordinates xf directly yield reprojection error, and the triangulation problem is simply: argxmwinv? [sent-521, score-0.35]
48 Equation 7 (8) To control for varying magnification across the image due to lens distortion, we sample the angular resolution of the lens model at the feature location and scale the feature camera focal length so that all feature camera frames have compatible pixel scale. [sent-526, score-0.577]
49 Generalized bundle adjustment The previous section provides the essential elements required for bundle adjustment with general cameras. [sent-528, score-0.552]
50 ever, in contrast to global shutter cameras, the triangulation sub-problems of bundle adjustment are coupled through the camera trajectory r? [sent-530, score-0.904]
51 Using feature cameras, the bundle adjustment optimization for generalized cameras is: {axrw,g? [sent-532, score-0.551]
52 Equation 7 (9) The twist with bundle adjustment using generalized cameras is that the camera trajectory influences the camera model. [sent-539, score-0.902]
53 With traditional cameras, only the instantaneous pose of the camera is used (as camera extrinsics). [sent-540, score-0.477]
54 For generalized cameras, the trajectory of the camera during the rolling shutter becomes part of the camera intrinsics. [sent-541, score-1.051]
55 In the context of bundle adjustment and triangulation, one must choose a representation for the trajectory that constrains the feature cameras’ relative poses. [sent-542, score-0.485]
56 [ 15], for rolling-shutter video, the trajectory is represented by a single key pose for each frame, with the pose during the rolling shutter linearly interpolated between successive key poses. [sent-544, score-0.962]
57 If a relative pose prior is available, for example from a calibrated IMU, then the prior may be used to constrain bundle adjustment in a variety of ways depending on the accuracy of the prior and the nature of the camera motion. [sent-546, score-0.751]
58 For example, one may model low-order deviations from the relative pose prior as a way of both constraining the number of free pose variables and regularizing the bundled pose. [sent-547, score-0.428]
59 We use calibrated camera rigs with an IMU rigidly attached, mounted on a vehicle. [sent-548, score-0.286]
60 From a separate pose optimization step, which is outside the scope of this paper but locally dominated by the IMU, we have accurate relative pose on the timescale of the rosette exposure. [sent-549, score-0.729]
61 We therefore “bake in” to the camera model the relative pose that spans the rosette exposure so that bundling does not adjust the known high frequencies of pose. [sent-550, score-0.914]
62 nominal rosette frame “n” and a nominal rosette exposure time tn: ? [sent-553, score-1.0]
63 shutter delay from the nominal r)o −set tte exposure time tn for pixel xˆim. [sent-711, score-0.551]
64 ) ·xw (11) 995566 and during bundling we solve for the nominal rosette poses n? [sent-781, score-0.511]
65 Equation 11 (12) This now has the form of traditional bundle adjustment with linear cameras: • • • The first term f? [sent-790, score-0.276]
66 The relative pose of the feature camera from tn to t(ˆ xim) due to motion during the rolling shutter is baked into this term. [sent-793, score-1.088]
67 − The second term represents the 6 DOF pose of the rTohseett see ionn dth tee rwmor rledp rfersaemnets a tht ethe 6 DcoOnFsta pnot neo omfi tnhael rosette exposure time tn; this pose is a free variable. [sent-794, score-0.753]
68 The result xf is the projection of the world point into the feature camera for pixel xˆim; because the feature camera is centered on pixel xˆim, xf is literally the reprojection residual. [sent-796, score-0.873]
69 With this approach, all the feature cameras within each panorama form a rigid assembly (porcupine) given by the relative pose prior. [sent-797, score-0.567]
70 This assembly may move rigidly during bundling by changing the nominal rosette pose. [sent-798, score-0.588]
71 Thus, the highest frequencies of pose are baked into the camera model; the medium frequency errors are reduced by bundling; the lowest frequency errors remain and must be addressed via loop closing. [sent-799, score-0.424]
72 Figure 8 shows an example point cloud before and after bundle adjustment. [sent-800, score-0.268]
73 SfM at scale with feature cameras The previous section assembles many of the pieces required for SfM with generalized cameras, in particular the use of feature cameras and a relative pose prior for bundle adjustment. [sent-803, score-0.868]
74 In this section, we leverage the relative pose prior for the other half of SfM, track generation. [sent-804, score-0.287]
75 We fuse this sensor data to establish an initial trajectory for the vehicle uninformed by the imagery. [sent-811, score-0.265]
76 A comparison of an unbundled (top) to bundled (bottom) SfM point cloud, including top-down, block-level details of the clouds and inset histograms of reprojection error for all tracked feature points. [sent-813, score-0.251]
77 • • The absolute pose of the vehicle path is accurate to aTbhoeut a b1s0o lmuteete pros ien othfe th ewo vresth case, dthue i sto a multipath GPS issues in dense urban cores. [sent-814, score-0.396]
78 The relative pose along the vehicle path is extremely aTchceur raetlea (sub-centimeter) e be vcaeuhsicel eit p aist hdo ism eixntarteemd by calibrated IMU integration over short time scales. [sent-815, score-0.455]
79 Similar to SfM with video, the natural linear path ofthe vehicle trajectory establishes potential connectivity between images: only images within a fixed window of each other along the path are considered for joint participation in image tracks. [sent-817, score-0.35]
80 Tracking A strong relative pose prior lets us avoid visual odometry via RANSAC-based relative pose estimation, which is often the first step in an SfM system. [sent-821, score-0.487]
81 The relative pose is good enough for triangulation along this trajectory, so we can bootstrap the system by tracking features rather than by matching im- ages. [sent-823, score-0.327]
82 2 Geometric confirmation: The track triangulates with a 3eoDm triangulation error nbe:l oTwhe e1 t rmacekte trr iaanndg a reprojection error below 1 degree. [sent-829, score-0.289]
83 The combination of visual matching with geometric confirmation to the pose prior eliminates virtually all noise because all features in a track must be similar in appearance and conform to the geometric dictates of the camera model and vehicle trajectory. [sent-837, score-0.715]
84 This is not relevant for bundle adjustment, but it is immensely useful for image-based loop closing as discussed in Section 6. [sent-842, score-0.263]
85 Results Using the camera model and pose prior described in Sections 3-5, we have created an appearance-augmented, 3D SfM point cloud for a substantial subset of all Street View imagery, comprising over 404 billion tracked feature points 2We call this highly stable matching method BFF matching. [sent-845, score-0.594]
86 In practice, we spend about 10 seconds per 4-megapixel panorama to extract, track, and bundle adjust features on a single modern CPU core. [sent-852, score-0.322]
87 The largest concurrent bundle adjustment problem solved is about 1500 cameras (a 100-panorama window of a 15-camera rosette). [sent-854, score-0.427]
88 Loop closing with SfM constraints The primary application of our SfM model is the correction of global pose error in our vehicle trajectory and hence our image collection. [sent-860, score-0.485]
89 We can correct this error by establishing relative pose constraints between pairs of panoramas that capture overlapping views of the street-level world. [sent-863, score-0.315]
90 Each panorama in the pair is associated with a set of augmented 3D points from our SfM model, namely, all the points that the panorama views. [sent-865, score-0.35]
91 A comparison of vehicle paths in downtown San Francisco before (a) and after (b) SfM-based correction. [sent-868, score-0.258]
92 We repeat this process for all candidate panorama pairs, generating billions of relative pose constraints linking geographically proximal panoramas that may have been captured minutes, days, or years apart in time. [sent-873, score-0.453]
93 It’s difficult to answer this question conclusively, as we lack ground truth pose for our vehicle trajectories. [sent-877, score-0.332]
94 Limitations As described in Section 3, we rely heavily on a good local pose estimate to make our complex, rolling-shutter camera systems useable for SfM. [sent-883, score-0.3]
95 Kinematics from lines in a single rolling shutter image. [sent-923, score-0.576]
96 Structure and kinematics triangulation with a rolling shutter stereo rig. [sent-928, score-0.687]
97 Synchronization and rolling shutter compensation for consumer video camera arrays. [sent-953, score-0.75]
98 A histogram of reprojection error, in pixels, for each of the 404 billion tracked features in our augmented point cloud. [sent-1017, score-0.236]
99 A generic rolling shutter camera model and its application to dynamic pose estimation. [sent-1044, score-0.876]
100 A minimal solution to the generalized 3-point pose problem. [sent-1049, score-0.244]
wordName wordTfidf (topN-words)
[('xim', 0.406), ('rosette', 0.362), ('rolling', 0.292), ('shutter', 0.284), ('sfm', 0.264), ('vehicle', 0.181), ('bundle', 0.168), ('cameras', 0.151), ('pose', 0.151), ('camera', 0.149), ('panorama', 0.128), ('xf', 0.113), ('triangulation', 0.111), ('adjustment', 0.108), ('xw', 0.1), ('iews', 0.097), ('generalized', 0.093), ('exposure', 0.089), ('im', 0.089), ('world', 0.085), ('trajectory', 0.084), ('lens', 0.081), ('confirmation', 0.078), ('nominal', 0.077), ('tn', 0.077), ('reprojection', 0.074), ('cloud', 0.072), ('rigs', 0.072), ('bundling', 0.072), ('gps', 0.071), ('panoramas', 0.069), ('imagery', 0.065), ('relative', 0.065), ('imu', 0.06), ('argxmwinv', 0.058), ('rosettes', 0.058), ('loop', 0.056), ('tracked', 0.054), ('lowe', 0.053), ('downtown', 0.052), ('projection', 0.052), ('billion', 0.05), ('hedborg', 0.048), ('meters', 0.046), ('track', 0.044), ('constellation', 0.041), ('assembly', 0.041), ('sensors', 0.041), ('billions', 0.04), ('panoramic', 0.04), ('baked', 0.039), ('bff', 0.039), ('ointsv', 0.039), ('planet', 0.039), ('rolls', 0.039), ('closing', 0.039), ('rays', 0.038), ('pages', 0.038), ('rigidly', 0.036), ('transform', 0.036), ('urban', 0.035), ('shutters', 0.034), ('bundled', 0.034), ('superlinear', 0.034), ('frame', 0.033), ('points', 0.032), ('irschara', 0.032), ('markings', 0.032), ('correcting', 0.032), ('equation', 0.031), ('feature', 0.031), ('slowly', 0.03), ('ceres', 0.03), ('google', 0.03), ('error', 0.03), ('augmented', 0.03), ('street', 0.029), ('snavely', 0.029), ('must', 0.029), ('calibrated', 0.029), ('path', 0.029), ('francisco', 0.029), ('geometric', 0.028), ('point', 0.028), ('tracks', 0.028), ('gordon', 0.028), ('instantaneous', 0.028), ('odometry', 0.028), ('prior', 0.027), ('downsample', 0.027), ('establishes', 0.027), ('grossberg', 0.027), ('adjust', 0.026), ('moving', 0.026), ('inertial', 0.026), ('paths', 0.025), ('geyer', 0.025), ('compensation', 0.025), ('pixel', 0.024), ('perhaps', 0.024)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999899 402 iccv-2013-Street View Motion-from-Structure-from-Motion
Author: Bryan Klingner, David Martin, James Roseborough
Abstract: We describe a structure-from-motion framework that handles “generalized” cameras, such as moving rollingshutter cameras, and works at an unprecedented scale— billions of images covering millions of linear kilometers of roads—by exploiting a good relative pose prior along vehicle paths. We exhibit a planet-scale, appearanceaugmented point cloud constructed with our framework and demonstrate its practical use in correcting the pose of a street-level image collection.
2 0.44547546 363 iccv-2013-Rolling Shutter Stereo
Author: Olivier Saurer, Kevin Köser, Jean-Yves Bouguet, Marc Pollefeys
Abstract: A huge fraction of cameras used nowadays is based on CMOS sensors with a rolling shutter that exposes the image line by line. For dynamic scenes/cameras this introduces undesired effects like stretch, shear and wobble. It has been shown earlier that rotational shake induced rolling shutter effects in hand-held cell phone capture can be compensated based on an estimate of the camera rotation. In contrast, we analyse the case of significant camera motion, e.g. where a bypassing streetlevel capture vehicle uses a rolling shutter camera in a 3D reconstruction framework. The introduced error is depth dependent and cannot be compensated based on camera motion/rotation alone, invalidating also rectification for stereo camera systems. On top, significant lens distortion as often present in wide angle cameras intertwines with rolling shutter effects as it changes the time at which a certain 3D point is seen. We show that naive 3D reconstructions (assuming global shutter) will deliver biased geometry already for very mild assumptions on vehicle speed and resolution. We then develop rolling shutter dense multiview stereo algorithms that solve for time of exposure and depth at the same time, even in the presence of lens distortion and perform an evaluation on ground truth laser scan models as well as on real street-level data.
3 0.34819886 32 iccv-2013-A Unified Rolling Shutter and Motion Blur Model for 3D Visual Registration
Author: Maxime Meilland, Tom Drummond, Andrew I. Comport
Abstract: Motion blur and rolling shutter deformations both inhibit visual motion registration, whether it be due to a moving sensor or a moving target. Whilst both deformations exist simultaneously, no models have been proposed to handle them together. Furthermore, neither deformation has been consideredpreviously in the context of monocularfullimage 6 degrees of freedom registration or RGB-D structure and motion. As will be shown, rolling shutter deformation is observed when a camera moves faster than a single pixel in parallax between subsequent scan-lines. Blur is a function of the pixel exposure time and the motion vector. In this paper a complete dense 3D registration model will be derived to accountfor both motion blur and rolling shutter deformations simultaneously. Various approaches will be compared with respect to ground truth and live real-time performance will be demonstratedfor complex scenarios where both blur and shutter deformations are dominant.
4 0.21858913 367 iccv-2013-SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels
Author: Jianxiong Xiao, Andrew Owens, Antonio Torralba
Abstract: Existing scene understanding datasets contain only a limited set of views of a place, and they lack representations of complete 3D spaces. In this paper, we introduce SUN3D, a large-scale RGB-D video database with camera pose and object labels, capturing the full 3D extent of many places. The tasks that go into constructing such a dataset are difficult in isolation hand-labeling videos is painstaking, and structure from motion (SfM) is unreliable for large spaces. But if we combine them together, we make the dataset construction task much easier. First, we introduce an intuitive labeling tool that uses a partial reconstruction to propagate labels from one frame to another. Then we use the object labels to fix errors in the reconstruction. For this, we introduce a generalization of bundle adjustment that incorporates object-to-object correspondences. This algorithm works by constraining points for the same object from different frames to lie inside a fixed-size bounding box, parameterized by its rotation and translation. The SUN3D database, the source code for the generalized bundle adjustment, and the web-based 3D annotation tool are all avail– able at http://sun3d.cs.princeton.edu.
5 0.16161026 286 iccv-2013-NYC3DCars: A Dataset of 3D Vehicles in Geographic Context
Author: Kevin Matzen, Noah Snavely
Abstract: Geometry and geography can play an important role in recognition tasks in computer vision. To aid in studying connections between geometry and recognition, we introduce NYC3DCars, a rich dataset for vehicle detection in urban scenes built from Internet photos drawn from the wild, focused on densely trafficked areas of New York City. Our dataset is augmented with detailed geometric and geographic information, including full camera poses derived from structure from motion, 3D vehicle annotations, and geographic information from open resources, including road segmentations and directions of travel. NYC3DCars can be used to study new questions about using geometric information in detection tasks, and to explore applications of Internet photos in understanding cities. To demonstrate the utility of our data, we evaluate the use of the geographic information in our dataset to enhance a parts-based detection method, and suggest other avenues for future exploration.
6 0.12604252 111 iccv-2013-Detecting Dynamic Objects with Multi-view Background Subtraction
7 0.12394218 17 iccv-2013-A Global Linear Method for Camera Pose Registration
8 0.11031759 184 iccv-2013-Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion
9 0.10401605 289 iccv-2013-Network Principles for SfM: Disambiguating Repeated Structures with Local Context
10 0.10337541 280 iccv-2013-Multi-view 3D Reconstruction from Uncalibrated Radially-Symmetric Cameras
11 0.1029467 348 iccv-2013-Refractive Structure-from-Motion on Underwater Images
12 0.097023591 115 iccv-2013-Direct Optimization of Frame-to-Frame Rotation
13 0.093850076 254 iccv-2013-Live Metric 3D Reconstruction on Mobile Phones
14 0.092571534 323 iccv-2013-Pose Estimation with Unknown Focal Length Using Points, Directions and Lines
15 0.091254026 228 iccv-2013-Large-Scale Multi-resolution Surface Reconstruction from RGB-D Sequences
16 0.090210274 49 iccv-2013-An Enhanced Structure-from-Motion Paradigm Based on the Absolute Dual Quadric and Images of Circular Points
17 0.08746551 366 iccv-2013-STAR3D: Simultaneous Tracking and Reconstruction of 3D Objects Using RGB-D Data
18 0.086367153 218 iccv-2013-Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data
19 0.083669782 341 iccv-2013-Real-Time Body Tracking with One Depth Camera and Inertial Sensors
topicId topicWeight
[(0, 0.165), (1, -0.169), (2, -0.032), (3, 0.079), (4, 0.01), (5, 0.009), (6, 0.041), (7, -0.115), (8, 0.034), (9, 0.107), (10, 0.046), (11, -0.068), (12, -0.09), (13, -0.071), (14, -0.056), (15, 0.099), (16, 0.103), (17, 0.073), (18, -0.036), (19, 0.06), (20, -0.043), (21, -0.259), (22, -0.098), (23, 0.15), (24, -0.042), (25, -0.152), (26, -0.155), (27, -0.064), (28, 0.15), (29, 0.158), (30, -0.096), (31, -0.175), (32, 0.115), (33, 0.17), (34, -0.045), (35, 0.129), (36, 0.07), (37, -0.014), (38, -0.058), (39, -0.11), (40, -0.036), (41, -0.035), (42, 0.03), (43, -0.023), (44, 0.041), (45, 0.011), (46, -0.052), (47, -0.029), (48, -0.037), (49, 0.013)]
simIndex simValue paperId paperTitle
1 0.93950236 363 iccv-2013-Rolling Shutter Stereo
Author: Olivier Saurer, Kevin Köser, Jean-Yves Bouguet, Marc Pollefeys
Abstract: A huge fraction of cameras used nowadays is based on CMOS sensors with a rolling shutter that exposes the image line by line. For dynamic scenes/cameras this introduces undesired effects like stretch, shear and wobble. It has been shown earlier that rotational shake induced rolling shutter effects in hand-held cell phone capture can be compensated based on an estimate of the camera rotation. In contrast, we analyse the case of significant camera motion, e.g. where a bypassing streetlevel capture vehicle uses a rolling shutter camera in a 3D reconstruction framework. The introduced error is depth dependent and cannot be compensated based on camera motion/rotation alone, invalidating also rectification for stereo camera systems. On top, significant lens distortion as often present in wide angle cameras intertwines with rolling shutter effects as it changes the time at which a certain 3D point is seen. We show that naive 3D reconstructions (assuming global shutter) will deliver biased geometry already for very mild assumptions on vehicle speed and resolution. We then develop rolling shutter dense multiview stereo algorithms that solve for time of exposure and depth at the same time, even in the presence of lens distortion and perform an evaluation on ground truth laser scan models as well as on real street-level data.
same-paper 2 0.89436394 402 iccv-2013-Street View Motion-from-Structure-from-Motion
Author: Bryan Klingner, David Martin, James Roseborough
Abstract: We describe a structure-from-motion framework that handles “generalized” cameras, such as moving rollingshutter cameras, and works at an unprecedented scale— billions of images covering millions of linear kilometers of roads—by exploiting a good relative pose prior along vehicle paths. We exhibit a planet-scale, appearanceaugmented point cloud constructed with our framework and demonstrate its practical use in correcting the pose of a street-level image collection.
3 0.85006779 32 iccv-2013-A Unified Rolling Shutter and Motion Blur Model for 3D Visual Registration
Author: Maxime Meilland, Tom Drummond, Andrew I. Comport
Abstract: Motion blur and rolling shutter deformations both inhibit visual motion registration, whether it be due to a moving sensor or a moving target. Whilst both deformations exist simultaneously, no models have been proposed to handle them together. Furthermore, neither deformation has been consideredpreviously in the context of monocularfullimage 6 degrees of freedom registration or RGB-D structure and motion. As will be shown, rolling shutter deformation is observed when a camera moves faster than a single pixel in parallax between subsequent scan-lines. Blur is a function of the pixel exposure time and the motion vector. In this paper a complete dense 3D registration model will be derived to accountfor both motion blur and rolling shutter deformations simultaneously. Various approaches will be compared with respect to ground truth and live real-time performance will be demonstratedfor complex scenarios where both blur and shutter deformations are dominant.
4 0.52864188 348 iccv-2013-Refractive Structure-from-Motion on Underwater Images
Author: Anne Jordt-Sedlazeck, Reinhard Koch
Abstract: In underwater environments, cameras need to be confined in an underwater housing, viewing the scene through a piece of glass. In case of flat port underwater housings, light rays entering the camera housing are refracted twice, due to different medium densities of water, glass, and air. This causes the usually linear rays of light to bend and the commonly used pinhole camera model to be invalid. When using the pinhole camera model without explicitly modeling refraction in Structure-from-Motion (SfM) methods, a systematic model error occurs. Therefore, in this paper, we propose a system for computing camera path and 3D points with explicit incorporation of refraction using new methods for pose estimation. Additionally, a new error function is introduced for non-linear optimization, especially bundle adjustment. The proposed method allows to increase reconstruction accuracy and is evaluated in a set of experiments, where the proposed method’s performance is compared to SfM with the perspective camera model.
Author: Lilian Calvet, Pierre Gurdjos
Abstract: This work aims at introducing a new unified Structurefrom-Motion (SfM) paradigm in which images of circular point-pairs can be combined with images of natural points. An imaged circular point-pair encodes the 2D Euclidean structure of a world plane and can easily be derived from the image of a planar shape, especially those including circles. A classical SfM method generally runs two steps: first a projective factorization of all matched image points (into projective cameras and points) and second a camera selfcalibration that updates the obtained world from projective to Euclidean. This work shows how to introduce images of circular points in these two SfM steps while its key contribution is to provide the theoretical foundations for combining “classical” linear self-calibration constraints with additional ones derived from such images. We show that the two proposed SfM steps clearly contribute to better results than the classical approach. We validate our contributions on synthetic and real images.
6 0.4515402 17 iccv-2013-A Global Linear Method for Camera Pose Registration
7 0.44652066 164 iccv-2013-Fibonacci Exposure Bracketing for High Dynamic Range Imaging
8 0.43915433 173 iccv-2013-Fluttering Pattern Generation Using Modified Legendre Sequence for Coded Exposure Imaging
9 0.43205121 280 iccv-2013-Multi-view 3D Reconstruction from Uncalibrated Radially-Symmetric Cameras
10 0.4125565 367 iccv-2013-SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels
11 0.38682935 286 iccv-2013-NYC3DCars: A Dataset of 3D Vehicles in Geographic Context
12 0.38654065 254 iccv-2013-Live Metric 3D Reconstruction on Mobile Phones
13 0.38179353 226 iccv-2013-Joint Subspace Stabilization for Stereoscopic Video
14 0.37374336 152 iccv-2013-Extrinsic Camera Calibration without a Direct View Using Spherical Mirror
15 0.36911917 228 iccv-2013-Large-Scale Multi-resolution Surface Reconstruction from RGB-D Sequences
16 0.36533093 9 iccv-2013-A Flexible Scene Representation for 3D Reconstruction Using an RGB-D Camera
17 0.35623285 436 iccv-2013-Unsupervised Intrinsic Calibration from a Single Frame Using a "Plumb-Line" Approach
18 0.35122555 289 iccv-2013-Network Principles for SfM: Disambiguating Repeated Structures with Local Context
19 0.3474609 184 iccv-2013-Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion
20 0.34170541 397 iccv-2013-Space-Time Tradeoffs in Photo Sequencing
topicId topicWeight
[(2, 0.036), (12, 0.015), (26, 0.043), (31, 0.035), (35, 0.011), (42, 0.098), (64, 0.048), (73, 0.032), (89, 0.185), (95, 0.015), (98, 0.37)]
simIndex simValue paperId paperTitle
1 0.79686832 435 iccv-2013-Unsupervised Domain Adaptation by Domain Invariant Projection
Author: Mahsa Baktashmotlagh, Mehrtash T. Harandi, Brian C. Lovell, Mathieu Salzmann
Abstract: Domain-invariant representations are key to addressing the domain shift problem where the training and test examples follow different distributions. Existing techniques that have attempted to match the distributions of the source and target domains typically compare these distributions in the original feature space. This space, however, may not be directly suitable for such a comparison, since some of the features may have been distorted by the domain shift, or may be domain specific. In this paper, we introduce a Domain Invariant Projection approach: An unsupervised domain adaptation method that overcomes this issue by extracting the information that is invariant across the source and target domains. More specifically, we learn a projection of the data to a low-dimensional latent space where the distance between the empirical distributions of the source and target examples is minimized. We demonstrate the effectiveness of our approach on the task of visual object recognition and show that it outperforms state-of-the-art methods on a standard domain adaptation benchmark dataset.
2 0.77903306 431 iccv-2013-Unbiased Metric Learning: On the Utilization of Multiple Datasets and Web Images for Softening Bias
Author: Chen Fang, Ye Xu, Daniel N. Rockmore
Abstract: Many standard computer vision datasets exhibit biases due to a variety of sources including illumination condition, imaging system, and preference of dataset collectors. Biases like these can have downstream effects in the use of vision datasets in the construction of generalizable techniques, especially for the goal of the creation of a classification system capable of generalizing to unseen and novel datasets. In this work we propose Unbiased Metric Learning (UML), a metric learning approach, to achieve this goal. UML operates in the following two steps: (1) By varying hyperparameters, it learns a set of less biased candidate distance metrics on training examples from multiple biased datasets. The key idea is to learn a neighborhood for each example, which consists of not only examples of the same category from the same dataset, but those from other datasets. The learning framework is based on structural SVM. (2) We do model validation on a set of weakly-labeled web images retrieved by issuing class labels as keywords to search engine. The metric with best validationperformance is selected. Although the web images sometimes have noisy labels, they often tend to be less biased, which makes them suitable for the validation set in our task. Cross-dataset image classification experiments are carried out. Results show significant performance improvement on four well-known computer vision datasets.
same-paper 3 0.77396882 402 iccv-2013-Street View Motion-from-Structure-from-Motion
Author: Bryan Klingner, David Martin, James Roseborough
Abstract: We describe a structure-from-motion framework that handles “generalized” cameras, such as moving rollingshutter cameras, and works at an unprecedented scale— billions of images covering millions of linear kilometers of roads—by exploiting a good relative pose prior along vehicle paths. We exhibit a planet-scale, appearanceaugmented point cloud constructed with our framework and demonstrate its practical use in correcting the pose of a street-level image collection.
4 0.76965773 434 iccv-2013-Unifying Nuclear Norm and Bilinear Factorization Approaches for Low-Rank Matrix Decomposition
Author: Ricardo Cabral, Fernando De_La_Torre, João P. Costeira, Alexandre Bernardino
Abstract: Low rank models have been widely usedfor the representation of shape, appearance or motion in computer vision problems. Traditional approaches to fit low rank models make use of an explicit bilinear factorization. These approaches benefit from fast numerical methods for optimization and easy kernelization. However, they suffer from serious local minima problems depending on the loss function and the amount/type of missing data. Recently, these lowrank models have alternatively been formulated as convex problems using the nuclear norm regularizer; unlike factorization methods, their numerical solvers are slow and it is unclear how to kernelize them or to impose a rank a priori. This paper proposes a unified approach to bilinear factorization and nuclear norm regularization, that inherits the benefits of both. We analyze the conditions under which these approaches are equivalent. Moreover, based on this analysis, we propose a new optimization algorithm and a “rank continuation ” strategy that outperform state-of-theart approaches for Robust PCA, Structure from Motion and Photometric Stereo with outliers and missing data.
5 0.76691318 271 iccv-2013-Modeling the Calibration Pipeline of the Lytro Camera for High Quality Light-Field Image Reconstruction
Author: Donghyeon Cho, Minhaeng Lee, Sunyeong Kim, Yu-Wing Tai
Abstract: Light-field imaging systems have got much attention recently as the next generation camera model. A light-field imaging system consists of three parts: data acquisition, manipulation, and application. Given an acquisition system, it is important to understand how a light-field camera converts from its raw image to its resulting refocused image. In this paper, using the Lytro camera as an example, we describe step-by-step procedures to calibrate a raw light-field image. In particular, we are interested in knowing the spatial and angular coordinates of the micro lens array and the resampling process for image reconstruction. Since Lytro uses a hexagonal arrangement of a micro lens image, additional treatments in calibration are required. After calibration, we analyze and compare the performances of several resampling methods for image reconstruction with and without calibration. Finally, a learning based interpolation method is proposed which demonstrates a higher quality image reconstruction than previous interpolation methods including a method used in Lytro software.
6 0.73698562 19 iccv-2013-A Learning-Based Approach to Reduce JPEG Artifacts in Image Matting
7 0.71294165 33 iccv-2013-A Unified Video Segmentation Benchmark: Annotation, Metrics and Analysis
8 0.63419086 181 iccv-2013-Frustratingly Easy NBNN Domain Adaptation
9 0.6317234 123 iccv-2013-Domain Adaptive Classification
10 0.62441278 438 iccv-2013-Unsupervised Visual Domain Adaptation Using Subspace Alignment
11 0.61699355 1 iccv-2013-3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding
12 0.61056238 427 iccv-2013-Transfer Feature Learning with Joint Distribution Adaptation
13 0.58734244 183 iccv-2013-Geometric Registration Based on Distortion Estimation
14 0.57922798 222 iccv-2013-Joint Learning of Discriminative Prototypes and Large Margin Nearest Neighbor Classifiers
15 0.57614529 141 iccv-2013-Enhanced Continuous Tabu Search for Parameter Estimation in Multiview Geometry
16 0.56435126 363 iccv-2013-Rolling Shutter Stereo
17 0.56226486 280 iccv-2013-Multi-view 3D Reconstruction from Uncalibrated Radially-Symmetric Cameras
18 0.5611341 367 iccv-2013-SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels
19 0.5580194 44 iccv-2013-Adapting Classification Cascades to New Domains
20 0.55798817 190 iccv-2013-Handling Occlusions with Franken-Classifiers