cvpr cvpr2013 cvpr2013-428 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Austin Abrams, Kylia Miskell, Robert Pless
Abstract: Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from webcams “in . wu st l. edu (b)(c) the wild.”
Reference: text
sentIndex sentText sentNum sentScore
1 Abstract Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. [sent-2, score-1.038]
2 Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. [sent-3, score-0.327]
3 Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. [sent-5, score-0.222]
4 Introduction A pixel under shadow has a dramatically different intensity than the same pixel under direct lighting. [sent-10, score-0.791]
5 Vision applications often incorporate shadows into their models, either by treating them as noise to be detected and ignored [8, 23], exploiting them as cues for camera calibration [5, 13], or incorporating them into larger image formation models [1, 3]. [sent-11, score-0.286]
6 In this paper, we treat shadows as a strong geometric cue: if a pixel is under shadow, then it must be the case that some other object along the lighting direction is casting a shadow onto it. [sent-12, score-1.004]
7 If the camera also has known geometric calibration, we can express this property as a linear constraint over the depth of each pixel involved. [sent-14, score-0.319]
8 From this geometry, we derive three novel results: • • An image-space constraint between a shadow and its oAcncl iumdeagr,e An approach to geometrically calibrate a camera from sAhnad aopwpr correspondences, aanlldy Figure 1. [sent-15, score-0.93]
9 In this paper, we exploit the inherent structure of cast shadows to recover shape from a single view. [sent-16, score-0.274]
10 Given a time-lapse sequence from a geographically-calibrated camera (a), we create correspondences (shown as a yellow line) between a shadow (blue) and its occluding object (red) (b). [sent-17, score-0.973]
11 Repeated across the image (c) and across many lighting directions, these tens of thousands of correspondences can be used as a cue to recover a sparse depth map from a single viewpoint (d). [sent-18, score-0.438]
12 Inferring depth from shadow correspondences has sev11111444440000057755 (a)(b)(c) Figure 2. [sent-21, score-1.007]
13 Where could the red point in (a) cast a shadow in the scene? [sent-23, score-0.798]
14 This solar plane intersects the image plane, defining the episolar line (c). [sent-25, score-0.694]
15 Finding the correct shadow correspondence therefore constrains the relative depth of each point. [sent-26, score-0.924]
16 First, shadow correspondences capture general shape: we do not require the ground to be planar or even visible, nor do we require the depth surface to be smooth or continuous. [sent-28, score-1.085]
17 Next, since we work directly with binary shadow masks, rather than intensities, we do not need to account for real-world photometric distortions such as variable exposure and radiometric response, so long as the shadow extraction pipeline is sufficiently robust. [sent-29, score-1.457]
18 Early work focused on interpreting shadows from line drawings: Shafer and Kanade [22] introduced a general theory for describing the orientation of surfaces by the shadows they cast onto each other. [sent-33, score-0.441]
19 Lowe and Binford [17] build a reasoning system to infer structure from line drawings, where one cue leverages manuallyspecified correspondences between a shadow and its caster. [sent-34, score-0.959]
20 Early work by Hatzitheodorou and Kender [9] introduces an approach to recover the shape of a one-dimensional surface slice from the shadows it casts on itself, extended by Raviv et al. [sent-36, score-0.35]
21 [21] leveraged epipolar geometry to carve out a surface from shadow labels across multiple views (see [15] for a survey on space carving). [sent-39, score-0.829]
22 Shadowgrams [7], shadow graphs [24], and shadow/antishadow constraints [6] all encode a con- straint similar to the one presented in this paper: all pixels on the image-space line between a shadow and its occluder should have a height below the corresponding 3D line. [sent-41, score-1.576]
23 In contrast, we do not place any constraint on the intermediate pixels between a shadow and its occluder, which removes the assumption that the depth surface is terrain-like. [sent-42, score-0.975]
24 Kawasaki and Furukawa [14] treat shape-from-shadows as a kind of structured light, where a wand is waved in front of the light source, and recover depth by constraining that the group of pixels shaded by the wand in any particular frame are coplanar in 3D. [sent-44, score-0.303]
25 In this work, we do not place assumptions on the shape of the object that casts shadows in each frame. [sent-45, score-0.289]
26 [4] implement a single-view shadow carving algorithm suitable for long-term timelapses. [sent-47, score-0.746]
27 Episolar Geometry In this section, we derive the geometric constraints tween a shaded pixel and the pixel that cast its shadow. [sent-51, score-0.265]
28 In an abuse of terminology, the phrase “y casts a shadow onto x” should be interpreted as “the 3D object that projects onto the image 1 1 14 4 40 0 068 6 Figure3. [sent-60, score-0.97]
29 at y casts a shadow onto the 3D object that projects onto the image at x”. [sent-63, score-0.97]
30 Suppose that some pixel y casts a shadow onto some other pixel x for some lighting direction Lt; we denote such a correspondence as y ? [sent-65, score-1.154]
31 Assuming directional lighting, this correspondence emplaces a constraint on the depths d of pixels at x and y: rxdx + Ltαxy = rydy, (1) where αxy is the unknown 3D distance between pixels x and y. [sent-67, score-0.379]
32 This constraint takes the form of a linear constraint involving the unknown depth of each pixel and the 3D distance between x and y. [sent-68, score-0.295]
33 This property holds a close relationship with well-known epipolar geometry, so we denote Equation 1 as the episolar constraint. [sent-69, score-0.484]
34 See Figure 2 for a visualization of the episolar constraint. [sent-70, score-0.459]
35 Nowhere do we make the assumption that our scene has a substantial ground plane, or that the depth surface is smooth or continuous. [sent-72, score-0.215]
36 This reduces the search space to 1D when determining shadow correspondences. [sent-75, score-0.707]
37 Second, we derive a nonlinear optimization to geometrically calibrate a camera from shadow correspondences. [sent-76, score-0.897]
38 Finally, given correspondences from a variety of lighting directions, we derive a convex optimization procedure which recovers the depths of all pixels involved. [sent-78, score-0.356]
39 The Episolar Line Generating correspondences between a shadow x and its occluder y is a challenging problem, but Equation 1 sheds some light on the shadow correspondence problem. [sent-81, score-1.783]
40 If y casts a shadow onto some unknown location x, then the point rxdx must lie in the linear subspace spanned by ry and Lt. [sent-82, score-1.043]
41 Therefore, if a pixel y casts a shadow, then its corresponding pixel x must lie on this episolar line. [sent-84, score-0.668]
42 Although this constraint alone does not dictate where on the episolar line the shadow truly comes from, it dramatically reduces the search space necessary for shadow correspondence. [sent-85, score-1.975]
43 Of practical interest is that the episolar line does not suffer from the common aperture problem seen in other correspondence problems. [sent-87, score-0.654]
44 For example, linking a roofline to its horizontal shadow would be ambiguous without using this constraint; any point on the roof could conceivably produce a shadow anywhere on the shadow edge. [sent-88, score-2.121]
45 However, this horizontal shadow will cross the episolar line at exactly one point, disambiguating the aperture problem. [sent-89, score-1.261]
46 If we could only generate correspondences on shadow corners, the constraint set might not be dense enough to use reliably. [sent-92, score-0.945]
47 However, since we can create correspondences across shadow edges, our overall correspondence set is much more informative of the underlying geometry. [sent-93, score-0.99]
48 Episolar Calibration Notice that in order to generate the episolar line, we need estimates of the camera’s calibration to determine pixel rays r in the same coordinate frame of L (in our case, the EastNorth-Up space). [sent-97, score-0.564]
49 However, estimating the geometric calibration of an outdoor camera is nontrivial. [sent-98, score-0.214]
50 Various approaches exist for calibration from outdoor cues such as sky color [16] or shadow trajectories cast onto the ground plane [5, 13]. [sent-99, score-1.103]
51 1A similar plane forms the basis for much of the work in the shadow carving approach presented in [21]. [sent-101, score-0.788]
52 1 1 14 4 40 0 079 7 However, cast shadows are abundant in most outdoor scenes. [sent-102, score-0.276]
53 Here, we leverage user-supplied shadow correspondences to calibrate a camera. [sent-103, score-0.925]
54 Through the episolar constraint, we find the camera calibration parameters θ that define a pinhole camera which produces episolar lines most consistent with the given correspondences. [sent-104, score-1.166]
55 More formally, if a user supplies a set of ground truth shadow correspondences G = {yi ? [sent-105, score-0.926]
56 ti xi}, and eθ (x, t) ∈ Rsha2 ddoewfin ceosr trhesep uonnidt-evneccetosr G episolar directio}n, faonrd a pixel x a∈t time t under camera parameters θ, we solve the nonlinear optimization θ∗= argθ,mβin? [sent-106, score-0.606]
57 ||xi+ βieθ(xi,ti) − yi||2, (2) where βi is the distance between xi and yi along the episolar line (analogous to α in Equation 1). [sent-107, score-0.506]
58 The correspondences used for calibration are not used for any other step. [sent-113, score-0.246]
59 Episolar Integration Given shadow correspondences C across a variety of lighting directions, the episolar constraint yields a depth inference process which can be cast as a constrained convex program: argd,mαiny? [sent-118, score-1.681]
60 Since our goal is to recover the depths d, we can again express the optimal αx∗y in terms of the following linear system: Ltαx∗y = rydy − rxdx (5) αx∗y = Lt? [sent-126, score-0.286]
61 Given a few ground truth correspondences (examples shown in (a)), we find the camera position most consistent with those correspondences (b). [sent-129, score-0.485]
62 That is, if some pixel y casts a shadow onto both x and x? [sent-139, score-0.943]
63 Because this process solves for a depth surface consistent with a set of depth differences, we denote the optimization in Equation 7 as episolar integration. [sent-146, score-0.735]
64 Correspondence Generation Although the episolar line reduces the search space for shadow correspondence to be along a line, it remains an open problem to robustly link a shadow to its caster. [sent-148, score-2.02]
65 From top to bottom, we show a crop from an example image, its shadow mask, and the extracted shadow correspondences (for two images). [sent-151, score-1.616]
66 Correspondences are shown as connections (yellow line) between an occluder (red point) and its shadow (blue). [sent-153, score-0.77]
67 Notice that the episolar line provides enough constraints to overcome the aperture problem, common in other correspondence problems. [sent-155, score-0.68]
68 Although there is never a time when the red point directly casts a shadow onto the blue point (a), there are enough intermediate constraints (b)-(f) to implicitly constrain the relative depths of the two points, and all intermediate points involved (green). [sent-158, score-0.985]
69 Given an input sequence of imagery from a diverse set of lighting directions, we first apply an in-house shadow estimation approach which returns a shadow-or-not label for all pixels in sequence. [sent-160, score-0.831]
70 When this method classifies some pixel y on a shadow edge as under direct illumination at time t, our goal is to find which pixel—if any—receives the shadow produced by y. [sent-161, score-1.456]
71 We employ a greedy strategy by taking incremental steps along the episolar line emerging from y. [sent-162, score-0.506]
72 , contiguous lit regions only generate correspondences on their edges). [sent-168, score-0.261]
73 In natural scenes, shadow correspondences tend to start in the same locations in the images (rooflines, convexities in mountain ridges, etc. [sent-170, score-0.89]
74 First, if a shadow is cast on the ground far away from its occluder, as in Figure 6(a), correspondences will be gener- (a)(b) (c)(d) (e)(f) Figure 6. [sent-188, score-1.04]
75 For all lit pixels on shadow boundaries, we follow their episolar lines until we find another pixel which is directly illuminated (three examples shown). [sent-191, score-1.312]
76 From here, we remove any correspondence that starts or ends in an unlikely place (c), detail crop in (d) (All correspondences marked in green are kept, red are removed; see text for details). [sent-192, score-0.349]
77 In this case, all correspondences that start on the ground are removed. [sent-193, score-0.219]
78 ated from one side of the cast shadow to the other. [sent-195, score-0.798]
79 Second, the initial rule will stop many correspondences at geometry edges when the background is lit and the foreground is not, as in Figure 6(f). [sent-197, score-0.343]
80 These false correspondences will be filtered out because it is rare for a true correspondence to stop in the same place repeatedly. [sent-198, score-0.337]
81 To recover lighting directions, we use the solar position algorithm from [20]. [sent-202, score-0.241]
82 The correspondences recovered from this scene (b) are rich enough to extract a depth map (c) very close to the ground truth (d). [sent-205, score-0.395]
83 Notice that our approach reliably extracts depth from a variety of complicated geometry and that although the resulting depth map is sparse, the network of constraints covers a large portion of the scene. [sent-210, score-0.354]
84 Our runtime is largely dependent on the complexity of the shadow masks and image resolution, but we report timing with respect to a camera with 135,000 pixels on a 2. [sent-212, score-0.837]
85 The most timeconsuming aspect is in computing the shadow masks, which took 4m40s. [sent-214, score-0.707]
86 Creating and filtering correspondences takes another 42 seconds, and solving for depths took 23 seconds. [sent-215, score-0.241]
87 Our recovered depth surface is almost exactly the ground truth. [sent-217, score-0.234]
88 After that, shadows that cast onto new parts (a) Example image (b) Recovered correspondences (c)Recover d epth(m)(d)Groundtruthdepth(m) Figure 9. [sent-224, score-0.46]
89 Given a sequence of images (example in (a)), we recover shadow correspondences (b) and a depth map (c). [sent-226, score-1.054]
90 For example, the shadow labeling between the tip of a vertical pole to its shadow on the ground plane will almost certainly not be entirely shaded, thus creating a false correspondence. [sent-239, score-1.492]
91 We anticipate that enforcing appearance similarity priors for nearby lighting directions will help leverage correspondence generation for more complicated cases. [sent-240, score-0.228]
92 Our approach only gives a sparse representation of the depth, reconstructing the depths of pixels which cast a shadow or had shadows cast onto them. [sent-242, score-1.159]
93 While this network of constraints still covers a large portion of the image, an ideal solution would merge this constraint with other depth inference processes such as outdoor photometric stereo [1, 3] or shape-from-clouds [10] to “fill in the gaps. [sent-243, score-0.394]
94 ” In this paper, we present an approach for recovering the depth surface of an outdoor scene by treating the sun as a second camera and establishing correspondences between a shadow and its caster. [sent-244, score-1.269]
95 This provides a nonlocal depth integration algorithm, as well as an image-space constraint which dictates which potential correspondences are geometrically feasible. [sent-245, score-0.408]
96 These constraints are particularly useful for shape reconstruction, because the correspondence step does not suffer from the aperture problem, and our derivation makes no assumptions on the shape of the depth surface. [sent-246, score-0.329]
97 A method for 3d scene recognition using shadow information and a single fixed viewpoint. [sent-276, score-0.727]
98 Camera calibration and light source orientation from solar shadows. [sent-281, score-0.211]
99 Using cloud shadows to infer scene structure and camera calibration. [sent-313, score-0.22]
100 Shape reconstruction and camera self-calibration using cast shadows and scene geometries. [sent-341, score-0.311]
wordName wordTfidf (topN-words)
[('shadow', 0.707), ('episolar', 0.459), ('correspondences', 0.183), ('solar', 0.125), ('casts', 0.125), ('depth', 0.117), ('shadows', 0.117), ('correspondence', 0.1), ('cast', 0.091), ('rxdx', 0.088), ('camera', 0.083), ('lit', 0.078), ('rydy', 0.071), ('onto', 0.069), ('lighting', 0.069), ('outdoor', 0.068), ('calibration', 0.063), ('occluder', 0.063), ('depths', 0.058), ('abrams', 0.058), ('constraint', 0.055), ('conference', 0.053), ('aperture', 0.048), ('recover', 0.047), ('line', 0.047), ('shaded', 0.044), ('photometric', 0.043), ('plane', 0.042), ('surface', 0.042), ('pixel', 0.042), ('connectedness', 0.041), ('recovered', 0.039), ('lt', 0.039), ('webcams', 0.039), ('carving', 0.039), ('rx', 0.039), ('geometry', 0.036), ('ground', 0.036), ('bamber', 0.035), ('directio', 0.035), ('epth', 0.035), ('hatzitheodorou', 0.035), ('ines', 0.035), ('ltlt', 0.035), ('pend', 0.035), ('pstart', 0.035), ('network', 0.035), ('calibrate', 0.035), ('earth', 0.033), ('directions', 0.032), ('kawasaki', 0.031), ('amos', 0.031), ('shafer', 0.031), ('raviv', 0.031), ('geometrically', 0.03), ('imagery', 0.029), ('timestamps', 0.029), ('place', 0.028), ('ry', 0.028), ('substitution', 0.027), ('generation', 0.027), ('sky', 0.027), ('stereo', 0.027), ('unknown', 0.026), ('constraints', 0.026), ('drawings', 0.026), ('frustum', 0.026), ('stop', 0.026), ('pixels', 0.026), ('jacobs', 0.026), ('sun', 0.026), ('epipolar', 0.025), ('notice', 0.025), ('international', 0.024), ('xy', 0.024), ('portion', 0.023), ('lowe', 0.023), ('away', 0.023), ('light', 0.023), ('treating', 0.023), ('nonlocal', 0.023), ('wand', 0.023), ('ray', 0.022), ('cue', 0.022), ('nonlinear', 0.022), ('express', 0.022), ('savarese', 0.022), ('threedimensional', 0.022), ('masks', 0.021), ('visualizing', 0.021), ('intersects', 0.021), ('scene', 0.02), ('derive', 0.02), ('rule', 0.02), ('calibrating', 0.02), ('remove', 0.019), ('pinhole', 0.019), ('leveraged', 0.019), ('shape', 0.019), ('crop', 0.019)]
simIndex simValue paperId paperTitle
same-paper 1 0.9999997 428 cvpr-2013-The Episolar Constraint: Monocular Shape from Shadow Correspondence
Author: Austin Abrams, Kylia Miskell, Robert Pless
Abstract: Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from webcams “in . wu st l. edu (b)(c) the wild.”
2 0.12090281 138 cvpr-2013-Efficient 2D-to-3D Correspondence Filtering for Scalable 3D Object Recognition
Author: Qiang Hao, Rui Cai, Zhiwei Li, Lei Zhang, Yanwei Pang, Feng Wu, Yong Rui
Abstract: 3D model-based object recognition has been a noticeable research trend in recent years. Common methods find 2D-to-3D correspondences and make recognition decisions by pose estimation, whose efficiency usually suffers from noisy correspondences caused by the increasing number of target objects. To overcome this scalability bottleneck, we propose an efficient 2D-to-3D correspondence filtering approach, which combines a light-weight neighborhoodbased step with a finer-grained pairwise step to remove spurious correspondences based on 2D/3D geometric cues. On a dataset of 300 3D objects, our solution achieves ∼10 times speed improvement over the baseline, with a comparable recognition accuracy. A parallel implementation on a quad-core CPU can run at ∼3fps for 1280× 720 images.
3 0.11570156 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
Author: Ju Shen, Sen-Ching S. Cheung
Abstract: The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature.
4 0.10370445 397 cvpr-2013-Simultaneous Super-Resolution of Depth and Images Using a Single Camera
Author: Hee Seok Lee, Kuoung Mu Lee
Abstract: In this paper, we propose a convex optimization framework for simultaneous estimation of super-resolved depth map and images from a single moving camera. The pixel measurement error in 3D reconstruction is directly related to the resolution of the images at hand. In turn, even a small measurement error can cause significant errors in reconstructing 3D scene structure or camera pose. Therefore, enhancing image resolution can be an effective solution for securing the accuracy as well as the resolution of 3D reconstruction. In the proposed method, depth map estimation and image super-resolution are formulated in a single energy minimization framework with a convex function and solved efficiently by a first-order primal-dual algorithm. Explicit inter-frame pixel correspondences are not required for our super-resolution procedure, thus we can avoid a huge computation time and obtain improved depth map in the accuracy and resolution as well as highresolution images with reasonable time. The superiority of our algorithm is demonstrated by presenting the improved depth map accuracy, image super-resolution results, and camera pose estimation.
5 0.099929705 466 cvpr-2013-Whitened Expectation Propagation: Non-Lambertian Shape from Shading and Shadow
Author: Brian Potetz, Mohammadreza Hajiarbabi
Abstract: For problems over continuous random variables, MRFs with large cliques pose a challenge in probabilistic inference. Difficulties in performing optimization efficiently have limited the probabilistic models explored in computer vision and other fields. One inference technique that handles large cliques well is Expectation Propagation. EP offers run times independent of clique size, which instead depend only on the rank, or intrinsic dimensionality, of potentials. This property would be highly advantageous in computer vision. Unfortunately, for grid-shaped models common in vision, traditional Gaussian EP requires quadratic space and cubic time in the number of pixels. Here, we propose a variation of EP that exploits regularities in natural scene statistics to achieve run times that are linear in both number of pixels and clique size. We test these methods on shape from shading, and we demonstrate strong performance not only for Lambertian surfaces, but also on arbitrary surface reflectance and lighting arrangements, which requires highly non-Gaussian potentials. Finally, we use large, non-local cliques to exploit cast shadow, which is traditionally ignored in shape from shading.
6 0.096446462 117 cvpr-2013-Detecting Changes in 3D Structure of a Scene from Multi-view Images Captured by a Vehicle-Mounted Camera
7 0.094789833 290 cvpr-2013-Motion Estimation for Self-Driving Cars with a Generalized Camera
8 0.091940865 361 cvpr-2013-Robust Feature Matching with Alternate Hough and Inverted Hough Transforms
9 0.089460798 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
10 0.086839691 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
11 0.086775273 423 cvpr-2013-Template-Based Isometric Deformable 3D Reconstruction with Sampling-Based Focal Length Self-Calibration
12 0.082504913 360 cvpr-2013-Robust Estimation of Nonrigid Transformation for Point Set Registration
13 0.081502408 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
14 0.081437722 76 cvpr-2013-Can a Fully Unconstrained Imaging Model Be Applied Effectively to Central Cameras?
15 0.075254411 380 cvpr-2013-Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
16 0.075132221 84 cvpr-2013-Cloud Motion as a Calibration Cue
17 0.073863201 111 cvpr-2013-Dense Reconstruction Using 3D Object Shape Priors
18 0.071743362 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
19 0.071572915 465 cvpr-2013-What Object Motion Reveals about Shape with Unknown BRDF and Lighting
20 0.071462363 108 cvpr-2013-Dense 3D Reconstruction from Severely Blurred Images Using a Single Moving Camera
topicId topicWeight
[(0, 0.131), (1, 0.155), (2, 0.02), (3, 0.037), (4, -0.002), (5, -0.066), (6, -0.059), (7, 0.041), (8, 0.024), (9, 0.009), (10, -0.026), (11, 0.016), (12, 0.039), (13, 0.031), (14, -0.023), (15, -0.069), (16, -0.015), (17, 0.048), (18, 0.002), (19, -0.026), (20, 0.035), (21, -0.055), (22, -0.017), (23, -0.015), (24, 0.054), (25, -0.074), (26, -0.049), (27, 0.003), (28, -0.017), (29, 0.038), (30, -0.003), (31, -0.014), (32, 0.016), (33, 0.04), (34, 0.046), (35, 0.026), (36, -0.028), (37, 0.016), (38, 0.063), (39, 0.07), (40, 0.014), (41, -0.022), (42, 0.001), (43, -0.015), (44, -0.031), (45, -0.01), (46, 0.028), (47, 0.03), (48, 0.04), (49, 0.002)]
simIndex simValue paperId paperTitle
same-paper 1 0.94869119 428 cvpr-2013-The Episolar Constraint: Monocular Shape from Shadow Correspondence
Author: Austin Abrams, Kylia Miskell, Robert Pless
Abstract: Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from webcams “in . wu st l. edu (b)(c) the wild.”
2 0.66869617 423 cvpr-2013-Template-Based Isometric Deformable 3D Reconstruction with Sampling-Based Focal Length Self-Calibration
Author: Adrien Bartoli, Toby Collins
Abstract: It has been shown that a surface deforming isometrically can be reconstructed from a single image and a template 3D shape. Methods from the literature solve this problem efficiently. However, they all assume that the camera model is calibrated, which drastically limits their applicability. We propose (i) a general variational framework that applies to (calibrated and uncalibrated) general camera models and (ii) self-calibrating 3D reconstruction algorithms for the weak-perspective and full-perspective camera models. In the former case, our algorithm returns the normal field and camera ’s scale factor. In the latter case, our algorithm returns the normal field, depth and camera ’s focal length. Our algorithms are the first to achieve deformable 3D reconstruction including camera self-calibration. They apply to much more general setups than existing methods. Experimental results on simulated and real data show that our algorithms give results with the same level of accuracy as existing methods (which use the true focal length) on perspective images, and correctly find the normal field on affine images for which the existing methods fail.
3 0.6652326 361 cvpr-2013-Robust Feature Matching with Alternate Hough and Inverted Hough Transforms
Author: Hsin-Yi Chen, Yen-Yu Lin, Bing-Yu Chen
Abstract: We present an algorithm that carries out alternate Hough transform and inverted Hough transform to establish feature correspondences, and enhances the quality of matching in both precision and recall. Inspired by the fact that nearby features on the same object share coherent homographies in matching, we cast the task of feature matching as a density estimation problem in the Hough space spanned by the hypotheses of homographies. Specifically, we project all the correspondences into the Hough space, and determine the correctness of the correspondences by their respective densities. In this way, mutual verification of relevant correspondences is activated, and the precision of matching is boosted. On the other hand, we infer the concerted homographies propagated from the locally grouped features, and enrich the correspondence candidates for each feature. The recall is hence increased. The two processes are tightly coupled. Through iterative optimization, plausible enrichments are gradually revealed while more correct correspondences are detected. Promising experimental results on three benchmark datasets manifest the effectiveness of the proposed approach.
4 0.66282725 138 cvpr-2013-Efficient 2D-to-3D Correspondence Filtering for Scalable 3D Object Recognition
Author: Qiang Hao, Rui Cai, Zhiwei Li, Lei Zhang, Yanwei Pang, Feng Wu, Yong Rui
Abstract: 3D model-based object recognition has been a noticeable research trend in recent years. Common methods find 2D-to-3D correspondences and make recognition decisions by pose estimation, whose efficiency usually suffers from noisy correspondences caused by the increasing number of target objects. To overcome this scalability bottleneck, we propose an efficient 2D-to-3D correspondence filtering approach, which combines a light-weight neighborhoodbased step with a finer-grained pairwise step to remove spurious correspondences based on 2D/3D geometric cues. On a dataset of 300 3D objects, our solution achieves ∼10 times speed improvement over the baseline, with a comparable recognition accuracy. A parallel implementation on a quad-core CPU can run at ∼3fps for 1280× 720 images.
5 0.65186942 290 cvpr-2013-Motion Estimation for Self-Driving Cars with a Generalized Camera
Author: Gim Hee Lee, Friedrich Faundorfer, Marc Pollefeys
Abstract: In this paper, we present a visual ego-motion estimation algorithm for a self-driving car equipped with a closeto-market multi-camera system. By modeling the multicamera system as a generalized camera and applying the non-holonomic motion constraint of a car, we show that this leads to a novel 2-point minimal solution for the generalized essential matrix where the full relative motion including metric scale can be obtained. We provide the analytical solutions for the general case with at least one inter-camera correspondence and a special case with only intra-camera correspondences. We show that up to a maximum of 6 solutions exist for both cases. We identify the existence of degeneracy when the car undergoes straight motion in the special case with only intra-camera correspondences where the scale becomes unobservable and provide a practical alternative solution. Our formulation can be efficiently implemented within RANSAC for robust estimation. We verify the validity of our assumptions on the motion model by comparing our results on a large real-world dataset collected by a car equipped with 4 cameras with minimal overlapping field-of-views against the GPS/INS ground truth.
6 0.64766198 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
7 0.63653415 279 cvpr-2013-Manhattan Scene Understanding via XSlit Imaging
8 0.62350714 397 cvpr-2013-Simultaneous Super-Resolution of Depth and Images Using a Single Camera
9 0.62315291 283 cvpr-2013-Megastereo: Constructing High-Resolution Stereo Panoramas
10 0.62120408 117 cvpr-2013-Detecting Changes in 3D Structure of a Scene from Multi-view Images Captured by a Vehicle-Mounted Camera
11 0.61692703 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
12 0.59401345 27 cvpr-2013-A Theory of Refractive Photo-Light-Path Triangulation
13 0.59321374 74 cvpr-2013-CLAM: Coupled Localization and Mapping with Efficient Outlier Handling
14 0.58605003 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
15 0.58039576 56 cvpr-2013-Bayesian Depth-from-Defocus with Shading Constraints
16 0.57171422 76 cvpr-2013-Can a Fully Unconstrained Imaging Model Be Applied Effectively to Central Cameras?
17 0.56864434 47 cvpr-2013-As-Projective-As-Possible Image Stitching with Moving DLT
18 0.56637579 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
19 0.56520432 360 cvpr-2013-Robust Estimation of Nonrigid Transformation for Point Set Registration
20 0.56026995 447 cvpr-2013-Underwater Camera Calibration Using Wavelength Triangulation
topicId topicWeight
[(10, 0.081), (16, 0.05), (26, 0.037), (28, 0.317), (33, 0.253), (67, 0.028), (69, 0.032), (87, 0.092)]
simIndex simValue paperId paperTitle
1 0.81828469 192 cvpr-2013-Graph Matching with Anchor Nodes: A Learning Approach
Author: Nan Hu, Raif M. Rustamov, Leonidas Guibas
Abstract: In this paper, we consider the weighted graph matching problem with partially disclosed correspondences between a number of anchor nodes. Our construction exploits recently introduced node signatures based on graph Laplacians, namely the Laplacian family signature (LFS) on the nodes, and the pairwise heat kernel map on the edges. In this paper, without assuming an explicit form of parametric dependence nor a distance metric between node signatures, we formulate an optimization problem which incorporates the knowledge of anchor nodes. Solving this problem gives us an optimized proximity measure specific to the graphs under consideration. Using this as a first order compatibility term, we then set up an integer quadratic program (IQP) to solve for a near optimal graph matching. Our experiments demonstrate the superior performance of our approach on randomly generated graphs and on two widelyused image sequences, when compared with other existing signature and adjacency matrix based graph matching methods.
same-paper 2 0.81053609 428 cvpr-2013-The Episolar Constraint: Monocular Shape from Shadow Correspondence
Author: Austin Abrams, Kylia Miskell, Robert Pless
Abstract: Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from webcams “in . wu st l. edu (b)(c) the wild.”
3 0.79482675 261 cvpr-2013-Learning by Associating Ambiguously Labeled Images
Author: Zinan Zeng, Shijie Xiao, Kui Jia, Tsung-Han Chan, Shenghua Gao, Dong Xu, Yi Ma
Abstract: We study in this paper the problem of learning classifiers from ambiguously labeled images. For instance, in the collection of new images, each image contains some samples of interest (e.g., human faces), and its associated caption has labels with the true ones included, while the samplelabel association is unknown. The task is to learn classifiers from these ambiguously labeled images and generalize to new images. An essential consideration here is how to make use of the information embedded in the relations between samples and labels, both within each image and across the image set. To this end, we propose a novel framework to address this problem. Our framework is motivated by the observation that samples from the same class repetitively appear in the collection of ambiguously labeled training images, while they are just ambiguously labeled in each image. If we can identify samples of the same class from each image and associate them across the image set, the matrix formed by the samples from the same class would be ideally low-rank. By leveraging such a low-rank assump- tion, we can simultaneously optimize a partial permutation matrix (PPM) for each image, which is formulated in order to exploit all information between samples and labels in a principled way. The obtained PPMs can be readily used to assign labels to samples in training images, and then a standard SVM classifier can be trained and used for unseen data. Experiments on benchmark datasets show the effectiveness of our proposed method.
4 0.77767134 134 cvpr-2013-Discriminative Sub-categorization
Author: Minh Hoai, Andrew Zisserman
Abstract: The objective of this work is to learn sub-categories. Rather than casting this as a problem of unsupervised clustering, we investigate a weakly supervised approach using both positive and negative samples of the category. We make the following contributions: (i) we introduce a new model for discriminative sub-categorization which determines cluster membership for positive samples whilst simultaneously learning a max-margin classifier to separate each cluster from the negative samples; (ii) we show that this model does not suffer from the degenerate cluster problem that afflicts several competing methods (e.g., Latent SVM and Max-Margin Clustering); (iii) we show that the method is able to discover interpretable sub-categories in various datasets. The model is evaluated experimentally over various datasets, and itsperformance advantages over k-means and Latent SVM are demonstrated. We also stress test the model and show its resilience in discovering sub-categories as the parameters are varied.
5 0.77363944 4 cvpr-2013-3D Visual Proxemics: Recognizing Human Interactions in 3D from a Single Image
Author: Ishani Chakraborty, Hui Cheng, Omar Javed
Abstract: We present a unified framework for detecting and classifying people interactions in unconstrained user generated images. 1 Unlike previous approaches that directly map people/face locations in 2D image space into features for classification, we first estimate camera viewpoint and people positions in 3D space and then extract spatial configuration features from explicit 3D people positions. This approach has several advantages. First, it can accurately estimate relative distances and orientations between people in 3D. Second, it encodes spatial arrangements of people into a richer set of shape descriptors than afforded in 2D. Our 3D shape descriptors are invariant to camera pose variations often seen in web images and videos. The proposed approach also estimates camera pose and uses it to capture the intent of the photo. To achieve accurate 3D people layout estimation, we develop an algorithm that robustly fuses semantic constraints about human interpositions into a linear camera model. This enables our model to handle large variations in people size, heights (e.g. age) and poses. An accurate 3D layout also allows us to construct features informed by Proxemics that improves our semantic classification. To characterize the human interaction space, we introduce visual proxemes; a set of prototypical patterns that represent commonly occurring social interactions in events. We train a discriminative classifier that classifies 3D arrangements of people into visual proxemes and quantitatively evaluate the performance on a large, challenging dataset.
6 0.76181531 328 cvpr-2013-Pedestrian Detection with Unsupervised Multi-stage Feature Learning
7 0.74590659 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
8 0.73513448 234 cvpr-2013-Joint Spectral Correspondence for Disparate Image Matching
9 0.690593 206 cvpr-2013-Human Pose Estimation Using Body Parts Dependent Joint Regressors
10 0.69058079 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
11 0.68853778 155 cvpr-2013-Exploiting the Power of Stereo Confidences
12 0.6881417 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
13 0.68707883 19 cvpr-2013-A Minimum Error Vanishing Point Detection Approach for Uncalibrated Monocular Images of Man-Made Environments
15 0.68533319 284 cvpr-2013-Mesh Based Semantic Modelling for Indoor and Outdoor Scenes
16 0.68505502 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
17 0.67932832 54 cvpr-2013-BRDF Slices: Accurate Adaptive Anisotropic Appearance Acquisition
18 0.67907709 104 cvpr-2013-Deep Convolutional Network Cascade for Facial Point Detection
19 0.6771906 432 cvpr-2013-Three-Dimensional Bilateral Symmetry Plane Estimation in the Phase Domain
20 0.67632854 299 cvpr-2013-Multi-source Multi-scale Counting in Extremely Dense Crowd Images