cvpr cvpr2013 cvpr2013-394 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Lap-Fai Yu, Sai-Kit Yeung, Yu-Wing Tai, Stephen Lin
Abstract: We present a shading-based shape refinement algorithm which uses a noisy, incomplete depth map from Kinect to help resolve ambiguities in shape-from-shading. In our framework, the partial depth information is used to overcome bas-relief ambiguity in normals estimation, as well as to assist in recovering relative albedos, which are needed to reliably estimate the lighting environment and to separate shading from albedo. This refinement of surface normals using a noisy depth map leads to high-quality 3D surfaces. The effectiveness of our algorithm is demonstrated through several challenging real-world examples.
Reference: text
sentIndex sentText sentNum sentScore
1 In our framework, the partial depth information is used to overcome bas-relief ambiguity in normals estimation, as well as to assist in recovering relative albedos, which are needed to reliably estimate the lighting environment and to separate shading from albedo. [sent-2, score-1.154]
2 This refinement of surface normals using a noisy depth map leads to high-quality 3D surfaces. [sent-3, score-0.979]
3 In this paper, we propose a shading-based shape refinement algorithm that utilizes Microsoft Kinect to address the ambiguities that exist among lighting, normals and albedo. [sent-12, score-0.618]
4 The Kinect records each RGB image together with a depth map. [sent-13, score-0.296]
5 Although the depth map is noisy and typically contains holes1 , we present a method that effec∗This work was done while Lap-Fai Yu was a visiting student at MSRA and at SUTD. [sent-14, score-0.423]
6 1Depth map holes result from scene areas in a Kinect depth image outside the depth sensing range or occluded from the infrared light projections, since the infrared projection and sensing directions are not the same. [sent-15, score-0.946]
7 Our shading-based shape refinement deals with the shape and reflectance ambiguities of SfS while effectively enhancing surface normals computed from the raw, noisy depth data of Kinect. [sent-17, score-1.14]
8 The depth information not only helps to resolve bas-relief ambiguity, but also aids in clustering pixels with similar normal directions. [sent-19, score-0.585]
9 Such grouping allows us to effectively estimate relative albedos and the environment illumination in terms of spherical harmonics. [sent-20, score-0.61]
10 To handle the holes in a depth map, we use edges from the RGB image to guide a structure-preserving hole filling process and create a reliable depth map proxy for our shading-based shape refinement algorithm. [sent-21, score-1.221]
11 The utilization of a noisy, incomplete depth map in our approach leads to high-quality 3D scene reconstruction, as exemplified in Figure 1. [sent-22, score-0.381]
12 Related Work Our work is related to SfS and depth map enhancement. [sent-24, score-0.381]
13 with the same reflectance properties as the target object to model the object’s shading under the environment illumination [16]. [sent-29, score-0.396]
14 Huang and Smith [7] interpolate the boundary normals of an object to obtain a rough shape prior to constrain SfS. [sent-36, score-0.506]
15 With regard to depth map enhancement, recent advances use an additional RGB image to denoise and upsample a depth map [20, 3, 13]. [sent-42, score-0.762]
16 With an RGB image that has a higher resolution and signal-to-noise ratio than the depth map, a direct approach is to apply a joint bilateral filter [20, 3] using the RGB image to define a neighborhood smoothness term. [sent-43, score-0.335]
17 formulate this as an optimization problem and show that with a small amount of user interaction, the depth map can be greatly improved. [sent-45, score-0.381]
18 But while these depth map enhancement methods can reduce noise and increase resolution, they also lose fine depth details during the smoothing process. [sent-46, score-0.7]
19 By contrast, our approach recovers fine depth details even if they are not captured in the initial noisy depth map, by making greater use of the RGB image through an analysis of its shading. [sent-47, score-0.634]
20 Depth-assisted SfS Approach To facilitate SfS, our approach utilizes partial depth information to separate shading from albedo, aid illumination estimation, and resolve surface normal ambiguity. [sent-49, score-0.834]
21 From the input RGB image and depth map, our method first computes a normal map from the captured depth map and segments the RGB image into regions of piecewise smooth color. [sent-54, score-0.961]
22 Through alternating optimization (AO), the relative albedos among the different regions are calculated, and the environment illumination is estimated from the albedo-normalized image. [sent-55, score-0.639]
23 After that, we estimate normals over the whole image using SfS with the help of a normal map computed from Kinect as a shape prior to resolve bas-relief ambiguity. [sent-56, score-0.826]
24 For regions that lack depth map values from Kinect, we use a constrained texture synthesis to fill in the missing depth values prior to applying our normal estimation algorithm. [sent-57, score-0.99]
25 As shown in Figure 1, the output of our method is a refined normal map without the shape and reflectance ambiguities of SfS nor the noise and holes of the Kinect range data. [sent-58, score-0.568]
26 Relative Albedo and Lighting Estimation The input from Kinect consists of an RGB image I = {Ii}, Ii = [Ii,r, Ii,g, Ii,b]T where iis the pixel index, and a depth map. [sent-61, score-0.296]
27 From the point cloud determined from the depth map, we calculate a rough normal map N = {ni}, where ni =, [ni,x, ni,y, en ai,z r]oTu gish tnhoer munalit m mnaoprm Nal =at pixel iw, oerbetained by a simple cross-product of the neighboring points. [sent-62, score-0.645]
28 For pixels with missing depth values, or whose neighboring pixels have any missing depth values, no initial normal is computed. [sent-63, score-0.851]
29 With this property, we solve for the relative albedos between different clusters using pixel-pairs of common normals from different clusters. [sent-73, score-0.886]
30 2 Data Structure To facilitate normal direction comparisons among clusters, we quantize all possible normal directions to vertices on an icosahedron, which provides a uniformly-distributed set of T = 642 normal directions over a sphere. [sent-77, score-0.685]
31 The normals in an image are stored in a data structure Bu,j,k which we refer to as bins, where u = 1, . [sent-78, score-0.388]
32 , 3 with Bu,j,0 as an indicator bit of whether the j-th normal direction exists within cluster Cu, and [Bu,j,1 , Bu,j,2, Bu,j,3] store the RGB values corresponding to the j-th normal direction in cluster Cu. [sent-87, score-0.482]
33 Then, for each cluster Cu, each normal n falls into a bin Bu,t,k, where n has the smallest dot-product with the t-th normal direction among all the T normal directions on the icosahedron. [sent-89, score-0.717]
34 If there are multiple pixels having normals that fall into the same bin Bu,t,k, the median of their RGB values is used. [sent-92, score-0.452]
35 hA,n G edge Eu1 e Exaischts c bleutswteeren C cluster Cu1 and Cu2 only if there are more than λ common normal directions between clusters Cu1 and Cu2 , with λ = 20 in our experiments. [sent-97, score-0.371]
36 After the MST is found, we calculate the relative albedos between all of its clusters in a depth-first search order along the tree. [sent-103, score-0.498]
37 The relative albedo between two clusters is computed by first determining the common bins (corresponding to common normals) utilized in clusters Cu1 and Cu2 , denoted by Q = {q : Bu1 ,q,0 = 1and Bu2,q,0 = 1}. [sent-104, score-0.454]
38 Pseudocode of this relative albedo estimation procedure is provided in the supplementary material. [sent-107, score-0.32]
39 4 Lighting Estimation The estimated relative albedos are highly useful. [sent-110, score-0.436]
40 By normalizing the albedos in different regions, we can then jointly use their rich variety of normal directions to more reliably estimate the environment lighting. [sent-111, score-0.711]
41 Suppose there are R pixels whose relative albedos are estimatedfromthe MST, and let nˆ i = [niT 1]T. [sent-112, score-0.442]
42 Using the RGB image I and initial normal map N computed from Kinect, Mk in (1) can be estimated up to a scale factor by linear least-squares minimization. [sent-117, score-0.308]
43 (a) Input image, (b) & (c) Relative albedos estimated at the 1st and 5th iteration, (d) & (e) Corresponding shading images at the 1st and 5th iteration. [sent-120, score-0.497]
44 Clusters without relative albedos in the 1st iteration are simply filled by original RGB values in (b). [sent-121, score-0.435]
45 5 Refinement by Alternating Optimization With the estimated lighting, we refine the relative albedos and calculate the relative albedos of those clusters not yet estimated. [sent-124, score-0.934]
46 For each cluster, an estimate of relative albedo for each RGB channel k is obtained for each normal the cluster as: pi,k= nˆiTIMi,kk nˆi. [sent-125, score-0.523]
47 ni in (2) RANSAC is again run on these estimates to obtain an updated relative albedo for each cluster. [sent-126, score-0.323]
48 Using the updated relative albedos of the MST clusters, we re-estimate the SH coefficients by (1). [sent-127, score-0.412]
49 We note that despite the noisy normals of the depth map, the relative albedos between two regions can be reliably determined when they have many normals in common, as is the case for connected nodes in the MST. [sent-131, score-1.551]
50 Moreover, the environment lighting can also be dependably recovered when the number and range of noisy normals is large, as again is the case with the MST. [sent-132, score-0.62]
51 In our work, we exploit the Kinect RGB-D data to obtain a structure-preserving shape prior, in the form of prior normals to be used later in a normal refinement step. [sent-138, score-0.788]
52 Kinect depth maps, however, frequently contain holes where there is no depth information for directly computing surface normals. [sent-139, score-0.742]
53 Right: by accounting for prior normals, the bed normals are correctly pointing upward. [sent-142, score-0.431]
54 Though holes may exist in the depth image, they do not appear in the corresponding RGB image. [sent-144, score-0.385]
55 We thus take advantage of the RGB image as a guide for depth completion in the hole region. [sent-145, score-0.596]
56 We then identify RGB edges that pass through a hole, referred to as a structural hole, in the depth image. [sent-147, score-0.296]
57 Along the edge, we generate hole patches which contain hole pixels whose depths need to be obtained, and known patches which contain no hole and are used for repairing the hole patches. [sent-148, score-1.312]
58 (a) Input RGB, (b) Input depth, (c) Depth gradient map, (d) Depth gradient map after patch repair, (e) Depth map after patch repair and poisson integration, (f) Prior normal map, (g) Resulting normal map after SfS. [sent-152, score-0.901]
59 Patch-based repairing allows propagation of existing structure to the hole region. [sent-155, score-0.409]
60 (c) & (d) Resultant normals using shape prior from (a) & (b). [sent-158, score-0.482]
61 The goal is to transfer the depth gradients from the known patches to the hole patches, after which the depth of the hole can be filled in by poisson integration while preserving the structure along the edge. [sent-159, score-1.289]
62 Denote the set of hole patches as H = {Hl } and the set of Dkneonwonte patches as oKl = p {tcKhmes} a. [sent-168, score-0.362]
63 RGB Data Cost: Let ZDrgb denote the number of pixels covered by hole patches. [sent-173, score-0.302]
64 The RGB data cost is defined so that the selected known patch closely matches the hole patch in the RGB image: CDrgb(H) =3ZD1rgb? [sent-174, score-0.385]
65 (4) Depth Gradient Data Cost: Let ZDdg be the number of non-hole pixels covered by hole patches, and D? [sent-179, score-0.302]
66 Since these pixels have depth values, their depth gradients can be calculated. [sent-181, score-0.654]
67 The depth gradient data cost favors solutions in which the computed depth gradients closely match the original depth gradients for the non-hole pixels: CDdg(H) =2ZD1dg? [sent-182, score-1.019]
68 a depth gradient (6) RGB Smoothness Cost: Suppose {Hl1, Hl2} is a pair of oRvGeBrlap Spminogothhonleespsat Cchoesst,: andKH−l11 andKH−l21 r eissp ae pcativireol yf denote their repairing known patches. [sent-190, score-0.432]
69 With Zov being the number of pixels in the overlapping regions of hole patches, we penalize solutions where the overlapping RGB values are inconsistent: CSrgb(H) =3Z1ov{H? [sent-192, score-0.302]
70 (7) Depth Gradient Smoothness Cost: Similar to the RGB smoothness cost, we have a corresponding cost for the depth gradient image: CSdg(H) =2Z1ov{H? [sent-197, score-0.402]
71 (8) After belief propagation is performed to minimize CBP (H), depth gradients of pixels within hole patches are replaced by depth gradients from the assigned known patches. [sent-204, score-1.036]
72 With the transferred depth gradients and the known depth values along the hole boundary as boundary conditions, poisson integration [15] is used to fill in the depth values of the hole. [sent-205, score-1.278]
73 2 Surface Normal Refinement The estimated relative albedos, lighting and shape prior serve as useful inputs for normal refinement over the whole 1 1 14 4 41 1 179 7 scene. [sent-209, score-0.583]
74 The surface normal refinement is formulated as a non-linear optimization using the total energy function: E(N) = wsfsEsfs(N) wpriorEprior(N) (9) + + wsmoothEsmooth(N) + wnormEnorm(N). [sent-211, score-0.367]
75 It constrains the normal according to the shading observed in the RGB image: Esfs(N) =Zto1tal? [sent-213, score-0.335]
76 1,2,3}(Ii,k− pi,k nˆiTMk nˆi)2(10) To resolve bas-relief ambiguity, Eprior(N) constrains the norTmoarlse otol vbeeb asism-rielalire ftoa mthbei prior Enormals computed from the repaired Kinect depth map (see Figure 5). [sent-215, score-0.542]
77 The to- tal energy function, which is non-linear in terms of normals ni, is optimized by the trust-region-reflective algorithm. [sent-234, score-0.388]
78 We initialize the normals to [0, 0, 1]T, facing the camera. [sent-235, score-0.388]
79 Lighting Estimation In Figure 9, we investigate our approach’s ability to estimate environment light in an indoor scene, by comparing it to ground truth obtained using a mirrored sphere convolved with 2nd-order spherical harmonics. [sent-239, score-0.3]
80 It can be observed that using more clusters and normals, which is made possible by the relative albedo estimation, leads to more accurate and robust light estimation. [sent-240, score-0.442]
81 As the normals throughout the MST are used, the major light directions and intensity resemble that obtained from the mirrored sphere. [sent-241, score-0.534]
82 5178), (e) Our estimated normal map, (f) Squared error map of our estimated normals (RMSE=0. [sent-273, score-0.72]
83 Ground Truth Comparison Next we validate our approach by conducting an analytical experiment in which we estimate normals of a Lambertian ball in an indoor scene (named jeans in the supplement). [sent-277, score-0.417]
84 Figure 11 shows the results of our approach in refining the raw normals computed directly from the depth map. [sent-278, score-0.722]
85 2While the RMSE of relative light intensity is in the range [0, 1], the RMSE of normals is in the range [0, 2], as the squared error of normals is in range [0, 4]. [sent-283, score-0.954]
86 For example, normals [0, 0, 1]T and [0, 0, −1]T result in a maximum squared error of 4. [sent-284, score-0.417]
87 In library, the structural holes on the books and shelf are repaired by the propagated patches, and the round surface of the stool is well reconstructed by shading despite the presence of noise and holes in the input depth and normal map. [sent-292, score-0.928]
88 In shoe cabinet, structural propagation enables the proper repair of the hole at the corner, which provides a correct shape prior compared to smoothing (see also Figure 8). [sent-296, score-0.486]
89 (b-d) Our recovered normals and two normal maps N shaded (ase- Ng) · R Lec wovitehre Ld n =orm (−al√s13 a,n√d13 s,ha√1d3ed)T ima ngde Ls o =f [1 (]√1 u3s,in√1g3, ge√1n3e)rTic. [sent-309, score-0.614]
90 Our approach uses only the regions with the highest-confidence relative albedos (from the MST) for lighting estimation, rather than the entire image. [sent-313, score-0.496]
91 Figure 14 compares our albedo normalization result with the state-of-the-art intrinsic image separation technique of [9], which also makes use of Kinect depth data. [sent-315, score-0.503]
92 Their work assumes the input to be a nearly flawless depth map obtained from video streams of a moving Kinect, and does not operate as well with a noisy depth map available from a single Kinect image. [sent-317, score-0.804]
93 In contrast, our technique performs more effective albedo normalization because the relative albedos are obtained with the help of estimated lighting. [sent-318, score-0.643]
94 Discussion High-quality normals are vital prerequisites for different practical applications. [sent-326, score-0.388]
95 Figure 15 shows a point cloud significantly refined with our resultant normals using the method of [10]. [sent-327, score-0.435]
96 In addition, the resultant normals enable realistic re-lighting and high-quality 3D surface reconstruction. [sent-328, score-0.496]
97 Limitations: Like other patch-based image completion methods, the effectiveness of our patch-based hole repairing step is subject to the quality and compatibility of the surrounding known patches. [sent-330, score-0.404]
98 While the RGB data is in general of higher quality than the depth data, its noise can still affect the quality of shape-from-shading. [sent-331, score-0.296]
99 Conclusion: We presented a useful postprocessing method to improve the quality of surface normals obtained from Kinect. [sent-334, score-0.449]
100 In future work, we plan to consider the lighting visibility of scene points based on the depth map, as this should improve the estimation of lighting, relative albedos, and shape-from-shading. [sent-336, score-0.493]
wordName wordTfidf (topN-words)
[('normals', 0.388), ('albedos', 0.337), ('depth', 0.296), ('hole', 0.272), ('rgb', 0.26), ('sfs', 0.24), ('albedo', 0.207), ('normal', 0.199), ('kinect', 0.197), ('shading', 0.136), ('khl', 0.13), ('refinement', 0.107), ('environment', 0.106), ('repairing', 0.104), ('reflectance', 0.097), ('holes', 0.089), ('hl', 0.087), ('mst', 0.086), ('clusters', 0.086), ('map', 0.085), ('lighting', 0.084), ('rmse', 0.079), ('relative', 0.075), ('light', 0.074), ('kh', 0.068), ('surface', 0.061), ('resolve', 0.06), ('repaired', 0.058), ('illumination', 0.057), ('cu', 0.056), ('repair', 0.053), ('poisson', 0.053), ('shape', 0.051), ('resultant', 0.047), ('ambiguities', 0.047), ('tai', 0.046), ('patches', 0.045), ('directions', 0.044), ('lambertian', 0.044), ('ambiguity', 0.044), ('andkh', 0.043), ('bracketed', 0.043), ('cbp', 0.043), ('cddg', 0.043), ('cdrgb', 0.043), ('csdg', 0.043), ('csrgb', 0.043), ('enorm', 0.043), ('eprior', 0.043), ('esfs', 0.043), ('itmk', 0.043), ('prior', 0.043), ('noisy', 0.042), ('cluster', 0.042), ('ni', 0.041), ('upsampling', 0.041), ('alternating', 0.04), ('patch', 0.039), ('smoothness', 0.039), ('icosahedron', 0.039), ('estimation', 0.038), ('raw', 0.038), ('spherical', 0.035), ('cost', 0.035), ('bin', 0.034), ('oxholm', 0.034), ('shoe', 0.034), ('cabinet', 0.034), ('fill', 0.033), ('propagation', 0.033), ('esmooth', 0.032), ('gradients', 0.032), ('gradient', 0.032), ('infrared', 0.031), ('pb', 0.03), ('pixels', 0.03), ('squared', 0.029), ('bedroom', 0.029), ('indoor', 0.029), ('suppose', 0.029), ('sphere', 0.028), ('supplement', 0.028), ('mirrored', 0.028), ('completion', 0.028), ('flowchart', 0.027), ('korea', 0.027), ('shaded', 0.027), ('photometric', 0.027), ('barron', 0.025), ('filling', 0.025), ('sh', 0.025), ('utilizes', 0.025), ('scenes', 0.025), ('yeung', 0.025), ('reliably', 0.025), ('rough', 0.024), ('estimated', 0.024), ('filled', 0.023), ('readers', 0.023), ('enhancement', 0.023)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000002 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
Author: Lap-Fai Yu, Sai-Kit Yeung, Yu-Wing Tai, Stephen Lin
Abstract: We present a shading-based shape refinement algorithm which uses a noisy, incomplete depth map from Kinect to help resolve ambiguities in shape-from-shading. In our framework, the partial depth information is used to overcome bas-relief ambiguity in normals estimation, as well as to assist in recovering relative albedos, which are needed to reliably estimate the lighting environment and to separate shading from albedo. This refinement of surface normals using a noisy depth map leads to high-quality 3D surfaces. The effectiveness of our algorithm is demonstrated through several challenging real-world examples.
2 0.3450506 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
Author: Jonathan T. Barron, Jitendra Malik
Abstract: In this paper we extend the “shape, illumination and reflectance from shading ” (SIRFS) model [3, 4], which recovers intrinsic scene properties from a single image. Though SIRFS performs well on images of segmented objects, it performs poorly on images of natural scenes, which contain occlusion and spatially-varying illumination. We therefore present Scene-SIRFS, a generalization of SIRFS in which we have a mixture of shapes and a mixture of illuminations, and those mixture components are embedded in a “soft” segmentation of the input image. We additionally use the noisy depth maps provided by RGB-D sensors (in this case, the Kinect) to improve shape estimation. Our model takes as input a single RGB-D image and produces as output an improved depth map, a set of surface normals, a reflectance image, a shading image, and a spatially varying model of illumination. The output of our model can be used for graphics applications, or for any application involving RGB-D images.
3 0.29863003 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
Author: Ju Shen, Sen-Ching S. Cheung
Abstract: The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature.
4 0.26030809 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
Author: Feng Lu, Yasuyuki Matsushita, Imari Sato, Takahiro Okabe, Yoichi Sato
Abstract: We propose an uncalibrated photometric stereo method that works with general and unknown isotropic reflectances. Our method uses a pixel intensity profile, which is a sequence of radiance intensities recorded at a pixel across multi-illuminance images. We show that for general isotropic materials, the geodesic distance between intensity profiles is linearly related to the angular difference of their surface normals, and that the intensity distribution of an intensity profile conveys information about the reflectance properties, when the intensity profile is obtained under uniformly distributed directional lightings. Based on these observations, we show that surface normals can be estimated up to a convex/concave ambiguity. A solution method based on matrix decomposition with missing data is developed for a reliable estimation. Quantitative and qualitative evaluations of our method are performed using both synthetic and real-world scenes.
5 0.23613852 196 cvpr-2013-HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
Author: Omar Oreifej, Zicheng Liu
Abstract: We present a new descriptor for activity recognition from videos acquired by a depth sensor. Previous descriptors mostly compute shape and motion features independently; thus, they often fail to capture the complex joint shapemotion cues at pixel-level. In contrast, we describe the depth sequence using a histogram capturing the distribution of the surface normal orientation in the 4D space of time, depth, and spatial coordinates. To build the histogram, we create 4D projectors, which quantize the 4D space and represent the possible directions for the 4D normal. We initialize the projectors using the vertices of a regular polychoron. Consequently, we refine the projectors using a discriminative density measure, such that additional projectors are induced in the directions where the 4D normals are more dense and discriminative. Through extensive experiments, we demonstrate that our descriptor better captures the joint shape-motion cues in the depth sequence, and thus outperforms the state-of-the-art on all relevant benchmarks.
6 0.23361938 305 cvpr-2013-Non-parametric Filtering for Geometric Detail Extraction and Material Representation
7 0.21617821 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
8 0.19928163 56 cvpr-2013-Bayesian Depth-from-Defocus with Shading Constraints
9 0.19888495 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
10 0.1913923 330 cvpr-2013-Photometric Ambient Occlusion
11 0.18801424 466 cvpr-2013-Whitened Expectation Propagation: Non-Lambertian Shape from Shading and Shadow
12 0.17819889 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
13 0.17347442 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
14 0.16406097 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
15 0.16367911 115 cvpr-2013-Depth Super Resolution by Rigid Body Self-Similarity in 3D
16 0.16163173 397 cvpr-2013-Simultaneous Super-Resolution of Depth and Images Using a Single Camera
17 0.15246519 75 cvpr-2013-Calibrating Photometric Stereo by Holistic Reflectance Symmetry Analysis
18 0.14640924 114 cvpr-2013-Depth Acquisition from Density Modulated Binary Patterns
19 0.14344741 329 cvpr-2013-Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
20 0.13608132 117 cvpr-2013-Detecting Changes in 3D Structure of a Scene from Multi-view Images Captured by a Vehicle-Mounted Camera
topicId topicWeight
[(0, 0.189), (1, 0.294), (2, 0.039), (3, 0.093), (4, -0.051), (5, -0.113), (6, -0.146), (7, 0.193), (8, 0.055), (9, -0.047), (10, -0.114), (11, -0.207), (12, -0.068), (13, 0.137), (14, 0.103), (15, 0.019), (16, -0.103), (17, -0.053), (18, -0.056), (19, -0.141), (20, 0.023), (21, -0.002), (22, -0.051), (23, -0.003), (24, 0.099), (25, 0.057), (26, -0.025), (27, -0.019), (28, 0.006), (29, 0.031), (30, 0.057), (31, -0.063), (32, 0.041), (33, 0.092), (34, -0.004), (35, 0.04), (36, -0.06), (37, -0.036), (38, -0.027), (39, 0.009), (40, -0.078), (41, -0.06), (42, -0.031), (43, -0.052), (44, 0.102), (45, 0.012), (46, 0.034), (47, -0.006), (48, 0.048), (49, -0.053)]
simIndex simValue paperId paperTitle
same-paper 1 0.95008355 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
Author: Lap-Fai Yu, Sai-Kit Yeung, Yu-Wing Tai, Stephen Lin
Abstract: We present a shading-based shape refinement algorithm which uses a noisy, incomplete depth map from Kinect to help resolve ambiguities in shape-from-shading. In our framework, the partial depth information is used to overcome bas-relief ambiguity in normals estimation, as well as to assist in recovering relative albedos, which are needed to reliably estimate the lighting environment and to separate shading from albedo. This refinement of surface normals using a noisy depth map leads to high-quality 3D surfaces. The effectiveness of our algorithm is demonstrated through several challenging real-world examples.
2 0.9158507 56 cvpr-2013-Bayesian Depth-from-Defocus with Shading Constraints
Author: Chen Li, Shuochen Su, Yasuyuki Matsushita, Kun Zhou, Stephen Lin
Abstract: We present a method that enhances the performance of depth-from-defocus (DFD) through the use of shading information. DFD suffers from important limitations namely coarse shape reconstruction and poor accuracy on textureless surfaces that can be overcome with the help of shading. We integrate both forms of data within a Bayesian framework that capitalizes on their relative strengths. Shading data, however, is challenging to recover accurately from surfaces that contain texture. To address this issue, we propose an iterative technique that utilizes depth information to improve shading estimation, which in turn is used to elevate depth estimation in the presence of textures. With this approach, we demonstrate improvements over existing DFD techniques, as well as effective shape reconstruction of textureless surfaces. – –
3 0.90027183 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
Author: Jonathan T. Barron, Jitendra Malik
Abstract: In this paper we extend the “shape, illumination and reflectance from shading ” (SIRFS) model [3, 4], which recovers intrinsic scene properties from a single image. Though SIRFS performs well on images of segmented objects, it performs poorly on images of natural scenes, which contain occlusion and spatially-varying illumination. We therefore present Scene-SIRFS, a generalization of SIRFS in which we have a mixture of shapes and a mixture of illuminations, and those mixture components are embedded in a “soft” segmentation of the input image. We additionally use the noisy depth maps provided by RGB-D sensors (in this case, the Kinect) to improve shape estimation. Our model takes as input a single RGB-D image and produces as output an improved depth map, a set of surface normals, a reflectance image, a shading image, and a spatially varying model of illumination. The output of our model can be used for graphics applications, or for any application involving RGB-D images.
4 0.79831594 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
Author: Eno Töppe, Claudia Nieuwenhuis, Daniel Cremers
Abstract: We introduce the concept of relative volume constraints in order to account for insufficient information in the reconstruction of 3D objects from a single image. The key idea is to formulate a variational reconstruction approach with shape priors in form of relative depth profiles or volume ratios relating object parts. Such shape priors can easily be derived either from a user sketch or from the object’s shading profile in the image. They can handle textured or shadowed object regions by propagating information. We propose a convex relaxation of the constrained optimization problem which can be solved optimally in a few seconds on graphics hardware. In contrast to existing single view reconstruction algorithms, the proposed algorithm provides substantially more flexibility to recover shape details such as self-occlusions, dents and holes, which are not visible in the object silhouette.
5 0.73363638 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
Author: Kevin Karsch, Zicheng Liao, Jason Rock, Jonathan T. Barron, Derek Hoiem
Abstract: Early work in computer vision considered a host of geometric cues for both shape reconstruction [11] and recognition [14]. However, since then, the vision community has focused heavily on shading cues for reconstruction [1], and moved towards data-driven approaches for recognition [6]. In this paper, we reconsider these perhaps overlooked “boundary” cues (such as self occlusions and folds in a surface), as well as many other established constraints for shape reconstruction. In a variety of user studies and quantitative tasks, we evaluate how well these cues inform shape reconstruction (relative to each other) in terms of both shape quality and shape recognition. Our findings suggest many new directions for future research in shape reconstruction, such as automatic boundary cue detection and relaxing assumptions in shape from shading (e.g. orthographic projection, Lambertian surfaces).
6 0.71752763 305 cvpr-2013-Non-parametric Filtering for Geometric Detail Extraction and Material Representation
7 0.69424605 21 cvpr-2013-A New Perspective on Uncalibrated Photometric Stereo
8 0.64542323 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
9 0.63575923 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
10 0.63444018 466 cvpr-2013-Whitened Expectation Propagation: Non-Lambertian Shape from Shading and Shadow
11 0.62638366 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
12 0.62588763 115 cvpr-2013-Depth Super Resolution by Rigid Body Self-Similarity in 3D
13 0.62456924 330 cvpr-2013-Photometric Ambient Occlusion
14 0.62418664 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
15 0.61143422 114 cvpr-2013-Depth Acquisition from Density Modulated Binary Patterns
16 0.60297227 196 cvpr-2013-HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
17 0.59048975 428 cvpr-2013-The Episolar Constraint: Monocular Shape from Shadow Correspondence
18 0.56972164 397 cvpr-2013-Simultaneous Super-Resolution of Depth and Images Using a Single Camera
19 0.53371823 423 cvpr-2013-Template-Based Isometric Deformable 3D Reconstruction with Sampling-Based Focal Length Self-Calibration
20 0.53299898 117 cvpr-2013-Detecting Changes in 3D Structure of a Scene from Multi-view Images Captured by a Vehicle-Mounted Camera
topicId topicWeight
[(10, 0.13), (16, 0.033), (26, 0.059), (28, 0.03), (33, 0.23), (39, 0.012), (67, 0.029), (69, 0.047), (87, 0.137), (91, 0.198)]
simIndex simValue paperId paperTitle
same-paper 1 0.8519792 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
Author: Lap-Fai Yu, Sai-Kit Yeung, Yu-Wing Tai, Stephen Lin
Abstract: We present a shading-based shape refinement algorithm which uses a noisy, incomplete depth map from Kinect to help resolve ambiguities in shape-from-shading. In our framework, the partial depth information is used to overcome bas-relief ambiguity in normals estimation, as well as to assist in recovering relative albedos, which are needed to reliably estimate the lighting environment and to separate shading from albedo. This refinement of surface normals using a noisy depth map leads to high-quality 3D surfaces. The effectiveness of our algorithm is demonstrated through several challenging real-world examples.
2 0.8231439 259 cvpr-2013-Learning a Manifold as an Atlas
Author: Nikolaos Pitelis, Chris Russell, Lourdes Agapito
Abstract: In this work, we return to the underlying mathematical definition of a manifold and directly characterise learning a manifold as finding an atlas, or a set of overlapping charts, that accurately describe local structure. We formulate the problem of learning the manifold as an optimisation that simultaneously refines the continuous parameters defining the charts, and the discrete assignment of points to charts. In contrast to existing methods, this direct formulation of a manifold does not require “unwrapping ” the manifold into a lower dimensional space and allows us to learn closed manifolds of interest to vision, such as those corresponding to gait cycles or camera pose. We report state-ofthe-art results for manifold based nearest neighbour classification on vision datasets, and show how the same techniques can be applied to the 3D reconstruction of human motion from a single image.
3 0.81879389 415 cvpr-2013-Structured Face Hallucination
Author: Chih-Yuan Yang, Sifei Liu, Ming-Hsuan Yang
Abstract: The goal of face hallucination is to generate highresolution images with fidelity from low-resolution ones. In contrast to existing methods based on patch similarity or holistic constraints in the image space, we propose to exploit local image structures for face hallucination. Each face image is represented in terms of facial components, contours and smooth regions. The image structure is maintained via matching gradients in the reconstructed highresolution output. For facial components, we align input images to generate accurate exemplars and transfer the high-frequency details for preserving structural consistency. For contours, we learn statistical priors to generate salient structures in the high-resolution images. A patch matching method is utilized on the smooth regions where the image gradients are preserved. Experimental results demonstrate that the proposed algorithm generates hallucinated face images with favorable quality and adaptability.
4 0.81531262 298 cvpr-2013-Multi-scale Curve Detection on Surfaces
Author: Michael Kolomenkin, Ilan Shimshoni, Ayellet Tal
Abstract: This paper extends to surfaces the multi-scale approach of edge detection on images. The common practice for detecting curves on surfaces requires the user to first select the scale of the features, apply an appropriate smoothing, and detect the edges on the smoothed surface. This approach suffers from two drawbacks. First, it relies on a hidden assumption that all the features on the surface are of the same scale. Second, manual user intervention is required. In this paper, we propose a general framework for automatically detecting the optimal scale for each point on the surface. We smooth the surface at each point according to this optimal scale and run the curve detection algorithm on the resulting surface. Our multi-scale algorithm solves the two disadvantages of the single-scale approach mentioned above. We demonstrate how to realize our approach on two commonly-used special cases: ridges & valleys and relief edges. In each case, the optimal scale is found in accordance with the mathematical definition of the curve.
Author: Jia Xu, Maxwell D. Collins, Vikas Singh
Abstract: We study the problem of interactive segmentation and contour completion for multiple objects. The form of constraints our model incorporates are those coming from user scribbles (interior or exterior constraints) as well as information regarding the topology of the 2-D space after partitioning (number of closed contours desired). We discuss how concepts from discrete calculus and a simple identity using the Euler characteristic of a planar graph can be utilized to derive a practical algorithm for this problem. We also present specialized branch and bound methods for the case of single contour completion under such constraints. On an extensive dataset of ∼ 1000 images, our experimOenn tasn suggest vthea dt a assmetal ol fa m∼ou 1n0t0 of ismidaeg knowledge can give strong improvements over fully unsupervised contour completion methods. We show that by interpreting user indications topologically, user effort is substantially reduced.
6 0.81375188 209 cvpr-2013-Hypergraphs for Joint Multi-view Reconstruction and Multi-object Tracking
7 0.8119297 365 cvpr-2013-Robust Real-Time Tracking of Multiple Objects by Volumetric Mass Densities
8 0.81057525 39 cvpr-2013-Alternating Decision Forests
9 0.81012857 337 cvpr-2013-Principal Observation Ray Calibration for Tiled-Lens-Array Integral Imaging Display
10 0.80805415 19 cvpr-2013-A Minimum Error Vanishing Point Detection Approach for Uncalibrated Monocular Images of Man-Made Environments
11 0.80759525 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
12 0.80748522 107 cvpr-2013-Deformable Spatial Pyramid Matching for Fast Dense Correspondences
13 0.80454874 155 cvpr-2013-Exploiting the Power of Stereo Confidences
14 0.80282158 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
15 0.80126488 61 cvpr-2013-Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics
16 0.80113649 274 cvpr-2013-Lost! Leveraging the Crowd for Probabilistic Visual Self-Localization
17 0.80039459 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
18 0.80027199 331 cvpr-2013-Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis
19 0.80019557 147 cvpr-2013-Ensemble Learning for Confidence Measures in Stereo Vision
20 0.79992533 393 cvpr-2013-Separating Signal from Noise Using Patch Recurrence across Scales