cvpr cvpr2013 cvpr2013-245 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ju Shen, Sen-Ching S. Cheung
Abstract: The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature.
Reference: text
sentIndex sentText sentNum sentScore
1 The quality of the depth measurements of these systems, however, is far from perfect. [sent-6, score-0.863]
2 Some depth values can have significant errors, while others can be missing altogether. [sent-7, score-0.906]
3 The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. [sent-8, score-0.933]
4 In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. [sent-9, score-0.863]
5 The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. [sent-10, score-2.773]
6 The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. [sent-11, score-1.328]
7 Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature. [sent-12, score-0.863]
8 Depth images obtained by such an imaging system have two major problems: missing and distorted depth values. [sent-18, score-0.903]
9 All the black regions in the depth image contain no depth measurements. [sent-20, score-1.47]
10 1st column: input depth image; completed depth by Camplani et al. [sent-23, score-1.521]
11 This represents a major source of depth error caused by the disparity between the projector and the sensor. [sent-28, score-0.84]
12 Missing and erroneous depth values can also be caused by absorption, poor reflection or even shadow reflection of 1 1 1 1 1 18 8 87 5 5 the light patterns. [sent-30, score-0.902]
13 Objects with darker colors, specular surfaces, or fine-grained surfaces like human hair are prime candidates for poor depth measurements [3]. [sent-31, score-0.924]
14 Surface orientation also plays a role in depth measurements as the surface normal deviates from the principal axis of the IR camera, the accuracy of the depth measurement declines and becomes unreliable near depth discontinuities [10]. [sent-32, score-2.504]
15 Our technical contribution is the use of depth “layer” in steering the completion process to produce well-defined depth edges. [sent-35, score-1.598]
16 We describe a novel stochastic framework that separates the depth image into multiple layers, and combines multiple RGB-D system noise models to robustly determine the depth layer label. [sent-36, score-1.771]
17 The goal is to denoise and complete missing values on the depth image that improve the quality for any subsequent RGB-D applications. [sent-37, score-0.935]
18 A common theme is to rely on information obtained from the companion color images to predict missing depth information. [sent-40, score-0.959]
19 The use of color information for depth enhancement is based on the assumption that certain correlation exists between depth continuality and color image consistency [9]. [sent-41, score-1.634]
20 While providing useful cue for interpolation, this assumption does not always hold as color edges and depth edges do not necessarily coincide with each other. [sent-42, score-0.817]
21 To interpolate the missing depth pixel, the scheme used neighboring depth pixels mapped into the same color segment as the target pixel. [sent-46, score-1.812]
22 This method relied strongly on the extrinsic alignment between the color and depth image. [sent-47, score-0.843]
23 proposed a stereoscopic in-painting algorithm to jointly complete missing texture and depth by using two pairs of RGB and depth cameras. [sent-50, score-1.672]
24 The system required an additional pair of color and depth cameras to achieve the goal. [sent-52, score-0.817]
25 Various probabilistic frameworks are often used in mod- eling depth measurements, fusing depth and color information, and predicting missing values. [sent-53, score-1.745]
26 demonstrated the use of Markov Random Field in the superresolution of depth data using high-resolution color data. [sent-55, score-0.845]
27 Similarly, in [17], a low resolution depth image was iteratively refined through the use of a high resolution color image. [sent-57, score-0.865]
28 These approaches work well for the super-resolution problem where missing depth pixels are uniformly distributed. [sent-60, score-0.915]
29 Depth images obtained by structured-light sensors often have large contiguous regions of missing depth measurements which cannot be handled by such approaches. [sent-61, score-1.059]
30 Though the approach could well preserve foreground edges, it may fail when the captured scene has complex depth variation. [sent-63, score-0.882]
31 These missing values can usually be inferred by the available depth pixels in the neighboring regions. [sent-67, score-1.02]
32 In addition, the corresponding RGB information can be used to steer the depth completion by using techniques such as Joint Bilateral Filter [9]. [sent-68, score-0.904]
33 The second type is the large and contiguous missing depth patches that often present along the boundary between close (foreground) and far objects (background). [sent-69, score-0.905]
34 An example can be found in Figure 2(a), where there are many missing depth (black) regions around the hand. [sent-70, score-0.877]
35 Missing and incorrect depth values due to disparity cannot be easily and correctly inferred by many existing works including Joint Bilateral Filter based schemes [9] [2] and probabilistic based schemes [4]. [sent-74, score-0.894]
36 The raw depth image shown in Figure 2(a) clearly indicates two distinct depth layers: foreground in dark gray and background in light gray. [sent-76, score-1.696]
37 No depth measurements are obtained in the black region. [sent-77, score-0.863]
38 In Figure 2(b), we overlay semitransparent green (foreground) and red (background) layers over the RGB image as an indicator of its corresponding depth information. [sent-78, score-0.911]
39 The missing depth patches (annotated as “B”) are often adjacent to “A”. [sent-80, score-0.877]
40 Most existing spatial approaches complete these missing values by using the erroneous depth in the neighboring areas. [sent-81, score-1.055]
41 The alignment between the depth and color images is based on the calibration result from Mircrosft Kinect diver little to rectify this problem because the RGB values from “A” are very similar to the ones from “B”, both of which are from the background color. [sent-85, score-0.916]
42 The depth completion result using a joint color-depth bilateral [2] shown in Figure 2(c) is indeed quite poor. [sent-86, score-0.975]
43 After an initial step of offline training on backgroundonly frames, our online algorithm consists of two main phases: layer labeling followed by depth denoising and completion. [sent-91, score-1.127]
44 Maximum A Posteriori (MAP) estimation is used in the labeling to prevent blurring along depth discontinuities. [sent-93, score-0.825]
45 In the second phase, the labels estimated in the first phase are used to steer the removal of outlier and the completion of missing depth values, from either the background model or from neighboring depth values with the same labels. [sent-94, score-2.046]
46 The robust labeling allows us to preserve the shape of object boundary and prevent noise propagation across objects with significant depth differences. [sent-95, score-0.919]
47 Let G be the support of the 2D color and depth images. [sent-99, score-0.817]
48 Due to the missing depth problem, Zd may not be directly observable. [sent-131, score-0.877]
49 We thus in- troduce an observable depth indicator random variable M which is 1 if the depth value is observed and 0 otherwise. [sent-132, score-1.535]
50 The depth distribution P(D, X) = P(X)P(D|X) is modeled as a mixture of Gaussian (MOG) model while the color distribution P(Ic, X) = P(X)P(Ic |X) is modeled as multiple color histograms on the quantized HSV space, one for each layer. [sent-147, score-0.995]
51 The observable depth indicator distribution P(M, X) = P(X)P(M|X) is based a simple Bernoulli distribution for each layer. [sent-148, score-0.844]
52 In fact, the parameter estimation for the depth indicator is a simpler version of the color distribution. [sent-149, score-0.845]
53 As such, our discussion will focus on the depth and color distributions. [sent-150, score-0.817]
54 During the offline training phase, we estimate the parameters for the negative (background) depth layers based on a set of training RGB-D frames of the static background. [sent-152, score-0.945]
55 During the global estimation, allthe pixels with both color and depth measurements will be aggregated to estimate a single pair of P(D, X) and P(Ic, X) using the Expectation-Maximization (EM) approach. [sent-154, score-0.983]
56 Parameter estimation for depth and color layer distributions note that the concept of layers is based only on depth but not on color. [sent-157, score-2.002]
57 As such, the EM process is primarily driven by the depth data in the sense that the E-step only estimates the layer posterior P(X|D) for the depth but not the color. [sent-158, score-1.759]
58 During the M-step, we use the depth data to update the estimates for the layer prior P(X) and the depth layer conditional P(D|X), only use the color data to update the color layer conditional P(Ic |X) using the posterior probability P(X |D) of the co-located depth pixel. [sent-159, score-3.184]
59 The example in Figure 4 shows this first step to have two separate background layers and obtain the global color and depth models. [sent-163, score-1.035]
60 For example, the local mean for layer Xs = iat location s is updated by a new depth value dnew as follows: μ(st,+i1) := λP(t)(Xs = i|Ds = dnew) · dnew + ? [sent-166, score-1.207]
61 Similar to the global distributions, the two layer conditional probabilities are updated based on the corresponding color or depth data. [sent-171, score-1.105]
62 The layer prior is updated using the depth data if available, or the color data if the depth data is missing. [sent-172, score-1.84]
63 The training data are obtained based on only those pixels with valid depth measurements and very low background posterior probability, i. [sent-177, score-0.997]
64 Next, the noisy depth measurement Zd is modeled based on an additive Gaussian model: Zd ? [sent-189, score-0.86]
65 (5) The noise standard deviation σθ reflects the uncertainty in the depth measurement. [sent-191, score-0.817]
66 As argued in Section 3, erroneous depth measurements occur predominantly near object boundaries. [sent-192, score-0.929]
67 To model this effect, we apply an edge detector on the depth map and use the spatial distance θ to the closest depth edge as a reliability measure. [sent-193, score-1.47]
68 Note that the actual depth measurement Id = MZd implies that P(Id = id |Zd = z, M = m) = δmid (z), the dirac delta function with the only nonzero value at z = mid. [sent-198, score-0.913]
69 2 Smoothing Term For the spatial MRF, the edge potential ψ(Xs , Xt) is defined based on the similarity in color and depth between the neighboring pixels: ψ(Xs,Xt,fst) ? [sent-213, score-0.871]
70 [1 − fst] footrhe Xrwsi=se Xt, (9) The similarity strength fst is a feature based on how close the color and depth of the neighboring pixels are. [sent-216, score-1.011]
71 It is modeled using the following equation: fst = max{α exp(−Cst) + (1 α) exp(−Dst) , nf } (10) − The color similarity ratio Cst and the depth similarity ratio Dst are defined in a similar fashion [13]: Cst = Dst = kc? [sent-217, score-0.979]
72 If either depth measurement is not present, nf will be used. [sent-230, score-0.868]
73 The parameter α ∈ [0, 1] is a trade-off between depth and color information. [sent-231, score-0.817]
74 If the depth measurements are reliable, most of the weight should be assigned to depth values as they are more reliable for foreground/background labeling; if the depth measurements are unreliable, they should not be used at all in computing the edge potential. [sent-232, score-2.49]
75 Depth Image Completion After assigning each pixel with a layer label, we need to identify erroneous depth measurements and complete missing depth values. [sent-241, score-2.136]
76 The erroneous depth values are essentially outliers that are significantly different from other depth values in the neighborhood. [sent-242, score-1.594]
77 However, as most measurement errors occur around object boundaries, it is imperative not to mistake true depth discontinuities as wrong depth values. [sent-243, score-1.59]
78 Time Performance Evaluation determine if a depth pixel is an outlier, we robustly estimate the depth distribution in the neighborhood around the pixel via a RANSAC-like procedure. [sent-247, score-1.592]
79 First, we only consider depth values in the neighborhood that share the same label as the target pixel. [sent-248, score-0.788]
80 Among those that survive the robustness test, the one with the smallest variance is used and the target depth pixel is declared an outlier if it is beyond two standard deviations from the mean. [sent-251, score-0.82]
81 The outlier depth pixel will join the rest of the missing depth pixels and will be completing using a joint color-depth bilateral filtering scheme similar to that in [9]. [sent-252, score-1.873]
82 The only difference is that we only consider the contributions from neighboring depth pixels that have the same layer label as the center pixel. [sent-253, score-1.09]
83 For the static background training, we captured 100 images for each scene to obtain the background color and depth statistics. [sent-260, score-0.989]
84 We can see that the background wall is clearly visible between the foreground leaves and the rapid spatial changes of depth layers lead to significant measurement error shown in the original depth image in the top right. [sent-263, score-1.913]
85 Scen withTwoLayers:inputRGBand epthimages (top); layer labeling result and completed depth steered by layers (bottom). [sent-269, score-1.31]
86 Scen withMultipeForegounda BackgroundLay- ers: input RGB and depth images (top); layer labeling result and completed depth steered by layers (bottom). [sent-274, score-2.045]
87 In particular, we have chosen [2] which represents one of the most recent efforts in using joint color-depth bilateral filtering for depth completion, and [4] which uses a similar MRF as ours in providing spatial smoothing. [sent-276, score-0.847]
88 To highlight the differences in depth completion, we generate a set of arbitrary views based on the produced depth image and the RGB image, which are presented in the second and fourth rows. [sent-282, score-1.498]
89 As shown in the depth maps, this approach enlarges the foreground shape by attaching unrelated pixels around objects’ boundaries. [sent-285, score-0.899]
90 The wrongly assigned depth values move some of the background pixels to the foreground or vice versa. [sent-286, score-1.023]
91 However, the MRF blurs the boundaries between foreground and background by generating intermediate depth values. [sent-291, score-0.931]
92 From the virtual views, these intermediate depth values can be seen spreading across the space between the foreground and background. [sent-292, score-0.939]
93 The depth completion steered by layers can better preserve the shape of object boundary and prevent noise propagation across objects with significant depth differences. [sent-294, score-1.931]
94 Conclusions In this paper, we have proposed a novel depth denoising and completion scheme by combining color-depth correlation, background modeling, spatial smoothness, and measurement error models pertinent to structured-light systems. [sent-296, score-1.101]
95 A probabilistic graphical model is used to fuse all these different factors together with the key latent variable being the depth layer at each pixel. [sent-297, score-1.027]
96 The depth layer labeling is formulated as a MAP problem and a MRF attuned to the uncertainty in depth measurements is used to spatially smooth the labeling process. [sent-298, score-2.063]
97 Driven by the obtained depth layer labels, depth noise is removed and depth interpolation is performed using a bilateral filter. [sent-299, score-2.627]
98 For example, additional depth features such as surface normal and texture may provide a more accurate uncertainty model in depth measurements. [sent-301, score-1.544]
99 We are currently investigating more computationally efficient approaches for depth layer labeling, and extending our model to support multiple Kinects which can provide more complete 3D scanning but may introduce new errors due to interference. [sent-302, score-1.049]
100 Stereoscopic inpainting: Joint color and depth completion from stereo images. [sent-410, score-0.945]
wordName wordTfidf (topN-words)
[('depth', 0.735), ('layer', 0.263), ('layers', 0.148), ('missing', 0.142), ('xs', 0.142), ('zd', 0.142), ('measurements', 0.128), ('completion', 0.128), ('foreground', 0.126), ('ic', 0.104), ('fst', 0.102), ('measurement', 0.099), ('dnew', 0.092), ('bilateral', 0.089), ('rgb', 0.083), ('color', 0.082), ('id', 0.079), ('ir', 0.077), ('projector', 0.074), ('background', 0.07), ('erroneous', 0.066), ('camplani', 0.061), ('cst', 0.057), ('steered', 0.057), ('labeling', 0.056), ('neighboring', 0.054), ('dst', 0.054), ('completed', 0.051), ('kinect', 0.05), ('virtual', 0.049), ('outlier', 0.047), ('attuned', 0.046), ('garro', 0.046), ('uncertainty', 0.044), ('phase', 0.043), ('denoising', 0.043), ('steer', 0.041), ('mrf', 0.041), ('mid', 0.04), ('distributions', 0.039), ('pixel', 0.038), ('pixels', 0.038), ('prime', 0.038), ('noise', 0.038), ('observable', 0.037), ('propagation', 0.035), ('ds', 0.035), ('xt', 0.035), ('nf', 0.034), ('prevent', 0.034), ('fingers', 0.034), ('interpolation', 0.032), ('static', 0.032), ('disparity', 0.031), ('stereoscopic', 0.031), ('light', 0.03), ('surface', 0.03), ('offline', 0.03), ('conference', 0.03), ('complete', 0.029), ('kc', 0.029), ('diebel', 0.029), ('probabilistic', 0.029), ('values', 0.029), ('indicator', 0.028), ('ps', 0.028), ('phases', 0.028), ('contiguous', 0.028), ('superresolution', 0.028), ('camera', 0.028), ('views', 0.028), ('extrinsic', 0.026), ('posterior', 0.026), ('sensors', 0.026), ('scheme', 0.026), ('distorted', 0.026), ('modeled', 0.026), ('wrongly', 0.025), ('updated', 0.025), ('resolution', 0.024), ('environment', 0.024), ('neighborhood', 0.024), ('schemes', 0.024), ('holes', 0.024), ('kd', 0.023), ('hair', 0.023), ('joint', 0.023), ('smoothing', 0.023), ('loopy', 0.023), ('labels', 0.022), ('inferred', 0.022), ('distribution', 0.022), ('markov', 0.022), ('belief', 0.022), ('fusing', 0.022), ('scanning', 0.022), ('discontinuities', 0.021), ('preserve', 0.021), ('reflection', 0.021), ('unreliable', 0.021)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999893 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
Author: Ju Shen, Sen-Ching S. Cheung
Abstract: The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature.
2 0.32427394 397 cvpr-2013-Simultaneous Super-Resolution of Depth and Images Using a Single Camera
Author: Hee Seok Lee, Kuoung Mu Lee
Abstract: In this paper, we propose a convex optimization framework for simultaneous estimation of super-resolved depth map and images from a single moving camera. The pixel measurement error in 3D reconstruction is directly related to the resolution of the images at hand. In turn, even a small measurement error can cause significant errors in reconstructing 3D scene structure or camera pose. Therefore, enhancing image resolution can be an effective solution for securing the accuracy as well as the resolution of 3D reconstruction. In the proposed method, depth map estimation and image super-resolution are formulated in a single energy minimization framework with a convex function and solved efficiently by a first-order primal-dual algorithm. Explicit inter-frame pixel correspondences are not required for our super-resolution procedure, thus we can avoid a huge computation time and obtain improved depth map in the accuracy and resolution as well as highresolution images with reasonable time. The superiority of our algorithm is demonstrated by presenting the improved depth map accuracy, image super-resolution results, and camera pose estimation.
3 0.31616768 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
Author: Ming-Yu Liu, Oncel Tuzel, Yuichi Taguchi
Abstract: We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution one. Though this is closely related to the all-pairshortest-path problem which has O(n2 log n) complexity, we develop a novel approximation algorithm whose complexity grows linearly with the image size and achieve realtime performance. We compare our algorithm with the state of the art on the benchmark dataset and show that our approach provides more accurate depth upsampling with fewer artifacts. In addition, we show that the proposed algorithm is well suited for upsampling depth images using binary edge maps, an important sensor fusion application.
4 0.29863003 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
Author: Lap-Fai Yu, Sai-Kit Yeung, Yu-Wing Tai, Stephen Lin
Abstract: We present a shading-based shape refinement algorithm which uses a noisy, incomplete depth map from Kinect to help resolve ambiguities in shape-from-shading. In our framework, the partial depth information is used to overcome bas-relief ambiguity in normals estimation, as well as to assist in recovering relative albedos, which are needed to reliably estimate the lighting environment and to separate shading from albedo. This refinement of surface normals using a noisy depth map leads to high-quality 3D surfaces. The effectiveness of our algorithm is demonstrated through several challenging real-world examples.
5 0.29793793 115 cvpr-2013-Depth Super Resolution by Rigid Body Self-Similarity in 3D
Author: unkown-author
Abstract: We tackle the problem of jointly increasing the spatial resolution and apparent measurement accuracy of an input low-resolution, noisy, and perhaps heavily quantized depth map. In stark contrast to earlier work, we make no use of ancillary data like a color image at the target resolution, multiple aligned depth maps, or a database of highresolution depth exemplars. Instead, we proceed by identifying and merging patch correspondences within the input depth map itself, exploiting patchwise scene self-similarity across depth such as repetition of geometric primitives or object symmetry. While the notion of ‘single-image ’ super resolution has successfully been applied in the context of color and intensity images, we are to our knowledge the first to present a tailored analogue for depth images. Rather than reason in terms of patches of 2D pixels as others have before us, our key contribution is to proceed by reasoning in terms of patches of 3D points, with matched patch pairs related by a respective 6 DoF rigid body motion in 3D. In support of obtaining a dense correspondence field in reasonable time, we introduce a new 3D variant of Patch- Match. A third contribution is a simple, yet effective patch upscaling and merging technique, which predicts sharp object boundaries at the target resolution. We show that our results are highly competitive with those of alternative techniques leveraging even a color image at the target resolution or a database of high-resolution depth exemplars.
6 0.29783767 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
8 0.26688722 114 cvpr-2013-Depth Acquisition from Density Modulated Binary Patterns
9 0.25404602 108 cvpr-2013-Dense 3D Reconstruction from Severely Blurred Images Using a Single Moving Camera
10 0.25218734 111 cvpr-2013-Dense Reconstruction Using 3D Object Shape Priors
11 0.24290441 407 cvpr-2013-Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera
12 0.23663138 196 cvpr-2013-HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
13 0.23639564 205 cvpr-2013-Hollywood 3D: Recognizing Actions in 3D Natural Scenes
14 0.23368397 357 cvpr-2013-Revisiting Depth Layers from Occlusions
15 0.19849938 307 cvpr-2013-Non-uniform Motion Deblurring for Bilayer Scenes
16 0.16658995 181 cvpr-2013-Fusing Depth from Defocus and Stereo with Coded Apertures
17 0.16385348 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
18 0.16148889 56 cvpr-2013-Bayesian Depth-from-Defocus with Shading Constraints
19 0.15974769 329 cvpr-2013-Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
20 0.15308866 380 cvpr-2013-Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images
topicId topicWeight
[(0, 0.246), (1, 0.304), (2, 0.08), (3, 0.081), (4, -0.08), (5, -0.043), (6, -0.078), (7, 0.225), (8, 0.043), (9, -0.019), (10, -0.065), (11, -0.126), (12, 0.036), (13, 0.257), (14, 0.054), (15, -0.036), (16, -0.372), (17, 0.078), (18, -0.062), (19, -0.071), (20, 0.004), (21, -0.044), (22, -0.089), (23, -0.135), (24, 0.07), (25, 0.062), (26, 0.023), (27, 0.043), (28, 0.006), (29, -0.009), (30, 0.005), (31, 0.087), (32, 0.093), (33, -0.052), (34, -0.011), (35, -0.09), (36, 0.016), (37, 0.011), (38, -0.011), (39, 0.032), (40, 0.01), (41, -0.05), (42, -0.006), (43, 0.102), (44, -0.161), (45, -0.023), (46, -0.057), (47, 0.008), (48, 0.02), (49, 0.074)]
simIndex simValue paperId paperTitle
same-paper 1 0.99580419 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
Author: Ju Shen, Sen-Ching S. Cheung
Abstract: The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature.
2 0.93226767 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
Author: Ming-Yu Liu, Oncel Tuzel, Yuichi Taguchi
Abstract: We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution one. Though this is closely related to the all-pairshortest-path problem which has O(n2 log n) complexity, we develop a novel approximation algorithm whose complexity grows linearly with the image size and achieve realtime performance. We compare our algorithm with the state of the art on the benchmark dataset and show that our approach provides more accurate depth upsampling with fewer artifacts. In addition, we show that the proposed algorithm is well suited for upsampling depth images using binary edge maps, an important sensor fusion application.
3 0.9059031 114 cvpr-2013-Depth Acquisition from Density Modulated Binary Patterns
Author: Zhe Yang, Zhiwei Xiong, Yueyi Zhang, Jiao Wang, Feng Wu
Abstract: This paper proposes novel density modulated binary patterns for depth acquisition. Similar to Kinect, the illumination patterns do not need a projector for generation and can be emitted by infrared lasers and diffraction gratings. Our key idea is to use the density of light spots in the patterns to carry phase information. Two technical problems are addressed here. First, we propose an algorithm to design the patterns to carry more phase information without compromising the depth reconstruction from a single captured image as with Kinect. Second, since the carried phase is not strictly sinusoidal, the depth reconstructed from the phase contains a systematic error. We further propose a pixelbased phase matching algorithm to reduce the error. Experimental results show that the depth quality can be greatly improved using the phase carried by the density of light spots. Furthermore, our scheme can achieve 20 fps depth reconstruction with GPU assistance.
4 0.86003178 115 cvpr-2013-Depth Super Resolution by Rigid Body Self-Similarity in 3D
Author: unkown-author
Abstract: We tackle the problem of jointly increasing the spatial resolution and apparent measurement accuracy of an input low-resolution, noisy, and perhaps heavily quantized depth map. In stark contrast to earlier work, we make no use of ancillary data like a color image at the target resolution, multiple aligned depth maps, or a database of highresolution depth exemplars. Instead, we proceed by identifying and merging patch correspondences within the input depth map itself, exploiting patchwise scene self-similarity across depth such as repetition of geometric primitives or object symmetry. While the notion of ‘single-image ’ super resolution has successfully been applied in the context of color and intensity images, we are to our knowledge the first to present a tailored analogue for depth images. Rather than reason in terms of patches of 2D pixels as others have before us, our key contribution is to proceed by reasoning in terms of patches of 3D points, with matched patch pairs related by a respective 6 DoF rigid body motion in 3D. In support of obtaining a dense correspondence field in reasonable time, we introduce a new 3D variant of Patch- Match. A third contribution is a simple, yet effective patch upscaling and merging technique, which predicts sharp object boundaries at the target resolution. We show that our results are highly competitive with those of alternative techniques leveraging even a color image at the target resolution or a database of high-resolution depth exemplars.
5 0.85275668 397 cvpr-2013-Simultaneous Super-Resolution of Depth and Images Using a Single Camera
Author: Hee Seok Lee, Kuoung Mu Lee
Abstract: In this paper, we propose a convex optimization framework for simultaneous estimation of super-resolved depth map and images from a single moving camera. The pixel measurement error in 3D reconstruction is directly related to the resolution of the images at hand. In turn, even a small measurement error can cause significant errors in reconstructing 3D scene structure or camera pose. Therefore, enhancing image resolution can be an effective solution for securing the accuracy as well as the resolution of 3D reconstruction. In the proposed method, depth map estimation and image super-resolution are formulated in a single energy minimization framework with a convex function and solved efficiently by a first-order primal-dual algorithm. Explicit inter-frame pixel correspondences are not required for our super-resolution procedure, thus we can avoid a huge computation time and obtain improved depth map in the accuracy and resolution as well as highresolution images with reasonable time. The superiority of our algorithm is demonstrated by presenting the improved depth map accuracy, image super-resolution results, and camera pose estimation.
6 0.80276698 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
8 0.78402317 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
9 0.78304315 407 cvpr-2013-Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera
10 0.72999048 56 cvpr-2013-Bayesian Depth-from-Defocus with Shading Constraints
11 0.70417339 196 cvpr-2013-HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
12 0.65912718 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
13 0.65129697 357 cvpr-2013-Revisiting Depth Layers from Occlusions
14 0.64265352 428 cvpr-2013-The Episolar Constraint: Monocular Shape from Shadow Correspondence
15 0.62280023 307 cvpr-2013-Non-uniform Motion Deblurring for Bilayer Scenes
16 0.62019962 181 cvpr-2013-Fusing Depth from Defocus and Stereo with Coded Apertures
17 0.61588418 108 cvpr-2013-Dense 3D Reconstruction from Severely Blurred Images Using a Single Moving Camera
18 0.61337006 219 cvpr-2013-In Defense of 3D-Label Stereo
19 0.60720921 111 cvpr-2013-Dense Reconstruction Using 3D Object Shape Priors
20 0.59255344 205 cvpr-2013-Hollywood 3D: Recognizing Actions in 3D Natural Scenes
topicId topicWeight
[(9, 0.106), (10, 0.171), (16, 0.068), (26, 0.036), (28, 0.02), (33, 0.274), (67, 0.046), (69, 0.075), (77, 0.01), (87, 0.102)]
simIndex simValue paperId paperTitle
1 0.93257898 219 cvpr-2013-In Defense of 3D-Label Stereo
Author: Carl Olsson, Johannes Ulén, Yuri Boykov
Abstract: It is commonly believed that higher order smoothness should be modeled using higher order interactions. For example, 2nd order derivatives for deformable (active) contours are represented by triple cliques. Similarly, the 2nd order regularization methods in stereo predominantly use MRF models with scalar (1D) disparity labels and triple clique interactions. In this paper we advocate a largely overlooked alternative approach to stereo where 2nd order surface smoothness is represented by pairwise interactions with 3D-labels, e.g. tangent planes. This general paradigm has been criticized due to perceived computational complexity of optimization in higher-dimensional label space. Contrary to popular beliefs, we demonstrate that representing 2nd order surface smoothness with 3D labels leads to simpler optimization problems with (nearly) submodular pairwise interactions. Our theoretical and experimental re- sults demonstrate advantages over state-of-the-art methods for 2nd order smoothness stereo. 1
same-paper 2 0.93047607 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
Author: Ju Shen, Sen-Ching S. Cheung
Abstract: The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature.
3 0.92563605 365 cvpr-2013-Robust Real-Time Tracking of Multiple Objects by Volumetric Mass Densities
Author: Horst Possegger, Sabine Sternig, Thomas Mauthner, Peter M. Roth, Horst Bischof
Abstract: Combining foreground images from multiple views by projecting them onto a common ground-plane has been recently applied within many multi-object tracking approaches. These planar projections introduce severe artifacts and constrain most approaches to objects moving on a common 2D ground-plane. To overcome these limitations, we introduce the concept of an occupancy volume exploiting the full geometry and the objects ’ center of mass and develop an efficient algorithm for 3D object tracking. Individual objects are tracked using the local mass density scores within a particle filter based approach, constrained by a Voronoi partitioning between nearby trackers. Our method benefits from the geometric knowledge given by the occupancy volume to robustly extract features and train classifiers on-demand, when volumetric information becomes unreliable. We evaluate our approach on several challenging real-world scenarios including the public APIDIS dataset. Experimental evaluations demonstrate significant improvements compared to state-of-theart methods, while achieving real-time performance. – –
4 0.92497545 61 cvpr-2013-Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics
Author: Bo Zheng, Yibiao Zhao, Joey C. Yu, Katsushi Ikeuchi, Song-Chun Zhu
Abstract: In this paper, we present an approach for scene understanding by reasoning physical stability of objects from point cloud. We utilize a simple observation that, by human design, objects in static scenes should be stable with respect to gravity. This assumption is applicable to all scene categories and poses useful constraints for the plausible interpretations (parses) in scene understanding. Our method consists of two major steps: 1) geometric reasoning: recovering solid 3D volumetric primitives from defective point cloud; and 2) physical reasoning: grouping the unstable primitives to physically stable objects by optimizing the stability and the scene prior. We propose to use a novel disconnectivity graph (DG) to represent the energy landscape and use a Swendsen-Wang Cut (MCMC) method for optimization. In experiments, we demonstrate that the algorithm achieves substantially better performance for i) object segmentation, ii) 3D volumetric recovery of the scene, and iii) better parsing result for scene understanding in comparison to state-of-the-art methods in both public dataset and our own new dataset.
5 0.92410988 248 cvpr-2013-Learning Collections of Part Models for Object Recognition
Author: Ian Endres, Kevin J. Shih, Johnston Jiaa, Derek Hoiem
Abstract: We propose a method to learn a diverse collection of discriminative parts from object bounding box annotations. Part detectors can be trained and applied individually, which simplifies learning and extension to new features or categories. We apply the parts to object category detection, pooling part detections within bottom-up proposed regions and using a boosted classifier with proposed sigmoid weak learners for scoring. On PASCAL VOC 2010, we evaluate the part detectors ’ ability to discriminate and localize annotated keypoints. Our detection system is competitive with the best-existing systems, outperforming other HOG-based detectors on the more deformable categories.
6 0.92121065 447 cvpr-2013-Underwater Camera Calibration Using Wavelength Triangulation
7 0.9209221 400 cvpr-2013-Single Image Calibration of Multi-axial Imaging Systems
8 0.92071158 331 cvpr-2013-Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis
9 0.92061603 98 cvpr-2013-Cross-View Action Recognition via a Continuous Virtual Path
10 0.92018336 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
11 0.9196344 225 cvpr-2013-Integrating Grammar and Segmentation for Human Pose Estimation
12 0.9189707 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
13 0.91892463 19 cvpr-2013-A Minimum Error Vanishing Point Detection Approach for Uncalibrated Monocular Images of Man-Made Environments
14 0.91877806 115 cvpr-2013-Depth Super Resolution by Rigid Body Self-Similarity in 3D
15 0.91828418 445 cvpr-2013-Understanding Bayesian Rooms Using Composite 3D Object Models
16 0.91788042 143 cvpr-2013-Efficient Large-Scale Structured Learning
17 0.9164229 360 cvpr-2013-Robust Estimation of Nonrigid Transformation for Point Set Registration
18 0.916309 74 cvpr-2013-CLAM: Coupled Localization and Mapping with Efficient Outlier Handling
19 0.91622263 414 cvpr-2013-Structure Preserving Object Tracking
20 0.91606343 285 cvpr-2013-Minimum Uncertainty Gap for Robust Visual Tracking