cvpr cvpr2013 cvpr2013-232 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Ming-Yu Liu, Oncel Tuzel, Yuichi Taguchi
Abstract: We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution one. Though this is closely related to the all-pairshortest-path problem which has O(n2 log n) complexity, we develop a novel approximation algorithm whose complexity grows linearly with the image size and achieve realtime performance. We compare our algorithm with the state of the art on the benchmark dataset and show that our approach provides more accurate depth upsampling with fewer artifacts. In addition, we show that the proposed algorithm is well suited for upsampling depth images using binary edge maps, an important sensor fusion application.
Reference: text
sentIndex sentText sentNum sentScore
1 com Abstract We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. [sent-2, score-1.291]
2 Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution one. [sent-3, score-1.377]
3 We compare our algorithm with the state of the art on the benchmark dataset and show that our approach provides more accurate depth upsampling with fewer artifacts. [sent-5, score-1.035]
4 In addition, we show that the proposed algorithm is well suited for upsampling depth images using binary edge maps, an important sensor fusion application. [sent-6, score-1.103]
5 However, unlike the conventional optical camera, the resolution of depth sensors advances at a much slower pace. [sent-10, score-0.56]
6 While the resolution of mainstream optical cameras is in the order of 10 megapixels, the resolution of mainstream time-of-flight depth sensors is still lower than 0. [sent-12, score-0.776]
7 One way to improve the resolution of depth images is to use a high resolution optical camera in tandem with the depth sensor. [sent-14, score-1.068]
8 In general, geometric and color boundaries of a scene are correlated: abrupt depth transition often leads to abrupt color transition. [sent-15, score-0.541]
9 In this paper, we study depth image upsampling problem using registered color images and propose a new algorithm based on geodesic curves in joint color and depth images. [sent-17, score-1.977]
10 Our algorithm is inspired by the joint bilateral upsampling algorithm [11] (current state of the art in terms of both accuracy and efficiency), which interpolates low resolution depths on the high resolution grid based on a set of weights computed as multiplications of spatial and color kernels. [sent-18, score-1.423]
11 These kernels utilize Euclidean distance to quantify the dissimilarities of the pixels and the word “joint“ is due to utilization of two channels: optical image for computing color distance and depth image for computing spatial distance. [sent-19, score-0.569]
12 We argue that using two separate kernels causes blurry depth boundaries and depth bleeding artifacts par- ticularly when the colors of the surfaces across the depth boundaries are similar. [sent-20, score-1.243]
13 We compute geodesic distances—lengths of the geodesic curves—from each pixel in the target high resolution depth image to all the pixels whose depths are known from the low resolution depth image. [sent-23, score-2.072]
14 These distances are used to propagate the known depths to the high resolution grid in a smooth and depth discontinuity preserving manner. [sent-24, score-0.852]
15 Since the geodesic distance integrates joint color and spatial changes along the curve, it is sensitive to thin contours around the surfaces, providing sharp depth boundaries even when the color difference between two sides of a contour is subtle. [sent-25, score-1.117]
16 In addition, the geodesic path can follow thin segments with uniform colors and therefore produce high quality depth images with fine details. [sent-26, score-0.896]
17 An example motivating the use of geodesic distances for depth upsampling is illustrated in Figure 1 (see Section 2 for further details). [sent-27, score-1.453]
18 For real-time processing, we propose a new approximation algorithm for simultaneously finding K nearest (in geodesic sense) nodes from each source node and show that its complexity grows linearly with the image size and K. [sent-29, score-0.542]
19 The result shows that the proposed algorithm produces more accurate high resolution depth images for both smooth surfaces and boundary re111666999 gions. [sent-31, score-0.666]
20 We also show that our method is well suited for upsampling depth images using binary edge maps (e. [sent-32, score-1.075]
21 Related Work Depth upsampling methods can be categorized as global or local. [sent-37, score-0.661]
22 Global methods [6, 14, 19] formulate depth upsampling as an optimization problem where a large cost is induced if two neighboring pixels having a similar color are assigned two very different depths. [sent-38, score-1.121]
23 Among them, joint bilateral upsampling is particularly popular, which uses bilateral filtering in a joint color-spatial space. [sent-41, score-1.057]
24 The proposed algorithm is also based on filtering where the upsampling filter weights are determined by geodesic distances. [sent-42, score-1.113]
25 Here, we define joint geodesic filtering in a color-spatial space and show its application for depth upsampling. [sent-44, score-0.868]
26 The fast marching algorithm [23] and the geodesic distance transform [18] are two common implementations for geodesic distance computation, both of which have a linear time complexity. [sent-45, score-0.979]
27 We derive a novel approximation algorithm for simultaneously finding K nearest nodes from each source node based on geodesic distance transforms and achieve real-time performance. [sent-47, score-0.558]
28 Recently, [8] proposed using geodesic distances to compute Voronoi cells for image tessellation, which are used for fitting planes to sparse depth measurements for depth interpolation. [sent-48, score-1.145]
29 • We propose a new joint filtering algorithm using geodesic odsiestan ac nese wfor j upsampling a depth image using a registered high resolution color image. [sent-53, score-1.796]
30 • We develop a fast optimization technique for finding approximate sKt noeptairmesitz antioodnes t ebcahsendiq on geodesic distance and achieving real-time upsampling performance. [sent-54, score-1.103]
31 In Section 2, we present the depth upsampling formulation. [sent-57, score-1.014]
32 Joint Geodesic Upsampling Let D↓ and I the low resolution depth and high resobe lution optical images, respectively, where the resolution of I r times larger than that of D↓. [sent-62, score-0.743]
33 Our goal is to construct a high resolution depth image D whose resolution is equal to that of I. [sent-65, score-0.682]
34 Depths of pixels in a sparse grid in D are known (which we refer to as seed pixels) from the corresponding low resolution depth image D↓. [sent-66, score-0.771]
35 We propose computing the affinity measure between two pixels using geodesic distances defined on the image grid1, which can be considered as a two dimensional embedding in a joint color-spatial space. [sent-72, score-0.583]
36 Comparison between joint bilateral upsampling and joint geodesic upsampling. [sent-87, score-1.302]
37 The triangles indicate the locations where depth measurements are collected using a low resolution depth sensor. [sent-91, score-0.888]
38 (c) The upsampled depth profile using joint bilateral upsampling. [sent-93, score-0.608]
39 (d) The upsampled depth profile using the proposed joint geodesic upsampling algorithm. [sent-94, score-1.562]
40 The proposed method integrates color changes along the geodesic path and accurately recovers high resolution depth profile, whereas joint bilateral upsampling smooths depths across occlusion boundary resulting in blurring effect. [sent-95, score-2.021]
41 We use the Gaussian kernel to convert the geodesic distance into the affinity measure: gG(x,y) = exp(−d2G2σ(x2,y)) (4) where σ is the kernel bandwidth parameter. [sent-97, score-0.494]
42 Figure 1 shows a 1D illustration comparing joint bilateral upsampling and joint geodesic upsampling. [sent-98, score-1.302]
43 Joint geodesic upsampling integrates color changes along the geodesic curves; therefore it is sensitive to thin structures and fine scale changes, producing smooth surfaces with sharp occlusion boundaries (Figure 1d). [sent-100, score-1.766]
44 In contrast, the joint bilateral upsampling algorithm incorrectly propagates depths across the depth boundary due to Euclidean color distance computation (Figure 1c). [sent-101, score-1.425]
45 Fast Geodesic Upsampling Computation The upsampling formulation in (1) requires computa- tion of shortest paths from each pixel to all the seed pixels, which is equivalent to the all-pair-shortest-path problem. [sent-103, score-0.943]
46 Here we derive an approximate formulation of the upsampling operation and present an O(Kn) algorithm which achieves real-time performance. [sent-106, score-0.661]
47 First, we make the assumption that to compute the depth of a pixel it is sufficient to propagate information from its K “nearest“ depth pixels. [sent-110, score-0.743]
48 TKh(ex )K b-ene thaeres set approximation teos geodesic upsampling (e4l) xis. [sent-113, score-1.132]
49 K(x) Our second assumption is that if the two seed pixels are spatially far away, they are unlikely to be simultaneously in the y↓ set of K nearest depth pixels of a given pixel. [sent-117, score-0.643]
50 Our algorithm consists of three major processing steps, as summarized in Figure 2: 1) we demultiplex the pixels into K channels, 2) for each channel we compute geodesic distance transform, and 3) we interpolate the depths according to computed geodesic distances. [sent-119, score-1.098]
51 1) Demultiplexing: For an input low resolution depth image, we first partition its pixels into K separate channels which is called demultiplexing. [sent-121, score-0.618]
52 2) Geodesic Distance Transform (GDT): For each channel, we compute the geodesic distance transform [18], which provides the shortest distance from each pixel to the nearest seed point. [sent-127, score-0.797]
53 2) For each channel, we compute the geodesic distance transform, which provides the shortest distance from each pixel to the nearest seed pixel. [sent-131, score-0.749]
54 3) The depths of the seed pixels are propagated to the high resolution grid using the computed distances. [sent-132, score-0.519]
55 The geodesic distance transform solves the following optimization problem Mk(x) =y m∈SinkdG(x,y) (6) where dG (x, y) is defined in (3). [sent-134, score-0.49]
56 3) Interpolation: After computing geodesic distance transform for each channel, we propagate the sparse depths given by the low resolution depth image to the high resolution grid using the computed distances. [sent-146, score-1.359]
57 The geodesic distance transform not only provides the geodesic distance but also the nearest seed coordinate and hence its depth. [sent-147, score-1.11]
58 Let Mk be the geodesic distance transform for channel k, where the distance from a pixel x to its nearest seed point in channel k is given by Mk (x) and its coordinate is given by y↓k (x). [sent-148, score-0.849]
59 The approximate geodesic upsampling is then given by D(x) =k? [sent-149, score-1.065]
60 The shortest paths are defined on the high resolution grid which is sparsely covered by the seed pixels, and a path can reach a spatially distant seed pixel if the pixels along the path have similar colors. [sent-152, score-0.761]
61 We also show a new upsampling application using binary edge maps. [sent-157, score-0.682]
62 The DISC metric measures the error rate in the depth discontinuity region, while SRMS 6 scenes, namely art, books, dolls, laundry, moebius, and reindeer. [sent-164, score-0.492]
63 We generate the low resolution depth images by downsampling the original ones with the downsampling rate varying from 2x to 16x. [sent-166, score-0.583]
64 The task is to upsample the low resolution depth image to the original resolution using the registered high resolution color image. [sent-167, score-1.006]
65 Metrics: We use three performance metrics for evaluation: 1) the error rate on the depth discontinuity regions (DISC), 2) the root-mean-square error (RMS), and 3) the root-mean-square error in the smooth region (SRMS). [sent-168, score-0.558]
66 DISC measures the reconstruction error in the depth discontinuity regions, a standard metric for benchmarking stereo reconstruction algorithms [16]. [sent-169, score-0.514]
67 We first extract depth edges in the ground truth depth image. [sent-170, score-0.706]
68 We compare the upsampled depth map to the ground truth only in the extracted discontinuity regions. [sent-173, score-0.503]
69 RMS is commonly used for comparing depth upsampling algorithms [11, 14]. [sent-176, score-1.014]
70 However, we argue that RMS favors blurry depth boundaries, which produce major artifacts for depth upsampling. [sent-177, score-0.746]
71 Square error magnifies large few pixel errors, which are commonly generated by algorithms producing sharp depth boundaries but minimized by blurry edges. [sent-178, score-0.528]
72 Therefore, this metric is less suited for evaluation of upsampling performances. [sent-179, score-0.723]
73 SRMS is a modified version ofRMS error where the error is computed only in the smooth region given by the complement of the extracted depth discontinuity region. [sent-180, score-0.533]
74 This metric ensures that error in smooth regions is not dominated by few boundary pixels having large errors and is better suited for measuring upsampling accuracy in the smooth region. [sent-181, score-0.92]
75 Algorithms: We compare the proposed algorithm with several filtering-based depth upsampling algorithms, 3Depths are given by disparities in the dataset. [sent-182, score-1.014]
76 bilinear interpolation (BL), joint bilateral upsampling (JBU) [11], non-local means filtering (NLM) [3], minimalspanning-tree based cost aggregation (MST) [20], and guided image filtering (GIF) [9]. [sent-183, score-0.98]
77 JBU (discussed in Section 2) is a popular choice for depth upsampling [4, 7]. [sent-186, score-1.014]
78 We note that real-time upsampling performance can be achieved by using the fast implementations described in [21, 1]. [sent-187, score-0.683]
79 NLM was used as a system component for a recent depth upsampling work [14], which transfers depth information from similarly-colored patches. [sent-191, score-1.367]
80 For depth upsampling, a linear regression function from the color image to the depth image is estimated for each low resolution patch, which is then used to transfer higher resolution color image to the output depth image. [sent-194, score-1.497]
81 We modify the algorithm for depth upsampling by prorogating depths based on the distances on the spanning tree. [sent-198, score-1.157]
82 QUAD was proposed for colorization of gray-scale images [12] and recently modified for depth upsampling [19]. [sent-199, score-1.066]
83 Upsampling is formulated as a quadratic optimization problem where the cost function enforces color and depth correlation subject to the linear constraints given by the low resolution depth image. [sent-200, score-0.939]
84 Visual comparison of the depth upsampling results at 8x upsampling rate. [sent-205, score-1.675]
85 The results were averaged over all the images in the dataset at different upsampling rates. [sent-215, score-0.661]
86 It can be seen that the proposed algorithm provides better depth recovery in the smooth regions at 4x, 8x, and 16x upsampling rates. [sent-225, score-1.052]
87 The results obtained by the JBU algorithm tends to include depth bleeding artifacts especially when the upsampling rate is large. [sent-230, score-1.04]
88 Existence of a thin edge in-between has no effect on the upsampling process. [sent-232, score-0.745]
89 Our algorithm uses geodesic paths for interpolation and is free from this drawback. [sent-233, score-0.491]
90 Approximation and Computation Analysis Our optimization scheme produces an approximation to the geodesic upsampling operation as explained in Section 3. [sent-302, score-1.162]
91 Table 2 reports the processing time for a 695 555 image aatb l8ex 2 upsampling rpartoec. [sent-306, score-0.661]
92 s Wsien compare rth are 6e9 5v×ar5ia5n5ts i mofthe geodesic upsampling algorithm: 1) exact, 2) approximation using multi-pass geodesic distance transform, and 3) approximation using two-pass geodesic distance trans- Table 2. [sent-307, score-2.083]
93 Depth Upsampling Using Binary Edge Maps In the last experiment, we show that our method is well suited for upsampling depth images using binary images and present an application for sensor fusion. [sent-324, score-1.082]
94 Specifically, we upsample a depth image from a low resolution depth sensor with a high resolution depth boundary map from a multi-flash camera (MFC)4 as shown in Figure 6. [sent-325, score-1.534]
95 This forces only those depth pixels in the same depth continuous region are used to compute the depths of pixels in the region, achieving boundary-confined upsampling. [sent-328, score-0.926]
96 Several advantages exist for upsampling using a depth 4MFC is a camera design that exploits illumination pattern change due to LED flashes from different directions to compute a depth boundary map. [sent-330, score-1.407]
97 Depth upsampling using binary edge (c) Input high resolution depth boundary map maps given by multi-flash from multi-flash camera. [sent-333, score-1.25]
98 (d) 16x upsampled depth image using joint bilateral upsampling (e) 16x upsampled depth image using the proposed algorithm. [sent-337, score-1.657]
99 Conclusion We presented a new joint filtering algorithm using geodesic distances for upsampling depth images using high resolution color images. [sent-344, score-1.79]
100 We developed an efficient approximation to the upsampling filter and achieved real-time performance. [sent-345, score-0.728]
wordName wordTfidf (topN-words)
[('upsampling', 0.661), ('geodesic', 0.404), ('depth', 0.353), ('resolution', 0.154), ('seed', 0.129), ('bilateral', 0.111), ('depths', 0.108), ('jbu', 0.106), ('discontinuity', 0.092), ('srms', 0.088), ('quad', 0.087), ('disc', 0.082), ('rms', 0.069), ('approximation', 0.067), ('thin', 0.063), ('joint', 0.063), ('paths', 0.062), ('upsampled', 0.058), ('pixels', 0.056), ('shortest', 0.054), ('demultiplexing', 0.053), ('channel', 0.053), ('colorization', 0.052), ('grid', 0.051), ('color', 0.051), ('upsample', 0.05), ('nearest', 0.049), ('transaction', 0.048), ('filtering', 0.048), ('transform', 0.048), ('mk', 0.046), ('boundaries', 0.044), ('xp', 0.044), ('backward', 0.042), ('registered', 0.041), ('nlm', 0.041), ('mfc', 0.041), ('blurry', 0.04), ('suited', 0.04), ('boundary', 0.04), ('smooth', 0.038), ('dg', 0.038), ('distance', 0.038), ('pass', 0.038), ('pixel', 0.037), ('traverses', 0.036), ('backsurfacefrontsurface', 0.035), ('demultiplex', 0.035), ('dolls', 0.035), ('oncel', 0.035), ('distances', 0.035), ('path', 0.034), ('exact', 0.034), ('optical', 0.033), ('middlebury', 0.031), ('gif', 0.031), ('mainstream', 0.031), ('megapixels', 0.031), ('produces', 0.03), ('surfaces', 0.03), ('sharp', 0.029), ('sharpest', 0.029), ('sensor', 0.028), ('low', 0.028), ('sensitivity', 0.027), ('baek', 0.027), ('yatziv', 0.027), ('bandwidth', 0.027), ('channels', 0.027), ('forward', 0.026), ('bleeding', 0.026), ('taguchi', 0.025), ('marching', 0.025), ('error', 0.025), ('interpolation', 0.025), ('affinity', 0.025), ('merl', 0.024), ('rounded', 0.024), ('downsampling', 0.024), ('aggregation', 0.024), ('profile', 0.023), ('mst', 0.023), ('grows', 0.022), ('stereo', 0.022), ('metric', 0.022), ('implementations', 0.022), ('joining', 0.021), ('lischinski', 0.021), ('abrupt', 0.021), ('fine', 0.021), ('integrates', 0.021), ('edge', 0.021), ('high', 0.021), ('neighbors', 0.021), ('state', 0.021), ('door', 0.021), ('industrial', 0.02), ('band', 0.02), ('sensors', 0.02), ('shaped', 0.02)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000007 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
Author: Ming-Yu Liu, Oncel Tuzel, Yuichi Taguchi
Abstract: We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution one. Though this is closely related to the all-pairshortest-path problem which has O(n2 log n) complexity, we develop a novel approximation algorithm whose complexity grows linearly with the image size and achieve realtime performance. We compare our algorithm with the state of the art on the benchmark dataset and show that our approach provides more accurate depth upsampling with fewer artifacts. In addition, we show that the proposed algorithm is well suited for upsampling depth images using binary edge maps, an important sensor fusion application.
2 0.31616768 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
Author: Ju Shen, Sen-Ching S. Cheung
Abstract: The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature.
3 0.18368454 115 cvpr-2013-Depth Super Resolution by Rigid Body Self-Similarity in 3D
Author: unkown-author
Abstract: We tackle the problem of jointly increasing the spatial resolution and apparent measurement accuracy of an input low-resolution, noisy, and perhaps heavily quantized depth map. In stark contrast to earlier work, we make no use of ancillary data like a color image at the target resolution, multiple aligned depth maps, or a database of highresolution depth exemplars. Instead, we proceed by identifying and merging patch correspondences within the input depth map itself, exploiting patchwise scene self-similarity across depth such as repetition of geometric primitives or object symmetry. While the notion of ‘single-image ’ super resolution has successfully been applied in the context of color and intensity images, we are to our knowledge the first to present a tailored analogue for depth images. Rather than reason in terms of patches of 2D pixels as others have before us, our key contribution is to proceed by reasoning in terms of patches of 3D points, with matched patch pairs related by a respective 6 DoF rigid body motion in 3D. In support of obtaining a dense correspondence field in reasonable time, we introduce a new 3D variant of Patch- Match. A third contribution is a simple, yet effective patch upscaling and merging technique, which predicts sharp object boundaries at the target resolution. We show that our results are highly competitive with those of alternative techniques leveraging even a color image at the target resolution or a database of high-resolution depth exemplars.
4 0.18273744 186 cvpr-2013-GeoF: Geodesic Forests for Learning Coupled Predictors
Author: Peter Kontschieder, Pushmeet Kohli, Jamie Shotton, Antonio Criminisi
Abstract: Conventional decision forest based methods for image labelling tasks like object segmentation make predictions for each variable (pixel) independently [3, 5, 8]. This prevents them from enforcing dependencies between variables and translates into locally inconsistent pixel labellings. Random field models, instead, encourage spatial consistency of labels at increased computational expense. This paper presents a new and efficient forest based model that achieves spatially consistent semantic image segmentation by encoding variable dependencies directly in the feature space the forests operate on. Such correlations are captured via new long-range, soft connectivity features, computed via generalized geodesic distance transforms. Our model can be thought of as a generalization of the successful Semantic Texton Forest, Auto-Context, and Entangled Forest models. A second contribution is to show the connection between the typical Conditional Random Field (CRF) energy and the forest training objective. This analysis yields a new objective for training decision forests that encourages more accurate structured prediction. Our GeoF model is validated quantitatively on the task of semantic image segmentation, on four challenging and very diverse image datasets. GeoF outperforms both stateof-the-art forest models and the conventional pairwise CRF.
5 0.18223678 397 cvpr-2013-Simultaneous Super-Resolution of Depth and Images Using a Single Camera
Author: Hee Seok Lee, Kuoung Mu Lee
Abstract: In this paper, we propose a convex optimization framework for simultaneous estimation of super-resolved depth map and images from a single moving camera. The pixel measurement error in 3D reconstruction is directly related to the resolution of the images at hand. In turn, even a small measurement error can cause significant errors in reconstructing 3D scene structure or camera pose. Therefore, enhancing image resolution can be an effective solution for securing the accuracy as well as the resolution of 3D reconstruction. In the proposed method, depth map estimation and image super-resolution are formulated in a single energy minimization framework with a convex function and solved efficiently by a first-order primal-dual algorithm. Explicit inter-frame pixel correspondences are not required for our super-resolution procedure, thus we can avoid a huge computation time and obtain improved depth map in the accuracy and resolution as well as highresolution images with reasonable time. The superiority of our algorithm is demonstrated by presenting the improved depth map accuracy, image super-resolution results, and camera pose estimation.
6 0.16406097 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
8 0.15401882 169 cvpr-2013-Fast Patch-Based Denoising Using Approximated Patch Geodesic Paths
9 0.14471814 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
10 0.14015898 108 cvpr-2013-Dense 3D Reconstruction from Severely Blurred Images Using a Single Moving Camera
11 0.12205018 111 cvpr-2013-Dense Reconstruction Using 3D Object Shape Priors
12 0.12001012 205 cvpr-2013-Hollywood 3D: Recognizing Actions in 3D Natural Scenes
13 0.11518624 407 cvpr-2013-Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera
14 0.11392651 196 cvpr-2013-HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
15 0.10711596 114 cvpr-2013-Depth Acquisition from Density Modulated Binary Patterns
16 0.098173589 352 cvpr-2013-Recovering Stereo Pairs from Anaglyphs
17 0.096295513 357 cvpr-2013-Revisiting Depth Layers from Occlusions
18 0.096052773 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
19 0.095040433 307 cvpr-2013-Non-uniform Motion Deblurring for Bilayer Scenes
20 0.092468105 384 cvpr-2013-Segment-Tree Based Cost Aggregation for Stereo Matching
topicId topicWeight
[(0, 0.162), (1, 0.193), (2, 0.045), (3, 0.065), (4, -0.043), (5, -0.01), (6, -0.053), (7, 0.097), (8, -0.003), (9, -0.038), (10, -0.03), (11, -0.066), (12, 0.001), (13, 0.149), (14, 0.053), (15, -0.057), (16, -0.238), (17, -0.023), (18, -0.01), (19, -0.065), (20, 0.01), (21, 0.033), (22, -0.055), (23, -0.066), (24, 0.018), (25, 0.045), (26, 0.005), (27, -0.001), (28, -0.014), (29, 0.002), (30, -0.04), (31, 0.036), (32, 0.104), (33, -0.005), (34, -0.026), (35, -0.051), (36, 0.05), (37, -0.03), (38, -0.091), (39, 0.167), (40, 0.054), (41, -0.051), (42, -0.068), (43, 0.081), (44, -0.108), (45, 0.014), (46, -0.057), (47, -0.031), (48, -0.002), (49, 0.03)]
simIndex simValue paperId paperTitle
same-paper 1 0.97370917 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
Author: Ming-Yu Liu, Oncel Tuzel, Yuichi Taguchi
Abstract: We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution one. Though this is closely related to the all-pairshortest-path problem which has O(n2 log n) complexity, we develop a novel approximation algorithm whose complexity grows linearly with the image size and achieve realtime performance. We compare our algorithm with the state of the art on the benchmark dataset and show that our approach provides more accurate depth upsampling with fewer artifacts. In addition, we show that the proposed algorithm is well suited for upsampling depth images using binary edge maps, an important sensor fusion application.
2 0.88108242 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
Author: Ju Shen, Sen-Ching S. Cheung
Abstract: The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms oth- er techniques in the literature.
3 0.85989606 115 cvpr-2013-Depth Super Resolution by Rigid Body Self-Similarity in 3D
Author: unkown-author
Abstract: We tackle the problem of jointly increasing the spatial resolution and apparent measurement accuracy of an input low-resolution, noisy, and perhaps heavily quantized depth map. In stark contrast to earlier work, we make no use of ancillary data like a color image at the target resolution, multiple aligned depth maps, or a database of highresolution depth exemplars. Instead, we proceed by identifying and merging patch correspondences within the input depth map itself, exploiting patchwise scene self-similarity across depth such as repetition of geometric primitives or object symmetry. While the notion of ‘single-image ’ super resolution has successfully been applied in the context of color and intensity images, we are to our knowledge the first to present a tailored analogue for depth images. Rather than reason in terms of patches of 2D pixels as others have before us, our key contribution is to proceed by reasoning in terms of patches of 3D points, with matched patch pairs related by a respective 6 DoF rigid body motion in 3D. In support of obtaining a dense correspondence field in reasonable time, we introduce a new 3D variant of Patch- Match. A third contribution is a simple, yet effective patch upscaling and merging technique, which predicts sharp object boundaries at the target resolution. We show that our results are highly competitive with those of alternative techniques leveraging even a color image at the target resolution or a database of high-resolution depth exemplars.
4 0.79261726 114 cvpr-2013-Depth Acquisition from Density Modulated Binary Patterns
Author: Zhe Yang, Zhiwei Xiong, Yueyi Zhang, Jiao Wang, Feng Wu
Abstract: This paper proposes novel density modulated binary patterns for depth acquisition. Similar to Kinect, the illumination patterns do not need a projector for generation and can be emitted by infrared lasers and diffraction gratings. Our key idea is to use the density of light spots in the patterns to carry phase information. Two technical problems are addressed here. First, we propose an algorithm to design the patterns to carry more phase information without compromising the depth reconstruction from a single captured image as with Kinect. Second, since the carried phase is not strictly sinusoidal, the depth reconstructed from the phase contains a systematic error. We further propose a pixelbased phase matching algorithm to reduce the error. Experimental results show that the depth quality can be greatly improved using the phase carried by the density of light spots. Furthermore, our scheme can achieve 20 fps depth reconstruction with GPU assistance.
5 0.76483184 397 cvpr-2013-Simultaneous Super-Resolution of Depth and Images Using a Single Camera
Author: Hee Seok Lee, Kuoung Mu Lee
Abstract: In this paper, we propose a convex optimization framework for simultaneous estimation of super-resolved depth map and images from a single moving camera. The pixel measurement error in 3D reconstruction is directly related to the resolution of the images at hand. In turn, even a small measurement error can cause significant errors in reconstructing 3D scene structure or camera pose. Therefore, enhancing image resolution can be an effective solution for securing the accuracy as well as the resolution of 3D reconstruction. In the proposed method, depth map estimation and image super-resolution are formulated in a single energy minimization framework with a convex function and solved efficiently by a first-order primal-dual algorithm. Explicit inter-frame pixel correspondences are not required for our super-resolution procedure, thus we can avoid a huge computation time and obtain improved depth map in the accuracy and resolution as well as highresolution images with reasonable time. The superiority of our algorithm is demonstrated by presenting the improved depth map accuracy, image super-resolution results, and camera pose estimation.
6 0.72743362 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
8 0.68455857 407 cvpr-2013-Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera
9 0.67646945 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
10 0.64433444 56 cvpr-2013-Bayesian Depth-from-Defocus with Shading Constraints
11 0.62546754 196 cvpr-2013-HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences
12 0.58472377 219 cvpr-2013-In Defense of 3D-Label Stereo
13 0.56383765 428 cvpr-2013-The Episolar Constraint: Monocular Shape from Shadow Correspondence
14 0.55655736 307 cvpr-2013-Non-uniform Motion Deblurring for Bilayer Scenes
15 0.55603379 108 cvpr-2013-Dense 3D Reconstruction from Severely Blurred Images Using a Single Moving Camera
16 0.55249125 354 cvpr-2013-Relative Volume Constraints for Single View 3D Reconstruction
17 0.54922575 181 cvpr-2013-Fusing Depth from Defocus and Stereo with Coded Apertures
18 0.54656208 205 cvpr-2013-Hollywood 3D: Recognizing Actions in 3D Natural Scenes
19 0.51697397 230 cvpr-2013-Joint 3D Scene Reconstruction and Class Segmentation
20 0.47472736 111 cvpr-2013-Dense Reconstruction Using 3D Object Shape Priors
topicId topicWeight
[(10, 0.167), (16, 0.061), (26, 0.035), (28, 0.206), (33, 0.243), (67, 0.036), (69, 0.04), (87, 0.089), (91, 0.011)]
simIndex simValue paperId paperTitle
1 0.88262624 192 cvpr-2013-Graph Matching with Anchor Nodes: A Learning Approach
Author: Nan Hu, Raif M. Rustamov, Leonidas Guibas
Abstract: In this paper, we consider the weighted graph matching problem with partially disclosed correspondences between a number of anchor nodes. Our construction exploits recently introduced node signatures based on graph Laplacians, namely the Laplacian family signature (LFS) on the nodes, and the pairwise heat kernel map on the edges. In this paper, without assuming an explicit form of parametric dependence nor a distance metric between node signatures, we formulate an optimization problem which incorporates the knowledge of anchor nodes. Solving this problem gives us an optimized proximity measure specific to the graphs under consideration. Using this as a first order compatibility term, we then set up an integer quadratic program (IQP) to solve for a near optimal graph matching. Our experiments demonstrate the superior performance of our approach on randomly generated graphs and on two widelyused image sequences, when compared with other existing signature and adjacency matrix based graph matching methods.
2 0.87755841 428 cvpr-2013-The Episolar Constraint: Monocular Shape from Shadow Correspondence
Author: Austin Abrams, Kylia Miskell, Robert Pless
Abstract: Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from webcams “in . wu st l. edu (b)(c) the wild.”
3 0.86833805 4 cvpr-2013-3D Visual Proxemics: Recognizing Human Interactions in 3D from a Single Image
Author: Ishani Chakraborty, Hui Cheng, Omar Javed
Abstract: We present a unified framework for detecting and classifying people interactions in unconstrained user generated images. 1 Unlike previous approaches that directly map people/face locations in 2D image space into features for classification, we first estimate camera viewpoint and people positions in 3D space and then extract spatial configuration features from explicit 3D people positions. This approach has several advantages. First, it can accurately estimate relative distances and orientations between people in 3D. Second, it encodes spatial arrangements of people into a richer set of shape descriptors than afforded in 2D. Our 3D shape descriptors are invariant to camera pose variations often seen in web images and videos. The proposed approach also estimates camera pose and uses it to capture the intent of the photo. To achieve accurate 3D people layout estimation, we develop an algorithm that robustly fuses semantic constraints about human interpositions into a linear camera model. This enables our model to handle large variations in people size, heights (e.g. age) and poses. An accurate 3D layout also allows us to construct features informed by Proxemics that improves our semantic classification. To characterize the human interaction space, we introduce visual proxemes; a set of prototypical patterns that represent commonly occurring social interactions in events. We train a discriminative classifier that classifies 3D arrangements of people into visual proxemes and quantitatively evaluate the performance on a large, challenging dataset.
4 0.86037236 261 cvpr-2013-Learning by Associating Ambiguously Labeled Images
Author: Zinan Zeng, Shijie Xiao, Kui Jia, Tsung-Han Chan, Shenghua Gao, Dong Xu, Yi Ma
Abstract: We study in this paper the problem of learning classifiers from ambiguously labeled images. For instance, in the collection of new images, each image contains some samples of interest (e.g., human faces), and its associated caption has labels with the true ones included, while the samplelabel association is unknown. The task is to learn classifiers from these ambiguously labeled images and generalize to new images. An essential consideration here is how to make use of the information embedded in the relations between samples and labels, both within each image and across the image set. To this end, we propose a novel framework to address this problem. Our framework is motivated by the observation that samples from the same class repetitively appear in the collection of ambiguously labeled training images, while they are just ambiguously labeled in each image. If we can identify samples of the same class from each image and associate them across the image set, the matrix formed by the samples from the same class would be ideally low-rank. By leveraging such a low-rank assump- tion, we can simultaneously optimize a partial permutation matrix (PPM) for each image, which is formulated in order to exploit all information between samples and labels in a principled way. The obtained PPMs can be readily used to assign labels to samples in training images, and then a standard SVM classifier can be trained and used for unseen data. Experiments on benchmark datasets show the effectiveness of our proposed method.
same-paper 5 0.85952413 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
Author: Ming-Yu Liu, Oncel Tuzel, Yuichi Taguchi
Abstract: We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution one. Though this is closely related to the all-pairshortest-path problem which has O(n2 log n) complexity, we develop a novel approximation algorithm whose complexity grows linearly with the image size and achieve realtime performance. We compare our algorithm with the state of the art on the benchmark dataset and show that our approach provides more accurate depth upsampling with fewer artifacts. In addition, we show that the proposed algorithm is well suited for upsampling depth images using binary edge maps, an important sensor fusion application.
6 0.85425526 134 cvpr-2013-Discriminative Sub-categorization
7 0.8396554 328 cvpr-2013-Pedestrian Detection with Unsupervised Multi-stage Feature Learning
8 0.82363766 234 cvpr-2013-Joint Spectral Correspondence for Disparate Image Matching
10 0.82203239 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
11 0.82166475 225 cvpr-2013-Integrating Grammar and Segmentation for Human Pose Estimation
12 0.81980163 406 cvpr-2013-Spatial Inference Machines
13 0.81972045 104 cvpr-2013-Deep Convolutional Network Cascade for Facial Point Detection
14 0.81832665 206 cvpr-2013-Human Pose Estimation Using Body Parts Dependent Joint Regressors
15 0.81766474 393 cvpr-2013-Separating Signal from Noise Using Patch Recurrence across Scales
16 0.81759942 394 cvpr-2013-Shading-Based Shape Refinement of RGB-D Images
17 0.81750906 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
18 0.81405443 447 cvpr-2013-Underwater Camera Calibration Using Wavelength Triangulation
20 0.81293774 170 cvpr-2013-Fast Rigid Motion Segmentation via Incrementally-Complex Local Models