iccv iccv2013 iccv2013-101 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Yaron Eshet, Simon Korman, Eyal Ofek, Shai Avidan
Abstract: We extend patch based methods to work on patches in 3D space. We start with Coherency Sensitive Hashing [12] (CSH), which is an algorithm for matching patches between two RGB images, and extend it to work with RGBD images. This is done by warping all 3D patches to a common virtual plane in which CSH is performed. To avoid noise due to warping of patches of various normals and depths, we estimate a group of dominant planes and compute CSH on each plane separately, before merging the matching patches. The result is DCSH - an algorithm that matches world (3D) patches in order to guide the search for image plane matches. An independent contribution is an extension of CSH, which we term Social-CSH. It allows a major speedup of the k nearest neighbor (kNN) version of CSH - its runtime growing linearly, rather than quadratically, in k. Social-CSH is used as a subcomponent of DCSH when many NNs are required, as in the case of image denoising. We show the benefits ofusing depth information to image reconstruction and image denoising, demonstrated on several RGBD images.
Reference: text
sentIndex sentText sentNum sentScore
1 DCSH - Matching Patches in RGBD Images Yaron Eshet Tel-Aviv University Simon Korman Tel-Aviv University Abstract We extend patch based methods to work on patches in 3D space. [sent-1, score-0.465]
2 We start with Coherency Sensitive Hashing [12] (CSH), which is an algorithm for matching patches between two RGB images, and extend it to work with RGBD images. [sent-2, score-0.285]
3 This is done by warping all 3D patches to a common virtual plane in which CSH is performed. [sent-3, score-0.391]
4 To avoid noise due to warping of patches of various normals and depths, we estimate a group of dominant planes and compute CSH on each plane separately, before merging the matching patches. [sent-4, score-0.454]
5 The result is DCSH - an algorithm that matches world (3D) patches in order to guide the search for image plane matches. [sent-5, score-0.422]
6 It allows a major speedup of the k nearest neighbor (kNN) version of CSH - its runtime growing linearly, rather than quadratically, in k. [sent-7, score-0.176]
7 Introduction Patch based methods rely on the observation that local image patches occur frequently within an image. [sent-11, score-0.244]
8 Virtually all patch based methods use square patches and measure similarity between patches using the Sum-ofSquared-Distances (SSD), no doubt for computational efficiency. [sent-13, score-0.733]
9 But these image patches are the deformed projections of patches in 3D. [sent-14, score-0.488]
10 We therefore propose to use patches in 3D space, in order to increase the quantity and the quality of similar patches. [sent-15, score-0.283]
11 In particular we propose to extend patch based methods to work on RGBD images. [sent-16, score-0.221]
12 Clearly, patch matching in 3D induces patch matches in the 2D image plane, which are defined by homographies (projective transformations). [sent-17, score-0.647]
13 As a core technology we develop DCSH, a method for matching patches between Eyal Ofek Shai Avidan Microsoft Research Tel-Aviv University (a) (b) Figure 1. [sent-19, score-0.285]
14 DCSH: (a) Each image pixel represents a point in 3D space and a normal direction according to the depth map. [sent-20, score-0.208]
15 (b) Each world patch (Pi) is projected to some quadrilateral (pi) on the image plane, by some homography (Hi). [sent-21, score-0.416]
16 The fact that world patches are repetitive in the scene is used to guide the search of similar (projected) patches in the image plane. [sent-22, score-0.537]
17 Specifically, given an RGBD image we use the depth values to compute the depth and normal of every patch and warp the patches to some virtual reference plane. [sent-25, score-0.788]
18 Experiments show that this depth information considerably improves the quality of patch matching. [sent-27, score-0.358]
19 Still, a single virtual plane might introduce strong warping and resampling artifacts that will affect matching, especially for patches with orientation that is perpendicular to that of the virtual plane. [sent-28, score-0.518]
20 SocialCSH finds only a small number of matches for each patch and enriches the list of matching candidates by incorporating their own candidate matches. [sent-34, score-0.361]
21 To do so, we used high quality color images, aligned using a multi-view stereo algorithm, resulting in a single RGBD image with reliable depth and high quality RGB components1 . [sent-39, score-0.176]
22 Background At their core, patch-based methods require efficient Approximate Nearest Neighbor (ANN) algorithms to find similar patches to each query patch. [sent-42, score-0.244]
23 PatchMatch [4], which is an extremely efficient algorithm that works by randomly finding possible patch candidates and propagating good matches across the image plane. [sent-45, score-0.279]
24 Such algorithms consider matching patches under 2D translations only, though some recent works have considered wider classes of transformations. [sent-47, score-0.285]
25 scale, rotation) do not suffice to capture the repetitive nature of patches in the 3D world. [sent-53, score-0.244]
26 An alternative method could search for matches on a dense SIFT [14] field, and since SIFT is a stable descriptor under affine transformations this can be seen as a proxy to patch matching in 3D space. [sent-54, score-0.346]
27 Given only a target image B and an NNF from a source image A to B, the goal is to reconstruct image A using the patches of B. [sent-57, score-0.244]
28 This is a standard building block in many patch based methods for image enhancement, such as denoising, super-resolution and retargetting. [sent-58, score-0.221]
29 [17] proposed a Multi-View Image Denoising algorithm where the goal is to collect similar patches across multiple views with independent noise, where depth is treated as a latent variable to be estimated from the data. [sent-66, score-0.342]
30 With RGBD images there are no multiple views available to aid the denoising process, but we show that single image denoising can still benefit from the use of depth information. [sent-69, score-0.416]
31 There is also some research on denoising Kinect images, where the goal is to denoise the depth map produced by the sensor. [sent-70, score-0.285]
32 [16] produce a high quality depth map by upsampling the original depth map using the high quality RGB image. [sent-72, score-0.274]
33 We deal with a different setting of denoising the RGB component of an RGBD image, using depth as a cue. [sent-73, score-0.257]
34 The general idea is to use the better matchings that occur between real-world patches in order to find correspondences between their projected image-patches. [sent-76, score-0.244]
35 2 we present a more × general patch normalization scheme, which will be used in the final DCSH matching algorithm, which is described in Algorithm 1. [sent-81, score-0.345]
36 Simulating a (per pixel) fronto-parallel view Given the 3D world coordinates ra = (Xa , Ya, Za) associated with each image pixel a, we first use a standard robust estimation of the normal direction na at the 3D point. [sent-84, score-0.246]
37 That is, we take na to be the least-squares solution to the stack of 49 equations of the form rb · na = 1for each pixel b in the 7 7 neighborhood of a. [sent-85, score-0.18]
38 Simulating a (per pixel) fronto-parallel view: (a) For each (green) pixel in the image, we simulate its local appearance, as if the surface was captured from a fronto-parallel view at a distance of zref . [sent-87, score-0.171]
39 (b) The inverse homography ha−1 can be used to sample a normalized patch (green) around the pixel. [sent-91, score-0.397]
40 We then (in-plane) rotate the normalized patch (by a rotation matrix Ra) such that it faces its dominant RGB texture orientation (white arrow). [sent-92, score-0.38]
41 A new normalized patch can be sampled using R−a1 · ha−1 (further details in text). [sent-93, score-0.278]
42 ×× Once we know the 3D location ra and normal na, we turn to compute the homography Ha that will enable sampling the surface at ra from the direction of na at a fixed distance of zref. [sent-94, score-0.409]
43 A world-patch, located at the origin and facing direction of z = (0, 0, 1), is first rotated in 3D so that its normal coincides with the surface normal direction na. [sent-96, score-0.192]
44 (1) One remaining degree of freedom of the warp is the surface in-plane rotation (perpendicular to the normal) which determines the orientation of the normalized patch. [sent-103, score-0.207]
45 We use the orientation normalization technique of SIFT [14], where a prominent orientation is chosen according to a weighted voting scheme, based on orientations and magnitudes of grayscale intensity gradients in a neighborhood of the central pixel. [sent-104, score-0.185]
46 The recovered in-plane rotation Ra is used to produce the final homography Ha = ha · Ra. [sent-105, score-0.236]
47 Simulating (several) general views In the previous section we normalized each pixel location to a patch, representing a canonical fronto-parallel 2For any vector v, ˆv is the its unit normalized version. [sent-108, score-0.187]
48 image clustered to 10 prototypical viewpoints, color-coded from 1 to 10 (Areas with invalid depth are coded with 0, dark blue). [sent-109, score-0.254]
49 Since pixel resolution in the image is limited, the normalization process for areas in the surface whose natural camera viewpoint is very different from being both fronto-parallel and at depth zref, will introduce warping and resampling errors that affect the quality of the matching. [sent-111, score-0.496]
50 For example, areas that are close to the camera (with depth smaller than zref) will be compared based on their normalized versions, which lack relevant information, e. [sent-112, score-0.23]
51 The normalization to different viewpoints results in a rich variety of candidate patches which will enable improved patch matching. [sent-116, score-0.586]
52 The gold standard, in this sense, would be to normalize all of the image patches to every single viewing point of each of the image’s patches (leaving the target image patch unchanged under the normalization). [sent-117, score-0.709]
53 This of course is infeasible, and we therefore compromise between speed and accuracy by selecting a set of L prototypical viewpoints, which are found by a clustering process on the surface normals and depths (these determine the natural viewpoint). [sent-118, score-0.263]
54 Each representative cluster center [xi, yi, zi, nix, niy] represents a specific prototypical viewpoint and induces a ho- mography Hi (following the exact formulation in the Section 3. [sent-124, score-0.193]
55 See Figure 3 for an example of clustering an image to L=10 areas with prototypical viewpoints. [sent-126, score-0.207]
56 The normalization of any patch a to the i’th viewpoint can now be synthesized by resampling through the concatenated homography: Na = Hi · H−a1 . [sent-127, score-0.375]
57 Nearest neighbor search The procedure above produces a (square) normalized patch for each location in images A and B. [sent-130, score-0.376]
58 The CSH al91 Algorithm 1DCSH: Matching Patches in RGBD Images Input: RGBD images A and B, intrinsic matrix K Output: k patch mappings (homographies) per pixel a ∈ A Step 1: Homographies to a fronto-parallel plane zref 1. [sent-131, score-0.443]
59 2) - Fit L homographies {Hi}iL=1 (each induced by a prototypical viewpoint). [sent-144, score-0.209]
60 For each prototypical homography Hi: (a) Create a normalized patch per location a ∈ A (Carneda tbe ∈ a B no) by sampling: hN ap =r Hocia ·t iHon−a1 a a· pa . [sent-146, score-0.608]
61 While it usually works with the entire set of overlapping square patches of an image, here, one patch per location is given to the algorithm, but these patches do not overlap in the regular sense. [sent-156, score-0.781]
62 This fact required the preprocessing stage of Walsh-Hadamard-Kernel patch projections to be computed directly (rather than using the more advanced GreyCode Kernel method [6], which requires true overlapping). [sent-157, score-0.221]
63 In addition, since neighboring patches underwent different homographies and orientation corrections, patch matches in the image plane are propagated in all four directions (up/down/left/right) rather than in the single expected di- rection. [sent-158, score-0.722]
64 Once matches have been found, we no longer need the ’bridging’ normalized patches and we turn back to the original image patches and construct direct mappings between them, avoiding excess interpolation and resampling. [sent-159, score-0.603]
65 Namely, if the normalized version H−b1 (pb) of the patch pb ∈ B was matched to H−a1 (pa) (a normalized version of a patch pa s∈ m Aat)c, we directly link the corresponding locations using t∈he A c)o,m wbein deidre homography: oHrrabe =: Hdi−ang1 ·l oHcab. [sent-160, score-0.625]
66 Simply put, this happens since each patch evaluates the k NNs of its k NNs. [sent-165, score-0.221]
67 In this process, given only a target image B and an NNF from a source image A to B, it is required to reconstruct image A using only the patches of B. [sent-224, score-0.244]
68 Row 3: Estimated normals maps, where gray areas are invalid due to noisy or missing depth values. [sent-230, score-0.231]
69 In the reconstructbioetnw process, aereach 8 patch ims replaced by . [sent-232, score-0.221]
70 it Isn nn etharees rte neighbor patch and since the NNF is dense, each final pixel will be an average of the 64 pixel values it receives through the 64 patches that contain it. [sent-233, score-0.608]
71 This is due to the fact that the matches were retrieved in a normalized plane, so the gaussian weighing should be done in the normalized plane itself, rather than on the image plane. [sent-239, score-0.243]
72 We therefore use at each patch p centered at pixel a, the same Gaussian weight kernel G after warping it back to image A, using the relevant inverse homography Ha, namely, H−a1 · G. [sent-240, score-0.427]
73 The introduction of patch normalization reduces the re- construction error dramatically, while the addition of orientation normalization (as done in SIFT) gives an additional improvement. [sent-254, score-0.438]
74 RGBD image denoising We choose to demonstrate the added value of using world 3D patch matches through a simple denoising pipeline. [sent-260, score-0.62]
75 These patches are then used to construct a lowrank PCA space, and denoising the original patch amounts to projecting it on the subspace and storing the denoised patch back in the image. [sent-262, score-0.92]
76 × Step 1: Find K NNs per patch In this stage we find k=200 NNs per image patch. [sent-266, score-0.267]
77 The reason we use the combination is that the 2D CSH patches have the advantage of not undergoing any kind of warping. [sent-268, score-0.244]
78 4 Step 2: Denoise each image patch using PCA Formally, let p be the query RGB patch (represented as a flat vector of length L) and let P = {pi}im=1 be the set of m matching patches )fo aunndd l eint step =1 . [sent-269, score-0.727]
79 We take an SVD decomposition of A = UDVT, and project p on the eigenvectors of the top c eigenvalues to obtain the denoised patch p? [sent-274, score-0.296]
80 [17] to automatically find the preferred dimensionality c, separately for each set of matching patches Step 3: Integrate the denoised patches to form the denoised image Using 8 8 patches, 64 values are assigned to each pixel andU are averaged attoc ohebsta,i 6n4 th vael udeenso airseed as image. [sent-278, score-0.727]
81 • DCSH-PCA: Steps 1-3, where in step 1 we use the bDeCstS 2H0-0P patches posut 1 o-3f:, w20he0r e2D in patches w(feo uunsed by CSH) in addition to 200 3D patches found by DCSH. [sent-283, score-0.732]
82 1 RGBD image denoising experiments In this section, we experiment with the 3 denoising versions. [sent-286, score-0.318]
83 mean gradient magnitude) of a patch - the lower the density of similar patches in the image (see e. [sent-292, score-0.465]
84 Another implication is that textured patches typically have few good NNs throughout the image. [sent-298, score-0.298]
85 For such patches, the introduction ofnormalized (3D) patches allows to increase the density of similar patches in the image, by searching across general homographies (image plane scales and orientations as well as out-of-image-plane rotations). [sent-299, score-0.636]
86 We therefore expect the main impact of depth normalization to occur in textured areas of the image. [sent-300, score-0.28]
87 An insight into the contribution of 3D patches to the denoising process can be observed by examining the statistics of the patches that manage to get into the final list of 200 patches fed to the PCA process. [sent-303, score-0.932]
88 Over all images, an average of around 80% of the patches originated from the (a)(b)(c) Figure 7. [sent-304, score-0.244]
89 A detailed example (see text for discussion): (a) noisy image (b) fraction of normalized (as opposed to regular) patches automatically chosen by the algorithm (the range [0,1] color coded by [blue,red]), (c) ’winner’ areas - DCSH (green) vs. [sent-305, score-0.401]
90 This can be seen visually in Figure 7 (b), where the per-patch ratio (in [0,1]) of 3D patches vs. [sent-308, score-0.244]
91 Namely, red areas are those where the vast majority of contributing patches came from the normalized list. [sent-310, score-0.376]
92 DCSH-PCA) the contribution of adding depth normalization is evident across all examples and amounts to an average of 0. [sent-313, score-0.181]
93 PSNR denoising results on iPhone data-set: The usage of 3D patches results in significantly improved denoising (comparing CSH-PCA to DCSH-PCA) across all images with an average gain of 0. [sent-319, score-0.592]
94 Across the different images, the contribution of (3D) patch normalization can be seen by comparing columns ’2D’ (CSH-PCA) and ’3D’ (DCSHPCA) (notice especially the doll’s ear, where fine details are revealed, or the cleaner letters in the text). [sent-324, score-0.304]
95 Conclusions We extended patch based methods to work on patches in 3D space. [sent-331, score-0.465]
96 In particular, we extended the CSH patch matching algorithm to work with RGBD images. [sent-332, score-0.262]
97 The novel algorithm, DCSH, runs CSH on a set of planes, representing prototypical viewpoints in the image. [sent-333, score-0.17]
98 We showed the added value of using depth information for improving the quality of patch matching and in partic5Note that our method uses depth information, which BM3D does not. [sent-337, score-0.497]
99 ’C&N;’: Clean patch (top) and Noisy patch (bottom), ’2D’: CSH-PCA denoised (top) and RMSE (bottom), and similarly for the others: ’3D’: DCSH-PCA, ’3D-BI’: DCSH-PCA-BI, ’BM3D’: BM3D. [sent-341, score-0.517]
100 The results point to depth as a new source of information in patch based methods and suggest that DCSH could prove useful in other ap- plications, such as super-resolution, inpainting and retargetting. [sent-344, score-0.319]
wordName wordTfidf (topN-words)
[('csh', 0.528), ('dcsh', 0.422), ('patches', 0.244), ('rgbd', 0.231), ('patch', 0.221), ('nns', 0.207), ('denoising', 0.159), ('prototypical', 0.132), ('homography', 0.119), ('nnf', 0.107), ('iphone', 0.099), ('depth', 0.098), ('ha', 0.089), ('rmse', 0.086), ('normalization', 0.083), ('zref', 0.08), ('homographies', 0.077), ('areas', 0.075), ('denoised', 0.075), ('rgb', 0.072), ('plane', 0.071), ('knn', 0.067), ('na', 0.066), ('coherency', 0.064), ('normal', 0.062), ('matches', 0.058), ('normalized', 0.057), ('bilateral', 0.056), ('quadrilateral', 0.053), ('quadratically', 0.051), ('patchmatch', 0.051), ('orientation', 0.051), ('simulating', 0.05), ('korman', 0.049), ('reconstruction', 0.049), ('speedup', 0.048), ('pixel', 0.048), ('neighbor', 0.047), ('hashing', 0.047), ('ra', 0.047), ('buades', 0.044), ('runtime', 0.043), ('surface', 0.043), ('list', 0.041), ('dress', 0.041), ('matching', 0.041), ('graylevels', 0.04), ('nlm', 0.04), ('socialcsh', 0.04), ('quality', 0.039), ('warping', 0.039), ('barnes', 0.039), ('resampling', 0.039), ('nearest', 0.038), ('viewpoints', 0.038), ('pb', 0.038), ('virtual', 0.037), ('hi', 0.036), ('ann', 0.035), ('normals', 0.034), ('viewpoint', 0.032), ('za', 0.031), ('subimage', 0.031), ('subroutine', 0.031), ('pa', 0.031), ('gain', 0.03), ('coll', 0.03), ('implication', 0.03), ('friends', 0.03), ('shechtman', 0.029), ('induces', 0.029), ('depths', 0.029), ('denoise', 0.028), ('hdi', 0.028), ('rotation', 0.028), ('versions', 0.028), ('warp', 0.028), ('cup', 0.027), ('ssd', 0.026), ('search', 0.026), ('grows', 0.025), ('speed', 0.025), ('origin', 0.025), ('sensors', 0.025), ('run', 0.025), ('webpage', 0.025), ('location', 0.025), ('planes', 0.025), ('merged', 0.025), ('text', 0.025), ('square', 0.024), ('invalid', 0.024), ('tables', 0.024), ('textured', 0.024), ('pipeline', 0.024), ('kinect', 0.024), ('per', 0.023), ('goldman', 0.023), ('texture', 0.023), ('world', 0.023)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000001 101 iccv-2013-DCSH - Matching Patches in RGBD Images
Author: Yaron Eshet, Simon Korman, Eyal Ofek, Shai Avidan
Abstract: We extend patch based methods to work on patches in 3D space. We start with Coherency Sensitive Hashing [12] (CSH), which is an algorithm for matching patches between two RGB images, and extend it to work with RGBD images. This is done by warping all 3D patches to a common virtual plane in which CSH is performed. To avoid noise due to warping of patches of various normals and depths, we estimate a group of dominant planes and compute CSH on each plane separately, before merging the matching patches. The result is DCSH - an algorithm that matches world (3D) patches in order to guide the search for image plane matches. An independent contribution is an extension of CSH, which we term Social-CSH. It allows a major speedup of the k nearest neighbor (kNN) version of CSH - its runtime growing linearly, rather than quadratically, in k. Social-CSH is used as a subcomponent of DCSH when many NNs are required, as in the case of image denoising. We show the benefits ofusing depth information to image reconstruction and image denoising, demonstrated on several RGBD images.
2 0.16539359 394 iccv-2013-Single-Patch Low-Rank Prior for Non-pointwise Impulse Noise Removal
Author: Ruixuan Wang, Emanuele Trucco
Abstract: This paper introduces a ‘low-rank prior’ for small oriented noise-free image patches: considering an oriented patch as a matrix, a low-rank matrix approximation is enough to preserve the texture details in the properly oriented patch. Based on this prior, we propose a single-patch method within a generalized joint low-rank and sparse matrix recovery framework to simultaneously detect and remove non-pointwise random-valued impulse noise (e.g., very small blobs). A weighting matrix is incorporated in the framework to encode an initial estimate of the spatial noise distribution. An accelerated proximal gradient method is adapted to estimate the optimal noise-free image patches. Experiments show the effectiveness of our framework in removing non-pointwise random-valued impulse noise.
3 0.15868263 424 iccv-2013-Tracking Revisited Using RGBD Camera: Unified Benchmark and Baselines
Author: Shuran Song, Jianxiong Xiao
Abstract: Despite significant progress, tracking is still considered to be a very challenging task. Recently, the increasing popularity of depth sensors has made it possible to obtain reliable depth easily. This may be a game changer for tracking, since depth can be used to prevent model drift and handle occlusion. We also observe that current tracking algorithms are mostly evaluated on a very small number of videos collectedandannotated by different groups. The lack of a reasonable size and consistently constructed benchmark has prevented a persuasive comparison among different algorithms. In this paper, we construct a unified benchmark dataset of 100 RGBD videos with high diversity, propose different kinds of RGBD tracking algorithms using 2D or 3D model, and present a quantitative comparison of various algorithms with RGB or RGBD input. We aim to lay the foundation for further research in both RGB and RGBD tracking, and our benchmark is available at http://tracking.cs.princeton.edu.
4 0.13870811 387 iccv-2013-Shape Anchors for Data-Driven Multi-view Reconstruction
Author: Andrew Owens, Jianxiong Xiao, Antonio Torralba, William Freeman
Abstract: We present a data-driven method for building dense 3D reconstructions using a combination of recognition and multi-view cues. Our approach is based on the idea that there are image patches that are so distinctive that we can accurately estimate their latent 3D shapes solely using recognition. We call these patches shape anchors, and we use them as the basis of a multi-view reconstruction system that transfers dense, complex geometry between scenes. We “anchor” our 3D interpretation from these patches, using them to predict geometry for parts of the scene that are relatively ambiguous. The resulting algorithm produces dense reconstructions from stereo point clouds that are sparse and noisy, and we demonstrate it on a challenging dataset of real-world, indoor scenes.
5 0.12397157 293 iccv-2013-Nonparametric Blind Super-resolution
Author: Tomer Michaeli, Michal Irani
Abstract: Super resolution (SR) algorithms typically assume that the blur kernel is known (either the Point Spread Function ‘PSF’ of the camera, or some default low-pass filter, e.g. a Gaussian). However, the performance of SR methods significantly deteriorates when the assumed blur kernel deviates from the true one. We propose a general framework for “blind” super resolution. In particular, we show that: (i) Unlike the common belief, the PSF of the camera is the wrong blur kernel to use in SR algorithms. (ii) We show how the correct SR blur kernel can be recovered directly from the low-resolution image. This is done by exploiting the inherent recurrence property of small natural image patches (either internally within the same image, or externally in a collection of other natural images). In particular, we show that recurrence of small patches across scales of the low-res image (which forms the basis for single-image SR), can also be used for estimating the optimal blur kernel. This leads to significant improvement in SR results.
6 0.10874195 23 iccv-2013-A New Image Quality Metric for Image Auto-denoising
7 0.10602347 156 iccv-2013-Fast Direct Super-Resolution by Simple Functions
8 0.10299322 209 iccv-2013-Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation
9 0.095096678 223 iccv-2013-Joint Noise Level Estimation from Personal Photo Collections
10 0.090929657 153 iccv-2013-Face Recognition Using Face Patch Networks
11 0.088680349 444 iccv-2013-Viewing Real-World Faces in 3D
12 0.087148219 105 iccv-2013-DeepFlow: Large Displacement Optical Flow with Deep Matching
13 0.086184025 9 iccv-2013-A Flexible Scene Representation for 3D Reconstruction Using an RGB-D Camera
14 0.085885994 174 iccv-2013-Forward Motion Deblurring
15 0.085404173 36 iccv-2013-Accurate and Robust 3D Facial Capture Using a Single RGBD Camera
16 0.084495656 133 iccv-2013-Efficient Hand Pose Estimation from a Single Depth Image
17 0.084474377 319 iccv-2013-Point-Based 3D Reconstruction of Thin Objects
18 0.084221907 410 iccv-2013-Support Surface Prediction in Indoor Scenes
19 0.083177157 404 iccv-2013-Structured Forests for Fast Edge Detection
20 0.082621604 309 iccv-2013-Partial Enumeration and Curvature Regularization
topicId topicWeight
[(0, 0.173), (1, -0.114), (2, -0.058), (3, -0.015), (4, -0.043), (5, 0.021), (6, 0.008), (7, -0.106), (8, -0.059), (9, -0.016), (10, -0.013), (11, -0.012), (12, 0.043), (13, -0.001), (14, 0.029), (15, -0.04), (16, -0.019), (17, -0.073), (18, 0.004), (19, 0.014), (20, -0.065), (21, 0.103), (22, 0.017), (23, 0.024), (24, 0.035), (25, 0.072), (26, -0.006), (27, 0.005), (28, -0.091), (29, -0.008), (30, -0.023), (31, 0.011), (32, 0.088), (33, 0.13), (34, -0.064), (35, -0.084), (36, 0.103), (37, -0.071), (38, 0.115), (39, -0.088), (40, -0.116), (41, 0.079), (42, 0.007), (43, -0.125), (44, 0.107), (45, 0.025), (46, 0.06), (47, 0.045), (48, 0.058), (49, -0.02)]
simIndex simValue paperId paperTitle
same-paper 1 0.96303427 101 iccv-2013-DCSH - Matching Patches in RGBD Images
Author: Yaron Eshet, Simon Korman, Eyal Ofek, Shai Avidan
Abstract: We extend patch based methods to work on patches in 3D space. We start with Coherency Sensitive Hashing [12] (CSH), which is an algorithm for matching patches between two RGB images, and extend it to work with RGBD images. This is done by warping all 3D patches to a common virtual plane in which CSH is performed. To avoid noise due to warping of patches of various normals and depths, we estimate a group of dominant planes and compute CSH on each plane separately, before merging the matching patches. The result is DCSH - an algorithm that matches world (3D) patches in order to guide the search for image plane matches. An independent contribution is an extension of CSH, which we term Social-CSH. It allows a major speedup of the k nearest neighbor (kNN) version of CSH - its runtime growing linearly, rather than quadratically, in k. Social-CSH is used as a subcomponent of DCSH when many NNs are required, as in the case of image denoising. We show the benefits ofusing depth information to image reconstruction and image denoising, demonstrated on several RGBD images.
2 0.81002659 394 iccv-2013-Single-Patch Low-Rank Prior for Non-pointwise Impulse Noise Removal
Author: Ruixuan Wang, Emanuele Trucco
Abstract: This paper introduces a ‘low-rank prior’ for small oriented noise-free image patches: considering an oriented patch as a matrix, a low-rank matrix approximation is enough to preserve the texture details in the properly oriented patch. Based on this prior, we propose a single-patch method within a generalized joint low-rank and sparse matrix recovery framework to simultaneously detect and remove non-pointwise random-valued impulse noise (e.g., very small blobs). A weighting matrix is incorporated in the framework to encode an initial estimate of the spatial noise distribution. An accelerated proximal gradient method is adapted to estimate the optimal noise-free image patches. Experiments show the effectiveness of our framework in removing non-pointwise random-valued impulse noise.
3 0.69213521 23 iccv-2013-A New Image Quality Metric for Image Auto-denoising
Author: Xiangfei Kong, Kuan Li, Qingxiong Yang, Liu Wenyin, Ming-Hsuan Yang
Abstract: This paper proposes a new non-reference image quality metric that can be adopted by the state-of-the-art image/video denoising algorithms for auto-denoising. The proposed metric is extremely simple and can be implemented in four lines of Matlab code1. The basic assumption employed by the proposed metric is that the noise should be independent of the original image. A direct measurement of this dependence is, however, impractical due to the relatively low accuracy of existing denoising method. The proposed metric thus aims at maximizing the structure similarity between the input noisy image and the estimated image noise around homogeneous regions and the structure similarity between the input noisy image and the denoised image around highly-structured regions, and is computed as the linear correlation coefficient of the two corresponding structure similarity maps. Numerous experimental results demonstrate that the proposed metric not only outperforms the current state-of-the-art non-reference quality metric quantitatively and qualitatively, but also better maintains temporal coherence when used for video denoising. ˜
4 0.66756546 387 iccv-2013-Shape Anchors for Data-Driven Multi-view Reconstruction
Author: Andrew Owens, Jianxiong Xiao, Antonio Torralba, William Freeman
Abstract: We present a data-driven method for building dense 3D reconstructions using a combination of recognition and multi-view cues. Our approach is based on the idea that there are image patches that are so distinctive that we can accurately estimate their latent 3D shapes solely using recognition. We call these patches shape anchors, and we use them as the basis of a multi-view reconstruction system that transfers dense, complex geometry between scenes. We “anchor” our 3D interpretation from these patches, using them to predict geometry for parts of the scene that are relatively ambiguous. The resulting algorithm produces dense reconstructions from stereo point clouds that are sparse and noisy, and we demonstrate it on a challenging dataset of real-world, indoor scenes.
5 0.64742172 351 iccv-2013-Restoring an Image Taken through a Window Covered with Dirt or Rain
Author: David Eigen, Dilip Krishnan, Rob Fergus
Abstract: Photographs taken through a window are often compromised by dirt or rain present on the window surface. Common cases of this include pictures taken from inside a vehicle, or outdoor security cameras mounted inside a protective enclosure. At capture time, defocus can be used to remove the artifacts, but this relies on achieving a shallow depth-of-field and placement of the camera close to the window. Instead, we present a post-capture image processing solution that can remove localized rain and dirt artifacts from a single image. We collect a dataset of clean/corrupted image pairs which are then used to train a specialized form of convolutional neural network. This learns how to map corrupted image patches to clean ones, implicitly capturing the characteristic appearance of dirt and water droplets in natural images. Our models demonstrate effective removal of dirt and rain in outdoor test conditions.
6 0.62845409 15 iccv-2013-A Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks
7 0.62202102 223 iccv-2013-Joint Noise Level Estimation from Personal Photo Collections
8 0.61142075 255 iccv-2013-Local Signal Equalization for Correspondence Matching
9 0.55793178 312 iccv-2013-Perceptual Fidelity Aware Mean Squared Error
10 0.54129821 135 iccv-2013-Efficient Image Dehazing with Boundary Constraint and Contextual Regularization
11 0.53945601 156 iccv-2013-Fast Direct Super-Resolution by Simple Functions
12 0.5324173 102 iccv-2013-Data-Driven 3D Primitives for Single Image Understanding
13 0.52096105 313 iccv-2013-Person Re-identification by Salience Matching
14 0.5189122 108 iccv-2013-Depth from Combining Defocus and Correspondence Using Light-Field Cameras
15 0.49675074 19 iccv-2013-A Learning-Based Approach to Reduce JPEG Artifacts in Image Matting
16 0.47610292 254 iccv-2013-Live Metric 3D Reconstruction on Mobile Phones
17 0.47581813 88 iccv-2013-Constant Time Weighted Median Filtering for Stereo Matching and Beyond
18 0.47025803 98 iccv-2013-Cross-Field Joint Image Restoration via Scale Map
19 0.46842003 18 iccv-2013-A Joint Intensity and Depth Co-sparse Analysis Model for Depth Map Super-resolution
20 0.46532115 437 iccv-2013-Unsupervised Random Forest Manifold Alignment for Lipreading
topicId topicWeight
[(2, 0.062), (16, 0.218), (26, 0.114), (31, 0.049), (42, 0.088), (48, 0.017), (64, 0.027), (73, 0.062), (89, 0.224), (98, 0.019)]
simIndex simValue paperId paperTitle
1 0.87581831 304 iccv-2013-PM-Huber: PatchMatch with Huber Regularization for Stereo Matching
Author: Philipp Heise, Sebastian Klose, Brian Jensen, Alois Knoll
Abstract: Most stereo correspondence algorithms match support windows at integer-valued disparities and assume a constant disparity value within the support window. The recently proposed PatchMatch stereo algorithm [7] overcomes this limitation of previous algorithms by directly estimating planes. This work presents a method that integrates the PatchMatch stereo algorithm into a variational smoothing formulation using quadratic relaxation. The resulting algorithm allows the explicit regularization of the disparity and normal gradients using the estimated plane parameters. Evaluation of our method in the Middlebury benchmark shows that our method outperforms the traditional integer-valued disparity strategy as well as the original algorithm and its variants in sub-pixel accurate disparity estimation.
same-paper 2 0.86401838 101 iccv-2013-DCSH - Matching Patches in RGBD Images
Author: Yaron Eshet, Simon Korman, Eyal Ofek, Shai Avidan
Abstract: We extend patch based methods to work on patches in 3D space. We start with Coherency Sensitive Hashing [12] (CSH), which is an algorithm for matching patches between two RGB images, and extend it to work with RGBD images. This is done by warping all 3D patches to a common virtual plane in which CSH is performed. To avoid noise due to warping of patches of various normals and depths, we estimate a group of dominant planes and compute CSH on each plane separately, before merging the matching patches. The result is DCSH - an algorithm that matches world (3D) patches in order to guide the search for image plane matches. An independent contribution is an extension of CSH, which we term Social-CSH. It allows a major speedup of the k nearest neighbor (kNN) version of CSH - its runtime growing linearly, rather than quadratically, in k. Social-CSH is used as a subcomponent of DCSH when many NNs are required, as in the case of image denoising. We show the benefits ofusing depth information to image reconstruction and image denoising, demonstrated on several RGBD images.
3 0.85633248 177 iccv-2013-From Point to Set: Extend the Learning of Distance Metrics
Author: Pengfei Zhu, Lei Zhang, Wangmeng Zuo, David Zhang
Abstract: Most of the current metric learning methods are proposed for point-to-point distance (PPD) based classification. In many computer vision tasks, however, we need to measure the point-to-set distance (PSD) and even set-to-set distance (SSD) for classification. In this paper, we extend the PPD based Mahalanobis distance metric learning to PSD and SSD based ones, namely point-to-set distance metric learning (PSDML) and set-to-set distance metric learning (SSDML), and solve them under a unified optimization framework. First, we generate positive and negative sample pairs by computing the PSD and SSD between training samples. Then, we characterize each sample pair by its covariance matrix, and propose a covariance kernel based discriminative function. Finally, we tackle the PSDML and SSDMLproblems by using standard support vector machine solvers, making the metric learning very efficient for multiclass visual classification tasks. Experiments on gender classification, digit recognition, object categorization and face recognition show that the proposed metric learning methods can effectively enhance the performance of PSD and SSD based classification.
4 0.84903026 42 iccv-2013-Active MAP Inference in CRFs for Efficient Semantic Segmentation
Author: Gemma Roig, Xavier Boix, Roderick De_Nijs, Sebastian Ramos, Koljia Kuhnlenz, Luc Van_Gool
Abstract: Most MAP inference algorithms for CRFs optimize an energy function knowing all the potentials. In this paper, we focus on CRFs where the computational cost of instantiating the potentials is orders of magnitude higher than MAP inference. This is often the case in semantic image segmentation, where most potentials are instantiated by slow classifiers fed with costly features. We introduce Active MAP inference 1) to on-the-fly select a subset of potentials to be instantiated in the energy function, leaving the rest of the parameters of the potentials unknown, and 2) to estimate the MAP labeling from such incomplete energy function. Results for semantic segmentation benchmarks, namely PASCAL VOC 2010 [5] and MSRC-21 [19], show that Active MAP inference achieves similar levels of accuracy but with major efficiency gains.
5 0.81571293 188 iccv-2013-Group Sparsity and Geometry Constrained Dictionary Learning for Action Recognition from Depth Maps
Author: Jiajia Luo, Wei Wang, Hairong Qi
Abstract: Human action recognition based on the depth information provided by commodity depth sensors is an important yet challenging task. The noisy depth maps, different lengths of action sequences, and free styles in performing actions, may cause large intra-class variations. In this paper, a new framework based on sparse coding and temporal pyramid matching (TPM) is proposed for depthbased human action recognition. Especially, a discriminative class-specific dictionary learning algorithm isproposed for sparse coding. By adding the group sparsity and geometry constraints, features can be well reconstructed by the sub-dictionary belonging to the same class, and the geometry relationships among features are also kept in the calculated coefficients. The proposed approach is evaluated on two benchmark datasets captured by depth cameras. Experimental results show that the proposed algorithm repeatedly hqi } @ ut k . edu GB ImagesR epth ImagesD setkonlSy0 896.5170d4ept.3h021 .x02y 19.876504.dep3th02.1 x02. achieves superior performance to the state of the art algorithms. Moreover, the proposed dictionary learning method also outperforms classic dictionary learning approaches.
6 0.81191546 17 iccv-2013-A Global Linear Method for Camera Pose Registration
7 0.7885946 196 iccv-2013-Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation
8 0.78017634 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
9 0.77838731 209 iccv-2013-Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation
10 0.77824455 156 iccv-2013-Fast Direct Super-Resolution by Simple Functions
11 0.77785897 60 iccv-2013-Bayesian Robust Matrix Factorization for Image and Video Processing
13 0.77592808 351 iccv-2013-Restoring an Image Taken through a Window Covered with Dirt or Rain
14 0.77584565 18 iccv-2013-A Joint Intensity and Depth Co-sparse Analysis Model for Depth Map Super-resolution
15 0.77505648 309 iccv-2013-Partial Enumeration and Curvature Regularization
16 0.77458215 411 iccv-2013-Symbiotic Segmentation and Part Localization for Fine-Grained Categorization
17 0.77439231 361 iccv-2013-Robust Trajectory Clustering for Motion Segmentation
18 0.77430856 150 iccv-2013-Exemplar Cut
19 0.77410203 386 iccv-2013-Sequential Bayesian Model Update under Structured Scene Prior for Semantic Road Scenes Labeling
20 0.77359354 258 iccv-2013-Low-Rank Sparse Coding for Image Classification