cvpr cvpr2013 cvpr2013-352 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Armand Joulin, Sing Bing Kang
Abstract: An anaglyph is a single image created by selecting complementary colors from a stereo color pair; the user can perceive depth by viewing it through color-filtered glasses. We propose a technique to reconstruct the original color stereo pair given such an anaglyph. We modified SIFT-Flow and use it to initially match the different color channels across the two views. Our technique then iteratively refines the matches, selects the good matches (which defines the “anchor” colors), and propagates the anchor colors. We use a diffusion-based technique for the color propagation, and added a step to suppress unwanted colors. Results on a variety of inputs demonstrate the robustness of our technique. We also extended our method to anaglyph videos by using optic flow between time frames.
Reference: text
sentIndex sentText sentNum sentScore
1 fr Abstract An anaglyph is a single image created by selecting complementary colors from a stereo color pair; the user can perceive depth by viewing it through color-filtered glasses. [sent-3, score-1.126]
2 We propose a technique to reconstruct the original color stereo pair given such an anaglyph. [sent-4, score-0.403]
3 We modified SIFT-Flow and use it to initially match the different color channels across the two views. [sent-5, score-0.392]
4 Our technique then iteratively refines the matches, selects the good matches (which defines the “anchor” colors), and propagates the anchor colors. [sent-6, score-0.243]
5 We use a diffusion-based technique for the color propagation, and added a step to suppress unwanted colors. [sent-7, score-0.197]
6 We also extended our method to anaglyph videos by using optic flow between time frames. [sent-9, score-0.521]
7 Introduction Arguably, the first 3D motion picture was shown in 1889 by William Friese-Green, who used the separation of colors from the stereo pair to generate color composites. [sent-11, score-0.705]
8 Glasses with appropriate color filters are used to perceive the depth effect. [sent-12, score-0.208]
9 Such images generated by color separation are called anaglyphs. [sent-13, score-0.201]
10 An example is shown in top-left part of Figure 1, where the color separation is red-cyan. [sent-14, score-0.201]
11 Here, anaglyph glasses comprising red-cyan filters are used to perceive depth. [sent-15, score-0.508]
12 There is much legacy anaglyph content available with no original color pairs. [sent-16, score-0.606]
13 In this paper, we show how we can reliably reconstruct a good approximation of the original color stereo pair given its anaglyph. [sent-17, score-0.397]
14 We assume the left-right color separation is red-cyan, which is the most common; the principles of our technique can be applied to other color separation schemes. [sent-18, score-0.446]
15 Given an anaglyph, recovering the stereo color pair is equivalent to transferring the intensities in the red channel to the right image and the intensities in the blue and green channels to the left one. [sent-19, score-0.985]
16 This complicates the use of stereo and standard color transfer methods. [sent-23, score-0.386]
17 Our technical contributions are as follow: • A completely automatic technique for reconstructing tAhe c sotmerpeole pair faruotmom an anaglyph (image or ovnidsetrou)c. [sent-25, score-0.536]
18 ’osf only specific color channels per view, and use of longer range influence. [sent-29, score-0.278]
19 We also added the step of suppressing visually inappropriate colors at disocclusions. [sent-30, score-0.333]
20 We made the following assumptions: • There are no new colors in the disoccluded regions. [sent-31, score-0.381]
21 • The information embedded in the anaglyph is enough tToh erec inofnosrtmruactti ofunl le mrgbbe2. [sent-32, score-0.453]
22 Dietz [3] proposed a method to recover the original images from anaglyphs captured using a modified camera with color filters. [sent-41, score-0.39]
23 There is unfortunately insufficient information in the paper on how his technique works; however, his results show that the colors often do not match across the reconstructed stereo pair. [sent-42, score-0.658]
24 2Certain types of anaglyphs, such as the “optimized” anaglyph, throws away the red channel; it fakes the red channel of the left image by combining green and blue channels. [sent-43, score-0.366]
25 a spx 222888999 respondences between the left and right images and recolorize the two images based on these correspondences. [sent-47, score-0.288]
26 They use an RGB color filter on the camera lens aperture to obtain three known shifted views. [sent-50, score-0.193]
27 They propose an algorithm to match the corresponding pixels across the three channels by exploiting the tendency of colors in natural images to form elongated clusters in the RGB space (color lines model of [16]), with the assumption of small disparities (between -5 to 10 pixels only). [sent-51, score-0.609]
28 Colorization In our approach, we use a diffusion-based colorization method to propagate the colors of reliable correspondences to other pixels. [sent-55, score-0.758]
29 Colorization methods assume that the color of some input pixels is known and infer the color of the other pixels [11, 15, 22, 12, 20, 9]. [sent-56, score-0.422]
30 Most colorization methods also assume, explicitly or implicitly, that the greyscale values are known and used as part of the optimization. [sent-57, score-0.465]
31 Our method is based on diffusion of reliably transferred colors using a graph construct (e. [sent-59, score-0.335]
32 They assume that the input colors given by a user are approximate and may be refined depending on the image content. [sent-63, score-0.298]
33 Unlike [11], since we work in rgb space directly, we must pre- × serve edges while diffusing the color (especially over shadowed regions). [sent-65, score-0.229]
34 Another popular approach to colorize an image is to use the geodesic of an image to connect a pixel with unknown color to an input pixel [22]. [sent-67, score-0.341]
35 Dense correspondences Our approach relies on finding dense good correspondences between both images. [sent-71, score-0.257]
36 Most stereo matching methods have been proven to be successful when the cameras are radiometrically similar [19]. [sent-74, score-0.21]
37 These are not applicable in our case, since we are matching the red channel in one view with cyan in the other. [sent-75, score-0.25]
38 [5] have proposed a color transfer method based on local correspondence between similar images. [sent-82, score-0.219]
39 Color transfer A related subject of interest is color transfer, which is the process of modifying the color of a target image to match the color characteristics of a source image. [sent-88, score-0.595]
40 Overview of our approach Our approach to recover the stereo pair from an anaglyph is to iteratively find dense good correspondences between the two images and recolorizing them based on these correspondences. [sent-93, score-0.878]
41 As a post-processing stage, we detect “incompatible” colors in unmatched regions (defined as colors that exist in one image but not the other) and assign them to the nearest known colors. [sent-94, score-0.642]
42 This process prevents new colors from appearing in the occluded regions. [sent-95, score-0.335]
43 Notice the incorrect colors transferred for the original SIFTFlow in the darker lower region. [sent-100, score-0.334]
44 Each channel is then iteratively matched independently using an optical flow method to produce new good correspondences; the colors are updated using the new correspondences. [sent-105, score-0.593]
45 In the next section, we describe the dense matching algorithms to align the color channels of the two images. [sent-106, score-0.354]
46 Channel alignment Given a red-cyan anaglyph, we first need to align the red channel with the blue and green channels. [sent-109, score-0.317]
47 To accomplish this, we use two different graph matching algorithms, namely, modified versions of SIFT-Flow and an optical flow algorithm. [sent-110, score-0.22]
48 Initial rough channel alignment SIFT-Flow is used to compute correspondences between the initially color-separated images. [sent-114, score-0.345]
49 More precisely, given the left image as the input, for each pixel p, SIFT-Flow tries to find the best disparity d(p) = (dx (p) , dy (p)) as to match SL (p) to SR(p + d(p)) by minimizing ESF = ? [sent-122, score-0.265]
50 In our case, the scenes to be matched are the same, just shown as shifted versions and in different color bands. [sent-134, score-0.193]
51 Unfortunately, the original SIFT-Flow tends to not do well in less textured regions, which often results in artifacts if used as is for matching followed by color transfer. [sent-135, score-0.255]
52 atches SIFT features which are known to have a higher value on highly texture regions which encourage matching of highly texture regions over low texture ones. [sent-144, score-0.234]
53 Notice the artifacts for SIFT-Flow at the darker lower region caused by incorrect color transfer. [sent-146, score-0.248]
54 Finally, in the original SIFT-flow method, the cost of matching a pixel to the outside of the target image is independent of its proximity to the left and right borders of 222999111 × the image. [sent-157, score-0.208]
55 [10] propose a stereo matching algorithm which works for images with different intensity distribution. [sent-165, score-0.254]
56 [5] recover the various textures of the image but is not robust enough to deal with high intensity changes resulting in wrong color estimations. [sent-168, score-0.197]
57 Refining channel alignment Using the matches by ASIFT-Flow, we have a first rough colorization of the two images. [sent-174, score-0.684]
58 We compute optical flow independently on the three channels, namely, the red channel for the left image and the blue and green channels for the right image. [sent-181, score-0.624]
59 Finding good matches Given matches from left to right ML? [sent-185, score-0.273]
60 We use three different criteria: stereo consistency, texture consistency, and detection of textureless regions. [sent-188, score-0.257]
61 These criteria are based on comparing an original known channel with its reprojected (warped) version (ML? [sent-189, score-0.243]
62 satisfy this criterion exactly, we use a soft version where a pixel is allow to match a pixel within a tx ty (= 5 2) window. [sent-201, score-0.286]
63 A match is deemed good if both pixels are textureless. [sent-227, score-0.212]
64 We make the strong assumption that the reconstructed colors (anchor colors) using these matches is correct. [sent-229, score-0.397]
65 Our goal is now to propagate anchor colors to the other pixels, which is the colorization step. [sent-230, score-0.761]
66 Colorization There are many approaches for colorization [8, 22, 11, 2]. [sent-232, score-0.367]
67 We did not consider methods based on geodesic distance because they are not suitable for cases where the number of anchor pixels (i. [sent-234, score-0.187]
68 In this section, we describe a colorization method inspired by Levin et al. [sent-238, score-0.367]
69 The anchor colors (yk, “known”) are fixed, with the rest (yu, “unknown”) needed to be computed. [sent-249, score-0.394]
70 [11] In [11], the user-specified colors may be modified. [sent-261, score-0.298]
71 In our case, the grayscale image is unknown, and as such, this color space is not useable. [sent-265, score-0.192]
72 At the end of this process, it is possible for disoccluded regions to have colors not found elsewhere (we term such colors as “incompatible” colors). [sent-270, score-0.725]
73 In the next section, we describe a method to detect such colors and recolorize them. [sent-271, score-0.436]
74 Detecting incompatible colors Our approach uses the anchor colors to fill in the missing colors. [sent-273, score-0.843]
75 Examples of recolorization of occluded regions with incompatible colors. [sent-275, score-0.234]
76 such occurrences and subsequently color correct, we make the assumption that there are no new colors in the occluded regions in order to produce visually plausible outputs. [sent-276, score-0.534]
77 To recolorize, we first detect regions with incompatible colors before we recolorize them with longer range connections. [sent-277, score-0.633]
78 Given pixel p in the right image with color cp = ( rˆp, gp, bp), the goal is to find if its estimated red value rˆp is compatible with the red values observed in the rest of the image given the same blue and green values. [sent-279, score-0.442]
79 Pixel p is deemed to be an incompatible color if |rNpW −ˆ r p| is iasbo dveee a ecder ttaoin b eth arensh inocldo mλp. [sent-293, score-0.35]
80 We design our colorization algorithm with a longer range than that for ioanthbcoe r drev ciretlwrye,g rceio slnuolroetifdn. [sent-296, score-0.367]
81 Here we have ground truth, with the anaglyphs synthetically generated from the stereo pairs. [sent-309, score-0.36]
82 This tool was released by 3dtv; it reconstructs the right image given the anaglyph and the left image. [sent-313, score-0.558]
83 Because the blue and green channels are shifted as well, we matching the red channel to the blue and green independently. [sent-319, score-0.56]
84 Here we have ground truth, from which the input anaglyphs are generated and processed. [sent-324, score-0.193]
85 The red and white part of the flag appears as uniform in the red channel in the left view, so registration in that area is less reliable. [sent-339, score-0.336]
86 Figure 7 shows a variety of results for anaglyphs found from different online sources, including one from a comic book. [sent-342, score-0.193]
87 Here, our algorithm fails on part of the flag because the white and red colors produces an almost constant red distribution on the left view. [sent-347, score-0.51]
88 This causes partially incorrect transfer of blue and green to the left view. [sent-348, score-0.252]
89 To reduce flickering effects, take into account temporal information (in the form of optical flow across successive frames). [sent-352, score-0.182]
90 For efficiency reasons, we perform rough channel alignment each frame independently. [sent-369, score-0.252]
91 We impose additional criteria on good correspondences, that of temporal match reciprocity: two pixels connected by the optical flow between two consecutive frames of the left view are matched to pixels of the right view that are also connected by the optical flow. [sent-370, score-0.699]
92 The additional criteria for a pixel iof the t-th frame of the left view are OtR? [sent-373, score-0.218]
93 Our colorization algorithm is extended to videos by computing similarity matrices within a frame Wt and between consecutive frames, Wt,t+1 . [sent-378, score-0.403]
94 Figure 10 shows results for an anaglyph video, which illustrate the importance of temporal information on preventing flickering effects caused by inconsistent colors across time. [sent-392, score-0.8]
95 The highlighted areas (dotted ellipses) have inconsistent colors across time. [sent-397, score-0.298]
96 Concluding remarks We have described a new technique for reconstructing the stereo color pair from either an anaglyph image or video. [sent-402, score-0.856]
97 Current techniques cannot be used as is because of the unique problem ofmatching across different color channels, transferring only specific color channels, and making sure inappropriate colors do not appear in either reconstructed view. [sent-403, score-0.714]
98 ’s [11] colorization technique to ensure good performance, which is evidenced in our results. [sent-405, score-0.449]
99 Semantic colorization with internet im- [3] [4] [5] [6] [7] [8] ages. [sent-435, score-0.367]
100 A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. [sent-546, score-0.2]
wordName wordTfidf (topN-words)
[('anaglyph', 0.453), ('colorization', 0.367), ('colors', 0.298), ('anaglyphs', 0.193), ('stereo', 0.167), ('color', 0.153), ('incompatible', 0.151), ('bando', 0.138), ('recolorize', 0.138), ('channels', 0.125), ('channel', 0.124), ('greyscale', 0.098), ('anchor', 0.096), ('correspondences', 0.093), ('deanaglyph', 0.083), ('disoccluded', 0.083), ('ml', 0.076), ('levin', 0.074), ('msf', 0.073), ('hacohen', 0.073), ('rough', 0.072), ('match', 0.07), ('flow', 0.068), ('transfer', 0.066), ('optical', 0.065), ('matches', 0.065), ('hellinger', 0.064), ('criteria', 0.062), ('flag', 0.061), ('sift', 0.061), ('pixel', 0.06), ('left', 0.059), ('artifacts', 0.059), ('psnr', 0.058), ('pixels', 0.058), ('textureless', 0.057), ('reprojected', 0.057), ('mr', 0.056), ('alignment', 0.056), ('anaglyphleftright', 0.055), ('ashikhmin', 0.055), ('dtv', 0.055), ('otr', 0.055), ('recolorizing', 0.055), ('rnpw', 0.055), ('perceive', 0.055), ('ty', 0.051), ('disocclusions', 0.049), ('colorized', 0.049), ('armand', 0.049), ('flickering', 0.049), ('separation', 0.048), ('green', 0.048), ('rectified', 0.047), ('intensities', 0.047), ('regions', 0.046), ('deemed', 0.046), ('red', 0.046), ('right', 0.046), ('spx', 0.045), ('unfortunately', 0.045), ('tx', 0.045), ('technique', 0.044), ('modified', 0.044), ('intensity', 0.044), ('refining', 0.043), ('blue', 0.043), ('matching', 0.043), ('diffusing', 0.043), ('yuv', 0.043), ('col', 0.043), ('transferring', 0.041), ('normale', 0.041), ('shifted', 0.04), ('grayscale', 0.039), ('dy', 0.039), ('pair', 0.039), ('reciprocity', 0.039), ('finger', 0.039), ('good', 0.038), ('occluded', 0.037), ('disparity', 0.037), ('view', 0.037), ('diffusion', 0.037), ('window', 0.036), ('consecutive', 0.036), ('incorrect', 0.036), ('inappropriate', 0.035), ('mse', 0.035), ('unknown', 0.035), ('asymmetric', 0.035), ('sl', 0.035), ('rq', 0.035), ('reconstructed', 0.034), ('failure', 0.033), ('rgb', 0.033), ('dense', 0.033), ('geodesic', 0.033), ('siggraph', 0.033), ('texture', 0.033)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999988 352 cvpr-2013-Recovering Stereo Pairs from Anaglyphs
Author: Armand Joulin, Sing Bing Kang
Abstract: An anaglyph is a single image created by selecting complementary colors from a stereo color pair; the user can perceive depth by viewing it through color-filtered glasses. We propose a technique to reconstruct the original color stereo pair given such an anaglyph. We modified SIFT-Flow and use it to initially match the different color channels across the two views. Our technique then iteratively refines the matches, selects the good matches (which defines the “anchor” colors), and propagates the anchor colors. We use a diffusion-based technique for the color propagation, and added a step to suppress unwanted colors. Results on a variety of inputs demonstrate the robustness of our technique. We also extended our method to anaglyph videos by using optic flow between time frames.
2 0.27494043 327 cvpr-2013-Pattern-Driven Colorization of 3D Surfaces
Author: George Leifman, Ayellet Tal
Abstract: Colorization refers to the process of adding color to black & white images or videos. This paper extends the term to handle surfaces in three dimensions. This is important for applications in which the colors of an object need to be restored and no relevant image exists for texturing it. We focus on surfaces with patterns and propose a novel algorithm for adding colors to these surfaces. The user needs only to scribble a few color strokes on one instance of each pattern, and the system proceeds to automatically colorize the whole surface. For this scheme to work, we address not only the problem of colorization, but also the problem of pattern detection on surfaces.
3 0.13705236 147 cvpr-2013-Ensemble Learning for Confidence Measures in Stereo Vision
Author: Ralf Haeusler, Rahul Nair, Daniel Kondermann
Abstract: With the aim to improve accuracy of stereo confidence measures, we apply the random decision forest framework to a large set of diverse stereo confidence measures. Learning and testing sets were drawnfrom the recently introduced KITTI dataset, which currently poses higher challenges to stereo solvers than other benchmarks with ground truth for stereo evaluation. We experiment with semi global matching stereo (SGM) and a census dataterm, which is the best performing realtime capable stereo method known to date. On KITTI images, SGM still produces a significant amount of error. We obtain consistently improved area under curve values of sparsification measures in comparison to best performing single stereo confidence measures where numbers of stereo errors are large. More specifically, our method performs best in all but one out of 194 frames of the KITTI dataset.
4 0.12007552 345 cvpr-2013-Real-Time Model-Based Rigid Object Pose Estimation and Tracking Combining Dense and Sparse Visual Cues
Author: Karl Pauwels, Leonardo Rubio, Javier Díaz, Eduardo Ros
Abstract: We propose a novel model-based method for estimating and tracking the six-degrees-of-freedom (6DOF) pose of rigid objects of arbitrary shapes in real-time. By combining dense motion and stereo cues with sparse keypoint correspondences, and by feeding back information from the model to the cue extraction level, the method is both highly accurate and robust to noise and occlusions. A tight integration of the graphical and computational capability of Graphics Processing Units (GPUs) results in pose updates at framerates exceeding 60 Hz. Since a benchmark dataset that enables the evaluation of stereo-vision-based pose estimators in complex scenarios is currently missing in the literature, we have introduced a novel synthetic benchmark dataset with varying objects, background motion, noise and occlusions. Using this dataset and a novel evaluation methodology, we show that the proposed method greatly outperforms state-of-the-art methods. Finally, we demonstrate excellent performance on challenging real-world sequences involving object manipulation.
5 0.11787443 107 cvpr-2013-Deformable Spatial Pyramid Matching for Fast Dense Correspondences
Author: Jaechul Kim, Ce Liu, Fei Sha, Kristen Grauman
Abstract: We introduce a fast deformable spatial pyramid (DSP) matching algorithm for computing dense pixel correspondences. Dense matching methods typically enforce both appearance agreement between matched pixels as well as geometric smoothness between neighboring pixels. Whereas the prevailing approaches operate at the pixel level, we propose a pyramid graph model that simultaneously regularizes match consistency at multiple spatial extents—ranging from an entire image, to coarse grid cells, to every single pixel. This novel regularization substantially improves pixel-level matching in the face of challenging image variations, while the “deformable ” aspect of our model overcomes the strict rigidity of traditional spatial pyramids. Results on LabelMe and Caltech show our approach outperforms state-of-the-art methods (SIFT Flow [15] and PatchMatch [2]), both in terms of accuracy and run time.
6 0.11145677 216 cvpr-2013-Improving Image Matting Using Comprehensive Sampling Sets
7 0.1100532 110 cvpr-2013-Dense Object Reconstruction with Semantic Priors
8 0.10415705 362 cvpr-2013-Robust Monocular Epipolar Flow Estimation
9 0.098829016 155 cvpr-2013-Exploiting the Power of Stereo Confidences
10 0.098593391 130 cvpr-2013-Discriminative Color Descriptors
11 0.098444782 450 cvpr-2013-Unsupervised Joint Object Discovery and Segmentation in Internet Images
12 0.098173589 232 cvpr-2013-Joint Geodesic Upsampling of Depth Images
13 0.089849882 158 cvpr-2013-Exploring Weak Stabilization for Motion Feature Extraction
14 0.086817347 181 cvpr-2013-Fusing Depth from Defocus and Stereo with Coded Apertures
15 0.084112383 244 cvpr-2013-Large Displacement Optical Flow from Nearest Neighbor Fields
17 0.079530649 195 cvpr-2013-HDR Deghosting: How to Deal with Saturation?
18 0.079190612 332 cvpr-2013-Pixel-Level Hand Detection in Ego-centric Videos
19 0.078531988 453 cvpr-2013-Video Editing with Temporal, Spatial and Appearance Consistency
20 0.077821493 192 cvpr-2013-Graph Matching with Anchor Nodes: A Learning Approach
topicId topicWeight
[(0, 0.198), (1, 0.115), (2, 0.024), (3, 0.031), (4, -0.0), (5, -0.019), (6, -0.01), (7, -0.017), (8, -0.023), (9, 0.022), (10, 0.028), (11, 0.019), (12, 0.1), (13, -0.011), (14, 0.051), (15, -0.052), (16, -0.035), (17, -0.144), (18, 0.097), (19, 0.012), (20, -0.012), (21, 0.032), (22, 0.065), (23, -0.062), (24, -0.036), (25, -0.137), (26, 0.116), (27, 0.022), (28, -0.016), (29, 0.057), (30, 0.02), (31, -0.025), (32, -0.025), (33, 0.013), (34, 0.025), (35, -0.045), (36, -0.029), (37, 0.06), (38, 0.037), (39, 0.064), (40, 0.009), (41, -0.015), (42, 0.003), (43, 0.022), (44, -0.013), (45, -0.012), (46, 0.006), (47, 0.029), (48, -0.03), (49, -0.022)]
simIndex simValue paperId paperTitle
same-paper 1 0.95703667 352 cvpr-2013-Recovering Stereo Pairs from Anaglyphs
Author: Armand Joulin, Sing Bing Kang
Abstract: An anaglyph is a single image created by selecting complementary colors from a stereo color pair; the user can perceive depth by viewing it through color-filtered glasses. We propose a technique to reconstruct the original color stereo pair given such an anaglyph. We modified SIFT-Flow and use it to initially match the different color channels across the two views. Our technique then iteratively refines the matches, selects the good matches (which defines the “anchor” colors), and propagates the anchor colors. We use a diffusion-based technique for the color propagation, and added a step to suppress unwanted colors. Results on a variety of inputs demonstrate the robustness of our technique. We also extended our method to anaglyph videos by using optic flow between time frames.
Author: Jiangbo Lu, Hongsheng Yang, Dongbo Min, Minh N. Do
Abstract: Though many tasks in computer vision can be formulated elegantly as pixel-labeling problems, a typical challenge discouraging such a discrete formulation is often due to computational efficiency. Recent studies on fast cost volume filtering based on efficient edge-aware filters have provided a fast alternative to solve discrete labeling problems, with the complexity independent of the support window size. However, these methods still have to step through the entire cost volume exhaustively, which makes the solution speed scale linearly with the label space size. When the label space is huge, which is often the case for (subpixelaccurate) stereo and optical flow estimation, their computational complexity becomes quickly unacceptable. Developed to search approximate nearest neighbors rapidly, the PatchMatch method can significantly reduce the complexity dependency on the search space size. But, its pixel-wise randomized search and fragmented data access within the 3D cost volume seriously hinder the application of efficient cost slice filtering. This paper presents a generic and fast computational framework for general multi-labeling problems called PatchMatch Filter (PMF). For the very first time, we explore effective and efficient strategies to weave together these two fundamental techniques developed in isolation, i.e., PatchMatch-based randomized search and efficient edge-aware image filtering. By decompositing an image into compact superpixels, we also propose superpixelbased novel search strategies that generalize and improve the original PatchMatch method. Focusing on dense correspondence field estimation in this paper, we demonstrate PMF’s applications in stereo and optical flow. Our PMF methods achieve state-of-the-art correspondence accuracy but run much faster than other competing methods, often giving over 10-times speedup for large label space cases.
3 0.66937548 112 cvpr-2013-Dense Segmentation-Aware Descriptors
Author: Eduard Trulls, Iasonas Kokkinos, Alberto Sanfeliu, Francesc Moreno-Noguer
Abstract: In this work we exploit segmentation to construct appearance descriptors that can robustly deal with occlusion and background changes. For this, we downplay measurements coming from areas that are unlikely to belong to the same region as the descriptor’s center, as suggested by soft segmentation masks. Our treatment is applicable to any image point, i.e. dense, and its computational overhead is in the order of a few seconds. We integrate this idea with Dense SIFT, and also with Dense Scale and Rotation Invariant Descriptors (SID), delivering descriptors that are densely computable, invariant to scaling and rotation, and robust to background changes. We apply our approach to standard benchmarks on large displacement motion estimation using SIFT-flow and widebaseline stereo, systematically demonstrating that the introduction of segmentation yields clear improvements.
4 0.66872215 107 cvpr-2013-Deformable Spatial Pyramid Matching for Fast Dense Correspondences
Author: Jaechul Kim, Ce Liu, Fei Sha, Kristen Grauman
Abstract: We introduce a fast deformable spatial pyramid (DSP) matching algorithm for computing dense pixel correspondences. Dense matching methods typically enforce both appearance agreement between matched pixels as well as geometric smoothness between neighboring pixels. Whereas the prevailing approaches operate at the pixel level, we propose a pyramid graph model that simultaneously regularizes match consistency at multiple spatial extents—ranging from an entire image, to coarse grid cells, to every single pixel. This novel regularization substantially improves pixel-level matching in the face of challenging image variations, while the “deformable ” aspect of our model overcomes the strict rigidity of traditional spatial pyramids. Results on LabelMe and Caltech show our approach outperforms state-of-the-art methods (SIFT Flow [15] and PatchMatch [2]), both in terms of accuracy and run time.
5 0.66299909 195 cvpr-2013-HDR Deghosting: How to Deal with Saturation?
Author: Jun Hu, Orazio Gallo, Kari Pulli, Xiaobai Sun
Abstract: We present a novel method for aligning images in an HDR (high-dynamic-range) image stack to produce a new exposure stack where all the images are aligned and appear as if they were taken simultaneously, even in the case of highly dynamic scenes. Our method produces plausible results even where the image used as a reference is either too dark or bright to allow for an accurate registration.
6 0.64836276 177 cvpr-2013-FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps
7 0.64778537 22 cvpr-2013-A Non-parametric Framework for Document Bleed-through Removal
8 0.64266461 162 cvpr-2013-FasT-Match: Fast Affine Template Matching
9 0.6418224 384 cvpr-2013-Segment-Tree Based Cost Aggregation for Stereo Matching
10 0.63598394 147 cvpr-2013-Ensemble Learning for Confidence Measures in Stereo Vision
11 0.62802655 234 cvpr-2013-Joint Spectral Correspondence for Disparate Image Matching
12 0.61414903 453 cvpr-2013-Video Editing with Temporal, Spatial and Appearance Consistency
13 0.60303849 373 cvpr-2013-SWIGS: A Swift Guided Sampling Method
14 0.59916747 155 cvpr-2013-Exploiting the Power of Stereo Confidences
15 0.59914142 235 cvpr-2013-Jointly Aligning and Segmenting Multiple Web Photo Streams for the Inference of Collective Photo Storylines
16 0.59871417 327 cvpr-2013-Pattern-Driven Colorization of 3D Surfaces
17 0.59705341 216 cvpr-2013-Improving Image Matting Using Comprehensive Sampling Sets
18 0.5966714 450 cvpr-2013-Unsupervised Joint Object Discovery and Segmentation in Internet Images
19 0.59069985 211 cvpr-2013-Image Matting with Local and Nonlocal Smooth Priors
20 0.5704 283 cvpr-2013-Megastereo: Constructing High-Resolution Stereo Panoramas
topicId topicWeight
[(10, 0.128), (16, 0.072), (26, 0.045), (28, 0.018), (33, 0.244), (67, 0.044), (69, 0.042), (72, 0.231), (87, 0.089)]
simIndex simValue paperId paperTitle
1 0.84198993 62 cvpr-2013-Bilinear Programming for Human Activity Recognition with Unknown MRF Graphs
Author: Zhenhua Wang, Qinfeng Shi, Chunhua Shen, Anton van_den_Hengel
Abstract: Markov Random Fields (MRFs) have been successfully applied to human activity modelling, largely due to their ability to model complex dependencies and deal with local uncertainty. However, the underlying graph structure is often manually specified, or automatically constructed by heuristics. We show, instead, that learning an MRF graph and performing MAP inference can be achieved simultaneously by solving a bilinear program. Equipped with the bilinear program based MAP inference for an unknown graph, we show how to estimate parameters efficiently and effectively with a latent structural SVM. We apply our techniques to predict sport moves (such as serve, volley in tennis) and human activity in TV episodes (such as kiss, hug and Hi-Five). Experimental results show the proposed method outperforms the state-of-the-art.
2 0.83940059 149 cvpr-2013-Evaluation of Color STIPs for Human Action Recognition
Author: Ivo Everts, Jan C. van_Gemert, Theo Gevers
Abstract: This paper is concerned with recognizing realistic human actions in videos based on spatio-temporal interest points (STIPs). Existing STIP-based action recognition approaches operate on intensity representations of the image data. Because of this, these approaches are sensitive to disturbing photometric phenomena such as highlights and shadows. Moreover, valuable information is neglected by discarding chromaticity from the photometric representation. These issues are addressed by Color STIPs. Color STIPs are multi-channel reformulations of existing intensity-based STIP detectors and descriptors, for which we consider a number of chromatic representations derived from the opponent color space. This enhanced modeling of appearance improves the quality of subsequent STIP detection and description. Color STIPs are shown to substantially outperform their intensity-based counterparts on the challenging UCF sports, UCF11 and UCF50 action recognition benchmarks. Moreover, the results show that color STIPs are currently the single best low-level feature choice for STIP-based approaches to human action recognition.
same-paper 3 0.82968718 352 cvpr-2013-Recovering Stereo Pairs from Anaglyphs
Author: Armand Joulin, Sing Bing Kang
Abstract: An anaglyph is a single image created by selecting complementary colors from a stereo color pair; the user can perceive depth by viewing it through color-filtered glasses. We propose a technique to reconstruct the original color stereo pair given such an anaglyph. We modified SIFT-Flow and use it to initially match the different color channels across the two views. Our technique then iteratively refines the matches, selects the good matches (which defines the “anchor” colors), and propagates the anchor colors. We use a diffusion-based technique for the color propagation, and added a step to suppress unwanted colors. Results on a variety of inputs demonstrate the robustness of our technique. We also extended our method to anaglyph videos by using optic flow between time frames.
4 0.82406008 124 cvpr-2013-Determining Motion Directly from Normal Flows Upon the Use of a Spherical Eye Platform
Author: Tak-Wai Hui, Ronald Chung
Abstract: We address the problem of recovering camera motion from video data, which does not require the establishment of feature correspondences or computation of optical flows but from normal flows directly. We have designed an imaging system that has a wide field of view by fixating a number of cameras together to form an approximate spherical eye. With a substantially widened visual field, we discover that estimating the directions of translation and rotation components of the motion separately are possible and particularly efficient. In addition, the inherent ambiguities between translation and rotation also disappear. Magnitude of rotation is recovered subsequently. Experimental results on synthetic and real image data are provided. The results show that not only the accuracy of motion estimation is comparable to those of the state-of-the-art methods that require explicit feature correspondences or optical flows, but also a faster computation time.
5 0.81745183 325 cvpr-2013-Part Discovery from Partial Correspondence
Author: Subhransu Maji, Gregory Shakhnarovich
Abstract: We study the problem of part discovery when partial correspondence between instances of a category are available. For visual categories that exhibit high diversity in structure such as buildings, our approach can be used to discover parts that are hard to name, but can be easily expressed as a correspondence between pairs of images. Parts naturally emerge from point-wise landmark matches across many instances within a category. We propose a learning framework for automatic discovery of parts in such weakly supervised settings, and show the utility of the rich part library learned in this way for three tasks: object detection, category-specific saliency estimation, and fine-grained image parsing.
6 0.81358844 229 cvpr-2013-It's Not Polite to Point: Describing People with Uncertain Attributes
7 0.77661389 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
8 0.77565962 365 cvpr-2013-Robust Real-Time Tracking of Multiple Objects by Volumetric Mass Densities
9 0.77426744 349 cvpr-2013-Reconstructing Gas Flows Using Light-Path Approximation
10 0.77326745 303 cvpr-2013-Multi-view Photometric Stereo with Spatially Varying Isotropic Materials
11 0.77219278 225 cvpr-2013-Integrating Grammar and Segmentation for Human Pose Estimation
12 0.77201122 98 cvpr-2013-Cross-View Action Recognition via a Continuous Virtual Path
13 0.77198923 245 cvpr-2013-Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras
14 0.77178031 115 cvpr-2013-Depth Super Resolution by Rigid Body Self-Similarity in 3D
15 0.77147746 331 cvpr-2013-Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis
16 0.77126914 447 cvpr-2013-Underwater Camera Calibration Using Wavelength Triangulation
17 0.77103621 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
18 0.7708621 286 cvpr-2013-Mirror Surface Reconstruction from a Single Image
19 0.7708261 61 cvpr-2013-Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics
20 0.77046406 27 cvpr-2013-A Theory of Refractive Photo-Light-Path Triangulation