cvpr cvpr2013 cvpr2013-110 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Sid Yingze Bao, Manmohan Chandraker, Yuanqing Lin, Silvio Savarese
Abstract: We present a dense reconstruction approach that overcomes the drawbacks of traditional multiview stereo by incorporating semantic information in the form of learned category-level shape priors and object detection. Given training data comprised of 3D scans and images of objects from various viewpoints, we learn a prior comprised of a mean shape and a set of weighted anchor points. The former captures the commonality of shapes across the category, while the latter encodes similarities between instances in the form of appearance and spatial consistency. We propose robust algorithms to match anchor points across instances that enable learning a mean shape for the category, even with large shape variations across instances. We model the shape of an object instance as a warped version of the category mean, along with instance-specific details. Given multiple images of an unseen instance, we collate information from 2D object detectors to align the structure from motion point cloud with the mean shape, which is subsequently warped and refined to approach the actual shape. Extensive experiments demonstrate that our model is general enough to learn semantic priors for different object categories, yet powerful enough to reconstruct individual shapes with large variations. Qualitative and quantitative evaluations show that our framework can produce more accurate reconstructions than alternative state-of-the-art multiview stereo systems.
Reference: text
sentIndex sentText sentNum sentScore
1 Given training data comprised of 3D scans and images of objects from various viewpoints, we learn a prior comprised of a mean shape and a set of weighted anchor points. [sent-2, score-0.969]
2 We propose robust algorithms to match anchor points across instances that enable learning a mean shape for the category, even with large shape variations across instances. [sent-4, score-1.256]
3 We model the shape of an object instance as a warped version of the category mean, along with instance-specific details. [sent-5, score-0.317]
4 Given multiple images of an unseen instance, we collate information from 2D object detectors to align the structure from motion point cloud with the mean shape, which is subsequently warped and refined to approach the actual shape. [sent-6, score-0.301]
5 We propose a framework for semantic dense reconstruction that learns a category-level shape prior, which is used with weighted warping and refinement mechanisms to reconstruct regularized, high-quality 3D shapes. [sent-29, score-0.641]
6 This paper presents a framework for dense 3D reconstruction that overcomes the drawbacks of traditional MVS by leveraging semantic information in the form of object detection and shape priors learned from a database of training images and 3D shapes. [sent-31, score-0.592]
7 We postulate in Section 3 that while object instances within a category might have very different shapes and appearances, they share certain similarities at a semantic level. [sent-40, score-0.312]
8 We model semantic similarity as a shape prior, which consists of a set of automatically learned anchor points across several instances, along with a learned mean shape that captures the – 11111222226666624422 shared commonality of the entire category. [sent-42, score-1.379]
9 In the learning phase (Section 4), the anchor points encode attributes such as frequency, appearance and location similarity of features across instances. [sent-44, score-0.798]
10 Based on matched anchor points, the shape prior for a category is determined by a series of weighted thin-plate spline (TPS) warps over the scans of training objects. [sent-46, score-1.094]
11 Our reconstruction phase (Section 5) starts with a point cloud obtained by applying a structure-from-motion (SFM) or MVS system to images of an unseen instance (with a shape different from training objects). [sent-47, score-0.434]
12 This guides the process of matching anchor points shown by green stars in right panel in Figure 2 – between the learned prior and the test object’s SFM point cloud, followed by a warping of the prior shape in order to closely resemble the true shape. [sent-49, score-1.326]
13 Finer details not captured by the shape prior may be recovered by a refinement step, using guidance from SFM or MVS output. [sent-50, score-0.295]
14 The refinement combines confidence scores from anchor points and photoconsistency in order to produce a regularized, high quality output shape. [sent-51, score-0.909]
15 This paper provides a framework to augment traditional multiview stereo (MVS) reconstruction methods with semantic information. [sent-56, score-0.318]
16 A set of example shapes is used by active shape models (ASM) to encode patterns of variability, thereby ensuring a fitted shape consistent with deformations observed in training [8]. [sent-63, score-0.389]
17 Subsequent works on statistical shape analysis [10] allow nonrigid TPS warps between shapes [5], but often require landmark identification and initial rigid alignment based on point distributions, which is not feasible for general scenes [24]. [sent-66, score-0.393]
18 We use semantic information, namely object detection for localization and anchor point matching, to overcome those drawbacks. [sent-67, score-0.841]
19 Learned anchor points yield confidence scores, which guide our deformation process through a weighted TPS [26]. [sent-68, score-0.797]
20 Morphable models in 3D demonstrate realistic shape recovery, but are limited to categories like faces with low shape variation that can be accurately modeled with a linear PCA basis [4]. [sent-69, score-0.32]
21 By exploiting semantics in the form of object detection and anchor point matching, we handle both greater shape variation and noisy, incomplete, image-based MVS inputs. [sent-72, score-0.929]
22 Determining correspondence across instances with varying shape is a key step in shape matching. [sent-74, score-0.418]
23 The demands on correspondences for 3D reconstruction are far higher than 2D shape matching competing factors like high localization accuracy, stringent outlier rejection and good density are all crucial to obtaining a high quality dense reconstruction. [sent-79, score-0.384]
24 However, the complexity of 3D shapes and the accuracy demands of 3D reconstruction necessitate far greater control over the deformation process, so we consider it advantageous to compute priors in the mesh space. [sent-86, score-0.324]
25 Our Model We assume that for each object category, there exists a prior that consists of a 3D mean shape S∗ that captures the commonality of shapes across all instances and a set of anchor points A that captures similarities between subsets of instances. [sent-88, score-1.321]
26 The shape of any particular object Si is a transformation of S∗, plus specific details Δi not shared by other instances: Si = T({S∗, A}, θi) + Δi, (1) where T is a warping (transformation) function and θi is the warping parameter that is unique to each object instance. [sent-89, score-0.506]
27 We leverage on certain reliable features associated with the shape prior, which we call anchor points. [sent-93, score-0.848]
28 Anchor points form the backbone of our framework, since they are representative of object shape and the relative importance of different object structures. [sent-94, score-0.322]
29 Anchor points with high weights, ω, are considered stable in terms of location and appearance, and thus, more representative of object shape across instances. [sent-95, score-0.311]
30 1, we detail the mechanism of learning anchor points from training data. [sent-98, score-0.768]
31 In particular, prior work on shape matching [2, 19] has demonstrated inspiring results using regularized thin-plate spline (TPS) transformations [5] to capture deformations. [sent-101, score-0.315]
32 i}, i= 1, · · · , n, be two sets of anchor points for object ins{taxnc}es Oi = =an 1d, ·O··? [sent-103, score-0.809]
33 Semantic information of this nature is determined automatically in our framework by the anchor point learning mechanism. [sent-123, score-0.728]
34 To incorporate semantic information from anchor points, in the form of a weight matrix W = diag(ω1 , · · · , ωn), we use an extension of TPS [26]: (K + nλW−1)β + Φα = x? [sent-124, score-0.76]
35 Details specific to each object that are not captured in the shape prior are recovered by a refinement step. [sent-128, score-0.336]
36 This refinement is used in both mean shape learning and during reconstruction of a particular test object. [sent-129, score-0.414]
37 To refine a shape Si (a mesh) towards shape Sj , we compute displacements for vertices in Si. [sent-130, score-0.346]
38 The vertices of the refined shape are obtained as pik + dik and it inherits the connectivity of Si. [sent-149, score-0.415]
39 This is because the above mechanism can be used, with minor changes, for both mean shape learning with the shape Sj being a mesh and for reconstruction with Sj being the oriented point cloud output of MVS, as elaborated in Sections 4. [sent-151, score-0.644]
40 Learning Reconstruction Priors For each object category, we use a set of object instances {On} to learn a mean shape S∗ and a set of anchor points A. [sent-155, score-1.118]
41 They also serve as the initial{izSatio}n for the anchor point learning, as described in the following. [sent-160, score-0.728]
42 Learning Anchor Points An anchor point, A = {Γ, χ, ω}, consists of a feature vector Γ that describes appearance, t,hωe} 3D location χ with respect to the mean shape and a scalar weight ω. [sent-163, score-0.888]
43 For cars, most anchor points are located around wheels and body corners since those parts are shared across instances. [sent-167, score-0.798]
44 For fruits, anchor points are distributed around the stem and bottom. [sent-168, score-0.768]
45 We also show image patches associated with the features of a few example anchor points. [sent-170, score-0.688]
46 For an anchor point A, if V are the indices of objects across which the corresponding SFMV points are matched and Ωi are the indices of images of Oi where A is visible, the corresponding feature vector is: Γ = {{fkii }ki∈Ωi }i∈V. [sent-172, score-0.901]
47 Then, the location for the anchor point is χj=|V1|? [sent-175, score-0.728]
48 (7) The weight ω reflects “importance” of an anchor point. [sent-177, score-0.688]
49 We consider an anchor point important if it appears across many instances, with low position and appearance variance. [sent-178, score-0.758]
50 In contrast to applications like shape matching, the quality of dense reconstruction is greatly affected by the order and extent of deformations. [sent-189, score-0.326]
51 Thus, the learned anchor point weights ω are crucial to the success of dense reconstruction. [sent-190, score-0.833]
52 The key precursor to learning anchor points is matching 3D points across instances, which is far from trivial. [sent-193, score-0.91]
53 Such points usually dominate an SFM point cloud, but do not generalize across instances Algorithm 1 Learning anchor points Set Parameters δf, δp. [sent-195, score-0.986]
54 endU wphdailtee Output: denser anchor point set A. [sent-213, score-0.728]
55 Learned shape prior and anchor points for keyboard category. [sent-215, score-1.018]
56 since they do not correspond to the object shape, thus, may not be anchor point candidates. [sent-220, score-0.769]
57 Moreover, the density of anchor points cannot be too low, since they guide the deformation process that computes the mean shape and fits it to the 3D point cloud. [sent-221, score-1.063]
58 To ensure the robustness of anchor point matching and good density, we propose an iterative algorithm, detailed in Algorithm 1. [sent-222, score-0.76]
59 The distribution and weights of the learned anchor points are visualized in Figure 3 and 4. [sent-223, score-0.837]
60 Mean Shape Construction The learned anchor points are used to compute a mean shape for an object category. [sent-226, score-1.052]
61 Recall that we have a mapping from the set of anchor points to each instance in the training set. [sent-227, score-0.797]
62 Thus, we can warp successive shapes closer to a mean shape using the anchor points. [sent-228, score-1.041]
63 The mean shape is constructed by combining these aligned and warped shapes of different instances. [sent-229, score-0.351]
64 In our experiments, we use the weighted number of commonly matched anchor points as the similarity cue. [sent-232, score-0.831]
65 We combine the warped shapes T(Ssican) following the order of merging successive branches, to eventually obtain a single shape S∗, which represents the commonality of all training instances. [sent-238, score-0.348]
66 The mean shape learning procedure is shown for a subset of the car dataset in Fig. [sent-240, score-0.304]
67 Note that S∗ is computed by using the warped training examples, where the warping maps the 3D locations of learned anchor points. [sent-242, score-0.907]
68 Thus, the prior shape is always aligned with the anchor points. [sent-243, score-0.925]
69 Two shapes aligned by anchor points are eventually combined into a single one using displacement vectors computed by minimizing (5). [sent-245, score-0.863]
70 Semantic Reconstruction with Shape Priors Given a number of images of an object O, we can reconstruct its 3D shape by warping the learned prior shape S∗ based on the estimated θ and by recovering Δ in (1) subsequently. [sent-249, score-0.614]
71 The reconstruction consists of three steps: matching anchor points, warping by anchor points, and refinement. [sent-250, score-1.658]
72 Accurately recovering warp parameters θ requires accurate matches between anchor points in S∗ and SFM points in Ssfm. [sent-251, score-0.942]
73 Initial Alignment It is conventional in shape modeling literature to compute shape alignments using Procrustes analysis or ICP [8]. [sent-255, score-0.32]
74 Matching anchor points from leaned model (left) to new object (right). [sent-266, score-0.809]
75 Since we also know those for the shape prior, we can use a rigid transformation to coarsely align the prior shape and its anchor points to fit the SFM point cloud of the object. [sent-276, score-1.303]
76 The initial alignment for a car reconstruction is shown in Figure 6. [sent-277, score-0.321]
77 Reconstruction Given a set of images I an object with unknown shape of S, we use standard SFM to recover the 3D point cloud Ssfm. [sent-281, score-0.316]
78 Our goal is to use the mean shape S∗ to produce a dense reconstruction that closely resembles S. [sent-282, score-0.366]
79 Since the initial alignment uses the object’s location, pose and scale, anchor points are likely to be aligned to 3D locations in the vicinity of their true matches. [sent-284, score-0.881]
80 Thus, the burden of identifying the point in Ssfm that corresponds to an anchor point in S∗ is reduced to a local search. [sent-285, score-0.768]
81 We use HOG features to match anchor points to SFM points. [sent-286, score-0.768]
82 Examples of robust anchor point matches from our algorithm are shown in Figure 7. [sent-288, score-0.762]
83 11111222226666668866 Algorithm 2 Matching anchor points Set parameters δ1δ2η. [sent-289, score-0.768]
84 Warping of the shape prior with the learned anchor points matched to SFM points using Algorithm 2. [sent-340, score-1.165]
85 Note that while the shape prior represents the commonality of all instances, anchor point-based warping recovers coarse aspects of instance-specific shape, such as the back geometry of Car 2. [sent-341, score-1.082]
86 Assume S∗ is the shape prior after the initial alignment of Section 5. [sent-343, score-0.298]
87 We use the above matches between anchor points in S∗ and SFM points in Ssfm to estimate parameters θ for the weighted TPS warping (4) and obtain S? [sent-345, score-1.002]
88 Notice that, this warping not only reduces the alignment error from the initial detection-based alignment, it also deforms the prior to fit the actual shape of the object. [sent-347, score-0.418]
89 This refined shape is the final output of our dense reconstruction framework. [sent-361, score-0.326]
90 The efficacy of using anchor points and their learned weights can be demonstrated by Table 2. [sent-386, score-0.837]
91 Using anchor points can greatly reduce the reconstruction error compared to only using object detection for alignment. [sent-387, score-0.939]
92 Learning anchor point weights further enhances the reconstruction accuracy. [sent-388, score-0.884]
93 RGD: Rigidly align mean shape to test object using matched anchor points. [sent-399, score-1.041]
94 WP: Align and warp mean shape using matched anchor points (without refinement). [sent-400, score-1.091]
95 RGD: Rigidly align the mean shape to a test object by using matched anchor points. [sent-416, score-1.041]
96 WP: Align and warp the mean shape by using matched anchor points (Section 5. [sent-417, score-1.091]
97 In contrast, we successfully learn meaningful semantic priors across shape variations and use them in our reconstruction, to produce the much higher quality reconstructions in (d), that closely resemble the ground truth (e). [sent-427, score-0.366]
98 Discussion and Future Work We have presented a comprehensive framework for dense object reconstruction that uses data-driven semantic priors to recover shape in situations unfavorable to traditional MVS. [sent-429, score-0.522]
99 Our learned priors, combined with robust anchor point matching and refinement mechanisms, are shown to produce visually high quality and quantitatively accurate results. [sent-430, score-0.887]
100 Evaluating shape correspondence for statistical shape analysis: A benchmark study. [sent-599, score-0.32]
wordName wordTfidf (topN-words)
[('anchor', 0.688), ('mvs', 0.327), ('sfm', 0.205), ('pik', 0.172), ('shape', 0.16), ('tps', 0.136), ('reconstruction', 0.13), ('warping', 0.12), ('car', 0.104), ('ssifm', 0.092), ('refinement', 0.084), ('points', 0.08), ('cloud', 0.075), ('fruits', 0.074), ('semantic', 0.072), ('shapes', 0.069), ('instances', 0.068), ('commonality', 0.063), ('matched', 0.063), ('alignment', 0.061), ('stars', 0.061), ('warp', 0.06), ('dik', 0.057), ('photoconsistency', 0.057), ('priors', 0.057), ('warped', 0.056), ('sj', 0.056), ('pkj', 0.055), ('ssfm', 0.055), ('multiview', 0.053), ('prior', 0.051), ('bk', 0.051), ('rear', 0.05), ('pj', 0.05), ('align', 0.049), ('reconstructions', 0.047), ('furukawa', 0.044), ('psr', 0.043), ('learned', 0.043), ('oi', 0.042), ('pages', 0.042), ('fruit', 0.041), ('object', 0.041), ('point', 0.04), ('mean', 0.04), ('keyboard', 0.039), ('mesh', 0.039), ('reconstruct', 0.039), ('regularized', 0.038), ('holes', 0.038), ('stereo', 0.037), ('fkii', 0.037), ('nki', 0.037), ('rgd', 0.037), ('spoiler', 0.037), ('ssican', 0.037), ('warps', 0.037), ('dense', 0.036), ('curless', 0.034), ('matches', 0.034), ('spline', 0.034), ('specularities', 0.033), ('plj', 0.033), ('matching', 0.032), ('reconstructed', 0.032), ('similarities', 0.031), ('category', 0.031), ('scans', 0.03), ('chui', 0.03), ('nik', 0.03), ('across', 0.03), ('cars', 0.03), ('bao', 0.029), ('instance', 0.029), ('morphable', 0.029), ('pji', 0.029), ('schematic', 0.029), ('deformation', 0.029), ('diffuse', 0.028), ('inset', 0.027), ('gallup', 0.027), ('poisson', 0.027), ('drawbacks', 0.027), ('seitz', 0.027), ('si', 0.026), ('aligned', 0.026), ('nec', 0.026), ('scan', 0.026), ('traditional', 0.026), ('density', 0.026), ('vertices', 0.026), ('weights', 0.026), ('initial', 0.026), ('michigan', 0.025), ('pauly', 0.025), ('rigidly', 0.024), ('scanner', 0.024), ('mi', 0.024), ('closer', 0.024), ('unique', 0.024)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 110 cvpr-2013-Dense Object Reconstruction with Semantic Priors
Author: Sid Yingze Bao, Manmohan Chandraker, Yuanqing Lin, Silvio Savarese
Abstract: We present a dense reconstruction approach that overcomes the drawbacks of traditional multiview stereo by incorporating semantic information in the form of learned category-level shape priors and object detection. Given training data comprised of 3D scans and images of objects from various viewpoints, we learn a prior comprised of a mean shape and a set of weighted anchor points. The former captures the commonality of shapes across the category, while the latter encodes similarities between instances in the form of appearance and spatial consistency. We propose robust algorithms to match anchor points across instances that enable learning a mean shape for the category, even with large shape variations across instances. We model the shape of an object instance as a warped version of the category mean, along with instance-specific details. Given multiple images of an unseen instance, we collate information from 2D object detectors to align the structure from motion point cloud with the mean shape, which is subsequently warped and refined to approach the actual shape. Extensive experiments demonstrate that our model is general enough to learn semantic priors for different object categories, yet powerful enough to reconstruct individual shapes with large variations. Qualitative and quantitative evaluations show that our framework can produce more accurate reconstructions than alternative state-of-the-art multiview stereo systems.
2 0.25705382 192 cvpr-2013-Graph Matching with Anchor Nodes: A Learning Approach
Author: Nan Hu, Raif M. Rustamov, Leonidas Guibas
Abstract: In this paper, we consider the weighted graph matching problem with partially disclosed correspondences between a number of anchor nodes. Our construction exploits recently introduced node signatures based on graph Laplacians, namely the Laplacian family signature (LFS) on the nodes, and the pairwise heat kernel map on the edges. In this paper, without assuming an explicit form of parametric dependence nor a distance metric between node signatures, we formulate an optimization problem which incorporates the knowledge of anchor nodes. Solving this problem gives us an optimized proximity measure specific to the graphs under consideration. Using this as a first order compatibility term, we then set up an integer quadratic program (IQP) to solve for a near optimal graph matching. Our experiments demonstrate the superior performance of our approach on randomly generated graphs and on two widelyused image sequences, when compared with other existing signature and adjacency matrix based graph matching methods.
Author: Ken Sakurada, Takayuki Okatani, Koichiro Deguchi
Abstract: This paper proposes a method for detecting temporal changes of the three-dimensional structure of an outdoor scene from its multi-view images captured at two separate times. For the images, we consider those captured by a camera mounted on a vehicle running in a city street. The method estimates scene structures probabilistically, not deterministically, and based on their estimates, it evaluates the probability of structural changes in the scene, where the inputs are the similarity of the local image patches among the multi-view images. The aim of the probabilistic treatment is to maximize the accuracy of change detection, behind which there is our conjecture that although it is difficult to estimate the scene structures deterministically, it should be easier to detect their changes. The proposed method is compared with the methods that use multi-view stereo (MVS) to reconstruct the scene structures of the two time points and then differentiate them to detect changes. The experimental results show that the proposed method outperforms such MVS-based methods.
4 0.11021759 97 cvpr-2013-Correspondence-Less Non-rigid Registration of Triangular Surface Meshes
Author: Zsolt Sánta, Zoltan Kato
Abstract: A novel correspondence-less approach is proposed to find a thin plate spline map between a pair of deformable 3D objects represented by triangular surface meshes. The proposed method works without landmark extraction and feature correspondences. The aligning transformation is found simply by solving a system of nonlinear equations. Each equation is generated by integrating a nonlinear function over the object’s domains. We derive recursive formulas for the efficient computation of these integrals. Based on a series of comparative tests on a large synthetic dataset, our triangular mesh-based algorithm outperforms state of the art methods both in terms of computing time and accuracy. The applicability of the proposed approach has been demonstrated on the registration of 3D lung CT volumes.
5 0.1100532 352 cvpr-2013-Recovering Stereo Pairs from Anaglyphs
Author: Armand Joulin, Sing Bing Kang
Abstract: An anaglyph is a single image created by selecting complementary colors from a stereo color pair; the user can perceive depth by viewing it through color-filtered glasses. We propose a technique to reconstruct the original color stereo pair given such an anaglyph. We modified SIFT-Flow and use it to initially match the different color channels across the two views. Our technique then iteratively refines the matches, selects the good matches (which defines the “anchor” colors), and propagates the anchor colors. We use a diffusion-based technique for the color propagation, and added a step to suppress unwanted colors. Results on a variety of inputs demonstrate the robustness of our technique. We also extended our method to anaglyph videos by using optic flow between time frames.
6 0.1052368 113 cvpr-2013-Dense Variational Reconstruction of Non-rigid Surfaces from Monocular Video
7 0.10337775 306 cvpr-2013-Non-rigid Structure from Motion with Diffusion Maps Prior
8 0.10172585 166 cvpr-2013-Fast Image Super-Resolution Based on In-Place Example Regression
9 0.097364001 230 cvpr-2013-Joint 3D Scene Reconstruction and Class Segmentation
10 0.095129862 87 cvpr-2013-Compressed Hashing
11 0.085891291 424 cvpr-2013-Templateless Quasi-rigid Shape Modeling with Implicit Loop-Closure
12 0.085328124 284 cvpr-2013-Mesh Based Semantic Modelling for Indoor and Outdoor Scenes
13 0.084903836 321 cvpr-2013-PDM-ENLOR: Learning Ensemble of Local PDM-Based Regressions
14 0.083035596 111 cvpr-2013-Dense Reconstruction Using 3D Object Shape Priors
15 0.078424312 360 cvpr-2013-Robust Estimation of Nonrigid Transformation for Point Set Registration
16 0.07818611 333 cvpr-2013-Plane-Based Content Preserving Warps for Video Stabilization
17 0.077221826 242 cvpr-2013-Label Propagation from ImageNet to 3D Point Clouds
18 0.075531855 268 cvpr-2013-Leveraging Structure from Motion to Learn Discriminative Codebooks for Scalable Landmark Classification
19 0.071567804 74 cvpr-2013-CLAM: Coupled Localization and Mapping with Efficient Outlier Handling
20 0.069823191 425 cvpr-2013-Tensor-Based High-Order Semantic Relation Transfer for Semantic Scene Segmentation
topicId topicWeight
[(0, 0.188), (1, 0.083), (2, 0.003), (3, 0.023), (4, 0.048), (5, -0.038), (6, -0.027), (7, 0.015), (8, -0.01), (9, -0.045), (10, -0.008), (11, 0.045), (12, -0.034), (13, 0.014), (14, 0.059), (15, -0.123), (16, -0.016), (17, 0.063), (18, 0.009), (19, 0.03), (20, -0.046), (21, -0.03), (22, 0.044), (23, 0.001), (24, 0.0), (25, -0.059), (26, 0.007), (27, -0.032), (28, 0.024), (29, 0.029), (30, -0.071), (31, -0.092), (32, -0.058), (33, -0.051), (34, 0.085), (35, 0.042), (36, -0.056), (37, -0.056), (38, -0.02), (39, -0.069), (40, -0.008), (41, 0.07), (42, -0.045), (43, -0.074), (44, 0.062), (45, -0.082), (46, -0.041), (47, 0.024), (48, -0.134), (49, 0.028)]
simIndex simValue paperId paperTitle
same-paper 1 0.92861521 110 cvpr-2013-Dense Object Reconstruction with Semantic Priors
Author: Sid Yingze Bao, Manmohan Chandraker, Yuanqing Lin, Silvio Savarese
Abstract: We present a dense reconstruction approach that overcomes the drawbacks of traditional multiview stereo by incorporating semantic information in the form of learned category-level shape priors and object detection. Given training data comprised of 3D scans and images of objects from various viewpoints, we learn a prior comprised of a mean shape and a set of weighted anchor points. The former captures the commonality of shapes across the category, while the latter encodes similarities between instances in the form of appearance and spatial consistency. We propose robust algorithms to match anchor points across instances that enable learning a mean shape for the category, even with large shape variations across instances. We model the shape of an object instance as a warped version of the category mean, along with instance-specific details. Given multiple images of an unseen instance, we collate information from 2D object detectors to align the structure from motion point cloud with the mean shape, which is subsequently warped and refined to approach the actual shape. Extensive experiments demonstrate that our model is general enough to learn semantic priors for different object categories, yet powerful enough to reconstruct individual shapes with large variations. Qualitative and quantitative evaluations show that our framework can produce more accurate reconstructions than alternative state-of-the-art multiview stereo systems.
2 0.70061082 424 cvpr-2013-Templateless Quasi-rigid Shape Modeling with Implicit Loop-Closure
Author: Ming Zeng, Jiaxiang Zheng, Xuan Cheng, Xinguo Liu
Abstract: This paper presents a method for quasi-rigid objects modeling from a sequence of depth scans captured at different time instances. As quasi-rigid objects, such as human bodies, usually have shape motions during the capture procedure, it is difficult to reconstruct their geometries. We represent the shape motion by a deformation graph, and propose a model-to-partmethod to gradually integrate sampled points of depth scans into the deformation graph. Under an as-rigid-as-possible assumption, the model-to-part method can adjust the deformation graph non-rigidly, so as to avoid error accumulation in alignment, which also implicitly achieves loop-closure. To handle the drift and topological error for the deformation graph, two algorithms are introduced. First, we use a two-stage registration to largely keep the rigid motion part. Second, in the step of graph integration, we topology-adaptively integrate new parts and dynamically control the regularization effect of the deformation graph. We demonstrate the effectiveness and robustness of our method by several depth sequences of quasi-rigid objects, and an application in human shape modeling.
3 0.63362688 113 cvpr-2013-Dense Variational Reconstruction of Non-rigid Surfaces from Monocular Video
Author: Ravi Garg, Anastasios Roussos, Lourdes Agapito
Abstract: This paper offers the first variational approach to the problem of dense 3D reconstruction of non-rigid surfaces from a monocular video sequence. We formulate nonrigid structure from motion (NRSfM) as a global variational energy minimization problem to estimate dense low-rank smooth 3D shapes for every frame along with the camera motion matrices, given dense 2D correspondences. Unlike traditional factorization based approaches to NRSfM, which model the low-rank non-rigid shape using a fixed number of basis shapes and corresponding coefficients, we minimize the rank of the matrix of time-varying shapes directly via trace norm minimization. In conjunction with this low-rank constraint, we use an edge preserving total-variation regularization term to obtain spatially smooth shapes for every frame. Thanks to proximal splitting techniques the optimization problem can be decomposed into many point-wise sub-problems and simple linear systems which can be easily solved on GPU hardware. We show results on real sequences of different objects (face, torso, beating heart) where, despite challenges in tracking, illumination changes and occlusions, our method reconstructs highly deforming smooth surfaces densely and accurately directly from video, without the need for any prior models or shape templates.
4 0.63048518 321 cvpr-2013-PDM-ENLOR: Learning Ensemble of Local PDM-Based Regressions
Author: Yen H. Le, Uday Kurkure, Ioannis A. Kakadiaris
Abstract: Statistical shape models, such as Active Shape Models (ASMs), sufferfrom their inability to represent a large range of variations of a complex shape and to account for the large errors in detection of model points. We propose a novel method (dubbed PDM-ENLOR) that overcomes these limitations by locating each shape model point individually using an ensemble of local regression models and appearance cues from selected model points. Our method first detects a set of reference points which were selected based on their saliency during training. For each model point, an ensemble of regressors is built. From the locations of the detected reference points, each regressor infers a candidate location for that model point using local geometric constraints, encoded by a point distribution model (PDM). The final location of that point is determined as a weighted linear combination, whose coefficients are learnt from the training data, of candidates proposed from its ensemble ’s component regressors. We use different subsets of reference points as explanatory variables for the component regressors to provide varying degrees of locality for the models in each ensemble. This helps our ensemble model to capture a larger range of shape variations as compared to a single PDM. We demonstrate the advantages of our method on the challenging problem of segmenting gene expression images of mouse brain.
5 0.62271434 96 cvpr-2013-Correlation Filters for Object Alignment
Author: Vishnu Naresh Boddeti, Takeo Kanade, B.V.K. Vijaya Kumar
Abstract: Alignment of 3D objects from 2D images is one of the most important and well studied problems in computer vision. A typical object alignment system consists of a landmark appearance model which is used to obtain an initial shape and a shape model which refines this initial shape by correcting the initialization errors. Since errors in landmark initialization from the appearance model propagate through the shape model, it is critical to have a robust landmark appearance model. While there has been much progress in designing sophisticated and robust shape models, there has been relatively less progress in designing robust landmark detection models. In thispaper wepresent an efficient and robust landmark detection model which is designed specifically to minimize localization errors thereby leading to state-of-the-art object alignment performance. We demonstrate the efficacy and speed of the proposed approach on the challenging task of multi-view car alignment.
6 0.60533732 106 cvpr-2013-Deformable Graph Matching
7 0.60341644 81 cvpr-2013-City-Scale Change Detection in Cadastral 3D Models Using Images
8 0.59596121 47 cvpr-2013-As-Projective-As-Possible Image Stitching with Moving DLT
9 0.59442008 195 cvpr-2013-HDR Deghosting: How to Deal with Saturation?
10 0.59332699 177 cvpr-2013-FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps
11 0.58773851 107 cvpr-2013-Deformable Spatial Pyramid Matching for Fast Dense Correspondences
12 0.58564883 192 cvpr-2013-Graph Matching with Anchor Nodes: A Learning Approach
13 0.58023947 360 cvpr-2013-Robust Estimation of Nonrigid Transformation for Point Set Registration
14 0.57959688 341 cvpr-2013-Procrustean Normal Distribution for Non-rigid Structure from Motion
15 0.57647896 97 cvpr-2013-Correspondence-Less Non-rigid Registration of Triangular Surface Meshes
16 0.56899005 80 cvpr-2013-Category Modeling from Just a Single Labeling: Use Depth Information to Guide the Learning of 2D Models
17 0.56643981 467 cvpr-2013-Wide-Baseline Hair Capture Using Strand-Based Refinement
18 0.56606179 105 cvpr-2013-Deep Learning Shape Priors for Object Segmentation
19 0.5572992 231 cvpr-2013-Joint Detection, Tracking and Mapping by Semantic Bundle Adjustment
20 0.54859197 306 cvpr-2013-Non-rigid Structure from Motion with Diffusion Maps Prior
topicId topicWeight
[(10, 0.113), (16, 0.036), (26, 0.073), (33, 0.242), (36, 0.227), (67, 0.038), (69, 0.049), (87, 0.1)]
simIndex simValue paperId paperTitle
1 0.86873335 236 cvpr-2013-K-Means Hashing: An Affinity-Preserving Quantization Method for Learning Binary Compact Codes
Author: Kaiming He, Fang Wen, Jian Sun
Abstract: In computer vision there has been increasing interest in learning hashing codes whose Hamming distance approximates the data similarity. The hashing functions play roles in both quantizing the vector space and generating similarity-preserving codes. Most existing hashing methods use hyper-planes (or kernelized hyper-planes) to quantize and encode. In this paper, we present a hashing method adopting the k-means quantization. We propose a novel Affinity-Preserving K-means algorithm which simultaneously performs k-means clustering and learns the binary indices of the quantized cells. The distance between the cells is approximated by the Hamming distance of the cell indices. We further generalize our algorithm to a product space for learning longer codes. Experiments show our method, named as K-means Hashing (KMH), outperforms various state-of-the-art hashing encoding methods.
2 0.8492763 319 cvpr-2013-Optimized Product Quantization for Approximate Nearest Neighbor Search
Author: Tiezheng Ge, Kaiming He, Qifa Ke, Jian Sun
Abstract: Product quantization is an effective vector quantization approach to compactly encode high-dimensional vectors for fast approximate nearest neighbor (ANN) search. The essence of product quantization is to decompose the original high-dimensional space into the Cartesian product of a finite number of low-dimensional subspaces that are then quantized separately. Optimal space decomposition is important for the performance of ANN search, but still remains unaddressed. In this paper, we optimize product quantization by minimizing quantization distortions w.r.t. the space decomposition and the quantization codebooks. We present two novel methods for optimization: a nonparametric method that alternatively solves two smaller sub-problems, and a parametric method that is guaranteed to achieve the optimal solution if the input data follows some Gaussian distribution. We show by experiments that our optimized approach substantially improves the accuracy of product quantization for ANN search.
same-paper 3 0.84258777 110 cvpr-2013-Dense Object Reconstruction with Semantic Priors
Author: Sid Yingze Bao, Manmohan Chandraker, Yuanqing Lin, Silvio Savarese
Abstract: We present a dense reconstruction approach that overcomes the drawbacks of traditional multiview stereo by incorporating semantic information in the form of learned category-level shape priors and object detection. Given training data comprised of 3D scans and images of objects from various viewpoints, we learn a prior comprised of a mean shape and a set of weighted anchor points. The former captures the commonality of shapes across the category, while the latter encodes similarities between instances in the form of appearance and spatial consistency. We propose robust algorithms to match anchor points across instances that enable learning a mean shape for the category, even with large shape variations across instances. We model the shape of an object instance as a warped version of the category mean, along with instance-specific details. Given multiple images of an unseen instance, we collate information from 2D object detectors to align the structure from motion point cloud with the mean shape, which is subsequently warped and refined to approach the actual shape. Extensive experiments demonstrate that our model is general enough to learn semantic priors for different object categories, yet powerful enough to reconstruct individual shapes with large variations. Qualitative and quantitative evaluations show that our framework can produce more accurate reconstructions than alternative state-of-the-art multiview stereo systems.
4 0.82384145 179 cvpr-2013-From N to N+1: Multiclass Transfer Incremental Learning
Author: Ilja Kuzborskij, Francesco Orabona, Barbara Caputo
Abstract: Since the seminal work of Thrun [17], the learning to learnparadigm has been defined as the ability ofan agent to improve its performance at each task with experience, with the number of tasks. Within the object categorization domain, the visual learning community has actively declined this paradigm in the transfer learning setting. Almost all proposed methods focus on category detection problems, addressing how to learn a new target class from few samples by leveraging over the known source. But if one thinks oflearning over multiple tasks, there is a needfor multiclass transfer learning algorithms able to exploit previous source knowledge when learning a new class, while at the same time optimizing their overall performance. This is an open challenge for existing transfer learning algorithms. The contribution of this paper is a discriminative method that addresses this issue, based on a Least-Squares Support Vector Machine formulation. Our approach is designed to balance between transferring to the new class and preserving what has already been learned on the source models. Exten- sive experiments on subsets of publicly available datasets prove the effectiveness of our approach.
5 0.82171196 267 cvpr-2013-Least Soft-Threshold Squares Tracking
Author: Dong Wang, Huchuan Lu, Ming-Hsuan Yang
Abstract: In this paper, we propose a generative tracking method based on a novel robust linear regression algorithm. In contrast to existing methods, the proposed Least Soft-thresold Squares (LSS) algorithm models the error term with the Gaussian-Laplacian distribution, which can be solved efficiently. Based on maximum joint likelihood of parameters, we derive a LSS distance to measure the difference between an observation sample and the dictionary. Compared with the distance derived from ordinary least squares methods, the proposed metric is more effective in dealing with outliers. In addition, we present an update scheme to capture the appearance change of the tracked target and ensure that the model is properly updated. Experimental results on several challenging image sequences demonstrate that the proposed tracker achieves more favorable performance than the state-of-the-art methods.
6 0.80410367 63 cvpr-2013-Binary Code Ranking with Weighted Hamming Distance
7 0.79964304 79 cvpr-2013-Cartesian K-Means
8 0.78957695 284 cvpr-2013-Mesh Based Semantic Modelling for Indoor and Outdoor Scenes
9 0.78361493 365 cvpr-2013-Robust Real-Time Tracking of Multiple Objects by Volumetric Mass Densities
10 0.77833521 331 cvpr-2013-Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis
11 0.77829194 19 cvpr-2013-A Minimum Error Vanishing Point Detection Approach for Uncalibrated Monocular Images of Man-Made Environments
12 0.77787912 61 cvpr-2013-Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics
13 0.77755868 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
14 0.7763266 248 cvpr-2013-Learning Collections of Part Models for Object Recognition
15 0.77593499 69 cvpr-2013-Boosting Binary Keypoint Descriptors
16 0.77551603 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
17 0.77547532 443 cvpr-2013-Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
18 0.77509636 298 cvpr-2013-Multi-scale Curve Detection on Surfaces
19 0.77471423 242 cvpr-2013-Label Propagation from ImageNet to 3D Point Clouds
20 0.77429575 147 cvpr-2013-Ensemble Learning for Confidence Measures in Stereo Vision