iccv iccv2013 iccv2013-256 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Marius Leordeanu, Andrei Zanfir, Cristian Sminchisescu
Abstract: Estimating a dense correspondence field between successive video frames, under large displacement, is important in many visual learning and recognition tasks. We propose a novel sparse-to-dense matching method for motion field estimation and occlusion detection. As an alternative to the current coarse-to-fine approaches from the optical flow literature, we start from the higher level of sparse matching with rich appearance and geometric constraints collected over extended neighborhoods, using an occlusion aware, locally affine model. Then, we move towards the simpler, but denser classic flow field model, with an interpolation procedure that offers a natural transition between the sparse and the dense correspondence fields. We experimentally demonstrate that our appearance features and our complex geometric constraintspermit the correct motion estimation even in difficult cases of large displacements and significant appearance changes. We also propose a novel classification method for occlusion detection that works in conjunction with the sparse-to-dense matching model. We validate our approach on the newly released Sintel dataset and obtain state-of-the-art results.
Reference: text
sentIndex sentText sentNum sentScore
1 an Abstract Estimating a dense correspondence field between successive video frames, under large displacement, is important in many visual learning and recognition tasks. [sent-8, score-0.297]
2 We propose a novel sparse-to-dense matching method for motion field estimation and occlusion detection. [sent-9, score-0.751]
3 As an alternative to the current coarse-to-fine approaches from the optical flow literature, we start from the higher level of sparse matching with rich appearance and geometric constraints collected over extended neighborhoods, using an occlusion aware, locally affine model. [sent-10, score-1.255]
4 Then, we move towards the simpler, but denser classic flow field model, with an interpolation procedure that offers a natural transition between the sparse and the dense correspondence fields. [sent-11, score-0.766]
5 We experimentally demonstrate that our appearance features and our complex geometric constraintspermit the correct motion estimation even in difficult cases of large displacements and significant appearance changes. [sent-12, score-0.363]
6 We also propose a novel classification method for occlusion detection that works in conjunction with the sparse-to-dense matching model. [sent-13, score-0.52]
7 Introduction Estimating a dense correspondence field between video frames is essential for many higher-level visual tasks, such as tracking, grouping, image segmentation and object recognition. [sent-16, score-0.297]
8 This problem, known as optical flow estimation, consists of inferring the displacements in the image caused by camera motion or the moving objects in the scene, and has been initially formulated for small, differential apparent motions. [sent-17, score-0.597]
9 1 The task is difficult for many reasons: 1) finding a dense correspondence field is computa∗The first two authors contributed equally. [sent-18, score-0.297]
10 1The is part of the general problem of motion field estimation, although optical cues do not always fully constrain the field, as it is well known. [sent-19, score-0.476]
11 Recently, there has been growing interest in integrating sparse feature matching into models that process large motions [30, 6, 32, 22], with significant improvements over traditional approaches. [sent-25, score-0.348]
12 However, the sophistication of the sparse matching component used is still limited: in [6] authors use nearest neighbor schemes, whereas others [32, 22] rely on energy models rooted in the classic Total Variation (TV) formulation. [sent-26, score-0.381]
13 We make the following contributions: 1) we propose the first affine-invariant and occlusion-aware matching algorithm to estimate a dense correspondence field between images, under potentially large displacements. [sent-27, score-0.461]
14 We rely on a new sparse-to-dense approach starting from a complex sparse matching cost, then making the transition towards a dense motion field using a novel interpolation method. [sent-28, score-0.776]
15 The final motion field is obtained using local refinement based on continuous optimization. [sent-29, score-0.397]
16 2) We introduce a new occlusion classifier that operates in conjunction with the sparseto-dense interpolation method, with the two modules providing mutually informative cues, to improve performance. [sent-30, score-0.434]
17 11772211 Overview of our approach: We propose a hierarchical sparse-to-dense matching method that generalizes ideas both from the traditional coarse-to-fine variational approach and from recent methods based on sparse feature matching [4, 16, 19, 8, 6, 32, 22]. [sent-31, score-0.489]
18 It starts from optimizing a sparse matching energy with rich appearance unary terms and locally affine, occlusion sensitive, geometric constraints. [sent-33, score-0.834]
19 Next, using the same high-order geometric constraints, a second stage of sparseto-dense interpolation follows, making the transition from a set of sparse matches on a grid to a dense correspondence field. [sent-35, score-0.684]
20 After every matching stage, a classifier based on intermediate motion cues produces an occlusion probability map that is then passed to the next matching level. [sent-37, score-0.883]
21 At the end, all cues obtained from the different stages of correspondence field estimation are fed into the occlusion classifier that produces the final occlusion map. [sent-38, score-1.013]
22 Through this synergy between matching and occlusion estimation we obtain both improved optical flow and occlusion maps. [sent-39, score-1.218]
23 Sparse Matching: Initialize a discrete set of candidate matches for each point on a sparse grid (in the first im- age) using dense kNN matching (in the second image) based on local feature descriptors. [sent-41, score-0.495]
24 Use the output of a boundary detector and cues from the initial sparse matching model to infer a first occlusion probability map. [sent-42, score-0.772]
25 Then discretely optimize a sparse matching cost function using both unary, data terms (from local features), and geometric relationships based on a locally affine model with occlusion constraints. [sent-43, score-0.985]
26 Use the improved matches to refine the occlusion map. [sent-44, score-0.388]
27 Then apply the same occlusion sensitive geometric model to obtain an accurate sparse-to-dense interpolation of matches. [sent-47, score-0.496]
28 Dense matching refinement: use a TV model with continuous flow optimization to obtain the final dense correspondence field. [sent-49, score-0.611]
29 Use matching cues computed from all stages to obtain the final occlusion map. [sent-50, score-0.599]
30 Discussion: Phase 1ofour approach is related to both graph matching algorithms [4, 16, 19, 8] that use sparse local features and second or higher order geometric relationships, and recent optical flow methods that used descriptor matching [6, 32, 22]. [sent-51, score-0.833]
31 This stage aims to minimize the relative motion error even when the displacements are generally large, so to bring the solution in the neighborhood of the correct one. [sent-52, score-0.418]
32 The next sparse-to-dense interpolation phase ensures a natural transition towards a dense motion field. [sent-53, score-0.455]
33 The different phases complement each other: phase 1 is not sensitive to small displacements, since it uses sparse matches and descriptors extracted over relatively large neighborhoods; phase 3 uses a TV model and is most appropriate for small motions; phase 2 provides a transition between 1 and 3. [sent-59, score-0.467]
34 Our approach is × different from large displacement optical flow methods that append an energy term based on sparse matching to the variational framework [6]. [sent-60, score-0.896]
35 It is also different from methods that use descriptor matching only to find candidate flows, but then optimize an energy model with local smoothness constraints [32]. [sent-61, score-0.269]
36 The SIFT-flow energy model [22] uses rich descriptors [23], but a local flow spatial regularization term. [sent-62, score-0.311]
37 Our method is closer to the approach of [4] that first applies sparse graph matching formulated as IQP, then uses thin plate splines for dense interpolation. [sent-63, score-0.387]
38 Mathematical Formulation Before we detail our main sparse-to-dense algorithm, we first define the sparse matching task. [sent-66, score-0.276]
39 Given two images I1 and I2 of the same scene, the sparse matching problem consists of finding the correspondences in I2 for N points located on a uniform grid in I1. [sent-67, score-0.365]
40 Finding the correct matches is equivalent to solving for the sparse displacement field–a 2N 1 vector w, such that the displacement of feature iis [2wNix × , wiy] . [sent-68, score-0.362]
41 Sparse matching hise th diesnp lfaocrmemuelnatted of as athtuer optimization of a cost function that is a linear combination of a data term and a spatial energy term: 11772222 E(w) = ED(w) + λES(w). [sent-69, score-0.384]
42 Using edge information is not uncommon to optical flow estimation methods [6, 32]. [sent-76, score-0.366]
43 The motions of such points within a certain neighborhood are expected to closely follow an affine transformation. [sent-80, score-0.41]
44 Our novel quadratic affine error model effectively produces an extended neighborhood where pairs of cpronondeuccetesd a points (i, j) ceoingthrbiobrutheo otod th syes tteotmal error as wi> Qijwj . [sent-88, score-0.436]
45 Locally Affine Spatial Model For each point i, we estimate its affine transformation TNi = (Ai, ti) from neighboring motions by weighted least squares, with weights eij , ∀j ∈ Ni. [sent-101, score-0.443]
46 Discussion: S implicitly induces a different, extended neighborhood system in the spatial energy, as follows: for any two points (i, j) with a common neighbor in the original neighborhood system, that is ∃k s. [sent-141, score-0.284]
47 As mentioned previously our geometric constraints are affine invariant - stated in the next property: Property 2: Let wA be the motion field corresponding to some global affine transformation A over all points in I1. [sent-147, score-0.772]
48 Proof: Follows immediately from the fact that each individual motion will perfectly agree with the estimated affine transformation. [sent-149, score-0.313]
49 Regardless of the neighbors chosen and their soft memberships, the estimated affine model will always be the same as the global model. [sent-150, score-0.298]
50 The spatial error term will be zero for transformations such as rotation or scaling, regardless of the scaling factor or the rotation angle, as our model is affine invariant. [sent-153, score-0.312]
51 This is radically different and more powerful than both variational optical flow, as well as graph matching approaches based on second or third-order constraints, which are not affine invariant. [sent-154, score-0.576]
52 Moreover, the use of boundary and occlusion information could extend the class ofzero spatial error displacement fields to piecewise affine transformations, since it could decouple the graph. [sent-155, score-0.747]
53 Other formulations related to our sparse matching model include [20, 13, 14], with [20] being the only affine invariant model, but without occlusion. [sent-157, score-0.471]
54 fOeaurtu sparse matching approach oefefe §r3s a locally affine formulation based on only pairwise, secondorder relationships, allowing efficient optimization (§3). [sent-159, score-0.538]
55 Let us consider the individual update of the displacement wi at node i. [sent-167, score-0.367]
56 (5) |{z} {Kzi The energy is the sum o|f a constant t{erzm K0, indepe}ndent of wi and Ki, which is a function of wi. [sent-171, score-0.379]
57 At step S2 the list of candidate matches for the current site is temporarily augmented with its current match (if not already included) and the continuous solution From the augmented list the minimizer wi∗ of Ki is found (S3) and the current wi is updated (S4). [sent-176, score-0.436]
58 Since the total cost is non-negative, bounded below, and the energy is monotonically decreasing, the sparse matching method is guaranteed to converge. [sent-180, score-0.411]
59 Then, the sparse matches become input to an interpolation procedure that produces a dense field w(d). [sent-181, score-0.54]
60 Sparse-to-Dense Interpolation: we fix the sparse displacements w as anchors, and, for each point i at dense locations, we define a similar neighborhood system, with neighbors taken from the set of sparse anchors. [sent-182, score-0.684]
61 Having fixed the sparse matches, we can afford to drop the data term and use only the spatial constraints to predict the dense mo- in Ni(E). [sent-183, score-0.308]
62 tions based on the same locally affine model with occlusion and boundary constraints. [sent-185, score-0.678]
63 It is straightforward to show that the spatial, dense energy model, can be globally optimized by simply setting the value of the dense field w(d) to the predicted field w˜ (d) , for each site i: ← TNi (pi) − pi. [sent-186, score-0.579]
64 As presented at the beginning, the dense output of t)h−is pinterpolation step becomes input to a TV-based continuation method, for dense refinement. [sent-187, score-0.274]
65 The number of iterations in practice is between 5 to 10, which makes our sparse matching algorithm very efficient (30 sec). [sent-190, score-0.276]
66 The size and shapes of these regions of occlusion depend mainly on the magnitude and velocity of the relative motion and 3D geometric scene layout. [sent-199, score-0.504]
67 The true motion field inside occlusion regions is hard to predict and current motion models are not yet powerful enough to accomplish this task, even though there are many promising recent approaches. [sent-201, score-0.67]
68 Many occlusion estimation methods, including ours, take advantage of the fact that these optical flow energy models often fail in the areas of occlusion [27, 1, 11, 12]. [sent-202, score-1.119]
69 Also, occlusion regions are often around discontinuities in the image or in the matching field, which turns out to be an important cue for occlusion region detection. [sent-204, score-0.847]
70 Relative and absolute average motion field errors on the 996 image pairs of Sintel-TrainTest set, that we used for testing, between our method and three other optical flow algorithms: MDP [32], LDOF [6] and Classic-NL-full [28]. [sent-206, score-0.637]
71 We treat the occlusion estimation task as a binary classification problem [21, 15]. [sent-208, score-0.359]
72 We compute cues based on the ideas mentioned above and train a different Random Forest classifier for each matching phase, depending on the cues available at that stage. [sent-209, score-0.324]
73 It is worth mentioning here the most important features (out of 20) from the final occlusion estimation stage, automatically ranked by treebagger. [sent-211, score-0.39]
74 1) Cues from matching initialization stage: the most relevant feature in this category was the confidence of initial nearest neighbor matching–the number of neighbors close enough (within a fixed threshold) to the first nearest neighbor. [sent-213, score-0.267]
75 2) Cues from the sparse matching stage: we considered the different deviations of the output from the sparse matching model–errors in the data and spatial terms as well as errors in the affine model least squares fitting. [sent-215, score-0.87]
76 The deviation from the affinity model assumption was the most relevant, in this category, for estimating occlusion regions. [sent-216, score-0.324]
77 3) Dense matching cues: the most powerful feature computed from the final motion field is the number of points from I1 matched to the same point in I2. [sent-217, score-0.468]
78 4) Motion boundary features: once we have the final motion field we computed features from the continuous output of Gb [18] using the x, y motions as input layers. [sent-219, score-0.489]
79 The ground truth used in the evaluation is the actual motion field in both visible and occluded (unmatched) areas. [sent-222, score-0.284]
80 Average end point motion errors (EPE) over all dense locations for the 996 image pairs from Sintel-TrainTest. [sent-224, score-0.307]
81 Final results on Sintel optical flow benchmark; epe stands for end point errors, epem are errors for visible points, and epeo are errors for occluded points. [sent-231, score-0.586]
82 25781309 47 ficult cases of very large displacements (often around 40 pixels or more), motion and defocus blur, specular reflections and strong atmospheric effects. [sent-241, score-0.266]
83 Experiments on occlusion detection: In Table 3 we present quantitative comparisons of our occlusion classifier versus two state-of-the-art optical flow methods with occlusion estimation. [sent-257, score-1.303]
84 Our occlusion maps are raw, pixel-wise classifi11772266 Figure3. [sent-259, score-0.324]
85 Most methods published for occlusion detection produce contours, not large occluded regions. [sent-263, score-0.412]
86 During sparse matching initialization, we use 51 51 pixeDlus patches, saen dm 1at9c h×i 1g9 i patches during optimization. [sent-266, score-0.276]
87 Ttehme neighborhood size is adaptively increased, if necessary, until at least half of the neighbors considered are visible according to our occlusion map (maximum 60 neighbors allowed). [sent-268, score-0.628]
88 5 mins), sparse-to-dense interpolation (13 mins), occlusion detection (12 mins), continuous refinement (8 mins). [sent-276, score-0.604]
89 Conclusions We proposed an efficient sparse-to-dense matching method for motion and occlusion estimation, using locally affine and occlusion aware constraints. [sent-289, score-1.192]
90 Differently from the coarse-to-fine continuation approaches, we start from a sparse matching stage based on complex geometric relationships, and move towards a dense correspondence field based on a TV optical flow energy model. [sent-290, score-1.21]
91 This natural decomposition of the dense matching problem produces state of the art results on the most extensive motion field benchmark to date. [sent-291, score-0.571]
92 We also propose a novel occlusion classifier, with competitive performance, which works in good synergy with matching. [sent-292, score-0.364]
93 Future work will investigate improved models with tightly integrated matching and occlusion reasoning components. [sent-293, score-0.488]
94 Symmetrical dense optical flow estimation with occlusions 11772277 motion details by TV continuous refinement at the last stage. [sent-300, score-0.733]
95 Large displacement optical flow: descriptor matching in variational motion estimation. [sent-329, score-0.592]
96 A naturalistic open source movie for optical flow evaluation. [sent-339, score-0.331]
97 Estimation of occlusion and [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] dense motion fields in a bidirectional bayesian framework. [sent-422, score-0.553]
98 A probabilistic approach to large displacement optical flow and occlusion detection. [sent-457, score-0.748]
99 Occlusion boundary detection and figure/ground assignment from optical flow. [sent-474, score-0.292]
100 Robust computation of optical flow in a multiscale differential framework. [sent-479, score-0.331]
wordName wordTfidf (topN-words)
[('occlusion', 0.324), ('wi', 0.274), ('tni', 0.272), ('sintel', 0.265), ('affine', 0.195), ('optical', 0.168), ('matching', 0.164), ('flow', 0.163), ('displacements', 0.148), ('pi', 0.14), ('eij', 0.129), ('motion', 0.118), ('sparse', 0.112), ('ni', 0.112), ('dense', 0.111), ('field', 0.11), ('interpolation', 0.11), ('sip', 0.109), ('siw', 0.109), ('energy', 0.105), ('epe', 0.105), ('neighbors', 0.103), ('neighborhood', 0.098), ('mins', 0.096), ('displacement', 0.093), ('boundary', 0.092), ('tv', 0.082), ('cues', 0.08), ('correspondence', 0.076), ('leordeanu', 0.074), ('refinement', 0.072), ('intervening', 0.072), ('nnii', 0.072), ('oij', 0.072), ('motions', 0.072), ('locally', 0.067), ('continuous', 0.066), ('phase', 0.065), ('matches', 0.064), ('bij', 0.063), ('geometric', 0.062), ('occluded', 0.056), ('argminu', 0.054), ('dpt', 0.054), ('qiiu', 0.054), ('qiiwi', 0.054), ('qijwj', 0.054), ('xkpi', 0.054), ('stage', 0.054), ('continuation', 0.052), ('transition', 0.051), ('sites', 0.05), ('variational', 0.049), ('folders', 0.048), ('wiy', 0.048), ('destinations', 0.048), ('transformation', 0.047), ('errors', 0.047), ('phases', 0.045), ('sequential', 0.045), ('points', 0.045), ('qij', 0.045), ('mdp', 0.045), ('marius', 0.045), ('imar', 0.045), ('pni', 0.045), ('official', 0.045), ('grid', 0.044), ('spatial', 0.043), ('term', 0.042), ('argminw', 0.042), ('ki', 0.042), ('ldof', 0.04), ('synergy', 0.04), ('differently', 0.04), ('gij', 0.038), ('es', 0.038), ('di', 0.038), ('meant', 0.036), ('estimation', 0.035), ('benchmark', 0.035), ('discontinuities', 0.035), ('property', 0.034), ('abrupt', 0.034), ('quadratic', 0.034), ('squares', 0.033), ('produces', 0.033), ('move', 0.033), ('detection', 0.032), ('positions', 0.032), ('regardless', 0.032), ('dij', 0.032), ('site', 0.032), ('superior', 0.031), ('pairs', 0.031), ('final', 0.031), ('relationships', 0.031), ('connects', 0.03), ('cost', 0.03), ('iin', 0.03)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000015 256 iccv-2013-Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
Author: Marius Leordeanu, Andrei Zanfir, Cristian Sminchisescu
Abstract: Estimating a dense correspondence field between successive video frames, under large displacement, is important in many visual learning and recognition tasks. We propose a novel sparse-to-dense matching method for motion field estimation and occlusion detection. As an alternative to the current coarse-to-fine approaches from the optical flow literature, we start from the higher level of sparse matching with rich appearance and geometric constraints collected over extended neighborhoods, using an occlusion aware, locally affine model. Then, we move towards the simpler, but denser classic flow field model, with an interpolation procedure that offers a natural transition between the sparse and the dense correspondence fields. We experimentally demonstrate that our appearance features and our complex geometric constraintspermit the correct motion estimation even in difficult cases of large displacements and significant appearance changes. We also propose a novel classification method for occlusion detection that works in conjunction with the sparse-to-dense matching model. We validate our approach on the newly released Sintel dataset and obtain state-of-the-art results.
2 0.29069135 105 iccv-2013-DeepFlow: Large Displacement Optical Flow with Deep Matching
Author: Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Abstract: Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox & Malik [6], our approach, termed DeepFlow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable assetfor integration into an energy minimizationframework for optical flow estimation. DeepFlow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset [8].
3 0.26807988 317 iccv-2013-Piecewise Rigid Scene Flow
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
4 0.25225919 12 iccv-2013-A General Dense Image Matching Framework Combining Direct and Feature-Based Costs
Author: Jim Braux-Zin, Romain Dupont, Adrien Bartoli
Abstract: Dense motion field estimation (typically Romain Dupont1 romain . dupont @ cea . fr Adrien Bartoli2 adrien . bart o l @ gmai l com i . 2 ISIT, Universit e´ d’Auvergne/CNRS, France sions are explicitly modeled [32, 13]. Coarse-to-fine warping improves global convergence by making the assumption that optical flow, the motion of smaller structures is similar to the motion of stereo disparity and surface registration) is a key computer vision problem. Many solutions have been proposed to compute small or large displacements, narrow or wide baseline stereo disparity, but a unified methodology is still lacking. We here introduce a general framework that robustly combines direct and feature-based matching. The feature-based cost is built around a novel robust distance function that handles keypoints and “weak” features such as segments. It allows us to use putative feature matches which may contain mismatches to guide dense motion estimation out of local minima. Our framework uses a robust direct data term (AD-Census). It is implemented with a powerful second order Total Generalized Variation regularization with external and self-occlusion reasoning. Our framework achieves state of the art performance in several cases (standard optical flow benchmarks, wide-baseline stereo and non-rigid surface registration). Our framework has a modular design that customizes to specific application needs.
5 0.21607719 300 iccv-2013-Optical Flow via Locally Adaptive Fusion of Complementary Data Costs
Author: Tae Hyun Kim, Hee Seok Lee, Kyoung Mu Lee
Abstract: Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods.
6 0.17085725 190 iccv-2013-Handling Occlusions with Franken-Classifiers
7 0.16222952 269 iccv-2013-Modeling Occlusion by Discriminative AND-OR Structures
8 0.15986212 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
9 0.15825692 242 iccv-2013-Learning People Detectors for Tracking in Crowded Scenes
10 0.14941326 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
11 0.14413615 39 iccv-2013-Action Recognition with Improved Trajectories
12 0.14317439 424 iccv-2013-Tracking Revisited Using RGBD Camera: Unified Benchmark and Baselines
13 0.12866554 319 iccv-2013-Point-Based 3D Reconstruction of Thin Objects
14 0.11672729 238 iccv-2013-Learning Graphs to Match
15 0.11031373 143 iccv-2013-Estimating Human Pose with Flowing Puppets
16 0.10629375 270 iccv-2013-Modeling Self-Occlusions in Dynamic Shape and Appearance Tracking
17 0.10481373 237 iccv-2013-Learning Graph Matching: Oriented to Category Modeling from Cluttered Scenes
18 0.10481223 214 iccv-2013-Improving Graph Matching via Density Maximization
19 0.10344304 297 iccv-2013-Online Motion Segmentation Using Dynamic Label Propagation
20 0.10137062 108 iccv-2013-Depth from Combining Defocus and Correspondence Using Light-Field Cameras
topicId topicWeight
[(0, 0.27), (1, -0.136), (2, -0.014), (3, 0.065), (4, -0.002), (5, 0.047), (6, -0.057), (7, 0.034), (8, 0.052), (9, -0.022), (10, -0.039), (11, 0.058), (12, 0.227), (13, -0.026), (14, 0.052), (15, 0.006), (16, -0.039), (17, 0.06), (18, 0.216), (19, 0.082), (20, 0.042), (21, 0.028), (22, 0.014), (23, -0.088), (24, 0.144), (25, -0.136), (26, 0.058), (27, 0.06), (28, -0.012), (29, -0.067), (30, -0.018), (31, -0.052), (32, -0.028), (33, 0.027), (34, -0.068), (35, -0.027), (36, 0.059), (37, 0.002), (38, -0.079), (39, 0.012), (40, 0.057), (41, -0.119), (42, -0.021), (43, 0.05), (44, 0.009), (45, 0.012), (46, 0.026), (47, -0.004), (48, 0.039), (49, -0.041)]
simIndex simValue paperId paperTitle
same-paper 1 0.97742951 256 iccv-2013-Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
Author: Marius Leordeanu, Andrei Zanfir, Cristian Sminchisescu
Abstract: Estimating a dense correspondence field between successive video frames, under large displacement, is important in many visual learning and recognition tasks. We propose a novel sparse-to-dense matching method for motion field estimation and occlusion detection. As an alternative to the current coarse-to-fine approaches from the optical flow literature, we start from the higher level of sparse matching with rich appearance and geometric constraints collected over extended neighborhoods, using an occlusion aware, locally affine model. Then, we move towards the simpler, but denser classic flow field model, with an interpolation procedure that offers a natural transition between the sparse and the dense correspondence fields. We experimentally demonstrate that our appearance features and our complex geometric constraintspermit the correct motion estimation even in difficult cases of large displacements and significant appearance changes. We also propose a novel classification method for occlusion detection that works in conjunction with the sparse-to-dense matching model. We validate our approach on the newly released Sintel dataset and obtain state-of-the-art results.
2 0.90591598 105 iccv-2013-DeepFlow: Large Displacement Optical Flow with Deep Matching
Author: Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Abstract: Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox & Malik [6], our approach, termed DeepFlow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable assetfor integration into an energy minimizationframework for optical flow estimation. DeepFlow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset [8].
3 0.90411896 12 iccv-2013-A General Dense Image Matching Framework Combining Direct and Feature-Based Costs
Author: Jim Braux-Zin, Romain Dupont, Adrien Bartoli
Abstract: Dense motion field estimation (typically Romain Dupont1 romain . dupont @ cea . fr Adrien Bartoli2 adrien . bart o l @ gmai l com i . 2 ISIT, Universit e´ d’Auvergne/CNRS, France sions are explicitly modeled [32, 13]. Coarse-to-fine warping improves global convergence by making the assumption that optical flow, the motion of smaller structures is similar to the motion of stereo disparity and surface registration) is a key computer vision problem. Many solutions have been proposed to compute small or large displacements, narrow or wide baseline stereo disparity, but a unified methodology is still lacking. We here introduce a general framework that robustly combines direct and feature-based matching. The feature-based cost is built around a novel robust distance function that handles keypoints and “weak” features such as segments. It allows us to use putative feature matches which may contain mismatches to guide dense motion estimation out of local minima. Our framework uses a robust direct data term (AD-Census). It is implemented with a powerful second order Total Generalized Variation regularization with external and self-occlusion reasoning. Our framework achieves state of the art performance in several cases (standard optical flow benchmarks, wide-baseline stereo and non-rigid surface registration). Our framework has a modular design that customizes to specific application needs.
4 0.88336205 300 iccv-2013-Optical Flow via Locally Adaptive Fusion of Complementary Data Costs
Author: Tae Hyun Kim, Hee Seok Lee, Kyoung Mu Lee
Abstract: Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods.
5 0.80656332 317 iccv-2013-Piecewise Rigid Scene Flow
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
6 0.67341971 430 iccv-2013-Two-Point Gait: Decoupling Gait from Body Shape
7 0.64403951 301 iccv-2013-Optimal Orthogonal Basis and Image Assimilation: Motion Modeling
8 0.61857468 358 iccv-2013-Robust Non-parametric Data Fitting for Correspondence Modeling
9 0.61254638 270 iccv-2013-Modeling Self-Occlusions in Dynamic Shape and Appearance Tracking
10 0.57960552 420 iccv-2013-Topology-Constrained Layered Tracking with Latent Flow
11 0.57717216 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
12 0.57248145 263 iccv-2013-Measuring Flow Complexity in Videos
13 0.56860518 131 iccv-2013-EVSAC: Accelerating Hypotheses Generation by Modeling Matching Scores with Extreme Value Theory
14 0.56580764 39 iccv-2013-Action Recognition with Improved Trajectories
15 0.55354697 352 iccv-2013-Revisiting Example Dependent Cost-Sensitive Learning with Decision Trees
16 0.54838783 171 iccv-2013-Fix Structured Learning of 2013 ICCV paper k2opt.pdf
17 0.54739136 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
18 0.53332937 140 iccv-2013-Elastic Net Constraints for Shape Matching
19 0.52644897 82 iccv-2013-Compensating for Motion during Direct-Global Separation
20 0.52618068 255 iccv-2013-Local Signal Equalization for Correspondence Matching
topicId topicWeight
[(2, 0.079), (6, 0.197), (7, 0.018), (26, 0.078), (31, 0.059), (40, 0.017), (42, 0.095), (64, 0.057), (73, 0.056), (78, 0.014), (89, 0.236), (98, 0.019)]
simIndex simValue paperId paperTitle
1 0.9069559 11 iccv-2013-A Fully Hierarchical Approach for Finding Correspondences in Non-rigid Shapes
Author: Ivan Sipiran, Benjamin Bustos
Abstract: This paper presents a hierarchical method for finding correspondences in non-rigid shapes. We propose a new representation for 3D meshes: the decomposition tree. This structure characterizes the recursive decomposition process of a mesh into regions of interest and keypoints. The internal nodes contain regions of interest (which may be recursively decomposed) and the leaf nodes contain the keypoints to be matched. We also propose a hierarchical matching algorithm that performs in a level-wise manner. The matching process is guided by the similarity between regions in high levels of the tree, until reaching the keypoints stored in the leaves. This allows us to reduce the search space of correspondences, making also the matching process efficient. We evaluate the effectiveness of our approach using the SHREC’2010 robust correspondence benchmark. In addition, we show that our results outperform the state of the art.
2 0.89560831 261 iccv-2013-Markov Network-Based Unified Classifier for Face Identification
Author: Wonjun Hwang, Kyungshik Roh, Junmo Kim
Abstract: We propose a novel unifying framework using a Markov network to learn the relationship between multiple classifiers in face recognition. We assume that we have several complementary classifiers and assign observation nodes to the features of a query image and hidden nodes to the features of gallery images. We connect each hidden node to its corresponding observation node and to the hidden nodes of other neighboring classifiers. For each observation-hidden node pair, we collect a set of gallery candidates that are most similar to the observation instance, and the relationship between the hidden nodes is captured in terms of the similarity matrix between the collected gallery images. Posterior probabilities in the hidden nodes are computed by the belief-propagation algorithm. The novelty of the proposed framework is the method that takes into account the classifier dependency using the results of each neighboring classifier. We present extensive results on two different evaluation protocols, known and unknown image variation tests, using three different databases, which shows that the proposed framework always leads to good accuracy in face recognition.
3 0.8861382 297 iccv-2013-Online Motion Segmentation Using Dynamic Label Propagation
Author: Ali Elqursh, Ahmed Elgammal
Abstract: The vast majority of work on motion segmentation adopts the affine camera model due to its simplicity. Under the affine model, the motion segmentation problem becomes that of subspace separation. Due to this assumption, such methods are mainly offline and exhibit poor performance when the assumption is not satisfied. This is made evident in state-of-the-art methods that relax this assumption by using piecewise affine spaces and spectral clustering techniques to achieve better results. In this paper, we formulate the problem of motion segmentation as that of manifold separation. We then show how label propagation can be used in an online framework to achieve manifold separation. The performance of our framework is evaluated on a benchmark dataset and achieves competitive performance while being online.
4 0.8773281 121 iccv-2013-Discriminatively Trained Templates for 3D Object Detection: A Real Time Scalable Approach
Author: Reyes Rios-Cabrera, Tinne Tuytelaars
Abstract: In this paper we propose a new method for detecting multiple specific 3D objects in real time. We start from the template-based approach based on the LINE2D/LINEMOD representation introduced recently by Hinterstoisser et al., yet extend it in two ways. First, we propose to learn the templates in a discriminative fashion. We show that this can be done online during the collection of the example images, in just a few milliseconds, and has a big impact on the accuracy of the detector. Second, we propose a scheme based on cascades that speeds up detection. Since detection of an object is fast, new objects can be added with very low cost, making our approach scale well. In our experiments, we easily handle 10-30 3D objects at frame rates above 10fps using a single CPU core. We outperform the state-of-the-art both in terms of speed as well as in terms of accuracy, as validated on 3 different datasets. This holds both when using monocular color images (with LINE2D) and when using RGBD images (with LINEMOD). Moreover, wepropose a challenging new dataset made of12 objects, for future competing methods on monocular color images.
same-paper 5 0.86191416 256 iccv-2013-Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
Author: Marius Leordeanu, Andrei Zanfir, Cristian Sminchisescu
Abstract: Estimating a dense correspondence field between successive video frames, under large displacement, is important in many visual learning and recognition tasks. We propose a novel sparse-to-dense matching method for motion field estimation and occlusion detection. As an alternative to the current coarse-to-fine approaches from the optical flow literature, we start from the higher level of sparse matching with rich appearance and geometric constraints collected over extended neighborhoods, using an occlusion aware, locally affine model. Then, we move towards the simpler, but denser classic flow field model, with an interpolation procedure that offers a natural transition between the sparse and the dense correspondence fields. We experimentally demonstrate that our appearance features and our complex geometric constraintspermit the correct motion estimation even in difficult cases of large displacements and significant appearance changes. We also propose a novel classification method for occlusion detection that works in conjunction with the sparse-to-dense matching model. We validate our approach on the newly released Sintel dataset and obtain state-of-the-art results.
6 0.81836182 196 iccv-2013-Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation
7 0.81718236 361 iccv-2013-Robust Trajectory Clustering for Motion Segmentation
8 0.81676102 60 iccv-2013-Bayesian Robust Matrix Factorization for Image and Video Processing
9 0.81580675 351 iccv-2013-Restoring an Image Taken through a Window Covered with Dirt or Rain
10 0.81520468 89 iccv-2013-Constructing Adaptive Complex Cells for Robust Visual Tracking
11 0.81485832 204 iccv-2013-Human Attribute Recognition by Rich Appearance Dictionary
12 0.81468952 447 iccv-2013-Volumetric Semantic Segmentation Using Pyramid Context Features
13 0.81461024 128 iccv-2013-Dynamic Probabilistic Volumetric Models
14 0.81449443 396 iccv-2013-Space-Time Robust Representation for Action Recognition
15 0.81433445 57 iccv-2013-BOLD Features to Detect Texture-less Objects
16 0.81408155 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
17 0.81373715 238 iccv-2013-Learning Graphs to Match
18 0.81372672 127 iccv-2013-Dynamic Pooling for Complex Event Recognition
19 0.81350994 426 iccv-2013-Training Deformable Part Models with Decorrelated Features
20 0.81341141 151 iccv-2013-Exploiting Reflection Change for Automatic Reflection Removal