iccv iccv2013 iccv2013-300 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Tae Hyun Kim, Hee Seok Lee, Kyoung Mu Lee
Abstract: Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods.
Reference: text
sentIndex sentText sentNum sentScore
1 kr Abstract Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. [sent-6, score-0.894]
2 In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. [sent-7, score-1.064]
3 The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. [sent-8, score-0.565]
4 The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). [sent-9, score-0.341]
5 From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. [sent-10, score-0.973]
6 Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods. [sent-11, score-0.91]
7 Introduction Optical flow estimation is used to find the pixel-wise displacement field between two images. [sent-13, score-0.445]
8 The estimation of accurate flow vectors has become a key step in numerous vision applications, such as dense 3D reconstruction and segmentations of video or motion [23, 18, 11]. [sent-15, score-0.419]
9 However, optical flow estimation is difficult to solve because it is a highly ill-posed inverse imaging problem. [sent-16, score-0.672]
10 To address this problem, traditional approaches used the energy minimization formulation composed of both data term and regularization as follows: E = Edata(u) + λEreg(u), (1) where u = (u, v)T denotes the optical flow field between two input images. [sent-17, score-0.963]
11 Edata measures the data fidelity, Ereg enforces regularization of the flow field, and λ controls the ? [sent-18, score-0.531]
12 Our adaptive data fusion method consists of complementary data models that overcome the limitations of the single data model and provides the similar result to the ground truth flow. [sent-64, score-0.671]
13 Horn and Schunk [13] proposed the variational formulation for the first time in the optical flow estimation. [sent-66, score-0.679]
14 However, the data terms were restricted to the brightness constancy in which the brightness of the corresponding pixels does not change, which is not applicable under illumination changes or noise. [sent-71, score-0.751]
15 Some recent studies have focused on improving the optical flow constraints beyond the brightness constancy. [sent-72, score-0.787]
16 [6] averaged two data models with respect to the brightness constancy and gradient constancy, which as33333447 sumed that the intensity gradients of the corresponding points are same. [sent-74, score-0.622]
17 [21] compared the classical data model based on the brightness constancy with arbitrary non-convex data models and showed the superiority of the complex data models, such as block matching. [sent-77, score-0.802]
18 [24] also adopted the block matching and used truncated normalized cross correlation (NCC) as a pixel-wise data term to cope with the matching problems under illumination or exposure changes and to remove the ambiguity in the occluded regions. [sent-79, score-0.334]
19 Although many approaches have tried to improve the optical flow constraints and designed robust data models, the previously proposed data models still remained inadequate in practical situations because their inherent weaknesses. [sent-80, score-0.852]
20 For example, the data models based on the gradient constancy or block matching such as the sum of absolute difference (SAD) are not valid when a geometrical transformation significantly changes the appearance in the target im- age. [sent-81, score-0.596]
21 However, these models could complement each other, and thus, the problem of designing a locally adaptive data term is of the greatest importance. [sent-82, score-0.382]
22 selected the best data model for each pixel in the reference image from two data models which are the brightness and gradient constancy, and assumed that the weight variable is binary. [sent-86, score-0.594]
23 Because of the small noise and uncertainty of each data model even if all assumptions of data models are not violated, the continuous weight variables must be allowed rather than the binary variables, to suppress the noise by averaging. [sent-89, score-0.473]
24 Therefore, we propose a more general and unified variational framework that considers both the data fidelity and regularization to determine the locally varying continuous weight variables and thus, can be applicable to other vision problems. [sent-91, score-0.653]
25 Figure 1 shows that our new optical flow estimation model based on the generalized data fusion framework significantly improves the accuracy. [sent-92, score-0.877]
26 In addition, our model includes a novel data discriminability term, which defines the goodness of each data model to reduce the ambiguity in the homogeneous region. [sent-93, score-0.622]
27 The proposed minimization procedure is very efficient and practical; thus, it can handle many data models, and the complexity increases linearly as the number of used data models increases. [sent-94, score-0.303]
28 We also provide a method to select the complementary data models to be used in the optical flow estimation. [sent-95, score-0.91]
29 Therefore, this study also proposes the method learning complementary data models where the number of selected models is made as few as possible based on the minimum description length (MDL) [12] and then fuses the chosen models. [sent-98, score-0.51]
30 Finally, experimental results verify our claims and show that the proposed approach outperforms the other conventional approaches and achieves very satisfactory performance in the Middlebury optical flow benchmark site. [sent-99, score-0.646]
31 Optical Flow Estimation Model Using Locally Adaptive Data Fusion Most traditional optical flow estimation methods are based on the variational framework and it is easy to implement and parallelize on modern GPUs. [sent-101, score-0.742]
32 Therefore, our proposed optical flow estimation method is also based on the variational framework with a robust data term and regu- larizer. [sent-102, score-0.868]
33 Thus, designing a locally (pixel-wise) adaptive data term is desirable by the fusion of complementary data models while excluding the invalid data models. [sent-104, score-0.815]
34 In addition, data discriminability which indicates the goodness of each data model to reduce the ambiguity in the textureless region and smoothness prior on the weight variable, are also required. [sent-105, score-0.707]
35 Therefore, we can generalize (1) by employing the adaptive data fusion model as, E = Edata(u, w) + ηEdscr(u, w) + μEreg (w) + λEreg (u) . [sent-106, score-0.285]
36 sE ada staet(u o,f fw M) measures tahriea bdleatsa, fidelity coupled with the flow fields and weight variables, Edscr (u, w) measures the discriminability of each data model, and Ereg (w) and Ereg (u) enforces the regularization of the weights, respectively. [sent-111, score-1.057]
37 The constant η is used to define the importance of discriminability term and μ controls the influence of regularization of the weight variables. [sent-112, score-0.571]
38 In the following sections, more details of the major factors for the adaptive data fusion, which are the data fidelity, the data discriminability, and regularization, are provided. [sent-113, score-0.335]
39 Data Fidelity This study proposes an optical flow estimation model that combines the conventional but complementary optical flow constraints learnt by the method presented in Section 4. [sent-116, score-1.521]
40 33333458 One of the most important factors for the adaptive data fusion is data fidelity. [sent-117, score-0.37]
41 For example, a data model with respect to the brightness constancy gives unreliable data cost at the true matching under illumination changes, shades or noise; thus, the energy minimization procedure can be overfitted and can provide undesirable result to avoid high cost. [sent-118, score-0.839]
42 However, other data models such as the gradient constancy or NCC can provide lower cost where the brightness constancy is invalid. [sent-119, score-0.918]
43 Therefore, we can obtain the desired result and avoid over-fitting by favoring more reliable data models, which provide better data fidelity if given models are normalized to have similar costs in the true matching where the assumptions of the models are valid. [sent-121, score-0.632]
44 weight variable, wl ∈ w, has constraints, wl (x) ≥ 0 and ? [sent-132, score-0.575]
45 Two data models are considered at the corresponding points between the reference and target images. [sent-140, score-0.261]
46 Therefore, not only the data fidelity but also the data discriminability of the data model should be considered in our adaptive data fusion. [sent-147, score-0.851]
47 The notion of the discriminability term is similar to the concept of good feature to track in [20] and the Harris corner detector, which measures how reliable a point is under a given supporting region for feature matching. [sent-148, score-0.413]
48 Similar to the Harris corner detector, the discriminability of each data model can be measured by the smallest eigenvalue of the auto-correlation matrix corresponding to the auto-correlation function defined by cl(x,u0) = ? [sent-209, score-0.409]
49 ∈W where the given optical flow u0 can be obtained from the initial state of each level in the coarse to fine approach or from the previous result in the iterative optimization procedure. [sent-211, score-0.609]
50 Then the data discriminability term in (2) can be represented as = ? [sent-214, score-0.45]
51 =l Because the data model with a large discriminability yields a large el (x, u0), this is added to the costs of the other data models and prevents the other models from gaining more weights. [sent-225, score-0.719]
52 Regularization As optical flow estimation is a highly ill-posed problem, regularization enforcing the smoothness of variables is necessary to obtain a reliable solution. [sent-228, score-0.938]
53 In our energy model, two primal variables are the set of the weight variables w and the flow fields u, and the details of regularization for each variable are described in the following sections. [sent-229, score-0.785]
54 1 Regularization on w Regularizing the weight variables is also an important factor for adaptive data fusion. [sent-232, score-0.32]
55 In particular, if noise exists in the true matching, then favoring a model which gives the better data fidelity only may not be the best solution, but averaging or weighted summing of all the data models could give more reliable results. [sent-233, score-0.48]
56 Therefore, we allow the continuous, but not the binary, weight field to get a solution from the weighted average of the costs, and incorporate the smoothness prior on the weight variables to avoid assigning large weight to unreliable models. [sent-236, score-0.426]
57 We design the regularization of the weight variables to change smoothly but to have sparse discontinuities. [sent-237, score-0.254]
58 2 Regularization on u In general, conventional optical flow estimation models assume that the flow vectors vary smoothly and have the sparse discontinuities in the edges of reference image. [sent-247, score-1.194]
59 Therefore, the edge map [18, 25, 23] is coupled to the total variation regularizer which allows discontinuities in the flow fields. [sent-248, score-0.353]
60 Learning of Complementary Data Models Based on MDL Using many complementary data models could give better results, however, this process is inefficient because of ? [sent-254, score-0.332]
61 Therefore, we study the nature of the data models to be used in the data fusion on the Middlebury training datasets where the ground truth of the motion fields is known. [sent-325, score-0.51]
62 By normalization, fair comparison of the costs among the data models in the true matching is possible, and we can presume that the matching cost is unreliable when the cost is over one (σ = 1), otherwise it is reliable. [sent-329, score-0.433]
63 The candidate data models consists of total 3 1 models such as the brightness, gradient and block matching for the gray and RGB color channels. [sent-334, score-0.346]
64 ) For a set of data models to be complementary, at least one data model should give reliable cost where the others could not. [sent-337, score-0.369]
65 Therefore we should minimize the sum of error function with complementary data models, and it is given by ? [sent-338, score-0.228]
66 lL=1 33334470 however, with the aid of complementary data models, ? [sent-348, score-0.228]
67 rm in (12) is designed to choose complementary set of the data models and the second term is used to minimize the redundancy of the chosen data models based on MDL. [sent-365, score-0.581]
68 Ifγ is given, the function F can be minimized by [9], and our M data models to be used in the optical flow estimation can be learned. [sent-366, score-0.83]
69 with chosen models is allowed, thus, we can determine the preferred set of data models from the curve shown in Figure 4. [sent-368, score-0.304]
70 To be robust against geometrical changes, the three data models of the red, blue and gray channels are based on the brightness constancy in [26]. [sent-371, score-0.585]
71 In addition, to be robust against illumination changes, the four models of the green and blue channels are based on the vertical and horizontal gradient constancy in [25]. [sent-372, score-0.39]
72 Optimization The proposed optical flow estimation model introduced in the previous section includes the regularization, weighted sum of the multiple data models and the data discriminability, and the final objective function of this study is as follows: ? [sent-379, score-0.975]
73 ion and discriminability terms are convex, if all the data models are convex then (13) can be a jointly convex problem [10]. [sent-398, score-0.596]
74 ,M} 12:: wl (=x) { 1←, wl M(x)} − (Pl wl (x) − 1)/|T| , if l ∈ T 3: T ←(x) T ← − w {l(}x,) i−f wlP P(x) < 0x 34:: wl ←(x) T← − −0 ,{ }if, ,l /∈f wTP 5: Repeat steps 2-4 until PlM=1 wl (x) = 1, for all x 4. [sent-430, score-1.245]
75 It allows the computation of the large displacement optical flow [21] and it also makes possible to avoid staying in the local minima. [sent-433, score-0.638]
76 ction and Postprocessing The data models used in our optical flow estimation have weakness in occlusion, thus occlusion handling in the postprocessing is beneficial. [sent-443, score-0.94]
77 Generally, cross checking of the optical flows is effective however, it doubles the computational cost [19]. [sent-444, score-0.368]
78 Because the estimated optical flow could vhaalvuee some errors a Bnedc a rounding omfaft technique liso wus ceodu ltod count the number on the discrete grid, we regard the pixels as ambiguous when N(x+u) = 2 and o(x) is equal to 0. [sent-448, score-0.609]
79 To fill the occluded pixels and remove the artifacts in the homogeneous regions, we apply the joint bilateral filter with the occlusion states and the similarity of color and optical flow fields that follows [19]. [sent-451, score-0.752]
80 3) Propagate variables to the next pyramid level if exists Repeat steps 3-8 from coarse to fine pyramid level The proposed optical flow estimation model is based on the quadratic approximations of the original data model to 33334492 Figure? [sent-462, score-0.835]
81 The comparisons are made using the methods with single data term, which has the best score, mean of data models, and the proposed data fusion with and without the discriminability term. [sent-476, score-0.699]
82 Furthermore we estimate five affine motions from the flow fields similar to RANSAC, and update the initial flow field when the affine motion yields lower energy, Edata (u, w), than the propagated motion field from the coarser level. [sent-499, score-0.832]
83 Three image warps that use intermediate flow vectors are performed in a single pyramid level as in [22]. [sent-502, score-0.317]
84 Furthermore the proposed optical flow estimation model is implemented in C++ on the GPU with CUDA, and the computation time is significantly reduced using paral- lelization. [sent-508, score-0.672]
85 Experimental Results The end point error (EPE) and angular error (AE) of the flow estimation results are measured using the various data models and shown in Figure 5. [sent-511, score-0.538]
86 In the evaluation, the single data model giving the best result, simple mean of data models and their fusion by the proposed method with and without the data discriminability are compared. [sent-513, score-0.772]
87 The results suggest that the proposed fusion method with discriminability outperforms others and provides significantly better results. [sent-514, score-0.444]
88 This result shows that relying only on the regularization of u does not give a good accuracy compared with taking into account the smoothness of the weight variables w. [sent-534, score-0.326]
89 Fig- ure 8 shows the pixel-wise weights, and the weights shown in Figures 8(a)-(c) are related with the gradient constancy, and the weight shown in Figure 8(d) is obtained from the brightness constancy and that in Figure 8(e) represents the weight of SAD. [sent-535, score-0.618]
90 The arrows indicate the shaded regions in the reference image, and the data models related with the gradient constancy gain more weights than the other models, as expected, and the data model shown in Figure 8(c) has less weight where the data cost is high in Figure 3(a). [sent-536, score-0.797]
91 Our results are compared with the state-of-the-art optical flow estimation methods based on the variational framework [25, 22, 14, 3, 8, 16, 24], and the AEPE of each method is shown in Figure 6. [sent-537, score-0.742]
92 Discussion and Conclusion This study has presented a novel optical flow estimation method that fuses various data models. [sent-543, score-0.852]
93 By providing the locally adaptive data term, the limitation of a single data model can be overcome. [sent-544, score-0.312]
94 This study also provided an efficient and practical solution, and the proposed optical flow estimation model showed competitive results compared with the state-of-the art methods. [sent-545, score-0.732]
95 In addition, a method for learning the set of data models based on MDL was presented which provided the data models to be used in our optical flow estimation. [sent-546, score-0.925]
96 The from the single data model with brightness constancy. [sent-582, score-0.263]
97 A framework for the robust estimation of optical flow. [sent-613, score-0.355]
98 High accuracy optical flow estimation based on a theory for warping. [sent-628, score-0.672]
99 Joint motion estimation and segmentation of complex scenes with label costs and occlusion modeling. [sent-732, score-0.243]
100 A duality based approach for realtime TVL1 optical flow. [sent-750, score-0.329]
wordName wordTfidf (topN-words)
[('discriminability', 0.324), ('flow', 0.317), ('optical', 0.292), ('sad', 0.258), ('constancy', 0.249), ('wl', 0.249), ('brightness', 0.178), ('ereg', 0.173), ('complementary', 0.143), ('fusion', 0.12), ('mdl', 0.115), ('proceedings', 0.115), ('edata', 0.114), ('middlebury', 0.114), ('aepe', 0.112), ('fidelity', 0.107), ('regularization', 0.099), ('data', 0.085), ('conference', 0.085), ('edscr', 0.084), ('adaptive', 0.08), ('costs', 0.079), ('variables', 0.078), ('weight', 0.077), ('ql', 0.076), ('continuous', 0.075), ('models', 0.073), ('variational', 0.07), ('aae', 0.069), ('simplex', 0.066), ('estimation', 0.063), ('occlusion', 0.062), ('locally', 0.062), ('minimization', 0.06), ('study', 0.06), ('reference', 0.059), ('goodness', 0.057), ('convex', 0.057), ('werlberger', 0.055), ('primal', 0.055), ('ir', 0.052), ('favoring', 0.051), ('eight', 0.05), ('forensic', 0.05), ('snu', 0.05), ('fields', 0.048), ('postprocessing', 0.048), ('reliable', 0.048), ('cost', 0.047), ('block', 0.047), ('seoul', 0.046), ('ifth', 0.046), ('international', 0.044), ('target', 0.044), ('pock', 0.043), ('cker', 0.043), ('steinbr', 0.043), ('epe', 0.043), ('ll', 0.043), ('chosen', 0.041), ('term', 0.041), ('smoothness', 0.041), ('fl', 0.041), ('designing', 0.041), ('redundancy', 0.04), ('unreliable', 0.04), ('ncc', 0.04), ('motion', 0.039), ('cl', 0.038), ('bruhn', 0.038), ('ambiguity', 0.038), ('kr', 0.038), ('conventional', 0.037), ('duality', 0.037), ('gradient', 0.037), ('field', 0.036), ('discontinuities', 0.036), ('pattern', 0.035), ('optic', 0.035), ('fuses', 0.035), ('horn', 0.034), ('harris', 0.033), ('homogeneous', 0.033), ('energy', 0.033), ('uniqueness', 0.032), ('regularizing', 0.032), ('ieee', 0.032), ('curve', 0.032), ('matching', 0.031), ('illumination', 0.031), ('give', 0.031), ('unit', 0.031), ('horizontally', 0.031), ('vertically', 0.031), ('changes', 0.03), ('controls', 0.03), ('checking', 0.029), ('displacement', 0.029), ('zach', 0.029), ('dual', 0.028)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999952 300 iccv-2013-Optical Flow via Locally Adaptive Fusion of Complementary Data Costs
Author: Tae Hyun Kim, Hee Seok Lee, Kyoung Mu Lee
Abstract: Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods.
2 0.31198281 105 iccv-2013-DeepFlow: Large Displacement Optical Flow with Deep Matching
Author: Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Abstract: Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox & Malik [6], our approach, termed DeepFlow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable assetfor integration into an energy minimizationframework for optical flow estimation. DeepFlow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset [8].
3 0.30311975 317 iccv-2013-Piecewise Rigid Scene Flow
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
4 0.28198019 12 iccv-2013-A General Dense Image Matching Framework Combining Direct and Feature-Based Costs
Author: Jim Braux-Zin, Romain Dupont, Adrien Bartoli
Abstract: Dense motion field estimation (typically Romain Dupont1 romain . dupont @ cea . fr Adrien Bartoli2 adrien . bart o l @ gmai l com i . 2 ISIT, Universit e´ d’Auvergne/CNRS, France sions are explicitly modeled [32, 13]. Coarse-to-fine warping improves global convergence by making the assumption that optical flow, the motion of smaller structures is similar to the motion of stereo disparity and surface registration) is a key computer vision problem. Many solutions have been proposed to compute small or large displacements, narrow or wide baseline stereo disparity, but a unified methodology is still lacking. We here introduce a general framework that robustly combines direct and feature-based matching. The feature-based cost is built around a novel robust distance function that handles keypoints and “weak” features such as segments. It allows us to use putative feature matches which may contain mismatches to guide dense motion estimation out of local minima. Our framework uses a robust direct data term (AD-Census). It is implemented with a powerful second order Total Generalized Variation regularization with external and self-occlusion reasoning. Our framework achieves state of the art performance in several cases (standard optical flow benchmarks, wide-baseline stereo and non-rigid surface registration). Our framework has a modular design that customizes to specific application needs.
5 0.21607719 256 iccv-2013-Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
Author: Marius Leordeanu, Andrei Zanfir, Cristian Sminchisescu
Abstract: Estimating a dense correspondence field between successive video frames, under large displacement, is important in many visual learning and recognition tasks. We propose a novel sparse-to-dense matching method for motion field estimation and occlusion detection. As an alternative to the current coarse-to-fine approaches from the optical flow literature, we start from the higher level of sparse matching with rich appearance and geometric constraints collected over extended neighborhoods, using an occlusion aware, locally affine model. Then, we move towards the simpler, but denser classic flow field model, with an interpolation procedure that offers a natural transition between the sparse and the dense correspondence fields. We experimentally demonstrate that our appearance features and our complex geometric constraintspermit the correct motion estimation even in difficult cases of large displacements and significant appearance changes. We also propose a novel classification method for occlusion detection that works in conjunction with the sparse-to-dense matching model. We validate our approach on the newly released Sintel dataset and obtain state-of-the-art results.
6 0.17866465 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
7 0.14478788 39 iccv-2013-Action Recognition with Improved Trajectories
8 0.14434275 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
9 0.1422928 143 iccv-2013-Estimating Human Pose with Flowing Puppets
10 0.12693265 5 iccv-2013-A Color Constancy Model with Double-Opponency Mechanisms
11 0.12605864 82 iccv-2013-Compensating for Motion during Direct-Global Separation
12 0.1249155 263 iccv-2013-Measuring Flow Complexity in Videos
13 0.12319421 420 iccv-2013-Topology-Constrained Layered Tracking with Latent Flow
14 0.1096537 129 iccv-2013-Dynamic Scene Deblurring
15 0.10909396 421 iccv-2013-Total Variation Regularization for Functions with Values in a Manifold
16 0.10699767 219 iccv-2013-Internet Based Morphable Model
17 0.10563113 424 iccv-2013-Tracking Revisited Using RGBD Camera: Unified Benchmark and Baselines
18 0.098193534 304 iccv-2013-PM-Huber: PatchMatch with Huber Regularization for Stereo Matching
19 0.097175099 58 iccv-2013-Bayesian 3D Tracking from Monocular Video
20 0.087870695 444 iccv-2013-Viewing Real-World Faces in 3D
topicId topicWeight
[(0, 0.219), (1, -0.135), (2, -0.017), (3, 0.075), (4, -0.017), (5, 0.047), (6, -0.027), (7, 0.026), (8, 0.076), (9, -0.008), (10, -0.064), (11, 0.013), (12, 0.273), (13, -0.036), (14, 0.02), (15, -0.035), (16, -0.11), (17, 0.032), (18, 0.202), (19, 0.067), (20, 0.155), (21, -0.004), (22, 0.042), (23, -0.112), (24, 0.079), (25, -0.108), (26, 0.154), (27, -0.008), (28, 0.049), (29, -0.046), (30, -0.039), (31, -0.039), (32, -0.026), (33, 0.061), (34, -0.116), (35, -0.015), (36, -0.026), (37, -0.01), (38, -0.063), (39, 0.04), (40, 0.004), (41, -0.137), (42, 0.044), (43, 0.022), (44, 0.004), (45, -0.069), (46, -0.038), (47, -0.051), (48, -0.031), (49, -0.054)]
simIndex simValue paperId paperTitle
same-paper 1 0.96853048 300 iccv-2013-Optical Flow via Locally Adaptive Fusion of Complementary Data Costs
Author: Tae Hyun Kim, Hee Seok Lee, Kyoung Mu Lee
Abstract: Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods.
2 0.86326164 105 iccv-2013-DeepFlow: Large Displacement Optical Flow with Deep Matching
Author: Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Abstract: Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox & Malik [6], our approach, termed DeepFlow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable assetfor integration into an energy minimizationframework for optical flow estimation. DeepFlow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset [8].
3 0.85040283 12 iccv-2013-A General Dense Image Matching Framework Combining Direct and Feature-Based Costs
Author: Jim Braux-Zin, Romain Dupont, Adrien Bartoli
Abstract: Dense motion field estimation (typically Romain Dupont1 romain . dupont @ cea . fr Adrien Bartoli2 adrien . bart o l @ gmai l com i . 2 ISIT, Universit e´ d’Auvergne/CNRS, France sions are explicitly modeled [32, 13]. Coarse-to-fine warping improves global convergence by making the assumption that optical flow, the motion of smaller structures is similar to the motion of stereo disparity and surface registration) is a key computer vision problem. Many solutions have been proposed to compute small or large displacements, narrow or wide baseline stereo disparity, but a unified methodology is still lacking. We here introduce a general framework that robustly combines direct and feature-based matching. The feature-based cost is built around a novel robust distance function that handles keypoints and “weak” features such as segments. It allows us to use putative feature matches which may contain mismatches to guide dense motion estimation out of local minima. Our framework uses a robust direct data term (AD-Census). It is implemented with a powerful second order Total Generalized Variation regularization with external and self-occlusion reasoning. Our framework achieves state of the art performance in several cases (standard optical flow benchmarks, wide-baseline stereo and non-rigid surface registration). Our framework has a modular design that customizes to specific application needs.
4 0.82032734 317 iccv-2013-Piecewise Rigid Scene Flow
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
5 0.81567639 256 iccv-2013-Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
Author: Marius Leordeanu, Andrei Zanfir, Cristian Sminchisescu
Abstract: Estimating a dense correspondence field between successive video frames, under large displacement, is important in many visual learning and recognition tasks. We propose a novel sparse-to-dense matching method for motion field estimation and occlusion detection. As an alternative to the current coarse-to-fine approaches from the optical flow literature, we start from the higher level of sparse matching with rich appearance and geometric constraints collected over extended neighborhoods, using an occlusion aware, locally affine model. Then, we move towards the simpler, but denser classic flow field model, with an interpolation procedure that offers a natural transition between the sparse and the dense correspondence fields. We experimentally demonstrate that our appearance features and our complex geometric constraintspermit the correct motion estimation even in difficult cases of large displacements and significant appearance changes. We also propose a novel classification method for occlusion detection that works in conjunction with the sparse-to-dense matching model. We validate our approach on the newly released Sintel dataset and obtain state-of-the-art results.
6 0.68343884 430 iccv-2013-Two-Point Gait: Decoupling Gait from Body Shape
7 0.67736077 301 iccv-2013-Optimal Orthogonal Basis and Image Assimilation: Motion Modeling
8 0.65776592 420 iccv-2013-Topology-Constrained Layered Tracking with Latent Flow
9 0.60360104 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
10 0.6019246 263 iccv-2013-Measuring Flow Complexity in Videos
11 0.57124501 171 iccv-2013-Fix Structured Learning of 2013 ICCV paper k2opt.pdf
12 0.55463326 82 iccv-2013-Compensating for Motion during Direct-Global Separation
13 0.54365975 143 iccv-2013-Estimating Human Pose with Flowing Puppets
14 0.52682948 270 iccv-2013-Modeling Self-Occlusions in Dynamic Shape and Appearance Tracking
15 0.51178771 39 iccv-2013-Action Recognition with Improved Trajectories
16 0.51047581 358 iccv-2013-Robust Non-parametric Data Fitting for Correspondence Modeling
17 0.50886649 352 iccv-2013-Revisiting Example Dependent Cost-Sensitive Learning with Decision Trees
18 0.4913277 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
19 0.48018897 275 iccv-2013-Motion-Aware KNN Laplacian for Video Matting
20 0.47830093 164 iccv-2013-Fibonacci Exposure Bracketing for High Dynamic Range Imaging
topicId topicWeight
[(2, 0.055), (7, 0.022), (26, 0.08), (31, 0.054), (40, 0.026), (42, 0.093), (48, 0.028), (64, 0.06), (73, 0.084), (74, 0.189), (89, 0.206)]
simIndex simValue paperId paperTitle
same-paper 1 0.86163574 300 iccv-2013-Optical Flow via Locally Adaptive Fusion of Complementary Data Costs
Author: Tae Hyun Kim, Hee Seok Lee, Kyoung Mu Lee
Abstract: Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods.
2 0.85563552 82 iccv-2013-Compensating for Motion during Direct-Global Separation
Author: Supreeth Achar, Stephen T. Nuske, Srinivasa G. Narasimhan
Abstract: Separating the direct and global components of radiance can aid shape recovery algorithms and can provide useful information about materials in a scene. Practical methods for finding the direct and global components use multiple images captured under varying illumination patterns and require the scene, light source and camera to remain stationary during the image acquisition process. In this paper, we develop a motion compensation method that relaxes this condition and allows direct-global separation to beperformed on video sequences of dynamic scenes captured by moving projector-camera systems. Key to our method is being able to register frames in a video sequence to each other in the presence of time varying, high frequency active illumination patterns. We compare our motion compensated method to alternatives such as single shot separation and frame interleaving as well as ground truth. We present results on challenging video sequences that include various types of motions and deformations in scenes that contain complex materials like fabric, skin, leaves and wax.
Author: Masakazu Iwamura, Tomokazu Sato, Koichi Kise
Abstract: Approximate nearest neighbor search (ANNS) is a basic and important technique used in many tasks such as object recognition. It involves two processes: selecting nearest neighbor candidates and performing a brute-force search of these candidates. Only the former though has scope for improvement. In most existing methods, it approximates the space by quantization. It then calculates all the distances between the query and all the quantized values (e.g., clusters or bit sequences), and selects a fixed number of candidates close to the query. The performance of the method is evaluated based on accuracy as a function of the number of candidates. This evaluation seems rational but poses a serious problem; it ignores the computational cost of the process of selection. In this paper, we propose a new ANNS method that takes into account costs in the selection process. Whereas existing methods employ computationally expensive techniques such as comparative sort and heap, the proposed method does not. This realizes a significantly more efficient search. We have succeeded in reducing computation times by one-third compared with the state-of-the- art on an experiment using 100 million SIFT features.
4 0.83174181 122 iccv-2013-Distributed Low-Rank Subspace Segmentation
Author: Ameet Talwalkar, Lester Mackey, Yadong Mu, Shih-Fu Chang, Michael I. Jordan
Abstract: Vision problems ranging from image clustering to motion segmentation to semi-supervised learning can naturally be framed as subspace segmentation problems, in which one aims to recover multiple low-dimensional subspaces from noisy and corrupted input data. Low-Rank Representation (LRR), a convex formulation of the subspace segmentation problem, is provably and empirically accurate on small problems but does not scale to the massive sizes of modern vision datasets. Moreover, past work aimed at scaling up low-rank matrix factorization is not applicable to LRR given its non-decomposable constraints. In this work, we propose a novel divide-and-conquer algorithm for large-scale subspace segmentation that can cope with LRR ’s non-decomposable constraints and maintains LRR ’s strong recovery guarantees. This has immediate implications for the scalability of subspace segmentation, which we demonstrate on a benchmark face recognition dataset and in simulations. We then introduce novel applications of LRR-based subspace segmentation to large-scale semisupervised learning for multimedia event detection, concept detection, and image tagging. In each case, we obtain stateof-the-art results and order-of-magnitude speed ups.
5 0.823349 426 iccv-2013-Training Deformable Part Models with Decorrelated Features
Author: Ross Girshick, Jitendra Malik
Abstract: In this paper, we show how to train a deformable part model (DPM) fast—typically in less than 20 minutes, or four times faster than the current fastest method—while maintaining high average precision on the PASCAL VOC datasets. At the core of our approach is “latent LDA,” a novel generalization of linear discriminant analysis for learning latent variable models. Unlike latent SVM, latent LDA uses efficient closed-form updates and does not require an expensive search for hard negative examples. Our approach also acts as a springboard for a detailed experimental study of DPM training. We isolate and quantify the impact of key training factors for the first time (e.g., How important are discriminative SVM filters? How important is joint parameter estimation? How many negative images are needed for training?). Our findings yield useful insights for researchers working with Markov random fields and partbased models, and have practical implications for speeding up tasks such as model selection.
6 0.81123459 239 iccv-2013-Learning Hash Codes with Listwise Supervision
7 0.7977702 60 iccv-2013-Bayesian Robust Matrix Factorization for Image and Video Processing
8 0.79694176 196 iccv-2013-Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation
9 0.7948246 151 iccv-2013-Exploiting Reflection Change for Automatic Reflection Removal
10 0.79311502 376 iccv-2013-Scene Text Localization and Recognition with Oriented Stroke Detection
11 0.79217362 351 iccv-2013-Restoring an Image Taken through a Window Covered with Dirt or Rain
12 0.7916283 270 iccv-2013-Modeling Self-Occlusions in Dynamic Shape and Appearance Tracking
13 0.79113477 220 iccv-2013-Joint Deep Learning for Pedestrian Detection
14 0.79102874 89 iccv-2013-Constructing Adaptive Complex Cells for Robust Visual Tracking
15 0.79078603 358 iccv-2013-Robust Non-parametric Data Fitting for Correspondence Modeling
16 0.79074901 58 iccv-2013-Bayesian 3D Tracking from Monocular Video
17 0.79011691 420 iccv-2013-Topology-Constrained Layered Tracking with Latent Flow
18 0.78990901 433 iccv-2013-Understanding High-Level Semantics by Modeling Traffic Patterns
19 0.78820503 223 iccv-2013-Joint Noise Level Estimation from Personal Photo Collections
20 0.78755224 349 iccv-2013-Regionlets for Generic Object Detection