iccv iccv2013 iccv2013-317 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
Reference: text
sentIndex sentText sentNum sentScore
1 Piecewise Rigid Scene Flow Christoph Vogel Konrad Schindler Photogrammetry & Remote Sensing, ETH Zurich Abstract Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. [sent-1, score-1.707]
2 Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. [sent-3, score-0.96]
3 Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. [sent-5, score-0.585]
4 Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. [sent-6, score-0.504]
5 We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques. [sent-8, score-1.584]
6 Introduction Scene flow estimation is the task of estimating dense 3D surface shape as well as a dense 3D motion field from two (or more) views of a scene taken at two (or more) time steps [20]. [sent-10, score-1.017]
7 The 3D scene flow generalizes two classical problems of computer vision, dense stereo matching and dense optical flow estimation. [sent-12, score-1.647]
8 Yet, despite significant progress in both stereo [4, 9, 26] and 2D optical flow estimation [5, 16, 17], existing 3D scene flow techniques [e. [sent-13, score-1.613]
9 Perhaps surprisingly, the additional information available in stereo motion sequences has not been leveraged to the extent that 3D scene flow outperforms dedicated stereo or 2D optical flow techniques at their respective task. [sent-16, score-2.164]
10 Much like stereo or 2D motion estimation, scene flow Stefan Roth Department of Computer Science, TU Darmstadt estimation is ill-posed due to the 3D equivalent of the aperture problem, and thus requires prior assumptions on geom- etry and motion. [sent-17, score-1.165]
11 We posit that existing 3D scene flow techniques have been limited by the underlying representation, and propose to model the scene as a collection of planar regions, each undergoing a rigid motion. [sent-24, score-0.978]
12 Following prior work in stereo [4], we argue that most scenes of interest consist of regions with a consistent motion pattern, into which they can be segmented at least implicitly during scene flow estimation. [sent-25, score-1.125]
13 Since a larger support is required to fit a plane and its rigid motion (9 unknowns) reliably, we base the initial estimation not on individual pixels, but on a superpixel segmentation of the reference image. [sent-26, score-0.647]
14 Scene flow estimation is then cast as a labeling problem, which assigns each pixel to a segment and each segment to a rigidly moving 3D plane. [sent-28, score-1.297]
15 Although the superpixels significantly simplify and stabilize the inference, they lead to inaccuracies at flow boundaries, since the initial segmentation does not take into account depth or motion discontinuities. [sent-29, score-0.844]
16 We also show how to explicitly include occlusion reasoning both at the segment and pixel level. [sent-31, score-0.486]
17 In experiments on challenging, realistic data the proposed approach substantially outperforms three state-of-the-art 3D scene flow methods. [sent-33, score-0.623]
18 (right) Processing steps and final result of piecewise rigid scene flow estimation. [sent-35, score-0.855]
19 Estimated depth, the lateral 3D motion component, and the re-projected 2D flow are shown. [sent-36, score-0.675]
20 outperforms recent dedicated stereo and optical flow algorithms in challenging settings on their respective task. [sent-38, score-1.039]
21 Estimation proceeds in two independent steps: First, 2D optical flow fields are estimated for all views (without requiring that they must be projections of the same 3D flow). [sent-42, score-0.694]
22 Stereo disparity is precomputed for each time step; then the optical flow for a reference view and the disparity differences for the other view are estimated. [sent-46, score-1.019]
23 [13] integrate a Kalman filter into this approach to yield smooth flow fields over multiple frames. [sent-48, score-0.509]
24 Huguet and Devernay [10] were possibly the first to estimate geometry and flow in an integrated manner with a variational formulation. [sent-50, score-0.593]
25 [3] parameterize the scene flow by depth and a 3D motion vector w. [sent-52, score-0.885]
26 a reference view, and estimate all parameters jointly with a 3D extension of the widely used optical flow method of Brox et al. [sent-55, score-0.743]
27 The local rigidity assumption, which for sparse motion estimation dates back to at least Adiv [1], has also been used in 3D motion capture with explicit surface models [e. [sent-59, score-0.452]
28 Also related is the optical flow approach of Nir et al. [sent-62, score-0.654]
29 [12], in which the flow field is (over-) parameterized by explicitly searching for rigid motion parameters, and then encouraging their smoothness. [sent-63, score-0.841]
30 In the presence of a dominant rigid motion (“background motion”) they alternatingly estimate both the relative camera pose and the scene flow. [sent-66, score-0.492]
31 Common to these previous approaches to 3D scene flow is that they penalize deviations from spatial smoothness, typically in a robust way. [sent-67, score-0.702]
32 In the context of stereo disparity and optical flow, explicit modeling of discontinuities by means of segmentation or layer-based formulations has a long history [23] and has recently gained renewed attention: Bleyer et al. [sent-68, score-0.637]
33 [4] estimate disparity by assuming the scene to be segmented into planar superpixels and parameterizing their geometry. [sent-69, score-0.432]
34 optical flow that enforces epipolar motion as hard constraint [27]. [sent-74, score-0.922]
35 [17] compute optical flow by parameterizing the motion per segment with 2D affine transformations, and also perform occlusion handling. [sent-78, score-1.263]
36 Discrete optimization based on fusion of proposals has been applied before to 2D optical flow estimation by Lempitsky et al. [sent-80, score-0.753]
37 Piecewise Rigid Model for 3D Scene Flow In contrast to typical approaches to 3D scene flow, our novel model parameterizes the scene as a collection of piecewise planar regions, each of which moves rigidly over time. [sent-84, score-0.625]
38 To that end we define an energy function that assigns each pixel to a segment and each segment to the 3D geometry and motion of a plane. [sent-86, score-0.831]
39 This allows us to estimate the 3D scene flow and depth for every pixel of a reference view. [sent-87, score-0.856]
40 Parameterizing the scene with moving planes also conveniently allows the pixel locations assigned to each plane to be transformed easily between images or mapped into 3D space using the corresponding homographies. [sent-106, score-0.63]
41 Over-segmentation is deliberately accepted, both to ensure correct depth and flow estimation for nonplanar and articulated objects, and to maximize boundary recall, even at the cost of spurious segment boundaries that do not correspond to depth or motion discontinuities. [sent-110, score-1.127]
42 Model overview Our aim is to estimate depth and a 3D scene flow vector for each pixel ofthe reference frame Il0. [sent-113, score-0.856]
43 For now we assume that we have a finite set of possible rigidly moving proposal planes Π = {πj } in 3D. [sent-114, score-0.564]
44 , W Wweh itchhen assigns efoacrh tw pixel p i∈n gIsl0: tAo a segment s ∈ I S→; an Sd, a mapping gPn : aSc →h p iΠxe tlo p assign teoac ah segment sto ∈ a rigidly moving n3Dg plane π →∈ ΠΠ . [sent-116, score-0.871]
45 t We thus formulate scene flow estimation as minimizing E(P, S) = ED(P, S) + λER(P, S) + μES(S). [sent-117, score-0.663]
46 This amounts to a total of 4 constraints per pixel (two stereo constraints at time steps 0 and 1, and two optical flow constraints for the two left, respectively right images; see Fig. [sent-124, score-1.038]
47 Our representation with rigidly moving 3D planes induces homographies, which map pixels from the reference view Il0 to the remaining views: Hr0(π) = (M − mnt)K−1 (2a) Hl1(π) = K(R − tnt)K−1 (2b) Hr1(π) = (MR −? [sent-126, score-0.6]
48 Shape and motion regularization The regularization terms shall encourage piecewise smooth 3D shape, as well as a piecewise smooth 3D motion field. [sent-149, score-0.899]
49 Since each segment is assigned to one rigidly moving plane, smoothness within a segment is always satisfied; we thus only need to consider the segment boundaries. [sent-150, score-0.91]
50 If 2D regularization in the image plane is desired instead, one can mimic the regularizer of [18] by replacing the 3D distances with disparity differences, differences between 2D optical flow vectors, and changes of the disparity difference over time. [sent-200, score-1.066]
51 We employ a segment regularization term similar in spirit to the approach of [21], which encourages smooth segments whose boundaries coincide with image edges. [sent-204, score-0.57]
52 The motivation is twofold: First, this prevents segments from getting too large, such that the scene is not overly simplified by the assumed piecewise planarity and piecewise rigid motion. [sent-225, score-0.586]
53 The set of seed points E(si) for a segment contains the center pixel of tsheee segment si in the initial superpixelization, as well as all pixels from a regularly-spaced grid (with spacing NS) that fell into the respective initial superpixel. [sent-227, score-0.631]
54 term ED and the shape and motion regularization Recall that these regularizers can only incur penalties at segment boundaries, since neighboring pixels within a segment will be assigned to the same moving plane, thus incur no regularization penalty. [sent-247, score-1.14]
55 The optimization is performed over a set of proposal planes with their associated motion (see Sec. [sent-253, score-0.502]
56 Proposal generation To perform inference over the depth and motion of each segment, we require a comprehensive proposal set of 3D planes along with their rigid motions. [sent-266, score-0.69]
57 We can generate these from the output of other scene flow algorithms or by combining the results of stereo and optical flow algorithms (see Sec. [sent-267, score-1.573]
58 To that end we fit a 3D plane to each superpixel segment, and estimate its rigid motion from the flow field(s). [sent-270, score-0.928]
59 In either case, fitting must be robust to a potentially large amount of outliers, caused both by inaccurate depth and motion estimates and by superpixels not being aligned with surface or motion boundaries. [sent-271, score-0.522]
60 We address this by robustly minimizing the transfer error: We first generate an initial solution by minimizing the quadratic transfer error using efficient algebraic methods, and then refine the rigidly moving proposal planes by gradient descent on the robust transfer error (Lorentzian penalty). [sent-272, score-0.564]
61 We apply the well-known principle ofusing a fixed occlusion penalty θ, if a certain pixel p (in the reference view) is occluded in at least one view of a pair. [sent-286, score-0.482]
62 We now consider whether a pixel p is occluded or not, which depends both on its binary segment assignment xp, and on whether there is any other pixel q (or possibly multiple pixels) that occludes p. [sent-305, score-0.529]
63 The summand becomes ˆ u0p, if xp = 0 and the product equals 1, which is the case if there is no pixel that could possibly occlude p, or if all possibly occluding pixels q are assigned a segment xq in which they do not lead to an occlusion. [sent-320, score-0.622]
64 1 shows the estimated 3D scene flow (left), and the results after various processing stages (right). [sent-337, score-0.623]
65 The segment-based scene reconstruction (Segment) without occlusion handling already gives fairly plausible results in unoccluded areas, but assigns incorrect depth and motion to regions not visible in the reference image (best seen immediately left of the × pedestrian). [sent-338, score-0.731]
66 By adding the segment-based occlusion model (Segment & Occlusion), the occlusion regions are properly detected and their motion is extrapolated in a more realistic manner. [sent-339, score-0.5]
67 Comparison with 2D scene flow For comparison with other scene flow methods from the literature [10, 18, 24] we consider the synthetic scene of [10], which consists of two independently rotating hemispheres in front of a plane. [sent-351, score-1.4]
68 The dataset provides images (1240 376 pixels) recorded with a calibprroatveidd esste irmeoa rig on a car, f3o7r6 benchmarking eofd optical fclaolwiand stereo algorithms in the context of automotive applica- × tions. [sent-356, score-0.551]
69 Having stereo pairs from consecutive video frames, the dataset also fulfills the requirements for scene flow estimation (see Fig. [sent-365, score-0.959]
70 The KITTI dataset provides a very challenging testbed for today’s stereo, optical flow and scene flow algorithms: First, pixel displacements in the data set are large in general, exceeding 150 pixels for stereo and 250 pixels for optical flow. [sent-368, score-2.026]
71 not visible in all four images, we let the stereo and 2D flow algorithms from the proposal generator predict which pixels are out-of-bounds and encourage the scene flow estimate to stay near that prediction. [sent-380, score-1.659]
72 reference frame; (right) flow reprojected to the reference image. [sent-424, score-0.647]
73 Nonetheless, we believe that this dataset is better suited for scene flow evaluation than the very limited, synthetic datasets used before [e. [sent-428, score-0.623]
74 At time of writing the results of the 2D baseline rank 13th in both stereo and optical flow among published methods in the official KITTI ranking. [sent-433, score-0.95]
75 Finally, we distinguish the use of various proposal sets: All experiments default to proposals from the stereo and flowderived 2D baseline technique (S+F). [sent-442, score-0.514]
76 The suffix (+R) denotes that a proposal set composed from locally rigid scene flow (Rig, [22]) is additionally used. [sent-443, score-0.914]
77 Finally, we optionally make use of egomotion proposals (+E), by estimating the dominant 3D motion using our proposal fitting technique on the 2D baseline output. [sent-444, score-0.424]
78 Rather, epipolar motion is one of several proposal solutions, which are used to minimize an energy that can cope with general, non-epipolar motion. [sent-446, score-0.528]
79 2D and 3D regularization lead to rather similar results, possibly explained by the fact that the evaluation does not have ground truth for 3D scene flow, but only for disparity and 2D optical flow. [sent-448, score-0.556]
80 We thus mostly rely on the 2D regularizer, and note that better 3D benchmarks are needed for quantitative evaluation of 3D scene flow methods. [sent-449, score-0.623]
81 Additional occlusion reasoning (-O) improves the results, especially for motion estimates in occluded areas, but performance in the stereo case slightly decreases. [sent-450, score-0.737]
82 Still, already the 2D proposal set from S+F alone is sufficient to surpass all our baselines by a large margin, including two recent 3D scene flow techniques [3, 22], on average by 33%. [sent-456, score-0.782]
83 f rA pte rt-ipmixe ol fo writing our method (PRSPix-2D+R+E) ranks 1st out of 28 published approaches for optical flow in all measures, and 3rd out of25 published methods in stereo, while (PRSPix-2D) ranked 3th and 5th respectively. [sent-461, score-0.654]
84 Moreover, it strongly outperforms another 3D scene flow technique [7], even on its semi-dense output. [sent-465, score-0.655]
85 Finally, even the best general 2D optical flow method is surpassed. [sent-466, score-0.654]
86 To the best of our knowledge, this is the first scene flow method to outperform optical flow algorithms w. [sent-467, score-1.277]
87 Conclusion We have shown that modeling a dynamic scene with local regions corresponding to rigidly moving planes can lead to compelling results for the task of joint geometry and 3D 1383 threshold of Z pixels) in non-occluded areas (Noc) and over the full image (All). [sent-472, score-0.636]
88 The proposed model achieves accurate geometry and motion boundaries by refining an initial oversegmentation of the scene, and allows for occlusion reasoning. [sent-474, score-0.449]
89 We show that our method substantially outperforms previous dense scene flow approaches on a challenging data set, and even surpasses dedicated state-of-the-art stereo and optical flow techniques at their respective task. [sent-475, score-1.699]
90 Its main limitation are scenes with strongly non-rigid motion or extreme curvature, where the piecewise planar and rigid approximation does not hold. [sent-476, score-0.539]
91 Determining three-dimensional motion and structure from optical flow generated by several moving objects. [sent-482, score-0.98]
92 Multi-view scene flow estimation: A view centered variational approach. [sent-497, score-0.707]
93 High accuracy optical flow estimation based on a theory for warping. [sent-513, score-0.694]
94 A variational method for scene flow estimation from stereo sequences. [sent-542, score-1.002]
95 Dense, robust, and accurate motion field estimation from stereo image sequences in real-time. [sent-561, score-0.576]
96 Pushing the limits of stereo using variational stereo estimation. [sent-568, score-0.635]
97 Joint motion estimation and segmentation of complex scenes with label costs and occlusion modeling. [sent-590, score-0.452]
98 Differences between stereo and motion behaviour on synthetic and realworld stereo sequences. [sent-606, score-0.798]
99 3D scene flow estimation with a rigid motion prior. [sent-626, score-1.001]
100 Efficient dense scene flow from sparse or dense [25] [26] [27] [28] stereo data. [sent-640, score-0.993]
wordName wordTfidf (topN-words)
[('flow', 0.469), ('stereo', 0.296), ('motion', 0.206), ('segment', 0.198), ('optical', 0.185), ('kitti', 0.161), ('proposal', 0.159), ('scene', 0.154), ('rigidly', 0.148), ('occlusion', 0.147), ('planes', 0.137), ('rigid', 0.132), ('moving', 0.12), ('piecewise', 0.1), ('segments', 0.1), ('disparity', 0.097), ('reference', 0.089), ('pixel', 0.088), ('endpoints', 0.086), ('superpixelization', 0.084), ('rabe', 0.084), ('plane', 0.083), ('xp', 0.082), ('penalty', 0.082), ('regularization', 0.08), ('census', 0.073), ('opi', 0.071), ('rig', 0.07), ('planar', 0.069), ('pixels', 0.065), ('energy', 0.064), ('epipolar', 0.062), ('xq', 0.061), ('proposals', 0.059), ('segmentation', 0.059), ('parameterizing', 0.058), ('depth', 0.056), ('boundaries', 0.055), ('vogel', 0.055), ('regularizer', 0.055), ('coincide', 0.054), ('superpixels', 0.054), ('reasoning', 0.053), ('dedicated', 0.051), ('wedel', 0.05), ('exceeding', 0.05), ('assigned', 0.048), ('encourage', 0.047), ('boundary', 0.047), ('huguet', 0.047), ('prspix', 0.047), ('uip', 0.047), ('vedula', 0.047), ('yamaguchi', 0.047), ('ns', 0.046), ('seed', 0.044), ('unoccluded', 0.043), ('term', 0.043), ('variational', 0.043), ('endpoint', 0.042), ('noc', 0.042), ('qpbo', 0.042), ('valgaerts', 0.042), ('basha', 0.042), ('vaudrey', 0.042), ('view', 0.041), ('geometry', 0.041), ('thresholds', 0.041), ('assignment', 0.041), ('possibly', 0.04), ('estimation', 0.04), ('smooth', 0.04), ('deviations', 0.04), ('views', 0.04), ('penalties', 0.04), ('penalize', 0.039), ('dnm', 0.039), ('occludes', 0.039), ('nir', 0.039), ('superpixel', 0.038), ('respective', 0.038), ('cope', 0.037), ('dense', 0.037), ('unger', 0.037), ('assigns', 0.036), ('areas', 0.036), ('nq', 0.035), ('aggressive', 0.035), ('occluded', 0.035), ('field', 0.034), ('bleyer', 0.033), ('brightness', 0.032), ('bruhn', 0.032), ('lempitsky', 0.032), ('strongly', 0.032), ('incur', 0.031), ('boolean', 0.031), ('intrinsics', 0.031), ('np', 0.031), ('normal', 0.03)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999988 317 iccv-2013-Piecewise Rigid Scene Flow
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
2 0.36964157 12 iccv-2013-A General Dense Image Matching Framework Combining Direct and Feature-Based Costs
Author: Jim Braux-Zin, Romain Dupont, Adrien Bartoli
Abstract: Dense motion field estimation (typically Romain Dupont1 romain . dupont @ cea . fr Adrien Bartoli2 adrien . bart o l @ gmai l com i . 2 ISIT, Universit e´ d’Auvergne/CNRS, France sions are explicitly modeled [32, 13]. Coarse-to-fine warping improves global convergence by making the assumption that optical flow, the motion of smaller structures is similar to the motion of stereo disparity and surface registration) is a key computer vision problem. Many solutions have been proposed to compute small or large displacements, narrow or wide baseline stereo disparity, but a unified methodology is still lacking. We here introduce a general framework that robustly combines direct and feature-based matching. The feature-based cost is built around a novel robust distance function that handles keypoints and “weak” features such as segments. It allows us to use putative feature matches which may contain mismatches to guide dense motion estimation out of local minima. Our framework uses a robust direct data term (AD-Census). It is implemented with a powerful second order Total Generalized Variation regularization with external and self-occlusion reasoning. Our framework achieves state of the art performance in several cases (standard optical flow benchmarks, wide-baseline stereo and non-rigid surface registration). Our framework has a modular design that customizes to specific application needs.
3 0.3398594 105 iccv-2013-DeepFlow: Large Displacement Optical Flow with Deep Matching
Author: Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Abstract: Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox & Malik [6], our approach, termed DeepFlow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable assetfor integration into an energy minimizationframework for optical flow estimation. DeepFlow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset [8].
4 0.30311975 300 iccv-2013-Optical Flow via Locally Adaptive Fusion of Complementary Data Costs
Author: Tae Hyun Kim, Hee Seok Lee, Kyoung Mu Lee
Abstract: Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods.
5 0.28223455 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
Author: Manjunath Narayana, Allen Hanson, Erik Learned-Miller
Abstract: In moving camera videos, motion segmentation is commonly performed using the image plane motion of pixels, or optical flow. However, objects that are at different depths from the camera can exhibit different optical flows even if they share the same real-world motion. This can cause a depth-dependent segmentation of the scene. Our goal is to develop a segmentation algorithm that clusters pixels that have similar real-world motion irrespective of their depth in the scene. Our solution uses optical flow orientations instead of the complete vectors and exploits the well-known property that under camera translation, optical flow orientations are independent of object depth. We introduce a probabilistic model that automatically estimates the number of observed independent motions and results in a labeling that is consistent with real-world motion in the scene. The result of our system is that static objects are correctly identified as one segment, even if they are at different depths. Color features and information from previous frames in the video sequence are used to correct occasional errors due to the orientation-based segmentation. We present results on more than thirty videos from different benchmarks. The system is particularly robust on complex background scenes containing objects at significantly different depths.
6 0.26807988 256 iccv-2013-Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
7 0.2613121 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
8 0.226467 442 iccv-2013-Video Segmentation by Tracking Many Figure-Ground Segments
9 0.21583751 382 iccv-2013-Semi-dense Visual Odometry for a Monocular Camera
10 0.21400218 304 iccv-2013-PM-Huber: PatchMatch with Huber Regularization for Stereo Matching
11 0.20247003 39 iccv-2013-Action Recognition with Improved Trajectories
12 0.19421275 322 iccv-2013-Pose Estimation and Segmentation of People in 3D Movies
13 0.18864723 263 iccv-2013-Measuring Flow Complexity in Videos
14 0.18175651 143 iccv-2013-Estimating Human Pose with Flowing Puppets
15 0.16768508 82 iccv-2013-Compensating for Motion during Direct-Global Separation
16 0.16698483 37 iccv-2013-Action Recognition and Localization by Hierarchical Space-Time Segments
17 0.16048056 420 iccv-2013-Topology-Constrained Layered Tracking with Latent Flow
18 0.1505637 219 iccv-2013-Internet Based Morphable Model
19 0.14998682 424 iccv-2013-Tracking Revisited Using RGBD Camera: Unified Benchmark and Baselines
20 0.14336273 282 iccv-2013-Multi-view Object Segmentation in Space and Time
topicId topicWeight
[(0, 0.307), (1, -0.284), (2, 0.016), (3, 0.185), (4, 0.045), (5, 0.051), (6, -0.065), (7, -0.015), (8, 0.083), (9, 0.005), (10, -0.022), (11, 0.117), (12, 0.296), (13, -0.013), (14, -0.052), (15, -0.104), (16, -0.178), (17, 0.006), (18, 0.179), (19, 0.085), (20, 0.133), (21, -0.02), (22, -0.011), (23, -0.068), (24, 0.127), (25, -0.129), (26, 0.081), (27, -0.056), (28, -0.024), (29, 0.001), (30, -0.088), (31, -0.007), (32, -0.112), (33, -0.069), (34, -0.076), (35, 0.046), (36, -0.001), (37, 0.03), (38, -0.037), (39, -0.084), (40, -0.007), (41, -0.06), (42, -0.021), (43, 0.005), (44, 0.014), (45, 0.007), (46, -0.078), (47, -0.028), (48, 0.004), (49, -0.011)]
simIndex simValue paperId paperTitle
same-paper 1 0.98150539 317 iccv-2013-Piecewise Rigid Scene Flow
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
2 0.86062008 12 iccv-2013-A General Dense Image Matching Framework Combining Direct and Feature-Based Costs
Author: Jim Braux-Zin, Romain Dupont, Adrien Bartoli
Abstract: Dense motion field estimation (typically Romain Dupont1 romain . dupont @ cea . fr Adrien Bartoli2 adrien . bart o l @ gmai l com i . 2 ISIT, Universit e´ d’Auvergne/CNRS, France sions are explicitly modeled [32, 13]. Coarse-to-fine warping improves global convergence by making the assumption that optical flow, the motion of smaller structures is similar to the motion of stereo disparity and surface registration) is a key computer vision problem. Many solutions have been proposed to compute small or large displacements, narrow or wide baseline stereo disparity, but a unified methodology is still lacking. We here introduce a general framework that robustly combines direct and feature-based matching. The feature-based cost is built around a novel robust distance function that handles keypoints and “weak” features such as segments. It allows us to use putative feature matches which may contain mismatches to guide dense motion estimation out of local minima. Our framework uses a robust direct data term (AD-Census). It is implemented with a powerful second order Total Generalized Variation regularization with external and self-occlusion reasoning. Our framework achieves state of the art performance in several cases (standard optical flow benchmarks, wide-baseline stereo and non-rigid surface registration). Our framework has a modular design that customizes to specific application needs.
3 0.81828761 300 iccv-2013-Optical Flow via Locally Adaptive Fusion of Complementary Data Costs
Author: Tae Hyun Kim, Hee Seok Lee, Kyoung Mu Lee
Abstract: Many state-of-the-art optical flow estimation algorithms optimize the data and regularization terms to solve ill-posed problems. In this paper, in contrast to the conventional optical flow framework that uses a single or fixed data model, we study a novel framework that employs locally varying data term that adaptively combines different multiple types of data models. The locally adaptive data term greatly reduces the matching ambiguity due to the complementary nature of the multiple data models. The optimal number of complementary data models is learnt by minimizing the redundancy among them under the minimum description length constraint (MDL). From these chosen data models, a new optical flow estimation energy model is designed with the weighted sum of the multiple data models, and a convex optimization-based highly effective and practical solution thatfinds the opticalflow, as well as the weights isproposed. Comparative experimental results on the Middlebury optical flow benchmark show that the proposed method using the complementary data models outperforms the state-ofthe art methods.
4 0.79551148 105 iccv-2013-DeepFlow: Large Displacement Optical Flow with Deep Matching
Author: Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui, Cordelia Schmid
Abstract: Optical flow computation is a key component in many computer vision systems designed for tasks such as action detection or activity recognition. However, despite several major advances over the last decade, handling large displacement in optical flow remains an open problem. Inspired by the large displacement optical flow of Brox & Malik [6], our approach, termed DeepFlow, blends a matching algorithm with a variational approach for optical flow. We propose a descriptor matching algorithm, tailored to the optical flow problem, that allows to boost performance on fast motions. The matching algorithm builds upon a multi-stage architecture with 6 layers, interleaving convolutions and max-pooling, a construction akin to deep convolutional nets. Using dense sampling, it allows to efficiently retrieve quasi-dense correspondences, and enjoys a built-in smoothing effect on descriptors matches, a valuable assetfor integration into an energy minimizationframework for optical flow estimation. DeepFlow efficiently handles large displacements occurring in realistic videos, and shows competitive performance on optical flow benchmarks. Furthermore, it sets a new state-of-the-art on the MPI-Sintel dataset [8].
5 0.77767086 256 iccv-2013-Locally Affine Sparse-to-Dense Matching for Motion and Occlusion Estimation
Author: Marius Leordeanu, Andrei Zanfir, Cristian Sminchisescu
Abstract: Estimating a dense correspondence field between successive video frames, under large displacement, is important in many visual learning and recognition tasks. We propose a novel sparse-to-dense matching method for motion field estimation and occlusion detection. As an alternative to the current coarse-to-fine approaches from the optical flow literature, we start from the higher level of sparse matching with rich appearance and geometric constraints collected over extended neighborhoods, using an occlusion aware, locally affine model. Then, we move towards the simpler, but denser classic flow field model, with an interpolation procedure that offers a natural transition between the sparse and the dense correspondence fields. We experimentally demonstrate that our appearance features and our complex geometric constraintspermit the correct motion estimation even in difficult cases of large displacements and significant appearance changes. We also propose a novel classification method for occlusion detection that works in conjunction with the sparse-to-dense matching model. We validate our approach on the newly released Sintel dataset and obtain state-of-the-art results.
6 0.73273897 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
7 0.69790661 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
8 0.66383415 430 iccv-2013-Two-Point Gait: Decoupling Gait from Body Shape
9 0.64973813 420 iccv-2013-Topology-Constrained Layered Tracking with Latent Flow
10 0.62976509 263 iccv-2013-Measuring Flow Complexity in Videos
11 0.59720874 304 iccv-2013-PM-Huber: PatchMatch with Huber Regularization for Stereo Matching
12 0.59615248 301 iccv-2013-Optimal Orthogonal Basis and Image Assimilation: Motion Modeling
13 0.56114513 82 iccv-2013-Compensating for Motion during Direct-Global Separation
14 0.55988115 143 iccv-2013-Estimating Human Pose with Flowing Puppets
15 0.54826623 39 iccv-2013-Action Recognition with Improved Trajectories
16 0.53508377 270 iccv-2013-Modeling Self-Occlusions in Dynamic Shape and Appearance Tracking
17 0.52940857 442 iccv-2013-Video Segmentation by Tracking Many Figure-Ground Segments
18 0.52639085 145 iccv-2013-Estimating the Material Properties of Fabric from Video
19 0.52159166 275 iccv-2013-Motion-Aware KNN Laplacian for Video Matting
20 0.50863266 171 iccv-2013-Fix Structured Learning of 2013 ICCV paper k2opt.pdf
topicId topicWeight
[(2, 0.074), (7, 0.012), (12, 0.012), (18, 0.113), (26, 0.096), (31, 0.044), (42, 0.099), (48, 0.013), (64, 0.075), (73, 0.062), (78, 0.013), (89, 0.253), (95, 0.013), (98, 0.016)]
simIndex simValue paperId paperTitle
1 0.96097761 87 iccv-2013-Conservation Tracking
Author: Martin Schiegg, Philipp Hanslovsky, Bernhard X. Kausler, Lars Hufnagel, Fred A. Hamprecht
Abstract: The quality of any tracking-by-assignment hinges on the accuracy of the foregoing target detection / segmentation step. In many kinds of images, errors in this first stage are unavoidable. These errors then propagate to, and corrupt, the tracking result. Our main contribution is the first probabilistic graphical model that can explicitly account for over- and undersegmentation errors even when the number of tracking targets is unknown and when they may divide, as in cell cultures. The tracking model we present implements global consistency constraints for the number of targets comprised by each detection and is solved to global optimality on reasonably large 2D+t and 3D+t datasets. In addition, we empirically demonstrate the effectiveness of a postprocessing that allows to establish target identity even across occlusion / undersegmentation. The usefulness and efficiency of this new tracking method is demonstrated on three different and challenging 2D+t and 3D+t datasets from developmental biology.
same-paper 2 0.93570691 317 iccv-2013-Piecewise Rigid Scene Flow
Author: Christoph Vogel, Konrad Schindler, Stefan Roth
Abstract: Estimating dense 3D scene flow from stereo sequences remains a challenging task, despite much progress in both classical disparity and 2D optical flow estimation. To overcome the limitations of existing techniques, we introduce a novel model that represents the dynamic 3D scene by a collection of planar, rigidly moving, local segments. Scene flow estimation then amounts to jointly estimating the pixelto-segment assignment, and the 3D position, normal vector, and rigid motion parameters of a plane for each segment. The proposed energy combines an occlusion-sensitive data term with appropriate shape, motion, and segmentation regularizers. Optimization proceeds in two stages: Starting from an initial superpixelization, we estimate the shape and motion parameters of all segments by assigning a proposal from a set of moving planes. Then the pixel-to-segment assignment is updated, while holding the shape and motion parameters of the moving planes fixed. We demonstrate the benefits of our model on different real-world image sets, including the challenging KITTI benchmark. We achieve leading performance levels, exceeding competing 3D scene flow methods, and even yielding better 2D motion estimates than all tested dedicated optical flow techniques.
3 0.93380624 150 iccv-2013-Exemplar Cut
Author: Jimei Yang, Yi-Hsuan Tsai, Ming-Hsuan Yang
Abstract: We present a hybrid parametric and nonparametric algorithm, exemplar cut, for generating class-specific object segmentation hypotheses. For the parametric part, we train a pylon model on a hierarchical region tree as the energy function for segmentation. For the nonparametric part, we match the input image with each exemplar by using regions to obtain a score which augments the energy function from the pylon model. Our method thus generates a set of highly plausible segmentation hypotheses by solving a series of exemplar augmented graph cuts. Experimental results on the Graz and PASCAL datasets show that the proposed algorithm achievesfavorable segmentationperformance against the state-of-the-art methods in terms of visual quality and accuracy.
4 0.92791927 196 iccv-2013-Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation
Author: Yuandong Tian, Srinivasa G. Narasimhan
Abstract: Real-world surfaces such as clothing, water and human body deform in complex ways. The image distortions observed are high-dimensional and non-linear, making it hard to estimate these deformations accurately. The recent datadriven descent approach [17] applies Nearest Neighbor estimators iteratively on a particular distribution of training samples to obtain a globally optimal and dense deformation field between a template and a distorted image. In this work, we develop a hierarchical structure for the Nearest Neighbor estimators, each of which can have only a local image support. We demonstrate in both theory and practice that this algorithm has several advantages over the nonhierarchical version: it guarantees global optimality with significantly fewer training samples, is several orders faster, provides a metric to decide whether a given image is “hard” (or “easy ”) requiring more (or less) samples, and can handle more complex scenes that include both global motion and local deformation. The proposed algorithm successfully tracks a broad range of non-rigid scenes including water, clothing, and medical images, and compares favorably against several other deformation estimation and tracking approaches that do not provide optimality guarantees.
5 0.92786646 9 iccv-2013-A Flexible Scene Representation for 3D Reconstruction Using an RGB-D Camera
Author: Diego Thomas, Akihiro Sugimoto
Abstract: Updating a global 3D model with live RGB-D measurements has proven to be successful for 3D reconstruction of indoor scenes. Recently, a Truncated Signed Distance Function (TSDF) volumetric model and a fusion algorithm have been introduced (KinectFusion), showing significant advantages such as computational speed and accuracy of the reconstructed scene. This algorithm, however, is expensive in memory when constructing and updating the global model. As a consequence, the method is not well scalable to large scenes. We propose a new flexible 3D scene representation using a set of planes that is cheap in memory use and, nevertheless, achieves accurate reconstruction of indoor scenes from RGB-D image sequences. Projecting the scene onto different planes reduces significantly the size of the scene representation and thus it allows us to generate a global textured 3D model with lower memory requirement while keeping accuracy and easiness to update with live RGB-D measurements. Experimental results demonstrate that our proposed flexible 3D scene representation achieves accurate reconstruction, while keeping the scalability for large indoor scenes.
6 0.92444223 127 iccv-2013-Dynamic Pooling for Complex Event Recognition
7 0.92430472 204 iccv-2013-Human Attribute Recognition by Rich Appearance Dictionary
8 0.92427039 89 iccv-2013-Constructing Adaptive Complex Cells for Robust Visual Tracking
9 0.92341602 65 iccv-2013-Breaking the Chain: Liberation from the Temporal Markov Assumption for Tracking Human Poses
10 0.92330855 396 iccv-2013-Space-Time Robust Representation for Action Recognition
11 0.92319232 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
12 0.9231286 107 iccv-2013-Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction
13 0.92311859 379 iccv-2013-Semantic Segmentation without Annotating Segments
14 0.92266548 433 iccv-2013-Understanding High-Level Semantics by Modeling Traffic Patterns
15 0.92234075 361 iccv-2013-Robust Trajectory Clustering for Motion Segmentation
16 0.92208946 351 iccv-2013-Restoring an Image Taken through a Window Covered with Dirt or Rain
17 0.92199111 200 iccv-2013-Higher Order Matching for Consistent Multiple Target Tracking
18 0.92179078 420 iccv-2013-Topology-Constrained Layered Tracking with Latent Flow
19 0.92121881 186 iccv-2013-GrabCut in One Cut
20 0.92109942 410 iccv-2013-Support Surface Prediction in Indoor Scenes