iccv iccv2013 iccv2013-172 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Chenliang Xu, Spencer Whitt, Jason J. Corso
Abstract: Supervoxel hierarchies provide a rich multiscale decomposition of a given video suitable for subsequent processing in video analysis. The hierarchies are typically computed by an unsupervised process that is susceptible to undersegmentation at coarse levels and over-segmentation at fine levels, which make it a challenge to adopt the hierarchies for later use. In this paper, we propose the first method to overcome this limitation and flatten the hierarchy into a single segmentation. Our method, called the uniform entropy slice, seeks a selection of supervoxels that balances the relative level of information in the selected supervoxels based on some post hoc feature criterion such as objectness. For example, with this criterion, in regions nearby objects, our method prefers finer supervoxels to capture the local details, but in regions away from any objects we prefer coarser supervoxels. We formulate the uniform entropy slice as a binary quadratic program and implement four different feature criteria, both unsupervised and supervised, to drive the flattening. Although we apply it only to supervoxel hierarchies in this paper, our method is generally applicable to segmentation tree hierarchies. Our experiments demonstrate both strong qualitative performance and superior quantitative performance to state of the art baselines on benchmark internet videos.
Reference: text
sentIndex sentText sentNum sentScore
1 The hierarchies are typically computed by an unsupervised process that is susceptible to undersegmentation at coarse levels and over-segmentation at fine levels, which make it a challenge to adopt the hierarchies for later use. [sent-3, score-0.401]
2 Our method, called the uniform entropy slice, seeks a selection of supervoxels that balances the relative level of information in the selected supervoxels based on some post hoc feature criterion such as objectness. [sent-5, score-1.251]
3 For example, with this criterion, in regions nearby objects, our method prefers finer supervoxels to capture the local details, but in regions away from any objects we prefer coarser supervoxels. [sent-6, score-0.518]
4 We formulate the uniform entropy slice as a binary quadratic program and implement four different feature criteria, both unsupervised and supervised, to drive the flattening. [sent-7, score-0.757]
5 Although we apply it only to supervoxel hierarchies in this paper, our method is generally applicable to segmentation tree hierarchies. [sent-8, score-0.849]
6 The uniform entropy slice (UES) selects supervoxels from multiple hierarchical levels in a principled way to balance the amount of information contributed by each selected supervoxel, according to some feature criterion (motion in this figure). [sent-16, score-1.296]
7 UES Selection shows what levels are used and UES Flattening shows the final supervoxel output. [sent-17, score-0.619]
8 Leveraging contributions in image segmentation evaluation [3] and criteria for good video segmentation [11], we have proposed the LIBSVX benchmark [33], which implements a suite of six supervoxel algorithms and tests them in a set of evaluation metrics with three video datasets. [sent-20, score-1.008]
9 [27] segmentation by weighted aggregation, perform best overall due to the way in which multiscale region similarity was reevaluated as the hierarchy was generated. [sent-23, score-0.347]
10 Trivial flattenings, by arbitrarily taking a level, would carry over intrinsic limitations of the bottom-up supervoxel methods, as Fig. [sent-26, score-0.515]
11 For example, taking a low level would mean very many supervoxels (over22224400 Input Video UES on Motion (Selection) Figure 2. [sent-28, score-0.33]
12 Example of supervoxel hierarchy selection by UES with a motion criterion on video boxers. [sent-29, score-1.106]
13 The motion criterion drives the algorithm to select finer levels of the hierarchy (brighter regions on bottom row) on the dominant moving objects. [sent-30, score-0.703]
14 The boxer on the right and the head of the boxer on the left are being selected from finer levels in the supervoxel hierarchy while the background segments are from coarser levels in the hierarchy. [sent-31, score-1.283]
15 We believe this is the key limitation to the adoption of supervoxels for early video analysis. [sent-34, score-0.419]
16 In this paper, we propose the first principled solution to overcome this key limitation of flattening a supervoxel hierarchy. [sent-35, score-0.706]
17 Our emphasis is on video supervoxel hierarchies, but the core contribution is generally applicable to image and other segmentation hierarchies, given certain assumptions are met. [sent-36, score-0.701]
18 Our approach includes a novel model—the uniform entropy slice (UES)—and a formulation for efficiently solving it via a binary quadratic program (QP). [sent-37, score-0.667]
19 A slice through the hierarchy is a flattened supervoxel segmentation generally consisting of supervoxels from various levels of the hierarchy. [sent-38, score-1.648]
20 Such a criterion enables us to pull out the most unique and dominant regions in a supervoxel hierarchy as shown in Figure 2. [sent-40, score-0.919]
21 The feature criterion, which drives the uniform entropy slice and hence the flattening, is independent of the supervoxel hierarchy itself. [sent-41, score-1.453]
22 We explore four different cases for the feature criterion underlying the uniform entropy slice: motion, object-ness, human-ness, and car-ness. [sent-42, score-0.441]
23 On the top of the figure, left, middle and right are bottom-up levels in a supervoxel hierarchy: T1, T2 and T3 respectively. [sent-46, score-0.645]
24 gree of versatility in the proposed method: indeed it can take any form of a subsequent criterion and apply it to a previously computed supervoxel hierarchy. [sent-50, score-0.642]
25 We have implemented and tested the uniform entropy slice on top of the state of the art graph-based segmentation (GBH) [16] and segmentation by weighted aggregation (SWA) [27] methods. [sent-51, score-0.835]
26 Our qualitative results demonstrate numerous clear cases in which the flattened supervoxels are precisely what is expected for various feature criteria, like human-ness. [sent-53, score-0.375]
27 Based on some hierarchical supervoxel algorithm, consider an h level hierarchical oversegmentation of the video: T {T1, T2 , . [sent-62, score-0.64]
28 , Th} and Vi sise gthmee nnotadteio snet o ifn t supervoxel le=vel { TTi, i∈ [1, h] . [sent-65, score-0.515]
29 We let node V0 be the root node of the supervoxel hierarchy T , and V1 is thbee s thete o rfo loeta fn ondoede osf tihn eT s . [sent-69, score-0.92]
30 u hWee s ecton ofsi ldeaefr only supervoxel hierarchies that are trees, i. [sent-70, score-0.63]
31 Figure 3 shows the general creation process of such a supervoxel tree; GBH generates such a tree. [sent-73, score-0.515]
32 Each slice is highlighted as a thick black curve, and nodes on the slice are darkened. [sent-77, score-0.708]
33 algorithm initially builds a 26-connected voxel lattice over the whole video clip, then iteratively constructs a region graph over the obtained segmentation based on the minimum spanning tree merging criterion [14], and forms a bottom-up segmentation tree structure of the regions. [sent-78, score-0.633]
34 Define a tree slice as a set of nodes from the hierarchy such that on each root-to-leaf path in the hierarchy, there is one and only one node in the slice set. [sent-83, score-1.221]
35 4 shows example tree slices for the segmentation tree from the previous Fig. [sent-87, score-0.421]
36 Note that we call this a tree slice rather than a tree cut to distinguish it from conventional use of the term cut, which generally indicates a set of edges and not nodes as we have in the slice. [sent-92, score-0.621]
37 In a valid slice, each root-to-leaf path in the segmentation tree T has one and only node being selected. [sent-99, score-0.386]
38 5 shows the path matrix P for our example supervoxel tFriege. [sent-107, score-0.577]
39 To ensure a tree slice is a valid, we have Px = 1p , (1) where 1p is an p-length vector of all ones. [sent-127, score-0.456]
40 The Uniform Entropy Slice The previous section presents the tree slice problem and a linear constraint to assess the validity of a slice; here we present a new model based on uniform entropy to quantify a slice. [sent-133, score-0.766]
41 The intuitive idea behind the uniform entropy slice is that we want to select nodes in the tree that balance the information contribution to the overall slice. [sent-134, score-0.833]
42 In our case for supervoxel hierarchies, one can relate finding the best slice in a hierarchy to such a compression 22224422 ? [sent-138, score-1.099]
43 Example hierarchy node entropy for the motion feature criterion. [sent-163, score-0.641]
44 (a) is the raw video girl from SegTrack, (b: coarse) (h: fine) are node entropy from various levels in the hierarchy. [sent-164, score-0.574]
45 The entropy color from dark blue to dark red maps entropy changing from low to high (using the jet colormap in Matlab). [sent-165, score-0.464]
46 Notice how the entropy of the girls limbs is relatively higher than that of the background for corresponding hierarchy levels. [sent-166, score-0.461]
47 F noord example, consider an unsupervised motion feature criterion in which we want the slice to focus on regions of the video that are moving uniquely relative to the rest of the video—e. [sent-174, score-0.708]
48 the hierarchy is computed by the entropy over F(·) : E(Vs) =. [sent-180, score-0.461]
49 We next propose the uniform entropy objective, which considers the node information content according to Eq. [sent-184, score-0.387]
50 2 and seeks a tree slice that balances the overall information content of the selected nodes. [sent-185, score-0.509]
51 Again, consider a valid tree slice x which is a vector of binary variables with one xs for each node Vs in the hierarchy taking value 1if the node is on the slice and 0 otherwise. [sent-186, score-1.241]
52 The uniform entropy objective hence seeks a valid tree slice that minimizes the difference in entropy of selected nodes: x∗ = argmin ? [sent-187, score-1.057]
53 First, in a common case, the entropy of a supervoxel in coarser levels of the hierarchy drops down when the segment breaks up into smaller pieces at finer levels. [sent-192, score-1.235]
54 6, which shows the node entropy for a motion feature criterion on the video girl from the SegTrack dataset [28]. [sent-194, score-0.678]
55 It is clear that the node entropy generally decreases from coarser to finer levels, and those informative supervoxels (the girl in this case) have overall more motion entropy than the background. [sent-195, score-1.117]
56 It is hence plausible the slice will select nodes around the girl at finer levels to match similar motion entropies to the background at coarser levels in the hierarchy. [sent-196, score-0.937]
57 Second, regions of the video that are salient for the specified feature criterion tend to have higher entropy than non-salient regions because of articulation and variability of the features near the salient region boundaries. [sent-197, score-0.513]
58 3 is complex because it requires enumerating all valid tree slices and includes a degenerate minimum which selects the root node only. [sent-202, score-0.326]
59 We instead reformulate the objective as the following binary quadratic program, which we call the uniform entropy slice. [sent-203, score-0.334]
60 Note the Px = 1p slice validity cbeontwstereanint t hfreo mtw Eq. [sent-209, score-0.333]
61 The linear term makes the slice prefer simpler segmentations when possible, i. [sent-213, score-0.378]
62 , prefer coarser levels in the hierarchy rather than finer levels in the hierarchy. [sent-215, score-0.634]
63 Ithne typical supervoxel trees, xthelesre in nis i a quadratic relationship between |Vi | and |Vi+1 | due to algorithm cco renlsattruiocntisohnip. [sent-217, score-0.539]
64 The quadratic term implements the uniform entropy objective βs,t = |E(Vs) − E(Vt) | |Vs | |Vt| (6) 22224433 Input VideoTop Level in GBH UES on Motion (Selection and Flat ening) UES on Human-ness (Selection and Flat ening) UES on Object-ness (Selection and Flat ening) Figure 7. [sent-218, score-0.334]
65 In the UES Selection images (left two columns), the dark red to dark blue means the finer levels to coarser levels in the supervoxel hierarchy tree. [sent-223, score-1.173]
66 Although vnoolduems ein o tfh teh coarser loexveellss Vof the tree have relatively higher entropy than nodes in the finer levels, the number ofcoarser level nodes is dramatically less than those in the finer levels. [sent-225, score-0.682]
67 By adding the volume factors, we push the selection down the hierarchy unless a uniform motion entropy has already been achieved. [sent-226, score-0.706]
68 See, for example, the level of the hierarchy selection for the video girl in Fig. [sent-228, score-0.537]
69 Feature Criteria The uniform entropy slice operates directly on a supervoxel hierarchy that was computed by an unsupervised method such as GBH. [sent-234, score-1.437]
70 However, the feature criteria, which drive the tree slice optimization, provide a doorway to apply situation-specific guidance post hoc. [sent-235, score-0.552]
71 Experiments We evaluate the uniform entropy slice (UES) both quantitatively (Sec. [sent-259, score-0.643]
72 To explore the generality of UES, we apply it to supervoxel hierarchies generated by two different methods, GBH [16] as implemented in [33] and SWA [27] as implemented in [9]. [sent-264, score-0.63]
73 For GBH, we construct a supervoxel tree directly from its output supervoxel hierarchy, since the method itself generates a tree structure. [sent-265, score-1.276]
74 For SWA, we simplify the supervoxel hierarchy, which is a general directed acyclic graph, to a tree structure by taking the most dominant parent for each child node and denote this variant of SWA as SWAT. [sent-266, score-0.739]
75 However, we have observed that, in practice, the relative hierarchy selection of supervoxels is not very sensitive to it. [sent-268, score-0.614]
76 We use the recently published supervoxel benchmark LIBSVX [33] to evaluate the UES with GBH and SWAT methods. [sent-273, score-0.533]
77 The benchmark provides six supervoxel methods and a set of supervoxel evaluation metrics. [sent-274, score-1.077]
78 The first is a simple trivial slice that takes a single level from the hierarchy, which we denote as “Base” in Table 1. [sent-306, score-0.386]
79 To the best of our knowledge, we are not aware of other video supervoxel selection algorithms. [sent-309, score-0.673]
80 The number of supervoxels from the input hierarchy varies from less than 10 to about 800. [sent-310, score-0.546]
81 Overall, the proposed UES achieves better performance for both SWAT and GBH supervoxel hierarchies than the other two baseline methods, and in some cases, such as 3D ACCU the improvement is significant for both methods. [sent-317, score-0.63]
82 In some cases, such as the video “cheetah” using SWAT, the scores are frequently the same for the three methods; this is a failure case of the overall supervoxel hierarchies, which we have observed to have little variation in supervoxels covered on the object at multiple levels in the hierarchy. [sent-319, score-1.004]
83 UES selects the coarse levels of the hierarchy for the background when doing so does not lead to foreground segments leaking, as in GBH. [sent-324, score-0.432]
84 A white circle means the bird has no segmentation leak, whereas a white rectangle means a segmentation leak with the surrounding tree branches. [sent-328, score-0.34]
85 Similarly, UES pushes the foreground and the corresponding leaking parts of the video down to the finer levels of the SWA hierarchy, while it still keeps the other background regions in the coarser levels of hierarchy. [sent-330, score-0.518]
86 The articulated pose generates more motion entropy over time than the surroundings do, which also allows UES to focus on the girl, as shown on the right half of the figure with both motion and object-ness criteria. [sent-335, score-0.364]
87 Sometimes, the motion criterion fails when the rigid objects have same motion as the camera in a video, or in a video with chaotic motion. [sent-347, score-0.349]
88 We show an example in Figure 10, where the motion completely fails to select the rigid object parachute, because the motion of it is uniform over the video (from left to right) with the camera. [sent-349, score-0.344]
89 The supervoxels in the top part of the object-ness selection image may seem to be errors, but indeed, these are expected: the parachute moves from left to right across the light and these selected supervoxels touch it at an earlier frame when it was passing by. [sent-351, score-0.703]
90 The top level hierarchy in GBH mistakes the woman in the left with the door and bench in the background. [sent-354, score-0.347]
91 However, the highperforming methods generate a hierarchy of supervoxels that often renders the user with more questions than at the outset due to the intrinsic limitations ofunsupervised grouping. [sent-368, score-0.546]
92 We have proposed the first principled method to flatten the hierarchy, called the uniform entropy slice (UES). [sent-369, score-0.691]
93 Our method seeks to balance the level of information across the selected supervoxels: choose bigger supervoxels in uninteresting regions of the video and smaller ones in interesting regions of the video. [sent-370, score-0.546]
94 A post hoc feature criterion is used to drive this information selection, and is independent of the original supervoxel process. [sent-371, score-0.746]
95 First, the hierarchy must be a tree (or adequately transformed into one as we did for SWA in this paper). [sent-375, score-0.374]
96 The proposed uniform entropy slice makes it plausible to provide an initial supervoxel map for further processing in problems like video object segmentation. [sent-378, score-1.277]
97 The feature criterion is independent of the supervoxel method. [sent-384, score-0.646]
98 In some respects, this fact is a clear benefit of the method, but it can also be considered a limitation: there is no guarantee that the uniform entropy slice is the optimal supervoxel segmentation for a given video and feature criterion. [sent-385, score-1.37]
99 In other words, since the supervoxel hierarchy is computed independent of the feature criterion, its segments may not coincide with the natural ones for a given criterion. [sent-386, score-0.814]
100 Our experiments demonstrate that for typical feature criteria this limitation is not critical, but further work is needed to better understand the induced error for a feature criterion-hierarchical supervoxel method pair. [sent-387, score-0.658]
wordName wordTfidf (topN-words)
[('supervoxel', 0.515), ('ues', 0.348), ('slice', 0.333), ('supervoxels', 0.295), ('hierarchy', 0.251), ('entropy', 0.21), ('gbh', 0.16), ('flattening', 0.151), ('tree', 0.123), ('hierarchies', 0.115), ('criterion', 0.105), ('levels', 0.104), ('swa', 0.103), ('uniform', 0.1), ('segmentation', 0.096), ('girl', 0.093), ('video', 0.09), ('sas', 0.087), ('coarser', 0.08), ('slices', 0.079), ('motion', 0.077), ('node', 0.077), ('finer', 0.075), ('criteria', 0.074), ('selection', 0.068), ('vs', 0.064), ('path', 0.062), ('dancers', 0.057), ('boxer', 0.056), ('swat', 0.056), ('flattened', 0.054), ('libsvx', 0.05), ('nodes', 0.042), ('ening', 0.042), ('sharon', 0.04), ('segtrack', 0.04), ('woman', 0.038), ('hierarchical', 0.036), ('drive', 0.036), ('level', 0.035), ('post', 0.034), ('vi', 0.033), ('seeks', 0.033), ('streaming', 0.032), ('hoc', 0.03), ('six', 0.029), ('plausible', 0.029), ('valid', 0.028), ('buffalo', 0.028), ('uesflateni', 0.028), ('unsupervised', 0.028), ('middle', 0.026), ('feature', 0.026), ('balance', 0.025), ('leftward', 0.025), ('parachute', 0.025), ('bivariate', 0.025), ('flatten', 0.025), ('leak', 0.025), ('segmentations', 0.025), ('moving', 0.025), ('dominant', 0.024), ('quadratic', 0.024), ('regions', 0.024), ('qp', 0.024), ('man', 0.023), ('merged', 0.023), ('leaking', 0.023), ('hunter', 0.023), ('bench', 0.023), ('principled', 0.023), ('superpixel', 0.022), ('segments', 0.022), ('focuses', 0.022), ('vt', 0.022), ('dark', 0.022), ('versatility', 0.022), ('px', 0.021), ('sel', 0.021), ('undersegmentation', 0.021), ('prefer', 0.02), ('supervised', 0.02), ('selected', 0.02), ('selects', 0.019), ('corso', 0.019), ('xs', 0.019), ('ok', 0.018), ('chairs', 0.018), ('videos', 0.018), ('coarse', 0.018), ('foreground', 0.018), ('pix', 0.018), ('oversegmentation', 0.018), ('drives', 0.018), ('benchmark', 0.018), ('trivial', 0.018), ('quantitative', 0.018), ('limitation', 0.017), ('early', 0.017), ('salient', 0.017)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999923 172 iccv-2013-Flattening Supervoxel Hierarchies by the Uniform Entropy Slice
Author: Chenliang Xu, Spencer Whitt, Jason J. Corso
Abstract: Supervoxel hierarchies provide a rich multiscale decomposition of a given video suitable for subsequent processing in video analysis. The hierarchies are typically computed by an unsupervised process that is susceptible to undersegmentation at coarse levels and over-segmentation at fine levels, which make it a challenge to adopt the hierarchies for later use. In this paper, we propose the first method to overcome this limitation and flatten the hierarchy into a single segmentation. Our method, called the uniform entropy slice, seeks a selection of supervoxels that balances the relative level of information in the selected supervoxels based on some post hoc feature criterion such as objectness. For example, with this criterion, in regions nearby objects, our method prefers finer supervoxels to capture the local details, but in regions away from any objects we prefer coarser supervoxels. We formulate the uniform entropy slice as a binary quadratic program and implement four different feature criteria, both unsupervised and supervised, to drive the flattening. Although we apply it only to supervoxel hierarchies in this paper, our method is generally applicable to segmentation tree hierarchies. Our experiments demonstrate both strong qualitative performance and superior quantitative performance to state of the art baselines on benchmark internet videos.
2 0.56362724 76 iccv-2013-Coarse-to-Fine Semantic Video Segmentation Using Supervoxel Trees
Author: Aastha Jain, Shuanak Chatterjee, René Vidal
Abstract: We propose an exact, general and efficient coarse-to-fine energy minimization strategy for semantic video segmentation. Our strategy is based on a hierarchical abstraction of the supervoxel graph that allows us to minimize an energy defined at the finest level of the hierarchy by minimizing a series of simpler energies defined over coarser graphs. The strategy is exact, i.e., it produces the same solution as minimizing over the finest graph. It is general, i.e., it can be used to minimize any energy function (e.g., unary, pairwise, and higher-order terms) with any existing energy minimization algorithm (e.g., graph cuts and belief propagation). It also gives significant speedups in inference for several datasets with varying degrees of spatio-temporal continuity. We also discuss the strengths and weaknesses of our strategy relative to existing hierarchical approaches, and the kinds of image and video data that provide the best speedups.
3 0.2277464 414 iccv-2013-Temporally Consistent Superpixels
Author: Matthias Reso, Jörn Jachalsky, Bodo Rosenhahn, Jörn Ostermann
Abstract: Superpixel algorithms represent a very useful and increasingly popular preprocessing step for a wide range of computer vision applications, as they offer the potential to boost efficiency and effectiveness. In this regards, this paper presents a highly competitive approach for temporally consistent superpixelsfor video content. The approach is based on energy-minimizing clustering utilizing a novel hybrid clustering strategy for a multi-dimensional feature space working in a global color subspace and local spatial subspaces. Moreover, a new contour evolution based strategy is introduced to ensure spatial coherency of the generated superpixels. For a thorough evaluation the proposed approach is compared to state of the art supervoxel algorithms using established benchmarks and shows a superior performance.
4 0.19345599 165 iccv-2013-Find the Best Path: An Efficient and Accurate Classifier for Image Hierarchies
Author: Min Sun, Wan Huang, Silvio Savarese
Abstract: Many methods have been proposed to solve the image classification problem for a large number of categories. Among them, methods based on tree-based representations achieve good trade-off between accuracy and test time efficiency. While focusing on learning a tree-shaped hierarchy and the corresponding set of classifiers, most of them [11, 2, 14] use a greedy prediction algorithm for test time efficiency. We argue that the dramatic decrease in accuracy at high efficiency is caused by the specific design choice of the learning and greedy prediction algorithms. In this work, we propose a classifier which achieves a better trade-off between efficiency and accuracy with a given tree-shaped hierarchy. First, we convert the classification problem as finding the best path in the hierarchy, and a novel branchand-bound-like algorithm is introduced to efficiently search for the best path. Second, we jointly train the classifiers using a novel Structured SVM (SSVM) formulation with additional bound constraints. As a result, our method achieves a significant 4.65%, 5.43%, and 4.07% (relative 24.82%, 41.64%, and 109.79%) improvement in accuracy at high efficiency compared to state-of-the-art greedy “tree-based” methods [14] on Caltech-256 [15], SUN [32] and ImageNet 1K [9] dataset, respectively. Finally, we show that our branch-and-bound-like algorithm naturally ranks the paths in the hierarchy (Fig. 8) so that users can further process them.
5 0.15628204 33 iccv-2013-A Unified Video Segmentation Benchmark: Annotation, Metrics and Analysis
Author: Fabio Galasso, Naveen Shankar Nagaraja, Tatiana Jiménez Cárdenas, Thomas Brox, Bernt Schiele
Abstract: Video segmentation research is currently limited by the lack of a benchmark dataset that covers the large variety of subproblems appearing in video segmentation and that is large enough to avoid overfitting. Consequently, there is little analysis of video segmentation which generalizes across subtasks, and it is not yet clear which and how video segmentation should leverage the information from the still-frames, as previously studied in image segmentation, alongside video specific information, such as temporal volume, motion and occlusion. In this work we provide such an analysis based on annotations of a large video dataset, where each video is manually segmented by multiple persons. Moreover, we introduce a new volume-based metric that includes the important aspect of temporal consistency, that can deal with segmentation hierarchies, and that reflects the tradeoff between over-segmentation and segmentation accuracy.
7 0.099763803 448 iccv-2013-Weakly Supervised Learning of Image Partitioning Using Decision Trees with Structured Split Criteria
8 0.093400143 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
9 0.088623852 263 iccv-2013-Measuring Flow Complexity in Videos
10 0.087663718 395 iccv-2013-Slice Sampling Particle Belief Propagation
11 0.087655768 299 iccv-2013-Online Video SEEDS for Temporal Window Objectness
12 0.077812009 78 iccv-2013-Coherent Motion Segmentation in Moving Camera Videos Using Optical Flow Orientations
13 0.072482064 282 iccv-2013-Multi-view Object Segmentation in Space and Time
14 0.0707644 447 iccv-2013-Volumetric Semantic Segmentation Using Pyramid Context Features
15 0.06888584 442 iccv-2013-Video Segmentation by Tracking Many Figure-Ground Segments
16 0.068032205 37 iccv-2013-Action Recognition and Localization by Hierarchical Space-Time Segments
17 0.067845285 432 iccv-2013-Uncertainty-Driven Efficiently-Sampled Sparse Graphical Models for Concurrent Tumor Segmentation and Atlas Registration
18 0.066404648 81 iccv-2013-Combining the Right Features for Complex Event Recognition
19 0.065290019 379 iccv-2013-Semantic Segmentation without Annotating Segments
20 0.064910382 429 iccv-2013-Tree Shape Priors with Connectivity Constraints Using Convex Relaxation on General Graphs
topicId topicWeight
[(0, 0.149), (1, -0.009), (2, 0.048), (3, 0.04), (4, 0.061), (5, 0.072), (6, -0.086), (7, 0.088), (8, 0.057), (9, -0.098), (10, -0.052), (11, 0.087), (12, 0.062), (13, 0.066), (14, -0.019), (15, 0.092), (16, -0.078), (17, -0.117), (18, -0.111), (19, 0.016), (20, 0.01), (21, -0.117), (22, -0.04), (23, 0.057), (24, -0.224), (25, -0.118), (26, -0.028), (27, -0.029), (28, -0.078), (29, -0.048), (30, 0.176), (31, -0.152), (32, -0.144), (33, -0.102), (34, 0.157), (35, -0.254), (36, 0.162), (37, -0.115), (38, 0.01), (39, 0.059), (40, -0.013), (41, -0.122), (42, 0.024), (43, -0.173), (44, 0.09), (45, 0.081), (46, -0.147), (47, 0.016), (48, -0.136), (49, 0.031)]
simIndex simValue paperId paperTitle
same-paper 1 0.92307806 172 iccv-2013-Flattening Supervoxel Hierarchies by the Uniform Entropy Slice
Author: Chenliang Xu, Spencer Whitt, Jason J. Corso
Abstract: Supervoxel hierarchies provide a rich multiscale decomposition of a given video suitable for subsequent processing in video analysis. The hierarchies are typically computed by an unsupervised process that is susceptible to undersegmentation at coarse levels and over-segmentation at fine levels, which make it a challenge to adopt the hierarchies for later use. In this paper, we propose the first method to overcome this limitation and flatten the hierarchy into a single segmentation. Our method, called the uniform entropy slice, seeks a selection of supervoxels that balances the relative level of information in the selected supervoxels based on some post hoc feature criterion such as objectness. For example, with this criterion, in regions nearby objects, our method prefers finer supervoxels to capture the local details, but in regions away from any objects we prefer coarser supervoxels. We formulate the uniform entropy slice as a binary quadratic program and implement four different feature criteria, both unsupervised and supervised, to drive the flattening. Although we apply it only to supervoxel hierarchies in this paper, our method is generally applicable to segmentation tree hierarchies. Our experiments demonstrate both strong qualitative performance and superior quantitative performance to state of the art baselines on benchmark internet videos.
2 0.89330816 76 iccv-2013-Coarse-to-Fine Semantic Video Segmentation Using Supervoxel Trees
Author: Aastha Jain, Shuanak Chatterjee, René Vidal
Abstract: We propose an exact, general and efficient coarse-to-fine energy minimization strategy for semantic video segmentation. Our strategy is based on a hierarchical abstraction of the supervoxel graph that allows us to minimize an energy defined at the finest level of the hierarchy by minimizing a series of simpler energies defined over coarser graphs. The strategy is exact, i.e., it produces the same solution as minimizing over the finest graph. It is general, i.e., it can be used to minimize any energy function (e.g., unary, pairwise, and higher-order terms) with any existing energy minimization algorithm (e.g., graph cuts and belief propagation). It also gives significant speedups in inference for several datasets with varying degrees of spatio-temporal continuity. We also discuss the strengths and weaknesses of our strategy relative to existing hierarchical approaches, and the kinds of image and video data that provide the best speedups.
3 0.60948133 414 iccv-2013-Temporally Consistent Superpixels
Author: Matthias Reso, Jörn Jachalsky, Bodo Rosenhahn, Jörn Ostermann
Abstract: Superpixel algorithms represent a very useful and increasingly popular preprocessing step for a wide range of computer vision applications, as they offer the potential to boost efficiency and effectiveness. In this regards, this paper presents a highly competitive approach for temporally consistent superpixelsfor video content. The approach is based on energy-minimizing clustering utilizing a novel hybrid clustering strategy for a multi-dimensional feature space working in a global color subspace and local spatial subspaces. Moreover, a new contour evolution based strategy is introduced to ensure spatial coherency of the generated superpixels. For a thorough evaluation the proposed approach is compared to state of the art supervoxel algorithms using established benchmarks and shows a superior performance.
4 0.47510418 299 iccv-2013-Online Video SEEDS for Temporal Window Objectness
Author: Michael Van_Den_Bergh, Gemma Roig, Xavier Boix, Santiago Manen, Luc Van_Gool
Abstract: Superpixel and objectness algorithms are broadly used as a pre-processing step to generate support regions and to speed-up further computations. Recently, many algorithms have been extended to video in order to exploit the temporal consistency between frames. However, most methods are computationally too expensive for real-time applications. We introduce an online, real-time video superpixel algorithm based on the recently proposed SEEDS superpixels. A new capability is incorporated which delivers multiple diverse samples (hypotheses) of superpixels in the same image or video sequence. The multiple samples are shown to provide a strong cue to efficiently measure the objectness of image windows, and we introduce the novel concept of objectness in temporal windows. Experiments show that the video superpixels achieve comparable performance to state-of-the-art offline methods while running at 30 fps on a single 2.8 GHz i7 CPU. State-of-the-art performance on objectness is also demonstrated, yet orders of magnitude faster and extended to temporal windows in video.
5 0.46192956 33 iccv-2013-A Unified Video Segmentation Benchmark: Annotation, Metrics and Analysis
Author: Fabio Galasso, Naveen Shankar Nagaraja, Tatiana Jiménez Cárdenas, Thomas Brox, Bernt Schiele
Abstract: Video segmentation research is currently limited by the lack of a benchmark dataset that covers the large variety of subproblems appearing in video segmentation and that is large enough to avoid overfitting. Consequently, there is little analysis of video segmentation which generalizes across subtasks, and it is not yet clear which and how video segmentation should leverage the information from the still-frames, as previously studied in image segmentation, alongside video specific information, such as temporal volume, motion and occlusion. In this work we provide such an analysis based on annotations of a large video dataset, where each video is manually segmented by multiple persons. Moreover, we introduce a new volume-based metric that includes the important aspect of temporal consistency, that can deal with segmentation hierarchies, and that reflects the tradeoff between over-segmentation and segmentation accuracy.
6 0.38716903 165 iccv-2013-Find the Best Path: An Efficient and Accurate Classifier for Image Hierarchies
8 0.34290296 145 iccv-2013-Estimating the Material Properties of Fabric from Video
9 0.33550602 447 iccv-2013-Volumetric Semantic Segmentation Using Pyramid Context Features
10 0.33164221 274 iccv-2013-Monte Carlo Tree Search for Scheduling Activity Recognition
11 0.31630975 176 iccv-2013-From Large Scale Image Categorization to Entry-Level Categories
13 0.31187865 395 iccv-2013-Slice Sampling Particle Belief Propagation
14 0.3094112 282 iccv-2013-Multi-view Object Segmentation in Space and Time
15 0.30418584 42 iccv-2013-Active MAP Inference in CRFs for Efficient Semantic Segmentation
16 0.28787261 448 iccv-2013-Weakly Supervised Learning of Image Partitioning Using Decision Trees with Structured Split Criteria
17 0.28205949 329 iccv-2013-Progressive Multigrid Eigensolvers for Multiscale Spectral Segmentation
18 0.27084091 443 iccv-2013-Video Synopsis by Heterogeneous Multi-source Correlation
19 0.27006626 128 iccv-2013-Dynamic Probabilistic Volumetric Models
20 0.26761791 110 iccv-2013-Detecting Curved Symmetric Parts Using a Deformable Disc Model
topicId topicWeight
[(2, 0.071), (7, 0.014), (12, 0.012), (17, 0.26), (26, 0.11), (31, 0.022), (35, 0.014), (40, 0.027), (42, 0.073), (48, 0.01), (64, 0.063), (73, 0.027), (89, 0.164), (98, 0.026)]
simIndex simValue paperId paperTitle
same-paper 1 0.76909977 172 iccv-2013-Flattening Supervoxel Hierarchies by the Uniform Entropy Slice
Author: Chenliang Xu, Spencer Whitt, Jason J. Corso
Abstract: Supervoxel hierarchies provide a rich multiscale decomposition of a given video suitable for subsequent processing in video analysis. The hierarchies are typically computed by an unsupervised process that is susceptible to undersegmentation at coarse levels and over-segmentation at fine levels, which make it a challenge to adopt the hierarchies for later use. In this paper, we propose the first method to overcome this limitation and flatten the hierarchy into a single segmentation. Our method, called the uniform entropy slice, seeks a selection of supervoxels that balances the relative level of information in the selected supervoxels based on some post hoc feature criterion such as objectness. For example, with this criterion, in regions nearby objects, our method prefers finer supervoxels to capture the local details, but in regions away from any objects we prefer coarser supervoxels. We formulate the uniform entropy slice as a binary quadratic program and implement four different feature criteria, both unsupervised and supervised, to drive the flattening. Although we apply it only to supervoxel hierarchies in this paper, our method is generally applicable to segmentation tree hierarchies. Our experiments demonstrate both strong qualitative performance and superior quantitative performance to state of the art baselines on benchmark internet videos.
2 0.67187238 343 iccv-2013-Real-World Normal Map Capture for Nearly Flat Reflective Surfaces
Author: Bastien Jacquet, Christian Häne, Kevin Köser, Marc Pollefeys
Abstract: Although specular objects have gained interest in recent years, virtually no approaches exist for markerless reconstruction of reflective scenes in the wild. In this work, we present a practical approach to capturing normal maps in real-world scenes using video only. We focus on nearly planar surfaces such as windows, facades from glass or metal, or frames, screens and other indoor objects and show how normal maps of these can be obtained without the use of an artificial calibration object. Rather, we track the reflections of real-world straight lines, while moving with a hand-held or vehicle-mounted camera in front of the object. In contrast to error-prone local edge tracking, we obtain the reflections by a robust, global segmentation technique of an ortho-rectified 3D video cube that also naturally allows efficient user interaction. Then, at each point of the reflective surface, the resulting 2D-curve to 3D-line correspondence provides a novel quadratic constraint on the local surface normal. This allows to globally solve for the shape by integrability and smoothness constraints and easily supports the usage of multiple lines. We demonstrate the technique on several objects and facades.
3 0.66872787 233 iccv-2013-Latent Task Adaptation with Large-Scale Hierarchies
Author: Yangqing Jia, Trevor Darrell
Abstract: Recent years have witnessed the success of large-scale image classification systems that are able to identify objects among thousands of possible labels. However, it is yet unclear how general classifiers such as ones trained on ImageNet can be optimally adapted to specific tasks, each of which only covers a semantically related subset of all the objects in the world. It is inefficient and suboptimal to retrain classifiers whenever a new task is given, and is inapplicable when tasks are not given explicitly, but implicitly specified as a set of image queries. In this paper we propose a novel probabilistic model that jointly identifies the underlying task and performs prediction with a lineartime probabilistic inference algorithm, given a set of query images from a latent task. We present efficient ways to estimate parameters for the model, and an open-source toolbox to train classifiers distributedly at a large scale. Empirical results based on the ImageNet data showed significant performance increase over several baseline algorithms.
4 0.65875244 414 iccv-2013-Temporally Consistent Superpixels
Author: Matthias Reso, Jörn Jachalsky, Bodo Rosenhahn, Jörn Ostermann
Abstract: Superpixel algorithms represent a very useful and increasingly popular preprocessing step for a wide range of computer vision applications, as they offer the potential to boost efficiency and effectiveness. In this regards, this paper presents a highly competitive approach for temporally consistent superpixelsfor video content. The approach is based on energy-minimizing clustering utilizing a novel hybrid clustering strategy for a multi-dimensional feature space working in a global color subspace and local spatial subspaces. Moreover, a new contour evolution based strategy is introduced to ensure spatial coherency of the generated superpixels. For a thorough evaluation the proposed approach is compared to state of the art supervoxel algorithms using established benchmarks and shows a superior performance.
5 0.65168792 160 iccv-2013-Fast Object Segmentation in Unconstrained Video
Author: Anestis Papazoglou, Vittorio Ferrari
Abstract: We present a technique for separating foreground objects from the background in a video. Our method isfast, , fully automatic, and makes minimal assumptions about the video. This enables handling essentially unconstrained settings, including rapidly moving background, arbitrary object motion and appearance, and non-rigid deformations and articulations. In experiments on two datasets containing over 1400 video shots, our method outperforms a state-of-theart background subtraction technique [4] as well as methods based on clustering point tracks [6, 18, 19]. Moreover, it performs comparably to recent video object segmentation methods based on objectproposals [14, 16, 27], while being orders of magnitude faster.
6 0.6488992 150 iccv-2013-Exemplar Cut
7 0.64861709 95 iccv-2013-Cosegmentation and Cosketch by Unsupervised Learning
8 0.64818871 411 iccv-2013-Symbiotic Segmentation and Part Localization for Fine-Grained Categorization
9 0.64744633 396 iccv-2013-Space-Time Robust Representation for Action Recognition
10 0.6472826 326 iccv-2013-Predicting Sufficient Annotation Strength for Interactive Foreground Segmentation
11 0.64715016 31 iccv-2013-A Unified Probabilistic Approach Modeling Relationships between Attributes and Objects
12 0.6455068 8 iccv-2013-A Deformable Mixture Parsing Model with Parselets
13 0.64407516 102 iccv-2013-Data-Driven 3D Primitives for Single Image Understanding
14 0.6431945 379 iccv-2013-Semantic Segmentation without Annotating Segments
15 0.64291859 299 iccv-2013-Online Video SEEDS for Temporal Window Objectness
16 0.64231062 404 iccv-2013-Structured Forests for Fast Edge Detection
17 0.64137483 107 iccv-2013-Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction
18 0.6412459 196 iccv-2013-Hierarchical Data-Driven Descent for Efficient Optimal Deformation Estimation
19 0.64101583 442 iccv-2013-Video Segmentation by Tracking Many Figure-Ground Segments
20 0.64084071 21 iccv-2013-A Method of Perceptual-Based Shape Decomposition