cvpr cvpr2013 cvpr2013-425 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Heesoo Myeong, Kyoung Mu Lee
Abstract: We propose a novel nonparametric approach for semantic segmentation using high-order semantic relations. Conventional context models mainly focus on learning pairwise relationships between objects. Pairwise relations, however, are not enough to represent high-level contextual knowledge within images. In this paper, we propose semantic relation transfer, a method to transfer high-order semantic relations of objects from annotated images to unlabeled images analogous to label transfer techniques where label information are transferred. Wefirst define semantic tensors representing high-order relations of objects. Semantic relation transfer problem is then formulated as semi-supervised learning using a quadratic objective function of the semantic tensors. By exploiting low-rank property of the semantic tensors and employing Kronecker sum similarity, an efficient approximation algorithm is developed. Based on the predicted high-order semantic relations, we reason semantic segmentation and evaluate the performance on several challenging datasets.
Reference: text
sentIndex sentText sentNum sentScore
1 http : / / cv Abstract We propose a novel nonparametric approach for semantic segmentation using high-order semantic relations. [sent-5, score-1.242]
2 Conventional context models mainly focus on learning pairwise relationships between objects. [sent-6, score-0.174]
3 In this paper, we propose semantic relation transfer, a method to transfer high-order semantic relations of objects from annotated images to unlabeled images analogous to label transfer techniques where label information are transferred. [sent-8, score-2.134]
4 Wefirst define semantic tensors representing high-order relations of objects. [sent-9, score-0.916]
5 Semantic relation transfer problem is then formulated as semi-supervised learning using a quadratic objective function of the semantic tensors. [sent-10, score-1.012]
6 By exploiting low-rank property of the semantic tensors and employing Kronecker sum similarity, an efficient approximation algorithm is developed. [sent-11, score-0.715]
7 Based on the predicted high-order semantic relations, we reason semantic segmentation and evaluate the performance on several challenging datasets. [sent-12, score-1.171]
8 Recently, with the increasing availability of large image collections of hand-labeled images, nonparametric label transfer approaches for this problem have attracted many computer vision researchers and shows very good performance [2, 3, 16, 23, 24, 25, 26]. [sent-15, score-0.432]
9 Compared to conventional parametric semantic segmentation methods [1, 6, 14, 22], these approaches do not need training model parameters, hence, they can be scalable to large datasets with an unknown number ofobject categories. [sent-16, score-0.628]
10 Typical label transfer approaches start by retrieving similar images for a given test image. [sent-17, score-0.296]
11 For a query image (a), our system finds the matched similar images (b) from a large dataset using global scene descriptors. [sent-22, score-0.203]
12 The high-order semantic relations are transferred from the annotated images (b) to the query image (a). [sent-23, score-0.966]
13 (We densely estimate high-order semantic relation across the image, but this figure displays only a few top scored relations for visualization purposes. [sent-24, score-0.986]
14 ) We then infer semantic segmentation (d) using estimated semantic relation (c). [sent-25, score-1.289]
15 Obviously, high-level semantic relationships between objects within the annotated image are very important cues to successful semantic segmentation. [sent-28, score-1.156]
16 To this end, recent approaches have advocated the use of nonparametric context models [10, 19]. [sent-29, score-0.185]
17 However, these methods use only pairwise relationships to model high-level semantic rela333000777311 tionships. [sent-31, score-0.635]
18 Since natural images typically contain more than three object categories, pairwise relations are not enough to represents high-level information within images. [sent-32, score-0.3]
19 In this paper, we develop a novel nonparametric approach for semantic segmentation by incorporating highorder semantic relations. [sent-33, score-1.341]
20 Specifically, similar to several label transfer methods [3, 16, 23, 25], we first find a set of small retrieved images from training images. [sent-34, score-0.425]
21 Our goal is to transfer high-order semantic relations of annotated objects from each matched image to the query image. [sent-35, score-1.217]
22 Since it is not feasible to obtain dense pixel-wise high-order semantic relations, we utilize “superpixel” regions obtained by oversegmentation of the query image. [sent-36, score-0.695]
23 We define semantic tensors to represent the higher-order semantic relations of regions. [sent-37, score-1.426]
24 We approach the problem of transferring the high-order semantic relations by defining a quadratic objective function of the semantic tensors. [sent-38, score-1.39]
25 To optimize our objective function, we develop an efficient approximate algorithm based on Kronecker sum similarity and low-rank property of semantic tensors. [sent-39, score-0.695]
26 To integrate our predicted semantic tensor into a semantic segmentation system, a fully connected Markov random field optimization is employed. [sent-40, score-1.349]
27 In Section 3, we introduce highorder semantic relation transfer algorithm and explain in detail. [sent-44, score-0.987]
28 3 presents a semantic segmentation method through semantic relation transfer. [sent-46, score-1.289]
29 Related work We now review related works on label transfer approaches and nonparametric context models. [sent-50, score-0.481]
30 The problem of label transfer was first addressed recently by Liu et al. [sent-51, score-0.296]
31 They first retrieved similar images using GIST matching [20] and constructed pixel-wise dense correspondence between each retrieved image and test image using SIFT flow [17]. [sent-53, score-0.315]
32 They then transferred the annotations based on dense correspondence and reasoned semantic segmentation. [sent-54, score-0.592]
33 Following the idea of label transfer [16], Zhang et al. [sent-55, score-0.296]
34 Tighe and Lazebnik [23, 24] considered superpixel-level matching to transfer label informa- (a)Pbcauirlwdinsge manticrelationbcaurildng(rboa)dThird-o ersmanctri elationrskady Figure 2. [sent-60, score-0.296]
35 The third-order semantic relations (b) can model complicated high-level semantic knowledges within an image compared with the pairwise semantic relation (a). [sent-62, score-2.013]
36 [16] claimed that the label transfer approach naturally embeds contextual information in the retrieval/alignment procedure, it is hard to tell how much contextual knowledge will help or what the effects will be. [sent-66, score-0.482]
37 On the other hand, recent nonparametric context models [10, 19] for semantic segmentation employed contextual relationships between objects to achieve more accurate results. [sent-67, score-0.936]
38 [10] learned which contextual relationships should be considered and predicted features weight for each relation in a nonparametric manner. [sent-69, score-0.539]
39 Since our semantic tensor can be viewed as a generalization of the context link [19], their work is most similar to our own. [sent-72, score-0.771]
40 On the contrary, our method focuses on high-order (mostly third-order) semantic relations, allowing us to model complex contextual relationships. [sent-75, score-0.603]
41 For example, triplet-wise semantic relations can be found such as (sky,car,road) by our method as illustrated in Figure 2. [sent-76, score-0.747]
42 These relations become important when considering complicated scenes with many object classes. [sent-77, score-0.237]
43 Second, we develop a quadratic objective function for the high-order semantic relation transfer problem. [sent-78, score-1.045]
44 High-order models are not well studied in the context of semantic segmentation. [sent-81, score-0.559]
45 However, their high-order model is not related to high-level semantic knowledge. [sent-84, score-0.51]
46 To our knowledge, there are no prior works explicitly considering high-order contextual relationships between objects in the literature on semantic segmentation. [sent-85, score-0.665]
47 The high-order algorithm semantic relation transfer × × 3. [sent-87, score-0.921]
48 We define high-order semantic relation transfer problem as a task to predict high-order relation between unlabeled regions in I1based on annotated regions in I2. [sent-91, score-1.276]
49 For simplicity, we will focus on third-order relations from now. [sent-92, score-0.237]
50 hir Cd- =ord {ecr semantic re}la i-s tions among region triplets (si, sj , sk) ∈ S S S is defined as a set of N N N third)-o∈r de Sr t×e n sSor ×s X S = is d{eXfi1n1e1d, X as11 a2, s eXt1 1of3, . [sent-100, score-0.796]
51 i Aon ss among region triplets on object class triplet (cα , cβ , cγ). [sent-110, score-0.277]
52 xiαjβkγ The variable indicates confidence score of how likely the region triplet (si , sj , sk) would be labeled as (cα , cβ , cγ), respectively. [sent-112, score-0.391]
53 , YKKK}, andrepresenteachelement o{fY Yαβ,γY as yiαjβkγ=⎨⎧01 oi(ftsh Gie,r(s wji),is e =k) c ∈α, SG2(sj) = cβ,G(sk) = cγ, , (2) where G⎩(si) denotes the ground truth class of region si and (si , sj , sk) ∈ S2 indicates that three regions si, sj, and sk are from t)he ∈ same image I2. [sent-119, score-0.483]
54 Since there are no semantic relations within S1 and across images, all yiαjβkγ is 0 for (si , sj , sk) ∈/ S2. [sent-120, score-0.877]
55 Objective function Now, the third-order semantic relation transfer problem (tcuasidne,sbojef,srekcgo)an frndi e dnfocaresalstcho erbpejs roctxbliαcejlβakmγs fot rfipelaseltism (uacptαien,rgpcβixt,hec lγtm)rbiap glsen tids- on Y. [sent-133, score-0.921]
56 We assume that there is no interaction between the semantic tensors. [sent-134, score-0.51]
57 Hence, we separately deal with the thirdorder semantic relations transfer problem with respect to Yαβγ. [sent-135, score-1.002]
58 Following the idea of link propagation [11], we want to enforce that two similar region triplets are likely to have the same confidence score. [sent-137, score-0.317]
59 ,j,k where wijk,lmn is the triplet-wise similarity between two region triplets (si, sj , sk) and (sl , sm, sn) and > 0 is the regularization parameter. [sent-143, score-0.319]
60 (5) is the λ × continuity constraint that two triplets should have the same confidence score if two triplets are similar. [sent-145, score-0.245]
61 The second term is the unary constraint that each region triplet xijk tends to have their target values yijk. [sent-146, score-0.309]
62 The cost function defined as pairwise and unary terms is a generalization of the cost function for label propagation [27]. [sent-147, score-0.202]
63 e expressed as F(X) =12vec(X)Tvec(X ×1LS+ X ×2LS+ X ×3LS) + λ(vec(X) − vec(Y))2 , (11) × where represents n-mode product of tensor [13]. [sent-171, score-0.193]
64 (12) (13) (14) That is, we sequentially estimate the semantic tensor for each mode product term. [sent-173, score-0.703]
65 The algorithm (b) first find similar region sl with respect to si while fixing sj and sk, (c) then find similar region sm with respect to sj while fixing sl and sk, (d) and finally find similar region sn with respect to sk while fixing sl and sm. [sent-195, score-0.993]
66 We predict the third-order semantic relations (d) by transferring semantic relations from each annotated image (c) to the query image (a). [sent-220, score-1.73]
67 We aggregate semantic relations (e) from multiple semantic relation candidates (d) and generate semantic segmentation (f). [sent-221, score-2.063]
68 Semantic segmentation through semantic relation transfer Now that we have the semantic relation transfer algorithm from annotated images to unlabeled images, we can infer semantic segmentation using estimated semantic tensors. [sent-224, score-3.136]
69 Hence, it is essential to to extract closely-related images from large dataset with respect to a query image for successful semantic relation transfer. [sent-228, score-0.84]
70 Unreliable semantic tensors can be predicted between two unrelated images. [sent-229, score-0.744]
71 This candidate image set will be used to transfer its high-order semantic relations into the query image. [sent-231, score-1.12]
72 1, we transfer high-order semantic relations from each candidate image to the query image and obtain multiple sets of predicted semantic tensors {X}u=1:M. [sent-235, score-1.864]
73 Our goal is to assign object c selamssa fnotirc ce taecnhs region i}n the query image. [sent-236, score-0.179]
74 To integrate the sets of predicted semantic tensors with a conventional unary and pairwise potential, we build high-order fully connected Markov random field model. [sent-237, score-0.9]
75 ∈Sin {ce1, we Kwa}n tis t toh ela ibnedl txhe o regions cinl sthse query image, the energy function is only defined on the regions of image I1. [sent-250, score-0.24]
76 These two terms are typically used to conventional nonparametric scene parsing approaches [16, 23, 24]. [sent-253, score-0.255]
77 However, it is nontrivial how to integrate the sets of predicted semantic tensors to semantic segmentation framework. [sent-254, score-1.373]
78 (23) The first clique potential EmHax take maximum confidence score among M number of candidate scores for region triplet (si, sj , sk) and for object triplet (cα , cβ, cγ). [sent-257, score-0.633]
79 This means that we only consider the strongest one from the set of relation candidates. [sent-258, score-0.183]
80 u (24) Meanwhile, the second clique potential EmHax takes summation of M number of confidence scores. [sent-266, score-0.177]
81 This potential picks average scores from the set of relation candidates. [sent-267, score-0.247]
82 Experiements In this section, we (1) evaluate our method’s semantic segmentation performance and compare against pairwise semantic segmentation [19] and (2) analyze integration of our predicted semantic tensors. [sent-272, score-1.83]
83 Table 1 summarizes our semantic segmentation accuracy compared with the state-of-the-art methods. [sent-276, score-0.596]
84 Proposed (max) indicates the accuracy of the semantic segmentation with the max high-order term Eq. [sent-277, score-0.651]
85 Recognition rate of two different high-order potential as a function of the number of the retrieved images M on the LMO dataset. [sent-287, score-0.193]
86 To compute ED, we employ the nonparametric superpixel parsing [23] for the LMO dataset and the boosted decision tree classifier [8] for the other datasets. [sent-297, score-0.188]
87 The semantic segmentation accuracy on this dataset is 81. [sent-308, score-0.596]
88 This is relatively good dataset to evaluate high-order semantic relations. [sent-310, score-0.51]
89 The semantic segmentation accuracy of the proposed method on this dataset is 76. [sent-320, score-0.596]
90 We design two different high-order potential for incorporating the set of the predicted semantic tensors. [sent-336, score-0.639]
91 As shown in Figure 5, sum potential, taking summarization of candidates confidence scores, provides more better semantic segmentation results at some point. [sent-337, score-0.71]
92 On the other hand, max potential, taking maximum of candidates confidence scores, is more robust to the number of retrieved images M. [sent-338, score-0.207]
93 As gradually adding retrieved images, wrong matched images become larger and the performance of sum potential decreases faster. [sent-339, score-0.277]
94 Conclusion We have presented a novel approach to learn high-order semantic relations of regions in a nonparametric manner. [sent-341, score-0.918]
95 We cast the high-order semantic relation transfer problem as a quadratic objective function of semantic tensors and propose an efficient approximate algorithm. [sent-342, score-1.722]
96 We develop a novel semantic tensor representation of the high-order semantic relations. [sent-343, score-1.198]
97 While we have presented this representation in the context of semantic segmentation, it can be applicable to various computer vision problem including object detection, scene classification, and total scene understanding. [sent-344, score-0.629]
98 Nonparametric scene parsing: Label transfer via dense scene alignment. [sent-469, score-0.328]
99 Partial similarity based nonparametric scene parsing in certain environment. [sent-539, score-0.256]
100 Supervised label transfer for semantic segmentation of street scenes. [sent-545, score-0.892]
wordName wordTfidf (topN-words)
[('semantic', 0.51), ('vec', 0.262), ('relations', 0.237), ('transfer', 0.228), ('relation', 0.183), ('tensors', 0.169), ('ls', 0.162), ('tensor', 0.145), ('emhax', 0.142), ('nonparametric', 0.136), ('sj', 0.13), ('retrieved', 0.129), ('myeong', 0.126), ('sk', 0.124), ('triplet', 0.121), ('query', 0.12), ('lmo', 0.11), ('si', 0.105), ('jik', 0.101), ('triplets', 0.097), ('contextual', 0.093), ('labelme', 0.092), ('tvec', 0.088), ('jain', 0.088), ('segmentation', 0.086), ('ws', 0.079), ('xijk', 0.076), ('tighe', 0.075), ('annotated', 0.074), ('label', 0.068), ('link', 0.067), ('kronecker', 0.066), ('highorder', 0.066), ('polo', 0.066), ('predicted', 0.065), ('potential', 0.064), ('pairwise', 0.063), ('clique', 0.062), ('relationships', 0.062), ('region', 0.059), ('eshum', 0.057), ('ak', 0.053), ('parsing', 0.052), ('objective', 0.052), ('confidence', 0.051), ('patchmatchgraph', 0.05), ('context', 0.049), ('sl', 0.048), ('matched', 0.048), ('product', 0.048), ('gould', 0.047), ('propagation', 0.043), ('transferring', 0.042), ('seoul', 0.042), ('vee', 0.04), ('quadratic', 0.039), ('gist', 0.039), ('sum', 0.036), ('korea', 0.036), ('scene', 0.035), ('regions', 0.035), ('fixing', 0.034), ('similarity', 0.033), ('integrate', 0.033), ('develop', 0.033), ('hence', 0.032), ('conventional', 0.032), ('approximate', 0.031), ('scored', 0.03), ('lj', 0.03), ('outdoor', 0.03), ('lazebnik', 0.03), ('diagonal', 0.03), ('indicates', 0.03), ('dense', 0.03), ('yi', 0.03), ('yuen', 0.029), ('kohli', 0.029), ('unlabeled', 0.028), ('unary', 0.028), ('correspondence', 0.027), ('candidates', 0.027), ('respect', 0.027), ('layout', 0.027), ('lk', 0.026), ('outer', 0.026), ('densely', 0.026), ('transferred', 0.025), ('candidate', 0.025), ('term', 0.025), ('liu', 0.025), ('muax', 0.025), ('eisc', 0.025), ('creative', 0.025), ('wisi', 0.025), ('sthse', 0.025), ('unfolded', 0.025), ('rela', 0.025), ('xns', 0.025), ('cinl', 0.025)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 425 cvpr-2013-Tensor-Based High-Order Semantic Relation Transfer for Semantic Scene Segmentation
Author: Heesoo Myeong, Kyoung Mu Lee
Abstract: We propose a novel nonparametric approach for semantic segmentation using high-order semantic relations. Conventional context models mainly focus on learning pairwise relationships between objects. Pairwise relations, however, are not enough to represent high-level contextual knowledge within images. In this paper, we propose semantic relation transfer, a method to transfer high-order semantic relations of objects from annotated images to unlabeled images analogous to label transfer techniques where label information are transferred. Wefirst define semantic tensors representing high-order relations of objects. Semantic relation transfer problem is then formulated as semi-supervised learning using a quadratic objective function of the semantic tensors. By exploiting low-rank property of the semantic tensors and employing Kronecker sum similarity, an efficient approximation algorithm is developed. Based on the predicted high-order semantic relations, we reason semantic segmentation and evaluate the performance on several challenging datasets.
2 0.31645396 309 cvpr-2013-Nonparametric Scene Parsing with Adaptive Feature Relevance and Semantic Context
Author: Gautam Singh, Jana Kosecka
Abstract: This paper presents a nonparametric approach to semantic parsing using small patches and simple gradient, color and location features. We learn the relevance of individual feature channels at test time using a locally adaptive distance metric. To further improve the accuracy of the nonparametric approach, we examine the importance of the retrieval set used to compute the nearest neighbours using a novel semantic descriptor to retrieve better candidates. The approach is validated by experiments on several datasets used for semantic parsing demonstrating the superiority of the method compared to the state of art approaches.
3 0.20466484 146 cvpr-2013-Enriching Texture Analysis with Semantic Data
Author: Tim Matthews, Mark S. Nixon, Mahesan Niranjan
Abstract: We argue for the importance of explicit semantic modelling in human-centred texture analysis tasks such as retrieval, annotation, synthesis, and zero-shot learning. To this end, low-level attributes are selected and used to define a semantic space for texture. 319 texture classes varying in illumination and rotation are positioned within this semantic space using a pairwise relative comparison procedure. Low-level visual features used by existing texture descriptors are then assessed in terms of their correspondence to the semantic space. Textures with strong presence ofattributes connoting randomness and complexity are shown to be poorly modelled by existing descriptors. In a retrieval experiment semantic descriptors are shown to outperform visual descriptors. Semantic modelling of texture is thus shown to provide considerable value in both feature selection and in analysis tasks.
4 0.15021476 460 cvpr-2013-Weakly-Supervised Dual Clustering for Image Semantic Segmentation
Author: Yang Liu, Jing Liu, Zechao Li, Jinhui Tang, Hanqing Lu
Abstract: In this paper, we propose a novel Weakly-Supervised Dual Clustering (WSDC) approach for image semantic segmentation with image-level labels, i.e., collaboratively performing image segmentation and tag alignment with those regions. The proposed approach is motivated from the observation that superpixels belonging to an object class usually exist across multiple images and hence can be gathered via the idea of clustering. In WSDC, spectral clustering is adopted to cluster the superpixels obtained from a set of over-segmented images. At the same time, a linear transformation between features and labels as a kind of discriminative clustering is learned to select the discriminative features among different classes. The both clustering outputs should be consistent as much as possible. Besides, weakly-supervised constraints from image-level labels are imposed to restrict the labeling of superpixels. Finally, the non-convex and non-smooth objective function are efficiently optimized using an iterative CCCP procedure. Extensive experiments conducted on MSRC andLabelMe datasets demonstrate the encouraging performance of our method in comparison with some state-of-the-arts.
5 0.14926861 242 cvpr-2013-Label Propagation from ImageNet to 3D Point Clouds
Author: Yan Wang, Rongrong Ji, Shih-Fu Chang
Abstract: Recent years have witnessed a growing interest in understanding the semantics of point clouds in a wide variety of applications. However, point cloud labeling remains an open problem, due to the difficulty in acquiring sufficient 3D point labels towards training effective classifiers. In this paper, we overcome this challenge by utilizing the existing massive 2D semantic labeled datasets from decadelong community efforts, such as ImageNet and LabelMe, and a novel “cross-domain ” label propagation approach. Our proposed method consists of two major novel components, Exemplar SVM based label propagation, which effectively addresses the cross-domain issue, and a graphical model based contextual refinement incorporating 3D constraints. Most importantly, the entire process does not require any training data from the target scenes, also with good scalability towards large scale applications. We evaluate our approach on the well-known Cornell Point Cloud Dataset, achieving much greater efficiency and comparable accuracy even without any 3D training data. Our approach shows further major gains in accuracy when the training data from the target scenes is used, outperforming state-ofthe-art approaches with far better efficiency.
6 0.1488438 73 cvpr-2013-Bringing Semantics into Focus Using Visual Abstraction
7 0.14735027 43 cvpr-2013-Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs
8 0.13240816 179 cvpr-2013-From N to N+1: Multiclass Transfer Incremental Learning
9 0.12794256 301 cvpr-2013-Multi-target Tracking by Rank-1 Tensor Approximation
10 0.12758242 107 cvpr-2013-Deformable Spatial Pyramid Matching for Fast Dense Correspondences
11 0.12357303 173 cvpr-2013-Finding Things: Image Parsing with Regions and Per-Exemplar Detectors
12 0.12203736 341 cvpr-2013-Procrustean Normal Distribution for Non-rigid Structure from Motion
13 0.11791138 165 cvpr-2013-Fast Energy Minimization Using Learned State Filters
14 0.11504539 446 cvpr-2013-Understanding Indoor Scenes Using 3D Geometric Phrases
15 0.11435012 325 cvpr-2013-Part Discovery from Partial Correspondence
16 0.11029638 247 cvpr-2013-Learning Class-to-Image Distance with Object Matchings
17 0.10925123 284 cvpr-2013-Mesh Based Semantic Modelling for Indoor and Outdoor Scenes
18 0.10620687 329 cvpr-2013-Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
19 0.10554844 186 cvpr-2013-GeoF: Geodesic Forests for Learning Coupled Predictors
20 0.10298663 86 cvpr-2013-Composite Statistical Inference for Semantic Segmentation
topicId topicWeight
[(0, 0.211), (1, -0.043), (2, 0.018), (3, -0.007), (4, 0.143), (5, 0.028), (6, -0.043), (7, 0.074), (8, -0.113), (9, 0.003), (10, 0.116), (11, -0.002), (12, -0.001), (13, 0.074), (14, -0.046), (15, -0.058), (16, 0.089), (17, 0.007), (18, -0.043), (19, -0.021), (20, 0.083), (21, -0.077), (22, 0.015), (23, 0.104), (24, -0.046), (25, -0.071), (26, 0.046), (27, 0.029), (28, -0.004), (29, -0.068), (30, -0.086), (31, -0.137), (32, -0.129), (33, -0.092), (34, -0.129), (35, -0.073), (36, -0.053), (37, 0.061), (38, -0.048), (39, -0.028), (40, -0.049), (41, 0.028), (42, -0.027), (43, 0.048), (44, 0.073), (45, -0.044), (46, -0.032), (47, 0.112), (48, -0.042), (49, 0.013)]
simIndex simValue paperId paperTitle
same-paper 1 0.97229928 425 cvpr-2013-Tensor-Based High-Order Semantic Relation Transfer for Semantic Scene Segmentation
Author: Heesoo Myeong, Kyoung Mu Lee
Abstract: We propose a novel nonparametric approach for semantic segmentation using high-order semantic relations. Conventional context models mainly focus on learning pairwise relationships between objects. Pairwise relations, however, are not enough to represent high-level contextual knowledge within images. In this paper, we propose semantic relation transfer, a method to transfer high-order semantic relations of objects from annotated images to unlabeled images analogous to label transfer techniques where label information are transferred. Wefirst define semantic tensors representing high-order relations of objects. Semantic relation transfer problem is then formulated as semi-supervised learning using a quadratic objective function of the semantic tensors. By exploiting low-rank property of the semantic tensors and employing Kronecker sum similarity, an efficient approximation algorithm is developed. Based on the predicted high-order semantic relations, we reason semantic segmentation and evaluate the performance on several challenging datasets.
2 0.84578073 309 cvpr-2013-Nonparametric Scene Parsing with Adaptive Feature Relevance and Semantic Context
Author: Gautam Singh, Jana Kosecka
Abstract: This paper presents a nonparametric approach to semantic parsing using small patches and simple gradient, color and location features. We learn the relevance of individual feature channels at test time using a locally adaptive distance metric. To further improve the accuracy of the nonparametric approach, we examine the importance of the retrieval set used to compute the nearest neighbours using a novel semantic descriptor to retrieve better candidates. The approach is validated by experiments on several datasets used for semantic parsing demonstrating the superiority of the method compared to the state of art approaches.
3 0.67763621 406 cvpr-2013-Spatial Inference Machines
Author: Roman Shapovalov, Dmitry Vetrov, Pushmeet Kohli
Abstract: This paper addresses the problem of semantic segmentation of 3D point clouds. We extend the inference machines framework of Ross et al. by adding spatial factors that model mid-range and long-range dependencies inherent in the data. The new model is able to account for semantic spatial context. During training, our method automatically isolates and retains factors modelling spatial dependencies between variables that are relevant for achieving higher prediction accuracy. We evaluate the proposed method by using it to predict 1 7-category semantic segmentations on sets of stitched Kinect scans. Experimental results show that the spatial dependencies learned by our method significantly improve the accuracy of segmentation. They also show that our method outperforms the existing segmentation technique of Koppula et al.
4 0.67005622 146 cvpr-2013-Enriching Texture Analysis with Semantic Data
Author: Tim Matthews, Mark S. Nixon, Mahesan Niranjan
Abstract: We argue for the importance of explicit semantic modelling in human-centred texture analysis tasks such as retrieval, annotation, synthesis, and zero-shot learning. To this end, low-level attributes are selected and used to define a semantic space for texture. 319 texture classes varying in illumination and rotation are positioned within this semantic space using a pairwise relative comparison procedure. Low-level visual features used by existing texture descriptors are then assessed in terms of their correspondence to the semantic space. Textures with strong presence ofattributes connoting randomness and complexity are shown to be poorly modelled by existing descriptors. In a retrieval experiment semantic descriptors are shown to outperform visual descriptors. Semantic modelling of texture is thus shown to provide considerable value in both feature selection and in analysis tasks.
5 0.6661461 157 cvpr-2013-Exploring Implicit Image Statistics for Visual Representativeness Modeling
Author: Xiaoshuai Sun, Xin-Jing Wang, Hongxun Yao, Lei Zhang
Abstract: In this paper, we propose a computational model of visual representativeness by integrating cognitive theories of representativeness heuristics with computer vision and machine learning techniques. Unlike previous models that build their representativeness measure based on the visible data, our model takes the initial inputs as explicit positive reference and extend the measure by exploring the implicit negatives. Given a group of images that contains obvious visual concepts, we create a customized image ontology consisting of both positive and negative instances by mining the most related and confusable neighbors of the positive concept in ontological semantic knowledge bases. The representativeness of a new item is then determined by its likelihoods for both the positive and negative references. To ensure the effectiveness of probability inference as well as the cognitive plausibility, we discover the potential prototypes and treat them as an intermediate representation of semantic concepts. In the experiment, we evaluate the performance of representativeness models based on both human judgements and user-click logs of commercial image search engine. Experimental results on both ImageNet and image sets of general concepts demonstrate the superior performance of our model against the state-of-the-arts.
6 0.66126233 242 cvpr-2013-Label Propagation from ImageNet to 3D Point Clouds
7 0.64378721 73 cvpr-2013-Bringing Semantics into Focus Using Visual Abstraction
8 0.64238751 43 cvpr-2013-Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs
9 0.63806635 460 cvpr-2013-Weakly-Supervised Dual Clustering for Image Semantic Segmentation
10 0.61582512 25 cvpr-2013-A Sentence Is Worth a Thousand Pixels
11 0.61482203 284 cvpr-2013-Mesh Based Semantic Modelling for Indoor and Outdoor Scenes
12 0.61142337 339 cvpr-2013-Probabilistic Graphlet Cut: Exploiting Spatial Structure Cue for Weakly Supervised Image Segmentation
13 0.57337528 86 cvpr-2013-Composite Statistical Inference for Semantic Segmentation
14 0.57110411 132 cvpr-2013-Discriminative Re-ranking of Diverse Segmentations
15 0.55223095 173 cvpr-2013-Finding Things: Image Parsing with Regions and Per-Exemplar Detectors
16 0.50856984 247 cvpr-2013-Learning Class-to-Image Distance with Object Matchings
17 0.50540262 186 cvpr-2013-GeoF: Geodesic Forests for Learning Coupled Predictors
18 0.50477999 366 cvpr-2013-Robust Region Grouping via Internal Patch Statistics
19 0.50389314 446 cvpr-2013-Understanding Indoor Scenes Using 3D Geometric Phrases
20 0.49406454 329 cvpr-2013-Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
topicId topicWeight
[(10, 0.121), (16, 0.032), (26, 0.053), (33, 0.292), (39, 0.019), (66, 0.018), (67, 0.067), (69, 0.086), (76, 0.014), (77, 0.011), (87, 0.096), (89, 0.088)]
simIndex simValue paperId paperTitle
1 0.96274233 212 cvpr-2013-Image Segmentation by Cascaded Region Agglomeration
Author: Zhile Ren, Gregory Shakhnarovich
Abstract: We propose a hierarchical segmentation algorithm that starts with a very fine oversegmentation and gradually merges regions using a cascade of boundary classifiers. This approach allows the weights of region and boundary features to adapt to the segmentation scale at which they are applied. The stages of the cascade are trained sequentially, with asymetric loss to maximize boundary recall. On six segmentation data sets, our algorithm achieves best performance under most region-quality measures, and does it with fewer segments than the prior work. Our algorithm is also highly competitive in a dense oversegmentation (superpixel) regime under boundary-based measures.
Author: Gunhee Kim, Eric P. Xing
Abstract: With an explosion of popularity of online photo sharing, we can trivially collect a huge number of photo streams for any interesting topics such as scuba diving as an outdoor recreational activity class. Obviously, the retrieved photo streams are neither aligned nor calibrated since they are taken in different temporal, spatial, and personal perspectives. However, at the same time, they are likely to share common storylines that consist of sequences of events and activities frequently recurred within the topic. In this paper, as a first technical step to detect such collective storylines, we propose an approach to jointly aligning and segmenting uncalibrated multiple photo streams. The alignment task discovers the matched images between different photo streams, and the image segmentation task parses each image into multiple meaningful regions to facilitate the image understanding. We close a loop between the two tasks so that solving one task helps enhance the performance of the other in a mutually rewarding way. To this end, we design a scalable message-passing based optimization framework to jointly achieve both tasks for the whole input image set at once. With evaluation on the new Flickr dataset of 15 outdoor activities that consist of 1.5 millions of images of 13 thousands of photo streams, our empirical results show that the proposed algorithms are more successful than other candidate methods for both tasks.
3 0.9546327 61 cvpr-2013-Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics
Author: Bo Zheng, Yibiao Zhao, Joey C. Yu, Katsushi Ikeuchi, Song-Chun Zhu
Abstract: In this paper, we present an approach for scene understanding by reasoning physical stability of objects from point cloud. We utilize a simple observation that, by human design, objects in static scenes should be stable with respect to gravity. This assumption is applicable to all scene categories and poses useful constraints for the plausible interpretations (parses) in scene understanding. Our method consists of two major steps: 1) geometric reasoning: recovering solid 3D volumetric primitives from defective point cloud; and 2) physical reasoning: grouping the unstable primitives to physically stable objects by optimizing the stability and the scene prior. We propose to use a novel disconnectivity graph (DG) to represent the energy landscape and use a Swendsen-Wang Cut (MCMC) method for optimization. In experiments, we demonstrate that the algorithm achieves substantially better performance for i) object segmentation, ii) 3D volumetric recovery of the scene, and iii) better parsing result for scene understanding in comparison to state-of-the-art methods in both public dataset and our own new dataset.
4 0.95279264 365 cvpr-2013-Robust Real-Time Tracking of Multiple Objects by Volumetric Mass Densities
Author: Horst Possegger, Sabine Sternig, Thomas Mauthner, Peter M. Roth, Horst Bischof
Abstract: Combining foreground images from multiple views by projecting them onto a common ground-plane has been recently applied within many multi-object tracking approaches. These planar projections introduce severe artifacts and constrain most approaches to objects moving on a common 2D ground-plane. To overcome these limitations, we introduce the concept of an occupancy volume exploiting the full geometry and the objects ’ center of mass and develop an efficient algorithm for 3D object tracking. Individual objects are tracked using the local mass density scores within a particle filter based approach, constrained by a Voronoi partitioning between nearby trackers. Our method benefits from the geometric knowledge given by the occupancy volume to robustly extract features and train classifiers on-demand, when volumetric information becomes unreliable. We evaluate our approach on several challenging real-world scenarios including the public APIDIS dataset. Experimental evaluations demonstrate significant improvements compared to state-of-theart methods, while achieving real-time performance. – –
5 0.95185244 248 cvpr-2013-Learning Collections of Part Models for Object Recognition
Author: Ian Endres, Kevin J. Shih, Johnston Jiaa, Derek Hoiem
Abstract: We propose a method to learn a diverse collection of discriminative parts from object bounding box annotations. Part detectors can be trained and applied individually, which simplifies learning and extension to new features or categories. We apply the parts to object category detection, pooling part detections within bottom-up proposed regions and using a boosted classifier with proposed sigmoid weak learners for scoring. On PASCAL VOC 2010, we evaluate the part detectors ’ ability to discriminate and localize annotated keypoints. Our detection system is competitive with the best-existing systems, outperforming other HOG-based detectors on the more deformable categories.
6 0.95027041 446 cvpr-2013-Understanding Indoor Scenes Using 3D Geometric Phrases
7 0.94840324 30 cvpr-2013-Accurate Localization of 3D Objects from RGB-D Data Using Segmentation Hypotheses
8 0.94817215 202 cvpr-2013-Hierarchical Saliency Detection
9 0.94732517 227 cvpr-2013-Intrinsic Scene Properties from a Single RGB-D Image
10 0.94725937 292 cvpr-2013-Multi-agent Event Detection: Localization and Role Assignment
11 0.946823 372 cvpr-2013-SLAM++: Simultaneous Localisation and Mapping at the Level of Objects
12 0.94568205 242 cvpr-2013-Label Propagation from ImageNet to 3D Point Clouds
13 0.94528282 445 cvpr-2013-Understanding Bayesian Rooms Using Composite 3D Object Models
same-paper 14 0.94508564 425 cvpr-2013-Tensor-Based High-Order Semantic Relation Transfer for Semantic Scene Segmentation
15 0.94400102 70 cvpr-2013-Bottom-Up Segmentation for Top-Down Detection
16 0.94349235 98 cvpr-2013-Cross-View Action Recognition via a Continuous Virtual Path
17 0.94341719 256 cvpr-2013-Learning Structured Hough Voting for Joint Object Detection and Occlusion Reasoning
18 0.94329536 19 cvpr-2013-A Minimum Error Vanishing Point Detection Approach for Uncalibrated Monocular Images of Man-Made Environments
19 0.94281608 225 cvpr-2013-Integrating Grammar and Segmentation for Human Pose Estimation
20 0.94269723 331 cvpr-2013-Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis