iccv iccv2013 iccv2013-52 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Adriana Kovashka, Kristen Grauman
Abstract: Current methods learn monolithic attribute predictors, with the assumption that a single model is sufficient to reflect human understanding of a visual attribute. However, in reality, humans vary in how they perceive the association between a named property and image content. For example, two people may have slightly different internal models for what makes a shoe look “formal”, or they may disagree on which of two scenes looks “more cluttered”. Rather than discount these differences as noise, we propose to learn user-specific attribute models. We adapt a generic model trained with annotations from multiple users, tailoring it to satisfy user-specific labels. Furthermore, we propose novel techniques to infer user-specific labels based on transitivity and contradictions in the user’s search history. We demonstrate that adapted attributes improve accuracy over both existing monolithic models as well as models that learn from scratch with user-specific data alone. In addition, we show how adapted attributes are useful to personalize image search, whether with binary or relative attributes.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract Current methods learn monolithic attribute predictors, with the assumption that a single model is sufficient to reflect human understanding of a visual attribute. [sent-3, score-0.482]
2 We demonstrate that adapted attributes improve accuracy over both existing monolithic models as well as models that learn from scratch with user-specific data alone. [sent-9, score-0.741]
3 In addition, we show how adapted attributes are useful to personalize image search, whether with binary or relative attributes. [sent-10, score-0.801]
4 Introduction Visual attributes are human understandable properties to describe images, e. [sent-12, score-0.443]
5 Thus far, training an attribute predictor largely follows the same procedure used for training any image classification system: one collects labeled image exemplars, extracts image descriptors, and applies discriminative learning. [sent-16, score-0.47]
6 Visual attribute interpretations vary slightly from viewer to viewer. [sent-24, score-0.452]
7 This is true whether attributes are modeled as categorical or relative properties. [sent-25, score-0.549]
8 We propose to adapt attribute models to take these differences in perception into account. [sent-27, score-0.605]
9 The differences may stem from several factors: the words for attributes are imprecise (when is the cat overweight vs. [sent-28, score-0.443]
10 Notably, their definitions vary whether we consider binary or relative attributes (see Fig. [sent-35, score-0.602]
11 This variability has important implications for any application where a human uses attributes to communicate with a vision system. [sent-37, score-0.443]
12 For example, in image search, a user requests images containing certain attributes [14, 23, 21, 13, 12]; in recognition, a user teaches a system about objects by describing their properties [15, 6, 3, 16, 17]. [sent-38, score-0.969]
13 Failing to account for user-specific notions of attributes will lead to discrepancies between the user’s precise intent and the message received by the system. [sent-39, score-0.525]
14 1 We propose to model attributes in a user-specific way, in order to capture the inherent differences in perception. [sent-41, score-0.443]
15 For binary properties, one takes the majority vote on the attribute present/absent label. [sent-44, score-0.592]
16 For relative properties, one takes a majority vote on the attribute more/less label. [sent-45, score-0.618]
17 Instead, we pose attribute learning as an adaptation problem. [sent-47, score-0.572]
18 First, we leverage any commonalities in perception to learn a generic prediction function, namely, a classifier for a binary attribute (e. [sent-48, score-0.887]
19 , pointy) or a ranking function for a relative attribute (e. [sent-50, score-0.539]
20 In the first, we connect relative attribute statements given by the user on multiple different images to obtain new implicit constraints via transitivity. [sent-56, score-0.852]
21 In the second, we detect discrepancies between the system’s generic attribute models and the user’s perception, and cre- ate implicit constraints to correct the models. [sent-57, score-0.785]
22 While our adapted attributes are applicable to any task demanding precise human-system communication about visual properties, we focus specifically on their impact for image search. [sent-59, score-0.639]
23 We demonstrate the advantages of personalized retrieval when a user queries for images with multi-attribute keywords or uses attributes to provide relevance feedback on selected reference images. [sent-60, score-0.992]
24 We compare our user-specific adapted attributes to a standard generic “consensus” model, as well as a baseline that trains exclusively with userspecific data. [sent-63, score-0.935]
25 We show that adapting learned models is an efficient way to capture person-dependent interpretations, particularly for fine-grained attribute distinctions where perception varies most. [sent-64, score-0.637]
26 Finally, we demonstrate the practical impact of adapted attributes for personalized search. [sent-66, score-0.794]
27 Related Work Learning visual attributes Visual attributes, originally introduced in [15, 6], offer a semantic representation shared among objects. [sent-69, score-0.443]
28 To our knowledge, all prior work assumes monolithic attribute predictors are sufficient, and none attempts to model user-specific perception, as we propose. [sent-76, score-0.482]
29 This includes prior methods that represent attributes relatively [16, 22]; though they permit looser comparative labels, they still assume a single underlying relative concept and learn a single “true” ordering of images. [sent-77, score-0.556]
30 Transfer learning and adaptation We adapt a generic attribute model to learn a user-specific one. [sent-78, score-0.912]
31 In contrast, we recover an individual user’s subjective attribute model from their annotations, by properly adapting a generic model over all previously seen users. [sent-93, score-0.795]
32 Personalization in information retrieval In information retrieval, personalization involves learning what a given user perceives as relevant, and producing user-specific 33442336 search results [18]. [sent-94, score-0.442]
33 Furthermore, whereas personalization generally entails learning a user-specific relevance function from scratch—there is no “universal” prior on relevance—we leverage a generic model for the attribute as a starting point, and efficiently adapt it towards the user’s preferences. [sent-97, score-0.914]
34 Approach We first train a generic model of an attribute using a large margin learning algorithm and data labeled with majority vote from multiple annotators. [sent-100, score-0.932]
35 Then, for a given user, we adapt the parameters of the generic model to account for any userspecific labeled data, while not straying too far from the prior generic model. [sent-102, score-0.738]
36 We refer to the resulting prediction function as an adapted attribute or user-specific attribute. [sent-103, score-0.593]
37 Then, we briefly describe how we use the adapted attributes to perform personalized content-based image search (Sec. [sent-108, score-0.842]
38 Thus, we are assured that the generic model is a valid prior for each novel user we aim to adapt to. [sent-123, score-0.589]
39 In the following, we do not notate individual attributes or users to avoid subscript clutter. [sent-128, score-0.547]
40 We assume the labeled examples originate from a pool ofpossibly many annotators who collectively represent the common denominator in attribute perception. [sent-132, score-0.532]
41 as input, and produce an adapted attribute f as output. [sent-139, score-0.593]
42 Adapting binary attribute classifiers Binary attributes predict whether or not an attribute is present in an image. [sent-140, score-1.371]
43 e generic attribute serves as a prior for the user-specific attribute, such that even with small amounts of user-labeled data we can learn an accurate predictor. [sent-167, score-0.695]
44 Hence, the adapted attribute prediction is a combination of the generic model’s prediction and similarities between the novel input x and (selected) user-specific instances xi. [sent-175, score-0.864]
45 Adapting relative attribute rankers Rather than make a hard decision about attribute presence, relative attributes predict the strength of an attribute in an image [16]. [sent-176, score-1.918]
46 =Ea {ch(x pair de)n}otes that image i1 exhibits the attribute more strongly than image i2—for example, that i1 is pointier than i2. [sent-181, score-0.496]
47 requires asking multiple annotators to vote on which of the two images exhibit the attribute more. [sent-183, score-0.548]
48 Again the solution requires solving a quadratic program [8], and the resulting adapted relative attribute predictor is: ? [sent-206, score-0.672]
49 Suitability for adapted attributes Having defined the two adaptation methods, we can now reflect on their strengths for our problem. [sent-213, score-0.733]
50 For example, suppose a user mostly agrees with the generic notion of formal shoes, but, unlike the average annotator, is also inclined to call loafers formal. [sent-227, score-0.62]
51 Second, training time is substantially lower than training each user model from scratch by pooling the generic and user-specific data. [sent-234, score-0.591]
52 The efficiency is especially valuable for personalized search, where we continually adapt a user’s attributes as his search history accumulates more user-specific data. [sent-239, score-0.768]
53 This is convenient, since in practice the data could be proprietary or simply unwieldy to pass around, yet one still would like to avoid learning personal attributes from scratch. [sent-242, score-0.47]
54 Personalized Image Search We next briefly describe how we use the adapted attributes to personalize image search results. [sent-245, score-0.717]
55 Similar to [14], the user states “I want images with attributes X, Y , and not Z”. [sent-248, score-0.692]
56 For relative attributes, we use the adapted rankers to retrieve images that agree with comparative relevance feedback. [sent-249, score-0.451]
57 Similar to [13], the user states “I want images that show more of attribute X than image A and less of attribute Y than image B”, etc. [sent-250, score-1.097]
58 Then, in both cases, the system sorts the database images according to how confidently the adapted attribute predictions agree with the attribute constraints mentioned in the query or feedback. [sent-251, score-1.109]
59 One can directly incorporate our adapted attributes into any existing attribute-search method. [sent-254, score-0.612]
60 Explicit collection Most directly, we ask the user to label a small set of images with the presence/absence of attributes (in the binary case) or pairs of images with comparative labels of the form “Image A is more/less/equally 33442358 [attribute name] than Image B” (in the relative case). [sent-259, score-0.945]
61 We convey the generic attribute meanings via qualification tests. [sent-261, score-0.695]
62 The first uses a margin criterion [26], requesting labels for those N images closest to the generic classifier’s hyperplane. [sent-266, score-0.465]
63 For the second, we devise a variant of the query-by-committee criterion, requesting user-specific labels for the N images where the human-given generic labels were most in disagreement. [sent-267, score-0.503]
64 While we find the margin criterion useful for binary attributes, for relative attributes it is less so. [sent-268, score-0.624]
65 Therefore, for relative attributes we adopt a simple diversitybased active selection scheme. [sent-270, score-0.522]
66 Therefore, we propose ways to infer “implicit” user-specific labels by mining the user’s relative attribute search history. [sent-276, score-0.704]
67 We discover which pairs of attributes are strongly correlated or anti-correlated3. [sent-289, score-0.476]
68 Now treating strongly (anti-)correlated attributes as the same (opposite) attribute, we detect contradictions as described above, for images A and B that have the same approximate attribute rank. [sent-290, score-1.017]
69 If the user requests images both more feminine than A and more sporty than B, where A and B are similarly feminine and similarly sporty, he seems to indicate that no images satisfy both constraints (green regions share no images). [sent-293, score-0.687]
70 This suggests his perception on one or both attributes differs from the current model fr? [sent-294, score-0.555]
71 For each constraint in a contradictory pair, we select an image C that violates it by a small margin, and create an implicit user-specific pair using A and C in the reverse order of how the current generic attribute ranks them. [sent-297, score-0.813]
72 By swapping the order, we correct the attribute model, and the theoretical set of images satisfying the user’s mental target is no longer empty (image C is now in both green regions). [sent-300, score-0.465]
73 Experiments We evaluate adapted attributes in terms of both their generalization accuracy (Sec. [sent-307, score-0.612]
74 Compared methods We compare our user-adaptive approach to the following three methods: 3We say two attributes third of the images in their are top strongly correlated if they share or bottom quartiles. [sent-312, score-0.476]
75 Datasets and features We use two datasets: Shoes [2], which contains 14,658 online shopping images describable by 10 attributes [13], and SUN Attributes [19], which contains 14,340 scenes. [sent-325, score-0.472]
76 We consider 12 attributes from SUN4 that appear frequently and are likely to be relevant for image search applications. [sent-326, score-0.518]
77 Adapted Attribute Accuracy First we evaluate generalization accuracy: will adapted attributes better agree with a user’s perception in novel images? [sent-338, score-0.777]
78 Our advantage over the generic model supports our main claim: we need to account for users’ individual perception when learning attributes. [sent-352, score-0.439]
79 So, when a useradapted model misclassifies, we cannot rule out the possibility that the worker himself was inconsistent with his personal perception of the attribute in that test case. [sent-363, score-0.585]
80 4 shows example attribute spectra for three generic 33443370 Shoes – Binary Attributes – “Feminine” aeepindagrtedclessmore aeepindagrtedclessmore Fidraine atg pdceurel 4s . [sent-367, score-0.773]
81 and adapted attribute predictions, sorted from least to most. [sent-375, score-0.593]
82 In the top set, it learns that this user perceives flat fancy shoes to be feminine, whereas the generic impression is that high-heeled shoes are more feminine. [sent-377, score-0.973]
83 In the middle set, it learns that for this user, shoes that are darker in color are more formal, whereas the generic model says shoes similar but brighter in color are formal. [sent-378, score-0.733]
84 First, we consider test case difficulty (a), as measured by the distance to the binary attribute generic hyperplane; closer instances are more difficult. [sent-382, score-0.777]
85 We sort the 10 test examples per split by difficulty, and average over all attributes and users. [sent-383, score-0.443]
86 We see that user-adapted attributes are often strongest when test cases are hardest. [sent-385, score-0.443]
87 Numbers in parens are standard error over all binary shoe attributes and random splits. [sent-390, score-0.565]
88 The margin between our adaptive method and the generic method is significantly increased for divergent workers (left col) compared to all workers (right col), as the generic model is insufficient when the user has a unique perception. [sent-391, score-0.958]
89 Personalized Search with Adapted Attributes Next we show that correctly capturing attribute perception is important for accurate search. [sent-395, score-0.536]
90 Search is a key application where adapted attributes can alleviate inconsistencies between what the user says, and what the (traditionally majority-vote-trained) machine understands. [sent-396, score-0.861]
91 For all search results, we use the attributes that seem most in need of adaptation, based on our previous results (5 for Shoes, 4 for SUN). [sent-397, score-0.518]
92 Accuracy is the percentage of test images where the binary predictions on all 3 query attributes agree with that user’s ground truth. [sent-402, score-0.549]
93 We see that the generalization power of the adapted attributes translates into the search setting. [sent-405, score-0.687]
94 This result demonstrates how our idea can benefit a number of prior binary attribute search systems [14, 23, 21]. [sent-407, score-0.552]
95 Relevance feedback with relative attributes Next we evaluate adapted attributes for relevance feedback. [sent-408, score-1.279]
96 We ask 10 users for whom we have trained user-specific relative attribute models to examine 10 target query images, and tell us whether they exhibit a specified attribute more/less/equally than 20 random reference images. [sent-409, score-1.07]
97 This result shows how our idea can improve prior systems for relative attribute search [13, 12]. [sent-444, score-0.578]
98 They apply only to the relative attribute search scenario, so we test on Shoes-R. [sent-446, score-0.578]
99 Conclusions and future work Our main contribution is the idea of adapting attributes to account for user-specific perception. [sent-452, score-0.541]
100 We plan to investigate extensions to detect when an attribute is perceived nearly the same by most users, to avoid requesting user-specific labels unnecessarily. [sent-456, score-0.602]
wordName wordTfidf (topN-words)
[('attributes', 0.443), ('attribute', 0.424), ('generic', 0.271), ('user', 0.249), ('shoes', 0.207), ('adapted', 0.169), ('feminine', 0.156), ('personalized', 0.155), ('adaptation', 0.121), ('contradictions', 0.117), ('perception', 0.112), ('formal', 0.1), ('sporty', 0.098), ('fr', 0.095), ('wb', 0.093), ('labels', 0.087), ('pointy', 0.08), ('relative', 0.079), ('search', 0.075), ('users', 0.075), ('feedback', 0.074), ('wr', 0.073), ('scratch', 0.071), ('relevance', 0.071), ('adapt', 0.069), ('adapting', 0.069), ('shoe', 0.069), ('transitivity', 0.069), ('implicit', 0.063), ('vote', 0.062), ('annotators', 0.062), ('workers', 0.059), ('requesting', 0.058), ('monolithic', 0.058), ('majority', 0.053), ('binary', 0.053), ('agree', 0.053), ('personalization', 0.052), ('userspecific', 0.052), ('fb', 0.051), ('margin', 0.049), ('worker', 0.049), ('says', 0.048), ('labeled', 0.046), ('sun', 0.046), ('rankers', 0.045), ('transfer', 0.044), ('target', 0.041), ('hyperplane', 0.041), ('viewers', 0.04), ('infer', 0.039), ('aeepindagrtedclessmore', 0.039), ('cool', 0.039), ('perceives', 0.039), ('pointier', 0.039), ('vacationing', 0.039), ('confidently', 0.039), ('statements', 0.037), ('ranking', 0.036), ('dr', 0.035), ('adriana', 0.035), ('kovashka', 0.034), ('comparative', 0.034), ('perceived', 0.033), ('strongly', 0.033), ('collecting', 0.032), ('distinctions', 0.032), ('deviate', 0.031), ('subjective', 0.031), ('yi', 0.031), ('declare', 0.03), ('personalize', 0.03), ('subscript', 0.029), ('account', 0.029), ('shades', 0.029), ('describable', 0.029), ('clickthrough', 0.029), ('contradictory', 0.029), ('extrapolate', 0.029), ('difficulty', 0.029), ('svm', 0.028), ('ranker', 0.028), ('entail', 0.028), ('requests', 0.028), ('interpretations', 0.028), ('whether', 0.027), ('classifier', 0.027), ('explicit', 0.027), ('learning', 0.027), ('impact', 0.027), ('speech', 0.027), ('db', 0.027), ('discrepancies', 0.027), ('men', 0.027), ('shiny', 0.027), ('history', 0.026), ('ranks', 0.026), ('gauge', 0.026), ('intent', 0.026), ('lagrange', 0.026)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999851 52 iccv-2013-Attribute Adaptation for Personalized Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: Current methods learn monolithic attribute predictors, with the assumption that a single model is sufficient to reflect human understanding of a visual attribute. However, in reality, humans vary in how they perceive the association between a named property and image content. For example, two people may have slightly different internal models for what makes a shoe look “formal”, or they may disagree on which of two scenes looks “more cluttered”. Rather than discount these differences as noise, we propose to learn user-specific attribute models. We adapt a generic model trained with annotations from multiple users, tailoring it to satisfy user-specific labels. Furthermore, we propose novel techniques to infer user-specific labels based on transitivity and contradictions in the user’s search history. We demonstrate that adapted attributes improve accuracy over both existing monolithic models as well as models that learn from scratch with user-specific data alone. In addition, we show how adapted attributes are useful to personalize image search, whether with binary or relative attributes.
2 0.50604153 31 iccv-2013-A Unified Probabilistic Approach Modeling Relationships between Attributes and Objects
Author: Xiaoyang Wang, Qiang Ji
Abstract: This paper proposes a unified probabilistic model to model the relationships between attributes and objects for attribute prediction and object recognition. As a list of semantically meaningful properties of objects, attributes generally relate to each other statistically. In this paper, we propose a unified probabilistic model to automatically discover and capture both the object-dependent and objectindependent attribute relationships. The model utilizes the captured relationships to benefit both attribute prediction and object recognition. Experiments on four benchmark attribute datasets demonstrate the effectiveness of the proposed unified model for improving attribute prediction as well as object recognition in both standard and zero-shot learning cases.
3 0.45844528 54 iccv-2013-Attribute Pivots for Guiding Relevance Feedback in Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: In interactive image search, a user iteratively refines his results by giving feedback on exemplar images. Active selection methods aim to elicit useful feedback, but traditional approaches suffer from expensive selection criteria and cannot predict informativeness reliably due to the imprecision of relevance feedback. To address these drawbacks, we propose to actively select “pivot” exemplars for which feedback in the form of a visual comparison will most reduce the system’s uncertainty. For example, the system might ask, “Is your target image more or less crowded than this image? ” Our approach relies on a series of binary search trees in relative attribute space, together with a selection function that predicts the information gain were the user to compare his envisioned target to the next node deeper in a given attribute ’s tree. It makes interactive search more efficient than existing strategies—both in terms of the system ’s selection time as well as the user’s feedback effort.
4 0.36760411 399 iccv-2013-Spoken Attributes: Mixing Binary and Relative Attributes to Say the Right Thing
Author: Amir Sadovnik, Andrew Gallagher, Devi Parikh, Tsuhan Chen
Abstract: In recent years, there has been a great deal of progress in describing objects with attributes. Attributes have proven useful for object recognition, image search, face verification, image description, and zero-shot learning. Typically, attributes are either binary or relative: they describe either the presence or absence of a descriptive characteristic, or the relative magnitude of the characteristic when comparing two exemplars. However, prior work fails to model the actual way in which humans use these attributes in descriptive statements of images. Specifically, it does not address the important interactions between the binary and relative aspects of an attribute. In this work we propose a spoken attribute classifier which models a more natural way of using an attribute in a description. For each attribute we train a classifier which captures the specific way this attribute should be used. We show that as a result of using this model, we produce descriptions about images of people that are more natural and specific than past systems.
5 0.35567969 213 iccv-2013-Implied Feedback: Learning Nuances of User Behavior in Image Search
Author: Devi Parikh, Kristen Grauman
Abstract: User feedback helps an image search system refine its relevance predictions, tailoring the search towards the user’s preferences. Existing methods simply take feedback at face value: clicking on an image means the user wants things like it; commenting that an image lacks a specific attribute means the user wants things that have it. However, we expect there is actually more information behind the user’s literal feedback. In particular, a user’s (possibly subconscious) search strategy leads him to comment on certain images rather than others, based on how any of the visible candidate images compare to the desired content. For example, he may be more likely to give negative feedback on an irrelevant image that is relatively close to his target, as opposed to bothering with one that is altogether different. We introduce novel features to capitalize on such implied feedback cues, and learn a ranking function that uses them to improve the system’s relevance estimates. We validate the approach with real users searching for shoes, faces, or scenes using two different modes of feedback: binary relevance feedback and relative attributes-based feedback. The results show that retrieval improves significantly when the system accounts for the learned behaviors. We show that the nuances learned are domain-invariant, and useful for both generic user-independent search as well as personalized user-specific search.
6 0.35001692 53 iccv-2013-Attribute Dominance: What Pops Out?
7 0.29499653 204 iccv-2013-Human Attribute Recognition by Rich Appearance Dictionary
8 0.27620551 380 iccv-2013-Semantic Transform: Weakly Supervised Semantic Inference for Relating Visual Attributes
9 0.22436801 123 iccv-2013-Domain Adaptive Classification
10 0.2151525 7 iccv-2013-A Deep Sum-Product Architecture for Robust Facial Attributes Analysis
11 0.1954267 449 iccv-2013-What Do You Do? Occupation Recognition in a Photo via Social Context
12 0.194593 276 iccv-2013-Multi-attributed Dictionary Learning for Sparse Coding
13 0.17924701 445 iccv-2013-Visual Reranking through Weakly Supervised Multi-graph Learning
14 0.1791465 238 iccv-2013-Learning Graphs to Match
15 0.17697626 378 iccv-2013-Semantic-Aware Co-indexing for Image Retrieval
16 0.17514335 237 iccv-2013-Learning Graph Matching: Oriented to Category Modeling from Cluttered Scenes
17 0.16526024 192 iccv-2013-Handwritten Word Spotting with Corrected Attributes
18 0.15196484 107 iccv-2013-Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction
19 0.119187 246 iccv-2013-Learning the Visual Interpretation of Sentences
20 0.11592255 350 iccv-2013-Relative Attributes for Large-Scale Abandoned Object Detection
topicId topicWeight
[(0, 0.214), (1, 0.226), (2, -0.104), (3, -0.261), (4, 0.22), (5, -0.048), (6, -0.147), (7, -0.255), (8, 0.352), (9, 0.275), (10, -0.086), (11, 0.049), (12, 0.047), (13, -0.001), (14, -0.048), (15, -0.071), (16, -0.011), (17, 0.013), (18, -0.064), (19, -0.031), (20, 0.04), (21, 0.002), (22, -0.021), (23, -0.034), (24, -0.04), (25, 0.004), (26, 0.022), (27, 0.033), (28, 0.002), (29, -0.046), (30, -0.092), (31, 0.022), (32, -0.033), (33, 0.04), (34, 0.041), (35, 0.015), (36, 0.019), (37, -0.002), (38, 0.059), (39, -0.003), (40, -0.004), (41, 0.005), (42, 0.005), (43, 0.004), (44, -0.015), (45, -0.031), (46, -0.026), (47, -0.005), (48, -0.038), (49, 0.009)]
simIndex simValue paperId paperTitle
same-paper 1 0.98189503 52 iccv-2013-Attribute Adaptation for Personalized Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: Current methods learn monolithic attribute predictors, with the assumption that a single model is sufficient to reflect human understanding of a visual attribute. However, in reality, humans vary in how they perceive the association between a named property and image content. For example, two people may have slightly different internal models for what makes a shoe look “formal”, or they may disagree on which of two scenes looks “more cluttered”. Rather than discount these differences as noise, we propose to learn user-specific attribute models. We adapt a generic model trained with annotations from multiple users, tailoring it to satisfy user-specific labels. Furthermore, we propose novel techniques to infer user-specific labels based on transitivity and contradictions in the user’s search history. We demonstrate that adapted attributes improve accuracy over both existing monolithic models as well as models that learn from scratch with user-specific data alone. In addition, we show how adapted attributes are useful to personalize image search, whether with binary or relative attributes.
2 0.93859684 53 iccv-2013-Attribute Dominance: What Pops Out?
Author: Naman Turakhia, Devi Parikh
Abstract: When we look at an image, some properties or attributes of the image stand out more than others. When describing an image, people are likely to describe these dominant attributes first. Attribute dominance is a result of a complex interplay between the various properties present or absent in the image. Which attributes in an image are more dominant than others reveals rich information about the content of the image. In this paper we tap into this information by modeling attribute dominance. We show that this helps improve the performance of vision systems on a variety of human-centric applications such as zero-shot learning, image search and generating textual descriptions of images.
3 0.93406069 399 iccv-2013-Spoken Attributes: Mixing Binary and Relative Attributes to Say the Right Thing
Author: Amir Sadovnik, Andrew Gallagher, Devi Parikh, Tsuhan Chen
Abstract: In recent years, there has been a great deal of progress in describing objects with attributes. Attributes have proven useful for object recognition, image search, face verification, image description, and zero-shot learning. Typically, attributes are either binary or relative: they describe either the presence or absence of a descriptive characteristic, or the relative magnitude of the characteristic when comparing two exemplars. However, prior work fails to model the actual way in which humans use these attributes in descriptive statements of images. Specifically, it does not address the important interactions between the binary and relative aspects of an attribute. In this work we propose a spoken attribute classifier which models a more natural way of using an attribute in a description. For each attribute we train a classifier which captures the specific way this attribute should be used. We show that as a result of using this model, we produce descriptions about images of people that are more natural and specific than past systems.
4 0.90943313 54 iccv-2013-Attribute Pivots for Guiding Relevance Feedback in Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: In interactive image search, a user iteratively refines his results by giving feedback on exemplar images. Active selection methods aim to elicit useful feedback, but traditional approaches suffer from expensive selection criteria and cannot predict informativeness reliably due to the imprecision of relevance feedback. To address these drawbacks, we propose to actively select “pivot” exemplars for which feedback in the form of a visual comparison will most reduce the system’s uncertainty. For example, the system might ask, “Is your target image more or less crowded than this image? ” Our approach relies on a series of binary search trees in relative attribute space, together with a selection function that predicts the information gain were the user to compare his envisioned target to the next node deeper in a given attribute ’s tree. It makes interactive search more efficient than existing strategies—both in terms of the system ’s selection time as well as the user’s feedback effort.
5 0.89518166 31 iccv-2013-A Unified Probabilistic Approach Modeling Relationships between Attributes and Objects
Author: Xiaoyang Wang, Qiang Ji
Abstract: This paper proposes a unified probabilistic model to model the relationships between attributes and objects for attribute prediction and object recognition. As a list of semantically meaningful properties of objects, attributes generally relate to each other statistically. In this paper, we propose a unified probabilistic model to automatically discover and capture both the object-dependent and objectindependent attribute relationships. The model utilizes the captured relationships to benefit both attribute prediction and object recognition. Experiments on four benchmark attribute datasets demonstrate the effectiveness of the proposed unified model for improving attribute prediction as well as object recognition in both standard and zero-shot learning cases.
6 0.8091591 380 iccv-2013-Semantic Transform: Weakly Supervised Semantic Inference for Relating Visual Attributes
7 0.72740078 213 iccv-2013-Implied Feedback: Learning Nuances of User Behavior in Image Search
8 0.72387612 449 iccv-2013-What Do You Do? Occupation Recognition in a Photo via Social Context
9 0.70141715 7 iccv-2013-A Deep Sum-Product Architecture for Robust Facial Attributes Analysis
10 0.61640507 350 iccv-2013-Relative Attributes for Large-Scale Abandoned Object Detection
11 0.58913916 445 iccv-2013-Visual Reranking through Weakly Supervised Multi-graph Learning
12 0.58379942 192 iccv-2013-Handwritten Word Spotting with Corrected Attributes
13 0.56461388 204 iccv-2013-Human Attribute Recognition by Rich Appearance Dictionary
14 0.50468314 285 iccv-2013-NEIL: Extracting Visual Knowledge from Web Data
15 0.45929787 378 iccv-2013-Semantic-Aware Co-indexing for Image Retrieval
16 0.44115773 246 iccv-2013-Learning the Visual Interpretation of Sentences
17 0.43778998 248 iccv-2013-Learning to Rank Using Privileged Information
18 0.43738204 416 iccv-2013-The Interestingness of Images
19 0.43692192 123 iccv-2013-Domain Adaptive Classification
20 0.43459019 332 iccv-2013-Quadruplet-Wise Image Similarity Learning
topicId topicWeight
[(2, 0.071), (7, 0.019), (12, 0.023), (26, 0.066), (31, 0.046), (34, 0.068), (40, 0.012), (42, 0.17), (64, 0.049), (69, 0.135), (73, 0.035), (78, 0.012), (89, 0.143), (97, 0.015), (98, 0.039)]
simIndex simValue paperId paperTitle
same-paper 1 0.89850128 52 iccv-2013-Attribute Adaptation for Personalized Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: Current methods learn monolithic attribute predictors, with the assumption that a single model is sufficient to reflect human understanding of a visual attribute. However, in reality, humans vary in how they perceive the association between a named property and image content. For example, two people may have slightly different internal models for what makes a shoe look “formal”, or they may disagree on which of two scenes looks “more cluttered”. Rather than discount these differences as noise, we propose to learn user-specific attribute models. We adapt a generic model trained with annotations from multiple users, tailoring it to satisfy user-specific labels. Furthermore, we propose novel techniques to infer user-specific labels based on transitivity and contradictions in the user’s search history. We demonstrate that adapted attributes improve accuracy over both existing monolithic models as well as models that learn from scratch with user-specific data alone. In addition, we show how adapted attributes are useful to personalize image search, whether with binary or relative attributes.
2 0.85097587 335 iccv-2013-Random Faces Guided Sparse Many-to-One Encoder for Pose-Invariant Face Recognition
Author: Yizhe Zhang, Ming Shao, Edward K. Wong, Yun Fu
Abstract: One of the most challenging task in face recognition is to identify people with varied poses. Namely, the test faces have significantly different poses compared with the registered faces. In this paper, we propose a high-level feature learning scheme to extract pose-invariant identity feature for face recognition. First, we build a single-hiddenlayer neural network with sparse constraint, to extractposeinvariant feature in a supervised fashion. Second, we further enhance the discriminative capability of the proposed feature by using multiple random faces as the target values for multiple encoders. By enforcing the target values to be uniquefor inputfaces over differentposes, the learned highlevel feature that is represented by the neurons in the hidden layer is pose free and only relevant to the identity information. Finally, we conduct face identification on CMU MultiPIE, and verification on Labeled Faces in the Wild (LFW) databases, where identification rank-1 accuracy and face verification accuracy with ROC curve are reported. These experiments demonstrate that our model is superior to oth- er state-of-the-art approaches on handling pose variations.
3 0.84911507 54 iccv-2013-Attribute Pivots for Guiding Relevance Feedback in Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: In interactive image search, a user iteratively refines his results by giving feedback on exemplar images. Active selection methods aim to elicit useful feedback, but traditional approaches suffer from expensive selection criteria and cannot predict informativeness reliably due to the imprecision of relevance feedback. To address these drawbacks, we propose to actively select “pivot” exemplars for which feedback in the form of a visual comparison will most reduce the system’s uncertainty. For example, the system might ask, “Is your target image more or less crowded than this image? ” Our approach relies on a series of binary search trees in relative attribute space, together with a selection function that predicts the information gain were the user to compare his envisioned target to the next node deeper in a given attribute ’s tree. It makes interactive search more efficient than existing strategies—both in terms of the system ’s selection time as well as the user’s feedback effort.
4 0.84205872 44 iccv-2013-Adapting Classification Cascades to New Domains
Author: Vidit Jain, Sachin Sudhakar Farfade
Abstract: Classification cascades have been very effective for object detection. Such a cascade fails to perform well in data domains with variations in appearances that may not be captured in the training examples. This limited generalization severely restricts the domains for which they can be used effectively. A common approach to address this limitation is to train a new cascade of classifiers from scratch for each of the new domains. Building separate detectors for each of the different domains requires huge annotation and computational effort, making it not scalable to a large number of data domains. Here we present an algorithm for quickly adapting a pre-trained cascade of classifiers using a small number oflabeledpositive instancesfrom a different yet similar data domain. In our experiments with images of human babies and human-like characters from movies, we demonstrate that the adapted cascade significantly outperforms both of the original cascade and the one trained from scratch using the given training examples. –
5 0.84135884 399 iccv-2013-Spoken Attributes: Mixing Binary and Relative Attributes to Say the Right Thing
Author: Amir Sadovnik, Andrew Gallagher, Devi Parikh, Tsuhan Chen
Abstract: In recent years, there has been a great deal of progress in describing objects with attributes. Attributes have proven useful for object recognition, image search, face verification, image description, and zero-shot learning. Typically, attributes are either binary or relative: they describe either the presence or absence of a descriptive characteristic, or the relative magnitude of the characteristic when comparing two exemplars. However, prior work fails to model the actual way in which humans use these attributes in descriptive statements of images. Specifically, it does not address the important interactions between the binary and relative aspects of an attribute. In this work we propose a spoken attribute classifier which models a more natural way of using an attribute in a description. For each attribute we train a classifier which captures the specific way this attribute should be used. We show that as a result of using this model, we produce descriptions about images of people that are more natural and specific than past systems.
6 0.83922601 123 iccv-2013-Domain Adaptive Classification
7 0.83728707 31 iccv-2013-A Unified Probabilistic Approach Modeling Relationships between Attributes and Objects
8 0.83551526 427 iccv-2013-Transfer Feature Learning with Joint Distribution Adaptation
9 0.83453321 80 iccv-2013-Collaborative Active Learning of a Kernel Machine Ensemble for Recognition
10 0.83423132 259 iccv-2013-Manifold Based Face Synthesis from Sparse Samples
11 0.83406395 398 iccv-2013-Sparse Variation Dictionary Learning for Face Recognition with a Single Training Sample per Person
12 0.83399171 205 iccv-2013-Human Re-identification by Matching Compositional Template with Cluster Sampling
13 0.83137566 328 iccv-2013-Probabilistic Elastic Part Model for Unsupervised Face Detector Adaptation
14 0.83134687 26 iccv-2013-A Practical Transfer Learning Algorithm for Face Verification
15 0.8302319 45 iccv-2013-Affine-Constrained Group Sparse Coding and Its Application to Image-Based Classifications
16 0.83012742 277 iccv-2013-Multi-channel Correlation Filters
17 0.83006775 392 iccv-2013-Similarity Metric Learning for Face Recognition
18 0.82858366 180 iccv-2013-From Where and How to What We See
20 0.82710457 161 iccv-2013-Fast Sparsity-Based Orthogonal Dictionary Learning for Image Restoration