iccv iccv2013 iccv2013-54 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Adriana Kovashka, Kristen Grauman
Abstract: In interactive image search, a user iteratively refines his results by giving feedback on exemplar images. Active selection methods aim to elicit useful feedback, but traditional approaches suffer from expensive selection criteria and cannot predict informativeness reliably due to the imprecision of relevance feedback. To address these drawbacks, we propose to actively select “pivot” exemplars for which feedback in the form of a visual comparison will most reduce the system’s uncertainty. For example, the system might ask, “Is your target image more or less crowded than this image? ” Our approach relies on a series of binary search trees in relative attribute space, together with a selection function that predicts the information gain were the user to compare his envisioned target to the next node deeper in a given attribute ’s tree. It makes interactive search more efficient than existing strategies—both in terms of the system ’s selection time as well as the user’s feedback effort.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract In interactive image search, a user iteratively refines his results by giving feedback on exemplar images. [sent-3, score-0.9]
2 To address these drawbacks, we propose to actively select “pivot” exemplars for which feedback in the form of a visual comparison will most reduce the system’s uncertainty. [sent-5, score-0.642]
3 ” Our approach relies on a series of binary search trees in relative attribute space, together with a selection function that predicts the information gain were the user to compare his envisioned target to the next node deeper in a given attribute ’s tree. [sent-7, score-1.622]
4 It makes interactive search more efficient than existing strategies—both in terms of the system ’s selection time as well as the user’s feedback effort. [sent-8, score-0.703]
5 Instead, an interactive approach lets the user help the system refine the top-ranked results via iterative feedback [3, 19, 13, 23, 11, 26, 5]. [sent-14, score-0.875]
6 The most common form of interaction consists of binary relevance feedback, in which the user declares certain exemplars to be relevant or irrelevant, and then the system updates its relevance metric in response. [sent-15, score-0.86]
7 To formulate the optimal question to ask next, it unifies an entropy reduction criterion with binary search trees in attribute space. [sent-21, score-0.76]
8 Typically, the system simply displays a screen full of top-ranked images, leaving a user free to provide feedback on any of them. [sent-23, score-0.827]
9 This strategy has the appeal of simultaneously showing the current results and accepting feedback [26]. [sent-24, score-0.494]
10 Thus, methods to actively select exemplar images for user feedback are needed. [sent-27, score-0.966]
11 The goal is to solicit feedback on those exemplars that would most improve the system’s notion of relevance. [sent-28, score-0.551]
12 First, the imprecision of binary relevance feedback (“Image X is relevant; image Y is not. [sent-33, score-0.754]
13 We propose to guide the user through a coarse-to-fine search using a relative attribute image representation. [sent-38, score-0.852]
14 At each iteration of feedback, the user provides a visual comparison between the attribute in his en297 visioned target and a “pivot” exemplar, where a pivot separates all database images into two balanced sets. [sent-39, score-1.272]
15 Given a database of images, we first construct a binary search tree for each relative attribute of interest (e. [sent-43, score-0.717]
16 Initially, the pivot exemplar for each attribute is the database image with the median relative attribute value. [sent-47, score-1.335]
17 Starting at the roots of these trees, we predict the information gain that would result from asking the user how his target image compares to each of the current pivots. [sent-48, score-0.475]
18 To compute the expected gain, we introduce methods to estimate the likelihood of the user’s response given the feedback history. [sent-49, score-0.647]
19 Then, among the pivots, the most informative comparison is requested, generating a question to the user such as, “Is your target image more, equally, or less pointy than this image? [sent-50, score-0.506]
20 It also moves the current pivot down one level within the selected attribute’s tree (unless the response is “equally”, in which case we no longer need to explore this tree). [sent-52, score-0.47]
21 Whereas prior information-gain methods would require a naive scan through all database images for each iteration, the proposed attribute search trees allow us to limit the scan to just one image per attribute. [sent-55, score-0.736]
22 For example, in a database of ∼15K images, a user can typically leoxcaamtep lhei,s i nexaa dcat target image Kwiitmh just 1a2 u s roerucnadns yofp ifceaelldyback, whereas the standard approach requires 21 rounds to reach the same level of accuracy. [sent-60, score-0.511]
23 The benefits of interactive feedback for image search are well studied [3, 19, 26, 5]. [sent-64, score-0.627]
24 In practice, the images displayed to the user for feedback are usually those ranked best by the system’s current relevance model. [sent-65, score-0.995]
25 If feedback is binary, with the user labeling examples as relevant (positive) or irrelevant (negative), the selection can naturally be cast as an active learning problem: the best examples to show are those that the relevance classifier is most uncertain about [13, 23, 11, 26]. [sent-67, score-1.24]
26 Notably, prior efforts to display the exemplar set that minimizes uncertainty were forced to resort to sampling or clustering heuristics due to the combinatorial optimization problem inherent when categorical feedback is assumed (e. [sent-68, score-0.579]
27 In contrast, we show that eliciting comparative feedback on ordinal visual attributes naturally leads to an efficient sequential selection strategy, where each comparison is guaranteed to decrease the predicted relevance of half of the unexplored database images. [sent-71, score-1.009]
28 While binary relevance feedback is most common, our recent work [8] shows how relative visual attributes are useful for feedback (e. [sent-81, score-1.396]
29 While this work also uses relative attribute feedback, the similarity to [8] ends there. [sent-84, score-0.465]
30 Whereas in [8] search proceeds in a standard passive manner, with the user offering feedback on images of his choosing among the topranked ones, our main idea is an actively guided search procedure based on a sequence of system-requested comparisons. [sent-85, score-1.246]
31 This entails novel methods for active selection with binary attribute trees (Sec. [sent-86, score-0.714]
32 Furthermore, we refine the simple counting model of [8] to account for uncertainty in attribute predictions (Sec. [sent-91, score-0.49]
33 More distant from our work, other work investigates training classifiers with actively selected attribute labels. [sent-105, score-0.504]
34 Our goal is very different: we do active feedback requests for image search, not classification, and our approach requests visual comparisons, not attribute labels. [sent-107, score-1.118]
35 ”, where A is a semantic attribute and I an exemplar from the database is being searched. [sent-116, score-0.542]
36 Rather than exhaustively search all database images as potential exemplars, however, we consider only a small number of pivot exemplars—the internal nodes of binary search trees constructed for each attribute. [sent-119, score-0.731]
37 After reviewing an existing method [16] to predict relative attribute strengths (Sec. [sent-121, score-0.49]
38 1), we explain how we construct attribute binary search trees (Sec. [sent-123, score-0.635]
39 Next we present our model of image relevance that accounts for the user’s attribute-based feedback (Sec. [sent-126, score-0.67]
40 Relative Attribute Predictions In order to utilize attribute-based comparisons, we need to estimate the strength of each attribute in each database image. [sent-155, score-0.486]
41 For each attribute m, we use its associated training pairs to learn a (possibly kernelized) ranking function: am (Ii) = wmTxi, which maps the image descriptor xi for image Ii to its real-valued attribute strength. [sent-159, score-0.865]
42 Using standard features and kernels, we find that 75% of held-out human comparisons are preserved by attribute predictors trained with ∼200 pairs. [sent-166, score-0.46]
43 The tree recursively partitions all the database images into two balanced sets, where the key at a given node is the median relative attribute value occurring within the set of images passed to that node. [sent-175, score-0.652]
44 To build the m-th attribute tree, we start at the root with all database images, sort them by their predicted attribute values am (I1) , . [sent-176, score-0.927]
45 Then the splitting repeats recursively, each time storing the next pivot image and its relative attribute value at the appropriate node. [sent-186, score-0.793]
46 Note that both the relative attribute ranker training and the search tree construction are offline procedures; they are ≤ 299 performed once, before handling any user queries. [sent-187, score-0.918]
47 Already, one could imagine a search procedure that walks a user through one such attribute tree, at each successively deeper level requesting a comparison to the pivot, and then eliminating the appropriate portion of the database depending on whether the user says “more” or “less”. [sent-188, score-1.199]
48 First, we cannot assume that the attribute predictions are identical to the attribute strengths a user will perceive; thus, a hard pruning of a full sub-tree is error-prone. [sent-190, score-1.176]
49 Second, this approach fails to account for the variable information gain that could be achieved depending on which attribute is explored at any given round of feedback. [sent-191, score-0.483]
50 Let F = {(Ipm, r)k}kT=1 denote the set of comparative ceotn sFtrai=nts a(Iccumula}ted in the T rounds of feedback so far. [sent-201, score-0.533]
51 The k-th item in F consists of a pivot image Ipm for atTtrihbeut ke- m, atenmd a user response r ∈ a {“more”, “less”, “feoqrua atltlyri”bu}t. [sent-202, score-0.729]
52 For example, if the user’s k-th comparison yields response r = “more”, then Sk,i = 1 if the database image Ii has attribute m more than the corresponding pivot image Ipm . [sent-207, score-0.913]
53 The probability of relevance is thus the probability that all T feedback comparisons in F are satisfied: ? [sent-208, score-0.717]
54 The probability that the k-th individual constraint is satisfied given that the user’s response was r for pivot Ipm is: P(Sk,i=1|Ii)=⎪ ⎨⎧P P( A Am m( I i ) ><= A Am m( I p ) ) if r = “ lem qsour”ael”y . [sent-213, score-0.46]
55 To estimate thes⎩e probabilities, we map the attribute predictions am (·) to probabilistic outputs, by adapting Platt’s method [17] to the paired classification problem implicit in the large-margin ranking objective. [sent-214, score-0.523]
56 Our probabilistic model of relevance accounts for the fact that predicted attributes can deviate from true perceived attribute strengths. [sent-222, score-0.774]
57 In contrast, prior work using relative attribute feedback [8] makes hard decisions, simply counting how many predicted attribute values satisfy the user’s constraints to measure relevance. [sent-223, score-1.4]
58 We find that a hard pruning of images on irrelevant branches of an attribute tree eliminates the true target for 93% of the queries, clearly supporting the proposed probabilistic formulation. [sent-224, score-0.623]
59 Our system maintains a set of M current pivot images (one per attribute tree) at each iteration, denoted P = {Ip1 , . [sent-228, score-0.795]
60 t iTbhuet pivots are initially tiohen ,r dooent pivot Pim =ages from each} . [sent-232, score-0.529]
61 e During aarcetiv inei tsieallelyct ithoen, our goal ti ism mtoidentify the pivot in this set that, once compared by the user to his target, will most reduce the entropy of the relevance predictions on all database images. [sent-234, score-0.983]
62 Note that selecting a pivot corresponds to selecting both an image as well as an attribute along which we want it to be compared; Ipm refers to the pivot for attribute m. [sent-235, score-1.482]
63 Given the feedback history F, we want to predict the information gain across all N dFa,ta wbaese w images freodr ceatc thhe pivot mina Ptio. [sent-237, score-0.94]
64 du Wcees wthilel to retaqlu e rsetle avance entropy over all images—or equivalently, the pivot that minimizes the expected entropy when used to augment the current set of feedback constraints. [sent-239, score-0.957]
65 The entropy based on the feedback thus far is: XN H(F) = −XXP(yi iX= X1 = ? [sent-240, score-0.55]
66 Furthermore, as we will show in the results, the pivots also enhance selection accuracy, by essentially isolating those images likely to impact relevance predictions. [sent-253, score-0.472]
67 In each case, we use cues from the available feedback history to form a “proxy” for the user, essentially borrowing the probability that a new constraint is satisfied from previously seen feedback. [sent-264, score-0.556]
68 The assumption is that the images that are relevant to the user thus far are (on the whole) more likely to satisfy the user’s next feedback than those that are irrelevant. [sent-266, score-0.886]
69 Ideally we would average the P(Sc,i = 1|Ii) values among only the relevant images Ii, where c= =in 1de|Ixes the candidate new feedback for a (yet unknown) user response R. [sent-268, score-0.985]
70 Here, first the user is asked to compare his target to the boot pivot (1) in terms of pointiness; then he is asked to compare it to (2) in terms of shininess, followed by (3) in terms of pointiness, and so on. [sent-274, score-0.737]
71 h we call Similar Question, examines all previously answered feedback requests, and copies the answer from the question that is most similar to the new one. [sent-281, score-0.539]
72 We define question similarity in terms of the Euclidean distance between the pivot images’ descriptors plus the similarity of the two attributes involved in either question. [sent-282, score-0.502]
73 Let rk∗ denote the response to the most similar question k found in the history F for the new pivot Ipm under consideforuatniodn i. [sent-285, score-0.501]
74 At each iteration, we present the user with the pivot selected with Eqn. [sent-290, score-0.63]
75 In order for the user to monitor the search progress and stop if an image similar to his target has been found, we also show him the current topranked images. [sent-292, score-0.527]
76 If further feedback is given, we first update F with the user’s new image-attribute-response constraint. [sent-293, score-0.494]
77 This is because our active selection criterion considers which attribute will most benefit from more refined feedback at any point in time. [sent-300, score-1.071]
78 The cost ofour selection method per round offeedback is O(MN), where M is the size of the attribute vocabulary, N is the database size, and M ? [sent-302, score-0.56]
79 That is, for a given search session, the user is instructed to give feedback by comparing the target we specify to the various methods’ selected exemplars. [sent-319, score-0.988]
80 We compare our method ACTIVE ATTRIBUTE PIVOTS against the following six methods: • ATTRIBUTE PIVOTS is a simplified version of our mAethod that uses the proposed attribute trees to select candidate images, but cycles among the attributes in a round-robin fashion. [sent-327, score-0.628]
81 This method represents traditional interactive methods that assume an “impatient” user for whom feedback exemplars and search results must be one and • • • the same. [sent-330, score-0.986]
82 ACTIVE BINARY FEEDBACK does not use statements aAbout the relative attribute strength of images, but rather asks the user whether the exemplar is similar to the target. [sent-333, score-0.823]
83 Relative feedback methods use the same relevance prediction function and only differ in the feedback they gather. [sent-337, score-1.164]
84 ” using the difference in the predicted attribute values for the target and Ipm . [sent-342, score-0.548]
85 By extrapolating a sparse set of real human judgments through a learned ranking function, we can perform largescale comparisons and isolate the impact of our idea from the impact of the attribute rankers’ precision. [sent-344, score-0.534]
86 We initialize all attribute search methods with the same feedback constraint. [sent-345, score-0.992]
87 This suggests that our best guess at the target tends to be a sufficient proxy, having a fairly similar attribute signature. [sent-355, score-0.52]
88 te l ir knrPanecShoes CDN 4@G0Shoes k aintPnrce lreScenes N 04GCD@Scenes nlPraekitcAiveatrbuIFtearpcvtieosnt TopCDN04@GActFIivearbctienoasryfedback Attribute pivots Active attribute exhaustive Figure 4. [sent-361, score-0.649]
89 Comparison Passive Passive binary feedback to existing interactive search methods (higher and steeper curves are better). [sent-362, score-0.678]
90 This is likely because we cannot estimate attribute similarity reliably due to the distinct face attributes (e. [sent-373, score-0.564]
91 This shows that relative attribute feedback alone (the contribution of [8]) does not offer the most efficient search; rather, our idea to actively elicit comparisons is essential. [sent-381, score-1.126]
92 3 This shows that the attribute trees serve as a form of regularization, helping our method focus on those comparisons that a priori may be most informative. [sent-386, score-0.546]
93 The results confirm the striking advantage of attribute feedback compared to binary relevance feedback. [sent-388, score-1.134]
94 Binary feedback has an advantage only in the first few iterations, likely because we generously initialize it with 2 feedback statements. [sent-389, score-0.988]
95 We find that both feedback modes require similar user time: 6. [sent-390, score-0.796]
96 All methods share one simulated feedback statement at iteration 0, which we do not plot. [sent-406, score-0.52]
97 303 nePrtekneiaclrSIhtoera tsio-1nkSIcte rantieosnFaceItse-rUatino ique Active attribute pivots Attribute pivots Top Figure 5. [sent-414, score-0.815]
98 Using the user’s feedback on the left, we retrieve the images on the right at the top of the results list. [sent-421, score-0.517]
99 Conclusion Today’s visual search systems place the burden on the user to initiate useful feedback by labeling images as relevant. [sent-435, score-0.904]
100 In contrast, our system actively guides the search based on visual comparisons, helping a user navigate the image database via relative semantic properties. [sent-436, score-0.634]
wordName wordTfidf (topN-words)
[('feedback', 0.494), ('attribute', 0.413), ('pivot', 0.328), ('user', 0.302), ('pivots', 0.201), ('relevance', 0.176), ('ipm', 0.134), ('passive', 0.133), ('attributes', 0.129), ('active', 0.119), ('target', 0.107), ('response', 0.099), ('actively', 0.091), ('ii', 0.088), ('trees', 0.086), ('search', 0.085), ('database', 0.073), ('ip', 0.069), ('relevant', 0.067), ('pointiness', 0.065), ('exemplars', 0.057), ('entropy', 0.056), ('exemplar', 0.056), ('shoes', 0.054), ('relative', 0.052), ('binary', 0.051), ('shininess', 0.049), ('wmtxi', 0.049), ('predictions', 0.048), ('interactive', 0.048), ('comparisons', 0.047), ('requests', 0.046), ('question', 0.045), ('selection', 0.045), ('queries', 0.045), ('om', 0.044), ('equally', 0.044), ('tree', 0.043), ('gain', 0.041), ('mental', 0.041), ('requested', 0.04), ('smiling', 0.04), ('ranking', 0.039), ('comparative', 0.039), ('kovashka', 0.038), ('irrelevant', 0.037), ('ranks', 0.037), ('rank', 0.035), ('exhaustive', 0.035), ('judgments', 0.035), ('questions', 0.034), ('satisfied', 0.033), ('blueness', 0.033), ('heel', 0.033), ('imprecision', 0.033), ('reqou', 0.033), ('solicits', 0.033), ('topranked', 0.033), ('wmtxj', 0.033), ('request', 0.032), ('system', 0.031), ('likelihood', 0.031), ('percentile', 0.03), ('live', 0.03), ('multimedia', 0.03), ('round', 0.029), ('history', 0.029), ('whereas', 0.029), ('adriana', 0.029), ('feminine', 0.029), ('elicit', 0.029), ('uncertainty', 0.029), ('scan', 0.028), ('perceived', 0.028), ('proxy', 0.028), ('predicted', 0.028), ('pointy', 0.027), ('bluer', 0.027), ('envisioned', 0.027), ('isolating', 0.027), ('yi', 0.026), ('iteration', 0.026), ('informative', 0.025), ('iterations', 0.025), ('eliciting', 0.025), ('ndcg', 0.025), ('passed', 0.025), ('predict', 0.025), ('reduction', 0.024), ('informativeness', 0.024), ('requesting', 0.024), ('content', 0.024), ('paired', 0.023), ('ranker', 0.023), ('images', 0.023), ('expected', 0.023), ('users', 0.023), ('ij', 0.023), ('reliably', 0.022), ('pubfig', 0.022)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000007 54 iccv-2013-Attribute Pivots for Guiding Relevance Feedback in Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: In interactive image search, a user iteratively refines his results by giving feedback on exemplar images. Active selection methods aim to elicit useful feedback, but traditional approaches suffer from expensive selection criteria and cannot predict informativeness reliably due to the imprecision of relevance feedback. To address these drawbacks, we propose to actively select “pivot” exemplars for which feedback in the form of a visual comparison will most reduce the system’s uncertainty. For example, the system might ask, “Is your target image more or less crowded than this image? ” Our approach relies on a series of binary search trees in relative attribute space, together with a selection function that predicts the information gain were the user to compare his envisioned target to the next node deeper in a given attribute ’s tree. It makes interactive search more efficient than existing strategies—both in terms of the system ’s selection time as well as the user’s feedback effort.
2 0.62751287 213 iccv-2013-Implied Feedback: Learning Nuances of User Behavior in Image Search
Author: Devi Parikh, Kristen Grauman
Abstract: User feedback helps an image search system refine its relevance predictions, tailoring the search towards the user’s preferences. Existing methods simply take feedback at face value: clicking on an image means the user wants things like it; commenting that an image lacks a specific attribute means the user wants things that have it. However, we expect there is actually more information behind the user’s literal feedback. In particular, a user’s (possibly subconscious) search strategy leads him to comment on certain images rather than others, based on how any of the visible candidate images compare to the desired content. For example, he may be more likely to give negative feedback on an irrelevant image that is relatively close to his target, as opposed to bothering with one that is altogether different. We introduce novel features to capitalize on such implied feedback cues, and learn a ranking function that uses them to improve the system’s relevance estimates. We validate the approach with real users searching for shoes, faces, or scenes using two different modes of feedback: binary relevance feedback and relative attributes-based feedback. The results show that retrieval improves significantly when the system accounts for the learned behaviors. We show that the nuances learned are domain-invariant, and useful for both generic user-independent search as well as personalized user-specific search.
3 0.45844528 52 iccv-2013-Attribute Adaptation for Personalized Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: Current methods learn monolithic attribute predictors, with the assumption that a single model is sufficient to reflect human understanding of a visual attribute. However, in reality, humans vary in how they perceive the association between a named property and image content. For example, two people may have slightly different internal models for what makes a shoe look “formal”, or they may disagree on which of two scenes looks “more cluttered”. Rather than discount these differences as noise, we propose to learn user-specific attribute models. We adapt a generic model trained with annotations from multiple users, tailoring it to satisfy user-specific labels. Furthermore, we propose novel techniques to infer user-specific labels based on transitivity and contradictions in the user’s search history. We demonstrate that adapted attributes improve accuracy over both existing monolithic models as well as models that learn from scratch with user-specific data alone. In addition, we show how adapted attributes are useful to personalize image search, whether with binary or relative attributes.
4 0.37272897 31 iccv-2013-A Unified Probabilistic Approach Modeling Relationships between Attributes and Objects
Author: Xiaoyang Wang, Qiang Ji
Abstract: This paper proposes a unified probabilistic model to model the relationships between attributes and objects for attribute prediction and object recognition. As a list of semantically meaningful properties of objects, attributes generally relate to each other statistically. In this paper, we propose a unified probabilistic model to automatically discover and capture both the object-dependent and objectindependent attribute relationships. The model utilizes the captured relationships to benefit both attribute prediction and object recognition. Experiments on four benchmark attribute datasets demonstrate the effectiveness of the proposed unified model for improving attribute prediction as well as object recognition in both standard and zero-shot learning cases.
5 0.26583987 399 iccv-2013-Spoken Attributes: Mixing Binary and Relative Attributes to Say the Right Thing
Author: Amir Sadovnik, Andrew Gallagher, Devi Parikh, Tsuhan Chen
Abstract: In recent years, there has been a great deal of progress in describing objects with attributes. Attributes have proven useful for object recognition, image search, face verification, image description, and zero-shot learning. Typically, attributes are either binary or relative: they describe either the presence or absence of a descriptive characteristic, or the relative magnitude of the characteristic when comparing two exemplars. However, prior work fails to model the actual way in which humans use these attributes in descriptive statements of images. Specifically, it does not address the important interactions between the binary and relative aspects of an attribute. In this work we propose a spoken attribute classifier which models a more natural way of using an attribute in a description. For each attribute we train a classifier which captures the specific way this attribute should be used. We show that as a result of using this model, we produce descriptions about images of people that are more natural and specific than past systems.
6 0.25201881 305 iccv-2013-POP: Person Re-identification Post-rank Optimisation
7 0.24268529 204 iccv-2013-Human Attribute Recognition by Rich Appearance Dictionary
8 0.22446992 53 iccv-2013-Attribute Dominance: What Pops Out?
9 0.21084669 380 iccv-2013-Semantic Transform: Weakly Supervised Semantic Inference for Relating Visual Attributes
10 0.16444862 123 iccv-2013-Domain Adaptive Classification
11 0.1586718 7 iccv-2013-A Deep Sum-Product Architecture for Robust Facial Attributes Analysis
12 0.14184566 445 iccv-2013-Visual Reranking through Weakly Supervised Multi-graph Learning
13 0.12922721 276 iccv-2013-Multi-attributed Dictionary Learning for Sparse Coding
14 0.12549019 326 iccv-2013-Predicting Sufficient Annotation Strength for Interactive Foreground Segmentation
15 0.11588686 107 iccv-2013-Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction
16 0.11095748 449 iccv-2013-What Do You Do? Occupation Recognition in a Photo via Social Context
17 0.10292898 237 iccv-2013-Learning Graph Matching: Oriented to Category Modeling from Cluttered Scenes
18 0.10256775 41 iccv-2013-Active Learning of an Action Detector from Untrimmed Videos
19 0.098681144 378 iccv-2013-Semantic-Aware Co-indexing for Image Retrieval
20 0.097600006 192 iccv-2013-Handwritten Word Spotting with Corrected Attributes
topicId topicWeight
[(0, 0.189), (1, 0.175), (2, -0.082), (3, -0.217), (4, 0.199), (5, -0.03), (6, -0.115), (7, -0.209), (8, 0.291), (9, 0.254), (10, -0.08), (11, 0.055), (12, 0.033), (13, 0.007), (14, -0.045), (15, -0.063), (16, -0.023), (17, -0.036), (18, -0.077), (19, -0.034), (20, 0.036), (21, -0.032), (22, -0.083), (23, -0.031), (24, -0.028), (25, 0.078), (26, 0.012), (27, 0.115), (28, -0.017), (29, -0.177), (30, -0.143), (31, -0.041), (32, -0.064), (33, 0.103), (34, 0.102), (35, 0.083), (36, -0.042), (37, 0.064), (38, 0.134), (39, 0.048), (40, -0.045), (41, -0.073), (42, 0.114), (43, -0.126), (44, -0.018), (45, -0.121), (46, -0.107), (47, -0.075), (48, -0.074), (49, 0.102)]
simIndex simValue paperId paperTitle
same-paper 1 0.97965652 54 iccv-2013-Attribute Pivots for Guiding Relevance Feedback in Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: In interactive image search, a user iteratively refines his results by giving feedback on exemplar images. Active selection methods aim to elicit useful feedback, but traditional approaches suffer from expensive selection criteria and cannot predict informativeness reliably due to the imprecision of relevance feedback. To address these drawbacks, we propose to actively select “pivot” exemplars for which feedback in the form of a visual comparison will most reduce the system’s uncertainty. For example, the system might ask, “Is your target image more or less crowded than this image? ” Our approach relies on a series of binary search trees in relative attribute space, together with a selection function that predicts the information gain were the user to compare his envisioned target to the next node deeper in a given attribute ’s tree. It makes interactive search more efficient than existing strategies—both in terms of the system ’s selection time as well as the user’s feedback effort.
2 0.92745471 213 iccv-2013-Implied Feedback: Learning Nuances of User Behavior in Image Search
Author: Devi Parikh, Kristen Grauman
Abstract: User feedback helps an image search system refine its relevance predictions, tailoring the search towards the user’s preferences. Existing methods simply take feedback at face value: clicking on an image means the user wants things like it; commenting that an image lacks a specific attribute means the user wants things that have it. However, we expect there is actually more information behind the user’s literal feedback. In particular, a user’s (possibly subconscious) search strategy leads him to comment on certain images rather than others, based on how any of the visible candidate images compare to the desired content. For example, he may be more likely to give negative feedback on an irrelevant image that is relatively close to his target, as opposed to bothering with one that is altogether different. We introduce novel features to capitalize on such implied feedback cues, and learn a ranking function that uses them to improve the system’s relevance estimates. We validate the approach with real users searching for shoes, faces, or scenes using two different modes of feedback: binary relevance feedback and relative attributes-based feedback. The results show that retrieval improves significantly when the system accounts for the learned behaviors. We show that the nuances learned are domain-invariant, and useful for both generic user-independent search as well as personalized user-specific search.
3 0.84371811 52 iccv-2013-Attribute Adaptation for Personalized Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: Current methods learn monolithic attribute predictors, with the assumption that a single model is sufficient to reflect human understanding of a visual attribute. However, in reality, humans vary in how they perceive the association between a named property and image content. For example, two people may have slightly different internal models for what makes a shoe look “formal”, or they may disagree on which of two scenes looks “more cluttered”. Rather than discount these differences as noise, we propose to learn user-specific attribute models. We adapt a generic model trained with annotations from multiple users, tailoring it to satisfy user-specific labels. Furthermore, we propose novel techniques to infer user-specific labels based on transitivity and contradictions in the user’s search history. We demonstrate that adapted attributes improve accuracy over both existing monolithic models as well as models that learn from scratch with user-specific data alone. In addition, we show how adapted attributes are useful to personalize image search, whether with binary or relative attributes.
4 0.6923154 53 iccv-2013-Attribute Dominance: What Pops Out?
Author: Naman Turakhia, Devi Parikh
Abstract: When we look at an image, some properties or attributes of the image stand out more than others. When describing an image, people are likely to describe these dominant attributes first. Attribute dominance is a result of a complex interplay between the various properties present or absent in the image. Which attributes in an image are more dominant than others reveals rich information about the content of the image. In this paper we tap into this information by modeling attribute dominance. We show that this helps improve the performance of vision systems on a variety of human-centric applications such as zero-shot learning, image search and generating textual descriptions of images.
5 0.68863499 399 iccv-2013-Spoken Attributes: Mixing Binary and Relative Attributes to Say the Right Thing
Author: Amir Sadovnik, Andrew Gallagher, Devi Parikh, Tsuhan Chen
Abstract: In recent years, there has been a great deal of progress in describing objects with attributes. Attributes have proven useful for object recognition, image search, face verification, image description, and zero-shot learning. Typically, attributes are either binary or relative: they describe either the presence or absence of a descriptive characteristic, or the relative magnitude of the characteristic when comparing two exemplars. However, prior work fails to model the actual way in which humans use these attributes in descriptive statements of images. Specifically, it does not address the important interactions between the binary and relative aspects of an attribute. In this work we propose a spoken attribute classifier which models a more natural way of using an attribute in a description. For each attribute we train a classifier which captures the specific way this attribute should be used. We show that as a result of using this model, we produce descriptions about images of people that are more natural and specific than past systems.
6 0.62356019 31 iccv-2013-A Unified Probabilistic Approach Modeling Relationships between Attributes and Objects
7 0.61456299 305 iccv-2013-POP: Person Re-identification Post-rank Optimisation
8 0.59613287 380 iccv-2013-Semantic Transform: Weakly Supervised Semantic Inference for Relating Visual Attributes
9 0.51080412 449 iccv-2013-What Do You Do? Occupation Recognition in a Photo via Social Context
10 0.49013332 445 iccv-2013-Visual Reranking through Weakly Supervised Multi-graph Learning
11 0.47353283 7 iccv-2013-A Deep Sum-Product Architecture for Robust Facial Attributes Analysis
12 0.45659852 350 iccv-2013-Relative Attributes for Large-Scale Abandoned Object Detection
13 0.37091851 192 iccv-2013-Handwritten Word Spotting with Corrected Attributes
14 0.36565277 326 iccv-2013-Predicting Sufficient Annotation Strength for Interactive Foreground Segmentation
15 0.35742977 204 iccv-2013-Human Attribute Recognition by Rich Appearance Dictionary
16 0.34434518 378 iccv-2013-Semantic-Aware Co-indexing for Image Retrieval
17 0.33930925 416 iccv-2013-The Interestingness of Images
18 0.33787078 248 iccv-2013-Learning to Rank Using Privileged Information
19 0.33274195 446 iccv-2013-Visual Semantic Complex Network for Web Images
20 0.32596228 285 iccv-2013-NEIL: Extracting Visual Knowledge from Web Data
topicId topicWeight
[(0, 0.109), (2, 0.078), (7, 0.018), (12, 0.022), (13, 0.011), (26, 0.07), (31, 0.03), (34, 0.043), (42, 0.217), (64, 0.054), (69, 0.014), (73, 0.023), (78, 0.02), (89, 0.139), (95, 0.012), (98, 0.011)]
simIndex simValue paperId paperTitle
same-paper 1 0.93390214 54 iccv-2013-Attribute Pivots for Guiding Relevance Feedback in Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: In interactive image search, a user iteratively refines his results by giving feedback on exemplar images. Active selection methods aim to elicit useful feedback, but traditional approaches suffer from expensive selection criteria and cannot predict informativeness reliably due to the imprecision of relevance feedback. To address these drawbacks, we propose to actively select “pivot” exemplars for which feedback in the form of a visual comparison will most reduce the system’s uncertainty. For example, the system might ask, “Is your target image more or less crowded than this image? ” Our approach relies on a series of binary search trees in relative attribute space, together with a selection function that predicts the information gain were the user to compare his envisioned target to the next node deeper in a given attribute ’s tree. It makes interactive search more efficient than existing strategies—both in terms of the system ’s selection time as well as the user’s feedback effort.
2 0.90206951 52 iccv-2013-Attribute Adaptation for Personalized Image Search
Author: Adriana Kovashka, Kristen Grauman
Abstract: Current methods learn monolithic attribute predictors, with the assumption that a single model is sufficient to reflect human understanding of a visual attribute. However, in reality, humans vary in how they perceive the association between a named property and image content. For example, two people may have slightly different internal models for what makes a shoe look “formal”, or they may disagree on which of two scenes looks “more cluttered”. Rather than discount these differences as noise, we propose to learn user-specific attribute models. We adapt a generic model trained with annotations from multiple users, tailoring it to satisfy user-specific labels. Furthermore, we propose novel techniques to infer user-specific labels based on transitivity and contradictions in the user’s search history. We demonstrate that adapted attributes improve accuracy over both existing monolithic models as well as models that learn from scratch with user-specific data alone. In addition, we show how adapted attributes are useful to personalize image search, whether with binary or relative attributes.
3 0.90162939 213 iccv-2013-Implied Feedback: Learning Nuances of User Behavior in Image Search
Author: Devi Parikh, Kristen Grauman
Abstract: User feedback helps an image search system refine its relevance predictions, tailoring the search towards the user’s preferences. Existing methods simply take feedback at face value: clicking on an image means the user wants things like it; commenting that an image lacks a specific attribute means the user wants things that have it. However, we expect there is actually more information behind the user’s literal feedback. In particular, a user’s (possibly subconscious) search strategy leads him to comment on certain images rather than others, based on how any of the visible candidate images compare to the desired content. For example, he may be more likely to give negative feedback on an irrelevant image that is relatively close to his target, as opposed to bothering with one that is altogether different. We introduce novel features to capitalize on such implied feedback cues, and learn a ranking function that uses them to improve the system’s relevance estimates. We validate the approach with real users searching for shoes, faces, or scenes using two different modes of feedback: binary relevance feedback and relative attributes-based feedback. The results show that retrieval improves significantly when the system accounts for the learned behaviors. We show that the nuances learned are domain-invariant, and useful for both generic user-independent search as well as personalized user-specific search.
4 0.89577997 231 iccv-2013-Latent Multitask Learning for View-Invariant Action Recognition
Author: Behrooz Mahasseni, Sinisa Todorovic
Abstract: This paper presents an approach to view-invariant action recognition, where human poses and motions exhibit large variations across different camera viewpoints. When each viewpoint of a given set of action classes is specified as a learning task then multitask learning appears suitable for achieving view invariance in recognition. We extend the standard multitask learning to allow identifying: (1) latent groupings of action views (i.e., tasks), and (2) discriminative action parts, along with joint learning of all tasks. This is because it seems reasonable to expect that certain distinct views are more correlated than some others, and thus identifying correlated views could improve recognition. Also, part-based modeling is expected to improve robustness against self-occlusion when actors are imaged from different views. Results on the benchmark datasets show that we outperform standard multitask learning by 21.9%, and the state-of-the-art alternatives by 4.5–6%.
Author: De-An Huang, Yu-Chiang Frank Wang
Abstract: Cross-domain image synthesis and recognition are typically considered as two distinct tasks in the areas of computer vision and pattern recognition. Therefore, it is not clear whether approaches addressing one task can be easily generalized or extended for solving the other. In this paper, we propose a unified model for coupled dictionary and feature space learning. The proposed learning model not only observes a common feature space for associating cross-domain image data for recognition purposes, the derived feature space is able to jointly update the dictionaries in each image domain for improved representation. This is why our method can be applied to both cross-domain image synthesis and recognition problems. Experiments on a variety of synthesis and recognition tasks such as single image super-resolution, cross-view action recognition, and sketchto-photo face recognition would verify the effectiveness of our proposed learning model.
6 0.88976878 44 iccv-2013-Adapting Classification Cascades to New Domains
7 0.88768882 167 iccv-2013-Finding Causal Interactions in Video Sequences
8 0.88753349 46 iccv-2013-Allocentric Pose Estimation
9 0.88709444 93 iccv-2013-Correlation Adaptive Subspace Segmentation by Trace Lasso
10 0.88655871 398 iccv-2013-Sparse Variation Dictionary Learning for Face Recognition with a Single Training Sample per Person
11 0.88588512 161 iccv-2013-Fast Sparsity-Based Orthogonal Dictionary Learning for Image Restoration
12 0.88469243 184 iccv-2013-Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion
13 0.8841083 335 iccv-2013-Random Faces Guided Sparse Many-to-One Encoder for Pose-Invariant Face Recognition
14 0.88400626 14 iccv-2013-A Generalized Iterated Shrinkage Algorithm for Non-convex Sparse Coding
15 0.88375992 70 iccv-2013-Cascaded Shape Space Pruning for Robust Facial Landmark Detection
16 0.88190722 259 iccv-2013-Manifold Based Face Synthesis from Sparse Samples
17 0.8782059 422 iccv-2013-Toward Guaranteed Illumination Models for Non-convex Objects
18 0.87664676 277 iccv-2013-Multi-channel Correlation Filters
19 0.87663782 45 iccv-2013-Affine-Constrained Group Sparse Coding and Its Application to Image-Based Classifications
20 0.87548429 154 iccv-2013-Face Recognition via Archetype Hull Ranking