iccv iccv2013 iccv2013-43 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Chengjiang Long, Gang Hua, Ashish Kapoor
Abstract: We present a noise resilient probabilistic model for active learning of a Gaussian process classifier from crowds, i.e., a set of noisy labelers. It explicitly models both the overall label noises and the expertise level of each individual labeler in two levels of flip models. Expectation propagation is adopted for efficient approximate Bayesian inference of our probabilistic model for classification, based on which, a generalized EM algorithm is derived to estimate both the global label noise and the expertise of each individual labeler. The probabilistic nature of our model immediately allows the adoption of the prediction entropy and estimated expertise for active selection of data sample to be labeled, and active selection of high quality labelers to label the data, respectively. We apply the proposed model for three visual recognition tasks, i.e, object category recognition, gender recognition, and multi-modal activity recognition, on three datasets with real crowd-sourced labels from Amazon Mechanical Turk. The experiments clearly demonstrated the efficacy of the proposed model.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract We present a noise resilient probabilistic model for active learning of a Gaussian process classifier from crowds, i. [sent-2, score-0.443]
2 It explicitly models both the overall label noises and the expertise level of each individual labeler in two levels of flip models. [sent-5, score-0.953]
3 Expectation propagation is adopted for efficient approximate Bayesian inference of our probabilistic model for classification, based on which, a generalized EM algorithm is derived to estimate both the global label noise and the expertise of each individual labeler. [sent-6, score-0.409]
4 The probabilistic nature of our model immediately allows the adoption of the prediction entropy and estimated expertise for active selection of data sample to be labeled, and active selection of high quality labelers to label the data, respectively. [sent-7, score-1.692]
5 Although it is cheap to obtain large quantity of labels through crowdsourcing, it has been well known that the collected labels could be very noisy. [sent-13, score-0.248]
6 So it is desirable to model the expertise level of the labelers to ensure the quality of the labels [6, 23, 1]. [sent-14, score-0.931]
7 The higher the expertise level a labeler is at, the lower the label noises he/she will produce. [sent-15, score-0.856]
8 The first approach attempts to evaluate the labelers by adopting a pre-labeled gold standard dataset [1]. [sent-17, score-0.639]
9 When a labeler is constantly generating contradicting labels on data samples from the gold standard dataset, all labels from that labeler may be discarded as he/she is highly likely to be an irresponsible one. [sent-18, score-1.423]
10 com ing the labels by collecting multiple labels for each data sample [6, 23]. [sent-20, score-0.281]
11 Then online or postmortem majority voting, or majority model consistency check is conducted to obtain the more likely ground-truth label of the data sample. [sent-21, score-0.304]
12 The basic assumption is that majority of the labelers are behaving in good faith. [sent-22, score-0.659]
13 The first approach is able to evaluate the labelers online, which is desirable. [sent-23, score-0.579]
14 It does not explicitly evaluate the labelers, although it may be extended to do so by online tracking how often a labeler is contradicting with the majority. [sent-26, score-0.571]
15 There lacks a principled approach to jointly model the global noise level of the labels and the expertise level of each individual labeler, in the absence of gold standard labels, which is what we want to achieve in this paper. [sent-28, score-0.42]
16 We present a Bayesian model (Figure 1), which explicitly models the global noise level of the labels and the expertise level of each individual labeler from crowds (i. [sent-29, score-0.872]
17 The resulting classifier is more resilient to label noises, adapting to the expertise of labelers. [sent-35, score-0.361]
18 Another improvement that can be made to current crowdsourcing labeling system such as Amazon Mechan- ical Turk (AMT) is to make it actively guide the labelers for more efficient labeling. [sent-36, score-0.655]
19 The proposed Bayesian model enables not only active selection of data samples to be labeled, but also active selection of quality labelers. [sent-37, score-0.782]
20 These are enabled by the probabilistic nature of our model and the explicit modeling of both global label noise and expertise of each individual labeler, thereby allowing entropy based uncertainty measure to be readily adopted for these purposes. [sent-38, score-0.398]
21 Several aspects distinguish our work from previous active learning based labeling [23, 2, 11, 15, 9]: first of all, our work deals with active learning with multiple labelers, a topic which has not been sufficiently explored before. [sent-39, score-0.578]
22 In other words, the labeler may label an example incorrectly. [sent-41, score-0.602]
23 Most previous work on active learning has assumed that the labels provided by the human oracle is noise free. [sent-42, score-0.452]
24 Thirdly, our model allows online evaluation of the quality of the labelers without relying on any additional pre-labeled gold standard data. [sent-43, score-0.736]
25 Hence we can select higher quality labelers and reduce the noise level of the labels we collected. [sent-44, score-0.824]
26 Related work Related works can be grouped into three categories including noise resilient Gaussian process classifiers [26, 10, 12], approximate Bayesian inference methods [17, 10, 18, 16], and active learning algorithms embracing crowdsourced labels [8, 25, 1, 23]. [sent-51, score-0.582]
27 A noise resilient likelihood model, namely flip noise model, is introduced in [16] to better handle label noises in Gaussian process classifier. [sent-53, score-0.415]
28 More recently, Kim and Ghahramani [12] exploited the flip noise model to explicitly handle outlier labels in Gaussian process classifier. [sent-54, score-0.26]
29 Several previous works have explored active learning from noisy crowd-sourced labels [1, 23] in different domains, where the two aforementioned approaches are exploited to handle label noise. [sent-57, score-0.551]
30 To better mitigate label noises online in the absence of gold standard labels, Donmez et al. [sent-58, score-0.286]
31 [8, 7] have explored confidence interval based estimation and sequential Bayesian estimation method to evaluate the label quality of the annotators in both stationary and non-stationary cases. [sent-59, score-0.233]
32 [29] proposed an incremental relabeling mechanism which employed active learning to not only select the unlabeled data to be labeled by the crowds, but also select already labeled data samples to be relabeled until sufficient confidence is built. [sent-61, score-0.525]
33 Later, they [5] proposed a method for pruning low-quality lablers by using the model trained from the entire labeled dataset from all labelers as ground truth. [sent-65, score-0.659]
34 The assumption is that good labelers will behave similarly. [sent-68, score-0.579]
35 These works build insights on how to deal with label noises and evaluate labeler quality. [sent-71, score-0.684]
36 In this sense, their models provided a more principled way for active data re-labeling. [sent-80, score-0.273]
37 In contrast, our proposed model actively induces a classifier which directly operates on visual features that directly extracted from images, which models the labelers’ quality in a principled way to facilitate active selection of annotators for providing better quality labels. [sent-81, score-0.619]
38 We denote ti = {tij }jM=1 as the set of labels from the M labelers for xi. [sent-91, score-0.703]
39 Hence, εj naturally represents the expertise or quality of the labels induced by labeler j. [sent-132, score-0.851]
40 Inference As a matter of fact, this two-level flip model can be conveniently collapsed by integrating yi out to obtain the joint probability p(ti |si, ε) = p(yi = +1|si , ξg) ? [sent-136, score-0.233]
41 Bayesian Active Learning For pool based active learning, we assume that we are given a pool of both labeled and unlabeled data samples X = {XL , XU}, and TL is the label set for XL from M Xlab =eler {sX. [sent-226, score-0.576]
42 The pro}p, oasnedd T Tmodel conveniently allows for both active selection of unlabeled data samples to be labeled, and also active selection of higher quality labelers. [sent-227, score-0.846]
43 j in our model directly models the labeler j’s quality. [sent-238, score-0.499]
44 It can be regarded as the probability that labeler j would label the data correctly. [sent-239, score-0.602]
45 In our active learning process, we can naturally select the top K < M labelers with the top K ? [sent-242, score-0.894]
46 The joint active selection of both labelers and data samples greatly facilitates to obtain higher quality labels. [sent-246, score-1.037]
47 Another active learning strategy is to only actively select the data sample to be labeled by all M labelers. [sent-247, score-0.426]
48 In total there are 2682 labeled video clips, each has 7 copies of labels from Amazon Mechanical Turk. [sent-271, score-0.245]
49 Since many of the classes have limited labeled clips, and also considering that the raw label accuracy of action 3 and action 5 is less than 50%, which fail all the classifiers we tried. [sent-288, score-0.238]
50 We choose to × work on the classification problem of action 9 only, which has sufficient number of labeled clips and its label accuracy is 75. [sent-289, score-0.235]
51 We collected 5 copies of gender labels for 9441 face images. [sent-296, score-0.27]
52 The rest of the images in “Meerkat, meerkat” and an equal number of images from the other two classes are put together to form the active learning pool. [sent-309, score-0.315]
53 We simulate the case that there are 2, 3, 4 bad labelers, who would randomly assign a label to the sample, so there is 50% chance that the label from them will be erroneous. [sent-310, score-0.297]
54 Therefore, We run our proposed active learning algorithm for both active selection of data samples and labelers. [sent-312, score-0.667]
55 We name our algorithm as JGPC-ASAL, stands for joint learning GPC with active selection of both samples and labelers (we call it joint learning in the sense that the multiple labels of a single example is jointly considered). [sent-316, score-1.211]
56 We compare with a combination ofother learning strategies with our model, such as active selection of samples but random selection of labelers, random selection of samples and active selection of labelers, and random selection of both samples and labelers. [sent-317, score-1.127]
57 For all these online learning algorithms based on JGPC, we select 3 labelers to provide the label using the corresponding criterion for labeler selection. [sent-319, score-1.313]
58 One algorithm we compare against is an active learning GP classifier with the global flip noise observation model similar to the model in [12]. [sent-320, score-0.456]
59 For this method, at each round, we use the prediction entropy to select the next sample to be labeled and majority voting is performed to obtain a single label from all 7 copies of labels. [sent-321, score-0.442]
60 We name it as majority vote active learning GPC with flip noise model, or in short GPC-MVAS-F. [sent-322, score-0.505]
61 The corresponding algorithm performing random sample selection using majority voted label, is named as GPC-MVRS-F. [sent-323, score-0.246]
62 Another algorithm we compare against is based on the active learning GP classifier proposed by Kapoor et al. [sent-324, score-0.32]
63 [2], where a Gaussian observation model is adopted, and a confidence criterion normalized by the variance of the posterior prediction is adopted for active learning. [sent-325, score-0.303]
64 Again, majority voting is performed at each active learning step to obtain a single label from all 7 copies of noisy labels. [sent-326, score-0.634]
65 Since there are no labeler selection mechanism, we simply gather majority voted labels from all 7 labelers. [sent-328, score-0.836]
66 Suggesting that when the labels are less noisy, active selection of samples are more important than active selection of labelers, which intuitively makes sense as the label quality is high. [sent-333, score-1.009]
67 In all cases for all algorithms, the active sample selection strategy always outperforms its random sample selection counterpart, which suggest that the proposed active learning criterion is robust against label noises. [sent-337, score-0.93]
68 Figure 3 visualizes the top three labelers selected at each active learning step when running the proposed JGPCASAL algorithm on the “Meerkat, meerkat” class with 4 bad labelers (labeler 4, 5, 6, and 7 are bad labelers). [sent-338, score-1.629]
69 The red, blue, and green color circles represents the top three labelers selected based on the estimated labeler quality measure at each active learning step. [sent-339, score-1.423]
70 Then with the progression of the active learning process, the three good labelers (labeler 1,2,and 3) got constantly selected. [sent-341, score-0.896]
71 Labelers with different expertise: To better understand the behavior of our algorithm, we run two sets of experiments with simulated label noises from the ground-truth labels on the 3 categories of ImageNet dataset. [sent-343, score-0.336]
72 In the first experiment, we simulated the case that each labeler produces 10%, 15%, 20%, 25%, 30%, 35% and 40% erroneous labels, respectively. [sent-345, score-0.526]
73 In the second experiment, we increase the label noise level to have each labeler to produce 15%, 20%, 25%, 30%, 35%, 40%, and 45% erroneous labels, respectively. [sent-346, score-0.641]
74 We also impose a naive majority voting consensus based labeler selection scheme to the GPC-MVAS-F and GPC-MVRS-F algorithm. [sent-347, score-0.76]
75 For each labeler, we record online his rate of consistent labels with the corresponding majority voted labels. [sent-348, score-0.278]
76 We call the GPC-MVAS-F and GPC-MVRS-F algorithm equipped with this simple active labeler selection scheme as GPC-MVASAL-F and GPCMVRSAL-F, respectively. [sent-350, score-0.847]
77 To validate its efficacy, we also compare against its corresponding random labeler selection version, namely GPC-MVASRL-F and GPC-MVRSRL-F. [sent-351, score-0.599]
78 Again, at each step of the online learning process, we select 3 labelers to provide the labels. [sent-352, score-0.687]
79 (2) The naive majority voting consensus based labeler selection criterion is also effective, as it achieved better accuracy than its random labeler selection counterpart. [sent-355, score-1.383]
80 The rest of the examples in the target class and an equal number of examples from the other two classes are put in the active learning pool. [sent-358, score-0.315]
81 Since the 7 copies of labels we collected from Amazon Mechanical Turk do not really entail labels from bad labelers, we found that active selection of higher quality labelers does not really improve recognition accuracy much. [sent-359, score-1.394]
82 Hence in this experiments we only do active sample selection, and assume all the 7 labelers will all label the selected example. [sent-360, score-0.963]
83 We argue that our joint treatment of multiple labels in GPC in general is superior than the majority voting strategy (GPC-MVAS-F and GPC-MVAS-K), as manifested by the results shown in Figure 5. [sent-362, score-0.309]
84 We also compare against two versions of the active learning algorithm proposed by Yan et al. [sent-363, score-0.289]
85 333000000555 Number of label s (b) Hold-out testing set - 10%-40% for each labeler irnygctoARaue0. [sent-365, score-0.634]
86 785901 2530G P C-MJ G VP4 AC0RS-A RSLA -F4L50 Number of labe l s (c) Active learning pool - 15%-45% for each labeler giuAyacoRrnte0 . [sent-366, score-0.621]
87 Again, active sample selection always achieves better performance than random sample selection. [sent-372, score-0.414]
88 The rest 404 clips of action 9 and the same number of clips from the other actions are used as the active learning pool. [sent-377, score-0.425]
89 The rest of the face images with different percentage of label inconsistency are used as the active learning pool. [sent-383, score-0.392]
90 As shown in Figure 7, the proposed JGPC-AS algorithm again showed superior recognition accuracy when compared with the GPC-MVASF and GPC-MVAS-K algorithms, in both the active learning pool and the hold-out testing set. [sent-384, score-0.378]
91 It is also obvious that algorithms performing active learning always achieved better performance when compared with their random learning counterparts. [sent-385, score-0.33]
92 Conclusion In this paper, we present a hierarchical Bayesian model to learn a Gaussian process classifier from crowd-sourced labels by jointly considering multiple labels instead of taking the majority voting. [sent-387, score-0.359]
93 Our two-level flip model enables us to design principled active learning strategy to not only select data sample, but also select quality labelers. [sent-388, score-0.519]
94 Our experiments on three visual recognition datasets with realcrowdsourced labels clearly demonstrated that the active selection of labelers is beneficial when there are a lot of careless labelers. [sent-389, score-1.051]
95 Our joint treatment of multiple labels for each data sample is also proven to be superior to the online majority voting scheme. [sent-390, score-0.383]
96 Our future work will further explore how to design an active learning machine to jointly select both the user and sample in a single criterion. [sent-392, score-0.348]
97 Ralf: A reinforced active learning formulation for object class recognition. [sent-462, score-0.289]
98 Which faces to tag: Adding prior constraints into active learning. [sent-474, score-0.248]
99 Large-scale live active learning: Training object detectors with crawled data and crowds. [sent-559, score-0.248]
100 Incremental relabeling for active learning with noisy crowdsourced annotations. [sent-597, score-0.388]
wordName wordTfidf (topN-words)
[('labelers', 0.579), ('labeler', 0.499), ('active', 0.248), ('meerkat', 0.194), ('expertise', 0.172), ('labels', 0.124), ('label', 0.103), ('selection', 0.1), ('flip', 0.097), ('gpc', 0.094), ('tij', 0.093), ('bad', 0.091), ('noises', 0.082), ('majority', 0.08), ('annotators', 0.074), ('gender', 0.074), ('copies', 0.072), ('si', 0.072), ('xu', 0.07), ('imagenet', 0.066), ('yu', 0.062), ('gold', 0.06), ('bayesian', 0.059), ('dl', 0.058), ('pool', 0.057), ('xl', 0.056), ('quality', 0.056), ('voting', 0.055), ('resilient', 0.055), ('yi', 0.054), ('clips', 0.053), ('amazon', 0.052), ('kapoor', 0.051), ('mechanical', 0.051), ('ep', 0.05), ('labeled', 0.049), ('annotator', 0.049), ('welinder', 0.049), ('crowdsourcing', 0.047), ('gaussian', 0.047), ('sl', 0.047), ('tl', 0.044), ('efficacy', 0.042), ('donmez', 0.041), ('online', 0.041), ('logp', 0.041), ('learning', 0.041), ('crowdsourced', 0.04), ('noise', 0.039), ('dekel', 0.038), ('raykar', 0.038), ('stevens', 0.038), ('crowds', 0.038), ('turk', 0.035), ('inference', 0.035), ('noisy', 0.035), ('lcc', 0.034), ('sample', 0.033), ('voted', 0.033), ('unlabeled', 0.032), ('conveniently', 0.032), ('expectation', 0.032), ('testing', 0.032), ('contradicting', 0.031), ('kut', 0.031), ('lablers', 0.031), ('yisi', 0.031), ('yitij', 0.031), ('adopted', 0.031), ('classifier', 0.031), ('vu', 0.031), ('action', 0.03), ('samples', 0.03), ('probabilistic', 0.029), ('actively', 0.029), ('constantly', 0.028), ('fung', 0.028), ('irresponsible', 0.028), ('rosales', 0.028), ('simulated', 0.027), ('jm', 0.027), ('zhao', 0.027), ('classes', 0.026), ('consensus', 0.026), ('select', 0.026), ('collapsed', 0.026), ('treatment', 0.026), ('principled', 0.025), ('criterion', 0.024), ('joint', 0.024), ('labe', 0.024), ('relabeling', 0.024), ('entropy', 0.024), ('convolutional', 0.023), ('carbonell', 0.023), ('su', 0.023), ('integral', 0.023), ('mu', 0.022), ('twist', 0.022), ('cvprw', 0.022)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999958 43 iccv-2013-Active Visual Recognition with Expertise Estimation in Crowdsourcing
Author: Chengjiang Long, Gang Hua, Ashish Kapoor
Abstract: We present a noise resilient probabilistic model for active learning of a Gaussian process classifier from crowds, i.e., a set of noisy labelers. It explicitly models both the overall label noises and the expertise level of each individual labeler in two levels of flip models. Expectation propagation is adopted for efficient approximate Bayesian inference of our probabilistic model for classification, based on which, a generalized EM algorithm is derived to estimate both the global label noise and the expertise of each individual labeler. The probabilistic nature of our model immediately allows the adoption of the prediction entropy and estimated expertise for active selection of data sample to be labeled, and active selection of high quality labelers to label the data, respectively. We apply the proposed model for three visual recognition tasks, i.e, object category recognition, gender recognition, and multi-modal activity recognition, on three datasets with real crowd-sourced labels from Amazon Mechanical Turk. The experiments clearly demonstrated the efficacy of the proposed model.
2 0.73101473 80 iccv-2013-Collaborative Active Learning of a Kernel Machine Ensemble for Recognition
Author: Gang Hua, Chengjiang Long, Ming Yang, Yan Gao
Abstract: Active learning is an effective way of engaging users to interactively train models for visual recognition. The vast majority of previous works, if not all of them, focused on active learning with a single human oracle. The problem of active learning with multiple oracles in a collaborative setting has not been well explored. Moreover, most of the previous works assume that the labels provided by the human oracles are noise free, which may often be violated in reality. We present a collaborative computational model for active learning with multiple human oracles. It leads to not only an ensemble kernel machine that is robust to label noises, but also a principled label quality measure to online detect irresponsible labelers. Instead of running independent active learning processes for each individual human oracle, our model captures the inherent correlations among the labelers through shared data among them. Our simulation experiments and experiments with real crowd-sourced noisy labels demonstrated the efficacy of our model.
3 0.1676835 6 iccv-2013-A Convex Optimization Framework for Active Learning
Author: Ehsan Elhamifar, Guillermo Sapiro, Allen Yang, S. Shankar Sasrty
Abstract: In many image/video/web classification problems, we have access to a large number of unlabeled samples. However, it is typically expensive and time consuming to obtain labels for the samples. Active learning is the problem of progressively selecting and annotating the most informative unlabeled samples, in order to obtain a high classification performance. Most existing active learning algorithms select only one sample at a time prior to retraining the classifier. Hence, they are computationally expensive and cannot take advantage of parallel labeling systems such as Mechanical Turk. On the other hand, algorithms that allow the selection of multiple samples prior to retraining the classifier, may select samples that have significant information overlap or they involve solving a non-convex optimization. More importantly, the majority of active learning algorithms are developed for a certain classifier type such as SVM. In this paper, we develop an efficient active learning framework based on convex programming, which can select multiple samples at a time for annotation. Unlike the state of the art, our algorithm can be used in conjunction with any type of classifiers, including those of the fam- ily of the recently proposed Sparse Representation-based Classification (SRC). We use the two principles of classifier uncertainty and sample diversity in order to guide the optimization program towards selecting the most informative unlabeled samples, which have the least information overlap. Our method can incorporate the data distribution in the selection process by using the appropriate dissimilarity between pairs of samples. We show the effectiveness of our framework in person detection, scene categorization and face recognition on real-world datasets.
4 0.14922369 41 iccv-2013-Active Learning of an Action Detector from Untrimmed Videos
Author: Sunil Bandla, Kristen Grauman
Abstract: Collecting and annotating videos of realistic human actions is tedious, yet critical for training action recognition systems. We propose a method to actively request the most useful video annotations among a large set of unlabeled videos. Predicting the utility of annotating unlabeled video is not trivial, since any given clip may contain multiple actions of interest, and it need not be trimmed to temporal regions of interest. To deal with this problem, we propose a detection-based active learner to train action category models. We develop a voting-based framework to localize likely intervals of interest in an unlabeled clip, and use them to estimate the total reduction in uncertainty that annotating that clip would yield. On three datasets, we show our approach can learn accurate action detectors more efficiently than alternative active learning strategies that fail to accommodate the “untrimmed” nature of real video data.
5 0.069947667 126 iccv-2013-Dynamic Label Propagation for Semi-supervised Multi-class Multi-label Classification
Author: Bo Wang, Zhuowen Tu, John K. Tsotsos
Abstract: In graph-based semi-supervised learning approaches, the classification rate is highly dependent on the size of the availabel labeled data, as well as the accuracy of the similarity measures. Here, we propose a semi-supervised multi-class/multi-label classification scheme, dynamic label propagation (DLP), which performs transductive learning through propagation in a dynamic process. Existing semi-supervised classification methods often have difficulty in dealing with multi-class/multi-label problems due to the lack in consideration of label correlation; our algorithm instead emphasizes dynamic metric fusion with label information. Significant improvement over the state-of-the-art methods is observed on benchmark datasets for both multiclass and multi-label tasks.
6 0.069684304 233 iccv-2013-Latent Task Adaptation with Large-Scale Hierarchies
7 0.065136045 52 iccv-2013-Attribute Adaptation for Personalized Image Search
8 0.065006606 42 iccv-2013-Active MAP Inference in CRFs for Efficient Semantic Segmentation
9 0.059743337 54 iccv-2013-Attribute Pivots for Guiding Relevance Feedback in Image Search
10 0.055292644 142 iccv-2013-Ensemble Projection for Semi-supervised Image Classification
11 0.054958742 26 iccv-2013-A Practical Transfer Learning Algorithm for Face Verification
12 0.052524291 191 iccv-2013-Handling Uncertain Tags in Visual Recognition
13 0.051404405 326 iccv-2013-Predicting Sufficient Annotation Strength for Interactive Foreground Segmentation
14 0.051349759 194 iccv-2013-Heterogeneous Image Features Integration via Multi-modal Semi-supervised Learning Model
15 0.051121794 386 iccv-2013-Sequential Bayesian Model Update under Structured Scene Prior for Semantic Road Scenes Labeling
16 0.051042292 338 iccv-2013-Randomized Ensemble Tracking
17 0.047468822 354 iccv-2013-Robust Dictionary Learning by Error Source Decomposition
18 0.046737179 223 iccv-2013-Joint Noise Level Estimation from Personal Photo Collections
19 0.046061419 188 iccv-2013-Group Sparsity and Geometry Constrained Dictionary Learning for Action Recognition from Depth Maps
20 0.044888165 321 iccv-2013-Pose-Free Facial Landmark Fitting via Optimized Part Mixtures and Cascaded Deformable Shape Model
topicId topicWeight
[(0, 0.134), (1, 0.079), (2, -0.02), (3, -0.017), (4, 0.013), (5, -0.011), (6, 0.005), (7, 0.026), (8, 0.022), (9, -0.048), (10, -0.025), (11, -0.038), (12, -0.029), (13, -0.038), (14, 0.1), (15, -0.058), (16, -0.09), (17, -0.052), (18, -0.08), (19, -0.011), (20, -0.06), (21, -0.073), (22, -0.218), (23, 0.08), (24, 0.002), (25, -0.028), (26, 0.307), (27, 0.199), (28, 0.219), (29, -0.111), (30, -0.209), (31, -0.028), (32, 0.293), (33, -0.3), (34, -0.019), (35, 0.012), (36, 0.155), (37, -0.141), (38, 0.012), (39, 0.007), (40, -0.117), (41, 0.023), (42, 0.115), (43, -0.08), (44, -0.18), (45, 0.2), (46, -0.023), (47, 0.034), (48, 0.051), (49, -0.028)]
simIndex simValue paperId paperTitle
same-paper 1 0.94532025 43 iccv-2013-Active Visual Recognition with Expertise Estimation in Crowdsourcing
Author: Chengjiang Long, Gang Hua, Ashish Kapoor
Abstract: We present a noise resilient probabilistic model for active learning of a Gaussian process classifier from crowds, i.e., a set of noisy labelers. It explicitly models both the overall label noises and the expertise level of each individual labeler in two levels of flip models. Expectation propagation is adopted for efficient approximate Bayesian inference of our probabilistic model for classification, based on which, a generalized EM algorithm is derived to estimate both the global label noise and the expertise of each individual labeler. The probabilistic nature of our model immediately allows the adoption of the prediction entropy and estimated expertise for active selection of data sample to be labeled, and active selection of high quality labelers to label the data, respectively. We apply the proposed model for three visual recognition tasks, i.e, object category recognition, gender recognition, and multi-modal activity recognition, on three datasets with real crowd-sourced labels from Amazon Mechanical Turk. The experiments clearly demonstrated the efficacy of the proposed model.
2 0.92676777 80 iccv-2013-Collaborative Active Learning of a Kernel Machine Ensemble for Recognition
Author: Gang Hua, Chengjiang Long, Ming Yang, Yan Gao
Abstract: Active learning is an effective way of engaging users to interactively train models for visual recognition. The vast majority of previous works, if not all of them, focused on active learning with a single human oracle. The problem of active learning with multiple oracles in a collaborative setting has not been well explored. Moreover, most of the previous works assume that the labels provided by the human oracles are noise free, which may often be violated in reality. We present a collaborative computational model for active learning with multiple human oracles. It leads to not only an ensemble kernel machine that is robust to label noises, but also a principled label quality measure to online detect irresponsible labelers. Instead of running independent active learning processes for each individual human oracle, our model captures the inherent correlations among the labelers through shared data among them. Our simulation experiments and experiments with real crowd-sourced noisy labels demonstrated the efficacy of our model.
3 0.66808915 6 iccv-2013-A Convex Optimization Framework for Active Learning
Author: Ehsan Elhamifar, Guillermo Sapiro, Allen Yang, S. Shankar Sasrty
Abstract: In many image/video/web classification problems, we have access to a large number of unlabeled samples. However, it is typically expensive and time consuming to obtain labels for the samples. Active learning is the problem of progressively selecting and annotating the most informative unlabeled samples, in order to obtain a high classification performance. Most existing active learning algorithms select only one sample at a time prior to retraining the classifier. Hence, they are computationally expensive and cannot take advantage of parallel labeling systems such as Mechanical Turk. On the other hand, algorithms that allow the selection of multiple samples prior to retraining the classifier, may select samples that have significant information overlap or they involve solving a non-convex optimization. More importantly, the majority of active learning algorithms are developed for a certain classifier type such as SVM. In this paper, we develop an efficient active learning framework based on convex programming, which can select multiple samples at a time for annotation. Unlike the state of the art, our algorithm can be used in conjunction with any type of classifiers, including those of the fam- ily of the recently proposed Sparse Representation-based Classification (SRC). We use the two principles of classifier uncertainty and sample diversity in order to guide the optimization program towards selecting the most informative unlabeled samples, which have the least information overlap. Our method can incorporate the data distribution in the selection process by using the appropriate dissimilarity between pairs of samples. We show the effectiveness of our framework in person detection, scene categorization and face recognition on real-world datasets.
4 0.43950629 41 iccv-2013-Active Learning of an Action Detector from Untrimmed Videos
Author: Sunil Bandla, Kristen Grauman
Abstract: Collecting and annotating videos of realistic human actions is tedious, yet critical for training action recognition systems. We propose a method to actively request the most useful video annotations among a large set of unlabeled videos. Predicting the utility of annotating unlabeled video is not trivial, since any given clip may contain multiple actions of interest, and it need not be trimmed to temporal regions of interest. To deal with this problem, we propose a detection-based active learner to train action category models. We develop a voting-based framework to localize likely intervals of interest in an unlabeled clip, and use them to estimate the total reduction in uncertainty that annotating that clip would yield. On three datasets, we show our approach can learn accurate action detectors more efficiently than alternative active learning strategies that fail to accommodate the “untrimmed” nature of real video data.
5 0.36657688 42 iccv-2013-Active MAP Inference in CRFs for Efficient Semantic Segmentation
Author: Gemma Roig, Xavier Boix, Roderick De_Nijs, Sebastian Ramos, Koljia Kuhnlenz, Luc Van_Gool
Abstract: Most MAP inference algorithms for CRFs optimize an energy function knowing all the potentials. In this paper, we focus on CRFs where the computational cost of instantiating the potentials is orders of magnitude higher than MAP inference. This is often the case in semantic image segmentation, where most potentials are instantiated by slow classifiers fed with costly features. We introduce Active MAP inference 1) to on-the-fly select a subset of potentials to be instantiated in the energy function, leaving the rest of the parameters of the potentials unknown, and 2) to estimate the MAP labeling from such incomplete energy function. Results for semantic segmentation benchmarks, namely PASCAL VOC 2010 [5] and MSRC-21 [19], show that Active MAP inference achieves similar levels of accuracy but with major efficiency gains.
6 0.34406957 234 iccv-2013-Learning CRFs for Image Parsing with Adaptive Subgradient Descent
7 0.33420506 126 iccv-2013-Dynamic Label Propagation for Semi-supervised Multi-class Multi-label Classification
8 0.31215325 178 iccv-2013-From Semi-supervised to Transfer Counting of Crowds
9 0.28442597 386 iccv-2013-Sequential Bayesian Model Update under Structured Scene Prior for Semantic Road Scenes Labeling
10 0.2769078 194 iccv-2013-Heterogeneous Image Features Integration via Multi-modal Semi-supervised Learning Model
11 0.27633095 125 iccv-2013-Drosophila Embryo Stage Annotation Using Label Propagation
12 0.27288505 141 iccv-2013-Enhanced Continuous Tabu Search for Parameter Estimation in Multiview Geometry
13 0.26816729 142 iccv-2013-Ensemble Projection for Semi-supervised Image Classification
14 0.25971261 191 iccv-2013-Handling Uncertain Tags in Visual Recognition
15 0.23914415 20 iccv-2013-A Max-Margin Perspective on Sparse Representation-Based Classification
16 0.23148607 211 iccv-2013-Image Segmentation with Cascaded Hierarchical Models and Logistic Disjunctive Normal Networks
17 0.2166397 75 iccv-2013-CoDeL: A Human Co-detection and Labeling Framework
18 0.21043643 285 iccv-2013-NEIL: Extracting Visual Knowledge from Web Data
19 0.20989195 324 iccv-2013-Potts Model, Parametric Maxflow and K-Submodular Functions
20 0.20423634 60 iccv-2013-Bayesian Robust Matrix Factorization for Image and Video Processing
topicId topicWeight
[(2, 0.085), (7, 0.023), (26, 0.067), (30, 0.252), (31, 0.047), (33, 0.012), (34, 0.016), (42, 0.104), (48, 0.017), (64, 0.031), (73, 0.043), (78, 0.017), (89, 0.134), (98, 0.034)]
simIndex simValue paperId paperTitle
same-paper 1 0.76944649 43 iccv-2013-Active Visual Recognition with Expertise Estimation in Crowdsourcing
Author: Chengjiang Long, Gang Hua, Ashish Kapoor
Abstract: We present a noise resilient probabilistic model for active learning of a Gaussian process classifier from crowds, i.e., a set of noisy labelers. It explicitly models both the overall label noises and the expertise level of each individual labeler in two levels of flip models. Expectation propagation is adopted for efficient approximate Bayesian inference of our probabilistic model for classification, based on which, a generalized EM algorithm is derived to estimate both the global label noise and the expertise of each individual labeler. The probabilistic nature of our model immediately allows the adoption of the prediction entropy and estimated expertise for active selection of data sample to be labeled, and active selection of high quality labelers to label the data, respectively. We apply the proposed model for three visual recognition tasks, i.e, object category recognition, gender recognition, and multi-modal activity recognition, on three datasets with real crowd-sourced labels from Amazon Mechanical Turk. The experiments clearly demonstrated the efficacy of the proposed model.
2 0.72897995 334 iccv-2013-Query-Adaptive Asymmetrical Dissimilarities for Visual Object Retrieval
Author: Cai-Zhi Zhu, Hervé Jégou, Shin'Ichi Satoh
Abstract: Visual object retrieval aims at retrieving, from a collection of images, all those in which a given query object appears. It is inherently asymmetric: the query object is mostly included in the database image, while the converse is not necessarily true. However, existing approaches mostly compare the images with symmetrical measures, without considering the different roles of query and database. This paper first measure the extent of asymmetry on large-scale public datasets reflecting this task. Considering the standard bag-of-words representation, we then propose new asymmetrical dissimilarities accounting for the different inlier ratios associated with query and database images. These asymmetrical measures depend on the query, yet they are compatible with an inverted file structure, without noticeably impacting search efficiency. Our experiments show the benefit of our approach, and show that the visual object retrieval task is better treated asymmetrically, in the spirit of state-of-the-art text retrieval.
3 0.72207916 80 iccv-2013-Collaborative Active Learning of a Kernel Machine Ensemble for Recognition
Author: Gang Hua, Chengjiang Long, Ming Yang, Yan Gao
Abstract: Active learning is an effective way of engaging users to interactively train models for visual recognition. The vast majority of previous works, if not all of them, focused on active learning with a single human oracle. The problem of active learning with multiple oracles in a collaborative setting has not been well explored. Moreover, most of the previous works assume that the labels provided by the human oracles are noise free, which may often be violated in reality. We present a collaborative computational model for active learning with multiple human oracles. It leads to not only an ensemble kernel machine that is robust to label noises, but also a principled label quality measure to online detect irresponsible labelers. Instead of running independent active learning processes for each individual human oracle, our model captures the inherent correlations among the labelers through shared data among them. Our simulation experiments and experiments with real crowd-sourced noisy labels demonstrated the efficacy of our model.
4 0.6782161 120 iccv-2013-Discriminative Label Propagation for Multi-object Tracking with Sporadic Appearance Features
Author: K.C. Amit Kumar, Christophe De_Vleeschouwer
Abstract: Given a set of plausible detections, detected at each time instant independently, we investigate how to associate them across time. This is done by propagating labels on a set of graphs that capture how the spatio-temporal and the appearance cues promote the assignment of identical or distinct labels to a pair of nodes. The graph construction is driven by the locally linear embedding (LLE) of either the spatio-temporal or the appearance features associated to the detections. Interestingly, the neighborhood of a node in each appearance graph is defined to include all nodes for which the appearance feature is available (except the ones that coexist at the same time). This allows to connect the nodes that share the same appearance even if they are temporally distant, which gives our framework the uncommon ability to exploit the appearance features that are available only sporadically along the sequence of detections. Once the graphs have been defined, the multi-object tracking is formulated as the problem of finding a label assignment that is consistent with the constraints captured by each of the graphs. This results into a difference of convex program that can be efficiently solved. Experiments are performed on a basketball and several well-known pedestrian datasets in order to validate the effectiveness of the proposed solution.
Author: Basura Fernando, Tinne Tuytelaars
Abstract: In this paper we present a new method for object retrieval starting from multiple query images. The use of multiple queries allows for a more expressive formulation of the query object including, e.g., different viewpoints and/or viewing conditions. This, in turn, leads to more diverse and more accurate retrieval results. When no query images are available to the user, they can easily be retrieved from the internet using a standard image search engine. In particular, we propose a new method based on pattern mining. Using the minimal description length principle, we derive the most suitable set of patterns to describe the query object, with patterns corresponding to local feature configurations. This results in apowerful object-specific mid-level image representation. The archive can then be searched efficiently for similar images based on this representation, using a combination of two inverted file systems. Since the patterns already encode local spatial information, good results on several standard image retrieval datasets are obtained even without costly re-ranking based on geometric verification.
6 0.63148201 384 iccv-2013-Semi-supervised Robust Dictionary Learning via Efficient l-Norms Minimization
7 0.6306352 126 iccv-2013-Dynamic Label Propagation for Semi-supervised Multi-class Multi-label Classification
8 0.63032895 427 iccv-2013-Transfer Feature Learning with Joint Distribution Adaptation
9 0.6299814 180 iccv-2013-From Where and How to What We See
10 0.62853163 123 iccv-2013-Domain Adaptive Classification
11 0.62829232 181 iccv-2013-Frustratingly Easy NBNN Domain Adaptation
12 0.62748033 194 iccv-2013-Heterogeneous Image Features Integration via Multi-modal Semi-supervised Learning Model
13 0.62626636 137 iccv-2013-Efficient Salient Region Detection with Soft Image Abstraction
14 0.62579507 44 iccv-2013-Adapting Classification Cascades to New Domains
15 0.62560248 328 iccv-2013-Probabilistic Elastic Part Model for Unsupervised Face Detector Adaptation
16 0.62528944 233 iccv-2013-Latent Task Adaptation with Large-Scale Hierarchies
17 0.62462848 150 iccv-2013-Exemplar Cut
18 0.62425226 327 iccv-2013-Predicting an Object Location Using a Global Image Representation
19 0.62417328 197 iccv-2013-Hierarchical Joint Max-Margin Learning of Mid and Top Level Representations for Visual Recognition
20 0.62410104 406 iccv-2013-Style-Aware Mid-level Representation for Discovering Visual Connections in Space and Time