cvpr cvpr2013 cvpr2013-15 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Peter Welinder, Max Welling, Pietro Perona
Abstract: How many labeled examples are needed to estimate a classifier’s performance on a new dataset? We study the case where data is plentiful, but labels are expensive. We show that by making a few reasonable assumptions on the structure of the data, it is possible to estimate performance curves, with confidence bounds, using a small number of ground truth labels. Our approach, which we call Semisupervised Performance Evaluation (SPE), is based on a generative model for the classifier’s confidence scores. In addition to estimating the performance of classifiers on new datasets, SPE can be used to recalibrate a classifier by reestimating the class-conditional confidence distributions.
Reference: text
sentIndex sentText sentNum sentScore
1 We show that by making a few reasonable assumptions on the structure of the data, it is possible to estimate performance curves, with confidence bounds, using a small number of ground truth labels. [sent-8, score-0.21]
2 In addition to estimating the performance of classifiers on new datasets, SPE can be used to recalibrate a classifier by reestimating the class-conditional confidence distributions. [sent-10, score-0.423]
3 The pedestrian detector was laboriously trained by a research group who labeled thousands of training and validation examples and publishes good experimental results (see e. [sent-14, score-0.184]
4 Should the urban planner trust the published performance figures and assume that the detector will perform equally well on her images? [sent-17, score-0.246]
5 In order to be sure, the planner needs to compute precision and recall on her dataset. [sent-20, score-0.205]
6 Is it possible at all to obtain reliable bounds on the performance of a detector / classifier without relabeling a new dataset? [sent-26, score-0.323]
7 A pedestrian detector trained on vacation images (the INRIA dataset [5]) performs well on images taken in natural environments (top), and fails miserably on images taken in an urban environment (bottom). [sent-28, score-0.263]
8 Can we estimate the performance of a pretrained classifier / detector on a novel data set? [sent-29, score-0.308]
9 A: Histogram of classifier scores si obtained by running the “ChnFtrs” detector [7] on the INRIA dataset [5]. [sent-34, score-0.435]
10 The red and green curves show the Gamma-Normal mixture model fitting the histogrammed scores with highest likelihood. [sent-35, score-0.348]
11 The red and green bars show the labels of the 10 randomly sampled labels (by chance, the scores for some of the samples are close to each other, thus only 6 bars are shown; the height of the bars has no meaning). [sent-38, score-0.533]
12 B: Precision and recall curves computed from the mixture model in A. [sent-39, score-0.223]
13 C: In black, precision-recall curve computed after all items have been labeled. [sent-40, score-0.297]
14 In red, precision-recall curve estimated using SPE from only 10 labeled examples (with 90% confidence interval shown as the magenta band). [sent-41, score-0.264]
15 We propose a method for achieving minimally supervised evaluation of classifiers, requiring as few as 10 labels to accurately estimate classifier performance. [sent-43, score-0.318]
16 Our method ×× is based on a generative Bayesian model for the confidence scores produced by the classifier, borrowing from the literature on semisupervised learning [16, 20, 2 1]. [sent-44, score-0.27]
17 We show how to use the model to re-calibrate classifiers to new datasets by choosing thresholds to satisfy performance constraints with high likelihood. [sent-45, score-0.184]
18 Modeling the classifier score Let us start with a set of N data items, (xi, yi) ∈ RD {0, 1}, drawn from some unknown distribution p)(x ∈, yR) an×d i{n0d,e1x}e,d d by in r∈o m{ s1,o . [sent-48, score-0.277]
19 e τ is some scalar threshold, has been used to classify all data items into two classes, yˆi ∈ {0, 1}. [sent-56, score-0.18]
20 While the “ground truth” labels yi are assumed t∈o {be0 ,u1n}k. [sent-57, score-0.205]
21 This is an expensive operation and our goal is to ask the oracle for as few labels as possible. [sent-71, score-0.229]
22 The set of items that have been labeled by the oracle at time t is denoted by Lt and its complement, the soerta colef i attem tims feor t w ish dicehn othteed ground truth is unknown, is denoted Ut. [sent-72, score-0.37]
23 A: Estimation error, as measured by the area between the true and predicted precision-recall curves, versus the number of labels sampled, for the ChnFtrs detector on the CPD dataset. [sent-95, score-0.184]
24 The red curve is SPE and the green curve shows the median error of the naive method (RND). [sent-96, score-0.345]
25 B: The performance curve estimated using SPE (red) with 90% confidence intervals (magenta) with 20 known labels. [sent-98, score-0.261]
26 The ground truth performance with all label known is shown as a black curve (GT), and the performance curve computed on 20 labels using the naive method from 5 random samples is shown in green (RND). [sent-99, score-0.567]
27 C–D: same as A–B, but for the logres8 classifier on the DGT dataset (hand-picked as an example where SPE does not work well). [sent-101, score-0.184]
28 E: Comparison of estimation error (area between curves) of SPE and naive method for 20 known labels and different datasets. [sent-102, score-0.199]
29 T anhed component d thenes miti exstu p0 waneidg p1 could be modeled parametrically by Normal distributions, Gamma distributions, or some other probability distributions appropriate for the given classifier (see Section 4. [sent-108, score-0.325]
30 This approach of applying a generative model to score distributions, when all labels are known, has been used in the past to obtain error estimates on classifier performance [ 12, 9, 1 1], and for classifier calibration [ 1]. [sent-110, score-0.566]
31 However, previous approaches require that the all items used to estimate the performance have been labeled. [sent-111, score-0.262]
32 We suggest that it may be possible to estimate classifier performance even when only a fraction of the ground truth labels are known. [sent-112, score-0.386]
33 In this case, the labels for the unlabeled items i ∈ Ut can be marginalized out, p(S, Yt | θ) = ? [sent-113, score-0.38]
34 This allows the model to make use of the scores of unlabeled items in addition to the labeled items, which enables accurate performance estimates with only a handful of labels. [sent-118, score-0.51]
35 Figure 2a shows a histogram of the scores obtained from classifier on a public dataset (see Section 4 for more information about the datasets we use). [sent-122, score-0.37]
36 At first glance, it is difficult to guess the performance of the classifier unless the oracle provides a lot of labels. [sent-123, score-0.337]
37 A–F: Standard parametric distributions py (s | θy) (black solid curve) fitted to the class conditional scores for a few example datasets and classifiers. [sent-128, score-0.47]
38 In all cases, we normalized the scores to be in the interval si ∈ (0, 1], and made the truncation at s = 0 for the truncated distributions. [sent-130, score-0.26]
39 G: Comparison of st∈and (a0r,d1 parametric d thisetri trbuunticoantiso n be astt representing empirical class-conditional score distributions (for a subset of the 78 cases we tried). [sent-133, score-0.254]
40 The distribution families we tried included (with abbreviations used in last three columns in parentheses) the truncated Normal (n), truncated Student’s t (t), Gamma (g), log-normal (ln), left- and right-skewed Gumbel (g-l and g-r), Gompertz (gz), and Frechet right (f-r) distribution. [sent-137, score-0.257]
41 For example, two often used performance measures are the precision P(τ; θ) and recall R(τ; θ) at a particular score threshold τ. [sent-145, score-0.24]
42 We can define these quantities in terms of the conditional distributions py (si | θy). [sent-146, score-0.23]
43 yi = 1, examples that have scores above a given threshold, R(τ;θ) =? [sent-149, score-0.215]
44 The posterior on θ may be used to obtain confidence bounds on the performance of the classifier. [sent-158, score-0.226]
45 For example, for some choice of parameters θ, the precision and recall can be computed for a range of score thresholds τ to obtain a curve (see solid curves in Figure 2b). [sent-159, score-0.398]
46 Similarly, given the posterior on θ, the distribution of P(τ; θ) and R(τ; θ) can be computed for a fixed τ to obtain confidence intervals (shown as colored bands in Figure 2b). [sent-160, score-0.274]
47 The same applies to the precision-recall curve: for some recall r, the distribution of precisions, found using Pr (r; θ) can be used to compute confidence intervals on the curve (see Figure 2c). [sent-161, score-0.345]
48 Since the previous approach views the scores (and the associated labels) as a finite sample from p(S, Y | θ), there will always be uncertainty inp teh fer performance |es θt)im, tahteer. [sent-164, score-0.255]
49 eW whileln aalwl aityesm bse eh uavnebeen labeled by the oracle, the remaining uncertainty in the performance represents the variability in sampling (S, Y ) from p(S, Y | θ). [sent-165, score-0.217]
50 Thus, when the oracle has labeled 333222666533 the whole test set, there should not be any uncertainty in the performance; it can simply be computed directly from (S, Y ). [sent-169, score-0.218]
51 To estimate the sample performance, we need to account for uncertainty in the unlabeled items, i ∈ Ut. [sent-170, score-0.219]
52 On the second line of (6) we rely on the assumption ofa mixture model to factor the joint probability distribution on θ and Yt? [sent-179, score-0.18]
53 and t|h Se scores S to trace out a performance curve (e. [sent-185, score-0.281]
54 The main difference between the sample and population performance estimates will be at the tails of the score distribution, p(S | θ), wtimheartee sin wdiilvli bdeua atl titheem ta lialsbe olfs can choarvee a large impact on tθh)e, performance curve. [sent-193, score-0.233]
55 A more generally applicable method, which we will describe here, is to split the sampling into three steps: (a) sample θ¯ from p(θ | S, Yt), (b) fix the mixture parameters to θ¯ and sample tph(eθ l a| Sbe,lsY Yt? [sent-201, score-0.234]
56 The first step, sampling from the posterior p(θ | S, Yt), can h bee cirasrtri setdep ,o suta using importance sampling (I |S S). [sent-205, score-0.239]
57 In IS, we sample from a proposal distribution q(θ) in order to estimate properties of the desired distribution p(θ | S, Yt). [sent-207, score-0.235]
58 We now have all steps needed to estimate the performance of the classifier, given the scores S and some labels Yt obtained from the oracle: 1. [sent-231, score-0.324]
59 Estimate performance measures using the scores S, labels Y¯t,m = Yt ∪ Y¯t? [sent-249, score-0.28]
60 Datasets We surveyed the literature for published classifier scores with ground truth labels. [sent-254, score-0.314]
61 The blue curve and confidence band show SPE applied to the ChnFtrs detector on the CPD dataset with 100 observed labels (black curve is ground truth). [sent-266, score-0.558]
62 Based on a curve like this, a practitioner can “recalibrate” a pre-trained classifier by picking a threshold for new dataset such that some pre-defined criteria (e. [sent-268, score-0.327]
63 detector scores and ground truth labels are available for a wide variety of detectors [7]. [sent-271, score-0.34]
64 Moreover, the CPD website also has scores and labels available, using the same detectors, for other pedestrian detection datasets, such as the IN- RIA (abbr. [sent-272, score-0.311]
65 We made use of the detections in the CPD and INR datasets as if they were classifier outputs. [sent-274, score-0.218]
66 To complement the pedestrian datasets, we also used a basic linear SVM classifier and a logistic regression classifier on the “optdigits” (abbr. [sent-280, score-0.385]
67 In the figures, the naming convention is as follows: “svm3” is used to mean that the SVM classifier was used with category 3 in the dataset being assigned to the y = 1class, and “logres9” denotes that the logistic regression classifier was used with category 9 being the y = 1 class, and so on. [sent-285, score-0.342]
68 Choosing class conditionals Which distribution families should one use for the class conditional py (s | θy) distributions? [sent-289, score-0.239]
69 To explore this question, we took (thse |c θlassifier scores and split them into two groups, one for yi = 0 and one for yi = 1. [sent-290, score-0.304]
70 We used MLE to fit different families of probability distributions (see Figure 4 for a list of distributions) on 80% of the data (sampled randomly) in each group. [sent-291, score-0.212]
71 We then ranked the distributions by the log likelihood of the remaining 20% of the data (given the MLE-fitted parameters). [sent-292, score-0.185]
72 Figure 4G shows the top-3 distributions that explained the class-conditional scores with highest likelihood for a selection of the datasets and classifiers. [sent-294, score-0.371]
73 We found that the truncated Normal distribution was in the top-3 list for 48/78 dataset class-conditionals, and that the Gamma distribution was in the top-3 list 53/78 times; at least one of the two distributions were always in the top-3 list. [sent-295, score-0.358]
74 In some cases, like Figure 4C, a mixture model would have provided a better fit than the simple distributions we tried. [sent-297, score-0.233]
75 That said, we found that truncated Normal and Gamma distributions were good choices for most of the datasets. [sent-298, score-0.216]
76 As an example, for the truncated Normal distribution, we use a Normal and a Gamma distribution as priors on the mean and standard deviation respectively (since we use sampling for inference, we are not limited to conjugate priors). [sent-301, score-0.209]
77 One heuristic, which we found worked well in our experiments, is to try different combinations of distributions for p0 and p1, and then choose the combination achieving the highest maximum likelihood on the labeled and unlabeled data. [sent-304, score-0.347]
78 While SPE performs as well as the naive method in terms of estimation error, the score distribution is not well explained by the assumptions of the model, so there is a bias in the prediction. [sent-315, score-0.229]
79 Figure 3E compares the estimation error of SPE to the naive method for different datasets, when only 20 labels are known. [sent-318, score-0.199]
80 Classifier recalibration Applying SPE to a test dataset allows us to “recalibrate” the classifier to that dataset. [sent-323, score-0.234]
81 Unlike previous work on classifier calibration [1, 17], SPE does not require all items to be labeled. [sent-324, score-0.338]
82 Similarly, we can c |h So,oYse a threshold τ to use with the classifier h¯(xi; τ) based on some pre-determined criteria. [sent-328, score-0.184]
83 For example, the requirement might be that the classifier performs with recall R(τ) > ˆr and precision P(τ) > pˆ. [sent-329, score-0.273]
84 n Tish esant,i fsofrie eda by calculating teh per expectation ˆ p(C(τ) = 1) = E [C(τ)] over the unlabeled items Yt? [sent-332, score-0.264]
85 Related work Previous approaches for estimating classifier performance with few labels falls into two categories: stratified sampling and active estimation using importance sampling. [sent-337, score-0.531]
86 This work has since been generalized to other classifier performance metrics, such as precision and recall [8]. [sent-339, score-0.311]
87 proposed instead to use importance sampling to focus labeling effort on data items with high classifier uncertainty, and applied it to standard loss functions [19] and Fmeasures [18]. [sent-341, score-0.445]
88 Both of these approaches assume that the classifier threshold τ is fixed (see Section 2) and that a single scalar performance measure is desired. [sent-342, score-0.222]
89 Fitting mixture models to the class-conditional score distributions has been studied in previous work with the goal of obtaining smooth performance curves. [sent-344, score-0.332]
90 This allowed them to provide smooth performance estimates even when the classconditional distributions could not be explained by standard parametric distributions. [sent-350, score-0.266]
91 Similarly, previous work on classifier calibration has involved fitting mixture models to score distributions [1, 17]. [sent-351, score-0.487]
92 In contrast to previous work, which require all data items to be labeled, SPE also makes use of the unlabeled data. [sent-352, score-0.264]
93 This semisupervised approach allows SPE to estimate classifier performance with very few labels, or when the proportions of positive and negative examples are very unbalanced. [sent-353, score-0.313]
94 Discussion We explored the problem of estimating classifier performance from few labeled items. [sent-355, score-0.277]
95 A sampling scheme based on importance sampling enables efficient inference. [sent-360, score-0.181]
96 One disadvantage with using an approach like SPE is that there are no guarantees that the assumption of standard parametric distributions will hold for any dataset, something we hope to address in future work. [sent-361, score-0.193]
97 However, using four public datasets, and multiple classifiers, we showed that classifier score distributions are often well approximated by standard twocomponent mixture models in practice. [sent-363, score-0.452]
98 One possibility in this direction would be to employ importance weighted active sampling techniques [3, 6], so similar in spirit to [19, 18] but for performance curves. [sent-366, score-0.186]
99 That said, as shown by our experiments, SPE already works well for a broad range of classifiers and datasets, and can estimate classifier performance with as few as 10 labels (see Figure 2). [sent-369, score-0.402]
100 Using asymmetric distributions to improve classifier probabilities: A comparison of new and standard parametric methods. [sent-376, score-0.351]
wordName wordTfidf (topN-words)
[('spe', 0.606), ('yt', 0.319), ('items', 0.18), ('cpd', 0.168), ('classifier', 0.158), ('distributions', 0.139), ('dgt', 0.126), ('scores', 0.126), ('curve', 0.117), ('labels', 0.116), ('oracle', 0.113), ('chnftrs', 0.112), ('gamma', 0.108), ('mixture', 0.094), ('planner', 0.09), ('yi', 0.089), ('unlabeled', 0.084), ('naive', 0.083), ('truncated', 0.077), ('recalibrate', 0.076), ('sawade', 0.076), ('sampling', 0.074), ('semisupervised', 0.073), ('confidence', 0.071), ('normal', 0.071), ('pedestrian', 0.069), ('detector', 0.068), ('curves', 0.065), ('recall', 0.064), ('score', 0.061), ('py', 0.06), ('datasets', 0.06), ('bounds', 0.059), ('distribution', 0.058), ('uncertainty', 0.058), ('posterior', 0.058), ('roc', 0.057), ('si', 0.057), ('parametric', 0.054), ('wm', 0.053), ('bands', 0.052), ('precision', 0.051), ('urban', 0.05), ('erkanli', 0.05), ('hamiltonian', 0.05), ('hellmich', 0.05), ('landwehr', 0.05), ('recalibration', 0.05), ('vacation', 0.05), ('bars', 0.049), ('labeled', 0.047), ('classifiers', 0.046), ('likelihood', 0.046), ('families', 0.045), ('conditionals', 0.045), ('estimate', 0.044), ('band', 0.043), ('proposal', 0.042), ('bennett', 0.041), ('cikm', 0.041), ('active', 0.041), ('thresholds', 0.04), ('ut', 0.04), ('said', 0.039), ('dasgupta', 0.039), ('rnd', 0.039), ('dirichlet', 0.039), ('performance', 0.038), ('bayesian', 0.038), ('stratified', 0.037), ('caltech', 0.037), ('estimates', 0.035), ('fitting', 0.035), ('intervals', 0.035), ('sat', 0.034), ('repository', 0.034), ('estimating', 0.034), ('densities', 0.034), ('sample', 0.033), ('importance', 0.033), ('thse', 0.032), ('inr', 0.032), ('conditional', 0.031), ('uci', 0.031), ('worked', 0.031), ('truth', 0.03), ('item', 0.03), ('magenta', 0.029), ('peter', 0.029), ('guess', 0.028), ('lt', 0.028), ('probability', 0.028), ('green', 0.028), ('population', 0.028), ('multiclass', 0.027), ('assumptions', 0.027), ('editors', 0.026), ('dataset', 0.026), ('threshold', 0.026), ('carlo', 0.026)]
simIndex simValue paperId paperTitle
same-paper 1 1.0 15 cvpr-2013-A Lazy Man's Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration
Author: Peter Welinder, Max Welling, Pietro Perona
Abstract: How many labeled examples are needed to estimate a classifier’s performance on a new dataset? We study the case where data is plentiful, but labels are expensive. We show that by making a few reasonable assumptions on the structure of the data, it is possible to estimate performance curves, with confidence bounds, using a small number of ground truth labels. Our approach, which we call Semisupervised Performance Evaluation (SPE), is based on a generative model for the classifier’s confidence scores. In addition to estimating the performance of classifiers on new datasets, SPE can be used to recalibrate a classifier by reestimating the class-conditional confidence distributions.
2 0.1393895 285 cvpr-2013-Minimum Uncertainty Gap for Robust Visual Tracking
Author: Junseok Kwon, Kyoung Mu Lee
Abstract: We propose a novel tracking algorithm that robustly tracks the target by finding the state which minimizes uncertainty of the likelihood at current state. The uncertainty of the likelihood is estimated by obtaining the gap between the lower and upper bounds of the likelihood. By minimizing the gap between the two bounds, our method finds the confident and reliable state of the target. In the paper, the state that gives the Minimum Uncertainty Gap (MUG) between likelihood bounds is shown to be more reliable than the state which gives the maximum likelihood only, especially when there are severe illumination changes, occlusions, and pose variations. A rigorous derivation of the lower and upper bounds of the likelihood for the visual tracking problem is provided to address this issue. Additionally, an efficient inference algorithm using Interacting Markov Chain Monte Carlo is presented to find the best state that maximizes the average of the lower and upper bounds of the likelihood and minimizes the gap between two bounds simultaneously. Experimental results demonstrate that our method successfully tracks the target in realistic videos and outperforms conventional tracking methods.
3 0.12158998 34 cvpr-2013-Adaptive Active Learning for Image Classification
Author: Xin Li, Yuhong Guo
Abstract: Recently active learning has attracted a lot of attention in computer vision field, as it is time and cost consuming to prepare a good set of labeled images for vision data analysis. Most existing active learning approaches employed in computer vision adopt most uncertainty measures as instance selection criteria. Although most uncertainty query selection strategies are very effective in many circumstances, they fail to take information in the large amount of unlabeled instances into account and are prone to querying outliers. In this paper, we present a novel adaptive active learning approach that combines an information density measure and a most uncertainty measure together to select critical instances to label for image classifications. Our experiments on two essential tasks of computer vision, object recognition and scene recognition, demonstrate the efficacy of the proposed approach.
4 0.10238775 398 cvpr-2013-Single-Pedestrian Detection Aided by Multi-pedestrian Detection
Author: Wanli Ouyang, Xiaogang Wang
Abstract: In this paper, we address the challenging problem of detecting pedestrians who appear in groups and have interaction. A new approach is proposed for single-pedestrian detection aided by multi-pedestrian detection. A mixture model of multi-pedestrian detectors is designed to capture the unique visual cues which are formed by nearby multiple pedestrians but cannot be captured by single-pedestrian detectors. A probabilistic framework is proposed to model the relationship between the configurations estimated by single- and multi-pedestrian detectors, and to refine the single-pedestrian detection result with multi-pedestrian detection. It can integrate with any single-pedestrian detector without significantly increasing the computation load. 15 state-of-the-art single-pedestrian detection approaches are investigated on three widely used public datasets: Caltech, TUD-Brussels andETH. Experimental results show that our framework significantly improves all these approaches. The average improvement is 9% on the Caltech-Test dataset, 11% on the TUD-Brussels dataset and 17% on the ETH dataset in terms of average miss rate. The lowest average miss rate is reduced from 48% to 43% on the Caltech-Test dataset, from 55% to 50% on the TUD-Brussels dataset and from 51% to 41% on the ETH dataset.
5 0.097036391 324 cvpr-2013-Part-Based Visual Tracking with Online Latent Structural Learning
Author: Rui Yao, Qinfeng Shi, Chunhua Shen, Yanning Zhang, Anton van_den_Hengel
Abstract: Despite many advances made in the area, deformable targets and partial occlusions continue to represent key problems in visual tracking. Structured learning has shown good results when applied to tracking whole targets, but applying this approach to a part-based target model is complicated by the need to model the relationships between parts, and to avoid lengthy initialisation processes. We thus propose a method which models the unknown parts using latent variables. In doing so we extend the online algorithm pegasos to the structured prediction case (i.e., predicting the location of the bounding boxes) with latent part variables. To better estimate the parts, and to avoid over-fitting caused by the extra model complexity/capacity introduced by theparts, wepropose a two-stage trainingprocess, based on the primal rather than the dual form. We then show that the method outperforms the state-of-the-art (linear and non-linear kernel) trackers.
6 0.094670661 142 cvpr-2013-Efficient Detector Adaptation for Object Detection in a Video
7 0.093802206 36 cvpr-2013-Adding Unlabeled Samples to Categories by Learned Attributes
8 0.091178469 260 cvpr-2013-Learning and Calibrating Per-Location Classifiers for Visual Place Recognition
9 0.091126397 217 cvpr-2013-Improving an Object Detector and Extracting Regions Using Superpixels
10 0.084711179 318 cvpr-2013-Optimized Pedestrian Detection for Multiple and Occluded People
11 0.082232438 388 cvpr-2013-Semi-supervised Learning of Feature Hierarchies for Object Detection in a Video
12 0.081961632 274 cvpr-2013-Lost! Leveraging the Crowd for Probabilistic Visual Self-Localization
13 0.077019513 363 cvpr-2013-Robust Multi-resolution Pedestrian Detection in Traffic Scenes
14 0.074827768 430 cvpr-2013-The SVM-Minus Similarity Score for Video Face Recognition
15 0.074418753 386 cvpr-2013-Self-Paced Learning for Long-Term Tracking
16 0.073657423 373 cvpr-2013-SWIGS: A Swift Guided Sampling Method
17 0.07348372 248 cvpr-2013-Learning Collections of Part Models for Object Recognition
18 0.072058365 328 cvpr-2013-Pedestrian Detection with Unsupervised Multi-stage Feature Learning
19 0.071876846 187 cvpr-2013-Geometric Context from Videos
20 0.071718521 387 cvpr-2013-Semi-supervised Domain Adaptation with Instance Constraints
topicId topicWeight
[(0, 0.174), (1, -0.034), (2, -0.012), (3, -0.015), (4, 0.07), (5, 0.01), (6, 0.024), (7, 0.003), (8, 0.01), (9, 0.033), (10, -0.031), (11, -0.065), (12, 0.018), (13, -0.073), (14, -0.068), (15, -0.032), (16, -0.038), (17, -0.036), (18, 0.024), (19, -0.018), (20, -0.049), (21, -0.047), (22, -0.07), (23, 0.07), (24, 0.007), (25, 0.004), (26, -0.04), (27, 0.033), (28, 0.047), (29, 0.042), (30, -0.01), (31, 0.06), (32, -0.096), (33, 0.036), (34, -0.036), (35, -0.002), (36, -0.076), (37, -0.072), (38, -0.063), (39, -0.066), (40, 0.02), (41, -0.047), (42, 0.032), (43, 0.035), (44, -0.065), (45, 0.061), (46, 0.054), (47, -0.028), (48, 0.025), (49, 0.034)]
simIndex simValue paperId paperTitle
same-paper 1 0.95657057 15 cvpr-2013-A Lazy Man's Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration
Author: Peter Welinder, Max Welling, Pietro Perona
Abstract: How many labeled examples are needed to estimate a classifier’s performance on a new dataset? We study the case where data is plentiful, but labels are expensive. We show that by making a few reasonable assumptions on the structure of the data, it is possible to estimate performance curves, with confidence bounds, using a small number of ground truth labels. Our approach, which we call Semisupervised Performance Evaluation (SPE), is based on a generative model for the classifier’s confidence scores. In addition to estimating the performance of classifiers on new datasets, SPE can be used to recalibrate a classifier by reestimating the class-conditional confidence distributions.
2 0.75998342 142 cvpr-2013-Efficient Detector Adaptation for Object Detection in a Video
Author: Pramod Sharma, Ram Nevatia
Abstract: In this work, we present a novel and efficient detector adaptation method which improves the performance of an offline trained classifier (baseline classifier) by adapting it to new test datasets. We address two critical aspects of adaptation methods: generalizability and computational efficiency. We propose an adaptation method, which can be applied to various baseline classifiers and is computationally efficient also. For a given test video, we collect online samples in an unsupervised manner and train a randomfern adaptive classifier . The adaptive classifier improves precision of the baseline classifier by validating the obtained detection responses from baseline classifier as correct detections or false alarms. Experiments demonstrate generalizability, computational efficiency and effectiveness of our method, as we compare our method with state of the art approaches for the problem of human detection and show good performance with high computational efficiency on two different baseline classifiers.
3 0.73026544 168 cvpr-2013-Fast Object Detection with Entropy-Driven Evaluation
Author: Raphael Sznitman, Carlos Becker, François Fleuret, Pascal Fua
Abstract: Cascade-style approaches to implementing ensemble classifiers can deliver significant speed-ups at test time. While highly effective, they remain challenging to tune and their overall performance depends on the availability of large validation sets to estimate rejection thresholds. These characteristics are often prohibitive and thus limit their applicability. We introduce an alternative approach to speeding-up classifier evaluation which overcomes these limitations. It involves maintaining a probability estimate of the class label at each intermediary response and stopping when the corresponding uncertainty becomes small enough. As a result, the evaluation terminates early based on the sequence of responses observed. Furthermore, it does so independently of the type of ensemble classifier used or the way it was trained. We show through extensive experimentation that our method provides 2 to 10 fold speed-ups, over existing state-of-the-art methods, at almost no loss in accuracy on a number of object classification tasks.
4 0.70421261 34 cvpr-2013-Adaptive Active Learning for Image Classification
Author: Xin Li, Yuhong Guo
Abstract: Recently active learning has attracted a lot of attention in computer vision field, as it is time and cost consuming to prepare a good set of labeled images for vision data analysis. Most existing active learning approaches employed in computer vision adopt most uncertainty measures as instance selection criteria. Although most uncertainty query selection strategies are very effective in many circumstances, they fail to take information in the large amount of unlabeled instances into account and are prone to querying outliers. In this paper, we present a novel adaptive active learning approach that combines an information density measure and a most uncertainty measure together to select critical instances to label for image classifications. Our experiments on two essential tasks of computer vision, object recognition and scene recognition, demonstrate the efficacy of the proposed approach.
5 0.66437989 320 cvpr-2013-Optimizing 1-Nearest Prototype Classifiers
Author: Paul Wohlhart, Martin Köstinger, Michael Donoser, Peter M. Roth, Horst Bischof
Abstract: The development of complex, powerful classifiers and their constant improvement have contributed much to the progress in many fields of computer vision. However, the trend towards large scale datasets revived the interest in simpler classifiers to reduce runtime. Simple nearest neighbor classifiers have several beneficial properties, such as low complexity and inherent multi-class handling, however, they have a runtime linear in the size of the database. Recent related work represents data samples by assigning them to a set of prototypes that partition the input feature space and afterwards applies linear classifiers on top of this representation to approximate decision boundaries locally linear. In this paper, we go a step beyond these approaches and purely focus on 1-nearest prototype classification, where we propose a novel algorithm for deriving optimal prototypes in a discriminative manner from the training samples. Our method is implicitly multi-class capable, parameter free, avoids noise overfitting and, since during testing only comparisons to the derived prototypes are required, highly efficient. Experiments demonstrate that we are able to outperform related locally linear methods, while even getting close to the results of more complex classifiers.
6 0.64670014 143 cvpr-2013-Efficient Large-Scale Structured Learning
7 0.62190694 383 cvpr-2013-Seeking the Strongest Rigid Detector
8 0.6152218 261 cvpr-2013-Learning by Associating Ambiguously Labeled Images
9 0.61353683 239 cvpr-2013-Kernel Null Space Methods for Novelty Detection
10 0.6094873 274 cvpr-2013-Lost! Leveraging the Crowd for Probabilistic Visual Self-Localization
11 0.60879338 385 cvpr-2013-Selective Transfer Machine for Personalized Facial Action Unit Detection
12 0.60854316 318 cvpr-2013-Optimized Pedestrian Detection for Multiple and Occluded People
13 0.59342301 134 cvpr-2013-Discriminative Sub-categorization
14 0.592094 353 cvpr-2013-Relative Hidden Markov Models for Evaluating Motion Skill
15 0.58555686 260 cvpr-2013-Learning and Calibrating Per-Location Classifiers for Visual Place Recognition
16 0.56987214 167 cvpr-2013-Fast Multiple-Part Based Object Detection Using KD-Ferns
17 0.56487072 390 cvpr-2013-Semi-supervised Node Splitting for Random Forest Construction
18 0.56391239 377 cvpr-2013-Sample-Specific Late Fusion for Visual Category Recognition
19 0.55958349 217 cvpr-2013-Improving an Object Detector and Extracting Regions Using Superpixels
20 0.55939358 264 cvpr-2013-Learning to Detect Partially Overlapping Instances
topicId topicWeight
[(10, 0.108), (16, 0.021), (26, 0.065), (28, 0.015), (33, 0.245), (67, 0.077), (69, 0.054), (76, 0.013), (84, 0.199), (87, 0.107)]
simIndex simValue paperId paperTitle
1 0.86998248 51 cvpr-2013-Auxiliary Cuts for General Classes of Higher Order Functionals
Author: Ismail Ben Ayed, Lena Gorelick, Yuri Boykov
Abstract: Several recent studies demonstrated that higher order (non-linear) functionals can yield outstanding performances in the contexts of segmentation, co-segmentation and tracking. In general, higher order functionals result in difficult problems that are not amenable to standard optimizers, and most of the existing works investigated particular forms of such functionals. In this study, we derive general bounds for a broad class of higher order functionals. By introducing auxiliary variables and invoking the Jensen ’s inequality as well as some convexity arguments, we prove that these bounds are auxiliary functionals for various non-linear terms, which include but are not limited to several affinity measures on the distributions or moments of segment appearance and shape, as well as soft constraints on segment volume. From these general-form bounds, we state various non-linear problems as the optimization of auxiliary functionals by graph cuts. The proposed bound optimizers are derivative-free, and consistently yield very steep functional decreases, thereby converging within a few graph cuts. We report several experiments on color and medical data, along with quantitative comparisons to stateof-the-art methods. The results demonstrate competitive performances of the proposed algorithms in regard to accuracy and convergence speed, and confirm their potential in various vision and medical applications.
same-paper 2 0.85956711 15 cvpr-2013-A Lazy Man's Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration
Author: Peter Welinder, Max Welling, Pietro Perona
Abstract: How many labeled examples are needed to estimate a classifier’s performance on a new dataset? We study the case where data is plentiful, but labels are expensive. We show that by making a few reasonable assumptions on the structure of the data, it is possible to estimate performance curves, with confidence bounds, using a small number of ground truth labels. Our approach, which we call Semisupervised Performance Evaluation (SPE), is based on a generative model for the classifier’s confidence scores. In addition to estimating the performance of classifiers on new datasets, SPE can be used to recalibrate a classifier by reestimating the class-conditional confidence distributions.
3 0.82351178 248 cvpr-2013-Learning Collections of Part Models for Object Recognition
Author: Ian Endres, Kevin J. Shih, Johnston Jiaa, Derek Hoiem
Abstract: We propose a method to learn a diverse collection of discriminative parts from object bounding box annotations. Part detectors can be trained and applied individually, which simplifies learning and extension to new features or categories. We apply the parts to object category detection, pooling part detections within bottom-up proposed regions and using a boosted classifier with proposed sigmoid weak learners for scoring. On PASCAL VOC 2010, we evaluate the part detectors ’ ability to discriminate and localize annotated keypoints. Our detection system is competitive with the best-existing systems, outperforming other HOG-based detectors on the more deformable categories.
4 0.82327372 365 cvpr-2013-Robust Real-Time Tracking of Multiple Objects by Volumetric Mass Densities
Author: Horst Possegger, Sabine Sternig, Thomas Mauthner, Peter M. Roth, Horst Bischof
Abstract: Combining foreground images from multiple views by projecting them onto a common ground-plane has been recently applied within many multi-object tracking approaches. These planar projections introduce severe artifacts and constrain most approaches to objects moving on a common 2D ground-plane. To overcome these limitations, we introduce the concept of an occupancy volume exploiting the full geometry and the objects ’ center of mass and develop an efficient algorithm for 3D object tracking. Individual objects are tracked using the local mass density scores within a particle filter based approach, constrained by a Voronoi partitioning between nearby trackers. Our method benefits from the geometric knowledge given by the occupancy volume to robustly extract features and train classifiers on-demand, when volumetric information becomes unreliable. We evaluate our approach on several challenging real-world scenarios including the public APIDIS dataset. Experimental evaluations demonstrate significant improvements compared to state-of-theart methods, while achieving real-time performance. – –
5 0.81767327 247 cvpr-2013-Learning Class-to-Image Distance with Object Matchings
Author: Guang-Tong Zhou, Tian Lan, Weilong Yang, Greg Mori
Abstract: We conduct image classification by learning a class-toimage distance function that matches objects. The set of objects in training images for an image class are treated as a collage. When presented with a test image, the best matching between this collage of training image objects and those in the test image is found. We validate the efficacy of the proposed model on the PASCAL 07 and SUN 09 datasets, showing that our model is effective for object classification and scene classification tasks. State-of-the-art image classification results are obtained, and qualitative results demonstrate that objects can be accurately matched.
6 0.81641358 71 cvpr-2013-Boundary Cues for 3D Object Shape Recovery
8 0.81389046 242 cvpr-2013-Label Propagation from ImageNet to 3D Point Clouds
9 0.81192112 331 cvpr-2013-Physically Plausible 3D Scene Tracking: The Single Actor Hypothesis
10 0.81182069 225 cvpr-2013-Integrating Grammar and Segmentation for Human Pose Estimation
11 0.8117491 98 cvpr-2013-Cross-View Action Recognition via a Continuous Virtual Path
12 0.81162268 61 cvpr-2013-Beyond Point Clouds: Scene Understanding by Reasoning Geometry and Physics
13 0.8115387 446 cvpr-2013-Understanding Indoor Scenes Using 3D Geometric Phrases
14 0.81087446 222 cvpr-2013-Incorporating User Interaction and Topological Constraints within Contour Completion via Discrete Calculus
15 0.810628 311 cvpr-2013-Occlusion Patterns for Object Class Detection
16 0.81042737 4 cvpr-2013-3D Visual Proxemics: Recognizing Human Interactions in 3D from a Single Image
17 0.81034869 414 cvpr-2013-Structure Preserving Object Tracking
18 0.81034416 119 cvpr-2013-Detecting and Aligning Faces by Image Retrieval
19 0.81026047 298 cvpr-2013-Multi-scale Curve Detection on Surfaces
20 0.80948299 72 cvpr-2013-Boundary Detection Benchmarking: Beyond F-Measures