nips nips2012 nips2012-256 knowledge-graph by maker-knowledge-mining

256 nips-2012-On the connections between saliency and tracking

Source: pdf

Author: Vijay Mahadevan, Nuno Vasconcelos

Abstract: A model connecting visual tracking and saliency has recently been proposed. This model is based on the saliency hypothesis for tracking which postulates that tracking is achieved by the top-down tuning, based on target features, of discriminant center-surround saliency mechanisms over time. In this work, we identify three main predictions that must hold if the hypothesis were true: 1) tracking reliability should be larger for salient than for non-salient targets, 2) tracking reliability should have a dependence on the deﬁning variables of saliency, namely feature contrast and distractor heterogeneity, and must replicate the dependence of saliency on these variables, and 3) saliency and tracking can be implemented with common low level neural mechanisms. We conﬁrm that the ﬁrst two predictions hold by reporting results from a set of human behavior studies on the connection between saliency and tracking. We also show that the third prediction holds by constructing a common neurophysiologically plausible architecture that can computationally solve both saliency and tracking. This architecture is fully compliant with the standard physiological models of V1 and MT, and with what is known about attentional control in area LIP, while explaining the results of the human behavior experiments.

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 On the connections between saliency and tracking Nuno Vasconcelos Statistical Visual Computing Laboratory UC San Diego, La Jolla, CA 92092 nuno@ece. [sent-1, score-1.118]

2 com Abstract A model connecting visual tracking and saliency has recently been proposed. [sent-5, score-1.181]

3 This model is based on the saliency hypothesis for tracking which postulates that tracking is achieved by the top-down tuning, based on target features, of discriminant center-surround saliency mechanisms over time. [sent-6, score-2.582]

4 We conﬁrm that the ﬁrst two predictions hold by reporting results from a set of human behavior studies on the connection between saliency and tracking. [sent-8, score-0.74]

5 We also show that the third prediction holds by constructing a common neurophysiologically plausible architecture that can computationally solve both saliency and tracking. [sent-9, score-0.772]

6 1 Introduction Biological vision systems have evolved sophisticated tracking mechanisms, capable of tracking complex objects, undergoing complex motion, in challenging environments. [sent-11, score-0.877]

7 Visual tracking has also been widely studied in computer vision, where numerous tracking algorithms [38] have been proposed. [sent-20, score-0.85]

8 1 The best results among tracking algorithms have recently been demonstrated for a class of methods that pose object tracking as incremental target/background classiﬁcation [22, 8, 2, 13]. [sent-26, score-0.88]

9 We refer to this as the saliency hypothesis for tracking. [sent-32, score-0.741]

10 Working under this hypothesis, [22] proposed a tracker based on the discriminant saliency principle of [12]. [sent-33, score-0.794]

11 This is a principle for bottom-up center-surround saliency, which poses saliency as discrimination between a target (center) and a null (surround) hypothesis. [sent-34, score-0.902]

12 Center-surround discriminant saliency has previously been shown to predict various psychophysical traits of human saliency and visual search performance [11]. [sent-35, score-1.545]

13 The extension proposed by [22], to the tracking problem, endows discriminant saliency with a top-down feature selection mechanism. [sent-36, score-1.252]

14 This mechanism enhances features that respond strongly to the target and weakly to the background, transforming the saliency operation from a search for locations where center is distinct from the surround, to a search for locations where target is present in the center but not in the surround. [sent-37, score-1.213]

15 We conﬁrm that the ﬁrst two of these predictions hold by performing several human behavior experiments on the dependence between target saliency and human tracking performance. [sent-40, score-1.404]

16 These experiments build on well understood properties of saliency, such as pop-out effects, to show that tracking requires discrimination between target and background using a center-surround mechanism. [sent-41, score-0.67]

17 In addition, we characterize the dependence of tracking performance on the extent of discrimination, by gradually varying feature contrast between target and distractors in the tracking tasks. [sent-42, score-1.341]

18 The results show that both tracking performance and saliency have highly similar patterns of dependency on feature contrast and distractor heterogeneity. [sent-43, score-1.473]

19 This network extends the substantial connections between discriminant saliency and the standard model that have already been shown [12] and is a biologically plausible optimal model for both saliency and tracking. [sent-45, score-1.538]

20 To the best of our knowledge, this is the ﬁrst report on psychophysics experiments studying the relation between attentional tracking of a single target and its saliency. [sent-47, score-0.714]

21 1 Experiment 1 : Saliency affects tracking performance The experimental setting was inspired by the tracking paradigm of Pylyshyn [29]. [sent-54, score-0.85]

22 Subjects viewed displays containing a green target disk surrounded by 70 red distractor disks and a static ﬁxation square. [sent-55, score-0.612]

23 Under the saliency hypothesis for tracking, the rate of successful target tracking should be much higher for salient than for non-salient displays. [sent-71, score-1.464]

24 However, this could be due to the fact that the target was the only green disk in salient displays, and since it continuously popped-out subjects could be acquiring the target at any time even after losing track. [sent-72, score-0.596]

25 In the latter, the tracking performance was almost at the chance level of 1 , suggesting complete tracking 3 failure. [sent-81, score-0.85]

26 In fact, the similarity of detection rates in the two experiments suggests that target pop-out does not aid human tracking performance at all. [sent-82, score-0.639]

27 This is consistent with the saliency hypothesis, since bottom-up saliency mechanisms are well known to have a center-surround structure [16, 12]. [sent-86, score-1.43]

28 The second, which follows from the fact that only target color varied between the two conditions, is that tracking performance depends on the discriminability of the target. [sent-90, score-0.611]

29 While the ﬁrst experiment used color as a discriminant cue, the conclusion that saliency affect tracking performance applies even when other features are salient. [sent-92, score-1.244]

30 For example, studies on multiple object tracking with identical targets and distractors have reported tracking failure when target and distractors are too close to each other [14]. [sent-93, score-1.514]

31 This is consistent with the discriminant hypothesis: when target and distractors are identical, the target must be spatio-temporally salient (due to its trajectory or position) in an immediate neighborhood to be tracked accurately. [sent-94, score-0.777]

32 2 Experiment 2: Tracking reliability as a function of feature contrast Experiment 2 aimed to investigate the connection between the two phenomena in greater detail, namely to quantify how tracking reliability depends on target saliency. [sent-96, score-0.799]

33 Since saliency is not an independent variable, this quantiﬁcation can only be done indirectly. [sent-97, score-0.693]

34 In fact, Nothdurft [26] has precisely quantiﬁed the dependence of saliency on orientation contrast in static displays. [sent-100, score-0.877]

35 His work has shown that perceived target saliency increases with the orientation contrast between target and neighboring distractors. [sent-101, score-1.224]

36 This increase is quite non-linear, exhibiting the threshold and saturation effects shown in Figure 1 (a), where we present curves of saliency as a function of orientation contrast between target and distractors for three levels of distractor homogeneity [26]. [sent-102, score-1.567]

37 The relationship between tracking reliability and target saliency can thus be characterized by manipulating orientation contrast and measuring the impact on tracking performance. [sent-103, score-1.945]

38 If the saliency 3 hypothesis for tracking holds, saliency and tracking reliability should be equivalent functions of orientation contrast. [sent-104, score-2.473]

39 In particular, increasing orientation contrast between target and distractors should result in a non-linear increase of tracking reliability, with threshold and saturation effects similar to those observed by Nothdurft. [sent-105, score-1.018]

40 The 7 conditions corresponded to different levels of orientation contrast between target and distractor ellipses. [sent-118, score-0.626]

41 To study the effect of distractor heterogeneity [26], three versions of the experiment were conducted with different numbers of ellipses in the target orientation. [sent-128, score-0.729]

42 Finally, in the third version, 13 ellipses were in distractor and 10 in target orientation, for the largest degree of distractor heterogeneity. [sent-133, score-0.832]

43 These curves are remarkably similar to Nothdurft’s saliency curves, shown in (a). [sent-136, score-0.693]

44 Again, there are 1) distinct threshold and saturation effects for tracking, with tracking accuracy saturating for orientation contrasts beyond 40◦ , and 2) decreasing tracking accuracy as distractor heterogeneity increases. [sent-137, score-1.461]

45 The co-variation of tracking accuracy and saliency is illustrated in Figure 1 (c), where the two quantities are presented as a scatter plot The correlation between the two variable is near perfect (r = 0. [sent-138, score-1.139]

46 In summary, tracking has a dependence on orientation contrast remarkably similar to that of saliency. [sent-140, score-0.609]

47 8 0 similar distractors 4 similar distractors 9 similar distractors 0. [sent-143, score-0.618]

48 7 0 similar distractors 4 similar distractors 9 similar distractors 0. [sent-151, score-0.618]

49 5 0 20 40 60 80 Target Orientation Contrast (deg) (d) Figure 1: (a) saliency vs. [sent-153, score-0.693]

50 orientation contrast (adapted from [26]) (b) human tracking success rate vs. [sent-154, score-0.612]

51 (c) scatter plot of saliency values from (a) vs tracking accuracy from (b), r = 0. [sent-156, score-1.139]

52 3 Experiment 3: The spatial structure of tracking It is well known that bottom-up saliency mechanisms are based on spatially localized centersurround processing [16, 6]. [sent-161, score-1.233]

53 Hence, the saliency hypothesis for tracking predicts that tracking performance depends only on distractors within a spatial neighborhood of the target. [sent-162, score-1.846]

54 The results 4 of Experiment 2 provide some evidence in support of this prediction, by showing that tracking performance depends on distractor heterogeneity. [sent-163, score-0.706]

55 In this experiment, the distance dcsd between the target and the closest distractor of the same orientation, denoted the closest similar distractor (CSD), was controlled so that dcsd = R, where R is a parameter. [sent-167, score-0.804]

56 This implies that the rate of tracking success does not depend on distractor heterogeneity for R > Rcritical . [sent-170, score-0.82]

57 The tracking accuracy for the case where there is no distractor heterogeneity (no distractor with the target orientation) is also shown, as a ﬂat line. [sent-183, score-1.308]

58 First, for a ﬁxed (non-zero) amount of distractor heterogeneity, tracking performance always increases with R. [sent-185, score-0.706]

59 Second, for large R tracking accuracy does not depend on distractor heterogeneity (it is nearly the same under the two heterogeneity conditions), converging to the accuracy observed when there is no distractor heterogeneity (Experiment 3). [sent-187, score-1.371]

60 When similar distractors are kept out of this region, the degree of distractor heterogeneity has no effect in tracking performance. [sent-191, score-1.026]

61 In summary, results of the human behavior experiments show that the ﬁrst two predictions made by the saliency hypothesis for tracking hold. [sent-192, score-1.213]

62 These predictions are that tracking reliability 1) is larger for salient than for non-salient targets (Experiment 1), 2) depends on the deﬁning variables of saliency, namely feature contrast and distractor heterogeneity (Experiment 2), and replicates the dependence of saliency on these variables. [sent-193, score-1.857]

63 This includes the threshold and saturation effects of the dependence of saliency on feature contrast (Experiment 2), and the spatially localized dependence of saliency on distractor heterogeneity (Experiment 3). [sent-194, score-1.988]

64 Overall, these experiments provide strong evidence in support of the saliency hypothesis for tracking. [sent-195, score-0.741]

65 We next consider the ﬁnal prediction, which is that saliency and tracking can be implemented with common neural mechanisms. [sent-196, score-1.118]

66 In [12], saliency is equated to optimal decision-making between two classes of visual stimuli, with label C ∈ {0, 1}, C = 1 for stimuli in a target class, and C = 0 for stimuli in a background class. [sent-198, score-1.052]

67 The saliency of location l is then equated to the expected accuracy of target/background classiﬁcation, 5 1 (a) 1 (b) tracking accuracy tracking accuracy 0. [sent-204, score-1.642]

68 75 0 similar distractors 4 similar distractors 9 similar distractors 0. [sent-209, score-0.618]

69 8 0 similar distractors 4 similar distractors 9 similar distractors 0. [sent-212, score-0.618]

70 6 2 3 4 5 average distance to nearest similar distractor (deg) 2 3 4 5 average distance to nearest similar distractor (deg) (c) (d) Figure 2: (a) and (b) Experiment 1: successful target tracking rate for targets that are (a) globally salient (pop-out), and (b) locally salient (do not pop-out). [sent-214, score-1.433]

71 (c) and (d) Experiment 3: the effect of background on tracking performance - (c) Tracking accuracy of human subjects for two versions of distractor homogeneities are plotted as a function of the average target-similar distractor distance. [sent-215, score-1.144]

72 Also shown, using a blue dashed line, is the tracking accuracy for the version with no similar distractors at target orientation of 40◦ from Experiment 2. [sent-216, score-0.97]

73 (d) model prediction for the same data using the saliency based model of Figure 3. [sent-217, score-0.693]

74 5 (1) The saliency measure Sk (l) is the expected conﬁdence with which the feature response Yk (l) is assigned to the target class. [sent-220, score-0.926]

75 This tunes the saliency measure to respond only to the presence of target stimuli, not to its absence. [sent-223, score-0.879]

76 This deﬁnition of saliency was shown, in [12], to be computable using units that conform to the standard neurophysiological model of cells in visual cortex area V1 [5], when the features are bandpass ﬁlters (e. [sent-224, score-0.823]

77 However, for the tracking task, the feature set Y for representing the target and background needs to contain spatiotemporal features that are tuned to the velocity of moving patterns. [sent-227, score-0.713]

78 It can be shown that saliency for such velocity tuned spatiotemporal features can be computed by combining the outputs of a set of V1 like units of [12], akin to the widely used approach for constructing models for MT cells from afferent V1 units [36, 33]. [sent-228, score-0.712]

79 1 Neurophysiologically plausible feature selection A key component of the saliency tracker of [22] is a feature selection procedure that continuously adapts the saliency measure of (1) to the target. [sent-231, score-1.593]

80 This changes the saliency from a bottom-up identiﬁcation of locations where center and surround differ, to a topdown identiﬁcation of locations containing the target in the center and background in the surround. [sent-233, score-1.075]

81 To derive a biologically plausible feature selection mechanism, we replace the saliency measure of (1) with a feature-weighted extension αk = 1 αk Sk (l), S(l) = (2) k k where αk is the weight given to the saliency of the k th feature channel. [sent-235, score-1.565]

82 (8) Hence, the posterior probability of feature k being the most salient at time t given that the target is ∗ ∗ at lt is computed by divisively normalizing a weighted version of Sk (lt ), the bottom-up saliency of ∗ the feature k at lt , by the total saliency summed over all features. [sent-244, score-1.886]

83 The weight applied to the saliency of each feature (corresponding to αk in (2)) is the posterior probability of the feature being the most salient at time t − τ . [sent-245, score-0.899]

84 2 Neurophysiologically plausible discriminant tracker A neurophysiologically plausible version of the discriminant tracker of [22] can be constructed with the discriminant saliency measure of (1), and the feature selection mechanism of (8). [sent-255, score-1.175]

85 In this case, there is no explicit top-down guidance about the object to recognize, and the saliency of location l is measured by the saliency of all unmodulated feature responses. [sent-257, score-1.499]

86 This consists of using the bottom-up saliency measure of (2) 0 0 with αk = PFk (1), where PFk (1) is a uniform prior for feature selection, at time t = 0. [sent-258, score-0.74]

87 The outputs of all features or neurons are then summed with equal weights to produce a ﬁnal saliency map. [sent-259, score-0.712]

88 Once the initial target location is attended, the feature selection mechanism modulates the saliency response of the individual feature channels, using the weights of (8). [sent-262, score-1.053]

89 The ﬁnal saliency value at that location also becomes the normalizing constant for the divisive normalization of (8). [sent-263, score-0.729]

90 The discriminant saliency network of [12] is used to construct a model for an MT neuron. [sent-268, score-0.779]

91 Feature selection, performed possibly in area LIP with weights being fed-back to MT, is achieved by the modulation of the response of each feature channel by its saliency value after divisive normalization across features. [sent-269, score-0.768]

92 After the latency due to feedback, say at time t + τ , the new feature weights and spatial weights, modulate the feature maps, which are again fed forward to LIP, where the updated saliency map is computed by simple summation. [sent-270, score-0.817]

93 The top-down saliency of location l at time t + τ is then given by S td (l) = td Sj (l) = j Sj (l)PF t |C(l∗ ) (1|1). [sent-271, score-0.729]

94 t j (9) j where Sj (l) is the modulated saliency response of the j th feature. [sent-272, score-0.693]

95 ∗ Spatial attention suppresses all but a neighborhood of the last known target location lt , and the feature-based attention suppresses all features except those present in the target and discriminative with respect to the background. [sent-273, score-0.614]

96 Therefore, the peak of the new saliency map corresponds to the position that best resembles the target at time t + τ , and attention is shifted to that position. [sent-274, score-0.936]

97 The remaining operations, possibly in LIP, compute the probabilities of (8) and the top-down saliency map of (9). [sent-279, score-0.693]

98 The model replicates the trend observed in both versions of the experiment, accurately tracking the target in the salient conditions, and losing track in the non-salient condition. [sent-282, score-0.789]

99 5 Conclusion We provide the ﬁrst veriﬁable evidence for a connection between saliency and tracking that was earlier only hypothesized [22]. [sent-286, score-1.118]

100 First, using psychophysics experiments we show that tracking requires discrimination between target and background using a center-surround mechanism, and that tracking reliability and saliency have a common dependence on feature contrast and distractor heterogeneity. [sent-288, score-2.278]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('saliency', 0.693), ('tracking', 0.425), ('distractor', 0.281), ('distractors', 0.206), ('target', 0.186), ('orientation', 0.132), ('heterogeneity', 0.114), ('salient', 0.112), ('lip', 0.084), ('ellipses', 0.084), ('surround', 0.076), ('pc', 0.069), ('discriminant', 0.068), ('visual', 0.063), ('reliability', 0.057), ('pfk', 0.056), ('lt', 0.054), ('mt', 0.053), ('psychophysics', 0.053), ('attentional', 0.05), ('hypothesis', 0.048), ('feature', 0.047), ('subjects', 0.047), ('disks', 0.047), ('disk', 0.046), ('mechanisms', 0.044), ('plausible', 0.042), ('experiment', 0.039), ('yk', 0.038), ('neurophysiologically', 0.037), ('attention', 0.037), ('stimuli', 0.037), ('location', 0.036), ('background', 0.036), ('targets', 0.036), ('deg', 0.034), ('displays', 0.033), ('tracker', 0.033), ('fk', 0.031), ('ellipse', 0.03), ('object', 0.03), ('spatial', 0.03), ('human', 0.028), ('dcsd', 0.028), ('pfj', 0.028), ('pylyshyn', 0.028), ('rcritical', 0.028), ('area', 0.028), ('sk', 0.027), ('contrast', 0.027), ('vision', 0.027), ('dependence', 0.025), ('mechanism', 0.025), ('responses', 0.025), ('versions', 0.025), ('motion', 0.025), ('biologically', 0.024), ('saturation', 0.024), ('discrimination', 0.023), ('localized', 0.023), ('replicates', 0.021), ('mahadevan', 0.021), ('locations', 0.021), ('sj', 0.021), ('center', 0.021), ('pami', 0.021), ('accuracy', 0.021), ('track', 0.02), ('peak', 0.02), ('bg', 0.02), ('enhances', 0.02), ('suppresses', 0.02), ('xation', 0.02), ('neurophysiological', 0.02), ('green', 0.019), ('features', 0.019), ('selection', 0.019), ('pixels', 0.019), ('trial', 0.019), ('parietal', 0.019), ('male', 0.019), ('pf', 0.019), ('predictions', 0.019), ('neighborhood', 0.019), ('attentive', 0.019), ('compliant', 0.019), ('covertly', 0.019), ('cued', 0.019), ('kazanovich', 0.019), ('keystroke', 0.019), ('makovski', 0.019), ('nothdurft', 0.019), ('spatially', 0.018), ('display', 0.018), ('network', 0.018), ('cognition', 0.018), ('supplement', 0.018), ('effects', 0.018), ('attached', 0.018), ('female', 0.018)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000012 256 nips-2012-On the connections between saliency and tracking

Author: Vijay Mahadevan, Nuno Vasconcelos

2 0.17035948 94 nips-2012-Delay Compensation with Dynamical Synapses

Author: Chi Fung, K. Wong, Si Wu

Abstract: Time delay is pervasive in neural information processing. To achieve real-time tracking, it is critical to compensate the transmission and processing delays in a neural system. In the present study we show that dynamical synapses with shortterm depression can enhance the mobility of a continuous attractor network to the extent that the system tracks time-varying stimuli in a timely manner. The state of the network can either track the instantaneous position of a moving stimulus perfectly (with zero-lag) or lead it with an effectively constant time, in agreement with experiments on the head-direction systems in rodents. The parameter regions for delayed, perfect and anticipative tracking correspond to network states that are static, ready-to-move and spontaneously moving, respectively, demonstrating the strong correlation between tracking performance and the intrinsic dynamics of the network. We also ﬁnd that when the speed of the stimulus coincides with the natural speed of the network state, the delay becomes effectively independent of the stimulus amplitude.

3 0.086981125 140 nips-2012-Fusion with Diffusion for Robust Visual Tracking

Author: Yu Zhou, Xiang Bai, Wenyu Liu, Longin J. Latecki

Abstract: A weighted graph is used as an underlying structure of many algorithms like semisupervised learning and spectral clustering. If the edge weights are determined by a single similarity measure, then it hard if not impossible to capture all relevant aspects of similarity when using a single similarity measure. In particular, in the case of visual object matching it is beneﬁcial to integrate different similarity measures that focus on different visual representations. In this paper, a novel approach to integrate multiple similarity measures is proposed. First pairs of similarity measures are combined with a diffusion process on their tensor product graph (TPG). Hence the diffused similarity of each pair of objects becomes a function of joint diffusion of the two original similarities, which in turn depends on the neighborhood structure of the TPG. We call this process Fusion with Diffusion (FD). However, a higher order graph like the TPG usually means signiﬁcant increase in time complexity. This is not the case in the proposed approach. A key feature of our approach is that the time complexity of the diffusion on the TPG is the same as the diffusion process on each of the original graphs. Moreover, it is not necessary to explicitly construct the TPG in our framework. Finally all diffused pairs of similarity measures are combined as a weighted sum. We demonstrate the advantages of the proposed approach on the task of visual tracking, where different aspects of the appearance similarity between the target object in frame t − 1 and target object candidates in frame t are integrated. The obtained method is tested on several challenge video sequences and the experimental results show that it outperforms state-of-the-art tracking methods. 1

4 0.083257295 114 nips-2012-Efficient coding provides a direct link between prior and likelihood in perceptual Bayesian inference

Author: Xue-xin Wei, Alan Stocker

Abstract: A common challenge for Bayesian models of perception is the fact that the two fundamental Bayesian components, the prior distribution and the likelihood function, are formally unconstrained. Here we argue that a neural system that emulates Bayesian inference is naturally constrained by the way it represents sensory information in populations of neurons. More speciﬁcally, we show that an efﬁcient coding principle creates a direct link between prior and likelihood based on the underlying stimulus distribution. The resulting Bayesian estimates can show biases away from the peaks of the prior distribution, a behavior seemingly at odds with the traditional view of Bayesian estimation, yet one that has been reported in human perception. We demonstrate that our framework correctly accounts for the repulsive biases previously reported for the perception of visual orientation, and show that the predicted tuning characteristics of the model neurons match the reported orientation tuning properties of neurons in primary visual cortex. Our results suggest that efﬁcient coding is a promising hypothesis in constraining Bayesian models of perceptual inference. 1 Motivation Human perception is not perfect. Biases have been observed in a large number of perceptual tasks and modalities, of which the most salient ones constitute many well-known perceptual illusions. It has been suggested, however, that these biases do not reﬂect a failure of perception but rather an observer’s attempt to optimally combine the inherently noisy and ambiguous sensory information with appropriate prior knowledge about the world [13, 4, 14]. This hypothesis, which we will refer to as the Bayesian hypothesis, has indeed proven quite successful in providing a normative explanation of perception at a qualitative and, more recently, quantitative level (see e.g. [15]). A major challenge in forming models based on the Bayesian hypothesis is the correct selection of two main components: the prior distribution (belief) and the likelihood function. This has encouraged some to criticize the Bayesian hypothesis altogether, claiming that arbitrary choices for these components always allow for unjustiﬁed post-hoc explanations of the data [1]. We do not share this criticism, referring to a number of successful attempts to constrain prior beliefs and likelihood functions based on principled grounds. For example, prior beliefs have been deﬁned as the relative distribution of the sensory variable in the environment in cases where these statistics are relatively easy to measure (e.g. local visual orientations [16]), or where it can be assumed that subjects have learned them over the course of the experiment (e.g. time perception [17]). Other studies have constrained the likelihood function according to known noise characteristics of neurons that are crucially involved in the speciﬁc perceptual process (e.g motion tuned neurons in visual cor∗ http://www.sas.upenn.edu/ astocker/lab 1 world neural representation efficient encoding percept Bayesian decoding Figure 1: Encoding-decoding framework. A stimulus representing a sensory variable θ elicits a ﬁring rate response R = {r1 , r2 , ..., rN } in a population of N neurons. The perceptual task is to generate a ˆ good estimate θ(R) of the presented value of the sensory variable based on this population response. Our framework assumes that encoding is efﬁcient, and decoding is Bayesian based on the likelihood p(R|θ), the prior p(θ), and a squared-error loss function. tex [18]). However, we agree that ﬁnding appropriate constraints is generally difﬁcult and that prior beliefs and likelihood functions have been often selected on the basis of mathematical convenience. Here, we propose that the efﬁcient coding hypothesis [19] offers a joint constraint on the prior and likelihood function in neural implementations of Bayesian inference. Efﬁcient coding provides a normative description of how neurons encode sensory information, and suggests a direct link between measured perceptual discriminability, neural tuning characteristics, and environmental statistics [11]. We show how this link can be extended to a full Bayesian account of perception that includes perceptual biases. We validate our model framework against behavioral as well as neural data characterizing the perception of visual orientation. We demonstrate that we can account not only for the reported perceptual biases away from the cardinal orientations, but also for the speciﬁc response characteristics of orientation-tuned neurons in primary visual cortex. Our work is a novel proposal of how two important normative hypotheses in perception science, namely efﬁcient (en)coding and Bayesian decoding, might be linked. 2 Encoding-decoding framework We consider perception as an inference process that takes place along the simpliﬁed neural encodingdecoding cascade illustrated in Fig. 11 . 2.1 Efﬁcient encoding Efﬁcient encoding proposes that the tuning characteristics of a neural population are adapted to the prior distribution p(θ) of the sensory variable such that the population optimally represents the sensory variable [19]. Different deﬁnitions of “optimally” are possible, and may lead to different results. Here, we assume an efﬁcient representation that maximizes the mutual information between the sensory variable and the population response. With this deﬁnition and an upper limit on the total ﬁring activity, the square-root of the Fisher Information must be proportional to the prior distribution [12, 21]. In order to constrain the tuning curves of individual neurons in the population we also impose a homogeneity constraint, requiring that there exists a one-to-one mapping F (θ) that transforms the ˜ physical space with units θ to a homogeneous space with units θ = F (θ) in which the stimulus distribution becomes uniform. This deﬁnes the mapping as θ F (θ) = p(χ)dχ , (1) −∞ which is the cumulative of the prior distribution p(θ). We then assume a neural population with identical tuning curves that evenly tiles the stimulus range in this homogeneous space. The population provides an efﬁcient representation of the sensory variable θ according to the above constraints [11]. ˜ The tuning curves in the physical space are obtained by applying the inverse mapping F −1 (θ). Fig. 2 1 In the context of this paper, we consider ‘inferring’, ‘decoding’, and ‘estimating’ as synonymous. 2 stimulus distribution d samples # a Fisher information discriminability and average firing rates and b firing rate [ Hz] efficient encoding F likelihood function F -1 likelihood c symmetric asymmetric homogeneous space physical space Figure 2: Efﬁcient encoding constrains the likelihood function. a) Prior distribution p(θ) derived from stimulus statistics. b) Efﬁcient coding deﬁnes the shape of the tuning curves in the physical space by transforming a set of homogeneous neurons using a mapping F −1 that is the inverse of the cumulative of the prior p(θ) (see Eq. (1)). c) As a result, the likelihood shape is constrained by the prior distribution showing heavier tails on the side of lower prior density. d) Fisher information, discrimination threshold, and average ﬁring rates are all uniform in the homogeneous space. illustrates the applied efﬁcient encoding scheme, the mapping, and the concept of the homogeneous space for the example of a symmetric, exponentially decaying prior distribution p(θ). The key idea here is that by assuming efﬁcient encoding, the prior (i.e. the stimulus distribution in the world) directly constrains the likelihood function. In particular, the shape of the likelihood is determined by the cumulative distribution of the prior. As a result, the likelihood is generally asymmetric, as shown in Fig. 2, exhibiting heavier tails on the side of the prior with lower density. 2.2 Bayesian decoding Let us consider a population of N sensory neurons that efﬁciently represents a stimulus variable θ as described above. A stimulus θ0 elicits a speciﬁc population response that is characterized by the vector R = [r1 , r2 , ..., rN ] where ri is the spike-count of the ith neuron over a given time-window τ . Under the assumption that the variability in the individual ﬁring rates is governed by a Poisson process, we can write the likelihood function over θ as N p(R|θ) = (τ fi (θ))ri −τ fi (θ) e , ri ! i=1 (2) ˆ with fi (θ) describing the tuning curve of neuron i. We then deﬁne a Bayesian decoder θLSE as the estimator that minimizes the expected squared-error between the estimate and the true stimulus value, thus θp(R|θ)p(θ)dθ ˆ θLSE (R) = , (3) p(R|θ)p(θ)dθ where we use Bayes’ rule to appropriately combine the sensory evidence with the stimulus prior p(θ). 3 Bayesian estimates can be biased away from prior peaks Bayesian models of perception typically predict perceptual biases toward the peaks of the prior density, a characteristic often considered a hallmark of Bayesian inference. This originates from the 3 a b prior attraction prior prior attraction likelihood repulsion! likelihood c prior prior repulsive bias likelihood likelihood mean posterior mean posterior mean Figure 3: Bayesian estimates biased away from the prior. a) If the likelihood function is symmetric, then the estimate (posterior mean) is, on average, shifted away from the actual value of the sensory variable θ0 towards the prior peak. b) Efﬁcient encoding typically leads to an asymmetric likelihood function whose normalized mean is away from the peak of the prior (relative to θ0 ). The estimate is determined by a combination of prior attraction and shifted likelihood mean, and can exhibit an overall repulsive bias. c) If p(θ0 ) < 0 and the likelihood is relatively narrow, then (1/p(θ)2 ) > 0 (blue line) and the estimate is biased away from the prior peak (see Eq. (6)). common approach of choosing a parametric description of the likelihood function that is computationally convenient (e.g. Gaussian). As a consequence, likelihood functions are typically assumed to be symmetric (but see [23, 24]), leaving the bias of the Bayesian estimator to be mainly determined by the shape of the prior density, i.e. leading to biases toward the peak of the prior (Fig. 3a). In our model framework, the shape of the likelihood function is constrained by the stimulus prior via efﬁcient neural encoding, and is generally not symmetric for non-ﬂat priors. It has a heavier tail on the side with lower prior density (Fig. 3b). The intuition is that due to the efﬁcient allocation of neural resources, the side with smaller prior density will be encoded less accurately, leading to a broader likelihood function on that side. The likelihood asymmetry pulls the Bayes’ least-squares estimate away from the peak of the prior while at the same time the prior pulls it toward its peak. Thus, the resulting estimation bias is the combination of these two counter-acting forces - and both are determined by the prior! 3.1 General derivation of the estimation bias In the following, we will formally derive the mean estimation bias b(θ) of the proposed encodingdecoding framework. Speciﬁcally, we will study the conditions for which the bias is repulsive i.e. away from the peak of the prior density. ˆ We ﬁrst re-write the estimator θLSE (3) by replacing θ with the inverse of its mapping to the homo−1 ˜ geneous space, i.e., θ = F (θ). The motivation for this is that the likelihood in the homogeneous space is symmetric (Fig. 2). Given a value θ0 and the elicited population response R, we can write the estimator as ˜ ˜ ˜ ˜ θp(R|θ)p(θ)dθ F −1 (θ)p(R|F −1 (θ))p(F −1 (θ))dF −1 (θ) ˆ θLSE (R) = = . ˜ ˜ ˜ p(R|θ)p(θ)dθ p(R|F −1 (θ))p(F −1 (θ))dF −1 (θ) Calculating the derivative of the inverse function and noting that F is the cumulative of the prior density, we get 1 1 1 ˜ ˜ ˜ ˜ ˜ ˜ dθ = dθ. dF −1 (θ) = (F −1 (θ)) dθ = dθ = −1 (θ)) ˜ F (θ) p(θ) p(F ˆ Hence, we can simplify θLSE (R) as ˆ θLSE (R) = ˜ ˜ ˜ F −1 (θ)p(R|F −1 (θ))dθ . ˜ ˜ p(R|F −1 (θ))dθ With ˜ K(R, θ) = ˜ p(R|F −1 (θ)) ˜ ˜ p(R|F −1 (θ))dθ 4 we can further simplify the notation and get ˆ θLSE (R) = ˜ ˜ ˜ F −1 (θ)K(R, θ)dθ . (4) ˆ ˜ In order to get the expected value of the estimate, θLSE (θ), we marginalize (4) over the population response space S, ˆ ˜ ˜ ˜ ˜ θLSE (θ) = p(R)F −1 (θ)K(R, θ)dθdR S = F −1 ˜ (θ)( ˜ ˜ p(R)K(R, θ)dR)dθ = ˜ ˜ ˜ F −1 (θ)L(θ)dθ, S where we deﬁne ˜ L(θ) = ˜ p(R)K(R, θ)dR. S ˜ ˜ ˜ It follows that L(θ)dθ = 1. Due to the symmetry in this space, it can be shown that L(θ) is ˜0 . Intuitively, L(θ) can be thought as the normalized ˜ symmetric around the true stimulus value θ average likelihood in the homogeneous space. We can then compute the expected bias at θ0 as b(θ0 ) = ˜ ˜ ˜ ˜ F −1 (θ)L(θ)dθ − F −1 (θ0 ) (5) ˜ This is expression is general where F −1 (θ) is deﬁned as the inverse of the cumulative of an arbitrary ˜ prior density p(θ) (see Eq. (1)) and the dispersion of L(θ) is determined by the internal noise level. ˜ ˜ Assuming the prior density to be smooth, we expand F −1 in a neighborhood (θ0 − h, θ0 + h) that is larger than the support of the likelihood function. Using Taylor’s theorem with mean-value forms of the remainder, we get 1 ˜ ˜ ˜ ˜ ˜ ˜ ˜ ˜ F −1 (θ) = F −1 (θ0 ) + F −1 (θ0 ) (θ − θ0 ) + F −1 (θx ) (θ − θ0 )2 , 2 ˜ ˜ ˜ with θx lying between θ0 and θ. By applying this expression to (5), we ﬁnd ˜ θ0 +h b(θ0 ) = = 1 2 ˜ θ0 −h 1 −1 ˜ ˜ ˜ ˜ ˜ 1 F (θx )θ (θ − θ0 )2 L(θ)dθ = ˜ 2 2 ˜ θ0 +h −( ˜ θ0 −h p(θx )θ ˜ ˜ 2 ˜ ˜ 1 )(θ − θ0 ) L(θ)dθ = p(θx )3 4 ˜ θ0 +h 1 ˜ − θ0 )2 L(θ)dθ ˜ ˜ ˜ ( ) ˜(θ ˜ p(F −1 (θx )) θ ( 1 ˜ ˜ ˜ ˜ ) (θ − θ0 )2 L(θ)dθ. p(θx )2 θ ˜ θ0 −h ˜ θ0 +h ˜ θ0 −h In general, there is no simple rule to judge the sign of b(θ0 ). However, if the prior is monotonic ˜ ˜ on the interval F −1 ((θ0 − h, θ0 + h)), then the sign of ( p(θ1 )2 ) is always the same as the sign of x 1 1 ( p(θ0 )2 ) . Also, if the likelihood is sufﬁciently narrow we can approximate ( p(θ1 )2 ) by ( p(θ0 )2 ) , x and therefore approximate the bias as b(θ0 ) ≈ C( 1 ) , p(θ0 )2 (6) where C is a positive constant. The result is quite surprising because it states that as long as the prior is monotonic over the support of the likelihood function, the expected estimation bias is always away from the peaks of the prior! 3.2 Internal (neural) versus external (stimulus) noise The above derivation of estimation bias is based on the assumption that all uncertainty about the sensory variable is caused by neural response variability. This level of internal noise depends on the response magnitude, and thus can be modulated e.g. by changing stimulus contrast. This contrastcontrolled noise modulation is commonly exploited in perceptual studies (e.g. [18]). Internal noise will always lead to repulsive biases in our framework if the prior is monotonic. If internal noise is low, the likelihood is narrow and thus the bias is small. Increasing internal noise leads to increasingly 5 larger biases up to the point where the likelihood becomes wide enough such that monotonicity of the prior over the support of the likelihood is potentially violated. Stimulus noise is another way to modulate the noise level in perception (e.g. random-dot motion stimuli). Such external noise, however, has a different effect on the shape of the likelihood function as compared to internal noise. It modiﬁes the likelihood function (2) by convolving it with the noise kernel. External noise is frequently chosen as additive and symmetric (e.g. zero-mean Gaussian). It is straightforward to prove that such symmetric external noise does not lead to a change in the mean of the likelihood, and thus does not alter the repulsive effect induced by its asymmetry. However, by increasing the overall width of the likelihood, the attractive inﬂuence of the prior increases, resulting in an estimate that is closer to the prior peak than without external noise2 . 4 Perception of visual orientation We tested our framework by modelling the perception of visual orientation. Our choice was based on the fact that i) we have pretty good estimates of the prior distribution of local orientations in natural images, ii) tuning characteristics of orientation selective neurons in visual cortex are wellstudied (monkey/cat), and iii) biases in perceived stimulus orientation have been well characterized. We start by creating an efﬁcient neural population based on measured prior distributions of local visual orientation, and then compare the resulting tuning characteristics of the population and the predicted perceptual biases with reported data in the literature. 4.1 Efﬁcient neural model population for visual orientation Previous studies measured the statistics of the local orientation in large sets of natural images and consistently found that the orientation distribution is multimodal, peaking at the two cardinal orientations as shown in Fig. 4a [16, 20]. We assumed that the visual system’s prior belief over orientation p(θ) follows this distribution and approximate it formally as p(θ) ∝ 2 − | sin(θ)| (black line in Fig. 4b) . (7) Based on this prior distribution we deﬁned an efﬁcient neural representation for orientation. We assumed a population of model neurons (N = 30) with tuning curves that follow a von-Mises distribution in the homogeneous space on top of a constant spontaneous ﬁring rate (5 Hz). We then ˜ applied the inverse transformation F −1 (θ) to all these tuning curves to get the corresponding tuning curves in the physical space (Fig. 4b - red curves), where F (θ) is the cumulative of the prior (7). The concentration parameter for the von-Mises tuning curves was set to κ ≈ 1.6 in the homogeneous space in order to match the measured average tuning width (∼ 32 deg) of neurons in area V1 of the macaque [9]. 4.2 Predicted tuning characteristics of neurons in primary visual cortex The orientation tuning characteristics of our model population well match neurophysiological data of neurons in primary visual cortex (V1). Efﬁcient encoding predicts that the distribution of neurons’ preferred orientation follows the prior, with more neurons tuned to cardinal than oblique orientations by a factor of approximately 1.5. A similar ratio has been found for neurons in area V1 of monkey/cat [9, 10]. Also, the tuning widths of the model neurons vary between 25-42 deg depending on their preferred tuning (see Fig. 4c), matching the measured tuning width ratio of 0.6 between neurons tuned to the cardinal versus oblique orientations [9]. An important prediction of our model is that most of the tuning curves should be asymmetric. Such asymmetries have indeed been reported for the orientation tuning of neurons in area V1 [6, 7, 8]. We computed the asymmetry index for our model population as deﬁned in previous studies [6, 7], and plotted it as a function of the preferred tuning of each neuron (Fig. 4d). The overall asymmetry index in our model population is 1.24 ± 0.11, which approximately matches the measured values for neurons in area V1 of the cat (1.26 ± 0.06) [6]. It also predicts that neurons tuned to the cardinal and oblique orientations should show less symmetry than those tuned to orientations in between. Finally, 2 Note, that these predictions are likely to change if the external noise is not symmetric. 6 a b 25 firing rate(Hz) 0 orientation(deg) asymmetry vs. tuning width 1.0 2.0 90 2.0 e asymmetry 1.0 0 asymmetry index 50 30 width (deg) 10 90 preferred tuning(deg) -90 0 d 0 0 90 asymmetry index 0 orientation(deg) tuning width -90 0 0 probability 0 -90 c efficient representation 0.01 0.01 image statistics -90 0 90 preferred tuning(deg) 25 30 35 40 tuning width (deg) Figure 4: Tuning characteristics of model neurons. a) Distribution of local orientations in natural images, replotted from [16]. b) Prior used in the model (black) and predicted tuning curves according to efﬁcient coding (red). c) Tuning width as a function of preferred orientation. d) Tuning curves of cardinal and oblique neurons are more symmetric than those tuned to orientations in between. e) Both narrowly and broadly tuned neurons neurons show less asymmetry than neurons with tuning widths in between. neurons with tuning widths at the lower and upper end of the range are predicted to exhibit less asymmetry than those neurons whose widths lie in between these extremes (illustrated in Fig. 4e). These last two predictions have not been tested yet. 4.3 Predicted perceptual biases Our model framework also provides speciﬁc predictions for the expected perceptual biases. Humans show systematic biases in perceived orientation of visual stimuli such as e.g. arrays of Gabor patches (Fig. 5a,d). Two types of biases can be distinguished: First, perceived orientations show an absolute bias away from the cardinal orientations, thus away from the peaks of the orientation prior [2, 3]. We refer to these biases as absolute because they are typically measured by adjusting a noise-free reference until it matched the orientation of the test stimulus. Interestingly, these repulsive absolute biases are the larger the smaller the external stimulus noise is (see Fig. 5b). Second, the relative bias between the perceived overall orientations of a high-noise and a low-noise stimulus is toward the cardinal orientations as shown in Fig. 5c, and thus toward the peak of the prior distribution [3, 16]. The predicted perceptual biases of our model are shown Fig. 5e,f. We computed the likelihood function according to (2) and used the prior in (7). External noise was modeled by convolving the stimulus likelihood function with a Gaussian (different widths for different noise levels). The predictions well match both, the reported absolute bias away as well as the relative biases toward the cardinal orientations. Note, that our model framework correctly accounts for the fact that less external noise leads to larger absolute biases (see also discussion in section 3.2). 5 Discussion We have presented a modeling framework for perception that combines efﬁcient (en)coding and Bayesian decoding. Efﬁcient coding imposes constraints on the tuning characteristics of a population of neurons according to the stimulus distribution (prior). It thus establishes a direct link between prior and likelihood, and provides clear constraints on the latter for a Bayesian observer model of perception. We have shown that the resulting likelihoods are in general asymmetric, with 7 absolute bias (data) b c relative bias (data) -4 0 bias(deg) 4 a low-noise stimulus -90 e 90 absolute bias (model) low external noise high external noise 3 high-noise stimulus -90 f 0 90 relative bias (model) 0 bias(deg) d 0 attraction -3 repulsion -90 0 orientation (deg) 90 -90 0 orientation (deg) 90 Figure 5: Biases in perceived orientation: Human data vs. Model prediction. a,d) Low- and highnoise orientation stimuli of the type used in [3, 16]. b) Humans show absolute biases in perceived orientation that are away from the cardinal orientations. Data replotted from [2] (pink squares) and [3] (green (black) triangles: bias for low (high) external noise). c) Relative bias between stimuli with different external noise level (high minus low). Data replotted from [3] (blue triangles) and [16] (red circles). e,f) Model predictions for absolute and relative bias. heavier tails away from the prior peaks. We demonstrated that such asymmetric likelihoods can lead to the counter-intuitive prediction that a Bayesian estimator is biased away from the peaks of the prior distribution. Interestingly, such repulsive biases have been reported for human perception of visual orientation, yet a principled and consistent explanation of their existence has been missing so far. Here, we suggest that these counter-intuitive biases directly follow from the asymmetries in the likelihood function induced by efﬁcient neural encoding of the stimulus. The good match between our model predictions and the measured perceptual biases and orientation tuning characteristics of neurons in primary visual cortex provides further support of our framework. Previous work has suggested that there might be a link between stimulus statistics, neuronal tuning characteristics, and perceptual behavior based on efﬁcient coding principles, yet none of these studies has recognized the importance of the resulting likelihood asymmetries [16, 11]. We have demonstrated here that such asymmetries can be crucial in explaining perceptual data, even though the resulting estimates appear “anti-Bayesian” at ﬁrst sight (see also models of sensory adaptation [23]). Note, that we do not provide a neural implementation of the Bayesian inference step. However, we and others have proposed various neural decoding schemes that can approximate Bayes’ leastsquares estimation using efﬁcient coding [26, 25, 22]. It is also worth pointing out that our estimator is set to minimize total squared-error, and that other choices of the loss function (e.g. MAP estimator) could lead to different predictions. Our framework is general and should be directly applicable to other modalities. In particular, it might provide a new explanation for perceptual biases that are hard to reconcile with traditional Bayesian approaches [5]. Acknowledgments We thank M. Jogan and A. Tank for helpful comments on the manuscript. This work was partially supported by grant ONR N000141110744. 8 References [1] M. Jones, and B. C. Love. Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behavioral and Brain Sciences, 34, 169–231,2011. [2] D. P. Andrews. Perception of contours in the central fovea. Nature, 205:1218- 1220, 1965. [3] A. Tomassini, M. J.Morgam. and J. A. Solomon. Orientation uncertainty reduces perceived obliquity. Vision Res, 50, 541–547, 2010. [4] W. S. Geisler, D. Kersten. Illusions, perception and Bayes. Nature Neuroscience, 5(6):508- 510, 2002. [5] M. O. Ernst Perceptual learning: inverting the size-weight illusion. Current Biology, 19:R23- R25, 2009. [6] G. H. Henry, B. Dreher, P. O. Bishop. Orientation speciﬁcity of cells in cat striate cortex. J Neurophysiol, 37(6):1394-409,1974. [7] D. Rose, C. Blakemore An analysis of orientation selectivity in the cat’s visual cortex. Exp Brain Res., Apr 30;20(1):1-17, 1974. [8] N. V. Swindale. Orientation tuning curves: empirical description and estimation of parameters. Biol Cybern., 78(1):45-56, 1998. [9] R. L. De Valois, E. W. Yund, N. Hepler. The orientation and direction selectivity of cells in macaque visual cortex. Vision Res.,22, 531544,1982. [10] B. Li, M. R. Peterson, R. D. Freeman. The oblique effect: a neural basis in the visual cortex. J. Neurophysiol., 90, 204217, 2003. [11] D. Ganguli and E.P. Simoncelli. Implicit encoding of prior probabilities in optimal neural populations. In Adv. Neural Information Processing Systems NIPS 23, vol. 23:658–666, 2011. [12] M. D. McDonnell, N. G. Stocks. Maximally Informative Stimuli and Tuning Curves for Sigmoidal RateCoding Neurons and Populations. Phys Rev Lett., 101(5):058103, 2008. [13] H Helmholtz. Treatise on Physiological Optics (transl.). Thoemmes Press, Bristol, U.K., 2000. Original publication 1867. [14] Y. Weiss, E. Simoncelli, and E. Adelson. Motion illusions as optimal percept. Nature Neuroscience, 5(6):598–604, June 2002. [15] D.C. Knill and W. Richards, editors. Perception as Bayesian Inference. Cambridge University Press, 1996. [16] A R Girshick, M S Landy, and E P Simoncelli. Cardinal rules: visual orientation perception reﬂects knowledge of environmental statistics. Nat Neurosci, 14(7):926–932, Jul 2011. [17] M. Jazayeri and M.N. Shadlen. Temporal context calibrates interval timing. Nature Neuroscience, 13(8):914–916, 2010. [18] A.A. Stocker and E.P. Simoncelli. Noise characteristics and prior expectations in human visual speed perception. Nature Neuroscience, pages 578–585, April 2006. [19] H.B. Barlow. Possible principles underlying the transformation of sensory messages. In W.A. Rosenblith, editor, Sensory Communication, pages 217–234. MIT Press, Cambridge, MA, 1961. [20] D.M. Coppola, H.R. Purves, A.N. McCoy, and D. Purves The distribution of oriented contours in the real world. Proc Natl Acad Sci U S A., 95(7): 4002–4006, 1998. [21] N. Brunel and J.-P. Nadal. Mutual information, Fisher information and population coding. Neural Computation, 10, 7, 1731–1757, 1998. [22] X-X. Wei and A.A. Stocker. Bayesian inference with efﬁcient neural population codes. In Lecture Notes in Computer Science, Artiﬁcial Neural Networks and Machine Learning - ICANN 2012, Lausanne, Switzerland, volume 7552, pages 523–530, 2012. [23] A.A. Stocker and E.P. Simoncelli. Sensory adaptation within a Bayesian framework for perception. In Y. Weiss, B. Sch¨ lkopf, and J. Platt, editors, Advances in Neural Information Processing Systems 18, pages o 1291–1298. MIT Press, Cambridge, MA, 2006. Oral presentation. [24] D.C. Knill. Robust cue integration: A Bayesian model and evidence from cue-conﬂict studies with stereoscopic and ﬁgure cues to slant. Journal of Vision, 7(7):1–24, 2007. [25] Deep Ganguli. Efﬁcient coding and Bayesian inference with neural populations. PhD thesis, Center for Neural Science, New York University, New York, NY, September 2012. [26] B. Fischer. Bayesian estimates from heterogeneous population codes. In Proc. IEEE Intl. Joint Conf. on Neural Networks. IEEE, 2010. 9

5 0.078086197 311 nips-2012-Shifting Weights: Adapting Object Detectors from Image to Video

Author: Kevin Tang, Vignesh Ramanathan, Li Fei-fei, Daphne Koller

Abstract: Typical object detectors trained on images perform poorly on video, as there is a clear distinction in domain between the two types of data. In this paper, we tackle the problem of adapting object detectors learned from images to work well on videos. We treat the problem as one of unsupervised domain adaptation, in which we are given labeled data from the source domain (image), but only unlabeled data from the target domain (video). Our approach, self-paced domain adaptation, seeks to iteratively adapt the detector by re-training the detector with automatically discovered target domain examples, starting with the easiest ﬁrst. At each iteration, the algorithm adapts by considering an increased number of target domain examples, and a decreased number of source domain examples. To discover target domain examples from the vast amount of video data, we introduce a simple, robust approach that scores trajectory tracks instead of bounding boxes. We also show how rich and expressive features speciﬁc to the target domain can be incorporated under the same framework. We show promising results on the 2011 TRECVID Multimedia Event Detection [1] and LabelMe Video [2] datasets that illustrate the beneﬁt of our approach to adapt object detectors to video. 1

6 0.07731685 210 nips-2012-Memorability of Image Regions

7 0.070730381 2 nips-2012-3D Social Saliency from Head-mounted Cameras

8 0.066737406 303 nips-2012-Searching for objects driven by context

9 0.065823659 90 nips-2012-Deep Learning of Invariant Features via Simulated Fixations in Video

10 0.063994624 195 nips-2012-Learning visual motion in recurrent neural networks

11 0.062831573 5 nips-2012-A Conditional Multinomial Mixture Model for Superset Label Learning

12 0.059317276 198 nips-2012-Learning with Target Prior

13 0.056459263 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

14 0.040918395 360 nips-2012-Visual Recognition using Embedded Feature Selection for Curvature Self-Similarity

15 0.038327333 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

16 0.038062971 62 nips-2012-Burn-in, bias, and the rationality of anchoring

17 0.038062971 116 nips-2012-Emergence of Object-Selective Features in Unsupervised Feature Learning

18 0.036263347 83 nips-2012-Controlled Recognition Bounds for Visual Learning and Exploration

19 0.036025338 23 nips-2012-A lattice filter model of the visual pathway

20 0.035580158 56 nips-2012-Bayesian active learning with localized priors for fast receptive field characterization

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.096), (1, 0.013), (2, -0.091), (3, 0.057), (4, 0.039), (5, 0.04), (6, 0.039), (7, 0.012), (8, -0.013), (9, 0.029), (10, -0.058), (11, -0.008), (12, 0.022), (13, -0.053), (14, 0.014), (15, 0.037), (16, 0.006), (17, 0.008), (18, -0.0), (19, 0.04), (20, 0.03), (21, 0.002), (22, -0.042), (23, 0.065), (24, 0.019), (25, -0.026), (26, 0.023), (27, 0.088), (28, 0.067), (29, -0.042), (30, -0.055), (31, 0.151), (32, 0.01), (33, -0.048), (34, 0.11), (35, 0.001), (36, -0.025), (37, -0.087), (38, -0.022), (39, 0.106), (40, 0.035), (41, -0.005), (42, 0.005), (43, 0.08), (44, -0.022), (45, -0.023), (46, -0.02), (47, 0.06), (48, -0.02), (49, -0.026)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.95967841 256 nips-2012-On the connections between saliency and tracking

Author: Vijay Mahadevan, Nuno Vasconcelos

2 0.7761113 94 nips-2012-Delay Compensation with Dynamical Synapses

Author: Chi Fung, K. Wong, Si Wu

3 0.58267224 140 nips-2012-Fusion with Diffusion for Robust Visual Tracking

Author: Yu Zhou, Xiang Bai, Wenyu Liu, Longin J. Latecki

4 0.55868345 24 nips-2012-A mechanistic model of early sensory processing based on subtracting sparse representations

Author: Shaul Druckmann, Tao Hu, Dmitri B. Chklovskii

Abstract: Early stages of sensory systems face the challenge of compressing information from numerous receptors onto a much smaller number of projection neurons, a so called communication bottleneck. To make more efficient use of limited bandwidth, compression may be achieved using predictive coding, whereby predictable, or redundant, components of the stimulus are removed. In the case of the retina, Srinivasan et al. (1982) suggested that feedforward inhibitory connections subtracting a linear prediction generated from nearby receptors implement such compression, resulting in biphasic center-surround receptive fields. However, feedback inhibitory circuits are common in early sensory circuits and furthermore their dynamics may be nonlinear. Can such circuits implement predictive coding as well? Here, solving the transient dynamics of nonlinear reciprocal feedback circuits through analogy to a signal-processing algorithm called linearized Bregman iteration we show that nonlinear predictive coding can be implemented in an inhibitory feedback circuit. In response to a step stimulus, interneuron activity in time constructs progressively less sparse but more accurate representations of the stimulus, a temporally evolving prediction. This analysis provides a powerful theoretical framework to interpret and understand the dynamics of early sensory processing in a variety of physiological experiments and yields novel predictions regarding the relation between activity and stimulus statistics.

5 0.53210557 114 nips-2012-Efficient coding provides a direct link between prior and likelihood in perceptual Bayesian inference

Author: Xue-xin Wei, Alan Stocker

6 0.51012051 2 nips-2012-3D Social Saliency from Head-mounted Cameras

7 0.49151444 311 nips-2012-Shifting Weights: Adapting Object Detectors from Image to Video

8 0.43260846 198 nips-2012-Learning with Target Prior

9 0.43160358 262 nips-2012-Optimal Neural Tuning Curves for Arbitrary Stimulus Distributions: Discrimax, Infomax and Minimum $L p$ Loss

10 0.41969144 28 nips-2012-A systematic approach to extracting semantic information from functional MRI data

11 0.40589258 195 nips-2012-Learning visual motion in recurrent neural networks

12 0.40122986 289 nips-2012-Recognizing Activities by Attribute Dynamics

13 0.40091604 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

14 0.39801016 273 nips-2012-Predicting Action Content On-Line and in Real Time before Action Onset – an Intracranial Human Study

15 0.38182446 46 nips-2012-Assessing Blinding in Clinical Trials

16 0.37844178 303 nips-2012-Searching for objects driven by context

17 0.3771992 210 nips-2012-Memorability of Image Regions

18 0.37542307 224 nips-2012-Multi-scale Hyper-time Hardware Emulation of Human Motor Nervous System Based on Spiking Neurons using FPGA

19 0.37481865 113 nips-2012-Efficient and direct estimation of a neural subunit model for sensory coding

20 0.36764109 50 nips-2012-Bandit Algorithms boost Brain Computer Interfaces for motor-task selection of a brain-controlled button

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.028), (11, 0.022), (17, 0.017), (21, 0.06), (38, 0.075), (39, 0.014), (42, 0.026), (46, 0.297), (54, 0.025), (55, 0.024), (74, 0.07), (76, 0.121), (77, 0.013), (80, 0.051), (92, 0.043)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.76969302 256 nips-2012-On the connections between saliency and tracking

Author: Vijay Mahadevan, Nuno Vasconcelos

2 0.67668289 244 nips-2012-Nonconvex Penalization Using Laplace Exponents and Concave Conjugates

Author: Zhihua Zhang, Bojun Tu

Abstract: In this paper we study sparsity-inducing nonconvex penalty functions using L´ vy e processes. We deﬁne such a penalty as the Laplace exponent of a subordinator. Accordingly, we propose a novel approach for the construction of sparsityinducing nonconvex penalties. Particularly, we show that the nonconvex logarithmic (LOG) and exponential (EXP) penalty functions are the Laplace exponents of Gamma and compound Poisson subordinators, respectively. Additionally, we explore the concave conjugate of nonconvex penalties. We ﬁnd that the LOG and EXP penalties are the concave conjugates of negative Kullback-Leiber (KL) distance functions. Furthermore, the relationship between these two penalties is due to asymmetricity of the KL distance. 1

3 0.67369914 183 nips-2012-Learning Partially Observable Models Using Temporally Abstract Decision Trees

Author: Erik Talvitie

Abstract: This paper introduces timeline trees, which are partial models of partially observable environments. Timeline trees are given some speciﬁc predictions to make and learn a decision tree over history. The main idea of timeline trees is to use temporally abstract features to identify and split on features of key events, spread arbitrarily far apart in the past (whereas previous decision-tree-based methods have been limited to a ﬁnite sufﬁx of history). Experiments demonstrate that timeline trees can learn to make high quality predictions in complex, partially observable environments with high-dimensional observations (e.g. an arcade game). 1

4 0.65243089 88 nips-2012-Cost-Sensitive Exploration in Bayesian Reinforcement Learning

Author: Dongho Kim, Kee-eung Kim, Pascal Poupart

Abstract: In this paper, we consider Bayesian reinforcement learning (BRL) where actions incur costs in addition to rewards, and thus exploration has to be constrained in terms of the expected total cost while learning to maximize the expected longterm total reward. In order to formalize cost-sensitive exploration, we use the constrained Markov decision process (CMDP) as the model of the environment, in which we can naturally encode exploration requirements using the cost function. We extend BEETLE, a model-based BRL method, for learning in the environment with cost constraints. We demonstrate the cost-sensitive exploration behaviour in a number of simulated problems. 1

5 0.50887871 210 nips-2012-Memorability of Image Regions

Author: Aditya Khosla, Jianxiong Xiao, Antonio Torralba, Aude Oliva

Abstract: While long term human visual memory can store a remarkable amount of visual information, it tends to degrade over time. Recent works have shown that image memorability is an intrinsic property of an image that can be reliably estimated using state-of-the-art image features and machine learning algorithms. However, the class of features and image information that is forgotten has not been explored yet. In this work, we propose a probabilistic framework that models how and which local regions from an image may be forgotten using a data-driven approach that combines local and global images features. The model automatically discovers memorability maps of individual images without any human annotation. We incorporate multiple image region attributes in our algorithm, leading to improved memorability prediction of images as compared to previous works. 1

6 0.50848889 90 nips-2012-Deep Learning of Invariant Features via Simulated Fixations in Video

7 0.50567377 201 nips-2012-Localizing 3D cuboids in single-view images

8 0.5035702 209 nips-2012-Max-Margin Structured Output Regression for Spatio-Temporal Action Localization

9 0.50350326 112 nips-2012-Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model

10 0.50291681 303 nips-2012-Searching for objects driven by context

11 0.50043064 18 nips-2012-A Simple and Practical Algorithm for Differentially Private Data Release

12 0.49998918 105 nips-2012-Dynamic Pruning of Factor Graphs for Maximum Marginal Prediction

13 0.49872544 113 nips-2012-Efficient and direct estimation of a neural subunit model for sensory coding

14 0.49834666 1 nips-2012-3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model

15 0.49825305 193 nips-2012-Learning to Align from Scratch

16 0.49741921 92 nips-2012-Deep Representations and Codes for Image Auto-Annotation

17 0.49720803 176 nips-2012-Learning Image Descriptors with the Boosting-Trick

18 0.49698839 167 nips-2012-Kernel Hyperalignment

19 0.49664846 260 nips-2012-Online Sum-Product Computation Over Trees

20 0.49645567 357 nips-2012-Unsupervised Template Learning for Fine-Grained Object Recognition