nips nips2011 nips2011-101 knowledge-graph by maker-knowledge-mining

101 nips-2011-Gaussian process modulated renewal processes

Source: pdf

Author: Yee W. Teh, Vinayak Rao

Abstract: Renewal processes are generalizations of the Poisson process on the real line whose intervals are drawn i.i.d. from some distribution. Modulated renewal processes allow these interevent distributions to vary with time, allowing the introduction of nonstationarity. In this work, we take a nonparametric Bayesian approach, modelling this nonstationarity with a Gaussian process. Our approach is based on the idea of uniformization, which allows us to draw exact samples from an otherwise intractable distribution. We develop a novel and efﬁcient MCMC sampler for posterior inference. In our experiments, we test these on a number of synthetic and real datasets. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 Gaussian process modulated renewal processes Yee Whye Teh Gatsby Computational Neuroscience Unit University College London ywteh@gatsby. [sent-1, score-0.954]

2 uk Abstract Renewal processes are generalizations of the Poisson process on the real line whose intervals are drawn i. [sent-7, score-0.205]

3 Modulated renewal processes allow these interevent distributions to vary with time, allowing the introduction of nonstationarity. [sent-11, score-0.849]

4 We develop a novel and efﬁcient MCMC sampler for posterior inference. [sent-14, score-0.156]

5 1 Introduction Renewal processes are stochastic point processes on the real line where intervals between successive points (times) are drawn i. [sent-16, score-0.189]

6 The simplest example of a renewal process is the homogeneous Poisson process, whose interevent times are exponentially distributed. [sent-20, score-0.96]

7 For example, immediately after ﬁring, a neuron is depleted of its resources and incapable of ﬁring again, and the gamma distribution is used to model interspike intervals [2]. [sent-22, score-0.153]

8 draws from a general renewal density can allow larger or smaller variances than an exponential with the same mean (overdispersion or underdispersion), but effectively encodes an ‘as good as new after a repair’ property. [sent-28, score-0.628]

9 Again, this is often only an approximation: because of age or other time-varying factors, the interevent distribution of the point process may vary with time. [sent-29, score-0.237]

10 Similarly, an external stimulus can modulate the ﬁring rate of the neuron, economic trends can modulate ﬁnancial transactions etc. [sent-31, score-0.142]

11 The most popular way of modelling this nonstationarity is via an inhomogeneous Poisson process whose intensity function determines the instantaneous event rate, and there has also been substantial work extending this to renewal processes in various ways (see section 2. [sent-32, score-1.199]

12 In this paper, we describe a nonparametric Bayesian approach where a renewal process is modulated by a random intensity function which is given a Gaussian process prior. [sent-34, score-1.187]

13 Our approach extends work by [6] on the Poisson process, using a generalization of the idea of Poisson thinning called uniformization [7] to draw exact samples from the model. [sent-35, score-0.216]

14 We extend recent ideas from [8] to develop a more natural and efﬁcient block Gibbs sampler than the incremental Metropolis-Hastings algorithm used in [6]. [sent-36, score-0.125]

15 In our experiments we demonstrate the usefulness of our model and sampler on a number of synthetic and real-world datasets. [sent-37, score-0.134]

16 1 2 Modulated renewal processes Consider a renewal process R over an interval [0, T ] whose interevent time is distributed according to a renewal density g. [sent-38, score-2.244]

17 } be the ordered set of event times sampled from this renewal process, i. [sent-42, score-0.723]

18 Associated with the renewal density g is a hazard function h, where h(τ )∆, for inﬁnitesimal ∆ > 0, is the probability of the interevent interval being in [τ, τ + ∆] conditioned on it being at least τ , i. [sent-45, score-1.038]

19 h(τ ) = g(τ ) τ 1 − 0 g(u)du (2) Let λ(t) be some time-varying intensity function. [sent-47, score-0.19]

20 A simple way to introduce nonstationarity into a renewal process is to modulate the hazard function by λ(t) so that it depends on both the time τ since the last event, and on the absolute time t [9, 10]: h(τ, t) ≡ m(h(τ ), λ(t)) (3) where m(·, ·) is some interaction function. [sent-48, score-1.024]

21 With a modulated hazard rate, the distribution of interevent times is no longer stationary. [sent-51, score-0.547]

22 Instead, plugging a multiplicative interaction into (2) and solving for g (see the supplementary material for details), we get τ g(τ |tprev ) = λ(tprev + τ )h(τ ) exp − λ(tprev + u)h(u)du (4) 0 where tprev is the previous event time. [sent-52, score-0.175]

23 Observe that equation (4) encompasses the inhomogeneous Poisson process as a special case (a constant hazard function with multiplicative modulation). [sent-53, score-0.449]

24 1 Gaussian process intensity functions In this paper we are interested in estimating both parameters of the hazard function h(τ ) as well as the intensity function λ(t) itself. [sent-55, score-0.702]

25 We call the resulting model a Gaussian process modulated renewal process. [sent-57, score-0.874]

26 Finally, we use a gamma family for the hazard function: γ−1 −γx e h(τ ) = ∞x γ−1 e−γu du where γ is the shape parameter2 . [sent-59, score-0.461]

27 We place a gamma hyperprior on λ∗ as well as hyperpriors on the GP hyperparameters. [sent-61, score-0.177]

28 2 Related work The idea of deﬁning a nonstationary renewal process by modulating the hazard function dates back to Cox [9]. [sent-63, score-1.114]

29 [13, 14, 1] proposed parametric (generalized linear) models where the intensity function was a linear combination of some known functions; these regression coefﬁcients were estimated via maximum likelihood. [sent-65, score-0.19]

30 [15] considers general modulated hazard functions as well; however they assume it has known form and are concerned with calculating statistical properties of the resulting process. [sent-66, score-0.376]

31 1 With renewal processes there is an ambiguity about the time of the ﬁrst event, which is typically taken to be exponentially distributed. [sent-67, score-0.708]

32 2 We parametrize the hazard function to produce 1 event per unit time; other parametrizations may be used. [sent-69, score-0.291]

33 A different approach to producing inhomogeneity is by ﬁrst sampling from a homogeneous renewal process and then rescaling time [16, 17]. [sent-71, score-0.859]

34 The trend renewal process [18] uses such an approach, and the authors propose an iterative kernel smoothing scheme to approximate a maximum likelihood estimate of the intensity function. [sent-72, score-0.914]

35 [2] uses time-rescaling to introduce inhomogeneity and, similar to us, a Gaussian process prior for the intensity function. [sent-73, score-0.327]

36 Finally, we note that our approach generalizes [6], who describe a doubly stochastic Poisson process and an MCMC sampler which does not require time discretization. [sent-75, score-0.196]

37 In the next sections we describe a generalization of their model to the inhomogeneous renewal process using a twist on a classical idea called uniformization. [sent-76, score-0.828]

38 2 we will show how a classical idea called uniformization allows us to efﬁciently draw exact samples from the model, without approximations due to discretization. [sent-80, score-0.14]

39 1 Modulated Poisson processes We start with thinning, a well-known result to sample from an inhomogeneous Poisson process with intensity λ(t). [sent-83, score-0.47]

40 Let E be a set of locations sampled from a homogeneous Poisson process with rate Ω. [sent-85, score-0.192]

41 The set F is a draw from a Poisson process with intensity function λ(t) . [sent-89, score-0.286]

42 2 Modulated renewal processes Less well-known is a generalization of this result to renewal processes [13]. [sent-91, score-1.416]

43 Note that the thinning result of the previous section builds on the memoryless property of the exponential distribution (or the complete randomness [20] of the Poisson process): events in disjoint sets occur independently of each other. [sent-92, score-0.188]

44 For a renewal process, events are no longer independent of their neighbours. [sent-93, score-0.716]

45 This idea of thinning a Poisson process by a subordinated Markov chain is called uniformization [7]. [sent-95, score-0.356]

46 [21] describes a uniformization scheme to sample from a homogeneous renewal process. [sent-96, score-0.833]

47 } from a homogeneous Poisson process with rate Ω and thin this set by running a discrete time Markov chain on the times in E. [sent-102, score-0.308]

48 For any Ω ≥ maxt,τ h(τ )λ(t), F is a sample from a modulated renewal process with hazard h(·) and modulating intensity λ(·). [sent-114, score-1.4]

49 The basic idea is to write down the probability p(E, Y ) of the whole generative process and marginalize out the thinned times, showing that the resulting interevent time is simply (4). [sent-116, score-0.403]

50 Our procedure to sample from a GP-modulated renewal process now follows: sample from a homogeneous Poisson process P(Ω) on [0, T ], instantiate the GP on this ﬁnite set of points and then thin the set by running the Markov ∗ chain described previously. [sent-120, score-0.971]

51 We imagine G was generated via uniformization, so that there exists an unobserved set ˜ of thinned events G. [sent-123, score-0.254]

52 In their work, [6] deﬁned a transition operator by proposing insertions and deletions of thinned events as well as by perturbing their locations. [sent-127, score-0.254]

53 In other words, given a set of event times G, the inhomogeneous Poisson process-distributed points ˜ G can be taken to be the events thinned in the procedure of section 3. [sent-140, score-0.453]

54 This is easily overcome by uniformization (in fact, just by thinning, since we’re dealing with a Poisson process). [sent-143, score-0.14]

55 Speciﬁcally, let G be the set of ˜ ˜ observed events and Gprev the previous set of thinned events. [sent-144, score-0.254]

56 This does involve the sigmoid likelihood function, and we proceed by elliptical slice sampling [22] 5 . [sent-148, score-0.125]

57 Algorithm 1 Blocked Gibbs sampler for GP-modulated renewal process on the interval [0, T ] ˜ ˜ Input: Set of event times G, set of thinned times Gprev and l instantiated at G ∪ Gprev . [sent-150, score-1.182]

58 ˜ new and a new instantiation l ˜ ˜ Output: A new set of thinned times G of the GP on G ∪ Gnew . [sent-151, score-0.196]

59 G∪Gnew 1: Sample A ⊂ [0, T ] from a Poisson process with rate Ω. [sent-152, score-0.127]

60 ˜ The gamma prior on λ∗ is conjugate to the Poisson, resulting in a gamma posterior. [sent-158, score-0.248]

61 We resampled the GP hyperparameters using slice sampling [23] 5 , while parameters of the hazard function were updated using Metropolis-Hastings moves along with equation (8). [sent-159, score-0.326]

62 One approach is to try reduce the number of thinned events |E|. [sent-170, score-0.254]

63 Recall that our generative approach is to thin a sample from a subordinating, homogeneous Poisson process whose rate upper bounds the modulated hazard rate. [sent-171, score-0.61]

64 We can reduce the number of thinned events by subordinating to an inhomogeneous Poisson process, one whose rate more closely resembles the instantaneous hazard rate. [sent-172, score-0.662]

65 We used gamma distributed interevent times with shape parameter γ ≥ 1. [sent-183, score-0.383]

66 When appropriate, we place a noninformative prior on the shape parameter: an exponential with rate 0. [sent-185, score-0.15]

67 Note that for shape parameters less than 1, the renewal process becomes ‘bursty’ and the hazard function becomes unbounded. [sent-187, score-1.038]

68 We ﬁnd from our experiments that this is only a problem when the length scale of the intensity function is comparable to the refractory period of the renewal process. [sent-190, score-0.861]

69 The base rate of the modulated renewal process (i. [sent-191, score-0.905]

70 the rate when the intensity function is ﬁxed at 1) is set to the empirical rate of the observed point process. [sent-193, score-0.252]

71 As a result the identiﬁability of the shape parameter is a consequence of the dispersion of the point process rather than of some sort of rate matching. [sent-194, score-0.215]

72 We ran three settings of our model: with the shape parameter ﬁxed to 1 (MRP Exp), with the shape parameter ﬁxed to the truth (MRP Gam3), and with a hyperprior on the shape parameter (MRP Full). [sent-197, score-0.316]

73 For comparison, we also ran an approximate discrete-time sampler where the Gaussian process was instantiated on a regular grid covering the interval of interest. [sent-198, score-0.29]

74 In this case, all intractable integrals were approximated numerically and we use elliptical slice sampling to run MCMC on this Gaussian vector. [sent-199, score-0.153]

75 Not surprisingly, the inhomogeneous Poisson process forms a poor approximation to the gamma renewal process; it underestimates the intensity function required to produce a sequence of events with refractory intervals. [sent-203, score-1.273]

76 Fixing the shape parameter to the truth signiﬁcantly reduces the l2 error and increases the predictive probabilities, but interestingly, for these datasets, the model with a prior on the shape parameter performs comparably with the ‘oracle’ model. [sent-204, score-0.176]

77 We have also included plots of the posterior distribution over the gamma parameter; these are peaked around 3. [sent-205, score-0.18]

78 Discretizing time into a 100 bins (Disc100) results in comparable performance for the ﬁrst two datasets on the l2 error; for the third, (which spans a longer interval and has a larger event count), we had to increase the resolution to 500 bins to improve accuracy. [sent-206, score-0.179]

79 For our next experiment, we ran our model on the coal mine disaster dataset commonly used in the point process literature. [sent-212, score-0.317]

80 Figure 2(left) shows the posterior mean of the intensity function (surrounded by 1 standard deviation) returned by our model. [sent-214, score-0.246]

81 Not included is the posterior distribution over the shape parameter; this concentrated in the interval 1 to around 1. [sent-215, score-0.187]

82 1 0 1 2 3 4 0 5 1 2 3 4 5 0 1 2 3 4 5 Figure 1: Synthetic Datasets 1-3: Posterior mean intensities plotted against time (top) and gamma shape posteriors (bottom) l2 error log pred. [sent-232, score-0.212]

83 With such a transformation, a homogeneous Poisson would reduce to a gamma renewal process with shape 2. [sent-271, score-1.001]

84 Note that the posteriors over intensity functions are similar (except for the obvious scaling factor of about 2). [sent-274, score-0.19]

85 Spike timing data We next ran our model on neural spike train data recorded from grasshopper auditory receptor cells [25]. [sent-275, score-0.137]

86 5 2 Figure 2: Left: Posterior mean intensity for coal mine data with 1 standard deviation error bars (plotted against time in years). [sent-281, score-0.343]

87 Centre: Posterior mean intensity for ‘thinned’ coalmine data with 1 standard deviation error bars. [sent-282, score-0.221]

88 Right: Gamma shape posterior for ‘thinned’ coal mine data. [sent-283, score-0.297]

89 5 Figure 3: Left: Posterior mean intensity for neural data with 1 standard deviation error bars. [sent-285, score-0.19]

90 We plot the posterior distribution over the intensity function given a sequence of 200 spikes in a 1. [sent-314, score-0.246]

91 We also included the posterior distribution over gamma shape parameters in ﬁgure 3; this concentrates around 1. [sent-316, score-0.268]

92 the shape parameter, the stimulus length-scale, the transformation from the stimulus to the input of the neuron etc) can be used to make more accurate inferences. [sent-320, score-0.158]

93 For our ﬁnal experiment, we compare our proposed blocked Gibbs sampler with the Metropolis-Hastings sampler of [6]. [sent-322, score-0.225]

94 We ran both algorithms on two datasets, synthetic dataset 1 from section 5 and the coal mine disaster dataset. [sent-323, score-0.255]

95 6 Discussion We have described how to produce exact samples from a nonstationary renewal process whose hazard function is modulated by a Gaussian process. [sent-329, score-1.131]

96 First is the restriction that the hazard function be bounded: while this covers a large and useful class of renewal processes, it is worth considering how our approach can be extended to produce exact or approximate samples for renewal processes with unbounded hazard functions. [sent-332, score-1.788]

97 Finally, even though we considered GP modulating functions, our uniformization-based sampler will also be useful for Bayesian inference involving simpler priors on modulating functions, eg. [sent-336, score-0.346]

98 Earthquake recurrence on the south Hayward fault is most consistent with a time dependent, renewal process. [sent-356, score-0.628]

99 Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities. [sent-371, score-0.229]

100 Transforming renewal processes for simulation of nonstationary arrival processes. [sent-432, score-0.739]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('renewal', 0.628), ('gi', 0.241), ('poisson', 0.23), ('hazard', 0.226), ('intensity', 0.19), ('thinned', 0.166), ('gp', 0.163), ('modulated', 0.15), ('interevent', 0.141), ('mrp', 0.141), ('uniformization', 0.14), ('gamma', 0.124), ('modulating', 0.11), ('inhomogeneous', 0.104), ('sampler', 0.1), ('process', 0.096), ('coal', 0.094), ('gprev', 0.094), ('events', 0.088), ('shape', 0.088), ('processes', 0.08), ('thinning', 0.076), ('ei', 0.072), ('ess', 0.068), ('event', 0.065), ('homogeneous', 0.065), ('gnew', 0.063), ('tprev', 0.063), ('mcmc', 0.061), ('mine', 0.059), ('iain', 0.059), ('posterior', 0.056), ('elliptical', 0.048), ('ryan', 0.048), ('slice', 0.048), ('subordinating', 0.047), ('murray', 0.047), ('fi', 0.047), ('chain', 0.044), ('refractory', 0.043), ('interval', 0.043), ('thin', 0.042), ('inhomogeneity', 0.041), ('lg', 0.041), ('disaster', 0.041), ('gaussian', 0.038), ('modulate', 0.038), ('yi', 0.037), ('nonstationarity', 0.036), ('stimulus', 0.035), ('synthetic', 0.034), ('traf', 0.034), ('markov', 0.033), ('ring', 0.032), ('ariel', 0.031), ('coalmine', 0.031), ('grasshopper', 0.031), ('markoff', 0.031), ('noninformative', 0.031), ('prescott', 0.031), ('repairable', 0.031), ('rie', 0.031), ('rokem', 0.031), ('unthinned', 0.031), ('rate', 0.031), ('discretizing', 0.031), ('nonstationary', 0.031), ('spike', 0.031), ('times', 0.03), ('adams', 0.03), ('lewis', 0.029), ('sampling', 0.029), ('intervals', 0.029), ('integrals', 0.028), ('hyperpriors', 0.028), ('deleting', 0.028), ('earthquake', 0.028), ('val', 0.028), ('nonparametric', 0.027), ('gatsby', 0.027), ('ran', 0.027), ('inference', 0.026), ('blocked', 0.025), ('receptor', 0.025), ('hyperprior', 0.025), ('ideas', 0.025), ('exp', 0.024), ('bins', 0.024), ('kass', 0.024), ('resample', 0.024), ('instantiated', 0.024), ('vely', 0.024), ('memoryless', 0.024), ('multiplicative', 0.023), ('du', 0.023), ('datasets', 0.023), ('dates', 0.023), ('resampled', 0.023), ('repair', 0.023), ('auditory', 0.023)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000002 101 nips-2011-Gaussian process modulated renewal processes

Author: Yee W. Teh, Vinayak Rao

2 0.12113745 8 nips-2011-A Model for Temporal Dependencies in Event Streams

Author: Asela Gunawardana, Christopher Meek, Puyang Xu

Abstract: We introduce the Piecewise-Constant Conditional Intensity Model, a model for learning temporal dependencies in event streams. We describe a closed-form Bayesian approach to learning these models, and describe an importance sampling algorithm for forecasting future events using these models, using a proposal distribution based on Poisson superposition. We then use synthetic data, supercomputer event logs, and web search query logs to illustrate that our learning algorithm can efﬁciently learn nonlinear temporal dependencies, and that our importance sampling algorithm can effectively forecast future events. 1

3 0.11670841 100 nips-2011-Gaussian Process Training with Input Noise

Author: Andrew Mchutchon, Carl E. Rasmussen

Abstract: In standard Gaussian Process regression input locations are assumed to be noise free. We present a simple yet effective GP model for training on input points corrupted by i.i.d. Gaussian noise. To make computations tractable we use a local linear expansion about each input point. This allows the input noise to be recast as output noise proportional to the squared gradient of the GP posterior mean. The input noise variances are inferred from the data as extra hyperparameters. They are trained alongside other hyperparameters by the usual method of maximisation of the marginal likelihood. Training uses an iterative scheme, which alternates between optimising the hyperparameters and calculating the posterior gradient. Analytic predictive moments can then be found for Gaussian distributed test points. We compare our model to others over a range of different regression problems and show that it improves over current methods. 1

4 0.11198165 131 nips-2011-Inference in continuous-time change-point models

Author: Florian Stimberg, Manfred Opper, Guido Sanguinetti, Andreas Ruttor

Abstract: We consider the problem of Bayesian inference for continuous-time multi-stable stochastic systems which can change both their diffusion and drift parameters at discrete times. We propose exact inference and sampling methodologies for two speciﬁc cases where the discontinuous dynamics is given by a Poisson process and a two-state Markovian switch. We test the methodology on simulated data, and apply it to two real data sets in ﬁnance and systems biology. Our experimental results show that the approach leads to valid inferences and non-trivial insights. 1

5 0.10444804 221 nips-2011-Priors over Recurrent Continuous Time Processes

Author: Ardavan Saeedi, Alexandre Bouchard-côté

Abstract: We introduce the Gamma-Exponential Process (GEP), a prior over a large family of continuous time stochastic processes. A hierarchical version of this prior (HGEP; the Hierarchical GEP) yields a useful model for analyzing complex time series. Models based on HGEPs display many attractive properties: conjugacy, exchangeability and closed-form predictive distribution for the waiting times, and exact Gibbs updates for the time scale parameters. After establishing these properties, we show how posterior inference can be carried efﬁciently using Particle MCMC methods [1]. This yields a MCMC algorithm that can resample entire sequences atomically while avoiding the complications of introducing slice and stick auxiliary variables of the beam sampler [2]. We applied our model to the problem of estimating the disease progression in multiple sclerosis [3], and to RNA evolutionary modeling [4]. In both domains, we found that our model outperformed the standard rate matrix estimation approach. 1

6 0.10285322 26 nips-2011-Additive Gaussian Processes

7 0.09643431 37 nips-2011-Analytical Results for the Error in Filtering of Gaussian Processes

8 0.093260042 24 nips-2011-Active learning of neural response functions with Gaussian processes

9 0.091623954 135 nips-2011-Information Rates and Optimal Decoding in Large Neural Populations

10 0.083362319 30 nips-2011-Algorithms for Hyper-Parameter Optimization

11 0.074525833 302 nips-2011-Variational Learning for Recurrent Spiking Networks

12 0.072290987 123 nips-2011-How biased are maximum entropy models?

13 0.062632881 206 nips-2011-Optimal Reinforcement Learning for Gaussian Systems

14 0.061609156 86 nips-2011-Empirical models of spiking in neural populations

15 0.059152927 78 nips-2011-Efficient Methods for Overlapping Group Lasso

16 0.058820337 77 nips-2011-Efficient Learning of Generalized Linear and Single Index Models with Isotonic Regression

17 0.057430841 219 nips-2011-Predicting response time and error rates in visual search

18 0.055064552 61 nips-2011-Contextual Gaussian Process Bandit Optimization

19 0.054634456 104 nips-2011-Generalized Beta Mixtures of Gaussians

20 0.053814482 301 nips-2011-Variational Gaussian Process Dynamical Systems

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.142), (1, 0.031), (2, 0.12), (3, -0.014), (4, -0.014), (5, -0.102), (6, -0.015), (7, -0.055), (8, 0.028), (9, 0.113), (10, -0.093), (11, -0.037), (12, 0.022), (13, -0.018), (14, -0.046), (15, 0.129), (16, 0.056), (17, 0.028), (18, -0.058), (19, -0.064), (20, 0.018), (21, 0.032), (22, -0.021), (23, -0.149), (24, -0.082), (25, 0.067), (26, -0.045), (27, 0.046), (28, 0.086), (29, -0.085), (30, -0.018), (31, 0.052), (32, -0.088), (33, 0.089), (34, -0.013), (35, -0.071), (36, 0.052), (37, -0.072), (38, 0.031), (39, 0.11), (40, 0.057), (41, 0.003), (42, -0.037), (43, -0.011), (44, 0.113), (45, 0.122), (46, 0.022), (47, 0.057), (48, -0.021), (49, 0.026)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.94206637 101 nips-2011-Gaussian process modulated renewal processes

Author: Yee W. Teh, Vinayak Rao

2 0.66778481 131 nips-2011-Inference in continuous-time change-point models

Author: Florian Stimberg, Manfred Opper, Guido Sanguinetti, Andreas Ruttor

3 0.63322961 37 nips-2011-Analytical Results for the Error in Filtering of Gaussian Processes

Author: Alex K. Susemihl, Ron Meir, Manfred Opper

Abstract: Bayesian ﬁltering of stochastic stimuli has received a great deal of attention recently. It has been applied to describe the way in which biological systems dynamically represent and make decisions about the environment. There have been no exact results for the error in the biologically plausible setting of inference on point process, however. We present an exact analysis of the evolution of the meansquared error in a state estimation task using Gaussian-tuned point processes as sensors. This allows us to study the dynamics of the error of an optimal Bayesian decoder, providing insights into the limits obtainable in this task. This is done for Markovian and a class of non-Markovian Gaussian processes. We ﬁnd that there is an optimal tuning width for which the error is minimized. This leads to a characterization of the optimal encoding for the setting as a function of the statistics of the stimulus, providing a mathematically sound primer for an ecological theory of sensory processing. 1

4 0.62793636 8 nips-2011-A Model for Temporal Dependencies in Event Streams

Author: Asela Gunawardana, Christopher Meek, Puyang Xu

5 0.61844462 221 nips-2011-Priors over Recurrent Continuous Time Processes

Author: Ardavan Saeedi, Alexandre Bouchard-côté

6 0.5826304 100 nips-2011-Gaussian Process Training with Input Noise

7 0.57656509 24 nips-2011-Active learning of neural response functions with Gaussian processes

8 0.57142001 173 nips-2011-Modelling Genetic Variations using Fragmentation-Coagulation Processes

9 0.55444014 30 nips-2011-Algorithms for Hyper-Parameter Optimization

10 0.5353753 26 nips-2011-Additive Gaussian Processes

11 0.49964246 206 nips-2011-Optimal Reinforcement Learning for Gaussian Systems

12 0.46806711 123 nips-2011-How biased are maximum entropy models?

13 0.45360914 135 nips-2011-Information Rates and Optimal Decoding in Large Neural Populations

14 0.42541242 147 nips-2011-Learning Patient-Specific Cancer Survival Distributions as a Sequence of Dependent Regressors

15 0.40659556 62 nips-2011-Continuous-Time Regression Models for Longitudinal Networks

16 0.40647021 55 nips-2011-Collective Graphical Models

17 0.39278471 61 nips-2011-Contextual Gaussian Process Bandit Optimization

18 0.38716701 86 nips-2011-Empirical models of spiking in neural populations

19 0.38391659 285 nips-2011-The Kernel Beta Process

20 0.37990552 42 nips-2011-Bayesian Bias Mitigation for Crowdsourcing

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.023), (3, 0.31), (4, 0.019), (20, 0.027), (26, 0.019), (31, 0.099), (33, 0.013), (43, 0.071), (45, 0.058), (57, 0.047), (63, 0.016), (74, 0.063), (83, 0.05), (84, 0.028), (99, 0.077)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.73715323 101 nips-2011-Gaussian process modulated renewal processes

Author: Yee W. Teh, Vinayak Rao

2 0.70200837 43 nips-2011-Bayesian Partitioning of Large-Scale Distance Data

Author: David Adametz, Volker Roth

Abstract: A Bayesian approach to partitioning distance matrices is presented. It is inspired by the Translation-invariant Wishart-Dirichlet process (TIWD) in [1] and shares a number of advantageous properties like the fully probabilistic nature of the inference model, automatic selection of the number of clusters and applicability in semi-supervised settings. In addition, our method (which we call fastTIWD) overcomes the main shortcoming of the original TIWD, namely its high computational costs. The fastTIWD reduces the workload in each iteration of a Gibbs sampler from O(n3 ) in the TIWD to O(n2 ). Our experiments show that the cost reduction does not compromise the quality of the inferred partitions. With this new method it is now possible to ‘mine’ large relational datasets with a probabilistic model, thereby automatically detecting new and potentially interesting clusters. 1

3 0.621746 33 nips-2011-An Exact Algorithm for F-Measure Maximization

Author: Krzysztof J. Dembczynski, Willem Waegeman, Weiwei Cheng, Eyke Hüllermeier

Abstract: The F-measure, originally introduced in information retrieval, is nowadays routinely used as a performance metric for problems such as binary classiﬁcation, multi-label classiﬁcation, and structured output prediction. Optimizing this measure remains a statistically and computationally challenging problem, since no closed-form maximizer exists. Current algorithms are approximate and typically rely on additional assumptions regarding the statistical distribution of the binary response variables. In this paper, we present an algorithm which is not only computationally efﬁcient but also exact, regardless of the underlying distribution. The algorithm requires only a quadratic number of parameters of the joint distribution (with respect to the number of binary responses). We illustrate its practical performance by means of experimental results for multi-label classiﬁcation. 1

4 0.61727071 134 nips-2011-Infinite Latent SVM for Classification and Multi-task Learning

Author: Jun Zhu, Ning Chen, Eric P. Xing

Abstract: Unlike existing nonparametric Bayesian models, which rely solely on specially conceived priors to incorporate domain knowledge for discovering improved latent representations, we study nonparametric Bayesian inference with regularization on the desired posterior distributions. While priors can indirectly affect posterior distributions through Bayes’ theorem, imposing posterior regularization is arguably more direct and in some cases can be much easier. We particularly focus on developing inﬁnite latent support vector machines (iLSVM) and multi-task inﬁnite latent support vector machines (MT-iLSVM), which explore the largemargin idea in combination with a nonparametric Bayesian model for discovering predictive latent features for classiﬁcation and multi-task learning, respectively. We present efﬁcient inference methods and report empirical studies on several benchmark datasets. Our results appear to demonstrate the merits inherited from both large-margin learning and Bayesian nonparametrics.

5 0.48201233 102 nips-2011-Generalised Coupled Tensor Factorisation

Author: Kenan Y. Yılmaz, Ali T. Cemgil, Umut Simsekli

Abstract: We derive algorithms for generalised tensor factorisation (GTF) by building upon the well-established theory of Generalised Linear Models. Our algorithms are general in the sense that we can compute arbitrary factorisations in a message passing framework, derived for a broad class of exponential family distributions including special cases such as Tweedie’s distributions corresponding to βdivergences. By bounding the step size of the Fisher Scoring iteration of the GLM, we obtain general updates for real data and multiplicative updates for non-negative data. The GTF framework is, then extended easily to address the problems when multiple observed tensors are factorised simultaneously. We illustrate our coupled factorisation approach on synthetic data as well as on a musical audio restoration problem. 1

6 0.47666603 135 nips-2011-Information Rates and Optimal Decoding in Large Neural Populations

7 0.47485003 37 nips-2011-Analytical Results for the Error in Filtering of Gaussian Processes

8 0.47394264 221 nips-2011-Priors over Recurrent Continuous Time Processes

9 0.47305542 249 nips-2011-Sequence learning with hidden units in spiking neural networks

10 0.47305346 57 nips-2011-Comparative Analysis of Viterbi Training and Maximum Likelihood Estimation for HMMs

11 0.47287786 273 nips-2011-Structural equations and divisive normalization for energy-dependent component analysis

12 0.47194433 301 nips-2011-Variational Gaussian Process Dynamical Systems

13 0.4702071 71 nips-2011-Directed Graph Embedding: an Algorithm based on Continuous Limits of Laplacian-type Operators

14 0.46988729 66 nips-2011-Crowdclustering

15 0.46984547 258 nips-2011-Sparse Bayesian Multi-Task Learning

16 0.46967492 86 nips-2011-Empirical models of spiking in neural populations

17 0.46677089 75 nips-2011-Dynamical segmentation of single trials from population neural data

18 0.46537083 183 nips-2011-Neural Reconstruction with Approximate Message Passing (NeuRAMP)

19 0.46528658 219 nips-2011-Predicting response time and error rates in visual search

20 0.46498609 186 nips-2011-Noise Thresholds for Spectral Clustering