nips nips2009 nips2009-168 knowledge-graph by maker-knowledge-mining

168 nips-2009-Non-stationary continuous dynamic Bayesian networks

Source: pdf

Author: Marco Grzegorczyk, Dirk Husmeier

Abstract: Dynamic Bayesian networks have been applied widely to reconstruct the structure of regulatory processes from time series data. The standard approach is based on the assumption of a homogeneous Markov chain, which is not valid in many realworld scenarios. Recent research efforts addressing this shortcoming have considered undirected graphs, directed graphs for discretized data, or over-ﬂexible models that lack any information sharing among time series segments. In the present article, we propose a non-stationary dynamic Bayesian network for continuous data, in which parameters are allowed to vary among segments, and in which a common network structure provides essential information sharing across segments. Our model is based on a Bayesian multiple change-point process, where the number and location of the change-points is sampled from the posterior distribution. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 Non-stationary continuous dynamic Bayesian networks Marco Grzegorczyk Department of Statistics, TU Dortmund University, 44221 Dortmund, Germany grzegorczyk@statistik. [sent-1, score-0.109]

2 uk Abstract Dynamic Bayesian networks have been applied widely to reconstruct the structure of regulatory processes from time series data. [sent-5, score-0.387]

3 Recent research efforts addressing this shortcoming have considered undirected graphs, directed graphs for discretized data, or over-ﬂexible models that lack any information sharing among time series segments. [sent-7, score-0.154]

4 In the present article, we propose a non-stationary dynamic Bayesian network for continuous data, in which parameters are allowed to vary among segments, and in which a common network structure provides essential information sharing across segments. [sent-8, score-0.471]

5 In particular, dynamic Bayesian networks (DBNs) have been applied, as they allow feedback loops and recurrent regulatory structures to be modelled while avoiding the ambiguity about edge directions common to static Bayesian networks. [sent-12, score-0.364]

6 However, regulatory interactions and signal transduction processes in the cell are usually adaptive and change in response to external stimuli. [sent-14, score-0.246]

7 Talih and Hengartner [4] proposed a time-varying Gaussian graphical model (GGM), in which the time-varying variance structure of the data was inferred with reversible jump (RJ) Markov chain Monte Carlo (MCMC). [sent-18, score-0.152]

8 A limitation of this approach is that changes of the network structure between different segments are restricted to changing at most a single edge, and the total number of segments is assumed known a priori. [sent-19, score-0.413]

9 (2008) Marginal Likelihood whole network Yes Ko et al. [sent-22, score-0.172]

10 (2007) BIC Continuous Change-point process Discrete Change-point process Continuous Change-point process Continuous Free allocation Continuous Free allocation node speciﬁc Yes Table 1: Overview of how our model compares with various related, recently published models. [sent-23, score-0.28]

11 The inference algorithm iterates between a convex optimization for determining the graph structure and a dynamic programming algorithm for calculating the segmentation. [sent-25, score-0.127]

12 Moreover, both the models of [4] and [5] are based on undirected graphs, whereas most processes in systems biology, like neural information ﬂow, signal transduction and transcriptional regulation, are intrinsically of a directed nature. [sent-27, score-0.152]

13 Both methods allow for different network structures in different segments of the time series, where the location of the change-points and the total number of segments are inferred from the data with RJMCMC. [sent-29, score-0.458]

14 Allowing the network structure to change between segments leads to a highly ﬂexible model. [sent-32, score-0.311]

15 Owing to the high costs of postgenomic high-throughput experiments, time series in systems biology are typically rather short. [sent-35, score-0.192]

16 Modelling short time series segments with separate network structures will almost inevitably lead to inﬂated inference uncertainty, which calls for some information sharing between the segments. [sent-36, score-0.464]

17 The conceptual problem is related to the very premise of a ﬂexible network structure. [sent-37, score-0.205]

18 This assumption is reasonable for some scenarios, like morphogenesis, where the different segments are e. [sent-38, score-0.102]

19 However, for most cellular processes on a shorter time scale, it is questionable whether it is the structure rather than just the strength of the regulatory interactions that changes with time. [sent-41, score-0.26]

20 To use the analogy of the trafﬁc ﬂow network invoked in [6]: it is not the road system (the network structure) that changes between off-peak and rush hours, but the intensity of the trafﬁc ﬂow (the strength of the interactions). [sent-42, score-0.344]

21 In the same vein, it is not the ability of a transcription factor to potentially bind to the promoter of a gene and thereby initiate transcription (the interaction structure), but the extent to which this happens (the interaction strength). [sent-43, score-0.174]

22 The objective of the present work is to propose and assess a non-stationary continuous-valued DBN that introduces information sharing among different time series segments via a constrained structure. [sent-44, score-0.256]

23 Our model is non-stationary with respect to the parameters, while the network structure is kept ﬁxed among segments. [sent-45, score-0.209]

24 We replace the free allocation model of [11] by a change-point process to incorporate our prior notion that adjacent time points in a time series are likely to be governed by similar distributions. [sent-49, score-0.308]

25 The objective of inference is to infer the 1 Note that as opposed to [7], [6] partially addresses this issue via a prior distribution that discourages changes in the network structure. [sent-52, score-0.172]

26 1 The dynamic BGe network DBNs are ﬂexible models for representing probabilistic relationships between interacting variables (nodes) X1 , . [sent-56, score-0.228]

27 An edge pointing from Xi to Xj indicates that the realization of Xj at time point t, symbolically: Xj (t), is conditionally dependent on the realization of Xi at time point t−1, symbolically: Xi (t−1). [sent-60, score-0.132]

28 The parent node set of node Xn in G, πn = πn (G), is the set of all nodes from which an edge points to node Xn in G. [sent-61, score-0.192]

29 (4) is effectively a mixture model with local probability distributions P (Xn |πn , θ k ) and it can hence, under a free allocation of the latent variables, approxn imate any probability distribution arbitrarily closely. [sent-82, score-0.248]

30 In the present work, we change the assignment of data points to mixture components from a free allocation to a change-point process. [sent-83, score-0.195]

31 However, formulating our method in terms of the BGe score is advantageous when adapting the proposed framework to non-linear static Bayesian networks along the lines of [12]. [sent-87, score-0.124]

32 (4), the marginal likelihood conditional on the latent variables V is given by N P (D|G, V, K)= Kn π Ψ(Dnn [k, Vn ], G) P (D|G, V, K, θ)P (θ)dθ = (5) n=1 k=1 m P Xn (t) = Dn,t |πn (t − 1) = D(πn ,t−1) , θ k n π Ψ(Dnn [k, Vn ], G)= δVn (t),k P (θ k |G)dθ k (6) n n t=2 π Eq. [sent-90, score-0.11]

33 Hence when the regularity conditions deﬁned in [10] are satisﬁed, then the expression in Eq. [sent-93, score-0.114]

34 (24) in [10] restricted to the subset of the data that has been assigned to the kth mixture component (or kth segment). [sent-95, score-0.133]

35 For node Xn the observation at time point t is assigned to the kth component, symbolically Vn (t) = k, if bn,k−1 ≤ t < bn,k . [sent-104, score-0.247]

36 The evennumbered order statistics prior on the change-point locations bn induces a prior distribution on the node-speciﬁc allocation vectors Vn . [sent-114, score-0.147]

37 Note that this approach is equivalent to the idea underlying the allocation sampler proposed in [13]. [sent-125, score-0.108]

38 The resulting algorithm is effectively an RJMCMC scheme [15] in the discrete space of network structures and latent allocation vectors, where the Jacobian in the acceptance criterion is always 1 and can be omitted. [sent-126, score-0.369]

39 5 we perform a structure MCMC move on the current graph G i and leave the latent variable matrix and the numbers of mixture components unchanged, symbolically: Vi+1 = Vi and Ki+1 = Ki . [sent-128, score-0.222]

40 (10) in [16] is computationally less efﬁcient than when applied to static Bayesian networks or stationary DBNs, since the local scores would have to be re-computed every time the positions of the change-points change. [sent-131, score-0.137]

41 Panel (d) shows a protein signal transduction network studied in [2], with an added feedback loop on the root node. [sent-134, score-0.284]

42 The graph is left unchanged, symbolically G i+1 := G i , if the move is not accepted. [sent-138, score-0.176]

43 With the complementary probability 1 − pG we leave the graph G i unchanged and perform a move i i i on (Vi , Ki ), where Vn is the latent variable vector of Xn in Vi , and Ki = (K1 , . [sent-139, score-0.171]

44 We i randomly select a node Xn and change its current number of components Kn via a change-point i birth or death move, or its latent variable vector Vn by a change-point re-allocation move. [sent-143, score-0.202]

45 i i The change-point reallocation move leaves Kn unchanged and may have an effect on Vn . [sent-145, score-0.118]

46 The structures in Figure panels 1a-c constitute elementary network motifs, as studied e. [sent-150, score-0.276]

47 The network in Figure 1d was extracted from the systems biology literature [2] and represents a well-studied protein signal transduction pathway. [sent-153, score-0.313]

48 We added an extra feedback loop on the root node to allow the generation of a Markov chain with non-zero autocorrelation; note that this modiﬁcation is not biologically implausible [21]. [sent-154, score-0.147]

49 For example, the network in Figure 1c was modelled as 2π + cW · φW (t) m Z(t + 1) = cX · X(t) + cY · Y (t) + ·sin(W (t)) + cZ · φZ (t + 1) (10) X(t + 1) = φX (t); Y (t + 1) = φY (t); W (t + 1) = W (t) + where the φ. [sent-157, score-0.172]

50 For each parameter conﬁguration, 25 time series with 41 time points where independently generated. [sent-165, score-0.166]

51 Each symbol shows a comparison of two average AUC scores, averaged over 25 (panels ac) or 5 (panel d) time series independently generated for a given SNR/ACT setting. [sent-242, score-0.12]

52 Note that for these models the parameters can be integrated out analytically, and only the network structure has to be learned. [sent-249, score-0.209]

53 Our comparative evaluation also included two non-linear/non-stationary models with a clearly deﬁned network structure (for the sake of comparability with our approach). [sent-252, score-0.252]

54 We replaced the authors’ free allocation model by the change-point process used for our model. [sent-260, score-0.142]

55 To assess the network reconstruction accuracy, various criteria have been proposed in the literature. [sent-266, score-0.172]

56 In the present study, we chose receiver-operator-characteristic (ROC) curves computed from the marginal posterior probabilities of the edges (and the ranking thereby induced). [sent-267, score-0.105]

57 Owing to the large number of simulations – for each network and parameter setting the simulations were repeated on 25 (Figures 2a-c) or 5 (Figures 2d) independently generated time series – we summarized the performance by the area under the curve (AUC), ranging between 0. [sent-268, score-0.292]

58 3 0 0 10 20 40 30 40 5 20 10 20 30 40 0 0 10 20 30 40 40 20 0 0 20 5 5 5 20 40 40 Figure 3: Results on the Arabidopsis gene expression time series. [sent-278, score-0.219]

59 Top panels: Average posterior probability of a change-point (vertical axis) at a speciﬁc transition time plotted against the transition time (horizontal axis) for two selected circadian genes (left: LHY, centre: TOC1) and averaged over all 9 genes (right). [sent-279, score-0.858]

60 The vertical dotted lines indicate the boundaries of the time series segments, which are related to different entrainment conditions and time intervals. [sent-280, score-0.244]

61 Bottom left and centre panels: Co-allocation matrices for the two selected genes LHY and TOC1. [sent-281, score-0.262]

62 The grey shading indicates the posterior probability of two time points being assigned to the same mixture component, ranging from 0 (black) to 1 (white). [sent-283, score-0.147]

63 Bottom right panel: Predicted regulatory network of nine circadian genes in Arabidopsis thaliana. [sent-284, score-0.766]

64 Edges indicate predicted interactions with a marginal posterior probability greater than 0. [sent-287, score-0.105]

65 investigation of how the signal-to-noise ratio and the autocorrelation parameters effect the relative performance of the methods has to be relegated to the supplementary material for lack of space. [sent-289, score-0.162]

66 4 Results on Arabidopsis gene expression time series We have applied our method to microarray gene expression time series related to the study of circadian regulation in plants. [sent-290, score-0.874]

67 We combined four time series, which differed with respect to the pre-experiment entrainment condition and the time intervals: Te ∈ {10h, 12h, 14h}, and τ ∈ {2h, 4h}. [sent-294, score-0.17]

68 We focused our analysis on 9 circadian genes6 (i. [sent-296, score-0.194]

69 We combined all four time series into a single set. [sent-299, score-0.12]

70 Since the gene expression values at the ﬁrst time point of a time series segment have no relation with the expression values at the last time point of the preceding segment, the corresponding boundary time points were appropriately removed from the data7 . [sent-301, score-0.504]

71 This ensures that for all pairs of consecutive time points a proper conditional dependence relation determined by the nature of the regulatory cellular processes is given. [sent-302, score-0.223]

72 The top panel of Figure 3 shows the marginal posterior 5 We used RMA rather than GCRMA for reasons discussed in [26]. [sent-303, score-0.151]

73 These 9 circadian genes are LHY, TOC1, CCA1, ELF4, ELF3, GI, PRR9, PRR5, and PRR3. [sent-304, score-0.456]

74 6 7 probability of a change-point for two selected genes (LHY and TOC1), and averaged over all genes. [sent-306, score-0.262]

75 This deviation indicates that the two genes are effected by the changing experimental conditions (entrainment, time interval) in different ways and thus provides a useful tool for further exploratory analysis. [sent-310, score-0.347]

76 The bottom right panel of Figure 3 shows the gene interaction network that is predicted when keeping all edges with marginal posterior probability above 0. [sent-311, score-0.46]

77 Empty circles in the ﬁgure represent morning genes (i. [sent-314, score-0.415]

78 genes whose expression peaks in the morning), shaded circles represent evening genes (i. [sent-316, score-0.806]

79 There are several directed edges pointing from the group of morning genes to the evening genes, mostly originating from gene CCA1. [sent-319, score-0.73]

80 This result is consistent with the ﬁndings in [29], where the morning genes were found to activate the evening genes, with CCA1 being a central regulator. [sent-320, score-0.59]

81 Our reconstructed network also contains edges pointing into the opposite direction, from the evening genes back to the morning genes. [sent-321, score-0.853]

82 This ﬁnding is also consistent with [29], where the evening genes were found to inhibit the morning genes via a negative feedback loop. [sent-322, score-0.895]

83 In the reconstructed network, the connectivity within the group of evening genes is sparser than within the group of morning genes. [sent-323, score-0.641]

84 This ﬁnding is consistent with the fact that following the light-dark cycle entrainment, the experiments were carried out in constant-light condition, resulting in a higher activity of the morning genes overall. [sent-324, score-0.415]

85 Within the group of evening genes, the reconstructed network contains an edge between GI and TOC1. [sent-325, score-0.398]

86 We have argued that a ﬂexible network structure can lead to practical and conceptual problems, and we therefore only allow the parameters to vary with time. [sent-330, score-0.242]

87 We have presented a comparative evaluation of the network reconstruction accuracy on synthetic data. [sent-331, score-0.215]

88 Note that such a study is missing from recent related studies on this topic, like [6] and [7], presumably because their overall network structure is not properly deﬁned. [sent-332, score-0.209]

89 The application of our model to gene expression time series from circadian clock-regulated genes in Arabidopsis thaliana has led to a plausible data segmentation, and the reconstructed network shows features that are consistent with the biological literature. [sent-334, score-1.011]

90 This scheme provides the approximation of a non-linear regulation process by a piecewise linear process under the assumption that the temporal processes are sufﬁciently smooth. [sent-336, score-0.133]

91 A straightforward modiﬁcation would be the replacement of the change-point process by the allocation model of [13] and [11]. [sent-337, score-0.108]

92 It would also provide a non-linear Bayesian network for static rather than time series data. [sent-339, score-0.33]

93 The development of more effective proposal moves, as well as a comparison with alternative non-linear Bayesian network models, like [31], is a promising subject for future research. [sent-341, score-0.172]

94 Modelling non-stationary gene regulatory processes with a non-homogeneous Bayesian network and the allocation sampler. [sent-425, score-0.557]

95 Bayesian ﬁnite mixtures with an unknown number of components: The allocation sampler. [sent-439, score-0.108]

96 Network motifs in the transcriptional regulation network of Escherichia coli. [sent-474, score-0.361]

97 Comparative analysis of microarray normalization procedures: effects on reverse engineering gene networks. [sent-528, score-0.1]

98 Flowering locus C mediates natural variation in the high-temperature response of the Arabidopsis circadian clock. [sent-548, score-0.194]

99 The diurnal project: Diurnal and circadian expression proﬁling, model-based pattern matching and promoter analysis. [sent-566, score-0.306]

100 Extension of a genetic network model by iterative experimentation and mathematical analysis. [sent-587, score-0.205]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('kn', 0.319), ('vn', 0.312), ('bge', 0.291), ('genes', 0.262), ('circadian', 0.194), ('bde', 0.175), ('cpbge', 0.175), ('evening', 0.175), ('network', 0.172), ('dnn', 0.155), ('morning', 0.153), ('regulatory', 0.138), ('arabidopsis', 0.117), ('grzegorczyk', 0.117), ('ko', 0.117), ('allocation', 0.108), ('segments', 0.102), ('xn', 0.1), ('gene', 0.1), ('lhy', 0.097), ('symbolically', 0.097), ('dbns', 0.097), ('regulation', 0.094), ('mcmc', 0.087), ('entrainment', 0.078), ('series', 0.074), ('expression', 0.073), ('biology', 0.072), ('bayesian', 0.071), ('transduction', 0.069), ('panels', 0.068), ('ki', 0.068), ('autocorrelation', 0.068), ('node', 0.064), ('husmeier', 0.058), ('relegated', 0.058), ('marginal', 0.057), ('dynamic', 0.056), ('ow', 0.055), ('latent', 0.053), ('networks', 0.053), ('owing', 0.053), ('mixture', 0.053), ('reconstructed', 0.051), ('motifs', 0.051), ('hartemink', 0.051), ('robinson', 0.051), ('posterior', 0.048), ('dirk', 0.047), ('panel', 0.046), ('time', 0.046), ('move', 0.045), ('exible', 0.044), ('bic', 0.044), ('plant', 0.044), ('transcriptional', 0.044), ('birth', 0.044), ('feedback', 0.043), ('comparative', 0.043), ('vi', 0.043), ('death', 0.041), ('stationarity', 0.041), ('reversible', 0.041), ('regularity', 0.041), ('chain', 0.04), ('kth', 0.04), ('pointing', 0.04), ('processes', 0.039), ('unchanged', 0.039), ('diurnal', 0.039), ('dortmund', 0.039), ('effected', 0.039), ('evennumbered', 0.039), ('locke', 0.039), ('netlab', 0.039), ('seedlings', 0.039), ('thaliana', 0.039), ('static', 0.038), ('structure', 0.037), ('interaction', 0.037), ('yes', 0.036), ('supplementary', 0.036), ('structures', 0.036), ('competing', 0.035), ('axis', 0.035), ('free', 0.034), ('peaks', 0.034), ('jump', 0.034), ('cw', 0.034), ('satoru', 0.034), ('reallocation', 0.034), ('cz', 0.034), ('talih', 0.034), ('bioinformatics', 0.034), ('sharing', 0.034), ('graph', 0.034), ('genetic', 0.033), ('conceptual', 0.033), ('score', 0.033), ('auc', 0.032)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.99999881 168 nips-2009-Non-stationary continuous dynamic Bayesian networks

Author: Marco Grzegorczyk, Dirk Husmeier

2 0.29318106 246 nips-2009-Time-Varying Dynamic Bayesian Networks

Author: Le Song, Mladen Kolar, Eric P. Xing

Abstract: Directed graphical models such as Bayesian networks are a favored formalism for modeling the dependency structures in complex multivariate systems such as those encountered in biology and neural science. When a system is undergoing dynamic transformation, temporally rewiring networks are needed for capturing the dynamic causal inﬂuences between covariates. In this paper, we propose time-varying dynamic Bayesian networks (TV-DBN) for modeling the structurally varying directed dependency structures underlying non-stationary biological/neural time series. This is a challenging problem due the non-stationarity and sample scarcity of time series data. We present a kernel reweighted 1 -regularized auto-regressive procedure for this problem which enjoys nice properties such as computational efﬁciency and provable asymptotic consistency. To our knowledge, this is the ﬁrst practical and statistically sound method for structure learning of TVDBNs. We applied TV-DBNs to time series measurements during yeast cell cycle and brain response to visual stimuli. In both cases, TV-DBNs reveal interesting dynamics underlying the respective biological systems. 1

3 0.09731634 97 nips-2009-Free energy score space

Author: Alessandro Perina, Marco Cristani, Umberto Castellani, Vittorio Murino, Nebojsa Jojic

Abstract: A score function induced by a generative model of the data can provide a feature vector of a ﬁxed dimension for each data sample. Data samples themselves may be of differing lengths (e.g., speech segments, or other sequence data), but as a score function is based on the properties of the data generation process, it produces a ﬁxed-length vector in a highly informative space, typically referred to as a “score space”. Discriminative classiﬁers have been shown to achieve higher performance in appropriately chosen score spaces than is achievable by either the corresponding generative likelihood-based classiﬁers, or the discriminative classiﬁers using standard feature extractors. In this paper, we present a novel score space that exploits the free energy associated with a generative model. The resulting free energy score space (FESS) takes into account latent structure of the data at various levels, and can be trivially shown to lead to classiﬁcation performance that at least matches the performance of the free energy classiﬁer based on the same generative model, and the same factorization of the posterior. We also show that in several typical vision and computational biology applications the classiﬁers optimized in FESS outperform the corresponding pure generative approaches, as well as a number of previous approaches to combining discriminating and generative models.

4 0.085169397 257 nips-2009-White Functionals for Anomaly Detection in Dynamical Systems

Author: Marco Cuturi, Jean-philippe Vert, Alexandre D'aspremont

Abstract: We propose new methodologies to detect anomalies in discrete-time processes taking values in a probability space. These methods are based on the inference of functionals whose evaluations on successive states visited by the process are stationary and have low autocorrelations. Deviations from this behavior are used to ﬂag anomalies. The candidate functionals are estimated in a subspace of a reproducing kernel Hilbert space associated with the original probability space considered. We provide experimental results on simulated datasets which show that these techniques compare favorably with other algorithms.

5 0.082543567 13 nips-2009-A Neural Implementation of the Kalman Filter

Author: Robert Wilson, Leif Finkel

Abstract: Recent experimental evidence suggests that the brain is capable of approximating Bayesian inference in the face of noisy input stimuli. Despite this progress, the neural underpinnings of this computation are still poorly understood. In this paper we focus on the Bayesian ﬁltering of stochastic time series and introduce a novel neural network, derived from a line attractor architecture, whose dynamics map directly onto those of the Kalman ﬁlter in the limit of small prediction error. When the prediction error is large we show that the network responds robustly to changepoints in a way that is qualitatively compatible with the optimal Bayesian model. The model suggests ways in which probability distributions are encoded in the brain and makes a number of testable experimental predictions. 1

6 0.082316376 225 nips-2009-Sparsistent Learning of Varying-coefficient Models with Structural Changes

7 0.076952979 23 nips-2009-Accelerating Bayesian Structural Inference for Non-Decomposable Gaussian Graphical Models

8 0.07628379 217 nips-2009-Sharing Features among Dynamical Systems with Beta Processes

9 0.07382293 40 nips-2009-Bayesian Nonparametric Models on Decomposable Graphs

10 0.071490437 72 nips-2009-Distribution Matching for Transduction

11 0.07112591 18 nips-2009-A Stochastic approximation method for inference in probabilistic graphical models

12 0.068379164 51 nips-2009-Clustering sequence sets for motif discovery

13 0.064887248 42 nips-2009-Bayesian Sparse Factor Models and DAGs Inference and Comparison

14 0.057831343 255 nips-2009-Variational Inference for the Nested Chinese Restaurant Process

15 0.05773855 131 nips-2009-Learning from Neighboring Strokes: Combining Appearance and Context for Multi-Domain Sketch Recognition

16 0.057718318 100 nips-2009-Gaussian process regression with Student-t likelihood

17 0.055832587 188 nips-2009-Perceptual Multistability as Markov Chain Monte Carlo Inference

18 0.053004764 174 nips-2009-Nonparametric Latent Feature Models for Link Prediction

19 0.051637273 207 nips-2009-Robust Nonparametric Regression with Metric-Space Valued Output

20 0.047932766 224 nips-2009-Sparse and Locally Constant Gaussian Graphical Models

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, -0.189), (1, -0.049), (2, 0.016), (3, -0.024), (4, 0.048), (5, -0.077), (6, 0.103), (7, 0.01), (8, -0.081), (9, -0.128), (10, 0.014), (11, -0.017), (12, -0.04), (13, -0.02), (14, -0.004), (15, 0.023), (16, -0.025), (17, -0.119), (18, -0.089), (19, -0.035), (20, 0.064), (21, 0.082), (22, 0.021), (23, 0.008), (24, 0.018), (25, 0.184), (26, 0.034), (27, 0.245), (28, 0.033), (29, -0.172), (30, 0.078), (31, 0.055), (32, 0.092), (33, -0.047), (34, -0.144), (35, 0.03), (36, -0.048), (37, -0.141), (38, 0.085), (39, -0.044), (40, -0.203), (41, -0.032), (42, -0.061), (43, 0.052), (44, 0.13), (45, -0.009), (46, 0.107), (47, -0.21), (48, 0.064), (49, -0.069)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.94577533 168 nips-2009-Non-stationary continuous dynamic Bayesian networks

Author: Marco Grzegorczyk, Dirk Husmeier

2 0.85352165 246 nips-2009-Time-Varying Dynamic Bayesian Networks

Author: Le Song, Mladen Kolar, Eric P. Xing

3 0.57580352 51 nips-2009-Clustering sequence sets for motif discovery

Author: Jong K. Kim, Seungjin Choi

Abstract: Most of existing methods for DNA motif discovery consider only a single set of sequences to ﬁnd an over-represented motif. In contrast, we consider multiple sets of sequences where we group sets associated with the same motif into a cluster, assuming that each set involves a single motif. Clustering sets of sequences yields clusters of coherent motifs, improving signal-to-noise ratio or enabling us to identify multiple motifs. We present a probabilistic model for DNA motif discovery where we identify multiple motifs through searching for patterns which are shared across multiple sets of sequences. Our model infers cluster-indicating latent variables and learns motifs simultaneously, where these two tasks interact with each other. We show that our model can handle various motif discovery problems, depending on how to construct multiple sets of sequences. Experiments on three different problems for discovering DNA motifs emphasize the useful behavior and conﬁrm the substantial gains over existing methods where only a single set of sequences is considered.

4 0.54016107 13 nips-2009-A Neural Implementation of the Kalman Filter

Author: Robert Wilson, Leif Finkel

5 0.46380249 225 nips-2009-Sparsistent Learning of Varying-coefficient Models with Structural Changes

Author: Mladen Kolar, Le Song, Eric P. Xing

Abstract: To estimate the changing structure of a varying-coefﬁcient varying-structure (VCVS) model remains an important and open problem in dynamic system modelling, which includes learning trajectories of stock prices, or uncovering the topology of an evolving gene network. In this paper, we investigate sparsistent learning of a sub-family of this model — piecewise constant VCVS models. We analyze two main issues in this problem: inferring time points where structural changes occur and estimating model structure (i.e., model selection) on each of the constant segments. We propose a two-stage adaptive procedure, which ﬁrst identiﬁes jump points of structural changes and then identiﬁes relevant covariates to a response on each of the segments. We provide an asymptotic analysis of the procedure, showing that with the increasing sample size, number of structural changes, and number of variables, the true model can be consistently selected. We demonstrate the performance of the method on synthetic data and apply it to the brain computer interface dataset. We also consider how this applies to structure estimation of time-varying probabilistic graphical models. 1

6 0.41746739 23 nips-2009-Accelerating Bayesian Structural Inference for Non-Decomposable Gaussian Graphical Models

7 0.4026666 97 nips-2009-Free energy score space

8 0.40076596 42 nips-2009-Bayesian Sparse Factor Models and DAGs Inference and Comparison

9 0.39934093 39 nips-2009-Bayesian Belief Polarization

10 0.38233376 197 nips-2009-Randomized Pruning: Efficiently Calculating Expectations in Large Dynamic Programs

11 0.37769136 56 nips-2009-Conditional Neural Fields

12 0.37299362 203 nips-2009-Replacing supervised classification learning by Slow Feature Analysis in spiking neural networks

13 0.35998183 150 nips-2009-Maximum likelihood trajectories for continuous-time Markov chains

14 0.35085452 224 nips-2009-Sparse and Locally Constant Gaussian Graphical Models

15 0.33943632 108 nips-2009-Heterogeneous multitask learning with joint sparsity constraints

16 0.33651194 40 nips-2009-Bayesian Nonparametric Models on Decomposable Graphs

17 0.32719022 155 nips-2009-Modelling Relational Data using Bayesian Clustered Tensor Factorization

18 0.31988779 188 nips-2009-Perceptual Multistability as Markov Chain Monte Carlo Inference

19 0.31597704 75 nips-2009-Efficient Large-Scale Distributed Training of Conditional Maximum Entropy Models

20 0.31584126 62 nips-2009-Correlation Coefficients are Insufficient for Analyzing Spike Count Dependencies

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(7, 0.023), (21, 0.066), (24, 0.026), (25, 0.08), (35, 0.073), (36, 0.07), (39, 0.049), (54, 0.249), (55, 0.011), (58, 0.091), (61, 0.018), (71, 0.062), (81, 0.031), (86, 0.055), (91, 0.017)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.78577167 168 nips-2009-Non-stationary continuous dynamic Bayesian networks

Author: Marco Grzegorczyk, Dirk Husmeier

2 0.74881768 189 nips-2009-Periodic Step Size Adaptation for Single Pass On-line Learning

Author: Chun-nan Hsu, Yu-ming Chang, Hanshen Huang, Yuh-jye Lee

Abstract: It has been established that the second-order stochastic gradient descent (2SGD) method can potentially achieve generalization performance as well as empirical optimum in a single pass (i.e., epoch) through the training examples. However, 2SGD requires computing the inverse of the Hessian matrix of the loss function, which is prohibitively expensive. This paper presents Periodic Step-size Adaptation (PSA), which approximates the Jacobian matrix of the mapping function and explores a linear relation between the Jacobian and Hessian to approximate the Hessian periodically and achieve near-optimal results in experiments on a wide variety of models and tasks. 1

3 0.71556032 36 nips-2009-Asymptotic Analysis of MAP Estimation via the Replica Method and Compressed Sensing

Author: Sundeep Rangan, Vivek Goyal, Alyson K. Fletcher

Abstract: The replica method is a non-rigorous but widely-accepted technique from statistical physics used in the asymptotic analysis of large, random, nonlinear problems. This paper applies the replica method to non-Gaussian maximum a posteriori (MAP) estimation. It is shown that with random linear measurements and Gaussian noise, the asymptotic behavior of the MAP estimate of an n-dimensional vector “decouples” as n scalar MAP estimators. The result is a counterpart to Guo and Verd´ ’s replica analysis of minimum mean-squared error estimation. u The replica MAP analysis can be readily applied to many estimators used in compressed sensing, including basis pursuit, lasso, linear estimation with thresholding, and zero norm-regularized estimation. In the case of lasso estimation the scalar estimator reduces to a soft-thresholding operator, and for zero normregularized estimation it reduces to a hard-threshold. Among other beneﬁts, the replica method provides a computationally-tractable method for exactly computing various performance metrics including mean-squared error and sparsity pattern recovery probability.

4 0.63340998 45 nips-2009-Beyond Convexity: Online Submodular Minimization

Author: Elad Hazan, Satyen Kale

Abstract: We consider an online decision problem over a discrete space in which the loss function is submodular. We give algorithms which are computationally efﬁcient and are Hannan-consistent in both the full information and bandit settings. 1

5 0.60231137 111 nips-2009-Hierarchical Modeling of Local Image Features through $L p$-Nested Symmetric Distributions

Author: Matthias Bethge, Eero P. Simoncelli, Fabian H. Sinz

Abstract: We introduce a new family of distributions, called Lp -nested symmetric distributions, whose densities are expressed in terms of a hierarchical cascade of Lp norms. This class generalizes the family of spherically and Lp -spherically symmetric distributions which have recently been successfully used for natural image modeling. Similar to those distributions it allows for a nonlinear mechanism to reduce the dependencies between its variables. With suitable choices of the parameters and norms, this family includes the Independent Subspace Analysis (ISA) model as a special case, which has been proposed as a means of deriving ﬁlters that mimic complex cells found in mammalian primary visual cortex. Lp -nested distributions are relatively easy to estimate and allow us to explore the variety of models between ISA and the Lp -spherically symmetric models. By ﬁtting the generalized Lp -nested model to 8 × 8 image patches, we show that the subspaces obtained from ISA are in fact more dependent than the individual ﬁlter coefﬁcients within a subspace. When ﬁrst applying contrast gain control as preprocessing, however, there are no dependencies left that could be exploited by ISA. This suggests that complex cell modeling can only be useful for redundancy reduction in larger image patches. 1

6 0.594347 246 nips-2009-Time-Varying Dynamic Bayesian Networks

7 0.58864802 31 nips-2009-An LP View of the M-best MAP problem

8 0.58182001 28 nips-2009-An Additive Latent Feature Model for Transparent Object Recognition

9 0.57274401 162 nips-2009-Neural Implementation of Hierarchical Bayesian Inference by Importance Sampling

10 0.57112366 113 nips-2009-Improving Existing Fault Recovery Policies

11 0.5693745 19 nips-2009-A joint maximum-entropy model for binary neural population patterns and continuous signals

12 0.56909525 241 nips-2009-The 'tree-dependent components' of natural scenes are edge filters

13 0.569089 131 nips-2009-Learning from Neighboring Strokes: Combining Appearance and Context for Multi-Domain Sketch Recognition

14 0.56775337 158 nips-2009-Multi-Label Prediction via Sparse Infinite CCA

15 0.56445932 155 nips-2009-Modelling Relational Data using Bayesian Clustered Tensor Factorization

16 0.56296301 254 nips-2009-Variational Gaussian-process factor analysis for modeling spatio-temporal data

17 0.56191695 174 nips-2009-Nonparametric Latent Feature Models for Link Prediction

18 0.56091541 215 nips-2009-Sensitivity analysis in HMMs with application to likelihood maximization

19 0.56040573 231 nips-2009-Statistical Models of Linear and Nonlinear Contextual Interactions in Early Visual Processing

20 0.56009132 62 nips-2009-Correlation Coefficients are Insufficient for Analyzing Spike Count Dependencies