jmlr jmlr2010 jmlr2010-64 jmlr2010-64-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Joshua W. Robinson, Alexander J. Hartemink
Abstract: Learning dynamic Bayesian network structures provides a principled mechanism for identifying conditional dependencies in time-series data. An important assumption of traditional DBN structure learning is that the data are generated by a stationary process, an assumption that is not true in many important settings. In this paper, we introduce a new class of graphical model called a nonstationary dynamic Bayesian network, in which the conditional dependence structure of the underlying data-generation process is permitted to change over time. Non-stationary dynamic Bayesian networks represent a new framework for studying problems in which the structure of a network is evolving over time. Some examples of evolving networks are transcriptional regulatory networks during an organism’s development, neural pathways during learning, and traffic patterns during the day. We define the non-stationary DBN model, present an MCMC sampling algorithm for learning the structure of the model from time-series data under different assumptions, and demonstrate the effectiveness of the algorithm on both simulated and biological data. Keywords: Bayesian networks, graphical models, model selection, structure learning, Monte Carlo methods
Amr Ahmed and Eric P. Xing. TESLA: Recovering time-varying networks of dependencies in social and biological studies. Proceedings of the National Academy of Sciences, 106(29):11878–11883, 2009. Michelle N. Arbeitman, Eileen E.M. Furlong, Farhad Imam, Eric Johnson, Brian H. Null, Bruce S. Baker, Mark A. Krasnow, Matthew P. Scott, Ronald W. Davis, and Kevin P. White. Gene expression during the life cycle of Drosophila melanogaster. Science, 5590(297):2270–2275, Sep 2002. Matthew J. Beal and Zoubin Ghahramani. Variational Bayesian learning of directed graphical models with hidden variables. Bayesian Analysis, 1(4):793–832, 2006. Allister Bernard and Alexander J. Hartemink. Informative structure priors: Joint learning of dynamic regulatory networks from multiple types of data. In Pacific Symposium on Biocomputing, volume 10, pages 459–470. World Scientific, Jan 2005. Wray Buntine. A guide to the literature on learning probabilistic networks from data. IEEE Transactions on Knowledge and Data Engineering, 8(2):195–210, 1996. Carlos M. Carvalho and Mike West. Dynamic matrix-variate graphical models. Bayesian Analysis, 2(1):69–98, 2007. David Maxwell Chickering, Dan Geiger, and David Heckerman. Learning Bayesian networks is NP-Hard. Microsoft Research Technical Report MSR-TR-94-17, Microsoft, Nov 1994. David Maxwell Chickering, Dan Geiger, and David Heckerman. Learning Bayesian networks: Search methods and experimental results. In Proceedings of the 5th International Workshop on Artificial Intelligence and Statistics, pages 112–128. Society for Artificial Intelligence in Statistics, Jan 1995. Lonnie Chrisman. A roadmap to research on Bayesian networks and other decomposable probabilistic models. CMU technical report, School of Computer Science, CMU, May 1998. Lu´s Miguel de Campos, Juan M. Fernandez-Luna, Jos´ Antonio G´ mez, and Jos´ Miguel Puerta. ı e a e Ant colony optimization for learning Bayesian networks. International Journal of Approximate Reasoning, 31(3):291–311, Nov 2002. Stuart J. Elgar, Jun Han, and Michael V. Taylor. mef2 activity levels differentially affect gene expression during Drosophila muscle development. Proceedings of the National Academy of Sciences, 105(3):918–923, Jan 2008. Nir Friedman. Learning belief networks in the presence of missing values and hidden variables. In Proceedings of the 14th International Conference on Machine Learning, pages 125–133. Morgan Kaufmann Publishers, 1997. Nir Friedman and Zohar Yakhini. On the sample complexity of learning Bayesian networks. In Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence, pages 274–282. Morgan Kaufmann Publishers Inc., Oct 1996. 3677 ROBINSON AND H ARTEMINK Nir Friedman, Kevin Murphy, and Stuart Russell. Learning the structure of dynamic probabilistic networks. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI98), pages 139–147. Morgan Kaufmann Publishers Inc., 1998. Nir Friedman, Michal Linial, Iftach Nachman, and Dana Pe’er. Using Bayesian networks to analyze expression data. In Research in Computational Molecular Biology (RECOMB00), volume 4, pages 127–135. ACM Press, Apr 2000. Zoubin Ghahramani and Geoffrey E. Hinton. Variational learning for switching state-space models. Neural Computation, 12(4):963–996, Apr 2000. Paolo Giudici and Robert Castelo. Improving Markov chain Monte Carlo model search for data mining. Machine Learning, 50(1–2), Jan 2003. Paolo Giudici, Peter Green, and Claudia Tarantola. Efficient model determination for discrete graphical models. Technical report, Athens University of Economics and Business, 1999. Marco Grzegorczyk, Dirk Husmeier, Kieron D. Edwards, Peter Ghazal, and Andrew J. Millar. Modelling non-stationary gene regulatory processes with a non-homogeneous Bayesian network and the allocation sampler. Bioinformatics, 24(18):2071–2078, Jul 2008. Fan Guo, Wenjie Fu, Yanxin Shi, and Eric P. Xing. Reverse engineering temporally rewiring gene networks. In NIPS workshop on New Problems and Methods in Computational Biology, Dec 2006. Fan Guo, Steve Hanneke, Wenjie Fu, and Eric P. Xing. Recovering temporally rewiring networks: A model-based approach. In Proceedings of the 24th International Conference on Machine Learning (ICML07), Jun 2007. Steve Hanneke and Eric P. Xing. Discrete temporal models of social networks. In Workshop on Statistical Network Analysis at the 23rd International Conference on Machine Learning, Jun 2006. Alexander J. Hartemink, David K. Gifford, Tommi S. Jaakkola, and Richard A. Young. Using graphical models and genomic expression data to statistically validate models of genetic regulatory networks. In Pacific Symposium on Biocomputing, volume 6, pages 422–433. World Scientific, Jan 2001. David Heckerman, Dan Geiger, and David Maxwell Chickering. Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20(3):197–243, Sep 1995. Reimar Hofmann and Volker Tresp. Discovering structure in continuous variables using Bayesian networks. In Advances in Neural Information Processing Systems 8 (NIPS95), pages 500–506. MIT Press, Dec 1995. Rhea R. Kimpo, Frederic E. Theunissen, and Allison J. Doupe. Propagation of correlated activity through multiple stages of a neural circuit. Journal of Neuroscience, 23(13):5750–5761, 2003. Mladen Kolar, Le Song, Amr Ahmed, and Eric P. Xing. Estimating time-varying networks. The Annals of Applied Statistics, 4(1):94–123, Mar 2010. 3678 L EARNING N ON -S TATIONARY DYNAMIC BAYESIAN N ETWORKS Paul K. Krause. Learning probabilistic networks. The Knowledge Engineering Review, 13(4):321– 351, 1998. Wai Lam and Fahiem Bacchus. Learning Bayesian belief networks: An approach based on the MDL principle. Computational Intelligence, 10(4):269–293, Jul 1994. Pedro Larra˜ aga, Mikel Poza, Yosu Yurramendi, Roberto H. Murga, and Cindy M.H. Kuijpers. n Structure learning of Bayesian networks by genetic algorithms: A performance analysis of control parameters. IEEE Journal on Pattern Analysis and Machine Intelligence, 18(9):912–926, Sep 1996. Nicholas M. Luscombe, M. Madan Babu, Haiyuan Yu, Michael Snyder, Sarah A. Teichmann, and Mark Gerstein. Genomic analysis of regulatory network dynamics reveals large topological changes. Nature, 431:308–312, Sep 2004. David Madigan, Jeremy York, and Denis Allard. Bayesian graphical models for discrete data. International Statistical Review, 63(2):215–232, Aug 1995. Dimitris Margaritis. Distribution-free learning of Bayesian network structure in continuous domains. In Proceedings of the 20th National Conference on Artificial Intelligence (AAAI05), pages 825–830. AAAI Press / The MIT Press, Jul 2005. Kevin Murphy. Learning Bayesian network structure from sparse data sets. UC Berkeley technical report 990, Computer Science Department, University of California at Berkeley, May 2001. Thomas Sandmann, Lars J. Jensen, Janus S. Jakobsen, Michal M. Karzynski, Michael P. Eichenlaub, Peer Bork, and Eileen E.M. Furlong. A temporal map of transcription factor activity: mef2 directly regulates target genes at all stages of muscle development. Developmental Cell, 10(6): 797–807, Jun 2006. V. Anne Smith, Erich D. Jarvis, and Alexander J. Hartemink. Influence of network topology and data collection on network inference. In Pacific Symposium on Biocomputing, volume 8, pages 164–175. World Scientific, Jan 2003. V. Anne Smith, Jing Yu, Tom V. Smulders, Alexander J. Hartemink, and Erich D. Jarvis. Computational inference of neural information flow networks. PLoS Computational Biology, 2(11): 1436–1449, Nov 2006. Joe Suzuki. Learning Bayesian belief networks based on the minimum description length principle: An efficient algorithm using the branch and bound technique. In Proceedings of the 13th International Conference on Machine Learning (ICML96), pages 462–470. Morgan Kaufmann Publishers Inc., Jul 1996. Makram Talih and Nicolas Hengartner. Structural learning with time-varying components: Tracking the cross-section of financial time series. Journal of the Royal Statistical Society B, 67(3):321– 341, Jun 2005. Claudia Tarantola. MCMC model determination for discrete graphical models. Statistical Modelling, 4(1):39–61, Apr 2004. 3679 ROBINSON AND H ARTEMINK Stanley Wasserman and Philippa E. Pattison. Logit models and logistic regressions for social networks: I. An introduction to Markov graphs and p∗. Psychometrika, 61(3):401–425, Sep 1996. Xiang Xuan and Kevin Murphy. Modeling changing dependency structure in multivariate time series. In Proceedings of the 24th International Conference on Machine Learning (ICML07), Jun 2007. Wentao Zhao, Erchin Serpedin, and Edward R. Dougherty. Inferring gene regulatory networks from time series data using the minimum description length principle. Bioinformatics, 22(17): 2129–2135, Sep 2006. 3680