jmlr jmlr2005 jmlr2005-17 jmlr2005-17-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Onno Zoeter, Tom Heskes
Abstract: We study the problem of learning two regimes (we have a normal and a prefault regime in mind) based on a train set of non-Markovian observation sequences. Key to the model is that we assume that once the system switches from the normal to the prefault regime it cannot restore and will eventually result in a fault. We refer to the particular setting as semi-supervised since we assume the only information given to the learner is whether a particular sequence ended with a stop (implying that the sequence was generated by the normal regime) or with a fault (implying that there was a switch from the normal to the fault regime). In the latter case the particular time point at which a switch occurred is not known. The underlying model used is a switching linear dynamical system (SLDS). The constraints in the regime transition probabilities result in an exact inference procedure that scales quadratically with the length of a sequence. Maximum aposteriori (MAP) parameter estimates can be found using an expectation maximization (EM) algorithm with this inference algorithm in the E-step. For long sequences this will not be practically feasible and an approximate inference and an approximate EM procedure is called for. We describe a Ä?Ĺš‚exible class of approximations corresponding to different choices of clusters in a Kikuchi free energy with weak consistency constraints. Keywords: change point problems, switching linear dynamical systems, strong junction trees, approximate inference, expectation propagation, Kikuchi free energies
Y. Bar-Shalom and X.-R. Li. Estimation and Tracking: Principles, Techniques, and Software. Artech House, 1993. A. T. Cemgil, H. J. Kappen, and D. Barber. A generative model for music transcription. Accepted to IEEE Transactions on Speech and Audio Processing, 2004. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, B, 39(1):1–38, 1977. P. Fearnhead. Exact and efÄ?Ĺš cient Bayesian inference for multiple changepoint problems. Technical report, Dept. of Math. and Stat., Lancaster University, 2003. P. J. Harrison and C. F. Stevens. Bayesian forecasting. Journal of the Royal Statistical Society Society B, 38:205–247, 1976. T. Heskes and O. Zoeter. Generalized belief propagation for approximate inference in hybrid Bayesian networks. In C. Bishop and B. Frey, editors, Proceedings of the Ninth International Workshop on ArtiÄ?Ĺš cial Intelligence and Statistics (AISTATS), 2003. Tom Heskes and Onno Zoeter. Expectation propagation for approximate inference in dynamic Bayesian networks. In Proceedings of the 18th Annual Conference on Uncertainty in ArtiÄ?Ĺš cial Intelligence (UAI 2002), San Francisco, CA, 2002. Morgan Kaufmann Publishers. T. Jaakkola. Tutorial on variational approximation methods. In Advanced Mean Field Methods, Theory and Practice. MIT Press, 2001. H. J. Kappen and W. Wiegerinck. Novel iteration schemes for the cluster variation method. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 415–422, Cambridge, MA, 2002. MIT Press. C.-J. Kim. Dynamic linear models with Markov-switching. Journal of Econometrics, 60:1–22, 1994. P. R. Krishnaiah and B. Q. Miao. Review about estimation of change points. In P. R. Krishnaiah and C. R. Rao, editors, Handbook of Statistics, volume 7. Elsevier, 1988. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger. Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47(2):498–519, 2001. Steffen L. Lauritzen. Propagation of probabilities, means, and variances in mixed graphical association models. Journal of the American Statistical Association, 87:1098–1108, 1992. T. Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of the 17th Annual Conference on Uncertainty in ArtiÄ?Ĺš cial Intelligence (UAI 2001). Morgan Kaufmann Publishers, 2001. T. Minka and Y. Qi. Tree-structured approximations by expectation propagation. In Sebastian ¨ Thrun, Lawrence Saul, and Bernhard Scholkopf, editors, Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004. 2025 Z OETER AND H ESKES J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan-Kaufmann, 1988. A. Schillings, B. van Wezel, T. Mulder, and J. Duysens. Mechanically induced stumbling during human treadmill walking. Journal of Neuroscience Methods, 67:11–17, 1996. R. H. Shumway and D. S. Stoffer. Dynamic linear models with switching. Journal of the American Statistical Association, 86:763–769, 1991. M. Welling, Y. W. Teh, and T. Minka. Structured regions graphs: morphing ep intro gbp. In Proceedings of the 21st Annual Conference on Uncertainty in ArtiÄ?Ĺš cial Intelligence (UAI-2005), Corvallis, Oregon, 2005. AUAI Press. M. West and J. Harrison. Bayesian Forecasting and Dynamic Models. Springer, 2nd edition, 1997. J. Whittaker. Graphical Models in Applied Multivariate Statistics. John Wiley & Sons, 1989. J. Yedidia, W. Freeman, and Y. Weiss. Constructing free energy approximations and generalized belief propagation algorithms. IEEE Transactions on Information Theory, 51(7):2282–2312, July 2005. O. Zoeter and T. Heskes. Deterministic approximate inference techniques for conditionally Gaussian state space models. Submitted, 2005. 2026