nips nips2013 nips2013-67 nips2013-67-reference knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Eunho Yang, Pradeep Ravikumar, Genevera I. Allen, Zhandong Liu
Abstract: Conditional random fields, which model the distribution of a multivariate response conditioned on a set of covariates using undirected graphs, are widely used in a variety of multivariate prediction applications. Popular instances of this class of models, such as categorical-discrete CRFs, Ising CRFs, and conditional Gaussian based CRFs, are not well suited to the varied types of response variables in many applications, including count-valued responses. We thus introduce a novel subclass of CRFs, derived by imposing node-wise conditional distributions of response variables conditioned on the rest of the responses and the covariates as arising from univariate exponential families. This allows us to derive novel multivariate CRFs given any univariate exponential distribution, including the Poisson, negative binomial, and exponential distributions. Also in particular, it addresses the common CRF problem of specifying “feature” functions determining the interactions between response variables and covariates. We develop a class of tractable penalized M -estimators to learn these CRF distributions from data, as well as a unified sparsistency analysis for this general class of CRFs showing exact structure recovery can be achieved with high probability. 1
[1] M. J. Wainwright and M. I. Jordan. Graphical models, exponential families and variational inference. Foundations and Trends in Machine Learning, 1(1–2):1—305, December 2008.
[2] J. Cheng, E. Levina, P. Wang, and J. Zhu. arXiv:1209.6419, 2012. Sparse ising models with covariates. Arxiv preprint
[3] S. Ding, G. Wahba, and X. Zhu. Learning Higher-Order Graph Structure with Features by Structure Penalty. In NIPS, 2011.
[4] T. Cai, H. Li, W. Liu, and J. Xie. Covariate adjusted precision matrix estimation with an application in genetical genomics. Biometrika, 2011.
[5] S. Kim and E. P. Xing. Statistical estimation of correlated genome associations to a quantitative trait network. PLoS Genetics, 2009.
[6] H. Liu, X. Chen, J. Lafferty, and L. Wasserman. Graph-valued regression. In NIPS, 2010.
[7] J. Yin and H. Li. A sparse conditional gaussian graphical model for analysis of genetical genomics data. Annals of Applied Statistics, 5(4):2630–2650, 2011.
[8] X. Yuan and T. Zhang. Partial gaussian graphical model estimation. Arxiv preprint arXiv:1209.6419, 2012.
[9] E. Yang, P. Ravikumar, G. I. Allen, and Z. Liu. Graphical models via generalized linear models. In Neur. Info. Proc. Sys., 25, 2012.
[10] E. Yang, P. Ravikumar, G. I. Allen, and Z. Liu. On graphical models via univariate exponential family distributions. Arxiv preprint arXiv:1301.4183, 2013.
[11] A. Jalali, P. Ravikumar, V. Vasuki, and S. Sanghavi. On learning discrete graphical models using groupsparse regularization. In Inter. Conf. on AI and Statistics (AISTATS), 14, 2011.
[12] N. Meinshausen and P. B¨ hlmann. High-dimensional graphs and variable selection with the Lasso. Annals u of Statistics, 34:1436–1462, 2006.
[13] P. Ravikumar, M. J. Wainwright, and J. Lafferty. High-dimensional ising model selection using `1 regularized logistic regression. Annals of Statistics, 38(3):1287–1319, 2010.
[14] J. Besag. Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society. Series B (Methodological), 36(2):192–236, 1974.
[15] A. Y. Ng and M. I. Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In Neur. Info. Proc. Sys., 2002.
[16] M. Schmidt, K. Murphy, G. Fung, and R. Rosales. Structure learning in random fields for heart motion abnormality detection. In Computer Vision and Pattern Recognition (CVPR), pages 1–8, 2008.
[17] A. Torralba, K. P. Murphy, and W. T. Freeman. Contextual models for object detection using boosted random fields. In NIPS, 2004.
[18] J. K. Bradley and C. Guestrin. Learning tree conditional random fields. In ICML, 2010.
[19] D. Shahaf, A. Chechetka, and C. Guestrin. Learning thin junction trees via graph cuts. In AISTATS, 2009.
[20] Cancer Genome Atlas Research Network. Comprehensive genomic characterization defines human glioblastoma genes and core pathways. Nature, 455(7216):1061–1068, October 2008.
[21] H. Liu, K. Roeder, and L. Wasserman. Stability approach to regularization selection (stars) for high dimensional graphical models. Arxiv preprint arXiv:1006.3316, 2010.
[22] J. Yang, S. A. Mani, J. L. Donaher, S. Ramaswamy, R. A. Itzykson, C. Come, P. Savagner, I. Gitelman, A. Richardson, and R. A. Weinberg. Twist, a master regulator of morphogenesis, plays an essential role in tumor metastasis. Cell, 117(7):927–939, 2004.
[23] S. A. Mikheeva, A. M. Mikheev, A. Petit, R. Beyer, R. G. Oxford, L. Khorasani, J.-P. Maxwell, C. A. Glackin, H. Wakimoto, I. Gonz´ lez-Herrero, et al. Twist1 promotes invasion through mesenchymal a change in human glioblastoma. Mol Cancer, 9:194, 2010.
[24] M. A. Smit, T. R. Geiger, J.-Y. Song, I. Gitelman, and D. S. Peeper. A twist-snail axis critical for trkbinduced epithelial-mesenchymal transition-like transformation, anoikis resistance, and metastasis. Molecular and cellular biology, 29(13):3722–3737, 2009.
[25] M. J. Wainwright. Sharp thresholds for high-dimensional and noisy sparsity recovery using `1 -constrained quadratic programming (Lasso). IEEE Trans. Information Theory, 55:2183–2202, May 2009.
[26] S. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu. A unified framework for high-dimensional analysis of m-estimators with decomposable regularizers. Arxiv preprint arXiv:1010.2731, 2010. 9