nips nips2012 nips2012-317 knowledge-graph by maker-knowledge-mining

317 nips-2012-Smooth-projected Neighborhood Pursuit for High-dimensional Nonparanormal Graph Estimation

Source: pdf

Author: Tuo Zhao, Kathryn Roeder, Han Liu

Abstract: We introduce a new learning algorithm, named smooth-projected neighborhood pursuit, for estimating high dimensional undirected graphs. In particularly, we focus on the nonparanormal graphical model and provide theoretical guarantees for graph estimation consistency. In addition to new computational and theoretical analysis, we also provide an alternative view to analyze the tradeoff between computational efﬁciency and statistical error under a smoothing optimization framework. Numerical results on both synthetic and real datasets are provided to support our theory. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 In particularly, we focus on the nonparanormal graphical model and provide theoretical guarantees for graph estimation consistency. [sent-2, score-0.739]

2 In addition to new computational and theoretical analysis, we also provide an alternative view to analyze the tradeoff between computational efﬁciency and statistical error under a smoothing optimization framework. [sent-3, score-0.202]

3 1 Introduction We consider the undirected graph estimation problem for a d-dimensional random vector X = (X1 , . [sent-5, score-0.213]

4 Most existing methods for high dimensional graph estimation assume that the random vector X follows a Gaussian distribution, i. [sent-16, score-0.213]

5 Under this parametric assumption, the graph estimation problem can be solved by estimating the sparsity pattern of the precision matrix Ω = Σ−1 , i. [sent-19, score-0.203]

6 There are two major approaches for learning high dimensional Gaussian graphical models: (i) graphical lasso (Yuan and Lin, 2007; Friedman et al. [sent-23, score-0.335]

7 , 2008) and (ii) neighborhood pursuit (Meinshausen and B¨ hlmann, 2006). [sent-25, score-0.572]

8 The graphical lasso maximizes the 1 -penalized Gaussian likelihood and u simultaneously estimates the precision matrix Ω and graph G. [sent-26, score-0.259]

9 In contrast, the neighborhood pursuit method maximizes the 1 -penalized pseudo-likelihood and can only estimate the graph G. [sent-27, score-0.696]

10 Theoretically, both methods are consistent in graph recovery for Gaussian models under certain regularity conditions. [sent-31, score-0.161]

11 (2011) suspect that the neighborhood pursuit approach has a better sample complexity in graph recovery than the graphical lasso. [sent-33, score-0.82]

12 (2009), an semiparametric nonparanormal model is proposed to relax the restrictive normality assumption. [sent-36, score-0.563]

13 (2009) show that for the nonparanormal distribution, the graph G can also be estimated by examining the sparsity pattern of Ω. [sent-44, score-0.637]

14 Different methods have been proposed to infer the nonparanormal model in high dimensions. [sent-45, score-0.481]

15 (2012), a rank-based estimator named nonparanormal SKEPTIC is proposed to directly estimate Ω. [sent-47, score-0.6]

16 Their main idea is to calculate a rank-correlation matrix (either based on the Spearman’s rho or Kendall’s tau correlation) and plug the estimated correlation matrix into the graphical lasso to estimate Ω and graph G. [sent-48, score-0.466]

17 Such a procedure has been proven to be robust and achieve the same parametric rates of convergence as the graphical lasso (Liu et al. [sent-49, score-0.206]

18 However, how to combine the nonparanormal SKEPTIC estimator with the neighborhood pursuit approach is still an open problem. [sent-51, score-1.138]

19 In this paper, we bridge this gap by proposing a novel smooth-projected neighborhood pursuit method. [sent-54, score-0.572]

20 The main idea is to project the possibly indeﬁnite nonparanormal SKEPTIC correlation matrix estimator into the cone of all positive semi-deﬁnite matrices with respect to a smoothed elementwise ∞ -norm. [sent-55, score-0.837]

21 Such a projection step is closely related to the dual smoothing approach in Nesterov (2005). [sent-56, score-0.157]

22 Computationally, our proposed smoothed elementwise ∞ -norm has nice structure so that we can √ develop an efﬁcient fast proximal gradient solver with a provable convergence rate O(1/ ) ( is the desired accuracy of the objective value, Nesterov (1988)). [sent-58, score-0.351]

23 Theoretically, we provide sufﬁcient conditions to guarantee that the proposed smooth-projected neighborhood pursuit approach is graph estimation consistent. [sent-59, score-0.769]

24 In addition to new computational and statistical analysis, we further provide an alternative view to analyze the fundamental tradeoff between computational efﬁciency and statistical error under the smoothing optimization framework. [sent-60, score-0.232]

25 , 2012) considers the dual smoothing approach as a tradeoff between computational efﬁciency and approximation error. [sent-62, score-0.144]

26 However, we directly consider the statistical error introduced by the smoothing approach, and show that the obtained estimator preserves the good statistical properties without losing the computational efﬁciency. [sent-65, score-0.314]

27 The rest of this paper is organized as follows: The next section reviews the nonparanormal SKEPTIC in Liu et al. [sent-67, score-0.552]

28 (2012); Section 3 introduces the smooth-projected neighborhood pursuit and derives the fast proximal gradient algorithm; Section 4 explores the statistical properties of the procedure; Section 5 and 6 present results on on both simulated and real datasets. [sent-68, score-0.713]

29 , vd )T ∈ Rd , we deﬁne the vector norms: 2 2 d×d ||v||1 = and j vj , and ||v||∞ = maxj |vi |. [sent-77, score-0.151]

30 Let A = [Ajk ] ∈ R j |vj |, ||v||2 = B = [Bjk ] ∈ Rd×d be two symmetric matrices, we deﬁne the matrix operator norms: ||A||1 = maxk j |Ajk |, ||A||∞ = maxj k |Ajk |, ||A||2 = max||v||2 =1 ||Av||2 and elementwise norms |||A|||1 = j,k |Ajk |, |||A|||∞ = maxj,k |Ajk |, ||A||2 = j,k |Ajk |2 . [sent-78, score-0.166]

31 The nonparanormal (nonparametric normal) distribution was initially motivated by the sparse additive models (Ravikumar et al. [sent-91, score-0.552]

32 , fd } be a collection of non-decreasing univariate functions and Σ∗ ∈ Rd×d be a correlation matrix with diag(Σ∗ ) = 1. [sent-100, score-0.096]

33 , Xd )T follows a nonparanormal distribution, denoted by X ∼ N P Nd (f, Σ∗ ), if f (X) = (f1 (X1 ), . [sent-104, score-0.481]

34 1) The nonparanormal family is equivalent to the Gaussian copula family for continuous distributions (Klaassen and Wellner, 1997; Tsukahara, 2005; Liu et al. [sent-109, score-0.61]

35 Similar to the Gaussian graphical model, the nonparanormal graphical model also encodes the conditional independence graph by the sparsity pattern of the precision matrix Ω∗ = (Σ∗ )−1 . [sent-111, score-0.811]

36 (2012) propose a rank-based procedure, named nonparanormal SKEPTIC, for learning nonparanromal graphical models. [sent-115, score-0.602]

37 , xi )T 1 d be n independent observations of X, we deﬁne the Spearman’s rho and Kendall’s tau correlation coefﬁcients as n i i ¯ ¯ i=1 (rj − rj )(rk − rk ) Spearman’s rho : ρjk = , (2. [sent-122, score-0.308]

38 2) n n i i ¯ 2 ¯ 2 i=1 (rj − rj ) · i=1 (rk − rk ) Kendall’s tau : τjk = 2 n(n − 1) sign xi − xi j j xi − xi , k k (2. [sent-123, score-0.16]

39 The smoothed elementwise ∞ -norm is a smooth convex function. [sent-136, score-0.174]

40 7) S we can also obtain a good correlation estimator without losing computational efﬁciency. [sent-144, score-0.225]

41 7) has a minimum eigenvalue constraint, regarding which, we exploit Nesterov (1988) and derive the following fast proximal gradient algorithm. [sent-147, score-0.111]

42 The main idea is to utilize the gradients in previous iterations to help ﬁnd the descent direction for the current iteration, and eventually achieves a faster convergence rate than the ordinary projected gradient algorithm. [sent-148, score-0.102]

43 Suppose A has the eigenvalue decomposition as A = j=1 σj vj vj , where σj ’s are the eigenvalues and vj ’s are corresponding eigenvectors. [sent-160, score-0.279]

44 2, the fast proximal gradient algorithm takes Q(W, W(t−1) , µ) = |||S − W(t−1) |||µ + G(t) , W − W(t−1) + ∞ W(t) = argmin Q(W, W(t−1) , µ) = Π+ W(t−1) − W 0 µ (t) G , θt (3. [sent-174, score-0.111]

45 14) The detailed proof can be found in the extended draft Zhao et al. [sent-182, score-0.112]

46 In next section, by directly analyzing the tradeoff between the computational efﬁciency and statistical error, we will show that choosing a suitable smooth parameter µ allows S concentrate to Σ∗ with a rate similar to Lemma 2. [sent-187, score-0.157]

47 Due to space limit, all the proofs of following theorems can be found in the extended draft Zhao et al. [sent-192, score-0.112]

48 The next theorem establishes the concentration property of S under the elementwise ∞ norm. [sent-194, score-0.11]

49 Given the nonparanormal SKEPTIC estimator S, for any large enough n, under the conditions that µ ≤ 4πϕ and ϕ > 0, we have the optimum to (3. [sent-198, score-0.592]

50 It implies that we can gain the computational efﬁciency without losing statistical rate in terms of elementwise sup-norm as long as µ is reasonably large. [sent-203, score-0.292]

51 We now show that our proposed smooth-projected neighborhood approach recovers the true neighborhood for each node with high probability under the following irrepresentable condition (Zhao and Yu, 2006; Zou, 2006; Wainwright, 2009). [sent-204, score-0.515]

52 Recall that Ij and Jj denote the true neighborhood and non-neighborhood of node j respectively. [sent-206, score-0.24]

53 3) The proposed projection approach can be also combined with other graph estimation method such as Zhou et al. [sent-214, score-0.316]

54 It guarantees that for each individual node, we can correctly recover its neighborhood with high probability. [sent-227, score-0.24]

55 Consequently, the following corollary can be implied so that we can asymptotically recover the underlying graph structure under given conditions. [sent-228, score-0.124]

56 (2012) recommend to use the Kendall’s tau for nonparanormal graph estimation because of its superior robustness property compared to the Spearman’s rho. [sent-237, score-0.733]

57 In this section, we use the Kendall’s tau in our smooth-projected neighborhood pursuit method. [sent-238, score-0.653]

58 We adopt the power function g(t) = sign(t)|t|4 to convert the Gaussian data to the nonparanormal data. [sent-241, score-0.481]

59 We use the ROC curve to evaluate the graph estimation performance. [sent-244, score-0.171]

60 Since d > n, the full solution paths cannot be obtained, therefore we restrict the range of false positive edge discovery rates to be from 0 to 0. [sent-245, score-0.119]

61 Nonparanormal SKEPTIC Estimator We ﬁrst evaluate the proposed smoothed elementwise ∞ -norm projection algorithm. [sent-249, score-0.248]

62 We study the empirical performance of the proposed fast proximal gradient algorithm using different smoothing parameters (µ = 1, 0. [sent-251, score-0.194]

63 The optimization and statistical error curves for different smoothing parameters (averaged over 50 replications) are presented in Figure 1. [sent-255, score-0.147]

64 We further compare the graph recovery performance of our proposed method with the naive indeﬁnite nonparanormal SKEPTIC estimator as in Liu et al. [sent-265, score-0.894]

65 The averaged ROC curves over 100 replications are presented in Figure 2. [sent-267, score-0.115]

66 We see that directly plugging the indeﬁnite nonparanormal SKEPTIC estimator into the neighborhood pursuit results in the worst performance. [sent-268, score-1.138]

67 0 our smoothed-projected neighborhood pursuit method signiﬁcantly outperforms the naive indeﬁnite nonparanormal SKEPTIC estimator. [sent-327, score-1.149]

68 4 False Positive Rate (c) Chain (d) Scale-free Figure 2: The averaged ROC curves of the neighborhood pursuit when combined with different correlation estimators. [sent-336, score-0.684]

69 “SKEPTIC” represents the indeﬁnite nonparanormal SKEPTIC estimator, and “Projection” represents our proposed projection approach. [sent-337, score-0.555]

70 (2012) to compare our proposed method with the naive neighborhood pursuit method. [sent-386, score-0.668]

71 The naive neighborhood pursuit directly exploits the Pearson correlation estimator under the neighborhood pursuit framework. [sent-387, score-1.379]

72 The averaged ROC curves over 100 replications are presented in Figure 3. [sent-389, score-0.115]

73 As can be seen, our proposed projection method outperforms the naive neighborhood pursuit throughout all four different graphs. [sent-390, score-0.742]

74 30 False Positive Rate (d) Scale-free Figure 3: The averaged ROC curves of the neighborhood pursuit when combined with different correlation estimators. [sent-398, score-0.684]

75 6 Real Data Analysis In this section we present a real data experiment to compare the nonparanormal graphical model to Gaussian graphical model. [sent-401, score-0.655]

76 For model selection, we use the stability graph procedure (Meinshausen 7 and B¨ hlmann, 2010; Liu et al. [sent-402, score-0.222]

77 The topic graph is ﬁrst used in Blei and Lafferty (2007) to illustrate the idea of correlated topic modeling. [sent-404, score-0.286]

78 Blei and Lafferty (2007) assume that the topic proportion approximately follows a normal distribution after the logarithmic-transformation. [sent-407, score-0.108]

79 Here we are interested in visualizing the relationship among the topics using an undirected graph: the nodes represent individual topics, and edges connecting different nodes represent highly related topics. [sent-408, score-0.151]

80 Evaluated by the Kolmogorov-Smirnov test, we ﬁnd many topic data highly violate the normality assumption (More details can be found in Zhao et al. [sent-411, score-0.183]

81 This motivates our choice of the smooth-projected neighborhood pursuit approach. [sent-413, score-0.572]

82 The smooth-projected neighborhood pursuit generates 6 mid-size modules and 6 small modules, while the naive neighborhood pursuit generated 1 large module, 2 mid-size modules and 6 small modules. [sent-415, score-1.382]

83 The nonparanormal approach discovers more reﬁned structures and improves the interpretability of the obtained graph. [sent-416, score-0.481]

84 ; (3) Topics closely related to modern physics are clustered in the same module such as “quantum-29”, “magnetic-55”, ”pressure-92”, “solar-62”, etc. [sent-418, score-0.101]

85 In contrast, the naive neighborhood pursuit mixes all these different topics in a large module. [sent-420, score-0.723]

86 (a) Our Proposed Method (b) Naive Neighborhood Pursuit Figure 4: Two topic graphs illustrating the difference of the estimated topic graphs. [sent-421, score-0.162]

87 The smoothprojected neighborhood pursuit (subﬁgure (a)) generates 6 mid-size modules and 6 small modules while the naive neighborhood pursuit (subﬁgure (b)) generates 1 large module, 2 mid-size modules and 6 small modules. [sent-422, score-1.453]

88 7 Conclusion and Acknowledgement In this paper, we study how to estimate the nonparanormal graph using the neighborhood pursuit in conjunction with the possible indeﬁnite nonparanormal skeptic estimator. [sent-423, score-2.052]

89 Using our proposed smoothed projection approach, the resulting estimator can be used as a positive semi-deﬁnite reﬁnement of the nonparanormal skeptic estimator. [sent-424, score-1.141]

90 Our estimator has better graph estimation performance with theoretical guarantee. [sent-425, score-0.256]

91 Our results suggest that it is possible to gain estimation robustness and modeling ﬂexibility without losing two important computational structures: convexity and smoothness. [sent-426, score-0.133]

92 A smoothing proximal gradient method for general structured sparse regression. [sent-447, score-0.194]

93 Efﬁcient estimation in the bivariate normal copula model: Normal margins are least-favorable. [sent-472, score-0.132]

94 The nonparanormal: Semiparametric estimation of high dimensional undirected graphs. [sent-491, score-0.131]

95 Stability approach to regularization selection for high dimensional graphical models. [sent-497, score-0.129]

96 Sparse graphical gaussian modeling of the isoprenoid gene network in arabidopsis thaliana. [sent-556, score-0.122]

97 Model selection and estimation in the gaussian graphical model. [sent-561, score-0.169]

98 The huge package for highdimensional undirected graph estimation in r. [sent-574, score-0.239]

99 A smoothing projection approach for high dimensional nonparanormal graph estimation. [sent-580, score-0.804]

100 Adaptive lasso for high dimensional regression and gaussian graphical modeling. [sent-588, score-0.212]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('nonparanormal', 0.481), ('skeptic', 0.394), ('pursuit', 0.332), ('neighborhood', 0.24), ('ajk', 0.142), ('nnp', 0.139), ('zhao', 0.135), ('inde', 0.124), ('graph', 0.124), ('iu', 0.123), ('afferty', 0.116), ('snp', 0.113), ('elementwise', 0.11), ('naive', 0.096), ('liu', 0.095), ('vj', 0.093), ('uhlmann', 0.093), ('roeder', 0.089), ('graphical', 0.087), ('estimator', 0.085), ('smoothing', 0.083), ('wasserman', 0.081), ('tau', 0.081), ('topic', 0.081), ('false', 0.076), ('roc', 0.075), ('proximal', 0.075), ('projection', 0.074), ('modules', 0.071), ('et', 0.071), ('module', 0.071), ('kendall', 0.071), ('rate', 0.066), ('smoothed', 0.064), ('spearman', 0.06), ('losing', 0.058), ('copula', 0.058), ('xd', 0.058), ('replications', 0.057), ('hao', 0.057), ('topics', 0.055), ('ij', 0.055), ('correlation', 0.054), ('nesterov', 0.051), ('semiparametric', 0.051), ('erd', 0.051), ('lasso', 0.048), ('blei', 0.047), ('rho', 0.047), ('estimation', 0.047), ('avikumar', 0.046), ('einshausen', 0.046), ('esterov', 0.046), ('kathryn', 0.046), ('tuo', 0.046), ('uan', 0.046), ('lafferty', 0.046), ('jj', 0.045), ('nyi', 0.045), ('rj', 0.045), ('positive', 0.043), ('dimensional', 0.042), ('fd', 0.042), ('undirected', 0.042), ('sj', 0.041), ('draft', 0.041), ('recovery', 0.037), ('jk', 0.036), ('gradient', 0.036), ('irrepresentable', 0.035), ('gaussian', 0.035), ('named', 0.034), ('curves', 0.034), ('hlmann', 0.034), ('meinshausen', 0.034), ('rk', 0.034), ('tradeoff', 0.033), ('sparsity', 0.032), ('ciency', 0.031), ('normality', 0.031), ('maxj', 0.03), ('statistical', 0.03), ('clustered', 0.03), ('computational', 0.028), ('wainwright', 0.028), ('vd', 0.028), ('normal', 0.027), ('stability', 0.027), ('nodes', 0.027), ('triplet', 0.027), ('pearson', 0.027), ('hopkins', 0.027), ('conditions', 0.026), ('johns', 0.026), ('norms', 0.026), ('highdimensional', 0.026), ('calculate', 0.025), ('annals', 0.025), ('rd', 0.025), ('averaged', 0.024)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0 317 nips-2012-Smooth-projected Neighborhood Pursuit for High-dimensional Nonparanormal Graph Estimation

Author: Tuo Zhao, Kathryn Roeder, Han Liu

2 0.25271073 352 nips-2012-Transelliptical Graphical Models

Author: Han Liu, Fang Han, Cun-hui Zhang

Abstract: We advocate the use of a new distribution family—the transelliptical—for robust inference of high dimensional graphical models. The transelliptical family is an extension of the nonparanormal family proposed by Liu et al. (2009). Just as the nonparanormal extends the normal by transforming the variables using univariate functions, the transelliptical extends the elliptical family in the same way. We propose a nonparametric rank-based regularization estimator which achieves the parametric rates of convergence for both graph recovery and parameter estimation. Such a result suggests that the extra robustness and ﬂexibility obtained by the semiparametric transelliptical modeling incurs almost no efﬁciency loss. We also discuss the relationship between this work with the transelliptical component analysis proposed by Han and Liu (2012). 1

3 0.13286375 351 nips-2012-Transelliptical Component Analysis

Author: Fang Han, Han Liu

Abstract: We propose a high dimensional semiparametric scale-invariant principle component analysis, named TCA, by utilize the natural connection between the elliptical distribution family and the principal component analysis. Elliptical distribution family includes many well-known multivariate distributions like multivariate Gaussian, t and logistic and it is extended to the meta-elliptical by Fang et.al (2002) using the copula techniques. In this paper we extend the meta-elliptical distribution family to a even larger family, called transelliptical. We prove that TCA can obtain a near-optimal s log d/n estimation consistency rate in recovering the leading eigenvector of the latent generalized correlation matrix under the transelliptical distribution family, even if the distributions are very heavy-tailed, have inﬁnite second moments, do not have densities and possess arbitrarily continuous marginal distributions. A feature selection result with explicit rate is also provided. TCA is further implemented in both numerical simulations and largescale stock data to illustrate its empirical usefulness. Both theories and experiments conﬁrm that TCA can achieve model ﬂexibility, estimation accuracy and robustness at almost no cost. 1

4 0.12967953 326 nips-2012-Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses

Author: Po-ling Loh, Martin J. Wainwright

Abstract: We investigate a curious relationship between the structure of a discrete graphical model and the support of the inverse of a generalized covariance matrix. We show that for certain graph structures, the support of the inverse covariance matrix of indicator variables on the vertices of a graph reﬂects the conditional independence structure of the graph. Our work extends results that have previously been established only in the context of multivariate Gaussian graphical models, thereby addressing an open question about the signiﬁcance of the inverse covariance matrix of a non-Gaussian distribution. Based on our population-level results, we show how the graphical Lasso may be used to recover the edge structure of certain classes of discrete graphical models, and present simulations to verify our theoretical results. 1

5 0.1092509 147 nips-2012-Graphical Models via Generalized Linear Models

Author: Eunho Yang, Genevera Allen, Zhandong Liu, Pradeep K. Ravikumar

Abstract: Undirected graphical models, also known as Markov networks, enjoy popularity in a variety of applications. The popular instances of these models such as Gaussian Markov Random Fields (GMRFs), Ising models, and multinomial discrete models, however do not capture the characteristics of data in many settings. We introduce a new class of graphical models based on generalized linear models (GLMs) by assuming that node-wise conditional distributions arise from exponential families. Our models allow one to estimate multivariate Markov networks given any univariate exponential distribution, such as Poisson, negative binomial, and exponential, by ﬁtting penalized GLMs to select the neighborhood for each node. A major contribution of this paper is the rigorous statistical analysis showing that with high probability, the neighborhood of our graphical models can be recovered exactly. We also provide examples of non-Gaussian high-throughput genomic networks learned via our GLM graphical models. 1

6 0.095151052 6 nips-2012-A Convex Formulation for Learning Scale-Free Networks via Submodular Relaxation

7 0.090259075 248 nips-2012-Nonparanormal Belief Propagation (NPNBP)

8 0.08845605 164 nips-2012-Iterative Thresholding Algorithm for Sparse Inverse Covariance Estimation

9 0.082924612 172 nips-2012-Latent Graphical Model Selection: Efficient Methods for Locally Tree-like Graphs

10 0.080456898 199 nips-2012-Link Prediction in Graphs with Autoregressive Features

11 0.070916295 282 nips-2012-Proximal Newton-type methods for convex optimization

12 0.070709422 180 nips-2012-Learning Mixtures of Tree Graphical Models

13 0.068153732 186 nips-2012-Learning as MAP Inference in Discrete Graphical Models

14 0.067272201 117 nips-2012-Ensemble weighted kernel estimators for multivariate entropy estimation

15 0.066613935 211 nips-2012-Meta-Gaussian Information Bottleneck

16 0.066338859 92 nips-2012-Deep Representations and Codes for Image Auto-Annotation

17 0.066298008 346 nips-2012-Topology Constraints in Graphical Models

18 0.06166967 335 nips-2012-The Bethe Partition Function of Log-supermodular Graphical Models

19 0.061300814 134 nips-2012-Finite Sample Convergence Rates of Zero-Order Stochastic Optimization Methods

20 0.060282733 69 nips-2012-Clustering Sparse Graphs

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.173), (1, 0.071), (2, 0.078), (3, -0.088), (4, -0.039), (5, 0.038), (6, 0.018), (7, -0.099), (8, -0.053), (9, 0.018), (10, 0.013), (11, -0.007), (12, 0.012), (13, 0.032), (14, 0.013), (15, -0.011), (16, 0.264), (17, 0.008), (18, -0.072), (19, 0.069), (20, 0.039), (21, -0.151), (22, -0.03), (23, -0.005), (24, -0.012), (25, -0.144), (26, -0.134), (27, -0.004), (28, 0.024), (29, -0.015), (30, 0.029), (31, -0.025), (32, -0.011), (33, -0.042), (34, -0.043), (35, -0.004), (36, 0.003), (37, 0.052), (38, -0.0), (39, -0.016), (40, 0.045), (41, 0.023), (42, -0.002), (43, 0.02), (44, -0.008), (45, -0.037), (46, -0.033), (47, 0.074), (48, -0.005), (49, 0.047)]

similar papers list:

simIndex simValue paperId paperTitle

1 0.91839331 352 nips-2012-Transelliptical Graphical Models

Author: Han Liu, Fang Han, Cun-hui Zhang

same-paper 2 0.91772783 317 nips-2012-Smooth-projected Neighborhood Pursuit for High-dimensional Nonparanormal Graph Estimation

Author: Tuo Zhao, Kathryn Roeder, Han Liu

3 0.84767997 351 nips-2012-Transelliptical Component Analysis

Author: Fang Han, Han Liu

4 0.66615468 211 nips-2012-Meta-Gaussian Information Bottleneck

Author: Melanie Rey, Volker Roth

Abstract: We present a reformulation of the information bottleneck (IB) problem in terms of copula, using the equivalence between mutual information and negative copula entropy. Focusing on the Gaussian copula we extend the analytical IB solution available for the multivariate Gaussian case to distributions with a Gaussian dependence structure but arbitrary marginal densities, also called meta-Gaussian distributions. This opens new possibles applications of IB to continuous data and provides a solution more robust to outliers. 1

5 0.58534789 326 nips-2012-Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses

Author: Po-ling Loh, Martin J. Wainwright

6 0.52767718 147 nips-2012-Graphical Models via Generalized Linear Models

7 0.47266078 192 nips-2012-Learning the Dependency Structure of Latent Factors

8 0.46521157 309 nips-2012-Semi-supervised Eigenvectors for Locally-biased Learning

9 0.45773706 346 nips-2012-Topology Constraints in Graphical Models

10 0.45735344 248 nips-2012-Nonparanormal Belief Propagation (NPNBP)

11 0.45701545 327 nips-2012-Structured Learning of Gaussian Graphical Models

12 0.45294619 199 nips-2012-Link Prediction in Graphs with Autoregressive Features

13 0.45085135 172 nips-2012-Latent Graphical Model Selection: Efficient Methods for Locally Tree-like Graphs

14 0.42475227 164 nips-2012-Iterative Thresholding Algorithm for Sparse Inverse Covariance Estimation

15 0.41653824 132 nips-2012-Fiedler Random Fields: A Large-Scale Spectral Approach to Statistical Network Modeling

16 0.41520518 287 nips-2012-Random function priors for exchangeable arrays with applications to graphs and relational data

17 0.41297472 254 nips-2012-On the Sample Complexity of Robust PCA

18 0.40818828 337 nips-2012-The Lovász ϑ function, SVMs and finding large dense subgraphs

19 0.40756619 6 nips-2012-A Convex Formulation for Learning Scale-Free Networks via Submodular Relaxation

20 0.4026885 139 nips-2012-Fused sparsity and robust estimation for linear models with unknown variance

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.06), (21, 0.019), (24, 0.29), (38, 0.117), (39, 0.052), (42, 0.045), (54, 0.022), (55, 0.012), (74, 0.045), (76, 0.158), (80, 0.059), (92, 0.04)]

similar papers list:

simIndex simValue paperId paperTitle

1 0.8962487 310 nips-2012-Semiparametric Principal Component Analysis

Author: Fang Han, Han Liu

Abstract: We propose two new principal component analysis methods in this paper utilizing a semiparametric model. The according methods are named Copula Component Analysis (COCA) and Copula PCA. The semiparametric model assumes that, after unspeciﬁed marginally monotone transformations, the distributions are multivariate Gaussian. The COCA and Copula PCA accordingly estimate the leading eigenvectors of the correlation and covariance matrices of the latent Gaussian distribution. The robust nonparametric rank-based correlation coefﬁcient estimator, Spearman’s rho, is exploited in estimation. We prove that, under suitable conditions, although the marginal distributions can be arbitrarily continuous, the COCA and Copula PCA estimators obtain fast estimation rates and are feature selection consistent in the setting where the dimension is nearly exponentially large relative to the sample size. Careful numerical experiments on the synthetic and real data are conducted to back up the theoretical results. We also discuss the relationship with the transelliptical component analysis proposed by Han and Liu (2012). 1

same-paper 2 0.75922847 317 nips-2012-Smooth-projected Neighborhood Pursuit for High-dimensional Nonparanormal Graph Estimation

Author: Tuo Zhao, Kathryn Roeder, Han Liu

3 0.6657719 355 nips-2012-Truncation-free Online Variational Inference for Bayesian Nonparametric Models

Author: Chong Wang, David M. Blei

Abstract: We present a truncation-free stochastic variational inference algorithm for Bayesian nonparametric models. While traditional variational inference algorithms require truncations for the model or the variational distribution, our method adapts model complexity on the ﬂy. We studied our method with Dirichlet process mixture models and hierarchical Dirichlet process topic models on two large data sets. Our method performs better than previous stochastic variational inference algorithms. 1

4 0.64102107 352 nips-2012-Transelliptical Graphical Models

Author: Han Liu, Fang Han, Cun-hui Zhang

5 0.60560298 166 nips-2012-Joint Modeling of a Matrix with Associated Text via Latent Binary Features

Author: Xianxing Zhang, Lawrence Carin

Abstract: A new methodology is developed for joint analysis of a matrix and accompanying documents, with the documents associated with the matrix rows/columns. The documents are modeled with a focused topic model, inferring interpretable latent binary features for each document. A new matrix decomposition is developed, with latent binary features associated with the rows/columns, and with imposition of a low-rank constraint. The matrix decomposition and topic model are coupled by sharing the latent binary feature vectors associated with each. The model is applied to roll-call data, with the associated documents deﬁned by the legislation. Advantages of the proposed model are demonstrated for prediction of votes on a new piece of legislation, based only on the observed text of legislation. The coupling of the text and legislation is also shown to yield insight into the properties of the matrix decomposition for roll-call data. 1

6 0.60475236 249 nips-2012-Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison

7 0.6044535 354 nips-2012-Truly Nonparametric Online Variational Inference for Hierarchical Dirichlet Processes

8 0.60118616 68 nips-2012-Clustering Aggregation as Maximum-Weight Independent Set

9 0.5993039 294 nips-2012-Repulsive Mixtures

10 0.59849465 316 nips-2012-Small-Variance Asymptotics for Exponential Family Dirichlet Process Mixture Models

11 0.59753144 163 nips-2012-Isotropic Hashing

12 0.59629112 47 nips-2012-Augment-and-Conquer Negative Binomial Processes

13 0.59623796 277 nips-2012-Probabilistic Low-Rank Subspace Clustering

14 0.59617746 221 nips-2012-Multi-Stage Multi-Task Feature Learning

15 0.59601492 69 nips-2012-Clustering Sparse Graphs

16 0.5951367 106 nips-2012-Dynamical And-Or Graph Learning for Object Shape Modeling and Detection

17 0.59510195 111 nips-2012-Efficient Sampling for Bipartite Matching Problems

18 0.59417659 58 nips-2012-Bayesian models for Large-scale Hierarchical Classification

19 0.59413314 335 nips-2012-The Bethe Partition Function of Log-supermodular Graphical Models

20 0.59379435 264 nips-2012-Optimal kernel choice for large-scale two-sample tests