nips nips2008 nips2008-235 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Piyush Rai, Hal Daume
Abstract: We propose a nonparametric Bayesian factor regression model that accounts for uncertainty in the number of factors, and the relationship between factors. To accomplish this, we propose a sparse variant of the Indian Buffet Process and couple this with a hierarchical model over factors, based on Kingman’s coalescent. We apply this model to two problems (factor analysis and factor regression) in gene-expression data analysis. 1
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract We propose a nonparametric Bayesian factor regression model that accounts for uncertainty in the number of factors, and the relationship between factors. [sent-3, score-0.497]
2 To accomplish this, we propose a sparse variant of the Indian Buffet Process and couple this with a hierarchical model over factors, based on Kingman’s coalescent. [sent-4, score-0.217]
3 We apply this model to two problems (factor analysis and factor regression) in gene-expression data analysis. [sent-5, score-0.311]
4 Factor regression couples this analysis with a prediction task, where the predictions are made solely on the basis of the factor representation. [sent-7, score-0.358]
5 The latent factor representation achieves two-fold benefits: (1) discovering the latent process underlying the data; (2) simpler predictive modeling through a compact data representation. [sent-8, score-0.433]
6 We address three fundamental shortcomings of standard factor analysis approaches [2, 3, 4, 1]: (1) we do not assume a known number of factors; (2) we do not assume factors are independent; (3) we do not assume all features are relevant to the factor analysis. [sent-10, score-0.754]
7 Our contributions thus parallel the needs of gene pathway modeling. [sent-13, score-0.144]
8 In addition, we couple predictive modeling (for factor regression) within the factor analysis framework itself, instead of having to model it separately. [sent-14, score-0.687]
9 In particular, we treat the gene-tofactor relationship nonparametrically by proposing a sparse variant of the Indian Buffet Process (IBP) [5], designed to account for the sparsity of relevant genes (features). [sent-16, score-0.51]
10 We couple this IBP with a hierarchical prior over the factors. [sent-17, score-0.191]
11 This prior explains the fact that pathways are fundamentally related: some are involved in transcription, some in signaling, some in synthesis. [sent-18, score-0.191]
12 The nonparametric nature of our sparse IBP requires that the hierarchical prior also be nonparametric. [sent-19, score-0.305]
13 A natural choice is Kingman’s coalescent [6], a popular distribution over infinite binary trees. [sent-20, score-0.398]
14 In particular, genes are features, samples are examples, and pathways are factors. [sent-22, score-0.364]
15 An alternative application might be to a collaborative filtering problem, in which case our genes might correspond to movies, our samples might correspond to users and our pathways might correspond to genres. [sent-24, score-0.364]
16 In this context, all three contributions of our model still make sense: we do not know how many movie genres there are; some genres are closely related (romance to comedy versus to action); many movies may be spurious. [sent-25, score-0.142]
17 We further use Kingman’s coalescent to model latent pathway hierarchies. [sent-29, score-0.48]
18 1 Indian Buffet Process The Indian Buffet Process [7] defines a distribution over infinite binary matrices, originally motivated by the need to model the latent factor structure of a given set of observations. [sent-31, score-0.403]
19 Thereafter, each incoming customer i selects a previously-selected dish k with a probability mk /(i − 1), where mk is the number of previous customers who have selected dish k. [sent-36, score-0.376]
20 We can easily define a binary matrix Z with value Zik = 1 precisely when customer i selects dish k. [sent-38, score-0.265]
21 2 Kingman’s Coalescent Our model makes use of a latent hierarchical structure over factors; we use Kingman’s coalescent [6] as a convenient prior distribution over hierarchies. [sent-47, score-0.59]
22 Kingman’s coalescent originated in the study of population genetics for a set of single-parent organisms. [sent-48, score-0.353]
23 The coalescent is a nonparametric model over a countable set of organisms. [sent-49, score-0.492]
24 We denote by ti the time at which the ith coalescent event occurs (note ti ≤ 0), and δi = ti−1 − ti the time between events (note δi > 0). [sent-56, score-0.566]
25 With probability one, a random draw 2 from the n-coalescent is a binary tree with a single root at t = −∞ and n individuals at time t = 0. [sent-58, score-0.163]
26 The marginal distribution over tree topologies is uniform and independent of coalescent times; and the model is infinitely exchangeable. [sent-60, score-0.464]
27 If we associate with each node in the tree a mean y and variance v message, we update messages as Eq (1), where i is the current node and li and ri are its children. [sent-72, score-0.259]
28 v i = (v li + (tli − ti )Λ)−1 + (v ri + (tri − ti )Λ)−1 −1 y i = y li (v li + (tli − ti )Λ) −1 (1) −1 −1 + y ri (v ri + (tri − ti )Λ) 2 vi 3 Nonparametric Bayesian Factor Regression Recall the standard factor analysis problem: X = AF + E, for standardized data X. [sent-73, score-0.742]
29 A is the factor loading matrix of size P × K and F = [f 1 , . [sent-78, score-0.45]
30 Recall that our goal is to treat the factor analysis problem nonparametrically, to model feature relevance, and to model hierarchical factors. [sent-87, score-0.415]
31 Next, we propose a variant of IBP to model gene relevance. [sent-90, score-0.164]
32 We then present the hierarchical model for inferring factor hierarchies. [sent-91, score-0.379]
33 We conclude with a presentation of the full model and our mechanism for modifying the factor analysis problem to factor regression. [sent-92, score-0.586]
34 Although IBP has been applied to nonparametric factor analysis in the past [5], the standard IBP formulation places IBP prior on the factor matrix (F) associating samples (i. [sent-95, score-0.788]
35 However, this assumption is inappropriate in the geneexpression context where it is not the factors themselves but the associations among genes and factors (i. [sent-99, score-0.773]
36 In such a context, each sample depends on all the factors but each gene within a sample usually depends only on a small number of factors. [sent-102, score-0.304]
37 Thus, it is more appropriate to model the factor loading matrix (A) with the IBP prior. [sent-103, score-0.486]
38 Note that since A and F are related with each other via the number of factors K, modeling A nonparametrically allows our model to also have an unbounded number of factors. [sent-104, score-0.348]
39 For most gene-expression problems [1], a binary factor loadings matrix (A) is inappropriate. [sent-105, score-0.59]
40 Therefore, we instead use the Hadamard (element-wise) product of a binary matrix Z and a matrix V of reals. [sent-106, score-0.143]
41 The factor analysis model, for each sample i, thus becomes: xi = (Z ⊙ V )f i + ei . [sent-108, score-0.275]
42 Our initial model assumes no factor hierarchies and hence 2 the prior over V would simply be a Gaussian: V ∼ Nor(0, σv I) with an inverse-gamma prior on σv . [sent-111, score-0.515]
43 F has a zero mean, unit variance Gaussian prior, as used in standard factor analysis. [sent-112, score-0.275]
44 Finally, ei = Nor(0, Ψ) models the idiosyncratic variations of genes where Ψ is a P × P diagonal matrix (diag(Ψ1 , . [sent-113, score-0.429]
45 A more realistic model is that certain genes simply do not participate in the factor analysis: for a culinary analogy, the genes enter the restaurant and leave before selecting any dishes. [sent-121, score-0.979]
46 ” We add an additional prior term to account for such spurious genes; effectively leading to a sparse solution (over the rows of the IBP matrix). [sent-123, score-0.241]
47 To see the difference, recall that the IBP contains a “rich get richer” phenomenon: frequently selected factors are more likely to get reselected. [sent-126, score-0.204]
48 Consider a truly spurious gene and ask whether it is likely to select any factors. [sent-127, score-0.207]
49 If some factor k is already frequently used, then a priori this gene is more likely to select it. [sent-128, score-0.375]
50 Our sparse-IBP prior is identical to the standard IBP prior with one exception. [sent-131, score-0.172]
51 To model the fact that factors are, in fact, re3 lated, we introduce a factor hierarchy. [sent-137, score-0.515]
52 Kingman’s coalescent [6] is an attractive prior for integration with IBP for several reasons. [sent-138, score-0.439]
53 The key aspects of this model are: the IBP prior over Z, the sparse binary vector T, and the Coalescent prior over V. [sent-146, score-0.301]
54 In standard Bayesian factor regression [1], factor analysis is followed by the regression task. [sent-147, score-0.716]
55 For example, a simple linear regression problem would involve estimating a K-dimensional parameter vector θ with regression value θ ⊤ F. [sent-149, score-0.166]
56 Our model, on the other hand, integrates factor regression component in the nonparametric factor analysis framework itself. [sent-150, score-0.736]
57 It is straightforward to see that it is equivalent to fitting another sparse model relating factors to responses. [sent-153, score-0.288]
58 Our model thus allows the factor analysis to take into account the regression task as well. [sent-154, score-0.394]
59 In case of binary responses, we add an extra probit regression step to predict binary outcomes from real-valued responses. [sent-155, score-0.173]
60 Sampling the IBP matrix Z: Sampling Z consists of sampling existing dishes, proposing new dishes and accepting or rejecting them based on the acceptance ratio in the associated M-H step. [sent-158, score-0.212]
61 Sampling V new from the coalescent is slightly involved. [sent-165, score-0.353]
62 As shown pictorially in figure 3, proposing a new column of V corresponds to adding a new leaf node to the existing coalescent tree. [sent-166, score-0.436]
63 In particular, we need to find a sibling (s) to the new node y ′ and need to find an insertion point on the branch joining the sibling s to its parent p (the grandparent of y ′ ). [sent-167, score-0.303]
64 Since the marginal distribution over trees under the coalescent is uniform, the sibling s is chosen uniformly over nodes in the tree. [sent-168, score-0.412]
65 We then use importance sampling to select an insertion time for the new node y ′ between ts and tp , according to the exponential distribution given by the coalescent prior (our proposal distribution is uniform). [sent-169, score-0.769]
66 This yields Nor(y 0 , v 0 ), given by: v 0 = [(v s + (ts − t)Λ)−1 + (v p + (t − tp )Λ)−1 ]−1 y 0 = [y s /(vs + (ts − t)Λ) + y p /(vp + (tp − t)Λ)]v 0 Here, ys and vs are the messages passed up through the tree, while yp and vp are the messages passed down through the tree (compare to Eq (1)). [sent-173, score-0.324]
67 Sampling the sparse IBP vector T: In the sparse IBP prior, recall that we have an additional P -many variables Tp , indicating whether gene p “eats” any dishes. [sent-174, score-0.196]
68 Sampling the real valued matrix V: For the case when V has a Gaus- Figure 3: Adding a sian prior on it, we sample V from its posterior p(Vg,j |X, Z, F, Ψ) ∝ new node to the tree 2 N Fj,i 1 Nor(Vg,j |µg,j , Σg,j ), where Σg,j = ( i=1 Ψg + σ2 )−1 and v N ∗ ∗ Σg,j ( i=1 Fj,i Xg,j )Ψ−1 . [sent-180, score-0.256]
69 To get around this, one can perform model selection via Reversible Jump MCMC [10] or evolutionary stochastic model search [11]. [sent-193, score-0.171]
70 A somewhat similar approach to ours is the infinite independent component analysis (iICA) model of [12] which treats factor analysis as a special case of ICA. [sent-197, score-0.311]
71 However, their model is limited to factor analysis and does not take into account feature selection, factor hierarchy and factor regression. [sent-198, score-0.942]
72 BFRM assumes a sparsity inducing mixture prior on the factor loading matrix A. [sent-204, score-0.563]
73 It is interesting to note that the nonparametric prior of our model (factor loading matrix defined as A = Z ⊙ V) is actually equivalent to the (parametric) sparse mixture prior of the BFRM as K → ∞. [sent-208, score-0.534]
74 To see this, note that our prior on the factor loading matrix A (composed of Z having an IBP prior, and V having a Gaussian prior), can be 2 written as Apk ∼ (1 − ρk )δ0 (Apk ) + ρk Nor(Apk |0, σv ), if we define ρk ∼ Bet(1, αβ/K). [sent-209, score-0.536]
75 We compare our nonparametric approach with the evolutionary search based approach proposed in [11], which is the nonparametric extension to BFRM. [sent-212, score-0.275]
76 We used the gene-factor connectivity matrix of E-coli network (described in [14]) to generate a synthetic dataset having 100 samples of 50 genes and 8 underlying factors. [sent-213, score-0.383]
77 Since we knew the ground truth for factor loadings in this case, this dataset was ideal to test for efficacy in recovering the factor loadings (binding sites and number of factors). [sent-214, score-0.992]
78 We also experimented with a real geneexpression data which is a breast cancer dataset having 251 samples of 226 genes and 5 prominent underlying factors (we know this from domain knowledge). [sent-215, score-0.584]
79 1 Nonparametric Gene-Factor Modeling and Variable Selection For the synthetic dataset generated by the E-coli network, the results are shown in figure 4 comparing the actual network used to generate the data and the inferred factor loading matrix. [sent-217, score-0.484]
80 As shown in figure 4, we recovered exactly the same number (8) of factors, and almost exactly the same factor loadings (binding sites and number of factors) as the ground truth. [sent-218, score-0.496]
81 In comparison, the evolutionary search based approach overestimated the number of factors and the inferred loadings clearly seem to be off from the actual loadings (even modulo column permutations). [sent-219, score-0.817]
82 (Right) Inferred factor loadings with the evolutionary search based approach. [sent-221, score-0.565]
83 To see the effect of variable selection for this data, we also introduced spurious genes by adding 50 random features in each sample. [sent-224, score-0.438]
84 We also investigated the effect of noise on the evolutionary search based approach and it resulted in an overestimated number of factor, plus false discovered factor loadings for spurious genes (see figure 5(c)). [sent-226, score-1.056]
85 To conserve space, we do not show here the cases when there are no spurious genes in the data but it turns out that variable selection does not filter out any of 226 relevant genes in such a case. [sent-227, score-0.739]
86 2 Hierarchical Factor Modeling Our results with hierarchical factor modeling are shown in figure 6 for synthetic and real data. [sent-229, score-0.415]
87 From the factor hierarchy for E-coli data (figure 6), we see that column-2 (corresponding to factor-2) of the V matrix is the most prominent one (it regulates the highest number of genes), and is closest to the tree-root, followed by column2, which it looks most similar to. [sent-232, score-0.445]
88 Columns corresponding to lesser prominent factors are located further down in the hierarchy (with appropriate relatedness). [sent-233, score-0.325]
89 The hierarchy can be used to find factors in order of their prominence. [sent-235, score-0.285]
90 The higher we chop off the tree along the hierarchy, the more prominent the factors, we discover, are. [sent-236, score-0.154]
91 For instance, if we are only interested in top 2 factors in E-coli data, we can chop off the tree above the sixth coalescent point. [sent-237, score-0.671]
92 In contrast, our model discovers the factor hierarchies as part of the inference procedure itself. [sent-239, score-0.343]
93 3 that hierarchical modeling results in better predictive performance for the factor regression task. [sent-242, score-0.49]
94 Empirical evidences also suggest that the factor hierarchy leads to faster convergence since most of the unlikely configurations will never be visited as they are constrained by the hierarchy. [sent-243, score-0.356]
95 04 220 50 1 2 3 4 5 6 7 8 (a) 3 5 8 7 4 6 1 1 2 2 3 Factors (b) (c) 4 5 1 2 3 5 4 (d) Figure 6: Hierarchical factor modeling results. [sent-260, score-0.314]
96 5 0 100 Iterations 200 300 400 500 600 700 800 900 1000 Iterations Figure 7: (a) MSE on the breast-cancer data for BFRM (horizontal line), our model with Gaussian (top red curved line) and Coalescent (bottom blue curved line) priors. [sent-307, score-0.148]
97 This MSE is the reconstruction error for the data - different from the MSE for the held-out real valued responses (fig 7 c) (b) Log-likelihoods for our model with Gaussian (bottom red curved line) and Coalescent (top blue curved line) priors. [sent-308, score-0.213]
98 (c) Factor regression results 7 Conclusions and Discussion We have presented a fully nonparametric Bayesian approach to sparse factor regression, modeling the gene-factor relationship using a sparse variant of the IBP. [sent-309, score-0.624]
99 Both gene selection and hierarchical factor modeling are straightforward extensions in our model that do not significantly complicate the inference procedure, but lead to improved model performance and more understandable outputs. [sent-311, score-0.584]
100 We applied Kingman’s coalescent as a hierarhical model on V, the matrix modulating the expression levels of genes in factors. [sent-312, score-0.739]
wordName wordTfidf (topN-words)
[('ibp', 0.441), ('coalescent', 0.353), ('genes', 0.301), ('factor', 0.275), ('loadings', 0.221), ('factors', 0.204), ('bfrm', 0.177), ('apk', 0.155), ('tp', 0.147), ('loading', 0.126), ('kingman', 0.118), ('spurious', 0.107), ('nonparametric', 0.103), ('gene', 0.1), ('indian', 0.098), ('buffet', 0.088), ('prior', 0.086), ('bet', 0.086), ('regression', 0.083), ('hierarchy', 0.081), ('dishes', 0.079), ('idiosyncratic', 0.079), ('tree', 0.075), ('customer', 0.074), ('mse', 0.074), ('ti', 0.071), ('evolutionary', 0.069), ('zik', 0.069), ('nonparametrically', 0.069), ('hierarchical', 0.068), ('regulatory', 0.066), ('dish', 0.063), ('pathways', 0.063), ('gure', 0.062), ('oisson', 0.059), ('sibling', 0.059), ('stu', 0.059), ('insertion', 0.059), ('pk', 0.059), ('curved', 0.056), ('overestimated', 0.052), ('mk', 0.051), ('parent', 0.051), ('inferred', 0.05), ('matrix', 0.049), ('sparse', 0.048), ('latent', 0.047), ('sampling', 0.047), ('transcription', 0.046), ('node', 0.046), ('binary', 0.045), ('pathway', 0.044), ('individuals', 0.043), ('bioinformatics', 0.042), ('agglomerative', 0.042), ('fundamentally', 0.042), ('bayesian', 0.041), ('prominent', 0.04), ('diffusion', 0.04), ('customers', 0.04), ('vp', 0.04), ('chop', 0.039), ('culinary', 0.039), ('daum', 0.039), ('geneexpression', 0.039), ('genres', 0.039), ('imputed', 0.039), ('modeling', 0.039), ('responses', 0.038), ('proposing', 0.037), ('couple', 0.037), ('model', 0.036), ('ri', 0.035), ('tli', 0.034), ('aat', 0.034), ('tri', 0.034), ('selects', 0.034), ('synthetic', 0.033), ('hierarchies', 0.032), ('eq', 0.032), ('lineages', 0.032), ('discovered', 0.031), ('proposal', 0.031), ('messages', 0.031), ('selection', 0.03), ('brownian', 0.029), ('joining', 0.029), ('variant', 0.028), ('movies', 0.028), ('exchangeable', 0.028), ('nite', 0.028), ('reconstruction', 0.027), ('sparsity', 0.027), ('reversible', 0.027), ('restaurant', 0.027), ('av', 0.027), ('li', 0.026), ('binding', 0.025), ('predictive', 0.025), ('associations', 0.025)]
simIndex simValue paperId paperTitle
same-paper 1 1.0000002 235 nips-2008-The Infinite Hierarchical Factor Regression Model
Author: Piyush Rai, Hal Daume
Abstract: We propose a nonparametric Bayesian factor regression model that accounts for uncertainty in the number of factors, and the relationship between factors. To accomplish this, we propose a sparse variant of the Indian Buffet Process and couple this with a hierarchical model over factors, based on Kingman’s coalescent. We apply this model to two problems (factor analysis and factor regression) in gene-expression data analysis. 1
2 0.20534314 18 nips-2008-An Efficient Sequential Monte Carlo Algorithm for Coalescent Clustering
Author: Dilan Gorur, Yee W. Teh
Abstract: We propose an efficient sequential Monte Carlo inference scheme for the recently proposed coalescent clustering model [1]. Our algorithm has a quadratic runtime while those in [1] is cubic. In experiments, we were surprised to find that in addition to being more efficient, it is also a better sequential Monte Carlo sampler than the best in [1], when measured in terms of variance of estimated likelihood and effective sample size. 1
3 0.20394021 234 nips-2008-The Infinite Factorial Hidden Markov Model
Author: Jurgen V. Gael, Yee W. Teh, Zoubin Ghahramani
Abstract: We introduce a new probability distribution over a potentially infinite number of binary Markov chains which we call the Markov Indian buffet process. This process extends the IBP to allow temporal dependencies in the hidden variables. We use this stochastic process to build a nonparametric extension of the factorial hidden Markov model. After constructing an inference scheme which combines slice sampling and dynamic programming we demonstrate how the infinite factorial hidden Markov model can be used for blind source separation. 1
4 0.18699418 26 nips-2008-Analyzing human feature learning as nonparametric Bayesian inference
Author: Thomas L. Griffiths, Joseph L. Austerweil
Abstract: Almost all successful machine learning algorithms and cognitive models require powerful representations capturing the features that are relevant to a particular problem. We draw on recent work in nonparametric Bayesian statistics to define a rational model of human feature learning that forms a featural representation from raw sensory data without pre-specifying the number of features. By comparing how the human perceptual system and our rational model use distributional and category information to infer feature representations, we seek to identify some of the forces that govern the process by which people separate and combine sensory primitives to form features. 1
5 0.1633459 9 nips-2008-A mixture model for the evolution of gene expression in non-homogeneous datasets
Author: Gerald Quon, Yee W. Teh, Esther Chan, Timothy Hughes, Michael Brudno, Quaid D. Morris
Abstract: We address the challenge of assessing conservation of gene expression in complex, non-homogeneous datasets. Recent studies have demonstrated the success of probabilistic models in studying the evolution of gene expression in simple eukaryotic organisms such as yeast, for which measurements are typically scalar and independent. Models capable of studying expression evolution in much more complex organisms such as vertebrates are particularly important given the medical and scientific interest in species such as human and mouse. We present Brownian Factor Phylogenetic Analysis, a statistical model that makes a number of significant extensions to previous models to enable characterization of changes in expression among highly complex organisms. We demonstrate the efficacy of our method on a microarray dataset profiling diverse tissues from multiple vertebrate species. We anticipate that the model will be invaluable in the study of gene expression patterns in other diverse organisms as well, such as worms and insects. 1
6 0.096818276 11 nips-2008-A spatially varying two-sample recombinant coalescent, with applications to HIV escape response
7 0.095474765 152 nips-2008-Non-stationary dynamic Bayesian networks
8 0.095124744 40 nips-2008-Bounds on marginal probability distributions
9 0.091053829 155 nips-2008-Nonparametric regression and classification with joint sparsity constraints
10 0.08998619 71 nips-2008-Efficient Sampling for Gaussian Process Inference using Control Variables
11 0.071561448 12 nips-2008-Accelerating Bayesian Inference over Nonlinear Differential Equations with Gaussian Processes
12 0.067701034 89 nips-2008-Gates
13 0.065079965 233 nips-2008-The Gaussian Process Density Sampler
14 0.062986217 37 nips-2008-Biasing Approximate Dynamic Programming with a Lower Discount Factor
15 0.059695303 216 nips-2008-Sparse probabilistic projections
16 0.058860503 70 nips-2008-Efficient Inference in Phylogenetic InDel Trees
17 0.057470303 221 nips-2008-Stochastic Relational Models for Large-scale Dyadic Data using MCMC
18 0.057124138 82 nips-2008-Fast Computation of Posterior Mode in Multi-Level Hierarchical Models
19 0.055834863 4 nips-2008-A Scalable Hierarchical Distributed Language Model
20 0.054003593 97 nips-2008-Hierarchical Fisher Kernels for Longitudinal Data
topicId topicWeight
[(0, -0.179), (1, -0.005), (2, 0.069), (3, 0.004), (4, 0.102), (5, -0.104), (6, -0.009), (7, 0.171), (8, 0.021), (9, -0.014), (10, -0.061), (11, 0.021), (12, 0.143), (13, -0.134), (14, 0.031), (15, 0.075), (16, 0.022), (17, -0.092), (18, 0.093), (19, 0.027), (20, -0.071), (21, 0.162), (22, -0.176), (23, 0.017), (24, -0.323), (25, -0.056), (26, -0.088), (27, -0.027), (28, -0.009), (29, 0.094), (30, 0.104), (31, 0.179), (32, 0.185), (33, 0.092), (34, -0.124), (35, 0.042), (36, -0.024), (37, -0.106), (38, -0.055), (39, -0.112), (40, -0.0), (41, 0.059), (42, -0.067), (43, 0.044), (44, -0.136), (45, -0.011), (46, 0.002), (47, 0.021), (48, 0.038), (49, 0.055)]
simIndex simValue paperId paperTitle
same-paper 1 0.9496876 235 nips-2008-The Infinite Hierarchical Factor Regression Model
Author: Piyush Rai, Hal Daume
Abstract: We propose a nonparametric Bayesian factor regression model that accounts for uncertainty in the number of factors, and the relationship between factors. To accomplish this, we propose a sparse variant of the Indian Buffet Process and couple this with a hierarchical model over factors, based on Kingman’s coalescent. We apply this model to two problems (factor analysis and factor regression) in gene-expression data analysis. 1
2 0.73816615 18 nips-2008-An Efficient Sequential Monte Carlo Algorithm for Coalescent Clustering
Author: Dilan Gorur, Yee W. Teh
Abstract: We propose an efficient sequential Monte Carlo inference scheme for the recently proposed coalescent clustering model [1]. Our algorithm has a quadratic runtime while those in [1] is cubic. In experiments, we were surprised to find that in addition to being more efficient, it is also a better sequential Monte Carlo sampler than the best in [1], when measured in terms of variance of estimated likelihood and effective sample size. 1
3 0.64889878 11 nips-2008-A spatially varying two-sample recombinant coalescent, with applications to HIV escape response
Author: Alexander Braunstein, Zhi Wei, Shane T. Jensen, Jon D. Mcauliffe
Abstract: Statistical evolutionary models provide an important mechanism for describing and understanding the escape response of a viral population under a particular therapy. We present a new hierarchical model that incorporates spatially varying mutation and recombination rates at the nucleotide level. It also maintains separate parameters for treatment and control groups, which allows us to estimate treatment effects explicitly. We use the model to investigate the sequence evolution of HIV populations exposed to a recently developed antisense gene therapy, as well as a more conventional drug therapy. The detection of biologically relevant and plausible signals in both therapy studies demonstrates the effectiveness of the method. 1
4 0.60774028 234 nips-2008-The Infinite Factorial Hidden Markov Model
Author: Jurgen V. Gael, Yee W. Teh, Zoubin Ghahramani
Abstract: We introduce a new probability distribution over a potentially infinite number of binary Markov chains which we call the Markov Indian buffet process. This process extends the IBP to allow temporal dependencies in the hidden variables. We use this stochastic process to build a nonparametric extension of the factorial hidden Markov model. After constructing an inference scheme which combines slice sampling and dynamic programming we demonstrate how the infinite factorial hidden Markov model can be used for blind source separation. 1
5 0.57390082 9 nips-2008-A mixture model for the evolution of gene expression in non-homogeneous datasets
Author: Gerald Quon, Yee W. Teh, Esther Chan, Timothy Hughes, Michael Brudno, Quaid D. Morris
Abstract: We address the challenge of assessing conservation of gene expression in complex, non-homogeneous datasets. Recent studies have demonstrated the success of probabilistic models in studying the evolution of gene expression in simple eukaryotic organisms such as yeast, for which measurements are typically scalar and independent. Models capable of studying expression evolution in much more complex organisms such as vertebrates are particularly important given the medical and scientific interest in species such as human and mouse. We present Brownian Factor Phylogenetic Analysis, a statistical model that makes a number of significant extensions to previous models to enable characterization of changes in expression among highly complex organisms. We demonstrate the efficacy of our method on a microarray dataset profiling diverse tissues from multiple vertebrate species. We anticipate that the model will be invaluable in the study of gene expression patterns in other diverse organisms as well, such as worms and insects. 1
6 0.49027777 26 nips-2008-Analyzing human feature learning as nonparametric Bayesian inference
7 0.43685514 152 nips-2008-Non-stationary dynamic Bayesian networks
8 0.43466368 70 nips-2008-Efficient Inference in Phylogenetic InDel Trees
9 0.40633711 236 nips-2008-The Mondrian Process
10 0.3929247 37 nips-2008-Biasing Approximate Dynamic Programming with a Lower Discount Factor
11 0.38021594 221 nips-2008-Stochastic Relational Models for Large-scale Dyadic Data using MCMC
12 0.37364662 89 nips-2008-Gates
13 0.34897393 82 nips-2008-Fast Computation of Posterior Mode in Multi-Level Hierarchical Models
14 0.34234393 155 nips-2008-Nonparametric regression and classification with joint sparsity constraints
15 0.33818084 188 nips-2008-QUIC-SVD: Fast SVD Using Cosine Trees
16 0.33230338 233 nips-2008-The Gaussian Process Density Sampler
17 0.33075187 134 nips-2008-Mixed Membership Stochastic Blockmodels
18 0.32771605 222 nips-2008-Stress, noradrenaline, and realistic prediction of mouse behaviour using reinforcement learning
19 0.32682791 4 nips-2008-A Scalable Hierarchical Distributed Language Model
20 0.32528797 71 nips-2008-Efficient Sampling for Gaussian Process Inference using Control Variables
topicId topicWeight
[(6, 0.05), (7, 0.069), (12, 0.028), (15, 0.013), (28, 0.179), (57, 0.106), (59, 0.012), (62, 0.251), (63, 0.015), (66, 0.014), (71, 0.015), (77, 0.053), (78, 0.017), (83, 0.059), (98, 0.032)]
simIndex simValue paperId paperTitle
same-paper 1 0.81173581 235 nips-2008-The Infinite Hierarchical Factor Regression Model
Author: Piyush Rai, Hal Daume
Abstract: We propose a nonparametric Bayesian factor regression model that accounts for uncertainty in the number of factors, and the relationship between factors. To accomplish this, we propose a sparse variant of the Indian Buffet Process and couple this with a hierarchical model over factors, based on Kingman’s coalescent. We apply this model to two problems (factor analysis and factor regression) in gene-expression data analysis. 1
2 0.77670944 113 nips-2008-Kernelized Sorting
Author: Novi Quadrianto, Le Song, Alex J. Smola
Abstract: Object matching is a fundamental operation in data analysis. It typically requires the definition of a similarity measure between the classes of objects to be matched. Instead, we develop an approach which is able to perform matching by requiring a similarity measure only within each of the classes. This is achieved by maximizing the dependency between matched pairs of observations by means of the Hilbert Schmidt Independence Criterion. This problem can be cast as one of maximizing a quadratic assignment problem with special structure and we present a simple algorithm for finding a locally optimal solution. 1
3 0.68384433 234 nips-2008-The Infinite Factorial Hidden Markov Model
Author: Jurgen V. Gael, Yee W. Teh, Zoubin Ghahramani
Abstract: We introduce a new probability distribution over a potentially infinite number of binary Markov chains which we call the Markov Indian buffet process. This process extends the IBP to allow temporal dependencies in the hidden variables. We use this stochastic process to build a nonparametric extension of the factorial hidden Markov model. After constructing an inference scheme which combines slice sampling and dynamic programming we demonstrate how the infinite factorial hidden Markov model can be used for blind source separation. 1
4 0.67453718 116 nips-2008-Learning Hybrid Models for Image Annotation with Partially Labeled Data
Author: Xuming He, Richard S. Zemel
Abstract: Extensive labeled data for image annotation systems, which learn to assign class labels to image regions, is difficult to obtain. We explore a hybrid model framework for utilizing partially labeled data that integrates a generative topic model for image appearance with discriminative label prediction. We propose three alternative formulations for imposing a spatial smoothness prior on the image labels. Tests of the new models and some baseline approaches on three real image datasets demonstrate the effectiveness of incorporating the latent structure. 1
5 0.67389882 200 nips-2008-Robust Kernel Principal Component Analysis
Author: Minh H. Nguyen, Fernando Torre
Abstract: Kernel Principal Component Analysis (KPCA) is a popular generalization of linear PCA that allows non-linear feature extraction. In KPCA, data in the input space is mapped to higher (usually) dimensional feature space where the data can be linearly modeled. The feature space is typically induced implicitly by a kernel function, and linear PCA in the feature space is performed via the kernel trick. However, due to the implicitness of the feature space, some extensions of PCA such as robust PCA cannot be directly generalized to KPCA. This paper presents a technique to overcome this problem, and extends it to a unified framework for treating noise, missing data, and outliers in KPCA. Our method is based on a novel cost function to perform inference in KPCA. Extensive experiments, in both synthetic and real data, show that our algorithm outperforms existing methods. 1
6 0.67048395 176 nips-2008-Partially Observed Maximum Entropy Discrimination Markov Networks
7 0.67038798 138 nips-2008-Modeling human function learning with Gaussian processes
8 0.66977656 197 nips-2008-Relative Performance Guarantees for Approximate Inference in Latent Dirichlet Allocation
9 0.66944265 118 nips-2008-Learning Transformational Invariants from Natural Movies
10 0.66648054 63 nips-2008-Dimensionality Reduction for Data in Multiple Feature Representations
11 0.6635856 66 nips-2008-Dynamic visual attention: searching for coding length increments
12 0.66352755 100 nips-2008-How memory biases affect information transmission: A rational analysis of serial reproduction
13 0.66344696 231 nips-2008-Temporal Dynamics of Cognitive Control
14 0.66310072 208 nips-2008-Shared Segmentation of Natural Scenes Using Dependent Pitman-Yor Processes
15 0.66258615 246 nips-2008-Unsupervised Learning of Visual Sense Models for Polysemous Words
16 0.66230756 4 nips-2008-A Scalable Hierarchical Distributed Language Model
17 0.66230088 79 nips-2008-Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning
18 0.6622383 92 nips-2008-Generative versus discriminative training of RBMs for classification of fMRI images
19 0.66195804 75 nips-2008-Estimating vector fields using sparse basis field expansions
20 0.66128761 62 nips-2008-Differentiable Sparse Coding