nips nips2005 nips2005-139 knowledge-graph by maker-knowledge-mining

139 nips-2005-Non-iterative Estimation with Perturbed Gaussian Markov Processes

Source: pdf

Author: Yunsong Huang, B. Keith Jenkins

Abstract: We develop an approach for estimation with Gaussian Markov processes that imposes a smoothness prior while allowing for discontinuities. Instead of propagating information laterally between neighboring nodes in a graph, we study the posterior distribution of the hidden nodes as a whole—how it is perturbed by invoking discontinuities, or weakening the edges, in the graph. We show that the resulting computation amounts to feed-forward fan-in operations reminiscent of V1 neurons. Moreover, using suitable matrix preconditioners, the incurred matrix inverse and determinant can be approximated, without iteration, in the same computational style. Simulation results illustrate the merits of this approach.

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 edu Abstract We develop an approach for estimation with Gaussian Markov processes that imposes a smoothness prior while allowing for discontinuities. [sent-4, score-0.081]

2 Instead of propagating information laterally between neighboring nodes in a graph, we study the posterior distribution of the hidden nodes as a whole—how it is perturbed by invoking discontinuities, or weakening the edges, in the graph. [sent-5, score-0.454]

3 We show that the resulting computation amounts to feed-forward fan-in operations reminiscent of V1 neurons. [sent-6, score-0.077]

4 Moreover, using suitable matrix preconditioners, the incurred matrix inverse and determinant can be approximated, without iteration, in the same computational style. [sent-7, score-0.216]

5 For generative models, often the ease of generation and the ease of inference are two conﬂicting features. [sent-10, score-0.038]

6 While the generation, or synthesis, of the input is immediate, the inference part is usually not. [sent-12, score-0.038]

7 In vision applications, it’s suitable to employ smoothness priors admitting discontinuities [4]. [sent-18, score-0.073]

8 Examples include weak membranes and plates [5], formulated in the context of variational energy minimization. [sent-19, score-0.171]

9 Typically, the inference for MRF or graphical models would incur lateral propagation of information between neighboring units [6]. [sent-20, score-0.038]

10 This is appealing in the sense that it consists of only simple, local operations carried out in parallel. [sent-21, score-0.043]

11 However, the resulting latency could undermine the plausibility that such algorithms are employed in human early vision inference tasks [7]. [sent-22, score-0.079]

12 In this paper we take the weak membrane and plate as instances of Gaussian processes (GP). [sent-23, score-0.389]

13 We show that the effect of marking each discontinuity (hereafter termed as “bond- breaking”) is to perturb the inverse of covariance matrix of the hidden nodes x by a matrix of rank 1. [sent-24, score-0.473]

14 When multiple bonds are broken, the computation of the posterior mean and covariance of x would involve the inversion of a matrix, which typically has large condition number, implying very slow convergence in straight-forward iterative approaches. [sent-25, score-0.344]

15 We show that there exists a family of preconditioners that can bring the condition number close to 1, thereby greatly speeding up the iteration—to the extent that a single step would sufﬁce in practice. [sent-26, score-0.191]

16 In what follows, we perturb the potential matrix Q−1 by reducing the coupling 0 energy of certain bonds2 . [sent-31, score-0.242]

17 This relieves the smoothness constraint on the nodes connected via those bonds. [sent-32, score-0.12]

18 Suppose the energy reduction of a bond connecting node i and j (whose state vectors are xi and xj , respectively) can be expressed as (xT fi + xT fj )2 , where fi and fj are coefﬁcient i j vectors. [sent-33, score-1.005]

19 This becomes (xT f )2 , if f is constructed to be a vector of same size as x, with the only non-zero entries fi and fj corresponding to node i and j. [sent-34, score-0.279]

20 This manipulation can be identiﬁed with a rank-1 perturbation of Q−1 , as Q−1 ← Q−1 − f f T , which is equivalent 0 1 0 to xT Q−1 x ← xT Q−1 x − (xT f )2 , ∀x. [sent-35, score-0.205]

21 We call this an elementary perturbation of Q−1 , 1 0 0 and f an elementary perturbation vector associated with the particular bond. [sent-36, score-0.488]

22 1), we form the L perturbation vectors into a matrix F1 = [f 1 , . [sent-39, score-0.269]

23 , f L ], and then the collective perturbations yield Q−1 1 and thus Q1 T = Q−1 − F1 F1 0 (1) = Q0 + Q0 F1 (I − T T F1 Q0 F1 )−1 F1 Q0 , (2) which follows from the Sherman-Morrison-Woodbury Formula (SMWF). [sent-42, score-0.036]

24 1 Perturbing a membrane and a plate In a membrane model [5], xi is scalar and the energy of the bond connecting xi and xj is (xi − xj )2 /q, where q is a parameter denoting the variance of state noise. [sent-44, score-1.14]

25 Upon perturba1 ensures positivity of the tion, this energy is reduced to η 2 (xi − xj )2 /q, where 0 < η energy. [sent-45, score-0.203]

26 Then, the energy reduction is (1 − η 2 )(xi − xj )2 /q, from which we can identify fi = (1 − η 2 )/q and fj = −fi . [sent-46, score-0.371]

27 In the case of a plate [5], xi = [ui , uh i , uv i ]T , in which ui represents the intensity, while uhi and uvi represent its gradient in the horizontal and vertical direction, respectively. [sent-47, score-0.761]

28 (−,i) We deﬁne the energy of a horizontal bond connecting node j and i as E0 = (uv i − 2 (−,i) T −1 (−,i) O d , where uv j ) /q + d d(−,i) = 1 2 ui uh i − 1 1 0 1 uj uhj and O = q 1/3 1/2 1/2 1 , Henceforth called bonds, as edge will refer to intensity discontinuity in an image. [sent-48, score-1.099]

29 the superscript (−, i) representing horizontal bond to the left of node i. [sent-51, score-0.42]

30 If E0 is re(−,i) duced to E1 = [(uvi − uv j )2 + (uhi − uhj )2 ]/q, i. [sent-54, score-0.141]

31 , coupling between node i and j exists only through their gradient values, one can show that the energy reduction (−,i) (−,i) E0 − E1 = [ui − uj − (uh i + uh j )/2]2 · 12/q. [sent-56, score-0.361]

32 Taking the actual energy reduction to (−,i) (−,i) − E1 ), we can identify fi (−,i) = 12(1 − η 2 )/q[1, −1/2, 0]T be (1 − η 2 )(E0 (−,i) T and fj = 12(1 − η 2 )/q[−1, −1/2, 0] , where 0 < η 1 ensures the positive deﬁniteness of the resulting potential matrix. [sent-57, score-0.367]

33 A similar procedure can be applied to a vertical bond in the plate, producing a perturbation vector f (|,i) , whose components are zero everywhere except for fi (|,i) = 12(1 − η 2 )/q[1, 0, −1/2]T and fj (|,i) = 12(1 − η 2 )/q[−1, 0, −1/2]T , for which node j is the lower neighbor of node i. [sent-58, score-0.896]

34 One can verify that xT f = 0 when the plate assumes the shape of a linear slope, meaning that this perturbation produces no energy difference in such a case. [sent-59, score-0.566]

35 (xT f )2 becomes signiﬁcant when the perturbed, or broken, bond associated with f straddles across a step discontinuity of the image. [sent-60, score-0.357]

36 2 Hidden state estimation Standard formulae exist for the posterior covariance K and mean x of x, given a noisy ˆ observation3 y = Cx + n, where n ∼ N (0, rI). [sent-63, score-0.044]

37 xα = Kα C T y/r, and Kα = [Q−1 + C T C/r]−1 , ˆ α for either the unperturbed (α = 0) or perturbed (α = 1) process. [sent-64, score-0.299]

38 For example, K1 equals K0 —a circulant matrix—plus a rank-L perturbation (cf. [sent-67, score-0.264]

39 Since each column of W1 is a spatially shifted copy of a prototypical vector, arising from breaking either a horizontal or a vertical −1 T bond, convolution can be utilized in computing W1 C T y. [sent-70, score-0.276]

40 For instance, z 1 r is the result of inner-products between the input y and the feed-forward fan-in weights CW , coded by the dendrites of identical −1 neurons, each situated at a broken bond. [sent-73, score-0.299]

41 , apply F1 and then F2 , both consisting of a set of perturbation vectors. [sent-78, score-0.205]

42 Quantities resulting from the α’th perturba3 4 The observation matrix C = I for a membrane, and C = I ⊗ [1, 0, 0] for a plate. [sent-79, score-0.064]

43 Solid and broken lines denote intact and broken bonds, respectively. [sent-86, score-0.7]

44 Open circles denote hidden nodes xi and ﬁlled circles denote observed nodes yi . [sent-87, score-0.248]

45 15 20 25 (c) Figure 2: The resulting receptive ﬁeld of the edge detector produced by breaking the shaded bond shown in Fig. [sent-88, score-0.544]

46 The central vertical dashed line in (a) and (b) marks the location of the vertical streak of bonds shown as broken in Fig. [sent-90, score-0.838]

47 In (a), those bonds are not actually broken; in (b), they are. [sent-92, score-0.307]

48 In (c), a central horizontal slice of (a) is plotted as a solid curve and the counterpart of (b) as a dashed curve. [sent-93, score-0.065]

49 , y x1 ˆ xc ˆ x0 ˆ Figure 3: Estimation of x given input y. [sent-94, score-0.093]

50 x0 : by unperturbed rod; x1 : coinciding perˆ ˆ fectly with y, is obtained by a rod whose two bonds at the step edges of y are broken; xc : ˆ correction term, engendered by the perturbed rod. [sent-95, score-0.799]

51 In particular, −1 T W2 = K1 F2 = K0 F2 + W1 H1 W1 F2 , g W2 (8) δW2 where W2 refers to the weights due to F2 in the absence of perturbation F1 , which, when indeed existent, would exert a contextual effect on F2 , thereby contributing to the term δW2 . [sent-98, score-0.242]

52 Figure 2 illustrates this effect on one perturbation vector (termed ‘edge detector’) in a membrane model, wherein ‘receptive ﬁeld’ refers to W2 and W2 in the case of panel (a) and (b), respectively. [sent-99, score-0.36]

53 Evidently, the receptive ﬁeld of W2 across the contextual boundary is pinched off. [sent-100, score-0.136]

54 We stress that once the relevant edges are detected, xc is computed almost instantly, ˆ without the need of iterative reﬁnement via lateral propagation. [sent-106, score-0.142]

55 3 Parameter estimation As edge inference/detection is outside the scope of this paper, we limit our attention to ﬁnding optimal values for the parameters r and q. [sent-109, score-0.125]

56 Given a possibly perturbed model Mα , in which x ∼ N (0, Qα ), we have y ∼ N (0, Sα ), where Sα = rI + CQα C T . [sent-112, score-0.24]

57 10 needs two identities, which are included here without proof (the second can be proven by using SMWF and its associated determinant identity): Eα = y T (y − C xα ) (cf. [sent-120, score-0.054]

58 3 Matrix Preconditioning Some of the foregoing computation necessitates matrix determinant and matrix inverse, e. [sent-123, score-0.182]

59 Methods exist in the literature for ﬁnding a matrix P ([9] and references therein) satisfying the following two criteria: (1) inverting P is easy; (2) the condition number κ(P −1 H) approaches 1. [sent-129, score-0.101]

60 Here we summarize our ﬁndings regarding the best class of preconditioners when H arises from some prototypical conﬁgurations of bond breaking. [sent-131, score-0.496]

61 When a streak of broken bonds forms a closed contour, with a consistent polarity convention (e. [sent-134, score-0.724]

62 , the excitatory region of the receptive ﬁeld of the edge detector associated with each bond lies inside the enclosed region), H and B (cf. [sent-136, score-0.563]

63 Let X be the unitary Fourier matrix of same size as H, then H e = X † HX would be approximately diagonal. [sent-139, score-0.129]

64 Let ΛH be diagonal: ΛH ij = δij H e ii , then H = XΛH X † is XΛH −1 X † approxa circulant matrix approximating H; i ΛH ii approximates |H|; −1 −1 1 imates H . [sent-140, score-0.123]

65 The quality of this preconditioner H can be evaluated by both the condition number κ(H −1 H) and the relative error between the inverse matrices: H −1 − H −1 F/ H −1 F, (12) where F denotes Frobenius norm. [sent-142, score-0.071]

66 The same X can approximately diagonalize B, and the product of the diagonal elements of the resulting matrix approximates |B|. [sent-143, score-0.064]

67 One end of the streak of broken bonds (target contour) abuts another contour, and the other end is open (i. [sent-145, score-0.796]

68 Imagine a vibrational mode of the membrane/plate given the conﬁguration of broken bonds. [sent-148, score-0.387]

69 The vibrational contrast of the nodes across the broken bond at a line-end has to be small, since in the immediate vicinity there exist paths of intact bonds linking the two nodes. [sent-149, score-1.159]

70 This suggests a Dirichlet boundary condition at the line-end. [sent-150, score-0.077]

71 , a T-junction), however, the vibrational contrast can be large, since the nodes on different sides of the contour are practically decoupled. [sent-153, score-0.272]

72 This analysis leads to using a transform (termed ‘HSWA’ in [10]) which we call ‘DCST’, denoting sine phase at the open end and cosine phase at the abutting end. [sent-155, score-0.281]

73 The unitary transform matrix X is given by: Xi,j = √ 2 2L + 1 cos(π(i − 1/2)(j − 1/2)/(L + 1/2)), 1 ≤ i, j ≤ L, where L is the number of broken bonds in the target contour. [sent-156, score-0.79]

74 When the streak of broken bonds form an open-ended contour, H can be approximately diagonalized by Sine Transform (cf. [sent-158, score-0.724]

75 the intuitive rationale stated in case (2)), of which the unitary transform matrix X is given by: Xi,j = 2/(L + 1) sin(πij/(L + 1)), 1 ≤ i, j ≤ L. [sent-159, score-0.184]

76 For a ‘clean’ prototypical contour, the performance of such preconditioners is remarkable, typically producing 1 ≤ κ < 1. [sent-160, score-0.216]

77 When contours in the image are interconnected in a complex way, we ﬁrst parse the image domain into non-overlapping enclosed regions, and then treat each region independently. [sent-163, score-0.25]

78 A contour segment dividing two regions is shared between them, and thus would contribute two copies, each belonging to one region[11]. [sent-164, score-0.101]

79 4 Experiment We test our approach on a real image (Fig. [sent-165, score-0.048]

80 We used both membrane and plate models, and in each case we used both the ‘direct’ method, which directly computes H −1 in Eq. [sent-172, score-0.389]

81 We ﬁrst apply a Canny detector to generate an edge map (Fig. [sent-175, score-0.146]

82 4g) for each noisy image, which is then converted to broken bonds. [sent-176, score-0.299]

83 The large number (over 104 ) of broken bonds makes the direct method impractical. [sent-177, score-0.649]

84 In order to attain a ‘direct’ result, we partition the image domain into a 5 × 5 array of blocks (one such block is delineated by the inner square in Fig. [sent-178, score-0.161]

85 4g), and focus on each of them in turn by retaining edges not more than 10 pixels from the target block (this block’s outer scope is delineated with the outer square in Fig. [sent-179, score-0.228]

86 When x is inferred given this partial edge map, only its pixels within the block ˆ are considered valid and are retained. [sent-181, score-0.135]

87 In ‘AD’, we parse the contours in each block and apply different diagonalizers accordingly, as summarized in Section 3. [sent-183, score-0.13]

88 4e and f illustrate the procedure to ﬁnd optimal q/r for a membrane and a plate, respectively, as explained in Section 2. [sent-188, score-0.155]

89 Note how good the cubic polynomial ﬁt is, and that the results of AD do not deviate much from those of the direct (rigorous) method. [sent-190, score-0.13]

90 4c and 4d show x by a perturbed and intact membrane model, respectively. [sent-192, score-0.497]

91 Estimation by (c) a perturbed membrane, and (d) an intact membrane. [sent-211, score-0.342]

92 The criterion function of varying q/r for (e) perturbed membrane, and (f) perturbed plate, which shares the same legend as in (e). [sent-212, score-0.48]

93 05 0 0 10 20 (a) DFT 30 0 0 100 (b) DST 200 0 Figure 5: Histograms of condition number κ after preconditioning, and relative error as deﬁned in Eq. [sent-234, score-0.037]

94 membrane model plate model direct AD direct AD q/r MSE q/r MSE q/r MSE q/r MSE 0. [sent-241, score-0.475]

95 031 121 Improved Entropic [12] MSE 121 138 166 5 Conclusions We have shown how the estimation with perturbed Gaussian Markov processes—hidden state and parameter estimation—can be carried out in non-iterative way. [sent-253, score-0.284]

96 Instead of focusing on each individual hidden node, we have taken each process as an entity under scrutiny. [sent-255, score-0.048]

97 This paradigm shift changes the way information is stored and represented—from the scenario where the global pattern of the process is embodied entirely by local couplings to the scenario where fan-in and fan-out weigths, in addition to local couplings, reﬂect the patterns of larger scales. [sent-256, score-0.044]

98 Although edge detection has not been treated in this paper, our formulation is capable of doing so, and our preliminary results are encouraging. [sent-257, score-0.081]

99 It may be premature at this stage to translate the operations of our model to neural substrate; we speculate nevertheless that our approach may have relevance to understanding biological visual systems. [sent-258, score-0.078]

100 Symmetric convolution and the discrete sine and cosine transforms. [sent-325, score-0.164]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('bonds', 0.307), ('broken', 0.299), ('bond', 0.28), ('perturbed', 0.24), ('plate', 0.234), ('perturbation', 0.205), ('membrane', 0.155), ('preconditioners', 0.154), ('ad', 0.144), ('energy', 0.127), ('mse', 0.123), ('dcst', 0.118), ('streak', 0.118), ('uh', 0.118), ('fi', 0.11), ('intact', 0.102), ('contour', 0.101), ('fj', 0.094), ('xc', 0.093), ('dft', 0.088), ('dst', 0.088), ('smwf', 0.088), ('vibrational', 0.088), ('sine', 0.087), ('cubic', 0.087), ('xt', 0.087), ('nodes', 0.083), ('uv', 0.082), ('edge', 0.081), ('discontinuity', 0.077), ('node', 0.075), ('unitary', 0.065), ('horizontal', 0.065), ('detector', 0.065), ('matrix', 0.064), ('prototypical', 0.062), ('receptive', 0.059), ('abutting', 0.059), ('circulant', 0.059), ('delineated', 0.059), ('gmp', 0.059), ('perturbing', 0.059), ('preconditioning', 0.059), ('uhi', 0.059), ('uhj', 0.059), ('unperturbed', 0.059), ('uvi', 0.059), ('breaking', 0.059), ('vertical', 0.057), ('transform', 0.055), ('ln', 0.055), ('block', 0.054), ('determinant', 0.054), ('ui', 0.053), ('termed', 0.052), ('canny', 0.051), ('rod', 0.051), ('perturb', 0.051), ('niteness', 0.051), ('edges', 0.049), ('image', 0.048), ('hidden', 0.048), ('deferred', 0.047), ('estimation', 0.044), ('variational', 0.044), ('cosine', 0.044), ('enclosed', 0.044), ('couplings', 0.044), ('direct', 0.043), ('operations', 0.043), ('connecting', 0.041), ('cw', 0.041), ('substrate', 0.041), ('fit', 0.041), ('uj', 0.041), ('latency', 0.041), ('boundary', 0.04), ('xj', 0.04), ('elementary', 0.039), ('parse', 0.039), ('von', 0.039), ('inference', 0.038), ('contextual', 0.037), ('contours', 0.037), ('condition', 0.037), ('smoothness', 0.037), ('markov', 0.036), ('end', 0.036), ('discontinuities', 0.036), ('mrf', 0.036), ('perturbations', 0.036), ('ensures', 0.036), ('visual', 0.035), ('snr', 0.035), ('amounts', 0.034), ('inverse', 0.034), ('region', 0.034), ('xi', 0.034), ('convolution', 0.033), ('outer', 0.033)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000002 139 nips-2005-Non-iterative Estimation with Perturbed Gaussian Markov Processes

Author: Yunsong Huang, B. Keith Jenkins

2 0.12990962 154 nips-2005-Preconditioner Approximations for Probabilistic Graphical Models

Author: John D. Lafferty, Pradeep K. Ravikumar

Abstract: We present a family of approximation techniques for probabilistic graphical models, based on the use of graphical preconditioners developed in the scientiﬁc computing literature. Our framework yields rigorous upper and lower bounds on event probabilities and the log partition function of undirected graphical models, using non-iterative procedures that have low time complexity. As in mean ﬁeld approaches, the approximations are built upon tractable subgraphs; however, we recast the problem of optimizing the tractable distribution parameters and approximate inference in terms of the well-studied linear systems problem of obtaining a good matrix preconditioner. Experiments are presented that compare the new approximation schemes to variational methods. 1

3 0.12648652 122 nips-2005-Logic and MRF Circuitry for Labeling Occluding and Thinline Visual Contours

Author: Eric Saund

Abstract: This paper presents representation and logic for labeling contrast edges and ridges in visual scenes in terms of both surface occlusion (border ownership) and thinline objects. In natural scenes, thinline objects include sticks and wires, while in human graphical communication thinlines include connectors, dividers, and other abstract devices. Our analysis is directed at both natural and graphical domains. The basic problem is to formulate the logic of the interactions among local image events, speciﬁcally contrast edges, ridges, junctions, and alignment relations, such as to encode the natural constraints among these events in visual scenes. In a sparse heterogeneous Markov Random Field framework, we deﬁne a set of interpretation nodes and energy/potential functions among them. The minimum energy conﬁguration found by Loopy Belief Propagation is shown to correspond to preferred human interpretation across a wide range of prototypical examples including important illusory contour ﬁgures such as the Kanizsa Triangle, as well as more difﬁcult examples. In practical terms, the approach delivers correct interpretations of inherently ambiguous hand-drawn box-and-connector diagrams at low computational cost.

4 0.10651117 133 nips-2005-Nested sampling for Potts models

Author: Iain Murray, David MacKay, Zoubin Ghahramani, John Skilling

Abstract: Nested sampling is a new Monte Carlo method by Skilling [1] intended for general Bayesian computation. Nested sampling provides a robust alternative to annealing-based methods for computing normalizing constants. It can also generate estimates of other quantities such as posterior expectations. The key technical requirement is an ability to draw samples uniformly from the prior subject to a constraint on the likelihood. We provide a demonstration with the Potts model, an undirected graphical model. 1

5 0.10577202 15 nips-2005-A Theoretical Analysis of Robust Coding over Noisy Overcomplete Channels

Author: Eizaburo Doi, Doru C. Balcan, Michael S. Lewicki

Abstract: Biological sensory systems are faced with the problem of encoding a high-ﬁdelity sensory signal with a population of noisy, low-ﬁdelity neurons. This problem can be expressed in information theoretic terms as coding and transmitting a multi-dimensional, analog signal over a set of noisy channels. Previously, we have shown that robust, overcomplete codes can be learned by minimizing the reconstruction error with a constraint on the channel capacity. Here, we present a theoretical analysis that characterizes the optimal linear coder and decoder for one- and twodimensional data. The analysis allows for an arbitrary number of coding units, thus including both under- and over-complete representations, and provides a number of important insights into optimal coding strategies. In particular, we show how the form of the code adapts to the number of coding units and to different data and noise conditions to achieve robustness. We also report numerical solutions for robust coding of highdimensional image data and show that these codes are substantially more robust compared against other image codes such as ICA and wavelets. 1

6 0.090956159 65 nips-2005-Estimating the wrong Markov random field: Benefits in the computation-limited setting

7 0.087086745 125 nips-2005-Message passing for task redistribution on sparse graphs

8 0.078989178 27 nips-2005-Analysis of Spectral Kernel Design based Semi-supervised Learning

9 0.073736534 46 nips-2005-Consensus Propagation

10 0.072434559 184 nips-2005-Structured Prediction via the Extragradient Method

11 0.068946213 24 nips-2005-An Approximate Inference Approach for the PCA Reconstruction Error

12 0.067198612 60 nips-2005-Dynamic Social Network Analysis using Latent Space Models

13 0.067165017 50 nips-2005-Convex Neural Networks

14 0.065768205 202 nips-2005-Variational EM Algorithms for Non-Gaussian Latent Variable Models

15 0.063178487 7 nips-2005-A Cortically-Plausible Inverse Problem Solving Method Applied to Recognizing Static and Kinematic 3D Objects

16 0.062775582 101 nips-2005-Is Early Vision Optimized for Extracting Higher-order Dependencies?

17 0.060705885 51 nips-2005-Correcting sample selection bias in maximum entropy density estimation

18 0.059579697 96 nips-2005-Inference with Minimal Communication: a Decision-Theoretic Variational Approach

19 0.058169864 70 nips-2005-Fast Information Value for Graphical Models

20 0.055844821 23 nips-2005-An Application of Markov Random Fields to Range Sensing

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.213), (1, 0.004), (2, -0.01), (3, 0.049), (4, -0.011), (5, -0.051), (6, -0.213), (7, 0.047), (8, 0.082), (9, 0.064), (10, 0.11), (11, 0.024), (12, -0.074), (13, -0.005), (14, -0.007), (15, 0.005), (16, 0.018), (17, 0.014), (18, -0.001), (19, -0.036), (20, 0.062), (21, -0.024), (22, -0.028), (23, 0.129), (24, 0.074), (25, 0.003), (26, 0.086), (27, -0.139), (28, 0.08), (29, 0.061), (30, -0.019), (31, 0.13), (32, -0.101), (33, -0.03), (34, 0.073), (35, -0.073), (36, 0.135), (37, -0.003), (38, -0.116), (39, -0.031), (40, 0.086), (41, 0.124), (42, -0.012), (43, 0.008), (44, -0.128), (45, 0.08), (46, 0.111), (47, -0.003), (48, -0.011), (49, 0.131)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.94039422 139 nips-2005-Non-iterative Estimation with Perturbed Gaussian Markov Processes

Author: Yunsong Huang, B. Keith Jenkins

2 0.71738106 122 nips-2005-Logic and MRF Circuitry for Labeling Occluding and Thinline Visual Contours

Author: Eric Saund

3 0.52927154 133 nips-2005-Nested sampling for Potts models

Author: Iain Murray, David MacKay, Zoubin Ghahramani, John Skilling

4 0.5021888 24 nips-2005-An Approximate Inference Approach for the PCA Reconstruction Error

Author: Manfred Opper

Abstract: The problem of computing a resample estimate for the reconstruction error in PCA is reformulated as an inference problem with the help of the replica method. Using the expectation consistent (EC) approximation, the intractable inference problem can be solved efﬁciently using only two variational parameters. A perturbative correction to the result is computed and an alternative simpliﬁed derivation is also presented. 1

5 0.46977928 90 nips-2005-Hot Coupling: A Particle Approach to Inference and Normalization on Pairwise Undirected Graphs

Author: Firas Hamze, Nando de Freitas

Abstract: This paper presents a new sampling algorithm for approximating functions of variables representable as undirected graphical models of arbitrary connectivity with pairwise potentials, as well as for estimating the notoriously difﬁcult partition function of the graph. The algorithm ﬁts into the framework of sequential Monte Carlo methods rather than the more widely used MCMC, and relies on constructing a sequence of intermediate distributions which get closer to the desired one. While the idea of using “tempered” proposals is known, we construct a novel sequence of target distributions where, rather than dropping a global temperature parameter, we sequentially couple individual pairs of variables that are, initially, sampled exactly from a spanning tree of the variables. We present experimental results on inference and estimation of the partition function for sparse and densely-connected graphs.

6 0.46251407 15 nips-2005-A Theoretical Analysis of Robust Coding over Noisy Overcomplete Channels

7 0.45389506 125 nips-2005-Message passing for task redistribution on sparse graphs

8 0.42897362 65 nips-2005-Estimating the wrong Markov random field: Benefits in the computation-limited setting

9 0.41047534 3 nips-2005-A Bayesian Framework for Tilt Perception and Confidence

10 0.41014236 154 nips-2005-Preconditioner Approximations for Probabilistic Graphical Models

11 0.40794173 51 nips-2005-Correcting sample selection bias in maximum entropy density estimation

12 0.38988334 180 nips-2005-Spectral Bounds for Sparse PCA: Exact and Greedy Algorithms

13 0.38191923 60 nips-2005-Dynamic Social Network Analysis using Latent Space Models

14 0.36667755 68 nips-2005-Factorial Switching Kalman Filters for Condition Monitoring in Neonatal Intensive Care

15 0.35247579 46 nips-2005-Consensus Propagation

16 0.34312528 204 nips-2005-Walk-Sum Interpretation and Analysis of Gaussian Belief Propagation

17 0.31798261 106 nips-2005-Large-scale biophysical parameter estimation in single neurons via constrained linear regression

18 0.31617281 184 nips-2005-Structured Prediction via the Extragradient Method

19 0.315514 7 nips-2005-A Cortically-Plausible Inverse Problem Solving Method Applied to Recognizing Static and Kinematic 3D Objects

20 0.309313 137 nips-2005-Non-Gaussian Component Analysis: a Semi-parametric Framework for Linear Dimension Reduction

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(3, 0.037), (6, 0.338), (10, 0.041), (18, 0.017), (27, 0.034), (31, 0.08), (34, 0.071), (39, 0.014), (55, 0.036), (60, 0.015), (65, 0.01), (69, 0.048), (73, 0.055), (88, 0.062), (91, 0.047), (94, 0.015)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.79086953 139 nips-2005-Non-iterative Estimation with Perturbed Gaussian Markov Processes

Author: Yunsong Huang, B. Keith Jenkins

2 0.71969795 20 nips-2005-Affine Structure From Sound

Author: Sebastian Thrun

Abstract: We consider the problem of localizing a set of microphones together with a set of external acoustic events (e.g., hand claps), emitted at unknown times and unknown locations. We propose a solution that approximates this problem under a far ﬁeld approximation deﬁned in the calculus of afﬁne geometry, and that relies on singular value decomposition (SVD) to recover the afﬁne structure of the problem. We then deﬁne low-dimensional optimization techniques for embedding the solution into Euclidean geometry, and further techniques for recovering the locations and emission times of the acoustic events. The approach is useful for the calibration of ad-hoc microphone arrays and sensor networks. 1

3 0.6865027 72 nips-2005-Fast Online Policy Gradient Learning with SMD Gain Vector Adaptation

Author: Jin Yu, Douglas Aberdeen, Nicol N. Schraudolph

Abstract: Reinforcement learning by direct policy gradient estimation is attractive in theory but in practice leads to notoriously ill-behaved optimization problems. We improve its robustness and speed of convergence with stochastic meta-descent, a gain vector adaptation method that employs fast Hessian-vector products. In our experiments the resulting algorithms outperform previously employed online stochastic, ofﬂine conjugate, and natural policy gradient methods. 1

4 0.41260463 78 nips-2005-From Weighted Classification to Policy Search

Author: Doron Blatt, Alfred O. Hero

Abstract: This paper proposes an algorithm to convert a T -stage stochastic decision problem with a continuous state space to a sequence of supervised learning problems. The optimization problem associated with the trajectory tree and random trajectory methods of Kearns, Mansour, and Ng, 2000, is solved using the Gauss-Seidel method. The algorithm breaks a multistage reinforcement learning problem into a sequence of single-stage reinforcement learning subproblems, each of which is solved via an exact reduction to a weighted-classiﬁcation problem that can be solved using off-the-self methods. Thus the algorithm converts a reinforcement learning problem into simpler supervised learning subproblems. It is shown that the method converges in a ﬁnite number of steps to a solution that cannot be further improved by componentwise optimization. The implication of the proposed algorithm is that a plethora of classiﬁcation methods can be applied to ﬁnd policies in the reinforcement learning problem. 1

5 0.40839878 154 nips-2005-Preconditioner Approximations for Probabilistic Graphical Models

Author: John D. Lafferty, Pradeep K. Ravikumar

6 0.4067561 96 nips-2005-Inference with Minimal Communication: a Decision-Theoretic Variational Approach

7 0.40429464 144 nips-2005-Off-policy Learning with Options and Recognizers

8 0.40426576 90 nips-2005-Hot Coupling: A Particle Approach to Inference and Normalization on Pairwise Undirected Graphs

9 0.39821708 102 nips-2005-Kernelized Infomax Clustering

10 0.39759532 153 nips-2005-Policy-Gradient Methods for Planning

11 0.39720684 46 nips-2005-Consensus Propagation

12 0.39573544 30 nips-2005-Assessing Approximations for Gaussian Process Classification

13 0.39509037 142 nips-2005-Oblivious Equilibrium: A Mean Field Approximation for Large-Scale Dynamic Games

14 0.39399454 43 nips-2005-Comparing the Effects of Different Weight Distributions on Finding Sparse Representations

15 0.39337522 124 nips-2005-Measuring Shared Information and Coordinated Activity in Neuronal Networks

16 0.39184842 132 nips-2005-Nearest Neighbor Based Feature Selection for Regression and its Application to Neural Activity

17 0.39163429 184 nips-2005-Structured Prediction via the Extragradient Method

18 0.39127308 178 nips-2005-Soft Clustering on Graphs

19 0.39089254 200 nips-2005-Variable KD-Tree Algorithms for Spatial Pattern Search and Discovery

20 0.38963187 45 nips-2005-Conditional Visual Tracking in Kernel Space