nips nips2012 nips2012-155 knowledge-graph by maker-knowledge-mining

155 nips-2012-Human memory search as a random walk in a semantic network

Source: pdf

Author: Joseph L. Austerweil, Joshua T. Abbott, Thomas L. Griffiths

Abstract: The human mind has a remarkable ability to store a vast amount of information in memory, and an even more remarkable ability to retrieve these experiences when needed. Understanding the representations and algorithms that underlie human memory search could potentially be useful in other information retrieval settings, including internet search. Psychological studies have revealed clear regularities in how people search their memory, with clusters of semantically related items tending to be retrieved together. These ﬁndings have recently been taken as evidence that human memory search is similar to animals foraging for food in patchy environments, with people making a rational decision to switch away from a cluster of related information as it becomes depleted. We demonstrate that the results that were taken as evidence for this account also emerge from a random walk on a semantic network, much like the random web surfer model used in internet search engines. This offers a simpler and more uniﬁed account of how people search their memory, postulating a single process rather than one process for exploring a cluster and one process for switching between clusters. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 Human memory search as a random walk in a semantic network Joseph L. [sent-1, score-0.613]

2 Understanding the representations and algorithms that underlie human memory search could potentially be useful in other information retrieval settings, including internet search. [sent-10, score-0.343]

3 Psychological studies have revealed clear regularities in how people search their memory, with clusters of semantically related items tending to be retrieved together. [sent-11, score-0.306]

4 These ﬁndings have recently been taken as evidence that human memory search is similar to animals foraging for food in patchy environments, with people making a rational decision to switch away from a cluster of related information as it becomes depleted. [sent-12, score-0.912]

5 We demonstrate that the results that were taken as evidence for this account also emerge from a random walk on a semantic network, much like the random web surfer model used in internet search engines. [sent-13, score-0.608]

6 This offers a simpler and more uniﬁed account of how people search their memory, postulating a single process rather than one process for exploring a cluster and one process for switching between clusters. [sent-14, score-0.324]

7 1 Introduction Human memory has a vast capacity, storing all the semantic knowledge, facts, and experiences that people accrue over a lifetime. [sent-15, score-0.498]

8 In fact, it is the same problem faced by libraries [1] and internet search engines [6] that need to efﬁciently organize information to facilitate retrieval of those items most likely to be relevant to a query. [sent-17, score-0.233]

9 It thus becomes interesting to try to understand exactly what kind of algorithms and representations are used when people search their memory. [sent-18, score-0.179]

10 One of the main tasks that has been used to explore memory search is the semantic ﬂuency task, in which people retrieve as many items belonging to a particular category (e. [sent-19, score-0.678]

11 Early studies using semantic ﬂuency tasks suggested a two-part memory retrieval process: clustering, in which the production of words form semantic subcategories, and switching, in which a transition is made from one subcategory to another [13, 21]. [sent-22, score-0.842]

12 Recently, it has been suggested that the clustering patterns observed in semantic ﬂuency tasks could reﬂect an optimal foraging strategy, with people searching for items distributed in memory in a way that is similar to animals searching for food in environments with patchy food resources [7]. [sent-24, score-1.09]

13 The idea behind this approach is that each cluster corresponds to a “patch” and people strategically choose to leave patches when the rate at which they retrieve relevant concepts drops below their average rate of retrieval. [sent-25, score-0.242]

14 Quantitative analyses of human data provide support for this account, ﬁnding shorter delays in retrieving relevant items after a change in clusters and a relationship between when people leave a cluster and their average retrieval time. [sent-26, score-0.495]

15 In this paper, we argue that there may be a simpler explanation for the patterns seen in semantic ﬂuency tasks, requiring only a single cognitive process rather than separate processes for exploring a cluster and deciding to switch between clusters. [sent-27, score-0.507]

16 We show that the results used to argue for the optimal foraging account can be reproduced by a random walk on a semantic network derived from human semantic associations. [sent-28, score-0.936]

17 Intriguingly, this is exactly the kind of process assumed by the PageRank algorithm [12], providing a suggestive link between human memory and internet search and a new piece of evidence supporting the claim [6] that this algorithm might be relevant to understanding human semantic memory. [sent-29, score-0.646]

18 Section 2 provides relevant background information on studies of human memory search with semantic ﬂuency tasks and outlines the retrieval phenomena predicted by an optimal foraging account. [sent-31, score-0.822]

19 Section 3 presents the parallels between searching the internet and search in human memory, and provides a structural analysis of semantic memory. [sent-32, score-0.474]

20 Section 4 evaluates our proposal that a random walk in a semantic network is consistent with the observed behavior in semantic ﬂuency tasks. [sent-33, score-0.739]

21 2 Semantic ﬂuency and optimal foraging Semantic ﬂuency tasks (also known as free recall from natural categories) are a classic methodological paradigm for examining how people recall relevant pieces of information from memory given a retrieval cue [2, 14, 19]. [sent-35, score-0.508]

22 Asking people to retrieve as many examples of a category as possible in a limited time is a simple task to carry out in clinical settings, and semantic ﬂuency has been used to study memory deﬁcits in patients with Alzheimer’s, Parkinson’s, and Huntington’s disease [9, 20, 21, 22]. [sent-36, score-0.569]

23 Both early and recent studies [2, 14, 21] have consistently found that clusters appear in the sequences of words that people produce, with bursts of semantically related words produced together and noticeable pauses between these bursts. [sent-37, score-0.348]

24 [21] had people retrieve examples of animals, and divided those animals into 22 nonexclusive clusters (“pets”, “African animals”, etc. [sent-39, score-0.349]

25 These clusters could be used to analyze patterns in people’s responses: if an item shares a cluster with the item immediately before it, it is considered part of the same cluster, otherwise, the current item deﬁnes a transition between clusters. [sent-41, score-0.498]

26 Observing fast transitions between items within a cluster but slow transitions between clusters led to the proposal that memory search might be decomposed into separate “clustering” and “switching” processes [21]. [sent-43, score-0.39]

27 The clusters that seem to appear in semantic memory suggest an analogy to the distribution of animal food sources in a patchy environment. [sent-44, score-0.661]

28 When animals search for food, they must consider the costs and beneﬁts of staying within a patch as opposed to searching for a new patch. [sent-45, score-0.374]

29 In particular, the marginal value theorem shows that a forager’s overall rate of return is optimized if it leaves a patch when the instantaneous rate (the marginal value) of ﬁnding food within the patch falls below the long-term average rate of ﬁnding food over the entire environment [3]. [sent-47, score-0.449]

30 [7] posited that search in human semantic memory is similarly guided by an optimal foraging policy. [sent-49, score-0.664]

31 The corresponding prediction is that people should leave a “patch” in memory (ie. [sent-50, score-0.222]

32 [7] had people perform a semantic ﬂuency task (“Name as many animals as you can in 3 minutes”) and analyzed the search paths taken through memory 2 1. [sent-53, score-0.717]

33 (a) The mean ratio between the inter-item response time (IRT) for an item and the participant’s long-term average IRT over the entire task, relative to the order of entry for the item (where “1” refers to the relative IRT between the ﬁrst word in a patch and the last word in the preceding patch). [sent-61, score-0.621]

34 (c) The relationship between a participant’s deviation from the marginal value theorem policy for patch departures (horizontalaxis) and the total number of words a participant produced. [sent-64, score-0.282]

35 in terms of the sequences of animal names produced, assessed with the predetermined animal subcategories of Troyer et al. [sent-65, score-0.295]

36 As a ﬁrst measure of correspondence with optimal foraging theory, the ratio between inter-item response times (IRTs) of items and the long-term average IRTs for each participant were examined at different retrieval positions relative to a patch switch. [sent-67, score-0.481]

37 The ﬁrst word in a patch (indicated by an order of entry of “1”) takes longer to produce than the overall long-term average IRT (indicated by the dotted line), and the second word in a patch (indicated by “2”) takes much less time to produce. [sent-69, score-0.481]

38 These results are in line with the marginal value theorem where IRTs up until a patch switch should increase monotonically towards the long-term average IRT and go above this average only for patch switch IRTs since it takes extra time to ﬁnd a new patch. [sent-70, score-0.583]

39 offered a two-part process model to account for this phenomenon: When the IRT following a word exceeds the long-term average IRT, search switches from local to global cues (e. [sent-72, score-0.194]

40 switching between using semantic similarity or overall frequency as search cues). [sent-74, score-0.425]

41 To formally examine how close the IRTs for words immediately preceding a patch switch were to the long-term average IRT, the per-participant average IRT for these pre-switch words was plotted against the per-participant long-term average IRT (see Figure 1 (b)). [sent-75, score-0.448]

42 As a further analysis of these preswitch IRTs, the absolute difference between the pre-switch IRT and long-term average IRT was plotted against the number of words a participant produced along with a regression line through this data (see Figure 1 (c)). [sent-77, score-0.205]

43 3 The structure of semantic memory The explanation proposed by Hills et al. [sent-79, score-0.444]

44 [7] for the patterns observed in people’s behavior in semantic ﬂuency tasks is relatively complex, assuming two separate processes and a strategic decision to switch between them. [sent-80, score-0.46]

45 In the remainder of the paper we consider a simpler alternative explanation, based on the structure of semantic memory. [sent-81, score-0.276]

46 Speciﬁcally, we consider the consequences of a single search process operating over a richer representation – a semantic network. [sent-82, score-0.35]

47 A semantic network represents the relationships between words (or concepts) as a directed graph, where each word is represented as a node and nodes are connected together with edges that represent pairwise association [4]. [sent-83, score-0.441]

48 Semantic networks derived from human behavior can be used to explore questions about the structure of human memory [5, 14, 17, 18]. [sent-84, score-0.241]

49 We will focus on a network derived from a word association task, in which people were asked to list the words that come to mind for a particular cue. [sent-85, score-0.251]

50 The result is a semantic network with 5018 nodes, from “a” to “zucchini”. [sent-88, score-0.312]

51 We explored whether the distance between the nodes corresponding to different animals in the semantic network could be predicted by their cluster membership. [sent-91, score-0.548]

52 produced 373 unique animals, of which 178 were included in the semantic network. [sent-93, score-0.345]

53 We analyzed whether the relationship between these animals in the semantic network showed evidence of the clustering seen in semantic ﬂuency tasks, based on the clusters identiﬁed by Troyer et al. [sent-96, score-0.877]

54 The features in the matrix F were deﬁned to be the twenty-two hand-coded subcategorization of animals from Troyer et al. [sent-100, score-0.177]

55 The two similarity matrices contain similar block structure, which supports the hypothesis that the clusters of animals are implicitly captured by the semantic network. [sent-103, score-0.509]

56 If the distance between animals in different clusters is greater than the distance between animals in the same cluster, as these results suggest, then a simple search process that is sensitive to this distance may be able to account for the results reported by Hills et al. [sent-104, score-0.482]

57 (a) (b) Figure 2: Visualizing the similarity between pairs of animals in our semantic network (darker colors represent stronger similarities). [sent-106, score-0.483]

58 The rows and columns of the two matrices were reordered to display animals in the clusters with largest weight ﬁrst. [sent-110, score-0.207]

59 4 4 Random walks and semantic ﬂuency One of the simplest processes that can operate over a semantic network is a random walk, stochastically jumping from one node to another by following edges. [sent-111, score-0.682]

60 Intuitively, this might provide a reasonable model for searching through semantic memory, being a meandering rather than a directed search. [sent-112, score-0.308]

61 Random walks on semantic networks have previously been proposed as a possible account of behavior on ﬂuency tasks: Grifﬁths et al. [sent-113, score-0.371]

62 [6] argued that the responses that people produce when asked to generate a word that begins with a particular letter were consistent with the stationary distribution of a random walk on the same semantic network used in the analysis presented in the previous section. [sent-114, score-0.64]

63 In addition to being simple, random walks on semantic networks have an interesting connection to methods used for information retrieval. [sent-115, score-0.297]

64 The link structure of n web pages on the Internet can be characterized by an n × n matrix L, where Lij is 1 if there is a link from web page j to web page i, and 0 otherwise. [sent-118, score-0.177]

65 If an internet user clicks uniformly at random over the outgoing links, then the probability that the user will click on page i given she is currently on page j is Lij Mij = n (2) k=1 Lkj where the denominator is the out-degree or number of web pages that page j links to. [sent-119, score-0.212]

66 [6] is that the prominence of words in human memory can be predicted by running the PageRank algorithm on a semantic network. [sent-123, score-0.54]

67 pointed out, multiple mechanisms exist that could produce this result, with only one possibility being that memory search is a random walk on a semantic network. [sent-125, score-0.603]

68 Exploring whether this kind of random walk can reproduce the phenomena identiﬁed by Hills et al. [sent-126, score-0.177]

69 We then evaluate our models of memory search by applying the analyses used by Hills et al. [sent-129, score-0.257]

70 [7] participants were asked to produce as many unique animals as possible in three minutes. [sent-133, score-0.234]

71 2 Computing inter-item retrieval times Random walk simulations for the models deﬁned above will produce a list of the nodes visited at each iteration. [sent-150, score-0.234]

72 In our analyses we consider only the ﬁrst time an animal node is visited, which we denote as τ (k) for the k th unique animal seen on the random walk (out of the K unique animals seen on the random walk). [sent-152, score-0.533]

73 Thus, according to the random walk models, the inter-item retrieval time (IRT) between animal k and k − 1 is IRT (k) = τ (k) − τ (k − 1) + L(Xτ (k) ) (4) where τ (k) is the ﬁrst hitting time of animal Xτ (k) and L(X) is the length of word X. [sent-159, score-0.479]

74 The number of iterations was selected to produce a similar mean number of animals to those produced by participants in Hills et al. [sent-165, score-0.335]

75 The left column shows the mean ratio between the inter-item retrieval time (IRT) for an item and the mean IRT over all 1750 iterations in the simulations, relative to the order of entry for the item. [sent-178, score-0.242]

76 Here we see that the ﬁrst word starting a patch (the bar labeled “1”) has the highest overall retrieval time. [sent-179, score-0.248]

77 as indicating the time it takes to switch clusters and generate a word from a new cluster. [sent-181, score-0.248]

78 The 2 A slightly lower overall total number of animals is to be expected, given the limited number of animals among the words included in our semantic network. [sent-182, score-0.611]

79 2 5 0 0 −2 −1 1 2 50 45 40 35 30 25 0 3 5 10 15 20 25 30 35 0 Mean IRT for the entry prior to switch Order of entry relative to patch switch 2 4 6 8 10 Abs(Last item IRT − Average IRT) 50 1. [sent-198, score-0.595]

80 4 (d) 15 35 Mean IRT for the entry prior to switch Order of entry relative to patch switch (c) 10 5 0 3 5 40 50 −2 20 Abs(Last item IRT − Average IRT) Number of words produced 0. [sent-199, score-0.709]

81 2 1 35 Mean IRT for the entry prior to switch Order of entry relative to patch switch (b) 40 15 0 3 45 Number of words produced 20 1 Average IRT Item IRT / Average IRT 25 1. [sent-202, score-0.595]

82 The right-most column displays the relationship between a simulation’s deviation from the marginal value theorem policy for patch departures (horizontal axis) and the total number of words the simulation produced. [sent-213, score-0.265]

83 7 emergence of the same phenomenon is seen across all four of our models which suggests that the structure of semantic memory, together with a simple undirected search process, is sufﬁcient to capture this effect. [sent-214, score-0.373]

84 [7] on the models, demonstrating that, like people, all four models take a signiﬁcantly longer amount of time for the word immediately following a patch (all t(999) > 44, p < 0. [sent-217, score-0.211]

85 0001) and take a signiﬁcantly shorter amount of time for the second item after a patch (all t(999) < −49, p < 0. [sent-218, score-0.257]

86 Intriguingly, all four models produce the basic phenomena taken as evidence for the use of the marginal value theorem in memory search. [sent-221, score-0.251]

87 There is a strong correlation between the IRT at the point of a cluster switch and the mean IRT (R2 = 0. [sent-222, score-0.173]

88 These results show that behavior consistent with following the marginal value theorem can be produced by surprisingly simple search algorithms, at least when measured along these metrics. [sent-233, score-0.194]

89 5 Discussion Understanding how people organize and search through information stored in memory has the potential to inform how we construct automated information retrieval systems. [sent-234, score-0.356]

90 In this paper, we considered two different accounts for the appearance of semantically-related clusters when people retrieve a sequence of items from memory. [sent-235, score-0.274]

91 In contrast, the proposal that memory search might just be a random walk on a semantic network [6] postulates a single, undirected process. [sent-238, score-0.636]

92 Our results show that four random walk models qualitatively reproduce a set of results predicted by optimal foraging theory, providing an alternative explanation for clustering in semantic ﬂuency tasks. [sent-239, score-0.625]

93 Finding that a random walk on a semantic network can account for some of the relatively complex phenomena that appear in the semantic ﬂuency task provides further support for the idea that memory search might simply be a random walk. [sent-240, score-0.948]

94 This result helps to clarify the possible mechanisms that could account for PageRank predicting the prominence of words in semantic memory [6], since PageRank is simply the stationary distribution of the Markov chain deﬁned by this random walk. [sent-241, score-0.491]

95 Demonstrating that the random walk models can produce behavior consistent with optimal foraging in semantic ﬂuency tasks generates some interesting directions for future research. [sent-243, score-0.6]

96 Having two competing accounts of the same phenomena suggests that the next step in exploring semantic ﬂuency is designing an experiment that distinguishes between these accounts. [sent-244, score-0.358]

97 Considering whether the optimal foraging account can also predict the prominence of words in semantic memory, where the random walk model is already known to succeed, is one possibility, as is exploring the predictions of the two accounts across a wider range of memory search tasks. [sent-245, score-0.866]

98 However, one of the most intriguing directions for future research is considering how these different proposals fare in accounting for changes in semantic ﬂuency in clinical populations. [sent-246, score-0.293]

99 Word association spaces for predicting semantic similarity effects in episodic memory. [sent-373, score-0.302]

100 The large-scale structure of semantic networks: Statistical analyses and a model of semantic growth. [sent-379, score-0.586]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('irt', 0.706), ('uency', 0.283), ('semantic', 0.276), ('hills', 0.197), ('animals', 0.145), ('foraging', 0.144), ('irts', 0.141), ('patch', 0.123), ('animal', 0.122), ('switch', 0.121), ('memory', 0.117), ('item', 0.114), ('walk', 0.11), ('people', 0.105), ('troyer', 0.082), ('search', 0.074), ('produced', 0.069), ('word', 0.065), ('pagerank', 0.063), ('participants', 0.063), ('clusters', 0.062), ('participant', 0.06), ('retrieval', 0.06), ('cat', 0.055), ('jumping', 0.054), ('human', 0.053), ('food', 0.053), ('cluster', 0.052), ('switching', 0.049), ('dog', 0.049), ('entry', 0.048), ('psychological', 0.046), ('words', 0.045), ('abs', 0.045), ('alzheimer', 0.043), ('items', 0.043), ('transition', 0.042), ('parkinson', 0.042), ('cue', 0.039), ('web', 0.039), ('internet', 0.039), ('xn', 0.038), ('retrieve', 0.037), ('network', 0.036), ('phenomena', 0.035), ('analyses', 0.034), ('steyvers', 0.034), ('grif', 0.034), ('clustering', 0.033), ('marginal', 0.033), ('et', 0.032), ('searching', 0.032), ('ths', 0.032), ('patchy', 0.031), ('average', 0.031), ('page', 0.03), ('surfer', 0.029), ('cits', 0.029), ('prominence', 0.029), ('psychology', 0.028), ('berkeley', 0.027), ('accounts', 0.027), ('jump', 0.027), ('produce', 0.026), ('similarity', 0.026), ('tasks', 0.026), ('house', 0.025), ('jumps', 0.025), ('account', 0.024), ('outgoing', 0.024), ('displays', 0.024), ('intriguingly', 0.024), ('moscovitch', 0.024), ('four', 0.023), ('proposal', 0.023), ('responses', 0.022), ('semantically', 0.022), ('walks', 0.021), ('preceding', 0.021), ('departures', 0.021), ('links', 0.02), ('exploring', 0.02), ('relative', 0.02), ('predicted', 0.02), ('shorter', 0.02), ('nodes', 0.019), ('processes', 0.019), ('neuropsychological', 0.019), ('subcategories', 0.019), ('huntington', 0.019), ('visited', 0.019), ('simulation', 0.019), ('explanation', 0.019), ('retrieving', 0.018), ('behavior', 0.018), ('evidence', 0.017), ('clinical', 0.017), ('disease', 0.017), ('relevant', 0.017), ('reproduced', 0.017)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.99999958 155 nips-2012-Human memory search as a random walk in a semantic network

Author: Joseph L. Austerweil, Joshua T. Abbott, Thomas L. Griffiths

2 0.10649782 306 nips-2012-Semantic Kernel Forests from Multiple Taxonomies

Author: Sung J. Hwang, Kristen Grauman, Fei Sha

Abstract: When learning features for complex visual recognition problems, labeled image exemplars alone can be insufﬁcient. While an object taxonomy specifying the categories’ semantic relationships could bolster the learning process, not all relationships are relevant to a given visual classiﬁcation task, nor does a single taxonomy capture all ties that are relevant. In light of these issues, we propose a discriminative feature learning approach that leverages multiple hierarchical taxonomies representing different semantic views of the object categories (e.g., for animal classes, one taxonomy could reﬂect their phylogenic ties, while another could reﬂect their habitats). For each taxonomy, we ﬁrst learn a tree of semantic kernels, where each node has a Mahalanobis kernel optimized to distinguish between the classes in its children nodes. Then, using the resulting semantic kernel forest, we learn class-speciﬁc kernel combinations to select only those relationships relevant to recognize each object class. To learn the weights, we introduce a novel hierarchical regularization term that further exploits the taxonomies’ structure. We demonstrate our method on challenging object recognition datasets, and show that interleaving multiple taxonomic views yields signiﬁcant accuracy improvements.

3 0.081621729 165 nips-2012-Iterative ranking from pair-wise comparisons

Author: Sahand Negahban, Sewoong Oh, Devavrat Shah

Abstract: The question of aggregating pairwise comparisons to obtain a global ranking over a collection of objects has been of interest for a very long time: be it ranking of online gamers (e.g. MSR’s TrueSkill system) and chess players, aggregating social opinions, or deciding which product to sell based on transactions. In most settings, in addition to obtaining ranking, ﬁnding ‘scores’ for each object (e.g. player’s rating) is of interest to understanding the intensity of the preferences. In this paper, we propose a novel iterative rank aggregation algorithm for discovering scores for objects from pairwise comparisons. The algorithm has a natural random walk interpretation over the graph of objects with edges present between two objects if they are compared; the scores turn out to be the stationary probability of this random walk. The algorithm is model independent. To establish the efﬁcacy of our method, however, we consider the popular Bradley-Terry-Luce (BTL) model in which each object has an associated score which determines the probabilistic outcomes of pairwise comparisons between objects. We bound the ﬁnite sample error rates between the scores assumed by the BTL model and those estimated by our algorithm. This, in essence, leads to order-optimal dependence on the number of samples required to learn the scores well by our algorithm. Indeed, the experimental evaluation shows that our (model independent) algorithm performs as well as the Maximum Likelihood Estimator of the BTL model and outperforms a recently proposed algorithm by Ammar and Shah [1]. 1

4 0.072316825 178 nips-2012-Learning Label Trees for Probabilistic Modelling of Implicit Feedback

Author: Andriy Mnih, Yee W. Teh

Abstract: User preferences for items can be inferred from either explicit feedback, such as item ratings, or implicit feedback, such as rental histories. Research in collaborative ﬁltering has concentrated on explicit feedback, resulting in the development of accurate and scalable models. However, since explicit feedback is often difﬁcult to collect it is important to develop effective models that take advantage of the more widely available implicit feedback. We introduce a probabilistic approach to collaborative ﬁltering with implicit feedback based on modelling the user’s item selection process. In the interests of scalability, we restrict our attention to treestructured distributions over items and develop a principled and efﬁcient algorithm for learning item trees from data. We also identify a problem with a widely used protocol for evaluating implicit feedback models and propose a way of addressing it using a small quantity of explicit feedback data. 1

5 0.06786783 356 nips-2012-Unsupervised Structure Discovery for Semantic Analysis of Audio

Author: Sourish Chaudhuri, Bhiksha Raj

Abstract: Approaches to audio classiﬁcation and retrieval tasks largely rely on detectionbased discriminative models. We submit that such models make a simplistic assumption in mapping acoustics directly to semantics, whereas the actual process is likely more complex. We present a generative model that maps acoustics in a hierarchical manner to increasingly higher-level semantics. Our model has two layers with the ﬁrst layer modeling generalized sound units with no clear semantic associations, while the second layer models local patterns over these sound units. We evaluate our model on a large-scale retrieval task from TRECVID 2011, and report signiﬁcant improvements over standard baselines. 1

6 0.061691254 153 nips-2012-How Prior Probability Influences Decision Making: A Unifying Probabilistic Model

7 0.058813743 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

8 0.057326119 126 nips-2012-FastEx: Hash Clustering with Exponential Families

9 0.055960476 69 nips-2012-Clustering Sparse Graphs

10 0.051155321 70 nips-2012-Clustering by Nonnegative Matrix Factorization Using Graph Random Walk

11 0.049821232 289 nips-2012-Recognizing Activities by Attribute Dynamics

12 0.048842516 111 nips-2012-Efficient Sampling for Bipartite Matching Problems

13 0.048743751 49 nips-2012-Automatic Feature Induction for Stagewise Collaborative Filtering

14 0.04816445 316 nips-2012-Small-Variance Asymptotics for Exponential Family Dirichlet Process Mixture Models

15 0.047234699 124 nips-2012-Factorial LDA: Sparse Multi-Dimensional Text Models

16 0.046510007 107 nips-2012-Effective Split-Merge Monte Carlo Methods for Nonparametric Models of Sequential Data

17 0.04472312 148 nips-2012-Hamming Distance Metric Learning

18 0.043904904 60 nips-2012-Bayesian nonparametric models for ranked data

19 0.04219972 196 nips-2012-Learning with Partially Absorbing Random Walks

20 0.041638717 68 nips-2012-Clustering Aggregation as Maximum-Weight Independent Set

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.106), (1, 0.014), (2, -0.055), (3, -0.02), (4, -0.042), (5, -0.044), (6, -0.003), (7, 0.019), (8, 0.019), (9, 0.109), (10, -0.007), (11, -0.046), (12, -0.043), (13, -0.008), (14, 0.026), (15, -0.006), (16, -0.046), (17, 0.045), (18, -0.012), (19, -0.044), (20, -0.008), (21, -0.017), (22, -0.026), (23, -0.065), (24, 0.031), (25, -0.046), (26, 0.036), (27, -0.005), (28, 0.053), (29, 0.012), (30, 0.106), (31, 0.017), (32, -0.01), (33, -0.083), (34, 0.04), (35, 0.012), (36, 0.017), (37, 0.005), (38, -0.018), (39, -0.032), (40, 0.065), (41, 0.072), (42, -0.037), (43, 0.025), (44, -0.025), (45, -0.051), (46, 0.004), (47, -0.013), (48, 0.003), (49, -0.06)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.92569596 155 nips-2012-Human memory search as a random walk in a semantic network

Author: Joseph L. Austerweil, Joshua T. Abbott, Thomas L. Griffiths

2 0.54998481 178 nips-2012-Learning Label Trees for Probabilistic Modelling of Implicit Feedback

Author: Andriy Mnih, Yee W. Teh

3 0.54702747 356 nips-2012-Unsupervised Structure Discovery for Semantic Analysis of Audio

Author: Sourish Chaudhuri, Bhiksha Raj

4 0.51201653 165 nips-2012-Iterative ranking from pair-wise comparisons

Author: Sahand Negahban, Sewoong Oh, Devavrat Shah

5 0.49695909 299 nips-2012-Scalable imputation of genetic data with a discrete fragmentation-coagulation process

Author: Lloyd Elliott, Yee W. Teh

Abstract: We present a Bayesian nonparametric model for genetic sequence data in which a set of genetic sequences is modelled using a Markov model of partitions. The partitions at consecutive locations in the genome are related by the splitting and merging of their clusters. Our model can be thought of as a discrete analogue of the continuous fragmentation-coagulation process [Teh et al 2011], preserving the important properties of projectivity, exchangeability and reversibility, while being more scalable. We apply this model to the problem of genotype imputation, showing improved computational efﬁciency while maintaining accuracies comparable to other state-of-the-art genotype imputation methods. 1

6 0.47667903 49 nips-2012-Automatic Feature Induction for Stagewise Collaborative Filtering

7 0.46530613 107 nips-2012-Effective Split-Merge Monte Carlo Methods for Nonparametric Models of Sequential Data

8 0.46118578 46 nips-2012-Assessing Blinding in Clinical Trials

9 0.45557594 75 nips-2012-Collaborative Ranking With 17 Parameters

10 0.44417837 99 nips-2012-Dip-means: an incremental clustering method for estimating the number of clusters

11 0.44109991 111 nips-2012-Efficient Sampling for Bipartite Matching Problems

12 0.42962348 276 nips-2012-Probabilistic Event Cascades for Alzheimer's disease

13 0.42926183 68 nips-2012-Clustering Aggregation as Maximum-Weight Independent Set

14 0.42682567 126 nips-2012-FastEx: Hash Clustering with Exponential Families

15 0.42298356 342 nips-2012-The variational hierarchical EM algorithm for clustering hidden Markov models

16 0.41197437 196 nips-2012-Learning with Partially Absorbing Random Walks

17 0.4114846 219 nips-2012-Modelling Reciprocating Relationships with Hawkes Processes

18 0.40685633 7 nips-2012-A Divide-and-Conquer Method for Sparse Inverse Covariance Estimation

19 0.40369993 306 nips-2012-Semantic Kernel Forests from Multiple Taxonomies

20 0.40309367 291 nips-2012-Reducing statistical time-series problems to binary classification

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.041), (17, 0.012), (21, 0.021), (36, 0.014), (38, 0.123), (39, 0.013), (42, 0.021), (54, 0.027), (55, 0.403), (74, 0.039), (76, 0.095), (80, 0.06), (92, 0.027)]

similar papers list:

simIndex simValue paperId paperTitle

1 0.83793145 340 nips-2012-The representer theorem for Hilbert spaces: a necessary and sufficient condition

Author: Francesco Dinuzzo, Bernhard Schölkopf

Abstract: The representer theorem is a property that lies at the foundation of regularization theory and kernel methods. A class of regularization functionals is said to admit a linear representer theorem if every member of the class admits minimizers that lie in the ﬁnite dimensional subspace spanned by the representers of the data. A recent characterization states that certain classes of regularization functionals with differentiable regularization term admit a linear representer theorem for any choice of the data if and only if the regularization term is a radial nondecreasing function. In this paper, we extend such result by weakening the assumptions on the regularization term. In particular, the main result of this paper implies that, for a sufﬁciently large family of regularization functionals, radial nondecreasing functions are the only lower semicontinuous regularization terms that guarantee existence of a representer theorem for any choice of the data. 1

2 0.83678311 158 nips-2012-ImageNet Classification with Deep Convolutional Neural Networks

Author: Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton

Abstract: We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of ﬁve convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a ﬁnal 1000-way softmax. To make training faster, we used non-saturating neurons and a very efﬁcient GPU implementation of the convolution operation. To reduce overﬁtting in the fully-connected layers we employed a recently-developed regularization method called “dropout” that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry. 1

3 0.83574384 52 nips-2012-Bayesian Nonparametric Modeling of Suicide Attempts

Author: Francisco Ruiz, Isabel Valera, Carlos Blanco, Fernando Pérez-Cruz

Abstract: The National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) database contains a large amount of information, regarding the way of life, medical conditions, etc., of a representative sample of the U.S. population. In this paper, we are interested in seeking the hidden causes behind the suicide attempts, for which we propose to model the subjects using a nonparametric latent model based on the Indian Buffet Process (IBP). Due to the nature of the data, we need to adapt the observation model for discrete random variables. We propose a generative model in which the observations are drawn from a multinomial-logit distribution given the IBP matrix. The implementation of an efﬁcient Gibbs sampler is accomplished using the Laplace approximation, which allows integrating out the weighting factors of the multinomial-logit likelihood model. Finally, the experiments over the NESARC database show that our model properly captures some of the hidden causes that model suicide attempts. 1

same-paper 4 0.83459395 155 nips-2012-Human memory search as a random walk in a semantic network

Author: Joseph L. Austerweil, Joshua T. Abbott, Thomas L. Griffiths

5 0.75914252 211 nips-2012-Meta-Gaussian Information Bottleneck

Author: Melanie Rey, Volker Roth

Abstract: We present a reformulation of the information bottleneck (IB) problem in terms of copula, using the equivalence between mutual information and negative copula entropy. Focusing on the Gaussian copula we extend the analytical IB solution available for the multivariate Gaussian case to distributions with a Gaussian dependence structure but arbitrary marginal densities, also called meta-Gaussian distributions. This opens new possibles applications of IB to continuous data and provides a solution more robust to outliers. 1

6 0.67959827 215 nips-2012-Minimizing Uncertainty in Pipelines

7 0.66079074 95 nips-2012-Density-Difference Estimation

8 0.55321074 306 nips-2012-Semantic Kernel Forests from Multiple Taxonomies

9 0.5459857 231 nips-2012-Multiple Operator-valued Kernel Learning

10 0.54016781 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

11 0.52949363 91 nips-2012-Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images

12 0.52636224 193 nips-2012-Learning to Align from Scratch

13 0.51749998 90 nips-2012-Deep Learning of Invariant Features via Simulated Fixations in Video

14 0.51472932 298 nips-2012-Scalable Inference of Overlapping Communities

15 0.50152218 4 nips-2012-A Better Way to Pretrain Deep Boltzmann Machines

16 0.50057191 210 nips-2012-Memorability of Image Regions

17 0.49960417 188 nips-2012-Learning from Distributions via Support Measure Machines

18 0.49706486 345 nips-2012-Topic-Partitioned Multinetwork Embeddings

19 0.49542066 93 nips-2012-Deep Spatio-Temporal Architectures and Learning for Protein Structure Prediction

20 0.49493483 190 nips-2012-Learning optimal spike-based representations