nips nips2012 nips2012-273 knowledge-graph by maker-knowledge-mining

273 nips-2012-Predicting Action Content On-Line and in Real Time before Action Onset – an Intracranial Human Study

Source: pdf

Author: Uri Maoz, Shengxuan Ye, Ian Ross, Adam Mamelak, Christof Koch

Abstract: The ability to predict action content from neural signals in real time before the action occurs has been long sought in the neuroscientiﬁc study of decision-making, agency and volition. On-line real-time (ORT) prediction is important for understanding the relation between neural correlates of decision-making and conscious, voluntary action as well as for brain-machine interfaces. Here, epilepsy patients, implanted with intracranial depth microelectrodes or subdural grid electrodes for clinical purposes, participated in a “matching-pennies” game against an opponent. In each trial, subjects were given a 5 s countdown, after which they had to raise their left or right hand immediately as the “go” signal appeared on a computer screen. They won a ﬁxed amount of money if they raised a different hand than their opponent and lost that amount otherwise. The question we here studied was the extent to which neural precursors of the subjects’ decisions can be detected in intracranial local ﬁeld potentials (LFP) prior to the onset of the action. We found that combined low-frequency (0.1–5 Hz) LFP signals from 10 electrodes were predictive of the intended left-/right-hand movements before the onset of the go signal. Our ORT system predicted which hand the patient would raise 0.5 s before the go signal with 68±3% accuracy in two patients. Based on these results, we constructed an ORT system that tracked up to 30 electrodes simultaneously, and tested it on retrospective data from 7 patients. On average, we could predict the correct hand choice in 83% of the trials, which rose to 92% if we let the system drop 3/10 of the trials on which it was less conﬁdent. Our system demonstrates— for the ﬁrst time—the feasibility of accurately predicting a binary action on single trials in real time for patients with intracranial recordings, well before the action occurs. 1 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 edu Abstract The ability to predict action content from neural signals in real time before the action occurs has been long sought in the neuroscientiﬁc study of decision-making, agency and volition. [sent-8, score-0.325]

2 On-line real-time (ORT) prediction is important for understanding the relation between neural correlates of decision-making and conscious, voluntary action as well as for brain-machine interfaces. [sent-9, score-0.393]

3 Here, epilepsy patients, implanted with intracranial depth microelectrodes or subdural grid electrodes for clinical purposes, participated in a “matching-pennies” game against an opponent. [sent-10, score-0.827]

4 In each trial, subjects were given a 5 s countdown, after which they had to raise their left or right hand immediately as the “go” signal appeared on a computer screen. [sent-11, score-0.354]

5 They won a ﬁxed amount of money if they raised a different hand than their opponent and lost that amount otherwise. [sent-12, score-0.216]

6 The question we here studied was the extent to which neural precursors of the subjects’ decisions can be detected in intracranial local ﬁeld potentials (LFP) prior to the onset of the action. [sent-13, score-0.409]

7 1–5 Hz) LFP signals from 10 electrodes were predictive of the intended left-/right-hand movements before the onset of the go signal. [sent-15, score-0.684]

8 Our ORT system predicted which hand the patient would raise 0. [sent-16, score-0.467]

9 5 s before the go signal with 68±3% accuracy in two patients. [sent-17, score-0.257]

10 Based on these results, we constructed an ORT system that tracked up to 30 electrodes simultaneously, and tested it on retrospective data from 7 patients. [sent-18, score-0.36]

11 On average, we could predict the correct hand choice in 83% of the trials, which rose to 92% if we let the system drop 3/10 of the trials on which it was less conﬁdent. [sent-19, score-0.385]

12 Our system demonstrates— for the ﬁrst time—the feasibility of accurately predicting a binary action on single trials in real time for patients with intracranial recordings, well before the action occurs. [sent-20, score-0.755]

13 1 1 Introduction The work of Benjamin Libet [1, 2] and others [3, 4] has challenged our intuitive notions of the relation between decision making and conscious voluntary action. [sent-21, score-0.289]

14 Using electrocorticography (EEG), these experiments measured brain potentials from subjects that were instructed to ﬂex their wrist at a time of their choice and note the position of a rotating dot on a clock when they felt the urge to move. [sent-22, score-0.206]

15 The results suggested that a slow cortical wave measured over motor areas—termed “readiness potential” [5], and known to precede voluntary movement [6]—begins a few hundred milliseconds before the average reported time of the subjective ‘urge’ to move. [sent-23, score-0.421]

16 This suggested that action onset and contents could be decoded from preparatory motor signals in the brain before the subject becomes aware of an intention to move and of the contents of the action. [sent-24, score-0.851]

17 However, the readiness potential was computed by averaging over 40 or more trials aligned to movement onset after the fact. [sent-25, score-0.612]

18 More recently, it was shown that action contents can be decoded using functional magnetic-resonance imaging (fMRI) several seconds before movement onset [7]. [sent-26, score-0.595]

19 But, while done on a single-trial basis, decoding the neural signals took place off-line, after the experiment was concluded, as the sluggish nature of fMRI hemodynamic signals precluded real-time analysis. [sent-27, score-0.16]

20 Moreover, the above studies focused on arbitrary and meaningless action—purposelessly raising the left or right hand—while we wanted to investigate prediction of reasoned action in more realistic, everyday situations with consequences for the subject. [sent-28, score-0.235]

21 Intracranial recordings are good candidates for single-trial, ORT analysis of action onset and contents [8, 9], because of the tight temporal pairing of LFP to the underlying neuronal signals. [sent-29, score-0.479]

22 We therefore took advantage of a rare opportunity to work with epilepsy patients implanted with intracranial electrodes for clinical purposes. [sent-31, score-0.765]

23 1) predicts, with far above chance accuracy, which one of two future actions is about to occur on this one trial and feeds the prediction back to the experimenter, all before the onset of the go signal that triggers the patient’s movement (see Experimental Methods). [sent-33, score-0.863]

24 We achieve relatively high prediction performance using only part of the data—learning from brain activity in past trials only (Fig. [sent-34, score-0.302]

25 1 Experimental Methods Subjects Subjects in this experiment were 8 consenting intractable epilepsy patients that were implanted with intracranial electrodes as part of their presurgical clinical evaluation (ages 18–60, 3 males). [sent-38, score-0.765]

26 They were inpatients in the neuro-telemetry ward at the Cedars Sinai Medical Center or the Huntington Memorial Hospital, and are designated with CS or HMH after their patient numbers, respectively. [sent-39, score-0.204]

27 Six of them—P12CS, P15CS, P22CS and P29–31HMH were implanted with intracortical depth electrodes targeting their bilateral anterior-cingulate cortex, amygdala, hippocampus and orbitofrontal cortex. [sent-40, score-0.323]

28 These electrodes had eight 40 µm microwires at their tips, 7 for recording and 1 serving as a local ground. [sent-41, score-0.329]

29 4) displaying the game screen (bottom right inset of Fig. [sent-49, score-0.265]

30 2 Experiment Design As part of our focus on purposeful, reasoned action, we had the subjects play a matching-pennies game—a 2-choice version of “rock paper scissors”—either against the experimenter or against a computer. [sent-55, score-0.428]

31 The subjects pressed down a button with their left hand and another with their right on a response box. [sent-56, score-0.206]

32 Then, in each trial, there was a 5 s countdown followed by a go signal, after which they had to immediately lift one of their hands. [sent-57, score-0.243]

33 It was agreed beforehand that the patient would win the trial if she lifted a different hand than her opponent, and lose if she raised the same hand as her opponent. [sent-58, score-0.516]

34 If a player lifted her hand before the go signal, did not lift her hand within 500 ms of the go signal, or lifted no hand or both hands at the go signal— an error trial—she lost $0. [sent-61, score-0.839]

35 The subjects were shown the countdown, the go signal, the overall score, and various instructions on a stimulus computer placed before them (Fig. [sent-63, score-0.243]

36 If, at the end of the game, the subject had more money than her opponent, she received that money in cash from the experimenter. [sent-66, score-0.166]

37 Before the experimental session began, the experimenter explained the rules of the game to the subject, and she could practice playing the game until she was familiar with it. [sent-67, score-0.591]

38 Consequently, patients usually made only few errors during the games (<6% of the trials). [sent-68, score-0.197]

39 Following the tutorial, the subject played 1–3 games against the computer and then once against the experimenter, depending on their availability and clinical circumstances. [sent-69, score-0.24]

40 The ﬁrst 2 games of P12CS were removed because the subject tended to constantly raise the right hand regardless of winning or losing. [sent-70, score-0.286]

41 In such sessions—3 games each—the subjects always played against the experimenter. [sent-72, score-0.224]

42 Half a second before go-signal onset, an arrow pointing towards the hand that the system predicted the experimenter had to raise to win the trial was displayed on that screen. [sent-77, score-0.701]

43 Simultaneously, a monophonic tone was played in the experimenter’s earphone ipsilateral to that hand. [sent-78, score-0.19]

44 The experimenter then lifted that hand at the go signal (see Supplemental Movie). [sent-79, score-0.635]

45 Neural signals ﬂow from the patient through the Cheetah machine to the analysis/stimulus computer, which controls the input and output of the game and computes the prediction of the hand the patient would raise at the go signal. [sent-81, score-1.046]

46 It displays it on a screen behind the patient and informs the experimenter which hand to raise by playing a tone in his ipsilateral ear using earphones. [sent-82, score-0.951]

47 1 The real-time system Hardware and software overview µV µV µV Neural data from the intracranial electrodes were transferred to a recording system (Neuralynx, Digital Lynx), where it was collected and saved to the local Cheetah machine, down sampled from 32 kHz to 2 kHz and buffered. [sent-84, score-0.627]

48 We found that this frequency range— generally comparable to that of the readiness potential—resulted in optimal prediction performance. [sent-90, score-0.152]

49 This computer also controlled the game screen, displaying the names of the players, their current scores and various instructions. [sent-92, score-0.153]

50 The buttons of the subject and her opponent 600 ﬂashed red or blue whenever she or her −5 −4 −3 −2 −1 0 opponent won, respectively. [sent-94, score-0.262]

51 Addition(b)100 ally, the analysis/stimulus computer sent 0 a unique transistor-transistor logic (TTL) −100 −200 pulse whenever the game screen changed −5 −4 −3 −2 −1 0 or a button was pressed on the response box, which synchronized the timing of (c) 100 0 these events with the LFP recordings. [sent-95, score-0.301]

52 −100 In real-time game sessions, the analy−200 −5 −4 −3 −2 −1 0 sis/stimulus computer also displayed the appropriate arrow on the computer screen (d) 1 behind the subject and played the tone 0 to the appropriate ear of the experimenter −1 0. [sent-96, score-0.81]

53 −5 −4 −3 −2 −1 0 The analysis software was based on a machine-learning algorithm that trained on past-trials data to predict the current trial and is detailed below. [sent-99, score-0.153]

54 The training phase included the ﬁrst 70% of the trials, with the prediction carried out on the remaining 30% using the trained parameters, together with an online weighting system (see below). [sent-100, score-0.22]

55 2b), the system found the mean and standard error over all leftward and rightward training trials, separately (Fig. [sent-103, score-0.222]

56 It then found the electrodes and time windows where the left/right separation was high (Fig. [sent-105, score-0.402]

57 The best electrode/time-window/classiﬁer (ETC) combinations were then used to predict the current trial in the prediction phase (Fig. [sent-108, score-0.283]

58 El 49−T1 (e) El 49−T2 El 49−T3 1 0 −1 −5 −4 −3 −2 −1 Countdown to go signal at t=0 (seconds) 0 (f) Classifier Cf1 Classifier Cf2 . [sent-111, score-0.22]

59 Mean±standard errors of signals preceeding left- and right-hand movments (c) are used to compute a left/right separability index (d), from which time windows with good separation are found (e). [sent-122, score-0.249]

60 Seven classiﬁers are then applied to all the time windows (f) and the best electrode/time-window/classiﬁer combinations are selected (g) and used in the prediction phase (Fig. [sent-123, score-0.259]

61 5 seconds before the go signal—is received in real time, and each electrode/time-window/classiﬁer combination (ETC) classiﬁes it as resulting in left- or right-hand movement. [sent-127, score-0.141]

62 2 Computing optimal left/right-separating time windows The algorithm focused on ﬁnding the time windows with the best left/right separation for the different recording electrodes over the training set (Fig. [sent-130, score-0.566]

63 That is, we wanted to predict whether the signal aN (t) on trial N will result in a leftward or rightward movement—i. [sent-132, score-0.364]

64 For each electrode, we looked at the N − 1 previous trials a1 (t), a2 (t), . [sent-135, score-0.148]

65 Now, let N −1 L(t) = {ai (t) | li = Lt}N −1 and R(t) = {ai (t) | li = Rt}i=1 be the set of previous leftward and i=1 rightward trials in the training set, respectively. [sent-142, score-0.28]

66 We deﬁne a consecutive time period of |δ(t)| > 0 for t < prediction time (the time before the go signal when we want the system to output a prediction; -0. [sent-146, score-0.401]

67 After all time windows are found for all electrodes, time windows less than M ms t apart are combined into one. [sent-149, score-0.301]

68 This resulted in 20–30 time windows over all 64 electrodes that we monitored. [sent-153, score-0.4]

69 At 400 ms before the go signal, the patient and experimenter are watching the game screen (inset on bottom right) on the analysis/stimulus computer (bottom left) and still pressing down the buttons of the response box. [sent-157, score-0.974]

70 The realtime system already computed a prediction, and thus displays an arrow on the screen behind the patient and plays a tone in the experimenter’s ear ipsilateral to the hand it predicts he should raise to beat the patient (see Supplemental Movie). [sent-158, score-1.04]

71 4 The prediction-phase weighting system In the prediction phase, each of the overall S binary ETCs calculates a prediction, ci ∈ {−1, 1} (for right and left, respectively), independently at the desired prediction time. [sent-173, score-0.272]

72 We then calculate ξ = i=1 wi · ci and predict left (right) if ξ > d (ξ < −d), or declare it an undetermined trial if −d < ξ < d. [sent-175, score-0.153]

73 Thus the larger d is, the more conﬁdent the system needs to be to make a prediction, and the larger the proportion of trials on which the system abstains—the dropoff rate. [sent-177, score-0.328]

74 The neural signals were collected by the Digital Lynx S system using Cheetah 5. [sent-183, score-0.17]

75 The simulated-ORT analyses carried out in this paper used real patient data saved on the Digital Lynx system. [sent-187, score-0.204]

76 5 0 Go-signal onset Figure 5: Across-subjects average of the prediction accuracy of simulated-ORT versus time before the go signal. [sent-200, score-0.499]

77 The mean accuracies over time when the system predicts on every trial, is allowed to drop 19% or 30% of the trials, are depicted in blue, green and red, respectively (±standard error shaded). [sent-201, score-0.215]

78 4 Results We tested our prediction system in actual real time on 2 patients—P15CS and P19CS (a depth and grid patient, respectively), with a prediction time of 0. [sent-204, score-0.309]

79 Because of computational limitations, the ORT system could only track 10 electrodes with just 1 ETC per electrode in real time. [sent-206, score-0.408]

80 For P15CS, we achieved an accuracy of 72±2% (±standard error; accuracy = number of accurately predicted trials / [total number of trials - number of dropped trials]; p = 10−8 , binomial test) without modifying the weights online during the prediction (see Section 3. [sent-207, score-0.461]

81 The prediction accuracy was signiﬁcantly above chance 63±2% (±standard error; p = 7 · 10−4 , binomial test). [sent-210, score-0.163]

82 To understand how much we could improve our accuracy with optimized hardware/software, we ran the simulated-ORT at various prediction times along 7 Accuracy the 5 s countdown leading to the go signal. [sent-211, score-0.371]

83 5; drop-off rate = number of dropped trials / total number of trials; these resulted from 3 drop-off thresholds—0, 0. [sent-215, score-0.186]

84 Averaged over all subjects, the accuracy rose from about 65% more than 1 4 s before the go signal to 83–92% close to go-signal onset, depending 0. [sent-221, score-0.257]

85 This suggests that the linear SVM and L2-norm comparisons (of aN to Lm and Rm ) together make up Figure 6: Simulated-ORT accuracy over time for individual nearly half of the overall weights at- patients with no drop off. [sent-245, score-0.209]

86 5 Discussion We constructed an ORT system that, based on intracranial recordings, predicted which hand a person would raise well before movement onset at accuracies much greater than chance in a competitive environment. [sent-247, score-0.924]

87 We further tested this system off-line, which suggested that with optimized hardware/software, such action contents would be predictable in real time at relatively high accuracies already several seconds before movement onset. [sent-248, score-0.536]

88 Both our prediction accuracy and drop-off rates close to movement onset are superior to those achieved before movement onset with noninvasive methods like EEG and fMRI [7, 12–14]. [sent-249, score-0.934]

89 Importantly, our subjects played a matching pennies game—a 2-choice version of rock-paper-scissors [15]—to keep their task realistic, with minor though real consequences, unlike the Libet-type paradigms whose outcome bears no consequences for the subjects. [sent-250, score-0.159]

90 It was suggested that accurate online, real-time prediction before movement onset is key to investigating the relation between the neural correlates of decisions, their awareness, and voluntary action [16, 17]. [sent-251, score-0.796]

91 For example, it would make it possible to study decision reversals on a single-trial basis, or to test whether subjects can guess above chance which of their action contents are predictable from their current brain activity, potentially before having consciously made up their mind [16, 18]. [sent-253, score-0.392]

92 Accurately decoding these preparatory motor signals may also result in earlier and improved classiﬁcation for brain-computer interfaces [13, 19, 20]. [sent-254, score-0.17]

93 Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential): The unconscious initiation of a freely voluntary act. [sent-265, score-0.639]

94 Unconscious cerebral initiative and the role of conscious will in voluntary action. [sent-269, score-0.289]

95 On the relation between brain potentials and the awareness of voluntary movements. [sent-274, score-0.304]

96 Altered awareness of voluntary action after damage to the parietal cortex. [sent-284, score-0.344]

97 Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. [sent-338, score-0.253]

98 Exploration of computational methods for classiﬁcation of movement intention during human voluntary movement from single trial EEG. [sent-361, score-0.718]

99 Using single-trial EEG data to predict laterality of voluntary motor decisions. [sent-368, score-0.287]

100 Detection of self-paced reaching movement intention from EEG signals. [sent-405, score-0.232]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('experimenter', 0.285), ('ort', 0.264), ('electrodes', 0.233), ('onset', 0.23), ('patient', 0.204), ('voluntary', 0.199), ('intracranial', 0.179), ('movement', 0.173), ('lm', 0.171), ('game', 0.153), ('trials', 0.148), ('go', 0.141), ('patients', 0.132), ('windows', 0.129), ('etcs', 0.122), ('trial', 0.114), ('screen', 0.112), ('el', 0.109), ('raise', 0.105), ('action', 0.103), ('subjects', 0.102), ('countdown', 0.102), ('prediction', 0.091), ('system', 0.09), ('conscious', 0.09), ('lfp', 0.09), ('implanted', 0.09), ('contents', 0.089), ('opponent', 0.089), ('electrode', 0.085), ('cheetah', 0.081), ('signals', 0.08), ('signal', 0.079), ('etc', 0.074), ('eeg', 0.074), ('pasadena', 0.072), ('tone', 0.072), ('clinical', 0.07), ('hand', 0.068), ('rm', 0.067), ('leftward', 0.066), ('rightward', 0.066), ('games', 0.065), ('brain', 0.063), ('lifted', 0.062), ('epilepsy', 0.061), ('ipsilateral', 0.061), ('lynx', 0.061), ('microwires', 0.061), ('readiness', 0.061), ('unconscious', 0.061), ('intention', 0.059), ('money', 0.059), ('played', 0.057), ('recordings', 0.057), ('hospital', 0.057), ('sign', 0.054), ('window', 0.052), ('classifier', 0.05), ('huntington', 0.05), ('motor', 0.049), ('subject', 0.048), ('ls', 0.047), ('ers', 0.047), ('koch', 0.047), ('player', 0.045), ('ear', 0.044), ('accuracies', 0.044), ('rs', 0.043), ('ms', 0.043), ('awareness', 0.042), ('predicts', 0.041), ('fmri', 0.041), ('bereitschaftspotential', 0.041), ('cedars', 0.041), ('etci', 0.041), ('libet', 0.041), ('maoz', 0.041), ('neuralynx', 0.041), ('preparatory', 0.041), ('reasoned', 0.041), ('subdural', 0.041), ('ttl', 0.041), ('und', 0.041), ('urge', 0.041), ('drop', 0.04), ('separation', 0.04), ('arrow', 0.039), ('phase', 0.039), ('predict', 0.039), ('resulted', 0.038), ('median', 0.038), ('accuracy', 0.037), ('tested', 0.037), ('buttons', 0.036), ('pressed', 0.036), ('ralph', 0.036), ('sinai', 0.036), ('chance', 0.035), ('recording', 0.035)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.99999917 273 nips-2012-Predicting Action Content On-Line and in Real Time before Action Onset – an Intracranial Human Study

Author: Uri Maoz, Shengxuan Ye, Ian Ross, Adam Mamelak, Christof Koch

2 0.17915261 266 nips-2012-Patient Risk Stratification for Hospital-Associated C. diff as a Time-Series Classification Task

Author: Jenna Wiens, Eric Horvitz, John V. Guttag

Abstract: A patient’s risk for adverse events is affected by temporal processes including the nature and timing of diagnostic and therapeutic activities, and the overall evolution of the patient’s pathophysiology over time. Yet many investigators ignore this temporal aspect when modeling patient outcomes, considering only the patient’s current or aggregate state. In this paper, we represent patient risk as a time series. In doing so, patient risk stratiﬁcation becomes a time-series classiﬁcation task. The task differs from most applications of time-series analysis, like speech processing, since the time series itself must ﬁrst be extracted. Thus, we begin by deﬁning and extracting approximate risk processes, the evolving approximate daily risk of a patient. Once obtained, we use these signals to explore different approaches to time-series classiﬁcation with the goal of identifying high-risk patterns. We apply the classiﬁcation to the speciﬁc task of identifying patients at risk of testing positive for hospital acquired Clostridium difﬁcile. We achieve an area under the receiver operating characteristic curve of 0.79 on a held-out set of several hundred patients. Our two-stage approach to risk stratiﬁcation outperforms classiﬁers that consider only a patient’s current state (p<0.05). 1

3 0.16356587 50 nips-2012-Bandit Algorithms boost Brain Computer Interfaces for motor-task selection of a brain-controlled button

Author: Joan Fruitet, Alexandra Carpentier, Maureen Clerc, Rémi Munos

Abstract: Brain-computer interfaces (BCI) allow users to “communicate” with a computer without using their muscles. BCI based on sensori-motor rhythms use imaginary motor tasks, such as moving the right or left hand, to send control signals. The performances of a BCI can vary greatly across users but also depend on the tasks used, making the problem of appropriate task selection an important issue. This study presents a new procedure to automatically select as fast as possible a discriminant motor task for a brain-controlled button. We develop for this purpose an adaptive algorithm, UCB-classif , based on the stochastic bandit theory. This shortens the training stage, thereby allowing the exploration of a greater variety of tasks. By not wasting time on inefﬁcient tasks, and focusing on the most promising ones, this algorithm results in a faster task selection and a more efﬁcient use of the BCI training session. Comparing the proposed method to the standard practice in task selection, for a ﬁxed time budget, UCB-classif leads to an improved classiﬁcation rate, and for a ﬁxed classiﬁcation rate, to a reduction of the time spent in training by 50%. 1

4 0.11857496 303 nips-2012-Searching for objects driven by context

Author: Bogdan Alexe, Nicolas Heess, Yee W. Teh, Vittorio Ferrari

Abstract: The dominant visual search paradigm for object class detection is sliding windows. Although simple and effective, it is also wasteful, unnatural and rigidly hardwired. We propose strategies to search for objects which intelligently explore the space of windows by making sequential observations at locations decided based on previous observations. Our strategies adapt to the class being searched and to the content of a particular test image, exploiting context as the statistical relation between the appearance of a window and its location relative to the object, as observed in the training set. In addition to being more elegant than sliding windows, we demonstrate experimentally on the PASCAL VOC 2010 dataset that our strategies evaluate two orders of magnitude fewer windows while achieving higher object detection performance. 1

5 0.10968255 109 nips-2012-Efficient Monte Carlo Counterfactual Regret Minimization in Games with Many Player Actions

Author: Neil Burch, Marc Lanctot, Duane Szafron, Richard G. Gibson

Abstract: Counterfactual Regret Minimization (CFR) is a popular, iterative algorithm for computing strategies in extensive-form games. The Monte Carlo CFR (MCCFR) variants reduce the per iteration time cost of CFR by traversing a smaller, sampled portion of the tree. The previous most effective instances of MCCFR can still be very slow in games with many player actions since they sample every action for a given player. In this paper, we present a new MCCFR algorithm, Average Strategy Sampling (AS), that samples a subset of the player’s actions according to the player’s average strategy. Our new algorithm is inspired by a new, tighter bound on the number of iterations required by CFR to converge to a given solution quality. In addition, we prove a similar, tighter bound for AS and other popular MCCFR variants. Finally, we validate our work by demonstrating that AS converges faster than previous MCCFR algorithms in both no-limit poker and Bluff. 1

6 0.10507084 276 nips-2012-Probabilistic Event Cascades for Alzheimer's disease

7 0.087268896 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

8 0.082167886 198 nips-2012-Learning with Target Prior

9 0.074748866 344 nips-2012-Timely Object Recognition

10 0.071610644 348 nips-2012-Tractable Objectives for Robust Policy Optimization

11 0.071555242 153 nips-2012-How Prior Probability Influences Decision Making: A Unifying Probabilistic Model

12 0.062921844 14 nips-2012-A P300 BCI for the Masses: Prior Information Enables Instant Unsupervised Spelling

13 0.062038332 224 nips-2012-Multi-scale Hyper-time Hardware Emulation of Human Motor Nervous System Based on Spiking Neurons using FPGA

14 0.061737463 231 nips-2012-Multiple Operator-valued Kernel Learning

15 0.057577968 209 nips-2012-Max-Margin Structured Output Regression for Spatio-Temporal Action Localization

16 0.054344155 46 nips-2012-Assessing Blinding in Clinical Trials

17 0.048676163 200 nips-2012-Local Supervised Learning through Space Partitioning

18 0.047092196 157 nips-2012-Identification of Recurrent Patterns in the Activation of Brain Networks

19 0.046521846 28 nips-2012-A systematic approach to extracting semantic information from functional MRI data

20 0.046464588 313 nips-2012-Sketch-Based Linear Value Function Approximation

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.123), (1, -0.048), (2, -0.072), (3, 0.033), (4, 0.033), (5, 0.039), (6, -0.002), (7, 0.071), (8, -0.042), (9, 0.019), (10, -0.039), (11, -0.001), (12, 0.048), (13, -0.032), (14, 0.008), (15, -0.005), (16, -0.049), (17, 0.056), (18, -0.075), (19, -0.073), (20, 0.0), (21, 0.05), (22, -0.172), (23, -0.111), (24, 0.145), (25, -0.189), (26, 0.032), (27, 0.106), (28, 0.036), (29, 0.13), (30, 0.006), (31, 0.117), (32, -0.125), (33, 0.081), (34, -0.005), (35, -0.028), (36, 0.029), (37, -0.051), (38, -0.132), (39, 0.044), (40, -0.04), (41, -0.181), (42, -0.017), (43, 0.051), (44, -0.095), (45, -0.0), (46, -0.09), (47, 0.04), (48, 0.17), (49, -0.1)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.96273619 273 nips-2012-Predicting Action Content On-Line and in Real Time before Action Onset – an Intracranial Human Study

Author: Uri Maoz, Shengxuan Ye, Ian Ross, Adam Mamelak, Christof Koch

2 0.68450809 50 nips-2012-Bandit Algorithms boost Brain Computer Interfaces for motor-task selection of a brain-controlled button

Author: Joan Fruitet, Alexandra Carpentier, Maureen Clerc, Rémi Munos

3 0.58517462 28 nips-2012-A systematic approach to extracting semantic information from functional MRI data

Author: Francisco Pereira, Matthew Botvinick

Abstract: This paper introduces a novel classiﬁcation method for functional magnetic resonance imaging datasets with tens of classes. The method is designed to make predictions using information from as many brain locations as possible, instead of resorting to feature selection, and does this by decomposing the pattern of brain activation into differently informative sub-regions. We provide results over a complex semantic processing dataset that show that the method is competitive with state-of-the-art feature selection and also suggest how the method may be used to perform group or exploratory analyses of complex class structure. 1

4 0.56791335 14 nips-2012-A P300 BCI for the Masses: Prior Information Enables Instant Unsupervised Spelling

Author: Pieter-jan Kindermans, Hannes Verschore, David Verstraeten, Benjamin Schrauwen

Abstract: The usability of Brain Computer Interfaces (BCI) based on the P300 speller is severely hindered by the need for long training times and many repetitions of the same stimulus. In this contribution we introduce a set of unsupervised hierarchical probabilistic models that tackle both problems simultaneously by incorporating prior knowledge from two sources: information from other training subjects (through transfer learning) and information about the words being spelled (through language models). We show, that due to this prior knowledge, the performance of the unsupervised models parallels and in some cases even surpasses that of supervised models, while eliminating the tedious training session. 1

5 0.54190648 266 nips-2012-Patient Risk Stratification for Hospital-Associated C. diff as a Time-Series Classification Task

Author: Jenna Wiens, Eric Horvitz, John V. Guttag

6 0.50829023 198 nips-2012-Learning with Target Prior

7 0.46932596 109 nips-2012-Efficient Monte Carlo Counterfactual Regret Minimization in Games with Many Player Actions

8 0.45731625 224 nips-2012-Multi-scale Hyper-time Hardware Emulation of Human Motor Nervous System Based on Spiking Neurons using FPGA

9 0.43312934 46 nips-2012-Assessing Blinding in Clinical Trials

10 0.38397652 289 nips-2012-Recognizing Activities by Attribute Dynamics

11 0.38387975 256 nips-2012-On the connections between saliency and tracking

12 0.37951404 276 nips-2012-Probabilistic Event Cascades for Alzheimer's disease

13 0.37390661 31 nips-2012-Action-Model Based Multi-agent Plan Recognition

14 0.36336163 363 nips-2012-Wavelet based multi-scale shape features on arbitrary surfaces for cortical thickness discrimination

15 0.35763818 157 nips-2012-Identification of Recurrent Patterns in the Activation of Brain Networks

16 0.35513976 238 nips-2012-Neurally Plausible Reinforcement Learning of Working Memory Tasks

17 0.33379251 303 nips-2012-Searching for objects driven by context

18 0.32422772 209 nips-2012-Max-Margin Structured Output Regression for Spatio-Temporal Action Localization

19 0.31945252 93 nips-2012-Deep Spatio-Temporal Architectures and Learning for Protein Structure Prediction

20 0.31327021 167 nips-2012-Kernel Hyperalignment

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(0, 0.035), (1, 0.028), (17, 0.035), (21, 0.03), (38, 0.085), (42, 0.036), (48, 0.352), (54, 0.026), (55, 0.033), (74, 0.039), (76, 0.107), (77, 0.014), (80, 0.062), (92, 0.036)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.76907319 273 nips-2012-Predicting Action Content On-Line and in Real Time before Action Onset – an Intracranial Human Study

Author: Uri Maoz, Shengxuan Ye, Ian Ross, Adam Mamelak, Christof Koch

2 0.4326039 188 nips-2012-Learning from Distributions via Support Measure Machines

Author: Krikamol Muandet, Kenji Fukumizu, Francesco Dinuzzo, Bernhard Schölkopf

Abstract: This paper presents a kernel-based discriminative learning framework on probability measures. Rather than relying on large collections of vectorial training examples, our framework learns using a collection of probability distributions that have been constructed to meaningfully represent training data. By representing these probability distributions as mean embeddings in the reproducing kernel Hilbert space (RKHS), we are able to apply many standard kernel-based learning techniques in straightforward fashion. To accomplish this, we construct a generalization of the support vector machine (SVM) called a support measure machine (SMM). Our analyses of SMMs provides several insights into their relationship to traditional SVMs. Based on such insights, we propose a ﬂexible SVM (FlexSVM) that places different kernel functions on each training example. Experimental results on both synthetic and real-world data demonstrate the effectiveness of our proposed framework. 1

3 0.42966202 77 nips-2012-Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models

Author: Jeff Beck, Alexandre Pouget, Katherine A. Heller

Abstract: Recent experiments have demonstrated that humans and animals typically reason probabilistically about their environment. This ability requires a neural code that represents probability distributions and neural circuits that are capable of implementing the operations of probabilistic inference. The proposed probabilistic population coding (PPC) framework provides a statistically efﬁcient neural representation of probability distributions that is both broadly consistent with physiological measurements and capable of implementing some of the basic operations of probabilistic inference in a biologically plausible way. However, these experiments and the corresponding neural models have largely focused on simple (tractable) probabilistic computations such as cue combination, coordinate transformations, and decision making. As a result it remains unclear how to generalize this framework to more complex probabilistic computations. Here we address this short coming by showing that a very general approximate inference algorithm known as Variational Bayesian Expectation Maximization can be naturally implemented within the linear PPC framework. We apply this approach to a generic problem faced by any given layer of cortex, namely the identiﬁcation of latent causes of complex mixtures of spikes. We identify a formal equivalent between this spike pattern demixing problem and topic models used for document classiﬁcation, in particular Latent Dirichlet Allocation (LDA). We then construct a neural network implementation of variational inference and learning for LDA that utilizes a linear PPC. This network relies critically on two non-linear operations: divisive normalization and super-linear facilitation, both of which are ubiquitously observed in neural circuits. We also demonstrate how online learning can be achieved using a variation of Hebb’s rule and describe an extension of this work which allows us to deal with time varying and correlated latent causes. 1 Introduction to Probabilistic Inference in Cortex Probabilistic (Bayesian) reasoning provides a coherent and, in many ways, optimal framework for dealing with complex problems in an uncertain world. It is, therefore, somewhat reassuring that behavioural experiments reliably demonstrate that humans and animals behave in a manner consistent with optimal probabilistic reasoning when performing a wide variety of perceptual [1, 2, 3], motor [4, 5, 6], and cognitive tasks[7]. This remarkable ability requires a neural code that represents probability distribution functions of task relevant stimuli rather than just single values. While there 1 are many ways to represent functions, Bayes rule tells us that when it comes to probability distribution functions, there is only one statistically optimal way to do it. More precisely, Bayes Rule states that any pattern of activity, r, that efﬁciently represents a probability distribution over some task relevant quantity s, must satisfy the relationship p(s|r) ∝ p(r|s)p(s), where p(r|s) is the stimulus conditioned likelihood function that speciﬁes the form of neural variability, p(s) gives the prior belief regarding the stimulus, and p(s|r) gives the posterior distribution over values of the stimulus, s given the representation r . Of course, it is unlikely that the nervous system consistently achieves this level of optimality. None-the-less, Bayes rule suggests the existence of a link between neural variability as characterized by the likelihood function p(r|s) and the state of belief of a mature statistical learning machine such as the brain. The so called Probabilistic Population Coding (or PPC) framework[8, 9, 10] takes this link seriously by proposing that the function encoded by a pattern of neural activity r is, in fact, the likelihood function p(r|s). When this is the case, the precise form of the neural variability informs the nature of the neural code. For example, the exponential family of statistical models with linear sufﬁcient statistics has been shown to be ﬂexible enough to model the ﬁrst and second order statistics of in vivo recordings in awake behaving monkeys[9, 11, 12] and anesthetized cats[13]. When the likelihood function is modeled in this way, the log posterior probability over the stimulus is linearly encoded by neural activity, i.e. log p(s|r) = h(s) · r − log Z(r) (1) Here, the stimulus dependent kernel, h(s), is a vector of functions of s, the dot represents a standard dot product, and Z(r) is the partition function which serves to normalize the posterior. This log linear form for a posterior distribution is highly computationally convenient and allows for evidence integration to be implemented via linear operations on neural activity[14, 8]. Proponents of this kind of linear PPC have demonstrated how to build biologically plausible neural networks capable of implementing the operations of probabilistic inference that are needed to optimally perform the behavioural tasks listed above. This includes, linear PPC implementations of cue combination[8], evidence integration over time, maximum likelihood and maximum a posterior estimation[9], coordinate transformation/auditory localization[10], object tracking/Kalman ﬁltering[10], explaining away[10], and visual search[15]. Moreover, each of these neural computations has required only a single recurrently connected layer of neurons that is capable of just two non-linear operations: coincidence detection and divisive normalization, both of which are widely observed in cortex[16, 17]. Unfortunately, this research program has been a piecemeal effort that has largely proceeded by building neural networks designed deal with particular problems. As a result, there have been no proposals for a general principle by which neural network implementations of linear PPCs might be generated and no suggestions regarding how to deal with complex (intractable) problems of probabilistic inference. In this work, we will partially address this short coming by showing that Variation Bayesian Expectation Maximization (VBEM) algorithm provides a general scheme for approximate inference and learning with linear PPCs. In section 2, we brieﬂy review the VBEM algorithm and show how it naturally leads to a linear PPC representation of the posterior as well as constraints on the neural network dynamics which build that PPC representation. Because this section describes the VB-PPC approach rather abstractly, the remainder of the paper is dedicated to concrete applications. As a motivating example, we consider the problem of inferring the concentrations of odors in an olfactory scene from a complex pattern of spikes in a population of olfactory receptor neurons (ORNs). In section 3, we argue that this requires solving a spike pattern demixing problem which is indicative of the generic problem faced by many layers of cortex. We then show that this demixing problem is equivalent to the problem addressed by a class of models for text documents know as probabilistic topic models, in particular Latent Dirichlet Allocation or LDA[18]. In section 4, we apply the VB-PPC approach to build a neural network implementation of probabilistic inference and learning for LDA. This derivation shows that causal inference with linear PPC’s also critically relies on divisive normalization. This result suggests that this particular non-linearity may be involved in very general and fundamental probabilistic computation, rather than simply playing a role in gain modulation. In this section, we also show how this formulation allows for a probabilistic treatment of learning and show that a simple variation of Hebb’s rule can implement Bayesian learning in neural circuits. 2 We conclude this work by generalizing this approach to time varying inputs by introducing the Dynamic Document Model (DDM) which can infer short term ﬂuctuations in the concentrations of individual topics/odors and can be used to model foraging and other tracking tasks. 2 Variational Bayesian Inference with linear Probabilistic Population Codes Variational Bayesian (VB) inference refers to a class of deterministic methods for approximating the intractable integrals which arise in the context of probabilistic reasoning. Properly implemented it can result a fast alternative to sampling based methods of inference such as MCMC[19] sampling. Generically, the goal of any Bayesian inference algorithm is to infer a posterior distribution over behaviourally relevant latent variables Z given observations X and a generative model which speciﬁes the joint distribution p(X, Θ, Z). This task is confounded by the fact that the generative model includes latent parameters Θ which must be marginalized out, i.e. we wish to compute, p(Z|X) ∝ p(X, Θ, Z)dΘ (2) When the number of latent parameters is large this integral can be quite unwieldy. The VB algorithms simplify this marginalization by approximating the complex joint distribution over behaviourally relevant latents and parameters, p(Θ, Z|X), with a distribution q(Θ, Z) for which integrals of this form are easier to deal with in some sense. There is some art to choosing the particular form for the approximating distribution to make the above integral tractable, however, a factorized approximation is common, i.e. q(Θ, Z) = qΘ (Θ)qZ (Z). Regardless, for any given observation X, the approximate posterior is found by minimizing the Kullback-Leibler divergence between q(Θ, Z) and p(Θ, Z|X). When a factorized posterior is assumed, the Variational Bayesian Expectation Maximization (VBEM) algorithm ﬁnds a local minimum of the KL divergence by iteratively updating, qΘ (Θ) and qZ (Z) according to the scheme n log qΘ (Θ) ∼ log p(X, Θ, Z) n qZ (Z) and n+1 log qZ (Z) ∼ log p(X, Θ, Z) n qΘ (Θ) (3) Here the brackets indicate an expected value taken with respect to the subscripted probability distribution function and the tilde indicates equality up to a constant which is independent of Θ and Z. The key property to note here is that the approximate posterior which results from this procedure is in an exponential family form and is therefore representable by a linear PPC (Eq. 1). This feature allows for the straightforward construction of networks which implement the VBEM algorithm with linear PPC’s in the following way. If rn and rn are patterns of activity that use a linear PPC representation Θ Z of the relevant posteriors, then n log qΘ (Θ) ∼ hΘ (Θ) · rn Θ and n+1 log qZ (Z) ∼ hZ (Z) · rn+1 . Z (4) Here the stimulus dependent kernels hZ (Z) and hΘ (Θ) are chosen so that their outer product results in a basis that spans the function space on Z × Θ given by log p(X, Θ, Z) for every X. This choice guarantees that there exist functions fΘ (X, rn ) and fZ (X, rn ) such that Z Θ rn = fΘ (X, rn ) Θ Z and rn+1 = fZ (X, rn ) Θ Z (5) satisfy Eq. 3. When this is the case, simply iterating the discrete dynamical system described by Eq. 5 until convergence will ﬁnd the VBEM approximation to the posterior. This is one way to build a neural network implementation of the VB algorithm. However, its not the only way. In general, any dynamical system which has stable ﬁxed points in common with Eq. 5 can also be said to implement the VBEM algorithm. In the example below we will take advantage of this ﬂexibility in order to build biologically plausible neural network implementations. 3 Response! to Mixture ! of Odors! Single Odor Response Cause Intensity Figure 1: (Left) Each cause (e.g. coffee) in isolation results in a pattern of neural activity (top). When multiple causes contribute to a scene this results in an overall pattern of neural activity which is a mixture of these patterns weighted by the intensities (bottom). (Right) The resulting pattern can be represented by a raster, where each spike is colored by its corresponding latent cause. 3 Probabilistic Topic Models for Spike Train Demixing Consider the problem of odor identiﬁcation depicted in Fig. 1. A typical mammalian olfactory system consists of a few hundred different types of olfactory receptor neurons (ORNs), each of which responds to a wide range of volatile chemicals. This results in a highly distributed code for each odor. Since, a typical olfactory scene consists of many different odors at different concentrations, the pattern of ORN spike trains represents a complex mixture. Described in this way, it is easy to see that the problem faced by early olfactory cortex can be described as the task of demixing spike trains to infer latent causes (odor intensities). In many ways this olfactory problem is a generic problem faced by each cortical layer as it tries to make sense of the activity of the neurons in the layer below. The input patterns of activity consist of spikes (or spike counts) labeled by the axons which deliver them and summarized by a histogram which indicates how many spikes come from each input neuron. Of course, just because a spike came from a particular neuron does not mean that it had a particular cause, just as any particular ORN spike could have been caused by any one of a large number of volatile chemicals. Like olfactory codes, cortical codes are often distributed and multiple latent causes can be present at the same time. Regardless, this spike or histogram demixing problem is formally equivalent to a class of demixing problems which arise in the context of probabilistic topic models used for document modeling. A simple but successful example of this kind of topic model is called Latent Dirichlet Allocation (LDA) [18]. LDA assumes that word order in documents is irrelevant and, therefore, models documents as histograms of word counts. It also assumes that there are K topics and that each of these topics appears in different proportions in each document, e.g. 80% of the words in a document might be concerned with coffee and 20% with strawberries. Words from a given topic are themselves drawn from a distribution over words associated with that topic, e.g. when talking about coffee you have a 5% chance of using the word ’bitter’. The goal of LDA is to infer both the distribution over topics discussed in each document and the distribution of words associated with each topic. We can map the generative model for LDA onto the task of spike demixing in cortex by letting topics become latent causes or odors, words become neurons, word occurrences become spikes, word distributions associated with each topic become patterns of neural activity associated with each cause, and different documents become the observed patterns of neural activity on different trials. This equivalence is made explicit in Fig. 2 which describes the standard generative model for LDA applied to documents on the left and mixtures of spikes on the right. 4 LDA Inference and Network Implementation In this section we will apply the VB-PPC formulation to build a biologically plausible network capable of approximating probabilistic inference for spike pattern demixing. For simplicity, we will use the equivalent Gamma-Poisson formulation of LDA which directly models word and topic counts 4 1. For each topic k = 1, . . . , K, (a) Distribution over words βk ∼ Dirichlet(η0 ) 2. For document d = 1, . . . , D, (a) Distribution over topics θd ∼ Dirichlet(α0 ) (b) For word m = 1, . . . , Ωd i. Topic assignment zd,m ∼ Multinomial(θd ) ii. Word assignment ωd,m ∼ Multinomial(βzm ) 1. For latent cause k = 1, . . . , K, (a) Pattern of neural activity βk ∼ Dirichlet(η0 ) 2. For scene d = 1, . . . , D, (a) Relative intensity of each cause θd ∼ Dirichlet(α0 ) (b) For spike m = 1, . . . , Ωd i. Cause assignment zd,m ∼ Multinomial(θd ) ii. Neuron assignment ωd,m ∼ Multinomial(βzm ) Figure 2: (Left) The LDA generative model in the context of document modeling. (Right) The corresponding LDA generative model mapped onto the problem of spike demixing. Text related attributes on the left, in red, have been replaced with neural attributes on the right, in green. rather than topic assignments. Speciﬁcally, we deﬁne, Rd,j to be the number of times neuron j ﬁres during trial d. Similarly, we let Nd,j,k to be the number of times a spike in neuron j comes from cause k in trial d. These new variables play the roles of the cause and neuron assignment variables, zd,m and ωd,m by simply counting them up. If we let cd,k be an un-normalized intensity of cause j such that θd,k = cd,k / k cd,k then the generative model, Rd,j = k Nd,j,k Nd,j,k ∼ Poisson(βj,k cd,k ) 0 cd,k ∼ Gamma(αk , C −1 ). (6) is equivalent to the topic models described above. Here the parameter C is a scale parameter which sets the expected total number of spikes from the population on each trial. Note that, the problem of inferring the wj,k and cd,k is a non-negative matrix factorization problem similar to that considered by Lee and Seung[20]. The primary difference is that, here, we are attempting to infer a probability distribution over these quantities rather than maximum likelihood estimates. See supplement for details. Following the prescription laid out in section 2, we approximate the posterior over latent variables given a set of input patterns, Rd , d = 1, . . . , D, with a factorized distribution of the form, qN (N)qc (c)qβ (β). This results in marginal posterior distributions q (β:,k |η:,k ), q cd,k |αd,k , C −1 + 1 ), and q (Nd,j,: | log pd,j,: , Rd,i ) which are Dirichlet, Gamma, and Multinomial respectively. Here, the parameters η:,k , αd,k , and log pd,j,: are the natural parameters of these distributions. The VBEM update algorithm yields update rules for these parameters which are summarized in Fig. 3 Algorithm1. Algorithm 1: Batch VB updates 1: while ηj,k not converged do 2: for d = 1, · · · , D do 3: while pd,j,k , αd,k not converged do 4: αd,k → α0 + j Rd,j pd,j,k 5: pd,j,k → Algorithm 2: Online VB updates 1: for d = 1, · · · , D do 2: reinitialize pj,k , αk ∀j, k 3: while pj,k , αk not converged do 4: αk → α0 + j Rd,j pj,k 5: pj,k → exp (ψ(ηj,k )−ψ(¯k )) exp ψ(αk ) η η i exp (ψ(ηj,i )−ψ(¯i )) exp ψ(αi ) exp (ψ(ηj,k )−ψ(¯k )) exp ψ(αd,k ) η η i exp (ψ(ηj,i )−ψ(¯i )) exp ψ(αd,i ) 6: end while 7: end for 8: ηj,k = η 0 + 9: end while end while ηj,k → (1 − dt)ηj,k + dt(η 0 + Rd,j pj,k ) 8: end for 6: 7: d Rd,j pd,j,k Figure 3: Here ηk = j ηj,k and ψ(x) is the digamma function so that exp ψ(x) is a smoothed ¯ threshold linear function. Before we move on to the neural network implementation, note that this standard formulation of variational inference for LDA utilizes a batch learning scheme that is not biologically plausible. Fortunately, an online version of this variational algorithm was recently proposed and shown to give 5 superior results when compared to the batch learning algorithm[21]. This algorithm replaces the sum over d in update equation for ηj,k with an incremental update based upon only the most recently observed pattern of spikes. See Fig. 3 Algorithm 2. 4.1 Neural Network Implementation Recall that the goal was to build a neural network that implements the VBEM algorithm for the underlying latent causes of a mixture of spikes using a neural code that represents the posterior distribution via a linear PPC. A linear PPC represents the natural parameters of a posterior distribution via a linear operation on neural activity. Since the primary quantity of interest here is the posterior distribution over odor concentrations, qc (c|α), this means that we need a pattern of activity rα which is linearly related to the αk ’s in the equations above. One way to accomplish this is to simply assume that the ﬁring rates of output neurons are equal to the positive valued αk parameters. Fig. 4 depicts the overall network architecture. Input patterns of activity, R, are transmitted to the synapses of a population of output neurons which represent the αk ’s. The output activity is pooled to ¯ form an un-normalized prediction of the activity of each input neuron, Rj , given the output layer’s current state of belief about the latent causes of the Rj . The activity at each synapse targeted by input neuron j is then inhibited divisively by this prediction. This results in a dendrite that reports to the ¯ soma a quantity, Nj,k , which represents the fraction of unexplained spikes from input neuron j that could be explained by latent cause k. A continuous time dynamical system with this feature and the property that it shares its ﬁxed points with the LDA algorithm is given by d ¯ Nj,k dt d αk dt ¯ ¯ = wj,k Rj − Rj Nj,k = (7) ¯ Nj,k exp (ψ (¯k )) (α0 − αk ) + exp (ψ (αk )) η (8) i ¯ where Rj = k wj,k exp (ψ (αk )), and wj,k = exp (ψ (ηj,k )). Note that, despite its form, it is Eq. 7 which implements the required divisive normalization operation since, in the steady state, ¯ ¯ Nj,k = wj,k Rj /Rj . Regardless, this network has a variety of interesting properties that align well with biology. It predicts that a balance of excitation and inhibition is maintained in the dendrites via divisive normalization and that the role of inhibitory neurons is to predict the input spikes which target individual dendrites. It also predicts superlinear facilitation. Speciﬁcally, the ﬁnal term on the right of Eq. 8 indicates that more active cells will be more sensitive to their dendritic inputs. Alternatively, this could be implemented via recurrent excitation at the population level. In either case, this is the mechanism by which the network implements a sparse prior on topic concentrations and stands in stark contrast to the winner take all mechanisms which rely on competitive mutual inhibition mechanisms. Additionally, the ηj in Eq. 8 represents a cell wide ’leak’ parameter that indicates that the total leak should be ¯ roughly proportional to the sum total weight of the synapses which drive the neuron. This predicts that cells that are highly sensitive to input should also decay back to baseline more quickly. This implementation also predicts Hebbian learning of synaptic weights. To observe this fact, note that the online update rule for the ηj,k parameters can be implemented by simply correlating the activity at ¯ each synapse, Nj,k with activity at the soma αj via the equation: τL d ¯ wj,k = exp (ψ (¯k )) (η0 − 1/2 − wj,k ) + Nj,k exp ψ (αk ) η dt (9) where τL is a long time constant for learning and we have used the fact that exp (ψ (ηjk )) ≈ ηjk −1/2 for x > 1. For a detailed derivation see the supplementary material. 5 Dynamic Document Model LDA is a rather simple generative model that makes several unrealistic assumptions about mixtures of sensory and cortical spikes. In particular, it assumes both that there are no correlations between the 6 Targeted Divisive Normalization Targeted Divisive Normalization αj Ri Input Neurons Recurrent Connections ÷ ÷ -1 -1 Σ μj Nij Ri Synapses Output Neurons Figure 4: The LDA network model. Dendritically targeted inhibition is pooled from the activity of all neurons in the output layer and acts divisively. Σ jj' Nij Input Neurons Synapses Output Neurons Figure 5: DDM network model also includes recurrent connections which target the soma with both a linear excitatory signal and an inhibitory signal that also takes the form of a divisive normalization. intensities of latent causes and that there are no correlations between the intensities of latent causes in temporally adjacent trials or scenes. This makes LDA a rather poor computational model for a task like olfactory foraging which requires the animal to track the rise a fall of odor intensities as it navigates its environment. We can model this more complicated task by replacing the static cause or odor intensity parameters with dynamic odor intensity parameters whose behavior is governed by an exponentiated Ornstein-Uhlenbeck process with drift and diffusion matrices given by (Λ and ΣD ). We call this variant of LDA the Dynamic Document Model (DDM) as it could be used to model smooth changes in the distribution of topics over the course of a single document. 5.1 DDM Model Thus the generative model for the DDM is as follows: 1. For latent cause k = 1, . . . , K, (a) Cause distribution over spikes βk ∼ Dirichlet(η0 ) 2. For scene t = 1, . . . , T , (a) Log intensity of causes c(t) ∼ Normal(Λct−1 , ΣD ) (b) Number of spikes in neuron j resulting from cause k, Nj,k (t) ∼ Poisson(βj,k exp ck (t)) (c) Number of spikes in neuron j, Rj (t) = k Nj,k (t) This model bears many similarities to the Correlated and Dynamic topic models[22], but models dynamics over a short time scale, where the dynamic relationship (Λ, ΣD ) is important. 5.2 Network Implementation Once again the quantity of interest is the current distribution of latent causes, p(c(t)|R(τ ), τ = 0..T ). If no spikes occur then no evidence is presented and posterior inference over c(t) is simply given by an undriven Kalman ﬁlter with parameters (Λ, ΣD ). A recurrent neural network which uses a linear PPC to encode a posterior that evolves according to a Kalman ﬁlter has the property that neural responses are linearly related to the inverse covariance matrix of the posterior as well as that inverse covariance matrix times the posterior mean. In the absence of evidence, it is easy to show that these quantities must evolve according to recurrent dynamics which implement divisive normalization[10]. Thus, the patterns of neural activity which linearly encode them must do so as well. When a new spike arrives, optimal inference is no longer possible and a variational approximation must be utilized. As is shown in the supplement, this variational approximation is similar to the variational approximation used for LDA. As a result, a network which can divisively inhibit its synapses is able to implement approximate Bayesian inference. Curiously, this implies that the addition of spatial and temporal correlations to the latent causes adds very little complexity to the VB-PPC network implementation of probabilistic inference. All that is required is an additional inhibitory population which targets the somata in the output population. See Fig. 5. 7 Natural Parameters Natural Parameters (α) 0.4 200 450 180 0.3 Network Estimate Network Estimate 500 400 350 300 250 200 150 100 0.1 0 50 100 150 200 250 300 350 400 450 500 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 140 120 0.4 0.3 100 0.2 80 0.1 0 60 40 0.4 20 50 0 0 0.2 160 0 0 0.3 0.2 20 40 60 80 100 120 VBEM Estimate VBEM Estimate 140 160 180 200 0.1 0 Figure 6: (Left) Neural network approximation to the natural parameters of the posterior distribution over topics (the α’s) as a function of the VBEM estimate of those same parameters for a variety of ’documents’. (Center) Same as left, but for the natural parameters of the DDM (i.e the entries of the matrix Σ−1 (t) and Σ−1 µ(t) of the distribution over log topic intensities. (Right) Three example traces for cause intensity in the DDM. Black shows true concentration, blue and red (indistinguishable) show MAP estimates for the network and VBEM algorithms. 6 Experimental Results We compared the PPC neural network implementations of the variational inference with the standard VBEM algorithm. This comparison is necessary because the two algorithms are not guaranteed to converge to the same solution due to the fact that we only required that the neural network dynamics have the same ﬁxed points as the standard VBEM algorithm. As a result, it is possible for the two algorithms to converge to different local minima of the KL divergence. For the network implementation of LDA we ﬁnd good agreement between the neural network and VBEM estimates of the natural parameters of the posterior. See Fig. 6(left) which shows the two algorithms estimates of the shape parameter of the posterior distribution over topic (odor) concentrations (a quantity which is proportional to the expected concentration). This agreement, however, is not perfect, especially when posterior predicted concentrations are low. In part, this is due to the fact we are presenting the network with difﬁcult inference problems for which the true posterior distribution over topics (odors) is highly correlated and multimodal. As a result, the objective function (KL divergence) is littered with local minima. Additionally, the discrete iterations of the VBEM algorithm can take very large steps in the space of natural parameters while the neural network implementation cannot. In contrast, the network implementation of the DDM is in much better agreement with the VBEM estimation. See Fig. 6(right). This is because the smooth temporal dynamics of the topics eliminate the need for the VBEM algorithm to take large steps. As a result, the smooth network dynamics are better able to accurately track the VBEM algorithms output. For simulation details please see the supplement. 7 Discussion and Conclusion In this work we presented a general framework for inference and learning with linear Probabilistic Population codes. This framework takes advantage of the fact that the Variational Bayesian Expectation Maximization algorithm generates approximate posterior distributions which are in an exponential family form. This is precisely the form needed in order to make probability distributions representable by a linear PPC. We then outlined a general means by which one can build a neural network implementation of the VB algorithm using this kind of neural code. We applied this VB-PPC framework to generate a biologically plausible neural network for spike train demixing. We chose this problem because it has many of the features of the canonical problem faced by nearly every layer of cortex, i.e. that of inferring the latent causes of complex mixtures of spike trains in the layer below. Curiously, this very complicated problem of probabilistic inference and learning ended up having a remarkably simple network solution, requiring only that neurons be capable of implementing divisive normalization via dendritically targeted inhibition and superlinear facilitation. Moreover, we showed that extending this approach to the more complex dynamic case in which latent causes change in intensity over time does not substantially increase the complexity of the neural circuit. Finally, we would like to note that, while we utilized a rate coding scheme for our linear PPC, the basic equations would still apply to any spike based log probability codes such as that considered Beorlin and Deneve[23]. 8 References [1] Daniel Kersten, Pascal Mamassian, and Alan Yuille. Object perception as Bayesian inference. Annual review of psychology, 55:271–304, January 2004. [2] Marc O Ernst and Martin S Banks. Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870):429–33, 2002. [3] Yair Weiss, Eero P Simoncelli, and Edward H Adelson. Motion illusions as optimal percepts. Nature neuroscience, 5(6):598–604, 2002. [4] P N Sabes. The planning and control of reaching movements. Current opinion in neurobiology, 10(6): 740–6, 2000. o [5] Konrad P K¨ rding and Daniel M Wolpert. Bayesian integration in sensorimotor learning. Nature, 427 (6971):244–7, 2004. [6] Emanuel Todorov. Optimality principles in sensorimotor control. Nature neuroscience, 7(9):907–15, 2004. [7] Erno T´ gl´ s, Edward Vul, Vittorio Girotto, Michel Gonzalez, Joshua B Tenenbaum, and Luca L Bonatti. e a Pure reasoning in 12-month-old infants as probabilistic inference. Science (New York, N.Y.), 332(6033): 1054–9, 2011. [8] W.J. Ma, J.M. Beck, P.E. Latham, and A. Pouget. Bayesian inference with probabilistic population codes. Nature Neuroscience, 2006. [9] Jeffrey M Beck, Wei Ji Ma, Roozbeh Kiani, Tim Hanks, Anne K Churchland, Jamie Roitman, Michael N Shadlen, Peter E Latham, and Alexandre Pouget. Probabilistic population codes for Bayesian decision making. Neuron, 60(6):1142–52, 2008. [10] J. M. Beck, P. E. Latham, and a. Pouget. Marginalization in Neural Circuits with Divisive Normalization. Journal of Neuroscience, 31(43):15310–15319, 2011. [11] Tianming Yang and Michael N Shadlen. Probabilistic reasoning by neurons. Nature, 447(7148):1075–80, 2007. [12] RHS Carpenter and MLL Williams. Neural computation of log likelihood in control of saccadic eye movements. Nature, 1995. [13] Arnulf B a Graf, Adam Kohn, Mehrdad Jazayeri, and J Anthony Movshon. Decoding the activity of neuronal populations in macaque primary visual cortex. Nature neuroscience, 14(2):239–45, 2011. [14] HB Barlow. Pattern Recognition and the Responses of Sensory Neurons. Annals of the New York Academy of Sciences, 1969. [15] Wei Ji Ma, Vidhya Navalpakkam, Jeffrey M Beck, Ronald Van Den Berg, and Alexandre Pouget. Behavior and neural basis of near-optimal visual search. Nature Neuroscience, (May), 2011. [16] DJ Heeger. Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9, 1992. [17] M Carandini, D J Heeger, and J a Movshon. Linearity and normalization in simple cells of the macaque primary visual cortex. The Journal of neuroscience : the ofﬁcial journal of the Society for Neuroscience, 17(21):8621–44, 1997. [18] D. Blei, A. Ng, and M. Jordan. Latent Dirichlet Allocation. JMLR, 2003. [19] M. Beal. Variational Algorithms for Approximate Bayesian Inference. PhD thesis, Gatsby Unit, UCL, 2003. [20] D D Lee and H S Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401 (6755):788–91, 1999. [21] M. Hoffman, D. Blei, and F. Bach. Online learning for Latent Dirichlet Allocation. In NIPS, 2010. [22] D. Blei and J. Lafferty. Dynamic topic models. In ICML, 2006. [23] M. Boerlin and S. Deneve. Spike-based population coding and working memory. PLOS computational biology, 2011. 9

4 0.42815018 209 nips-2012-Max-Margin Structured Output Regression for Spatio-Temporal Action Localization

Author: Du Tran, Junsong Yuan

Abstract: Structured output learning has been successfully applied to object localization, where the mapping between an image and an object bounding box can be well captured. Its extension to action localization in videos, however, is much more challenging, because we need to predict the locations of the action patterns both spatially and temporally, i.e., identifying a sequence of bounding boxes that track the action in video. The problem becomes intractable due to the exponentially large size of the structured video space where actions could occur. We propose a novel structured learning approach for spatio-temporal action localization. The mapping between a video and a spatio-temporal action trajectory is learned. The intractable inference and learning problems are addressed by leveraging an efﬁcient Max-Path search method, thus making it feasible to optimize the model over the whole structured space. Experiments on two challenging benchmark datasets show that our proposed method outperforms the state-of-the-art methods. 1

5 0.42618579 113 nips-2012-Efficient and direct estimation of a neural subunit model for sensory coding

Author: Brett Vintch, Andrew Zaharia, J Movshon, Hhmi) Hhmi), Eero P. Simoncelli

Abstract: Many visual and auditory neurons have response properties that are well explained by pooling the rectiﬁed responses of a set of spatially shifted linear ﬁlters. These ﬁlters cannot be estimated using spike-triggered averaging (STA). Subspace methods such as spike-triggered covariance (STC) can recover multiple ﬁlters, but require substantial amounts of data, and recover an orthogonal basis for the subspace in which the ﬁlters reside rather than the ﬁlters themselves. Here, we assume a linear-nonlinear–linear-nonlinear (LN-LN) cascade model in which the ﬁrst linear stage is a set of shifted (‘convolutional’) copies of a common ﬁlter, and the ﬁrst nonlinear stage consists of rectifying scalar nonlinearities that are identical for all ﬁlter outputs. We refer to these initial LN elements as the ‘subunits’ of the receptive ﬁeld. The second linear stage then computes a weighted sum of the responses of the rectiﬁed subunits. We present a method for directly ﬁtting this model to spike data, and apply it to both simulated and real neuronal data from primate V1. The subunit model signiﬁcantly outperforms STA and STC in terms of cross-validated accuracy and efﬁciency. 1

6 0.42610979 62 nips-2012-Burn-in, bias, and the rationality of anchoring

7 0.42610979 116 nips-2012-Emergence of Object-Selective Features in Unsupervised Feature Learning

8 0.42558604 43 nips-2012-Approximate Message Passing with Consistent Parameter Estimation and Applications to Sparse Learning

9 0.42499331 316 nips-2012-Small-Variance Asymptotics for Exponential Family Dirichlet Process Mixture Models

10 0.42476666 333 nips-2012-Synchronization can Control Regularization in Neural Systems via Correlated Noise Processes

11 0.42381111 83 nips-2012-Controlled Recognition Bounds for Visual Learning and Exploration

12 0.42318308 354 nips-2012-Truly Nonparametric Online Variational Inference for Hierarchical Dirichlet Processes

13 0.42309076 112 nips-2012-Efficient Spike-Coding with Multiplicative Adaptation in a Spike Response Model

14 0.42269766 90 nips-2012-Deep Learning of Invariant Features via Simulated Fixations in Video

15 0.42256504 65 nips-2012-Cardinality Restricted Boltzmann Machines

16 0.42221493 104 nips-2012-Dual-Space Analysis of the Sparse Linear Model

17 0.42191994 148 nips-2012-Hamming Distance Metric Learning

18 0.42087036 355 nips-2012-Truncation-free Online Variational Inference for Bayesian Nonparametric Models

19 0.42057055 266 nips-2012-Patient Risk Stratification for Hospital-Associated C. diff as a Time-Series Classification Task

20 0.42048156 353 nips-2012-Transferring Expectations in Model-based Reinforcement Learning