nips nips2010 nips2010-171 knowledge-graph by maker-knowledge-mining

171 nips-2010-Movement extraction by detecting dynamics switches and repetitions

Source: pdf

Author: Silvia Chiappa, Jan R. Peters

Abstract: Many time-series such as human movement data consist of a sequence of basic actions, e.g., forehands and backhands in tennis. Automatically extracting and characterizing such actions is an important problem for a variety of different applications. In this paper, we present a probabilistic segmentation approach in which an observed time-series is modeled as a concatenation of segments corresponding to different basic actions. Each segment is generated through a noisy transformation of one of a few hidden trajectories representing different types of movement, with possible time re-scaling. We analyze three different approximation methods for dealing with model intractability, and demonstrate how the proposed approach can successfully segment table tennis movements recorded using a robot arm as haptic input device. 1

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 Movement extraction by detecting dynamics switches and repetitions Silvia Chiappa Statistical Laboratory Wilberforce Road, Cambridge, UK silvia@statslab. [sent-1, score-0.071]

2 de Abstract Many time-series such as human movement data consist of a sequence of basic actions, e. [sent-7, score-0.227]

3 Automatically extracting and characterizing such actions is an important problem for a variety of different applications. [sent-10, score-0.098]

4 In this paper, we present a probabilistic segmentation approach in which an observed time-series is modeled as a concatenation of segments corresponding to different basic actions. [sent-11, score-0.25]

5 Each segment is generated through a noisy transformation of one of a few hidden trajectories representing different types of movement, with possible time re-scaling. [sent-12, score-0.336]

6 We analyze three different approximation methods for dealing with model intractability, and demonstrate how the proposed approach can successfully segment table tennis movements recorded using a robot arm as haptic input device. [sent-13, score-0.684]

7 1 Introduction Motion capture systems have become widespread in many application areas such as robotics [18], physical therapy, sports sciences [10], virtual reality [15], artiﬁcial movie generation [13], computer games [1], etc. [sent-14, score-0.318]

8 These systems are used for extracting the movement templates characterizing basic actions contained in their recordings. [sent-15, score-0.392]

9 In physical therapy and sports sciences, these templates are employed to analyze patient’s progress or sports professional’s movements; in robotics, virtual reality, movie generation or computer games, they become the basic elements for composing complex actions. [sent-16, score-0.473]

10 In order to obtain the movement templates, boundaries between actions need to be detected. [sent-17, score-0.274]

11 Furthermore, fundamental similarities and differences in the dynamics underlying different actions need to be captured. [sent-18, score-0.17]

12 For example, in a recording from a game of table tennis, observations corresponding to different actions can differ, due to different goals for hitting the ball, racket speeds, desired ball interaction, etc. [sent-19, score-0.248]

13 The system needs to determine whether this dissimilarity corresponds to substantially diverse types of underlying movements (such as in the case of a forehand and a backhand), or not (such as in the case of two forehands that differ only in speed). [sent-20, score-0.312]

14 In this paper, we present a probabilistic model in which actions are assumed to arise from noisy transformations of a small set of hidden trajectories, each representing a different movement template, with non-linear time re-scaling accounting for differences in action durations. [sent-22, score-0.461]

15 Segmentation is obtained by inferring, at each time-step, the position of the observations in the current action and the underlying movement template. [sent-24, score-0.327]

16 To guide segmentation, we impose constraints on the minimum and maximum duration that each action can have. [sent-25, score-0.167]

17 1 Hidden dynamics 75 67 68 61 σt+1 zt zt+1 vt vt+1 ··· 64 Observations 73 σt vt−1 97 σt−1 zt−1 ··· 94 56 53 53 (a) 51 h1:S 1:M (b) Figure 1: (a) The hidden dynamics shown on the top layer are assumed to generate the time-series at the bottom. [sent-26, score-0.733]

18 (b) Belief network representation of the proposed segmentation model. [sent-27, score-0.142]

19 We apply the model to a human game of table tennis recorded with a Barrett WAM used as a haptic input device, and show that we can obtain a meaningful segmentation of the time-series. [sent-29, score-0.53]

20 2 The Segmentation Model In the proposed segmentation approach, the observations originate from a set of continuous-valued hidden trajectories, each representing a different movement template. [sent-30, score-0.494]

21 Speciﬁcally, we assume that the observed time-series consists of a concatenation of segments (basic actions), each generated through a noisy transformation of one of the hidden trajectories, with possible time re-scaling. [sent-31, score-0.261]

22 This generation process is illustrated in Figure 1 (a), where the observations on the lower graph are generated from the three underlying hidden trajectories on the upper graph. [sent-32, score-0.42]

23 , the ﬁrst hidden trajectory of length 97 gives rise to three segments of length 75, 68 and 94 respectively. [sent-35, score-0.23]

24 The observed time-series and the S hidden trajectories are represented by the continuous random variables1 v1:T ≡ v1 , . [sent-36, score-0.281]

25 The ﬁrst set is used to infer which movement template generated the observations at each time-step, to detect action boundaries, and to deﬁne hard constraints on the minimum and maximum duration of each observed action. [sent-44, score-0.435]

26 The second set is used to model time re-scaling from the hidden trajectories to the observations. [sent-45, score-0.26]

27 We assume that the joint distribution of these variables factorizes as follows p(h1:S ) 1:M p(vt |h1:S , zt , σt )p(zt |zt−1 , σt−1:t )p(σt |σt−1 ). [sent-46, score-0.322]

28 The variable σt is a triple σt = {st , dt , ct } with a similar role as in regime-switching models with explicit regime-duration distribution (ERDMs) [4]. [sent-48, score-0.842]

29 , S} indicates which of the S hidden trajectories underlies the observations at time t. [sent-52, score-0.315]

30 The duration variables dt speciﬁes the time interval spanned by the observations forming the current action, and takes a value between dmin and dmax . [sent-53, score-0.499]

31 The count variable ct indicates the time distance to the beginning of the next action, taking value ct = dt and ct = 1 respectively at the beginning and end of an action. [sent-54, score-2.118]

32 More speciﬁcally, we deﬁne p(σt |σt−1 ) = p(ct |dt , ct−1 )p(dt |dt−1 , ct−1 )p(st |st−1 , ct−1 ) with2 1 For the sake of notational simplicity, we describe the model for the case of a single observed time-series and hidden trajectories of the same length M . [sent-55, score-0.26]

33 The variable zt indicates which of the M elements in the hidden trajectory generated the observations vt . [sent-59, score-0.714]

34 We deﬁne p(zt |zt−1 , σt−1:t ) = p(zt |zt−1 , dt , ct−1:t ) with p(zt |zt−1 , dt , ct−1:t ) = ˜dt ψzt ,ct if ct−1 = 1, dt ,ct ψzt ,zt−1 if ct−1 > 1. [sent-60, score-0.732]

35 ˜ The vector ψ dt ,ct and the matrix ψ dt ,ct encode two constraints3 . [sent-61, score-0.488]

36 , wmax } ensures that subsequent observations are generated by subsequent elements of the hidden trajectory and imposes a limit on the magnitude of time-warping. [sent-65, score-0.389]

37 , M − ct + 1} accounts for the dt − ct and ct − 1 observations preceding and following vt in the action. [sent-69, score-2.249]

38 The hidden trajectories follow independent linear Markovian dynamics with Gaussian noise, that is i i i i i i p(hi |hi m m−1 ) = N (F hm−1 , ΣH ), h1 ∼ N (µ , Σ ). [sent-70, score-0.331]

39 V H After learning Θ, we can sample a segmentation from p(σ1:T |v1:T ) or compute the most likely ∗ segmentation σ1:T = arg maxσ1:T p(σ1:T |v1:T )5 . [sent-74, score-0.284]

40 , S do generate hidden trajectory i hi ∼ N (µi , Σi ) 1 h h i hi = F i hi m m−1 + ηm , ηm ∼ N (0, ΣH ) set t = 1 for action a = 1, . [sent-78, score-0.672]

41 From a modeling point of view, the presented method builds on previous approaches that consider the observed time-series as time-warped transformations of one or several continuous-valued hidden trajectories. [sent-82, score-0.142]

42 In [12] and [14], the authors considered the case in which each time-series is generated by one of a set of different hidden trajectories. [sent-86, score-0.167]

43 None of these models can deal with the situation in which possibly different dynamics underlie different segments of the same time-series. [sent-87, score-0.116]

44 From an application point of view, previous segmentation systems for extracting basic movements employed considerable human intervention [16]. [sent-88, score-0.386]

45 On the other hand, automatic probabilistic methods for modeling movement templates assumed that the time-series data was pre-segmented into basic movements [5, 17]. [sent-89, score-0.441]

46 3 In the experiments we added the additional constraint that nearly complete movements are observed, that is zt−dt +ct ∈ {1, . [sent-90, score-0.147]

47 4 With π:,st−1 we indicate the vector of transition probabilities from dynamics type st−1 to any dynamics. [sent-97, score-0.071]

48 3 3 Inference and Learning The interaction between the continuous and discrete hidden variables renders the computation of the posterior distributions required for learning and sampling a segmentation intractable. [sent-99, score-0.378]

49 In the second (maximum a posteriori) method, we estimate the most likely set of hidden trajectories and Θ by maximizing p(h1:S , v1:T |Θ) using an 1:M EM approach. [sent-102, score-0.26]

50 1 Variational Method In the variational approximation, we introduce a distribution q in which the problematic dependence between the hidden dynamics and the segmentation and time-warping variables is relaxed, that is6 q(h1:S , z1:T , σ1:T ) = q(h1:S )q(z1:T |σ1:T )q(σ1:T ) . [sent-105, score-0.443]

51 We then use a variational EM algorithm in which B(q, Θ) is iteratively maximized with respect to q and the model parameters Θ until convergence7 . [sent-107, score-0.059]

52 (1) , (2) (3) Before describing how to perform inference on these distributions, we observe that all quantities required for learning Θ, sampling a segmentation, and updating q(h1:S ) can be formulated such 1:M that only partial inference on q(σ1:T ) and q(z1:T |σ1:T ) is required. [sent-109, score-0.094]

53 For example, we can write log p(v1:T |h1:S , z1:T , σ1:T ) 1:M q(z1:T ,σ1:T ) i,k,1 γt = i,k,1 i,k,t,m ξτ log p(vτ |hi , zτ = m, σt ), (4) m τ,m t,i,k i,k,1 i,k,1 i,k,1 i,k,1 i,k,t,m with γt = q(σt ), ξτ = q(zτ = m|σt ), σt = {st = i, dt = k, ct = 1}. [sent-110, score-0.842]

54 Thus, only posteriors for which the count variables take value 1 are required8 . [sent-111, score-0.059]

55 8 This is common to ERDMs using separate duration and count variables [4]. [sent-116, score-0.135]

56 7 4 where t i,k,1 γt vm ≡ 1/ai ˆi m t,k t i,k,t,m ξτ vτ , ˆ V,m ≡ 1/ai Σi , Σi m V τ =t−k+1 Therefore, inference on q(h1:S ) 1:M i,k,1 γt ai ≡ m t,k i,k,t,m ξτ . [sent-117, score-0.061]

57 This update has the form of the joint distribution of an ERDM using separate duration and count variables [4]. [sent-121, score-0.158]

58 More speciﬁcally γt = q(σt ) = q(vt+1:T |st = j, ct = i,k,1 i,1 i,k,1 1)q(σt , v1:t )/q(v1:T ) = βt αt /q(v1:T ), where j,l,1 i,k,1 j,l,1 p(σt |σt−k ) αt−k , i,k,1 i,k,1 αt = q(vt−k+1:t |σt ) jl j,1 βt = i,k,1 i,1 q(vt+1:t+k |σt+k )πi,j βt+k ρk . [sent-123, score-0.598]

59 i,k ρk πi,j Since we have imposed the constraints c0 = 1, cT = 1, we need to replace terms such as p(dt = k, ct = 1|ct−k = 1) = ρk with p(dt = k, ct = 1|ct−k = 1, c0 = 1, cT = 1). [sent-124, score-1.196]

60 The form of update (3) implies that inference on distributions of the i,k,1 type q(zt−k+1:t |σt ) can be accomplished with forward-backward routines similar to the ones used in hidden Markov models (HMMs). [sent-128, score-0.245]

62 q(σt |σt+1 , v1:T ) = σt+1 σt+1 ¨ αt+1 ¨¨ βt+1 Suppose that, at time t, ct = 1 and we have sampled dynamics type st = i and duration dt = k. [sent-131, score-1.129]

63 2 Maximum a Posteriori (MAP) Method Instead of approximating the posterior distribution of all hidden variables, we can approximate only p(h1:S |v1:T ) with a deterministic distribution, by using the variational method described above in 1:M which q(hi ) is a Dirac delta around its mean. [sent-134, score-0.201]

64 Notice that this is equivalent to compute the most m likely set of hidden trajectories and parameters by maximizing the joint distribution p(v1:T , h1:S |Θ) 1:M with respect to h1:S and Θ using an EM algorithm. [sent-135, score-0.26]

65 Each time-series (with V=2 or V=3) contains repeated occurrences of actions arising from the noisy transformation of up to three hidden trajectories with time-warping. [sent-149, score-0.388]

66 In the second row of Table 2, we give the correct segmentation for each time-series. [sent-150, score-0.142]

67 Each number indicates the time-step at which a new action starts, whilst the colors indicate the types of dynamics underlying the actions. [sent-151, score-0.188]

68 In the rows below, we give the segmentations obtained by each approximation method with 4 different initial random conditions (with minimum and maximum action duration between 5 and 30). [sent-152, score-0.191]

69 The 1:M continuous hidden variables are sampled given a single set of segmentation and time-warping variables (unlike update (1) in which we average over segmentation and time-warping variables), which may result in poor mixing. [sent-156, score-0.528]

70 The inferior performance of the variational method in comparison to the MAP method would seem to suggest that the posterior covariances of the continuous hidden variables cannot accurately be estimated. [sent-157, score-0.251]

71 4 Table Tennis Recordings using a Robot Arm In this section, we show how the proposed model performs in segmenting time-series obtained from table tennis recordings using a robot arm moved by a human. [sent-158, score-0.427]

72 The generic goal is to extract movement templates to be used for robot imitation learning [2, 9]. [sent-159, score-0.4]

73 We used the Barrett WAM robot arm shown in Figure 2 as a haptic input device for recording and replaying movements. [sent-161, score-0.384]

74 We recorded a game of table tennis where a human moved the robot arm making the typical moves occurring in this speciﬁc setup. [sent-162, score-0.576]

75 These naturally include forehands, going into an awaiting posture for a forehand, backhands, and going into an awaiting posture for a backhand. [sent-163, score-0.71]

76 The dashed vertical lines indicate the obtained action boundaries and the numbers the underlying movement templates. [sent-171, score-0.318]

77 This sequence includes moves to the right awaiting posture (1), moves to the left awaiting posture (2), forehands (3, 5), two incomplete moves towards the awaiting posture merged with a backhand (4), moves to the left awaiting posture with humerus rotation (6) and backhands (7). [sent-172, score-1.833]

78 The recorded time-series contains the joint positions and velocities of all seven degrees of freedom (DoF) of an anthropomorphic arm. [sent-173, score-0.149]

79 However, only the shoulder and upper arm DoF, which are the most signiﬁcant in such movements, were considered for the analysis. [sent-174, score-0.121]

80 The minimum and maximum durations dmin and dmax were set to 4 and 15 respectively, as prior knowledge about table tennis would suggest that basic-action durations are within this range. [sent-177, score-0.335]

81 We also imposed the constraint that nearly complete movements are observed (ι = 2, = M − 1). [sent-178, score-0.147]

82 The length of the hidden dynamics M was set to dmax , the variable wmax was Figure 2: The Barrett WAM used for recordset to10 4, and the number of movement templates ing the table tennis sequences. [sent-179, score-0.819]

83 During the S was set to 8, as this should be a reasonable upper experiment the robot is in a gravity compenbound on the number of different underlying move- sation and sequences can be replayed on the ments. [sent-180, score-0.144]

84 We assumed no prior knowledge on the dynamics of the hidden trajectories. [sent-183, score-0.213]

85 However, in a real application of the model we could simplify the problem by incorporating knowledge about previously identiﬁed movement templates. [sent-184, score-0.155]

86 As shown in Figure 3, the model segments the time-series into 59 basic movements of forehands (numbers 3, 5), backhands (7), and going into a right (1) and left (2, 6) awaiting posture. [sent-185, score-0.627]

87 In some cases, a more ﬂuid game results in incomplete moves towards an awaiting posture and hence into a composite movement that can no longer be segmented (4). [sent-186, score-0.583]

88 Also, there appear to be two types of moving back to the left awaiting posture: one which needs untwisting of the humerus rotation degree of freedom (6), and another which purely employs shoulder degrees of freedom (2). [sent-187, score-0.404]

89 The action boundaries estimated by the model are in strong agreement with manual visual segmentation, with the exception of movements 4 that should be segmented into two separate movements. [sent-188, score-0.284]

90 com we provide a visual interpretation of the segmentation from which the model accuracy can be appreciated. [sent-191, score-0.142]

91 10 This is the smallest value that ensures that complete actions can be observed. [sent-192, score-0.073]

92 7 5 Conclusions In this paper we have introduced a probabilistic model for detecting repeated occurrences of basic movements in time-series data. [sent-193, score-0.216]

93 This model may potentially be applicable in domains such as robotics, sports sciences, physical therapy, virtual reality, artiﬁcial movie generation, computer games, etc. [sent-194, score-0.164]

94 , for automatic extraction of the movement templates contained in a recording. [sent-195, score-0.254]

95 We have presented an evaluation on table tennis movements that we have recorded using a robot arm as haptic input device, showing that the model is able to accurately segment the time-series into basic movements that could be used for robot imitation learning. [sent-196, score-1.017]

96 Appendix Constraints on z1:T Consider an action starting at time 1 and ﬁnishing at time t with the constraints z1 ∈ {1, . [sent-197, score-0.091]

97 Marker-less human motion tracking opportunities for ﬁeld testing in sports. [sent-276, score-0.079]

98 Evaluating the effect of motion and body shape on the perceived sex of virtual characters. [sent-306, score-0.101]

99 Capture database through symbolization, recognition and generation of motion patterns. [sent-326, score-0.101]

100 Simultaneous tracking and balancing of humanoid robots for imitating human motion capture data. [sent-338, score-0.112]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('ct', 0.598), ('zt', 0.293), ('dt', 0.244), ('awaiting', 0.185), ('vt', 0.156), ('movement', 0.155), ('movements', 0.147), ('posture', 0.145), ('hidden', 0.142), ('segmentation', 0.142), ('st', 0.14), ('tennis', 0.14), ('hi', 0.132), ('wmax', 0.124), ('robot', 0.118), ('trajectories', 0.118), ('forehands', 0.103), ('templates', 0.099), ('action', 0.091), ('haptic', 0.09), ('arm', 0.088), ('backhands', 0.082), ('duration', 0.076), ('actions', 0.073), ('dynamics', 0.071), ('gibbs', 0.066), ('humerus', 0.062), ('therapy', 0.062), ('variational', 0.059), ('hq', 0.056), ('observations', 0.055), ('dmax', 0.054), ('barrett', 0.054), ('wam', 0.054), ('virtual', 0.054), ('sports', 0.054), ('generation', 0.054), ('device', 0.052), ('game', 0.05), ('em', 0.05), ('moves', 0.048), ('motion', 0.047), ('boundaries', 0.046), ('segments', 0.045), ('sampling', 0.044), ('trajectory', 0.043), ('recorded', 0.042), ('backhand', 0.041), ('dmin', 0.041), ('erdms', 0.041), ('gst', 0.041), ('lgssm', 0.041), ('listgarten', 0.041), ('yamane', 0.041), ('basic', 0.04), ('reality', 0.037), ('dof', 0.036), ('chiappa', 0.036), ('forehand', 0.036), ('vm', 0.036), ('recording', 0.036), ('freedom', 0.034), ('table', 0.034), ('durations', 0.033), ('humanoid', 0.033), ('routines', 0.033), ('shoulder', 0.033), ('movie', 0.033), ('template', 0.033), ('rotation', 0.033), ('games', 0.032), ('human', 0.032), ('silvia', 0.031), ('robotics', 0.031), ('count', 0.03), ('occurrences', 0.029), ('variables', 0.029), ('imitation', 0.028), ('hm', 0.027), ('velocities', 0.026), ('underlying', 0.026), ('transformation', 0.026), ('inference', 0.025), ('neal', 0.025), ('generated', 0.025), ('segment', 0.025), ('extracting', 0.025), ('going', 0.025), ('beginning', 0.025), ('segmentations', 0.024), ('map', 0.024), ('positions', 0.024), ('moved', 0.024), ('physical', 0.023), ('update', 0.023), ('degrees', 0.023), ('concatenation', 0.023), ('recordings', 0.023), ('accomplished', 0.022), ('continuous', 0.021)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.99999994 171 nips-2010-Movement extraction by detecting dynamics switches and repetitions

Author: Silvia Chiappa, Jan R. Peters

2 0.15017895 29 nips-2010-An Approximate Inference Approach to Temporal Optimization in Optimal Control

Author: Konrad Rawlik, Marc Toussaint, Sethu Vijayakumar

Abstract: Algorithms based on iterative local approximations present a practical approach to optimal control in robotic systems. However, they generally require the temporal parameters (for e.g. the movement duration or the time point of reaching an intermediate goal) to be speciﬁed a priori. Here, we present a methodology that is capable of jointly optimizing the temporal parameters in addition to the control command proﬁles. The presented approach is based on a Bayesian canonical time formulation of the optimal control problem, with the temporal mapping from canonical to real time parametrised by an additional control variable. An approximate EM algorithm is derived that efﬁciently optimizes both the movement duration and control commands offering, for the ﬁrst time, a practical approach to tackling generic via point problems in a systematic way under the optimal control framework. The proposed approach, which is applicable to plants with non-linear dynamics as well as arbitrary state dependent and quadratic control costs, is evaluated on realistic simulations of a redundant robotic plant.

3 0.10978809 179 nips-2010-Natural Policy Gradient Methods with Parameter-based Exploration for Control Tasks

Author: Atsushi Miyamae, Yuichi Nagata, Isao Ono, Shigenobu Kobayashi

Abstract: In this paper, we propose an efﬁcient algorithm for estimating the natural policy gradient using parameter-based exploration; this algorithm samples directly in the parameter space. Unlike previous methods based on natural gradients, our algorithm calculates the natural policy gradient using the inverse of the exact Fisher information matrix. The computational cost of this algorithm is equal to that of conventional policy gradients whereas previous natural policy gradient methods have a prohibitive computational cost. Experimental results show that the proposed method outperforms several policy gradient methods. 1

4 0.10943649 167 nips-2010-Mixture of time-warped trajectory models for movement decoding

Author: Elaine Corbett, Eric Perreault, Konrad Koerding

Abstract: Applications of Brain-Machine-Interfaces typically estimate user intent based on biological signals that are under voluntary control. For example, we might want to estimate how a patient with a paralyzed arm wants to move based on residual muscle activity. To solve such problems it is necessary to integrate obtained information over time. To do so, state of the art approaches typically use a probabilistic model of how the state, e.g. position and velocity of the arm, evolves over time – a so-called trajectory model. We wanted to further develop this approach using two intuitive insights: (1) At any given point of time there may be a small set of likely movement targets, potentially identified by the location of objects in the workspace or by gaze information from the user. (2) The user may want to produce movements at varying speeds. We thus use a generative model with a trajectory model incorporating these insights. Approximate inference on that generative model is implemented using a mixture of extended Kalman filters. We find that the resulting algorithm allows us to decode arm movements dramatically better than when we use a trajectory model with linear dynamics. 1 In trod u cti on When patients have lost a limb or the ability to communicate with the outside world, brain machine interfaces (BMIs) are often used to enable robotic prostheses or restore communication. To achieve this, the user's intended state of the device must be decoded from biological signals. In the context of Bayesian statistics, two aspects are important for the design of an estimator of a temporally evolving state: the observation model, which describes how measured variables relate to the system’s state and the trajectory model which describes how the state changes over time in a probabilistic manner. Following this logic many recent BMI applications have relied on Bayesian estimation for a wide range of problems including the decoding of intended human [1] and animal [2] movements. In the context of BMIs, Bayesian approaches offer a principled way of formalizing the uncertainty about signals and thus often result in improvements over other signal processing techniques [1]-[3]. Most work on state estimation in dynamical systems has assumed linear dynamics and Gaussian noise. Under these circumstances, efficient algorithms result from belief propagation. The most frequent application uses the Kalman filter (KF), which recursively combines noisy state observations with the probabilistic evolution of state defined by the trajectory model to estimate the marginal distribution over states [4]. Such approaches have been used widely for applications including upper [1] and lower [5] extremity prosthetic 1 devices, functional electric stimulation [6] and human computer interactions [7]. As these algorithms are so commonly used, it seems promising to develop extensions to nonlinear trajectory models that may better describe the probabilistic distribution of movements in everyday life. One salient departure from the standard assumptions is that people tend to produce both slow and fast movements, depending on the situation. Models with linear dynamics only allow such deviation through the noise term, which makes these models poor at describing the natural variation of movement speeds during real world tasks. Explicitly incorporating movement speed into the trajectory model should lead to better movement estimates. Knowledge of the target position should also strongly affect trajectory models. After all , we tend to accelerate our arm early during movement and slow down later on. Target information can be linearly incorporated into the trajectory model, and this has greatly improved predictions [8]-[12]. Alternatively, if there are a small number of potential targets then a mixture of trajectory models approach [13] can be used. Here we are interested in the case where available data provide a prior over potential t argets but where movement targets may be anywhere. We want to incorporate target uncertainty and allow generalization to novel targets. Prior information about potential targets could come from a number of sources but would generally be noisy. For example, activity in the dorsal premotor cortex provides information about intended target location prior to movement and may be used where such recordings are available [14]. Target information may also be found noninvasively by tracking eye movements. However, such data will generally provide non-zero priors for a number of possible target locations as the subject saccades over the scene. While subjects almost always look at a target before reaching for it [15], there may be a delay of up to a second between looking at the target and the reach – a time interval over which up to 3 saccades are typically made. Each of these fixations could be the target. Hence, a probabilistic distribution of targets is appropriate when using either neural recordings or eye tracking to estimate potential reach targets Here we present an algorithm that uses a mixture of extended Kalman Filters (EKFs) to combine our insights related to the variation of movement speed and the availability of probabilistic target knowledge. Each of the mixture component s allows the speed of the movement to vary continuously over time. We tested how well we could use EMGs and eye movements to decode hand position of humans performing a three -dimensional large workspace reaching task. We find that using a trajectory model that allows for probabilistic target information and variation of speed leads to dramatic improvements in decoding quality. 2 Gen e ral Decod i n g S etti n g We wanted to test how well different decoding algorithms can decode human movement, over a wide range of dynamics. While many recent studies have looked at more restrictive, two-dimensional movements, a system to restore arm function should produce a wide range of 3D trajectories. We recorded arm kinematics and EMGs of healthy subjects during unconstrained 3D reaches to targets over a large workspace. Two healthy subjects were asked to reach at slow, normal and fast speeds, as they would in everyday life. Subjects were seated as they reached towards 16 LEDs in blocks of 150s, which were located on two planes positioned such that all targets were just reachable (Fig 1A). The target LED was lit for one second prior to an auditory go cue, at which time the subject would reach to the target at the appropriate speed. Slow, normal and fast reaches were allotted 3 s, 1.5s and 1s respectively; however, subjects determined the speed. An approximate total of 450 reaches were performed per subject. The subjects provided informed consent, and the protocol was approved by the Northwestern University Institutional Review Board. EMG signals were measured from the pectoralis major, and the three deltoid muscles of the shoulder. This represents a small subset of the muscles involved in reaching, and approximates those muscles retaining some voluntary control following mid-level cervical spinal cord injuries. 2 The EMG signals were band-pass filtered between 10 and 1,000 Hz, and subsequently anti aliased filtered. Hand, wrist, shoulder and head positions were tracked using an Optotrak motion analysis system. We simultaneously recorded eye movements with an ASL EYETRAC-6 head mounted eye tracker. Approximately 25% of the reaches were assigned to the test set, and the rest were used for training. Reaches for which either the motion capture data was incomplete, or there was visible motion artifact on the EMG were removed. As the state we used hand positions and joint angles (3 shoulder, 2 elbow, position, velocity and acceleration, 24 dimensions). Joint angles were calculated from the shoulder and wrist marker data using digitized bony landmarks which defined a coordinate system for the upper limb as detailed by Wu et al. [16]. As the motion data were sampled at 60Hz, the mean absolute value o f the EMG in the corresponding 16.7ms windows was used as an observation of the state at each time-step. Algorithm accuracy was quantified by normalizing the root -mean-squared error by the straight line distance between the first and final position of the endpoint for each reach. We compared the algorithms statistically using repeated measures ANOVAs with Tukey post -hoc tests, treating reach and subject as random effects. In the rest of the paper we will ask how well these reaching movements can be decoded from EMG and eye-tracking data. Figure 1: A Experimental setup and B sample kinematics and processed EMGs for one reach 3 Kal man Fi l ters w i th Target i n f ormati on All models that we consider in this paper assume linear observations with Gaussian noise: (1) where x is the state, y is the observation and v is the measurement noise with p(v) ~ N(0,R), and R is the observation covariance matrix. The model fitted the measured EMGs with an average r2 of 0.55. This highlights the need to integrate information over time. The standard approach also assumes linear dynamics and Gaussian process noise: (2) where, x t represents the hand and joint angle positions, w is the process noise with p(w) ~ N(0,Q), and Q is the state covariance matrix. The Kalman filter does optimal inference for this generative model. This model can effectively capture the dynamics of stereotypical reaches to a single target by appropriately tuning its parameters. However, when used to describe reaches to multiple targets, the model cannot describe target dependent aspects of reaching but boils down to a random drift model. Fast velocities are underestimated as they are unlikely under the trajectory model and there is excessive drift close to the target (Fig. 2A). 3 In many decoding applications we may know the subject’s target. A range of recent studies have addressed the issue of incorporating this information into the trajectory model [8, 13], and we might assume the effect of the target on the dynamics to be linear. This naturally suggests adding the target to the state space, which works well in practice [9, 12]. By appending the target to the state vector (KFT), the simple linear format of the KF may be retained: (3) where xTt is the vector of target positions, with dimensionality less than or equal to that of xt. This trajectory model thus allows describing both the rapid acceleration that characterizes the beginning of a reach and the stabilization towards its end. We compared the accuracy of the KF and the KFT to the Single Target Model (STM), a KF trained only on reaches to the target being tested (Fig. 2). The STM represents the best possible prediction that could be obtained with a Kalman filter. Assuming the target is perfectly known, we implemented the KFT by correctly initializing the target state xT at the beginning of the reach. We will relax this assumption below. The initial hand and joint angle positions were also assumed to be known. Figure 2: A Sample reach and predictions and B average accuracies with standard errors for KFT, KF and MTM. Consistent with the recent literature, both methods that incorporated target information produced higher prediction accuracy than the standard KF (both p<0.0001). Interestingly, there was no significant difference between the KFT and the STM (p=0.9). It seems that when we have knowledge of the target, we do not lose much by training a single model over the whole workspace rather than modeling the targets individually. This is encouraging, as we desire a BMI system that can generalize to any target within the workspace, not just specifically to those that are available in the training data. Clearly, adding the target to the state space allows the dynamics of typical movements to be modeled effectively, resulting in dramatic increases in decoding performance. 4 Ti me Warp i n g 4.1 I m p l e m e n t i n g a t i m e - w a r p e d t r a j e c t o r y mo d e l While the KFT above can capture the general reach trajectory profile, it does not allow for natural variability in the speed of movements. Depending on our task objectives, which would not directly be observed by a BMI, we might lazily reach toward a target or move a t maximal speed. We aim to change the trajectory model to explicitly incorporate a warping factor by which the average movement speed is scaled, allowing for such variability. As the movement speed will be positive in all practical cases, we model the logarithm of this factor, 4 and append it to the state vector: (4) We create a time-warped trajectory model by noting that if the average rate of a trajectory is to be scaled by a factor S, the position at time t will equal that of the original trajectory at time St. Differentiating, the velocity will be multiplied by S, and the acceleration by S 2. For simplicity, the trajectory noise is assumed to be additive and Gaussian, and the model is assumed to be stationary: (5) where Ip is the p-dimensional identity matrix and is a p p matrix of zeros. Only the terms used to predict the acceleration states need to be estimated to build the state transition matrix, and they are scaled as a nonlinear function of xs. After adding the variable movement speed to the state space the system is no longer linear. Therefore we need a different solution strategy. Instead of the typical KFT we use the Extended Kalman Filter (EKFT) to implement a nonlinear trajectory model by linearizing the dynamics around the best estimate at each time-step [17]. With this approach we add only small computational overhead to the KFT recursions. 4.2 Tr a i n i n g t h e t i m e w a r p i n g mo d e l The filter parameters were trained using a variant of the Expectation Maximization (EM) algorithm [18]. For extended Kalman filter learning the initialization for the variables may matter. S was initialized with the ground truth average reach speeds for each movement relative to the average speed across all movements. The state transition parameters were estimated using nonlinear least squares regression, while C, Q and R were estimated linearly for the new system, using the maximum likelihood solution [18] (M-step). For the E-step we used a standard extended Kalman smoother. We thus found the expected values for t he states given the current filter parameters. For this computation, and later when testing the algorithm, xs was initialized to its average value across all reaches while the remaining states were initialized to their true values. The smoothed estimate fo r xs was then used, along with the true values for the other states, to re-estimate the filter parameters in the M-step as before. We alternated between the E and M steps until the log likelihood converged (which it did in all cases). Following the training procedure, the diagonal of the state covariance matrix Q corresponding to xs was set to the variance of the smoothed xs over all reaches, according to how much this state should be allowed to change during prediction. This allowed the estimate of xs to develop over the course of the reach due to the evidence provided by the observations, better capturing the dynamics of reaches at different speeds. 4.3 P e r f o r ma n c e o f t h e t i m e - w a r p e d E K F T Incorporating time warping explicitly into the trajectory model pro duced a noticeable increase in decoding performance over the KFT. As the speed state xs is estimated throughout the course of the reach, based on the evidence provided by the observations, the trajectory model has the flexibility to follow the dynamics of the reach more accurately (Fig. 3). While at the normal self-selected speed the difference between the algorithms is small, for the slow and fast speeds, where the dynamics deviate from average, there i s a clear advantage to the time warping model. 5 Figure 3: Hand positions and predictions of the KFT and EKFT for sample reaches at A slow, B normal and C fast speeds. Note the different time scales between reaches. The models were first trained using data from all speeds (Fig. 4A). The EKFT was 1.8% more accurate on average (p<0.01), and the effect was significant at the slow (1.9%, p<0.05) and the fast (2.8%, p<0.01), but not at the normal (p=0.3) speed. We also trained the models from data using only reaches at the self-selected normal speed, as we wanted to see if there was enough variation to effectively train the EKFT (Fig. 4B). Interestingly, the performance of the EKFT was reduced by only 0.6%, and the KFT by 1.1%. The difference in performance between the EKFT and KFT was even more pronounced on aver age (2.3%, p<0.001), and for the slow and fast speeds (3.6 and 4.1%, both p< 0.0001). At the normal speed, the algorithms again were not statistically different (p=0.6). This result demonstrates that the EKFT is a practical option for a real BMI system, as it is not necessary to greatly vary the speeds while collecting training data for the model to be effective over a wide range of intended speeds. Explicitly incorporating speed information into the trajectory model helps decoding, by modeling the natural variation in volitional speed. Figure 4: Mean and standard error of EKFT and KFT accuracy at the different subjectselected speeds. Models were trained on reaches at A all speeds and B just normal speed reaches. Asterisks indicate statistically significant differences between the algorithms. 5 Mi xtu res of Target s So far, we have assumed that the targets of our reaches are perfectly known. In a real-world system, there will be uncertainty about the intended target of the reach. However, in typical applications there are a small number of possible objectives. Here we address this situation. Drawing on the recent literature, we use a mixture model to consider each of the possible targets [11, 13]. We condition the posterior probability for the state on the N possible targets, T: (6) 6 Using Bayes' Rule, this equation becomes: (7) As we are dealing with a mixture model, we perform the Kalman filter recursion for each possible target, xT, and our solution is a weighted sum of the outputs. The weights are proportional to the prior for that target, , and the likelihood of the model given that target . is independent of the target and does not need to be calculated. We tested mixtures of both algorithms, the mKFT and mEKFT, with real uncert ain priors obtained from eye-tracking in the one-second period preceding movement. As the targets were situated on two planes, the three-dimensional location of the eye gaze was found by projecting its direction onto those planes. The first, middle and last eye samples were selected, and all other samples were assigned to a group according to which of the three was closest. The mean and variance of these three groups were used to initialize three Kalman filters in the mixture model. The priors of the three groups were assigned proportional to the number of samples in them. If the subject looks at multiple positions prior to reaching, this method ensures with a high probability that the correct target was accounted for in one of the filters in the mixture. We also compared the MTM approach of Yu et al. [13], where a different KF model was generated for each target, and a mixture is performed over these models. This approach explicitly captures the dynamics of stereotypical reaches to specific targets. Given perfect target information, it would reduce to the STM described above. Priors for the MTM were found by assigning each valid eye sample to its closest two targets, and weighting the models proportional to the number of samples assigned to the corresponding target, divided by its distance from the mean of those samples. We tried other ways of assigning priors and the one presented gave the best results. We calculated the reduction in decoding quality when instead of perfect priors we provide eye-movement based noisy priors (Fig. 5). The accuracies of the mEKFT, the mKFT and the MTM were only degraded by 0.8, 1.9 and 2.1% respectively, compared to the perfect prior situation. The mEKFT was still close to 10% better than the KF. The mixture model framework is effective in accounting for uncertain priors. Figure 5: Mean and standard errors of accuracy for algorithms with perfect priors, and uncertain priors with full and partial training set. The asterisk indicates a statistically significant effects between the two training types, where real priors are used. Here, only reaches at normal speed were used to train the models, as this is a more realistic training set for a BMI application. This accounts for the degraded performance of the MTM with perfect priors relative to the STM from above (Fig. 2). With even more stereotyped training data for each target, the MTM doesn't generalize as well to new speeds. 7 We also wanted to know if the algorithms could generalize to new targets. In a real application, the available training data will generally not span the entire useable worksp ace. We compared the algorithms where reaches to all targets except the one being tested had been used to train the models. The performance of the MTM was significantly de graded unsurprisingly, as it was designed for reaches to a set of known targets. Performance of the mKFT and mEKFT degraded by about 1%, but not significantly (both p>0.7), demonstrating that the continuous approach to target information is preferable when the target could be anywhere in space, not just at locations for which training data is available. 6 Di scu ssi on and concl u si on s The goal of this work was to design a trajectory model that would improve decoding for BMIs with an application to reaching. We incorporated two features that prominently influence the dynamics of natural reach: the movement speed and the target location. Our approach is appropriate where uncertain target information is available. The model generalizes well to new regions of the workspace for which there is no training data, and across a broad range of reaching dynamics to widely spaced targets in three dimensions. The advantages over linear models in decoding precision we report here could be equally obtained using mixtures over many targets and speeds. While mixture models [11, 13] could allow for slow versus fast movements and any number of potential targets, this strategy will generally require many mixture components. Such an approach would require a lot more training data, as we have shown that it does not generalize well. It would also be run-time intensive which is problematic for prosthetic devices that rely on low power controllers. In contrast, the algorithm introduced here only takes a small amount of additional run-time in comparison to the standard KF approach. The EKF is only marginally slower than the standard KF and the algorithm will not generally need to consider more than 3 mixture components assuming the subject fixates the target within the second pre ceding the reach. In this paper we assumed that subjects always would fixate a reach target – along with other non-targets. While this is close to the way humans usually coordinate eyes and reaches [15], there might be cases where people do not fixate a reach target. Our approach could be easily extended to deal with such situations by adding a dummy mixture component that all ows the description of movements to any target. As an alternative to mixture approaches, a system can explicitly estimate the target position in the state vector [9]. This approach, however, would not straightforwardly allow for the rich target information available; we look at the target but also at other locations, strongly suggesting mixture distributions. A combination of the two approaches could further improve decoding quality. We could both estimate speed and target position for the EKFT in a continuous manner while retaining the mixture over target priors. We believe that the issues that we have addressed here are almost universal. Virtually all types of movements are executed at varying speed. A probabilistic distribution for a small number of action candidates may also be expected in most BMI applications – after all there are usually only a small number of actions that make sense in a given environment. While this work is presented in the context of decoding human reaching, it may be applied to a wide range of BMI applications including lower limb prosthetic devices and human computer interactions, as well as different signal sources such as electrode grid recordings and electroencephalograms. The increased user control in conveying their intended movements would significantly improve the functionality of a neuroprosthetic device. A c k n o w l e d g e me n t s T h e a u t h o r s t h a n k T. H a s w e l l , E . K r e p k o v i c h , a n d V. Ravichandran for assistance with experiments. This work was funded by the NSF Program in Cyber-Physical Systems. R e f e re n c e s [1] L.R. Hochberg, M.D. Serruya, G.M. Friehs, J.A. Mukand, M. Saleh, A.H. Caplan, A. Branner, D. 8 [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] Chen, R.D. Penn, and J.P. Donoghue, “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature, vol. 442, 2006, pp. 164–171. W. Wu, Y. Gao, E. Bienenstock, J.P. Donoghue, and M.J. Black, “Bayesian population decoding of motor cortical activity using a Kalman filter,” Neural Computation, vol. 18, 2006, pp. 80–118. W. Wu, M.J. Black, Y. Gao, E. Bienenstock, M. Serruya, A. Shaikhouni, and J.P. Donoghue, “Neural decoding of cursor motion using a Kalman filter,” Advances in Neural Information Processing Systems 15: Proceedings of the 2002 Conference, 2003, p. 133. R.E. Kalman, “A new approach to linear filtering and prediction problems,” Journal of basic Engineering, vol. 82, 1960, pp. 35–45. G.G. Scandaroli, G.A. Borges, J.Y. Ishihara, M.H. Terra, A.F.D. Rocha, and F.A.D.O. Nascimento, “Estimation of foot orientation with respect to ground for an above knee robotic prosthesis,” Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems, St. Louis, MO, USA: IEEE Press, 2009, pp. 1112-1117. I. Cikajlo, Z. Matjačić, and T. Bajd, “Efficient FES triggering applying Kalman filter during sensory supported treadmill walking,” Journal of Medical Engineering & Technology, vol. 32, 2008, pp. 133144. S. Kim, J.D. Simeral, L.R. Hochberg, J.P. Donoghue, and M.J. Black, “Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia,” Journal of Neural Engineering, vol. 5, 2008, pp. 455-476. L. Srinivasan, U.T. Eden, A.S. Willsky, and E.N. Brown, “A state-space analysis for reconstruction of goal-directed movements using neural signals,” Neural computation, vol. 18, 2006, pp. 2465–2494. G.H. Mulliken, S. Musallam, and R.A. Andersen, “Decoding trajectories from posterior parietal cortex ensembles,” Journal of Neuroscience, vol. 28, 2008, p. 12913. W. Wu, J.E. Kulkarni, N.G. Hatsopoulos, and L. Paninski, “Neural Decoding of Hand Motion Using a Linear State-Space Model With Hidden States,” IEEE Transactions on neural systems and rehabilitation engineering, vol. 17, 2009, p. 1. J.E. Kulkarni and L. Paninski, “State-space decoding of goal-directed movements,” IEEE Signal Processing Magazine, vol. 25, 2008, p. 78. C. Kemere and T. Meng, “Optimal estimation of feed-forward-controlled linear systems,” IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005. Proceedings.(ICASSP'05), 2005. B.M. Yu, C. Kemere, G. Santhanam, A. Afshar, S.I. Ryu, T.H. Meng, M. Sahani, and K.V. Shenoy, “Mixture of trajectory models for neural decoding of goal-directed movements,” Journal of neurophysiology, vol. 97, 2007, p. 3763. N. Hatsopoulos, J. Joshi, and J.G. O'Leary, “Decoding continuous and discrete motor behaviors using motor and premotor cortical ensembles,” Journal of neurophysiology, vol. 92, 2004, p. 1165. R.S. Johansson, G. Westling, A. Backstrom, and J.R. Flanagan, “Eye-hand coordination in object manipulation,” Journal of Neuroscience, vol. 21, 2001, p. 6917. G. Wu, F.C. van der Helm, H.E.J. Veeger, M. Makhsous, P. Van Roy, C. Anglin, J. Nagels, A.R. Karduna, and K. McQuade, “ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion–Part II: shoulder, elbow, wrist and hand,” Journal of biomechanics, vol. 38, 2005, pp. 981–992. D. Simon, Optimal state estimation: Kalman, H [infinity] and nonlinear approaches, John Wiley and Sons, 2006. Z. Ghahramani and G.E. Hinton, “Parameter estimation for linear dynamical systems,” University of Toronto technical report CRG-TR-96-2, vol. 6, 1996. 9

5 0.097274952 222 nips-2010-Random Walk Approach to Regret Minimization

Author: Hariharan Narayanan, Alexander Rakhlin

Abstract: We propose a computationally efﬁcient random walk on a convex body which rapidly mixes to a time-varying Gibbs distribution. In the setting of online convex optimization and repeated games, the algorithm yields low regret and presents a novel efﬁcient method for implementing mixture forecasting strategies. 1

6 0.096168816 66 nips-2010-Double Q-learning

7 0.089718245 14 nips-2010-A Reduction from Apprenticeship Learning to Classification

8 0.085847653 198 nips-2010-Optimal Web-Scale Tiering as a Flow Problem

9 0.08187148 193 nips-2010-Online Learning: Random Averages, Combinatorial Parameters, and Learnability

10 0.08069627 187 nips-2010-Occlusion Detection and Motion Estimation with Convex Optimization

11 0.067609243 57 nips-2010-Decoding Ipsilateral Finger Movements from ECoG Signals in Humans

12 0.065868117 235 nips-2010-Self-Paced Learning for Latent Variable Models

13 0.064664237 77 nips-2010-Epitome driven 3-D Diffusion Tensor image segmentation: on extracting specific structures

14 0.062617213 39 nips-2010-Bayesian Action-Graph Games

15 0.060522497 33 nips-2010-Approximate inference in continuous time Gaussian-Jump processes

16 0.05876619 281 nips-2010-Using body-anchored priors for identifying actions in single images

17 0.058686215 206 nips-2010-Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine

18 0.058129855 196 nips-2010-Online Markov Decision Processes under Bandit Feedback

19 0.057873398 254 nips-2010-Stability Approach to Regularization Selection (StARS) for High Dimensional Graphical Models

20 0.05775572 40 nips-2010-Beyond Actions: Discriminative Models for Contextual Group Activities

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.153), (1, -0.046), (2, -0.067), (3, 0.028), (4, -0.065), (5, 0.002), (6, -0.06), (7, -0.004), (8, 0.004), (9, 0.163), (10, -0.004), (11, -0.077), (12, 0.06), (13, -0.02), (14, 0.056), (15, 0.092), (16, -0.092), (17, 0.049), (18, -0.016), (19, -0.023), (20, -0.084), (21, 0.201), (22, 0.042), (23, 0.136), (24, 0.0), (25, -0.089), (26, -0.064), (27, 0.09), (28, -0.013), (29, -0.053), (30, 0.152), (31, 0.081), (32, 0.079), (33, -0.123), (34, -0.05), (35, 0.173), (36, -0.0), (37, 0.017), (38, -0.047), (39, 0.095), (40, 0.037), (41, -0.005), (42, 0.013), (43, 0.088), (44, -0.094), (45, -0.047), (46, 0.153), (47, 0.03), (48, 0.048), (49, 0.019)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.96366429 171 nips-2010-Movement extraction by detecting dynamics switches and repetitions

Author: Silvia Chiappa, Jan R. Peters

2 0.70119381 167 nips-2010-Mixture of time-warped trajectory models for movement decoding

Author: Elaine Corbett, Eric Perreault, Konrad Koerding

3 0.68231887 29 nips-2010-An Approximate Inference Approach to Temporal Optimization in Optimal Control

Author: Konrad Rawlik, Marc Toussaint, Sethu Vijayakumar

4 0.51765686 57 nips-2010-Decoding Ipsilateral Finger Movements from ECoG Signals in Humans

Author: Yuzong Liu, Mohit Sharma, Charles Gaona, Jonathan Breshears, Jarod Roland, Zachary Freudenburg, Eric Leuthardt, Kilian Q. Weinberger

Abstract: Several motor related Brain Computer Interfaces (BCIs) have been developed over the years that use activity decoded from the contralateral hemisphere to operate devices. Contralateral primary motor cortex is also the region most severely affected by hemispheric stroke. Recent studies have identiﬁed ipsilateral cortical activity in planning of motor movements and its potential implications for a stroke relevant BCI. The most fundamental functional loss after a hemispheric stroke is the loss of ﬁne motor control of the hand. Thus, whether ipsilateral cortex encodes ﬁnger movements is critical to the potential feasibility of BCI approaches in the future. This study uses ipsilateral cortical signals from humans (using ECoG) to decode ﬁnger movements. We demonstrate, for the ﬁrst time, successful ﬁnger movement detection using machine learning algorithms. Our results show high decoding accuracies in all cases which are always above chance. We also show that signiﬁcant accuracies can be achieved with the use of only a fraction of all the features recorded and that these core features are consistent with previous physiological ﬁndings. The results of this study have substantial implications for advancing neuroprosthetic approaches to stroke populations not currently amenable to existing BCI techniques. 1

5 0.50015038 50 nips-2010-Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories

Author: George Konidaris, Scott Kuindersma, Roderic Grupen, Andre S. Barreto

Abstract: We introduce CST, an algorithm for constructing skill trees from demonstration trajectories in continuous reinforcement learning domains. CST uses a changepoint detection method to segment each trajectory into a skill chain by detecting a change of appropriate abstraction, or that a segment is too complex to model as a single skill. The skill chains from each trajectory are then merged to form a skill tree. We demonstrate that CST constructs an appropriate skill tree that can be further reﬁned through learning in a challenging continuous domain, and that it can be used to segment demonstration trajectories on a mobile manipulator into chains of skills where each skill is assigned an appropriate abstraction. 1

6 0.48856172 244 nips-2010-Sodium entry efficiency during action potentials: A novel single-parameter family of Hodgkin-Huxley models

7 0.44204599 2 nips-2010-A Bayesian Approach to Concept Drift

8 0.42397517 66 nips-2010-Double Q-learning

9 0.39788729 262 nips-2010-Switched Latent Force Models for Movement Segmentation

10 0.36369953 226 nips-2010-Repeated Games against Budgeted Adversaries

11 0.35600483 39 nips-2010-Bayesian Action-Graph Games

12 0.35425982 40 nips-2010-Beyond Actions: Discriminative Models for Contextual Group Activities

13 0.35321671 28 nips-2010-An Alternative to Low-level-Sychrony-Based Methods for Speech Detection

14 0.35244122 121 nips-2010-Improving Human Judgments by Decontaminating Sequential Dependencies

15 0.35041341 281 nips-2010-Using body-anchored priors for identifying actions in single images

16 0.34802637 156 nips-2010-Learning to combine foveal glimpses with a third-order Boltzmann machine

17 0.33422419 255 nips-2010-Static Analysis of Binary Executables Using Structural SVMs

18 0.33397019 34 nips-2010-Attractor Dynamics with Synaptic Depression

19 0.32950372 198 nips-2010-Optimal Web-Scale Tiering as a Flow Problem

20 0.32448781 19 nips-2010-A rational decision making framework for inhibitory control

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(13, 0.026), (16, 0.282), (17, 0.012), (27, 0.09), (30, 0.045), (45, 0.268), (50, 0.051), (52, 0.017), (60, 0.012), (77, 0.071), (90, 0.035)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.84854448 171 nips-2010-Movement extraction by detecting dynamics switches and repetitions

Author: Silvia Chiappa, Jan R. Peters

2 0.83204299 136 nips-2010-Large-Scale Matrix Factorization with Missing Data under Additional Constraints

Author: Kaushik Mitra, Sameer Sheorey, Rama Chellappa

Abstract: Matrix factorization in the presence of missing data is at the core of many computer vision problems such as structure from motion (SfM), non-rigid SfM and photometric stereo. We formulate the problem of matrix factorization with missing data as a low-rank semideﬁnite program (LRSDP) with the advantage that: 1) an efﬁcient quasi-Newton implementation of the LRSDP enables us to solve large-scale factorization problems, and 2) additional constraints such as orthonormality, required in orthographic SfM, can be directly incorporated in the new formulation. Our empirical evaluations suggest that, under the conditions of matrix completion theory, the proposed algorithm ﬁnds the optimal solution, and also requires fewer observations compared to the current state-of-the-art algorithms. We further demonstrate the effectiveness of the proposed algorithm in solving the afﬁne SfM problem, non-rigid SfM and photometric stereo problems.

3 0.81901461 219 nips-2010-Random Conic Pursuit for Semidefinite Programming

Author: Ariel Kleiner, Ali Rahimi, Michael I. Jordan

Abstract: We present a novel algorithm, Random Conic Pursuit, that solves semideﬁnite programs (SDPs) via repeated optimization over randomly selected two-dimensional subcones of the PSD cone. This scheme is simple, easily implemented, applicable to very general SDPs, scalable, and theoretically interesting. Its advantages are realized at the expense of an ability to readily compute highly exact solutions, though useful approximate solutions are easily obtained. This property renders Random Conic Pursuit of particular interest for machine learning applications, in which the relevant SDPs are generally based upon random data and so exact minima are often not a priority. Indeed, we present empirical results to this effect for various SDPs encountered in machine learning; these experiments demonstrate the potential practical usefulness of Random Conic Pursuit. We also provide a preliminary analysis that yields insight into the theoretical properties and convergence of the algorithm. 1

4 0.79946262 1 nips-2010-(RF)^2 -- Random Forest Random Field

Author: Nadia Payet, Sinisa Todorovic

Abstract: We combine random forest (RF) and conditional random ﬁeld (CRF) into a new computational framework, called random forest random ﬁeld (RF)2 . Inference of (RF)2 uses the Swendsen-Wang cut algorithm, characterized by MetropolisHastings jumps. A jump from one state to another depends on the ratio of the proposal distributions, and on the ratio of the posterior distributions of the two states. Prior work typically resorts to a parametric estimation of these four distributions, and then computes their ratio. Our key idea is to instead directly estimate these ratios using RF. RF collects in leaf nodes of each decision tree the class histograms of training examples. We use these class histograms for a nonparametric estimation of the distribution ratios. We derive the theoretical error bounds of a two-class (RF)2 . (RF)2 is applied to a challenging task of multiclass object recognition and segmentation over a random ﬁeld of input image regions. In our empirical evaluation, we use only the visual information provided by image regions (e.g., color, texture, spatial layout), whereas the competing methods additionally use higher-level cues about the horizon location and 3D layout of surfaces in the scene. Nevertheless, (RF)2 outperforms the state of the art on benchmark datasets, in terms of accuracy and computation time.

5 0.78497034 212 nips-2010-Predictive State Temporal Difference Learning

Author: Byron Boots, Geoffrey J. Gordon

Abstract: We propose a new approach to value function approximation which combines linear temporal difference reinforcement learning with subspace identiﬁcation. In practical applications, reinforcement learning (RL) is complicated by the fact that state is either high-dimensional or partially observable. Therefore, RL methods are designed to work with features of state rather than state itself, and the success or failure of learning is often determined by the suitability of the selected features. By comparison, subspace identiﬁcation (SSID) methods are designed to select a feature set which preserves as much information as possible about state. In this paper we connect the two approaches, looking at the problem of reinforcement learning with a large set of features, each of which may only be marginally useful for value function approximation. We introduce a new algorithm for this situation, called Predictive State Temporal Difference (PSTD) learning. As in SSID for predictive state representations, PSTD ﬁnds a linear compression operator that projects a large set of features down to a small set that preserves the maximum amount of predictive information. As in RL, PSTD then uses a Bellman recursion to estimate a value function. We discuss the connection between PSTD and prior approaches in RL and SSID. We prove that PSTD is statistically consistent, perform several experiments that illustrate its properties, and demonstrate its potential on a difﬁcult optimal stopping problem. 1

6 0.71783209 38 nips-2010-Batch Bayesian Optimization via Simulation Matching

7 0.71781272 277 nips-2010-Two-Layer Generalization Analysis for Ranking Using Rademacher Average

8 0.71776319 224 nips-2010-Regularized estimation of image statistics by Score Matching

9 0.71678233 239 nips-2010-Sidestepping Intractable Inference with Structured Ensemble Cascades

10 0.71670312 17 nips-2010-A biologically plausible network for the computation of orientation dominance

11 0.71599048 158 nips-2010-Learning via Gaussian Herding

12 0.7158249 218 nips-2010-Probabilistic latent variable models for distinguishing between cause and effect

13 0.71454698 103 nips-2010-Generating more realistic images using gated MRF's

14 0.71446389 98 nips-2010-Functional form of motion priors in human motion perception

15 0.71363586 155 nips-2010-Learning the context of a category

16 0.71318763 85 nips-2010-Exact learning curves for Gaussian process regression on large random graphs

17 0.71277684 29 nips-2010-An Approximate Inference Approach to Temporal Optimization in Optimal Control

18 0.71269214 282 nips-2010-Variable margin losses for classifier design

19 0.71148604 238 nips-2010-Short-term memory in neuronal networks through dynamical compressed sensing

20 0.71113008 185 nips-2010-Nonparametric Density Estimation for Stochastic Optimization with an Observable State Variable