nips nips2010 nips2010-29 knowledge-graph by maker-knowledge-mining

29 nips-2010-An Approximate Inference Approach to Temporal Optimization in Optimal Control

Source: pdf

Author: Konrad Rawlik, Marc Toussaint, Sethu Vijayakumar

Abstract: Algorithms based on iterative local approximations present a practical approach to optimal control in robotic systems. However, they generally require the temporal parameters (for e.g. the movement duration or the time point of reaching an intermediate goal) to be speciﬁed a priori. Here, we present a methodology that is capable of jointly optimizing the temporal parameters in addition to the control command proﬁles. The presented approach is based on a Bayesian canonical time formulation of the optimal control problem, with the temporal mapping from canonical to real time parametrised by an additional control variable. An approximate EM algorithm is derived that efﬁciently optimizes both the movement duration and control commands offering, for the ﬁrst time, a practical approach to tackling generic via point problems in a systematic way under the optimal control framework. The proposed approach, which is applicable to plants with non-linear dynamics as well as arbitrary state dependent and quadratic control costs, is evaluated on realistic simulations of a redundant robotic plant.

Reference: text

Summary: the most important sentenses genereted by tfidf model

sentIndex sentText sentNum sentScore

1 the movement duration or the time point of reaching an intermediate goal) to be speciﬁed a priori. [sent-5, score-1.043]

2 Here, we present a methodology that is capable of jointly optimizing the temporal parameters in addition to the control command proﬁles. [sent-6, score-0.263]

3 The presented approach is based on a Bayesian canonical time formulation of the optimal control problem, with the temporal mapping from canonical to real time parametrised by an additional control variable. [sent-7, score-0.746]

4 An approximate EM algorithm is derived that efﬁciently optimizes both the movement duration and control commands offering, for the ﬁrst time, a practical approach to tackling generic via point problems in a systematic way under the optimal control framework. [sent-8, score-1.238]

5 The proposed approach, which is applicable to plants with non-linear dynamics as well as arbitrary state dependent and quadratic control costs, is evaluated on realistic simulations of a redundant robotic plant. [sent-9, score-0.366]

6 Although control based on an optimality criterion is certainly attractive, practical approaches for stochastic systems are currently limited to the ﬁnite horizon [9, 16] or ﬁrst exit time formulation [14, 17]. [sent-13, score-0.402]

7 , duration or the time when costs for speciﬁc sub goals of the problem are incurred, assuming them as given a priori. [sent-16, score-0.607]

8 This question is non trivial and important even while considering a simple reaching problem. [sent-18, score-0.21]

9 This can result in not reaching the goal, having to use unrealistic range of control commands or excessive (wasteful) durations for short distance tasks. [sent-20, score-0.576]

10 Although this limitation could be overcome by chaining together individual time optimal single goal controllers, such a sequential approach has several drawbacks. [sent-23, score-0.141]

11 First, if we are interested in placing a cost on overall movement duration, we are restricted to linear costs if we wish to remain time optimal. [sent-24, score-0.757]

12 A second more important ﬂaw is that 1 future goals should inﬂuence our control even before we have achieved the previous goal, a problem which we highlight during our comparative simulation studies. [sent-25, score-0.29]

13 A wide variety of successful approaches to address stochastic optimal control problems have been described in the literature [6, 2, 7]. [sent-26, score-0.227]

14 The approach we present here builds on a class of approximate stochastic optimal control methods which have been successfully used in the domain of robotic manipulators and in particular, the iLQG [9] algorithm used by [10], and the Approximate Inference Control (AICO) algorithm [16]. [sent-27, score-0.276]

15 These approaches, as alluded to earlier, are ﬁnite horizon formulations and consequently require the temporal structure of the problem to be ﬁxed a priori. [sent-28, score-0.163]

16 This requirement is a direct consequence of a ﬁxed length discretization of the continuous problem and the structure of the temporally non-stationary cost function used, which binds incurrence of goal related costs to speciﬁc (discretised) time points. [sent-29, score-0.408]

17 The fundamental idea proposed here is to reformulate the problem in canonical time and alternately optimize the temporal and spatial trajectories. [sent-30, score-0.22]

18 We implement this general approach in the context of the approximate inference formulation of AICO, leading to an Expectation Maximisation (EM) algorithm where the E-Step reduces to the standard inference control problem. [sent-31, score-0.265]

19 The closed loop stochastic optimal control problem is to ﬁnd the policy π : x(t) → u(t) given by π ∗ = argmin Ex,u|π,x(0) {L(x(·), u(·))} . [sent-41, score-0.227]

20 k L(x1:K , u1:K ) = CK (xK ) + (5) k=0 Note that here we used the Euler Forward Method as the discretization scheme, which will prove advantageous if a linear cost on the movement duration is chosen, leading to closed form solution for certain optimization problems. [sent-43, score-0.956]

21 1 Approximate Inference Control Recently, it has been suggested to consider a Bayesian inference approach [16] to (discreet) optimal control problems formalised in Section 2. [sent-46, score-0.232]

22 xK rK (b) Figure 1: The graphical models for (a) standard inference control and (b) the AICO-T model with canonical time. [sent-53, score-0.292]

23 , we interpret the cost as a negative log likelihood of task fulﬁlment. [sent-58, score-0.206]

24 Inference control consists of computing the posterior conditioned on the observation r0:K = 1 within the resulting model (illustrated as a graphical model in Fig. [sent-59, score-0.224]

25 For cases, where the process and cost are linear and quadratic in u respectively, the controls can be marginalised in closed form and one is left with the problem of computing the posterior P (x0:K |r0:K = 1) = N (xk+1 |xk + F(xk )∆t , W∆t ) exp(−∆t Ck (xk )) , (7) k with W := Q + BH−1 B⊤. [sent-61, score-0.321]

26 3 Temporal Optimization for Optimal Control Often the state dependent cost term C(x, t) in (2) can be split into a set of costs which are incurred only at speciﬁc times: also referred to as goals, and others which are independent of time, that is N δt=tn Vn (x) . [sent-64, score-0.433]

27 For instance, in a reaching movement, generally a cost that is a function of the distance to the target is incurred only at the ﬁnal time T while collision costs are independent of time and incurred throughout the movement. [sent-66, score-0.823]

28 In order to allow the time point at which the goals are achieved to be inﬂuenced by the optimization, we will re-frame the goal driven part of the problem in a canonical time and in addition to optimizing the controls, also optimize the mapping from canonical to real time. [sent-67, score-0.394]

29 Speciﬁcally, we introduce into the problem deﬁned by (1) & (2) the canonical time variable τ with the associated mapping t τ = β(t) = 0 1 ds , θ(s) θ(·) > 0 , (9) with θ as an additional control. [sent-68, score-0.161]

30 The real time behaviour is mainly speciﬁed by the additional ˆ cost term T over the new controls θ which we have introduced. [sent-71, score-0.276]

31 , T is equivalent to a cost on the total movement duration. [sent-74, score-0.581]

32 Although here we will stick to the linear case, the proposed approach is also applicable to non-linear duration costs. [sent-75, score-0.337]

33 We brieﬂy note the similarity of the formulation to the canonical time formulation of [11] used in an imitation learning setting. [sent-76, score-0.184]

34 Using this time step sequence and (4) we can now obtain a discreet process in terms of the canonical time with an explicit dependence on θ0:K−1 . [sent-79, score-0.248]

35 Discretization of the cost in (10) gives K−1 N L(x1:K , u1:K , θ0:K−1 ) = T (θk ) + J (xk )θk + u⊤Hθk uk , k Vn (xkn ) + ˆ n=1 (11) k=0 ˆ for some given k1:N . [sent-80, score-0.206]

36 We now have a new formulation of the optimal control problem that no longer of the form of equations (4) & (5), e. [sent-81, score-0.227]

37 Proceeding as for standard inference control and treating the cost (11) as a neg-log likelihood of an auxiliary binary dynamic random variable, we obtain the inference problem illustrated by the Bayesian network in Figure 1(b). [sent-84, score-0.385]

38 (12) However, as will be shown below we actually only require the expectations xk x⊤ and xk x⊤ k k+1 during the M-Step. [sent-99, score-0.95]

39 As these are in general not tractable, we compute a Gaussian approximation to the posterior, following an approximate message passing approach with linear and quadratic approximations to the dynamics and cost respectively [16] (for details, refer to supplementary material). [sent-100, score-0.234]

40 The required expectations, 2 under the assumption of constant θ(·) during each step 4 J (xk ) and log P(xk+1 |xk , θk ) = − 1 Dx −1 (xk+1 − F(xk ))⊤ Wk (xk+1 − F(xk )) log |Wk | − 2 2 , (15) with F(xk ) = xk + F(xk )θk and Wk = θk W, are in general not tractable. [sent-105, score-0.475]

41 Therefore, we take approximations F(xk ) ≈ ak + Ak xk and J (xk ) ≈ 1 ⊤ x Jk xk − j⊤xk , k 2 k (16) choosing the mean of q i (xk ) as the point of approximation, consistent with the equivalent approximations made in the E-Step. [sent-106, score-1.141]

42 On the other hand, the algorithm may lead to control frequencies which are not achievable in practice. [sent-116, score-0.171]

43 In general, a ﬁxed control signal frequency may be prescribed by the hardware system. [sent-117, score-0.2]

44 Finally, although we have chosen to express the time cost in terms of a function of the θ’s, often it may be desirable to consider a cost directly over the duration T . [sent-119, score-0.65]

45 The state ˙ ˙ of the plant is given by x = (q, q), with q ∈ R2 the joint angles and q ∈ R2 associated angular 5 600 AICO-T(α = α0 ) AICO-T(α = 2α) AICO-T(α = 0. [sent-123, score-0.137]

46 (a & b) Effect of changing time-cost weight α, (effectively the ratio between reaching cost and duration cost) on (a) duration and (b) reaching cost (control + state cost). [sent-144, score-1.376]

47 (c) Comparison of reaching costs (control + error cost) for AICO-T and a ﬁxed duration approach, i. [sent-145, score-0.66]

48 For all experiments, we used a quadratic control cost and the state dependent cost term: ∗ ∗ δk=ki (φi (xk ) − yi )⊤Λi (φi (xk ) − yi ) , ˆ V(xk ) = (18) i ∗ ˆ for some given ki and employed a diagonal weight matrix Λi while yi represented the desired state in task space. [sent-151, score-0.65]

49 1 Variable Distance Reaching Task In order to evaluate the behaviour of AICO-T we applied it to a reaching task with varying starttarget distance. [sent-157, score-0.302]

50 Speciﬁcally, for a ﬁxed start point we considered a series of targets lying equally spaced along a line in task space. [sent-158, score-0.174]

51 It should be noted that although the targets are equally spaced in task space and results are shown with respect to movement distance in task space, the distances in joint space scale non linearly. [sent-159, score-0.688]

52 The state cost (18) contained a single term incurred at the ﬁnal discrete step with Λ = 106 · I and the control cost were given by H = 104 · I. [sent-160, score-0.589]

53 2(a & b) shows the movement duration (= θk ) and standard reaching cost3 for different temporal-cost parameters α (we used α0 = 2·107), demonstrating that AICO-T successfully trades-off the movement duration and standard reaching cost for varying movement distances. [sent-162, score-2.495]

54 2(c), we compare the reaching costs of AICO-T with those obtained with a ﬁxed duration approach, in this case AICO. [sent-164, score-0.66]

55 Note that although with a ﬁxed, long duration (e. [sent-165, score-0.314]

56 41) the control and error costs are reduced for short movements, these movements necessarily have up to 4× longer durations than those obtained with AICO-T. [sent-168, score-0.469]

57 2 application of AICO-T results in a optimised movement duration of 0. [sent-170, score-0.747]

58 2(a)), making the ﬁxed time approach impractical when temporal costs are considered. [sent-173, score-0.268]

59 Choosing a short duration on the other hand (AICO (T=0. [sent-174, score-0.314]

60 We further emphasis that the ﬁxed durations used in this comparison were chosen post hoc by exploiting the durations suggested by AICO-T in absence of this, there would have been no practical way of choosing them apart from experimentation. [sent-176, score-0.306]

61 Furthermore, we would like to highlight that, although the results suggests a simple scaling of duration with movement distance, in cluttered environments and plants with more complex forward kinematics, an efﬁcient decision on the movement duration cannot be based only on task space distance. [sent-177, score-1.609]

62 the standard reaching cost is the sum of control costs and cost on the endpoint error, without taking duration into account, i. [sent-182, score-1.127]

63 (a) End point task space trajectories for two different via points (circles) obtained for a ﬁxed start point (triangle). [sent-202, score-0.31]

64 (c) Movement durations and reaching costs (control + error costs) from 10 random start points. [sent-204, score-0.516]

65 The proportion of the movement duration spend before the via point is shown in light gray (mean in the AICO-T case). [sent-205, score-0.863]

66 This task has also seen some interest in the literature on modeling of human movement using the optimal control framework, e. [sent-208, score-0.69]

67 Here the common approach is to choose the time point at which one passes the via point such as to divide the movement duration in the same ratio as the distances between the start point, via point and end target. [sent-211, score-1.073]

68 This requires on the one hand prior knowledge of these movement distances and on the other, makes the implicit assumption that the two movements are in some sense independent. [sent-212, score-0.493]

69 In a ﬁrst experiment, we demonstrate the ability of our approach to solve such sequential problems, adjusting movement durations between sub goals in a principled manner, and show that it improves upon the standard modelling approach. [sent-213, score-0.774]

70 We again choose the movement duration for the standard case post hoc to coincide with the mean movement duration obtained with AICO-T for each of the individual via point tasks. [sent-217, score-1.628]

71 Each task is expressed using a cost function consisting of two point target cost terms. [sent-218, score-0.438]

72 Note that the cost function does not penalise velocity at the via point but encourages the stopping at the target. [sent-220, score-0.234]

73 Note that although for AICO-T this cost is incurred at the same discrete step, we allow θ before and after the via point to differ, but constrain them to be constant throughout each part of the movement, hence, allowing the cost to be incurred at an arbitrary point in real time. [sent-222, score-0.608]

74 3(a&b;), we show maximum a posteriori (MAP) trajectories in task space and joint space for controllers computed for the mean initial state. [sent-225, score-0.234]

75 Interestingly, although the end point trajectory for the near via point produced by AICO-T may look sub-optimal than that produced by the standard AICO algorithm, closer examination of the joint space trajectories reveal that our approach results in more efﬁcient actuation trajectories. [sent-226, score-0.368]

76 3(c), we illustrate the resulting average movement durations and costs of the mean trajectories. [sent-228, score-0.698]

77 late in the movement for the near and far via point, respectively. [sent-232, score-0.503]

78 This directly leads to a lower incurred cost compared to un-optimised movement durations. [sent-233, score-0.671]

79 3(a&b;) show MAP trajectories of controllers computed for the mean start state. [sent-235, score-0.165]

80 sequential (dashed) optimisation using AICO-T for a sequential (via point) task. [sent-256, score-0.146]

81 (a) Task space trajectories for a ﬁxed start point (triangle). [sent-257, score-0.166]

82 (c) The movement durations and reaching costs (control + error cost) for 10 random start points. [sent-260, score-0.949]

83 The mean proportion of the movement duration spend before the via point is shown in light gray. [sent-261, score-0.863]

84 In order to highlight the shortcomings of sequential time optimal control, next we compare planning a complete movement over sequential goals to planning a sequence of individual movements. [sent-262, score-0.882]

85 Speciﬁcally, using AICO-T, we compare planning the whole via point movement (joint planner) to planning a movement from the start to the via point followed by a second movement from the end point of the ﬁrst movement (n. [sent-263, score-2.107]

86 The joint planner used the same cost function as the previous experiment. [sent-266, score-0.275]

87 For the sequential planner, each of the two sub-trajectories had half the number of discrete time steps of the joint planner and the cost functions were given by appropriately splitting (19), i. [sent-267, score-0.388]

88 , ∗ ∗ ∗ ∗ V 1 (xk ) = δk= K (φ(xk )−yv )⊤Λv (φ(xk )−yv ) and V 2 (xk ) = δk= K (φ(xk )−ye )⊤Λe (φ(xk )−ye ) , 2 2 ∗ ∗ with Λv , Λe , yv , ye as for (19). [sent-269, score-0.158]

89 4(a&b;), we plot the MAP trajectories for the mean start state, in task as well as joint space. [sent-271, score-0.23]

90 The results illustrate that sequential planning leads to sub-optimal results as it does not take future goals into consideration. [sent-272, score-0.223]

91 One should however note that this effect would be less pronounced if the cost required stopping at the via point, as it is the velocity away from the end target which is the main problem for the sequential planner. [sent-277, score-0.299]

92 5 Conclusion The contribution of this paper is a novel method for jointly optimizing a movement trajectory and its time evolution (temporal scale and duration) in the stochastic optimal control framework. [sent-278, score-0.775]

93 As a special case, this solves the problem of an unknown goal horizon and the problem of trajectory optimization through via points when the timing of intermediate constraints is unknown and subject to optimization. [sent-279, score-0.186]

94 The method was derived in the form of an Expectation-Maximization algorithm where the E-step addresses the stochastic optimal control problem reformulated as an inference problem and the M-step re-adapts the time evolution of the trajectory. [sent-281, score-0.3]

95 We demonstrated the algorithm on a standard reaching task with and without via points. [sent-285, score-0.308]

96 In particular, in the via point case, it becomes obvious that ﬁxed horizon methods and sequenced ﬁrst exit time methods cannot ﬁnd equally efﬁcient motions as the proposed method. [sent-286, score-0.261]

97 A time-scaling method for near-timeoptimal control of an omni-directional robot along speciﬁed paths. [sent-294, score-0.202]

98 A linear theory for control of non-linear stochastic systems. [sent-307, score-0.199]

99 An iterative optimal control and estimation design for nonlinear stochastic system. [sent-314, score-0.227]

100 Optimal feedback control as a theory of motor coordination. [sent-337, score-0.196]

similar papers computed by tfidf model

tfidf for this paper:

wordName wordTfidf (topN-words)

[('xk', 0.475), ('movement', 0.433), ('duration', 0.314), ('aico', 0.279), ('reaching', 0.21), ('control', 0.171), ('cost', 0.148), ('costs', 0.136), ('durations', 0.129), ('temporal', 0.092), ('goals', 0.092), ('ak', 0.091), ('incurred', 0.09), ('wk', 0.088), ('canonical', 0.088), ('ye', 0.082), ('pos', 0.08), ('discreet', 0.08), ('ilqg', 0.08), ('trajectories', 0.079), ('yv', 0.076), ('planner', 0.075), ('trajectory', 0.075), ('sequential', 0.073), ('horizon', 0.071), ('rad', 0.07), ('exit', 0.064), ('discretization', 0.061), ('vel', 0.06), ('task', 0.058), ('uk', 0.058), ('planning', 0.058), ('controls', 0.054), ('posterior', 0.053), ('lqg', 0.053), ('plant', 0.053), ('joint', 0.052), ('tr', 0.051), ('edinburgh', 0.051), ('robotic', 0.049), ('emanuel', 0.048), ('point', 0.046), ('marc', 0.046), ('controllers', 0.045), ('jk', 0.042), ('start', 0.041), ('via', 0.04), ('robotics', 0.04), ('sethu', 0.04), ('time', 0.04), ('target', 0.038), ('commands', 0.035), ('ddp', 0.035), ('maximisation', 0.035), ('randomised', 0.035), ('ck', 0.034), ('quadratic', 0.034), ('behaviour', 0.034), ('angle', 0.033), ('dt', 0.033), ('vn', 0.033), ('movements', 0.033), ('ds', 0.033), ('inference', 0.033), ('bu', 0.032), ('marginalised', 0.032), ('toussaint', 0.032), ('state', 0.032), ('robot', 0.031), ('distance', 0.031), ('plants', 0.03), ('spend', 0.03), ('dynamical', 0.03), ('near', 0.03), ('rk', 0.03), ('targets', 0.029), ('prescribed', 0.029), ('icra', 0.029), ('stochastic', 0.028), ('optimal', 0.028), ('em', 0.028), ('formulation', 0.028), ('map', 0.028), ('highlight', 0.027), ('dependent', 0.027), ('distances', 0.027), ('approximations', 0.027), ('hoc', 0.026), ('stefan', 0.026), ('feedback', 0.025), ('sub', 0.025), ('passing', 0.025), ('automation', 0.024), ('aw', 0.024), ('hu', 0.024), ('ful', 0.024), ('applicable', 0.023), ('length', 0.023), ('adjusting', 0.022), ('post', 0.022)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 1.0000005 29 nips-2010-An Approximate Inference Approach to Temporal Optimization in Optimal Control

Author: Konrad Rawlik, Marc Toussaint, Sethu Vijayakumar

2 0.16851529 167 nips-2010-Mixture of time-warped trajectory models for movement decoding

Author: Elaine Corbett, Eric Perreault, Konrad Koerding

Abstract: Applications of Brain-Machine-Interfaces typically estimate user intent based on biological signals that are under voluntary control. For example, we might want to estimate how a patient with a paralyzed arm wants to move based on residual muscle activity. To solve such problems it is necessary to integrate obtained information over time. To do so, state of the art approaches typically use a probabilistic model of how the state, e.g. position and velocity of the arm, evolves over time – a so-called trajectory model. We wanted to further develop this approach using two intuitive insights: (1) At any given point of time there may be a small set of likely movement targets, potentially identified by the location of objects in the workspace or by gaze information from the user. (2) The user may want to produce movements at varying speeds. We thus use a generative model with a trajectory model incorporating these insights. Approximate inference on that generative model is implemented using a mixture of extended Kalman filters. We find that the resulting algorithm allows us to decode arm movements dramatically better than when we use a trajectory model with linear dynamics. 1 In trod u cti on When patients have lost a limb or the ability to communicate with the outside world, brain machine interfaces (BMIs) are often used to enable robotic prostheses or restore communication. To achieve this, the user's intended state of the device must be decoded from biological signals. In the context of Bayesian statistics, two aspects are important for the design of an estimator of a temporally evolving state: the observation model, which describes how measured variables relate to the system’s state and the trajectory model which describes how the state changes over time in a probabilistic manner. Following this logic many recent BMI applications have relied on Bayesian estimation for a wide range of problems including the decoding of intended human [1] and animal [2] movements. In the context of BMIs, Bayesian approaches offer a principled way of formalizing the uncertainty about signals and thus often result in improvements over other signal processing techniques [1]-[3]. Most work on state estimation in dynamical systems has assumed linear dynamics and Gaussian noise. Under these circumstances, efficient algorithms result from belief propagation. The most frequent application uses the Kalman filter (KF), which recursively combines noisy state observations with the probabilistic evolution of state defined by the trajectory model to estimate the marginal distribution over states [4]. Such approaches have been used widely for applications including upper [1] and lower [5] extremity prosthetic 1 devices, functional electric stimulation [6] and human computer interactions [7]. As these algorithms are so commonly used, it seems promising to develop extensions to nonlinear trajectory models that may better describe the probabilistic distribution of movements in everyday life. One salient departure from the standard assumptions is that people tend to produce both slow and fast movements, depending on the situation. Models with linear dynamics only allow such deviation through the noise term, which makes these models poor at describing the natural variation of movement speeds during real world tasks. Explicitly incorporating movement speed into the trajectory model should lead to better movement estimates. Knowledge of the target position should also strongly affect trajectory models. After all , we tend to accelerate our arm early during movement and slow down later on. Target information can be linearly incorporated into the trajectory model, and this has greatly improved predictions [8]-[12]. Alternatively, if there are a small number of potential targets then a mixture of trajectory models approach [13] can be used. Here we are interested in the case where available data provide a prior over potential t argets but where movement targets may be anywhere. We want to incorporate target uncertainty and allow generalization to novel targets. Prior information about potential targets could come from a number of sources but would generally be noisy. For example, activity in the dorsal premotor cortex provides information about intended target location prior to movement and may be used where such recordings are available [14]. Target information may also be found noninvasively by tracking eye movements. However, such data will generally provide non-zero priors for a number of possible target locations as the subject saccades over the scene. While subjects almost always look at a target before reaching for it [15], there may be a delay of up to a second between looking at the target and the reach – a time interval over which up to 3 saccades are typically made. Each of these fixations could be the target. Hence, a probabilistic distribution of targets is appropriate when using either neural recordings or eye tracking to estimate potential reach targets Here we present an algorithm that uses a mixture of extended Kalman Filters (EKFs) to combine our insights related to the variation of movement speed and the availability of probabilistic target knowledge. Each of the mixture component s allows the speed of the movement to vary continuously over time. We tested how well we could use EMGs and eye movements to decode hand position of humans performing a three -dimensional large workspace reaching task. We find that using a trajectory model that allows for probabilistic target information and variation of speed leads to dramatic improvements in decoding quality. 2 Gen e ral Decod i n g S etti n g We wanted to test how well different decoding algorithms can decode human movement, over a wide range of dynamics. While many recent studies have looked at more restrictive, two-dimensional movements, a system to restore arm function should produce a wide range of 3D trajectories. We recorded arm kinematics and EMGs of healthy subjects during unconstrained 3D reaches to targets over a large workspace. Two healthy subjects were asked to reach at slow, normal and fast speeds, as they would in everyday life. Subjects were seated as they reached towards 16 LEDs in blocks of 150s, which were located on two planes positioned such that all targets were just reachable (Fig 1A). The target LED was lit for one second prior to an auditory go cue, at which time the subject would reach to the target at the appropriate speed. Slow, normal and fast reaches were allotted 3 s, 1.5s and 1s respectively; however, subjects determined the speed. An approximate total of 450 reaches were performed per subject. The subjects provided informed consent, and the protocol was approved by the Northwestern University Institutional Review Board. EMG signals were measured from the pectoralis major, and the three deltoid muscles of the shoulder. This represents a small subset of the muscles involved in reaching, and approximates those muscles retaining some voluntary control following mid-level cervical spinal cord injuries. 2 The EMG signals were band-pass filtered between 10 and 1,000 Hz, and subsequently anti aliased filtered. Hand, wrist, shoulder and head positions were tracked using an Optotrak motion analysis system. We simultaneously recorded eye movements with an ASL EYETRAC-6 head mounted eye tracker. Approximately 25% of the reaches were assigned to the test set, and the rest were used for training. Reaches for which either the motion capture data was incomplete, or there was visible motion artifact on the EMG were removed. As the state we used hand positions and joint angles (3 shoulder, 2 elbow, position, velocity and acceleration, 24 dimensions). Joint angles were calculated from the shoulder and wrist marker data using digitized bony landmarks which defined a coordinate system for the upper limb as detailed by Wu et al. [16]. As the motion data were sampled at 60Hz, the mean absolute value o f the EMG in the corresponding 16.7ms windows was used as an observation of the state at each time-step. Algorithm accuracy was quantified by normalizing the root -mean-squared error by the straight line distance between the first and final position of the endpoint for each reach. We compared the algorithms statistically using repeated measures ANOVAs with Tukey post -hoc tests, treating reach and subject as random effects. In the rest of the paper we will ask how well these reaching movements can be decoded from EMG and eye-tracking data. Figure 1: A Experimental setup and B sample kinematics and processed EMGs for one reach 3 Kal man Fi l ters w i th Target i n f ormati on All models that we consider in this paper assume linear observations with Gaussian noise: (1) where x is the state, y is the observation and v is the measurement noise with p(v) ~ N(0,R), and R is the observation covariance matrix. The model fitted the measured EMGs with an average r2 of 0.55. This highlights the need to integrate information over time. The standard approach also assumes linear dynamics and Gaussian process noise: (2) where, x t represents the hand and joint angle positions, w is the process noise with p(w) ~ N(0,Q), and Q is the state covariance matrix. The Kalman filter does optimal inference for this generative model. This model can effectively capture the dynamics of stereotypical reaches to a single target by appropriately tuning its parameters. However, when used to describe reaches to multiple targets, the model cannot describe target dependent aspects of reaching but boils down to a random drift model. Fast velocities are underestimated as they are unlikely under the trajectory model and there is excessive drift close to the target (Fig. 2A). 3 In many decoding applications we may know the subject’s target. A range of recent studies have addressed the issue of incorporating this information into the trajectory model [8, 13], and we might assume the effect of the target on the dynamics to be linear. This naturally suggests adding the target to the state space, which works well in practice [9, 12]. By appending the target to the state vector (KFT), the simple linear format of the KF may be retained: (3) where xTt is the vector of target positions, with dimensionality less than or equal to that of xt. This trajectory model thus allows describing both the rapid acceleration that characterizes the beginning of a reach and the stabilization towards its end. We compared the accuracy of the KF and the KFT to the Single Target Model (STM), a KF trained only on reaches to the target being tested (Fig. 2). The STM represents the best possible prediction that could be obtained with a Kalman filter. Assuming the target is perfectly known, we implemented the KFT by correctly initializing the target state xT at the beginning of the reach. We will relax this assumption below. The initial hand and joint angle positions were also assumed to be known. Figure 2: A Sample reach and predictions and B average accuracies with standard errors for KFT, KF and MTM. Consistent with the recent literature, both methods that incorporated target information produced higher prediction accuracy than the standard KF (both p<0.0001). Interestingly, there was no significant difference between the KFT and the STM (p=0.9). It seems that when we have knowledge of the target, we do not lose much by training a single model over the whole workspace rather than modeling the targets individually. This is encouraging, as we desire a BMI system that can generalize to any target within the workspace, not just specifically to those that are available in the training data. Clearly, adding the target to the state space allows the dynamics of typical movements to be modeled effectively, resulting in dramatic increases in decoding performance. 4 Ti me Warp i n g 4.1 I m p l e m e n t i n g a t i m e - w a r p e d t r a j e c t o r y mo d e l While the KFT above can capture the general reach trajectory profile, it does not allow for natural variability in the speed of movements. Depending on our task objectives, which would not directly be observed by a BMI, we might lazily reach toward a target or move a t maximal speed. We aim to change the trajectory model to explicitly incorporate a warping factor by which the average movement speed is scaled, allowing for such variability. As the movement speed will be positive in all practical cases, we model the logarithm of this factor, 4 and append it to the state vector: (4) We create a time-warped trajectory model by noting that if the average rate of a trajectory is to be scaled by a factor S, the position at time t will equal that of the original trajectory at time St. Differentiating, the velocity will be multiplied by S, and the acceleration by S 2. For simplicity, the trajectory noise is assumed to be additive and Gaussian, and the model is assumed to be stationary: (5) where Ip is the p-dimensional identity matrix and is a p p matrix of zeros. Only the terms used to predict the acceleration states need to be estimated to build the state transition matrix, and they are scaled as a nonlinear function of xs. After adding the variable movement speed to the state space the system is no longer linear. Therefore we need a different solution strategy. Instead of the typical KFT we use the Extended Kalman Filter (EKFT) to implement a nonlinear trajectory model by linearizing the dynamics around the best estimate at each time-step [17]. With this approach we add only small computational overhead to the KFT recursions. 4.2 Tr a i n i n g t h e t i m e w a r p i n g mo d e l The filter parameters were trained using a variant of the Expectation Maximization (EM) algorithm [18]. For extended Kalman filter learning the initialization for the variables may matter. S was initialized with the ground truth average reach speeds for each movement relative to the average speed across all movements. The state transition parameters were estimated using nonlinear least squares regression, while C, Q and R were estimated linearly for the new system, using the maximum likelihood solution [18] (M-step). For the E-step we used a standard extended Kalman smoother. We thus found the expected values for t he states given the current filter parameters. For this computation, and later when testing the algorithm, xs was initialized to its average value across all reaches while the remaining states were initialized to their true values. The smoothed estimate fo r xs was then used, along with the true values for the other states, to re-estimate the filter parameters in the M-step as before. We alternated between the E and M steps until the log likelihood converged (which it did in all cases). Following the training procedure, the diagonal of the state covariance matrix Q corresponding to xs was set to the variance of the smoothed xs over all reaches, according to how much this state should be allowed to change during prediction. This allowed the estimate of xs to develop over the course of the reach due to the evidence provided by the observations, better capturing the dynamics of reaches at different speeds. 4.3 P e r f o r ma n c e o f t h e t i m e - w a r p e d E K F T Incorporating time warping explicitly into the trajectory model pro duced a noticeable increase in decoding performance over the KFT. As the speed state xs is estimated throughout the course of the reach, based on the evidence provided by the observations, the trajectory model has the flexibility to follow the dynamics of the reach more accurately (Fig. 3). While at the normal self-selected speed the difference between the algorithms is small, for the slow and fast speeds, where the dynamics deviate from average, there i s a clear advantage to the time warping model. 5 Figure 3: Hand positions and predictions of the KFT and EKFT for sample reaches at A slow, B normal and C fast speeds. Note the different time scales between reaches. The models were first trained using data from all speeds (Fig. 4A). The EKFT was 1.8% more accurate on average (p<0.01), and the effect was significant at the slow (1.9%, p<0.05) and the fast (2.8%, p<0.01), but not at the normal (p=0.3) speed. We also trained the models from data using only reaches at the self-selected normal speed, as we wanted to see if there was enough variation to effectively train the EKFT (Fig. 4B). Interestingly, the performance of the EKFT was reduced by only 0.6%, and the KFT by 1.1%. The difference in performance between the EKFT and KFT was even more pronounced on aver age (2.3%, p<0.001), and for the slow and fast speeds (3.6 and 4.1%, both p< 0.0001). At the normal speed, the algorithms again were not statistically different (p=0.6). This result demonstrates that the EKFT is a practical option for a real BMI system, as it is not necessary to greatly vary the speeds while collecting training data for the model to be effective over a wide range of intended speeds. Explicitly incorporating speed information into the trajectory model helps decoding, by modeling the natural variation in volitional speed. Figure 4: Mean and standard error of EKFT and KFT accuracy at the different subjectselected speeds. Models were trained on reaches at A all speeds and B just normal speed reaches. Asterisks indicate statistically significant differences between the algorithms. 5 Mi xtu res of Target s So far, we have assumed that the targets of our reaches are perfectly known. In a real-world system, there will be uncertainty about the intended target of the reach. However, in typical applications there are a small number of possible objectives. Here we address this situation. Drawing on the recent literature, we use a mixture model to consider each of the possible targets [11, 13]. We condition the posterior probability for the state on the N possible targets, T: (6) 6 Using Bayes' Rule, this equation becomes: (7) As we are dealing with a mixture model, we perform the Kalman filter recursion for each possible target, xT, and our solution is a weighted sum of the outputs. The weights are proportional to the prior for that target, , and the likelihood of the model given that target . is independent of the target and does not need to be calculated. We tested mixtures of both algorithms, the mKFT and mEKFT, with real uncert ain priors obtained from eye-tracking in the one-second period preceding movement. As the targets were situated on two planes, the three-dimensional location of the eye gaze was found by projecting its direction onto those planes. The first, middle and last eye samples were selected, and all other samples were assigned to a group according to which of the three was closest. The mean and variance of these three groups were used to initialize three Kalman filters in the mixture model. The priors of the three groups were assigned proportional to the number of samples in them. If the subject looks at multiple positions prior to reaching, this method ensures with a high probability that the correct target was accounted for in one of the filters in the mixture. We also compared the MTM approach of Yu et al. [13], where a different KF model was generated for each target, and a mixture is performed over these models. This approach explicitly captures the dynamics of stereotypical reaches to specific targets. Given perfect target information, it would reduce to the STM described above. Priors for the MTM were found by assigning each valid eye sample to its closest two targets, and weighting the models proportional to the number of samples assigned to the corresponding target, divided by its distance from the mean of those samples. We tried other ways of assigning priors and the one presented gave the best results. We calculated the reduction in decoding quality when instead of perfect priors we provide eye-movement based noisy priors (Fig. 5). The accuracies of the mEKFT, the mKFT and the MTM were only degraded by 0.8, 1.9 and 2.1% respectively, compared to the perfect prior situation. The mEKFT was still close to 10% better than the KF. The mixture model framework is effective in accounting for uncertain priors. Figure 5: Mean and standard errors of accuracy for algorithms with perfect priors, and uncertain priors with full and partial training set. The asterisk indicates a statistically significant effects between the two training types, where real priors are used. Here, only reaches at normal speed were used to train the models, as this is a more realistic training set for a BMI application. This accounts for the degraded performance of the MTM with perfect priors relative to the STM from above (Fig. 2). With even more stereotyped training data for each target, the MTM doesn't generalize as well to new speeds. 7 We also wanted to know if the algorithms could generalize to new targets. In a real application, the available training data will generally not span the entire useable worksp ace. We compared the algorithms where reaches to all targets except the one being tested had been used to train the models. The performance of the MTM was significantly de graded unsurprisingly, as it was designed for reaches to a set of known targets. Performance of the mKFT and mEKFT degraded by about 1%, but not significantly (both p>0.7), demonstrating that the continuous approach to target information is preferable when the target could be anywhere in space, not just at locations for which training data is available. 6 Di scu ssi on and concl u si on s The goal of this work was to design a trajectory model that would improve decoding for BMIs with an application to reaching. We incorporated two features that prominently influence the dynamics of natural reach: the movement speed and the target location. Our approach is appropriate where uncertain target information is available. The model generalizes well to new regions of the workspace for which there is no training data, and across a broad range of reaching dynamics to widely spaced targets in three dimensions. The advantages over linear models in decoding precision we report here could be equally obtained using mixtures over many targets and speeds. While mixture models [11, 13] could allow for slow versus fast movements and any number of potential targets, this strategy will generally require many mixture components. Such an approach would require a lot more training data, as we have shown that it does not generalize well. It would also be run-time intensive which is problematic for prosthetic devices that rely on low power controllers. In contrast, the algorithm introduced here only takes a small amount of additional run-time in comparison to the standard KF approach. The EKF is only marginally slower than the standard KF and the algorithm will not generally need to consider more than 3 mixture components assuming the subject fixates the target within the second pre ceding the reach. In this paper we assumed that subjects always would fixate a reach target – along with other non-targets. While this is close to the way humans usually coordinate eyes and reaches [15], there might be cases where people do not fixate a reach target. Our approach could be easily extended to deal with such situations by adding a dummy mixture component that all ows the description of movements to any target. As an alternative to mixture approaches, a system can explicitly estimate the target position in the state vector [9]. This approach, however, would not straightforwardly allow for the rich target information available; we look at the target but also at other locations, strongly suggesting mixture distributions. A combination of the two approaches could further improve decoding quality. We could both estimate speed and target position for the EKFT in a continuous manner while retaining the mixture over target priors. We believe that the issues that we have addressed here are almost universal. Virtually all types of movements are executed at varying speed. A probabilistic distribution for a small number of action candidates may also be expected in most BMI applications – after all there are usually only a small number of actions that make sense in a given environment. While this work is presented in the context of decoding human reaching, it may be applied to a wide range of BMI applications including lower limb prosthetic devices and human computer interactions, as well as different signal sources such as electrode grid recordings and electroencephalograms. The increased user control in conveying their intended movements would significantly improve the functionality of a neuroprosthetic device. A c k n o w l e d g e me n t s T h e a u t h o r s t h a n k T. H a s w e l l , E . K r e p k o v i c h , a n d V. Ravichandran for assistance with experiments. This work was funded by the NSF Program in Cyber-Physical Systems. R e f e re n c e s [1] L.R. Hochberg, M.D. Serruya, G.M. Friehs, J.A. Mukand, M. Saleh, A.H. Caplan, A. Branner, D. 8 [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] Chen, R.D. Penn, and J.P. Donoghue, “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature, vol. 442, 2006, pp. 164–171. W. Wu, Y. Gao, E. Bienenstock, J.P. Donoghue, and M.J. Black, “Bayesian population decoding of motor cortical activity using a Kalman filter,” Neural Computation, vol. 18, 2006, pp. 80–118. W. Wu, M.J. Black, Y. Gao, E. Bienenstock, M. Serruya, A. Shaikhouni, and J.P. Donoghue, “Neural decoding of cursor motion using a Kalman filter,” Advances in Neural Information Processing Systems 15: Proceedings of the 2002 Conference, 2003, p. 133. R.E. Kalman, “A new approach to linear filtering and prediction problems,” Journal of basic Engineering, vol. 82, 1960, pp. 35–45. G.G. Scandaroli, G.A. Borges, J.Y. Ishihara, M.H. Terra, A.F.D. Rocha, and F.A.D.O. Nascimento, “Estimation of foot orientation with respect to ground for an above knee robotic prosthesis,” Proceedings of the 2009 IEEE/RSJ international conference on Intelligent robots and systems, St. Louis, MO, USA: IEEE Press, 2009, pp. 1112-1117. I. Cikajlo, Z. Matjačić, and T. Bajd, “Efficient FES triggering applying Kalman filter during sensory supported treadmill walking,” Journal of Medical Engineering & Technology, vol. 32, 2008, pp. 133144. S. Kim, J.D. Simeral, L.R. Hochberg, J.P. Donoghue, and M.J. Black, “Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia,” Journal of Neural Engineering, vol. 5, 2008, pp. 455-476. L. Srinivasan, U.T. Eden, A.S. Willsky, and E.N. Brown, “A state-space analysis for reconstruction of goal-directed movements using neural signals,” Neural computation, vol. 18, 2006, pp. 2465–2494. G.H. Mulliken, S. Musallam, and R.A. Andersen, “Decoding trajectories from posterior parietal cortex ensembles,” Journal of Neuroscience, vol. 28, 2008, p. 12913. W. Wu, J.E. Kulkarni, N.G. Hatsopoulos, and L. Paninski, “Neural Decoding of Hand Motion Using a Linear State-Space Model With Hidden States,” IEEE Transactions on neural systems and rehabilitation engineering, vol. 17, 2009, p. 1. J.E. Kulkarni and L. Paninski, “State-space decoding of goal-directed movements,” IEEE Signal Processing Magazine, vol. 25, 2008, p. 78. C. Kemere and T. Meng, “Optimal estimation of feed-forward-controlled linear systems,” IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005. Proceedings.(ICASSP'05), 2005. B.M. Yu, C. Kemere, G. Santhanam, A. Afshar, S.I. Ryu, T.H. Meng, M. Sahani, and K.V. Shenoy, “Mixture of trajectory models for neural decoding of goal-directed movements,” Journal of neurophysiology, vol. 97, 2007, p. 3763. N. Hatsopoulos, J. Joshi, and J.G. O'Leary, “Decoding continuous and discrete motor behaviors using motor and premotor cortical ensembles,” Journal of neurophysiology, vol. 92, 2004, p. 1165. R.S. Johansson, G. Westling, A. Backstrom, and J.R. Flanagan, “Eye-hand coordination in object manipulation,” Journal of Neuroscience, vol. 21, 2001, p. 6917. G. Wu, F.C. van der Helm, H.E.J. Veeger, M. Makhsous, P. Van Roy, C. Anglin, J. Nagels, A.R. Karduna, and K. McQuade, “ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion–Part II: shoulder, elbow, wrist and hand,” Journal of biomechanics, vol. 38, 2005, pp. 981–992. D. Simon, Optimal state estimation: Kalman, H [infinity] and nonlinear approaches, John Wiley and Sons, 2006. Z. Ghahramani and G.E. Hinton, “Parameter estimation for linear dynamical systems,” University of Toronto technical report CRG-TR-96-2, vol. 6, 1996. 9

3 0.15017895 171 nips-2010-Movement extraction by detecting dynamics switches and repetitions

Author: Silvia Chiappa, Jan R. Peters

Abstract: Many time-series such as human movement data consist of a sequence of basic actions, e.g., forehands and backhands in tennis. Automatically extracting and characterizing such actions is an important problem for a variety of different applications. In this paper, we present a probabilistic segmentation approach in which an observed time-series is modeled as a concatenation of segments corresponding to different basic actions. Each segment is generated through a noisy transformation of one of a few hidden trajectories representing different types of movement, with possible time re-scaling. We analyze three different approximation methods for dealing with model intractability, and demonstrate how the proposed approach can successfully segment table tennis movements recorded using a robot arm as haptic input device. 1

4 0.11348338 248 nips-2010-Sparse Inverse Covariance Selection via Alternating Linearization Methods

Author: Katya Scheinberg, Shiqian Ma, Donald Goldfarb

Abstract: Gaussian graphical models are of great interest in statistical learning. Because the conditional independencies between different nodes correspond to zero entries in the inverse covariance matrix of the Gaussian distribution, one can learn the structure of the graph by estimating a sparse inverse covariance matrix from sample data, by solving a convex maximum likelihood problem with an ℓ1 -regularization term. In this paper, we propose a ﬁrst-order method based on an alternating linearization technique that exploits the problem’s special structure; in particular, the subproblems solved in each iteration have closed-form solutions. Moreover, our algorithm obtains an ϵ-optimal solution in O(1/ϵ) iterations. Numerical experiments on both synthetic and real data from gene association networks show that a practical version of this algorithm outperforms other competitive algorithms. 1

5 0.098670855 232 nips-2010-Sample Complexity of Testing the Manifold Hypothesis

Author: Hariharan Narayanan, Sanjoy Mitter

Abstract: The hypothesis that high dimensional data tends to lie in the vicinity of a low dimensional manifold is the basis of a collection of methodologies termed Manifold Learning. In this paper, we study statistical aspects of the question of ﬁtting a manifold with a nearly optimal least squared error. Given upper bounds on the dimension, volume, and curvature, we show that Empirical Risk Minimization can produce a nearly optimal manifold using a number of random samples that is independent of the ambient dimension of the space in which data lie. We obtain an upper bound on the required number of samples that depends polynomially on the curvature, exponentially on the intrinsic dimension, and linearly on the intrinsic volume. For constant error, we prove a matching minimax lower bound on the sample complexity that shows that this dependence on intrinsic dimension, volume log 1 and curvature is unavoidable. Whether the known lower bound of O( k + 2 δ ) 2 for the sample complexity of Empirical Risk minimization on k−means applied to data in a unit ball of arbitrary dimension is tight, has been an open question since 1997 [3]. Here is the desired bound on the error and δ is a bound on the probability of failure. We improve the best currently known upper bound [14] of 2 log 1 log4 k log 1 O( k2 + 2 δ ) to O k min k, 2 + 2 δ . Based on these results, we 2 devise a simple algorithm for k−means and another that uses a family of convex programs to ﬁt a piecewise linear curve of a speciﬁed length to high dimensional data, where the sample complexity is independent of the ambient dimension. 1

6 0.096350849 57 nips-2010-Decoding Ipsilateral Finger Movements from ECoG Signals in Humans

7 0.094590589 163 nips-2010-Lower Bounds on Rate of Convergence of Cutting Plane Methods

8 0.070817277 282 nips-2010-Variable margin losses for classifier design

9 0.062820308 251 nips-2010-Sphere Embedding: An Application to Part-of-Speech Induction

10 0.062519975 156 nips-2010-Learning to combine foveal glimpses with a third-order Boltzmann machine

11 0.061476812 210 nips-2010-Practical Large-Scale Optimization for Max-norm Regularization

12 0.061126258 208 nips-2010-Policy gradients in linearly-solvable MDPs

13 0.058890868 268 nips-2010-The Neural Costs of Optimal Control

14 0.055433154 27 nips-2010-Agnostic Active Learning Without Constraints

15 0.053347465 203 nips-2010-Parametric Bandits: The Generalized Linear Case

16 0.053011961 185 nips-2010-Nonparametric Density Estimation for Stochastic Optimization with an Observable State Variable

17 0.051879913 70 nips-2010-Efficient Optimization for Discriminative Latent Class Models

18 0.050308675 50 nips-2010-Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories

19 0.049660683 33 nips-2010-Approximate inference in continuous time Gaussian-Jump processes

20 0.0487547 226 nips-2010-Repeated Games against Budgeted Adversaries

similar papers computed by lsi model

lsi for this paper:

topicId topicWeight

[(0, 0.154), (1, -0.028), (2, -0.013), (3, 0.046), (4, -0.003), (5, 0.015), (6, 0.008), (7, -0.028), (8, -0.002), (9, 0.012), (10, 0.004), (11, -0.061), (12, 0.108), (13, -0.021), (14, 0.023), (15, 0.04), (16, -0.03), (17, 0.078), (18, -0.004), (19, 0.016), (20, -0.043), (21, 0.061), (22, 0.07), (23, 0.072), (24, 0.055), (25, -0.123), (26, -0.171), (27, 0.047), (28, 0.084), (29, -0.2), (30, 0.091), (31, 0.095), (32, 0.083), (33, -0.146), (34, -0.097), (35, 0.179), (36, 0.008), (37, -0.101), (38, 0.017), (39, 0.062), (40, 0.11), (41, 0.066), (42, 0.026), (43, 0.006), (44, 0.033), (45, -0.086), (46, 0.211), (47, -0.091), (48, -0.02), (49, 0.07)]

similar papers list:

simIndex simValue paperId paperTitle

same-paper 1 0.96535319 29 nips-2010-An Approximate Inference Approach to Temporal Optimization in Optimal Control

Author: Konrad Rawlik, Marc Toussaint, Sethu Vijayakumar

2 0.78068107 167 nips-2010-Mixture of time-warped trajectory models for movement decoding

Author: Elaine Corbett, Eric Perreault, Konrad Koerding

3 0.68730158 57 nips-2010-Decoding Ipsilateral Finger Movements from ECoG Signals in Humans

Author: Yuzong Liu, Mohit Sharma, Charles Gaona, Jonathan Breshears, Jarod Roland, Zachary Freudenburg, Eric Leuthardt, Kilian Q. Weinberger

Abstract: Several motor related Brain Computer Interfaces (BCIs) have been developed over the years that use activity decoded from the contralateral hemisphere to operate devices. Contralateral primary motor cortex is also the region most severely affected by hemispheric stroke. Recent studies have identiﬁed ipsilateral cortical activity in planning of motor movements and its potential implications for a stroke relevant BCI. The most fundamental functional loss after a hemispheric stroke is the loss of ﬁne motor control of the hand. Thus, whether ipsilateral cortex encodes ﬁnger movements is critical to the potential feasibility of BCI approaches in the future. This study uses ipsilateral cortical signals from humans (using ECoG) to decode ﬁnger movements. We demonstrate, for the ﬁrst time, successful ﬁnger movement detection using machine learning algorithms. Our results show high decoding accuracies in all cases which are always above chance. We also show that signiﬁcant accuracies can be achieved with the use of only a fraction of all the features recorded and that these core features are consistent with previous physiological ﬁndings. The results of this study have substantial implications for advancing neuroprosthetic approaches to stroke populations not currently amenable to existing BCI techniques. 1

4 0.67075253 171 nips-2010-Movement extraction by detecting dynamics switches and repetitions

Author: Silvia Chiappa, Jan R. Peters

5 0.57400024 248 nips-2010-Sparse Inverse Covariance Selection via Alternating Linearization Methods

Author: Katya Scheinberg, Shiqian Ma, Donald Goldfarb

6 0.47147071 50 nips-2010-Constructing Skill Trees for Reinforcement Learning Agents from Demonstration Trajectories

7 0.4541764 163 nips-2010-Lower Bounds on Rate of Convergence of Cutting Plane Methods

8 0.39978564 255 nips-2010-Static Analysis of Binary Executables Using Structural SVMs

9 0.33785862 19 nips-2010-A rational decision making framework for inhibitory control

10 0.32414952 2 nips-2010-A Bayesian Approach to Concept Drift

11 0.3235358 156 nips-2010-Learning to combine foveal glimpses with a third-order Boltzmann machine

12 0.3221162 90 nips-2010-Fast Large-scale Mixture Modeling with Component-specific Data Partitions

13 0.31939539 257 nips-2010-Structured Determinantal Point Processes

14 0.31354111 58 nips-2010-Decomposing Isotonic Regression for Efficiently Solving Large Problems

15 0.30891931 34 nips-2010-Attractor Dynamics with Synaptic Depression

16 0.30274582 244 nips-2010-Sodium entry efficiency during action potentials: A novel single-parameter family of Hodgkin-Huxley models

17 0.29911938 210 nips-2010-Practical Large-Scale Optimization for Max-norm Regularization

18 0.29852843 202 nips-2010-Parallelized Stochastic Gradient Descent

19 0.28630179 11 nips-2010-A POMDP Extension with Belief-dependent Rewards

20 0.28255865 121 nips-2010-Improving Human Judgments by Decontaminating Sequential Dependencies

similar papers computed by lda model

lda for this paper:

topicId topicWeight

[(4, 0.153), (13, 0.048), (17, 0.02), (27, 0.068), (30, 0.104), (45, 0.28), (50, 0.044), (52, 0.032), (60, 0.041), (77, 0.065), (78, 0.012), (90, 0.03)]

similar papers list:

simIndex simValue paperId paperTitle

1 0.93387163 158 nips-2010-Learning via Gaussian Herding

Author: Koby Crammer, Daniel D. Lee

Abstract: We introduce a new family of online learning algorithms based upon constraining the velocity ﬂow over a distribution of weight vectors. In particular, we show how to effectively herd a Gaussian weight vector distribution by trading off velocity constraints with a loss function. By uniformly bounding this loss function, we demonstrate how to solve the resulting optimization analytically. We compare the resulting algorithms on a variety of real world datasets, and demonstrate how these algorithms achieve state-of-the-art robust performance, especially with high label noise in the training data. 1

same-paper 2 0.91838342 29 nips-2010-An Approximate Inference Approach to Temporal Optimization in Optimal Control

Author: Konrad Rawlik, Marc Toussaint, Sethu Vijayakumar

3 0.89617658 155 nips-2010-Learning the context of a category

Author: Dan Navarro

Abstract: This paper outlines a hierarchical Bayesian model for human category learning that learns both the organization of objects into categories, and the context in which this knowledge should be applied. The model is ﬁt to multiple data sets, and provides a parsimonious method for describing how humans learn context speciﬁc conceptual representations.

4 0.89233571 63 nips-2010-Distributed Dual Averaging In Networks

Author: Alekh Agarwal, Martin J. Wainwright, John C. Duchi

Abstract: The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. We develop and analyze distributed algorithms based on dual averaging of subgradients, and provide sharp bounds on their convergence rates as a function of the network size and topology. Our analysis clearly separates the convergence of the optimization algorithm itself from the effects of communication constraints arising from the network structure. We show that the number of iterations required by our algorithm scales inversely in the spectral gap of the network. The sharpness of this prediction is conﬁrmed both by theoretical lower bounds and simulations for various networks. 1

5 0.8897897 88 nips-2010-Extensions of Generalized Binary Search to Group Identification and Exponential Costs

Author: Gowtham Bellala, Suresh Bhavnani, Clayton Scott

Abstract: Generalized Binary Search (GBS) is a well known greedy algorithm for identifying an unknown object while minimizing the number of “yes” or “no” questions posed about that object, and arises in problems such as active learning and active diagnosis. Here, we provide a coding-theoretic interpretation for GBS and show that GBS can be viewed as a top-down algorithm that greedily minimizes the expected number of queries required to identify an object. This interpretation is then used to extend GBS in two ways. First, we consider the case where the objects are partitioned into groups, and the objective is to identify only the group to which the object belongs. Then, we consider the case where the cost of identifying an object grows exponentially in the number of queries. In each case, we present an exact formula for the objective function involving Shannon or R´ nyi entropy, and e develop a greedy algorithm for minimizing it. 1

6 0.88966304 243 nips-2010-Smoothness, Low Noise and Fast Rates

7 0.88720345 27 nips-2010-Agnostic Active Learning Without Constraints

8 0.88651037 290 nips-2010-t-logistic regression

9 0.88599938 148 nips-2010-Learning Networks of Stochastic Differential Equations

10 0.88552749 87 nips-2010-Extended Bayesian Information Criteria for Gaussian Graphical Models

11 0.88493627 49 nips-2010-Computing Marginal Distributions over Continuous Markov Networks for Statistical Relational Learning

12 0.88445908 1 nips-2010-(RF)^2 -- Random Forest Random Field

13 0.88423932 280 nips-2010-Unsupervised Kernel Dimension Reduction

14 0.88346159 201 nips-2010-PAC-Bayesian Model Selection for Reinforcement Learning

15 0.88345909 202 nips-2010-Parallelized Stochastic Gradient Descent

16 0.88343632 287 nips-2010-Worst-Case Linear Discriminant Analysis

17 0.88340992 92 nips-2010-Fast global convergence rates of gradient methods for high-dimensional statistical recovery

18 0.88281542 134 nips-2010-LSTD with Random Projections

19 0.88274485 282 nips-2010-Variable margin losses for classifier design

20 0.88271946 85 nips-2010-Exact learning curves for Gaussian process regression on large random graphs