nips nips2002 nips2002-9 knowledge-graph by maker-knowledge-mining
Source: pdf
Author: Emanuel Todorov, Michael I. Jordan
Abstract: Behavioral goals are achieved reliably and repeatedly with movements rarely reproducible in their detail. Here we offer an explanation: we show that not only are variability and goal achievement compatible, but indeed that allowing variability in redundant dimensions is the optimal control strategy in the face of uncertainty. The optimal feedback control laws for typical motor tasks obey a “minimal intervention” principle: deviations from the average trajectory are only corrected when they interfere with the task goals. The resulting behavior exhibits task-constrained variability, as well as synergetic coupling among actuators—which is another unexplained empirical phenomenon.
Reference: text
sentIndex sentText sentNum sentScore
1 edu Abstract Behavioral goals are achieved reliably and repeatedly with movements rarely reproducible in their detail. [sent-6, score-0.048]
2 Here we offer an explanation: we show that not only are variability and goal achievement compatible, but indeed that allowing variability in redundant dimensions is the optimal control strategy in the face of uncertainty. [sent-7, score-1.004]
3 The optimal feedback control laws for typical motor tasks obey a “minimal intervention” principle: deviations from the average trajectory are only corrected when they interfere with the task goals. [sent-8, score-1.552]
4 The resulting behavior exhibits task-constrained variability, as well as synergetic coupling among actuators—which is another unexplained empirical phenomenon. [sent-9, score-0.129]
5 1 Introduction Both the difficulty and the fascination of the motor coordination problem lie in the apparent conflict between two fundamental properties of the motor system: the ability to accomplish its goal reliably and repeatedly, and the fact that it does so with variable movements [1]. [sent-10, score-0.989]
6 More precisely, trial-to-trial fluctuations in individual degrees of freedom are on average larger than fluctuations in task-relevant movement parameters—motor variability is constrained to a redundant or “uncontrolled” manifold [16] rather than being suppressed altogether. [sent-11, score-0.5]
7 In concordance with such naturally occurring variability, experimentally induced perturbations [1, 3, 12] are compensated in a way that maintains task performance rather than a specific stereotypical movement pattern. [sent-13, score-0.222]
8 This body of evidence is fundamentally incompatible with standard models of motor coordination that enforce a strict separation between trajectory planning and trajectory execution [2, 8, 17, 10]. [sent-14, score-0.855]
9 In such serial planning/execution models, the role of the planning stage is to resolve the redundancy inherent in the musculo-skeletal system, by replacing the behavioral goal (achievable via infinitely many movement trajectories) with a specific “desired trajectory. [sent-15, score-0.409]
10 ” Accurate execution of the desired trajectory guarantees achievement of the goal, and can be implemented with relatively simple trajectory-tracking algorithms. [sent-16, score-0.356]
11 This would be impossible if the behavioral goal were replaced with a specific trajectory. [sent-18, score-0.117]
12 Instead, these observations imply a very different control scheme, one which pursues the behavioral goal more directly. [sent-19, score-0.478]
13 Efforts to delineate such a control scheme have led to the idea of motor synergies, or high-level “control knobs,” that have invariant and predictable effects on the task-relevant movement parameters despite variability in individual degrees of freedom [9, 11]. [sent-20, score-1.059]
14 But the computational underpinnings of such an approach—how the synergies appropriate for a given task and plant can be constructed, what control scheme is capable of utilizing them, and why the motor system should prefer such a control scheme in the first place—remain unclear. [sent-21, score-1.527]
15 This general form of hierarchical control implies correlations among the control signals sent to multiple actuators (i. [sent-22, score-0.807]
16 , synergetic coupling) and a corresponding reduction in control space dimesionality. [sent-24, score-0.446]
17 Such phenonema have indeed been observed [4, 18], but the relationship to the hypothetical functional synergies remains to be established. [sent-25, score-0.17]
18 In this paper we aim to resolve the apparent conflict at the heart of the motor coordination problem, and clarify the relationship between variability, task goals, and motor synergies. [sent-26, score-0.993]
19 We treat motor coordination within the framework of stochastic optimal control, and postulate that the motor system approximates the best possible control scheme for a given task. [sent-27, score-1.429]
20 Such a control scheme will generally take the form of a feedback control law. [sent-28, score-0.91]
21 By postponing all decisions regarding movement details until the last possible moment, this control law takes advantage of the opportunities for more successful task completion that are constantly being created by unpredictable fluctuations away from the average trajectory. [sent-30, score-0.855]
22 Such exploitation of redundancy not only results in higher performance, but also gives rise to task-constrained variability and motor synergies—the phenomena we seek to explain. [sent-31, score-0.666]
23 If this principle holds, and the noise perturbs the system in all directions, the interplay of the noise and control processes will result in variability which is larger in task-irrelevant directions. [sent-35, score-0.676]
24 At the same time, the fact that certain deviations are not being corrected implies that the corresponding control subspace is not being used—which is the phenomenon typically interpreted as evidence for motor synergies [4, 18]. [sent-36, score-1.079]
25 An optimal feedback controller has nothing to gain from correcting task-irrelevant deviations, because its only concern is task performance and by definition such deviations do not interfere with performance. [sent-38, score-0.623]
26 On the other hand, generating a corrective control signal can be detrimental, because: 1) the noise in the motor system is known to be multiplicative [13] and therefore could increase; 2) the cost being minimized most likely includes a control-dependent effort penalty which could also increase. [sent-39, score-1.091]
27 1 Local analysis of a general class of optimal control problems Redundancy is not easy to define. [sent-42, score-0.486]
28 Consider the task of reaching, which requires the fingertip to be at a specified target at some point in time . [sent-43, score-0.096]
29 At time , all arm configurations for which the fingertip is at the target are redundant. [sent-44, score-0.075]
30 , control Consider a system with state , and dynamics , instantaneous scalar cost E ¦ ¡ ¢ @ 8 ¦ ¡ ¢ 3 1 F0 D%6§! [sent-47, score-0.606]
31 Control signals are generated by a feedback control law, which can be any mapping of the form . [sent-52, score-0.51]
32 The analysis below heavily relies on properties of the optimal cost-to-go function, defined as 1 ¦ ¢ P§¥¤I ¦¦ ¢ ¡ %§¤£§! [sent-53, score-0.125]
33 ¥¤¢ TR ¦¦¦ x¢ ¡ ¦ x¢ ¡ uw s q i YedW b 1 ¦ ¡ S where the minimum is achieved by the optimal control law . [sent-55, score-0.713]
34 Suppose that in a given task the system of interest (driven by the optimal control law) generates an average trajectory . [sent-56, score-0.78]
35 On a given trial, let be the deviation form the average trajectory at time . [sent-57, score-0.204]
36 Let be the change in the optimal cost-to-go due to ; i. [sent-58, score-0.125]
37 Now we are ready to define the deviation redundancy: the deviation is redundant iff . [sent-61, score-0.189]
38 Note that our definition reduces to the intuitive geometric definition at the end of the movement, where the cost function and optimal cost-to-go are identical. [sent-62, score-0.267]
39 SR ¡ SR ( 1 ¦¡ G#U ¢ ¡ ¦ d ¢¡ S R S VR ¦U¡)8© ¢¡ S R U¢ S R 1 ¦¡ S §¤©¡ ¤ ¦R ¢ ¡ To define the notion of “correction,” we need to separate the passive and active dynamics: ¦¡ U"! [sent-63, score-0.072]
40 ¤¢ S q R S R ¦¡ u ¢ @ ¦ ¡ q S q i ¦ ¡ ¢ @ q p n km 8 ¦ ¡ S q i ¦ ¡ ¢ 3 8 ¦ ¡ §¦7"6¡"! [sent-66, score-0.063]
41 ¤53 The (infinitesimal) expected change in be identified: signal is naturally defined as due to the control can now . [sent-79, score-0.361]
42 and , we obviously need to know In order to relate the quantities something about the optimal control law . [sent-81, score-0.747]
43 For problems in the above general form, the optimal control law is given [7] by the minimum where and are the gradient and Hessian of the optimal cost-to-go func. [sent-82, score-0.838]
44 Note that the latter formulation is still very general, and can represent realistic musculo-skeletal dynamics and motor tasks. [sent-91, score-0.431]
45 ¤ ¢ ©¨¦|p f n 1 ¦ ¦ ¤¢ p §¥| f n Using the fact1 that and , and eliminating terms that do not depend on , the expression that has to be minimized w. [sent-92, score-0.037]
46 ¤ i Therefore the optimal control law is ¦¡ U"! [sent-101, score-0.713]
47 We expand the optimal cost-to-go to second order: , also expand its gradient to first order: , and approximate all other quantities as being constant in a small neighborhood of . [sent-107, score-0.225]
48 Substituting in the above definitions yields i D¡ z $ U ¢¡ (d¡ ¦ ¢ ¦ "¦ ¢ ¦ ¡ ¢ Ud9% U ¢ S R ¦¡ ¦ q U ¢¡ tq ¡ ( 1 TU ¢¡ tq S d ¢¡ S q R ¡ ¦ q 8 ¦R ¦ ¡ ¢ U U"y ¦¡ UC¢ S R S R A 7 Bi ¡ 9e @d876¡ q e¡ ¦ q % ¦¡ ¢ 56h q c 0 3421h q c ! [sent-109, score-0.158]
49 When —which can happen for infinitely many when the Hessian is singular—the deviation is redundant and the optimal controller takes no corrective action. [sent-112, score-0.415]
50 Furthermore, and are positively correlated because 2 is a positive semi-definite matrix . [sent-113, score-0.035]
51 Thus the optimal controller resists single-trial deviations that take the system to more costly states, and magnifies deviations to less costly states. [sent-114, score-0.456]
52 To address such questions (and also build models of specific motor control experiments) we need to focus on a class of control problems for which the optimal control law can actually be found. [sent-118, score-1.814]
53 To that end, we have modified [15] the extensively studied LQG framework to include the multiplicative control noise characteristic of the motor system. [sent-119, score-0.838]
54 The control problems studied here and in ¡ D h C @C aD c iP c yv V xp v w aD X Be6`D gD fP a c G X e` c C @B6`D F X dR D QP EDa C bI a ` X R H a aD C c R I 1 Defining the unit vector as having a in position and in all other positions, we can write . [sent-120, score-0.361]
55 2 has to be positive semi-definite—or else we could find a control signal that makes the instantaneous cost negative, and that is impossible by definition. [sent-122, score-0.469]
56 q aD C X TD s rR p P @B6`D YWutV U QSQI 1 dv : dq 0. [sent-124, score-0.175]
57 the passive dynamics is stable); the last component of the state is (for similarity with motor control tasks), and are positive semi-definite, . [sent-128, score-0.909]
58 ¡ ( ©1 § ¨ £ the next section are in the form Dynamics Feedback Cost w w¡ w wi A8 w ¡ w £ wi ¡ 0 18 ¡ )1 w 7 ¡ z x w w ¡ w {{ w E ~ w H x s|{ w y8 w A8 w y)1 z ('w ¡ 7 Note that the system state is now partially observable, through noisy sensor readings . [sent-133, score-0.125]
59 Multiplicative noise complicates matters, but we have found [15] that for systems with stable passive dynamics a similar control strategy is very close to optimal. [sent-136, score-0.521]
60 , but also dynam- ¦ ¢ Specific motor control tasks are considered below. [sent-139, score-0.787]
61 Here we generate 100 random problems in the above form, compute the optimal control law in each case, and correlate the quantities and corr. [sent-140, score-0.747]
62 As the “dv : corr” curve in Figure 1 shows, they are positively correlated at all times. [sent-141, score-0.076]
63 We also show in Figure 1 that the Hessian of the optimal cost-to-go has similar shape to the task cost (“dv : dq” curve), and that the state covariance is smaller along dimensions where the task cost is larger; i. [sent-142, score-0.578]
64 ¥ ¨ © ¡ ¢ ¡ 0 () 21 V RrT R R r h Figure 2: Simulations of motor control tasks – see text. [sent-147, score-0.787]
65 3 Applications to motor coordination We have used the modified LQG framework to model a wide range of specific motor control tasks [14, 15], and always found that optimal feedback controllers generate variability that is elongated in redundant dimensions. [sent-148, score-1.928]
66 The first model (Figure 2, Bimanual Tasks) includes two 1D point masses with positions X1 and X2, each driven with a force actuator whose output is a noisy second-order low-pass filtered version of the corresponding control signal. [sent-150, score-0.483]
67 The feedback contains noisy position, velocity, and force information—delayed by 50 msec (by augmenting the system state with a sequence of recent sensor readings). [sent-151, score-0.299]
68 The “ Difference” task requires the two points to start moving 20cm apart, and stop at identical but unspecified locations. [sent-152, score-0.157]
69 The covariance of the final state is elongated in the task-irrelevant dimension: the two points always stop close to each other, but the final location can vary substantially from trial to trial. [sent-153, score-0.169]
70 A related phenomenon has been observed in the more complex bimanual task of inserting a pointer in a cup [6]. [sent-154, score-0.17]
71 We now modify the task: in “Sum,” the two points start at the same location and have to stop so that the midpoint between them is at zero. [sent-155, score-0.061]
72 We also illustrate a Via Point task, where a 2D point mass has to pass through a sequence of two intermediate targets and stop at a final target (tracing an S-shaped curve). [sent-157, score-0.061]
73 , the weight of the corresponding positional constraint is increased), the variability decreases at that point. [sent-161, score-0.188]
74 4 Multi-attribute costs and desired trajectory tracking As we stated earlier, replacing the task goal with a desired trajectory (which achieves the goal if executed precisely) is generally suboptimal. [sent-164, score-0.774]
75 Here we present a more general view of desired trajectory tracking which clarifies its relationship to optimal control. [sent-166, score-0.388]
76 Desired trajectory tracking can be incorporated in the present framework by using a modified cost, one that specifies a desired state at each point in time, and penalizes the deviations from that state. [sent-167, score-0.414]
77 Such a modified cost would normally include the original task cost (e. [sent-168, score-0.312]
78 , the terms that specify the desired terminal state), but also a large number of additional terms that do not need to be minimized in order to accomplish the actual task. [sent-170, score-0.145]
79 This raises the question: what happens to the expected values of the terms in the original cost, when we attempt to minimize other costs simultaneously? [sent-171, score-0.109]
80 Intuitively, one would expect the orig- inal costs to increase (relative to the costs obtained by the task-optimal controller). [sent-172, score-0.218]
81 Consider a family of optimal control problems parameterized by the vector , with cost functions . [sent-175, score-0.594]
82 Let be an optimal control law 3 , and be the vector of expected component costs achieved by ; i. [sent-180, score-0.822]
83 Then we can define the inverse mapping from the expected component cost manifold to the weight manifold , as illustrated in Figure 3. [sent-189, score-0.316]
84 From the definitions of and , the total expected cost achieved by is . [sent-190, score-0.108]
85 is an optimal control law for the problem defined by the weight vector , no other Since control law can achieve a smaller total expected cost, and so for all . [sent-191, score-1.335]
86 Therefore, if we construct the dimensional hyperplane that contains and is orthogonal to , the entire manifold has to lie in the half-space not containing the origin. [sent-192, score-0.087]
87 Thus is tangent to the manifold at point , has non-negative curvature, and the unit vector which is normal to at satisfies 4 . [sent-193, score-0.137]
88 Let , be a parametric curve that passes through the point of interest : . [sent-194, score-0.041]
89 By differentiating at we obtain the tangent to the curve at . [sent-196, score-0.131]
90 , the tangent cannot turn away The non-negative curvature of implies from the normal without crossing the hyperplane . [sent-201, score-0.083]
91 3 If we assume that the optimal control law is unique, all inequalities below become strict. [sent-203, score-0.713]
92 on the unit sphere For a general 2D manifold embedded in , the mapping that satisfies is known as the Gauss map, and plays an important role in surface differential geometry. [sent-204, score-0.087]
93 h h f igc e d c t rqq gP t rqq s p 4 The above result means that whenever we change the weight vector , the corresponding vector of expected component costs achieved by the (new) optimal control law will change in an “opposite” direction. [sent-205, score-0.97]
94 The coordination of arm movements: an experimentally confirmed mathematical model. [sent-248, score-0.181]
95 Multi-joint limbs permit a flexible response to unpredictable events. [sent-275, score-0.045]
96 The variation of hand tremor with force in healthy subjects. [sent-280, score-0.065]
97 Optimal feedback control as a theory of motor coordination. [sent-284, score-0.889]
98 Optimal feedback control under signal-dependent noise: Methodology for modeling biological movement. [sent-287, score-0.51]
99 The uncontrolled manifold concept: Identifying control variables for a functional task. [sent-296, score-0.497]
100 Formation and control of optimal trajectory in human multijoint arm movement: Minimum torque-change model. [sent-301, score-0.719]
wordName wordTfidf (topN-words)
[('motor', 0.379), ('control', 0.361), ('law', 0.227), ('intervention', 0.17), ('synergies', 0.17), ('trajectory', 0.158), ('variability', 0.154), ('feedback', 0.149), ('redundancy', 0.133), ('movement', 0.126), ('optimal', 0.125), ('todorov', 0.114), ('costs', 0.109), ('cost', 0.108), ('deviations', 0.106), ('coordination', 0.106), ('redundant', 0.097), ('task', 0.096), ('dq', 0.094), ('manifold', 0.087), ('actuators', 0.085), ('lqg', 0.085), ('synergetic', 0.085), ('dv', 0.081), ('controller', 0.079), ('tq', 0.079), ('behavioral', 0.078), ('arm', 0.075), ('ud', 0.075), ('achievement', 0.074), ('bimanual', 0.074), ('passive', 0.072), ('desired', 0.07), ('controllers', 0.068), ('corrective', 0.068), ('interfere', 0.068), ('ad', 0.067), ('hessian', 0.067), ('force', 0.065), ('corrected', 0.063), ('elongated', 0.063), ('km', 0.063), ('multiplicative', 0.062), ('stop', 0.061), ('uu', 0.059), ('res', 0.059), ('tu', 0.059), ('actuator', 0.057), ('bizzi', 0.057), ('dcov', 0.057), ('gelfand', 0.057), ('hogan', 0.057), ('ngertip', 0.057), ('nkel', 0.057), ('rqq', 0.057), ('tsetlin', 0.057), ('ttr', 0.057), ('uctuations', 0.057), ('execution', 0.054), ('dynamics', 0.052), ('tangent', 0.05), ('corr', 0.049), ('gur', 0.049), ('uncontrolled', 0.049), ('principle', 0.049), ('sr', 0.048), ('movements', 0.048), ('tasks', 0.047), ('minimal', 0.047), ('deviation', 0.046), ('state', 0.045), ('unpredictable', 0.045), ('coupling', 0.044), ('ict', 0.042), ('plant', 0.042), ('curve', 0.041), ('modi', 0.041), ('aq', 0.04), ('differentiating', 0.04), ('readings', 0.04), ('ru', 0.04), ('vd', 0.04), ('system', 0.04), ('scheme', 0.039), ('goal', 0.039), ('accomplish', 0.038), ('rms', 0.038), ('minimized', 0.037), ('correction', 0.036), ('suppressed', 0.036), ('noise', 0.036), ('tr', 0.035), ('tracking', 0.035), ('positively', 0.035), ('weight', 0.034), ('quantities', 0.034), ('geometric', 0.034), ('curvature', 0.033), ('expand', 0.033), ('resolve', 0.033)]
simIndex simValue paperId paperTitle
same-paper 1 0.99999923 9 nips-2002-A Minimal Intervention Principle for Coordinated Movement
Author: Emanuel Todorov, Michael I. Jordan
Abstract: Behavioral goals are achieved reliably and repeatedly with movements rarely reproducible in their detail. Here we offer an explanation: we show that not only are variability and goal achievement compatible, but indeed that allowing variability in redundant dimensions is the optimal control strategy in the face of uncertainty. The optimal feedback control laws for typical motor tasks obey a “minimal intervention” principle: deviations from the average trajectory are only corrected when they interfere with the task goals. The resulting behavior exhibits task-constrained variability, as well as synergetic coupling among actuators—which is another unexplained empirical phenomenon.
2 0.17964296 187 nips-2002-Spikernels: Embedding Spiking Neurons in Inner-Product Spaces
Author: Lavi Shpigelman, Yoram Singer, Rony Paz, Eilon Vaadia
Abstract: Inner-product operators, often referred to as kernels in statistical learning, define a mapping from some input space into a feature space. The focus of this paper is the construction of biologically-motivated kernels for cortical activities. The kernels we derive, termed Spikernels, map spike count sequences into an abstract vector space in which we can perform various prediction tasks. We discuss in detail the derivation of Spikernels and describe an efficient algorithm for computing their value on any two sequences of neural population spike counts. We demonstrate the merits of our modeling approach using the Spikernel and various standard kernels for the task of predicting hand movement velocities from cortical recordings. In all of our experiments all the kernels we tested outperform the standard scalar product used in regression with the Spikernel consistently achieving the best performance. 1
3 0.17202757 153 nips-2002-Neural Decoding of Cursor Motion Using a Kalman Filter
Author: W Wu, M. J. Black, Y. Gao, M. Serruya, A. Shaikhouni, J. P. Donoghue, Elie Bienenstock
Abstract: The direct neural control of external devices such as computer displays or prosthetic limbs requires the accurate decoding of neural activity representing continuous movement. We develop a real-time control system using the spiking activity of approximately 40 neurons recorded with an electrode array implanted in the arm area of primary motor cortex. In contrast to previous work, we develop a control-theoretic approach that explicitly models the motion of the hand and the probabilistic relationship between this motion and the mean firing rates of the cells in 70 bins. We focus on a realistic cursor control task in which the subject must move a cursor to “hit” randomly placed targets on a computer monitor. Encoding and decoding of the neural data is achieved with a Kalman filter which has a number of advantages over previous linear filtering techniques. In particular, the Kalman filter reconstructions of hand trajectories in off-line experiments are more accurate than previously reported results and the model provides insights into the nature of the neural coding of movement. ¨ ©§
4 0.15843599 155 nips-2002-Nonparametric Representation of Policies and Value Functions: A Trajectory-Based Approach
Author: Christopher G. Atkeson, Jun Morimoto
Abstract: A longstanding goal of reinforcement learning is to develop nonparametric representations of policies and value functions that support rapid learning without suffering from interference or the curse of dimensionality. We have developed a trajectory-based approach, in which policies and value functions are represented nonparametrically along trajectories. These trajectories, policies, and value functions are updated as the value function becomes more accurate or as a model of the task is updated. We have applied this approach to periodic tasks such as hopping and walking, which required handling discount factors and discontinuities in the task dynamics, and using function approximation to represent value functions at discontinuities. We also describe extensions of the approach to make the policies more robust to modeling error and sensor noise.
5 0.14960641 123 nips-2002-Learning Attractor Landscapes for Learning Motor Primitives
Author: Auke J. Ijspeert, Jun Nakanishi, Stefan Schaal
Abstract: Many control problems take place in continuous state-action spaces, e.g., as in manipulator robotics, where the control objective is often defined as finding a desired trajectory that reaches a particular goal state. While reinforcement learning offers a theoretical framework to learn such control policies from scratch, its applicability to higher dimensional continuous state-action spaces remains rather limited to date. Instead of learning from scratch, in this paper we suggest to learn a desired complex control policy by transforming an existing simple canonical control policy. For this purpose, we represent canonical policies in terms of differential equations with well-defined attractor properties. By nonlinearly transforming the canonical attractor dynamics using techniques from nonparametric regression, almost arbitrary new nonlinear policies can be generated without losing the stability properties of the canonical system. We demonstrate our techniques in the context of learning a set of movement skills for a humanoid robot from demonstrations of a human teacher. Policies are acquired rapidly, and, due to the properties of well formulated differential equations, can be re-used and modified on-line under dynamic changes of the environment. The linear parameterization of nonparametric regression moreover lends itself to recognize and classify previously learned movement skills. Evaluations in simulations and on an actual 30 degree-offreedom humanoid robot exemplify the feasibility and robustness of our approach. 1
6 0.12973644 128 nips-2002-Learning a Forward Model of a Reflex
7 0.10700818 116 nips-2002-Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior
8 0.10399019 55 nips-2002-Combining Features for BCI
9 0.093540087 144 nips-2002-Minimax Differential Dynamic Programming: An Application to Robust Biped Walking
10 0.08761131 47 nips-2002-Branching Law for Axons
11 0.079349205 65 nips-2002-Derivative Observations in Gaussian Process Models of Dynamic Systems
12 0.076236777 33 nips-2002-Approximate Linear Programming for Average-Cost Dynamic Programming
13 0.07605163 49 nips-2002-Charting a Manifold
14 0.067640595 36 nips-2002-Automatic Alignment of Local Representations
15 0.060301974 95 nips-2002-Gaussian Process Priors with Uncertain Inputs Application to Multiple-Step Ahead Time Series Forecasting
16 0.05894224 51 nips-2002-Classifying Patterns of Visual Motion - a Neuromorphic Approach
17 0.058633741 5 nips-2002-A Digital Antennal Lobe for Pattern Equalization: Analysis and Design
18 0.05737314 18 nips-2002-Adaptation and Unsupervised Learning
19 0.056964286 77 nips-2002-Effective Dimension and Generalization of Kernel Learning
20 0.056794494 60 nips-2002-Convergence Properties of Some Spike-Triggered Analysis Techniques
topicId topicWeight
[(0, -0.204), (1, 0.055), (2, -0.113), (3, -0.036), (4, -0.011), (5, 0.003), (6, -0.02), (7, 0.025), (8, 0.169), (9, 0.225), (10, -0.087), (11, 0.066), (12, 0.127), (13, -0.08), (14, -0.086), (15, 0.047), (16, -0.059), (17, -0.046), (18, -0.094), (19, -0.059), (20, -0.002), (21, 0.016), (22, 0.05), (23, -0.029), (24, 0.119), (25, -0.026), (26, 0.217), (27, -0.152), (28, 0.138), (29, -0.087), (30, -0.222), (31, 0.061), (32, 0.01), (33, -0.032), (34, 0.059), (35, -0.029), (36, -0.084), (37, -0.139), (38, 0.088), (39, 0.048), (40, 0.042), (41, 0.125), (42, 0.006), (43, 0.0), (44, 0.019), (45, -0.086), (46, -0.002), (47, -0.009), (48, -0.016), (49, -0.088)]
simIndex simValue paperId paperTitle
same-paper 1 0.98022437 9 nips-2002-A Minimal Intervention Principle for Coordinated Movement
Author: Emanuel Todorov, Michael I. Jordan
Abstract: Behavioral goals are achieved reliably and repeatedly with movements rarely reproducible in their detail. Here we offer an explanation: we show that not only are variability and goal achievement compatible, but indeed that allowing variability in redundant dimensions is the optimal control strategy in the face of uncertainty. The optimal feedback control laws for typical motor tasks obey a “minimal intervention” principle: deviations from the average trajectory are only corrected when they interfere with the task goals. The resulting behavior exhibits task-constrained variability, as well as synergetic coupling among actuators—which is another unexplained empirical phenomenon.
2 0.68694055 123 nips-2002-Learning Attractor Landscapes for Learning Motor Primitives
Author: Auke J. Ijspeert, Jun Nakanishi, Stefan Schaal
Abstract: Many control problems take place in continuous state-action spaces, e.g., as in manipulator robotics, where the control objective is often defined as finding a desired trajectory that reaches a particular goal state. While reinforcement learning offers a theoretical framework to learn such control policies from scratch, its applicability to higher dimensional continuous state-action spaces remains rather limited to date. Instead of learning from scratch, in this paper we suggest to learn a desired complex control policy by transforming an existing simple canonical control policy. For this purpose, we represent canonical policies in terms of differential equations with well-defined attractor properties. By nonlinearly transforming the canonical attractor dynamics using techniques from nonparametric regression, almost arbitrary new nonlinear policies can be generated without losing the stability properties of the canonical system. We demonstrate our techniques in the context of learning a set of movement skills for a humanoid robot from demonstrations of a human teacher. Policies are acquired rapidly, and, due to the properties of well formulated differential equations, can be re-used and modified on-line under dynamic changes of the environment. The linear parameterization of nonparametric regression moreover lends itself to recognize and classify previously learned movement skills. Evaluations in simulations and on an actual 30 degree-offreedom humanoid robot exemplify the feasibility and robustness of our approach. 1
3 0.66290301 153 nips-2002-Neural Decoding of Cursor Motion Using a Kalman Filter
Author: W Wu, M. J. Black, Y. Gao, M. Serruya, A. Shaikhouni, J. P. Donoghue, Elie Bienenstock
Abstract: The direct neural control of external devices such as computer displays or prosthetic limbs requires the accurate decoding of neural activity representing continuous movement. We develop a real-time control system using the spiking activity of approximately 40 neurons recorded with an electrode array implanted in the arm area of primary motor cortex. In contrast to previous work, we develop a control-theoretic approach that explicitly models the motion of the hand and the probabilistic relationship between this motion and the mean firing rates of the cells in 70 bins. We focus on a realistic cursor control task in which the subject must move a cursor to “hit” randomly placed targets on a computer monitor. Encoding and decoding of the neural data is achieved with a Kalman filter which has a number of advantages over previous linear filtering techniques. In particular, the Kalman filter reconstructions of hand trajectories in off-line experiments are more accurate than previously reported results and the model provides insights into the nature of the neural coding of movement. ¨ ©§
4 0.56379902 187 nips-2002-Spikernels: Embedding Spiking Neurons in Inner-Product Spaces
Author: Lavi Shpigelman, Yoram Singer, Rony Paz, Eilon Vaadia
Abstract: Inner-product operators, often referred to as kernels in statistical learning, define a mapping from some input space into a feature space. The focus of this paper is the construction of biologically-motivated kernels for cortical activities. The kernels we derive, termed Spikernels, map spike count sequences into an abstract vector space in which we can perform various prediction tasks. We discuss in detail the derivation of Spikernels and describe an efficient algorithm for computing their value on any two sequences of neural population spike counts. We demonstrate the merits of our modeling approach using the Spikernel and various standard kernels for the task of predicting hand movement velocities from cortical recordings. In all of our experiments all the kernels we tested outperform the standard scalar product used in regression with the Spikernel consistently achieving the best performance. 1
5 0.53169107 155 nips-2002-Nonparametric Representation of Policies and Value Functions: A Trajectory-Based Approach
Author: Christopher G. Atkeson, Jun Morimoto
Abstract: A longstanding goal of reinforcement learning is to develop nonparametric representations of policies and value functions that support rapid learning without suffering from interference or the curse of dimensionality. We have developed a trajectory-based approach, in which policies and value functions are represented nonparametrically along trajectories. These trajectories, policies, and value functions are updated as the value function becomes more accurate or as a model of the task is updated. We have applied this approach to periodic tasks such as hopping and walking, which required handling discount factors and discontinuities in the task dynamics, and using function approximation to represent value functions at discontinuities. We also describe extensions of the approach to make the policies more robust to modeling error and sensor noise.
6 0.521258 81 nips-2002-Expected and Unexpected Uncertainty: ACh and NE in the Neocortex
7 0.50967646 144 nips-2002-Minimax Differential Dynamic Programming: An Application to Robust Biped Walking
8 0.48951051 128 nips-2002-Learning a Forward Model of a Reflex
9 0.46559432 47 nips-2002-Branching Law for Axons
10 0.44829282 179 nips-2002-Scaling of Probability-Based Optimization Algorithms
11 0.38619697 55 nips-2002-Combining Features for BCI
12 0.38470146 60 nips-2002-Convergence Properties of Some Spike-Triggered Analysis Techniques
13 0.38381085 117 nips-2002-Intrinsic Dimension Estimation Using Packing Numbers
14 0.37766328 65 nips-2002-Derivative Observations in Gaussian Process Models of Dynamic Systems
15 0.34774336 33 nips-2002-Approximate Linear Programming for Average-Cost Dynamic Programming
16 0.31677213 18 nips-2002-Adaptation and Unsupervised Learning
17 0.2964232 49 nips-2002-Charting a Manifold
18 0.28979763 36 nips-2002-Automatic Alignment of Local Representations
19 0.28247708 116 nips-2002-Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior
20 0.28148484 5 nips-2002-A Digital Antennal Lobe for Pattern Equalization: Analysis and Design
topicId topicWeight
[(7, 0.01), (11, 0.021), (14, 0.018), (23, 0.03), (42, 0.051), (54, 0.118), (55, 0.347), (57, 0.021), (64, 0.022), (67, 0.038), (68, 0.03), (74, 0.053), (92, 0.033), (98, 0.102)]
simIndex simValue paperId paperTitle
Author: Alistair Bray, Dominique Martinez
Abstract: In Slow Feature Analysis (SFA [1]), it has been demonstrated that high-order invariant properties can be extracted by projecting inputs into a nonlinear space and computing the slowest changing features in this space; this has been proposed as a simple general model for learning nonlinear invariances in the visual system. However, this method is highly constrained by the curse of dimensionality which limits it to simple theoretical simulations. This paper demonstrates that by using a different but closely-related objective function for extracting slowly varying features ([2, 3]), and then exploiting the kernel trick, this curse can be avoided. Using this new method we show that both the complex cell properties of translation invariance and disparity coding can be learnt simultaneously from natural images when complex cells are driven by simple cells also learnt from the image. The notion of maximising an objective function based upon the temporal predictability of output has been progressively applied in modelling the development of invariances in the visual system. F6ldiak used it indirectly via a Hebbian trace rule for modelling the development of translation invariance in complex cells [4] (closely related to many other models [5,6,7]); this rule has been used to maximise invariance as one component of a hierarchical system for object and face recognition [8]. On the other hand, similar functions have been maximised directly in networks for extracting linear [2] and nonlinear [9, 1] visual invariances. Direct maximisation of such functions have recently been used to model complex cells [10] and as an alternative to maximising sparseness/independence in modelling simple cells [11]. Slow Feature Analysis [1] combines many of the best properties of these methods to provide a good general nonlinear model. That is, it uses an objective function that minimises the first-order temporal derivative of the outputs; it provides a closedform solution which maximises this function by projecting inputs into a nonlinear http://www.loria.fr/equipes/cortex/ space; it exploits sphering (or PCA-whitening) of the data to ensure that all outputs have unit variance and are uncorrelated. However, the method suffers from the curse of dimensionality in that the nonlinear feature space soon becomes very large as the input dimension grows, and yet this feature space must be represented explicitly in order for the essential sphering to occur. The alternative that we propose here is to use the objective function of Stone [2, 9], that maximises output variance over a long period whilst minimising variance over a shorter period; in the linear case, this can be implemented by a biologically plausible mixture of Hebbian and anti-Hebbian learning on the same synapses [2]. In recent work, Stone has proposed a closed-form solution for maximising this function in the linear domain of blind source separation that does not involve data-sphering. This paper describes how this method can be kernelised. The use of the
same-paper 2 0.91637969 9 nips-2002-A Minimal Intervention Principle for Coordinated Movement
Author: Emanuel Todorov, Michael I. Jordan
Abstract: Behavioral goals are achieved reliably and repeatedly with movements rarely reproducible in their detail. Here we offer an explanation: we show that not only are variability and goal achievement compatible, but indeed that allowing variability in redundant dimensions is the optimal control strategy in the face of uncertainty. The optimal feedback control laws for typical motor tasks obey a “minimal intervention” principle: deviations from the average trajectory are only corrected when they interfere with the task goals. The resulting behavior exhibits task-constrained variability, as well as synergetic coupling among actuators—which is another unexplained empirical phenomenon.
3 0.91501701 19 nips-2002-Adapting Codes and Embeddings for Polychotomies
Author: Gunnar Rätsch, Sebastian Mika, Alex J. Smola
Abstract: In this paper we consider formulations of multi-class problems based on a generalized notion of a margin and using output coding. This includes, but is not restricted to, standard multi-class SVM formulations. Differently from many previous approaches we learn the code as well as the embedding function. We illustrate how this can lead to a formulation that allows for solving a wider range of problems with for instance many classes or even “missing classes”. To keep our optimization problems tractable we propose an algorithm capable of solving them using twoclass classifiers, similar in spirit to Boosting.
4 0.89854348 95 nips-2002-Gaussian Process Priors with Uncertain Inputs Application to Multiple-Step Ahead Time Series Forecasting
Author: Agathe Girard, Carl Edward Rasmussen, Joaquin Quiñonero Candela, Roderick Murray-Smith
Abstract: We consider the problem of multi-step ahead prediction in time series analysis using the non-parametric Gaussian process model. -step ahead forecasting of a discrete-time non-linear dynamic system can be performed by doing repeated one-step ahead predictions. For a state-space model of the form , the prediction of at time is based on the point estimates of the previous outputs. In this paper, we show how, using an analytical Gaussian approximation, we can formally incorporate the uncertainty about intermediate regressor values, thus updating the uncertainty on the current prediction. ¡ % # ¢ ¡ ¢ ¡¨ ¦ ¤ ¢ $
5 0.7556535 193 nips-2002-Temporal Coherence, Natural Image Sequences, and the Visual Cortex
Author: Jarmo Hurri, Aapo Hyvärinen
Abstract: We show that two important properties of the primary visual cortex emerge when the principle of temporal coherence is applied to natural image sequences. The properties are simple-cell-like receptive fields and complex-cell-like pooling of simple cell outputs, which emerge when we apply two different approaches to temporal coherence. In the first approach we extract receptive fields whose outputs are as temporally coherent as possible. This approach yields simple-cell-like receptive fields (oriented, localized, multiscale). Thus, temporal coherence is an alternative to sparse coding in modeling the emergence of simple cell receptive fields. The second approach is based on a two-layer statistical generative model of natural image sequences. In addition to modeling the temporal coherence of individual simple cells, this model includes inter-cell temporal dependencies. Estimation of this model from natural data yields both simple-cell-like receptive fields, and complex-cell-like pooling of simple cell outputs. In this completely unsupervised learning, both layers of the generative model are estimated simultaneously from scratch. This is a significant improvement on earlier statistical models of early vision, where only one layer has been learned, and others have been fixed a priori.
6 0.72017413 10 nips-2002-A Model for Learning Variance Components of Natural Images
7 0.70995522 28 nips-2002-An Information Theoretic Approach to the Functional Classification of Neurons
8 0.70298088 2 nips-2002-A Bilinear Model for Sparse Coding
9 0.65291089 14 nips-2002-A Probabilistic Approach to Single Channel Blind Signal Separation
10 0.64419574 81 nips-2002-Expected and Unexpected Uncertainty: ACh and NE in the Neocortex
11 0.63959199 119 nips-2002-Kernel Dependency Estimation
12 0.6313116 159 nips-2002-Optimality of Reinforcement Learning Algorithms with Linear Function Approximation
13 0.63119483 123 nips-2002-Learning Attractor Landscapes for Learning Motor Primitives
14 0.6227597 149 nips-2002-Multiclass Learning by Probabilistic Embeddings
15 0.62244821 106 nips-2002-Hyperkernels
16 0.62187576 128 nips-2002-Learning a Forward Model of a Reflex
17 0.62152112 24 nips-2002-Adaptive Scaling for Feature Selection in SVMs
18 0.6188671 5 nips-2002-A Digital Antennal Lobe for Pattern Equalization: Analysis and Design
19 0.61826169 11 nips-2002-A Model for Real-Time Computation in Generic Neural Microcircuits
20 0.61678129 141 nips-2002-Maximally Informative Dimensions: Analyzing Neural Responses to Natural Signals