Dynamical Movement Primitives

Lunch and Learn - 2025/May/19

🦆 Alejandro Suárez Hernández

Motivation

Learn complex trajectories in an adaptable way

Biological inspiration

Simple motion templates lead to complex motions
Central Pattern Generator generate coordination signal
Not entirely accurate!

Enter DMPs

Mathematical framework for imitating trajectories
Conceived by Ijkspeert and colleagues between 2003 and 2013
Discrete and rhythmic (we focus on discrete)
Warning! Only kinematics, no control

Core idea

\[ \tau \ddot{y} = \alpha_z (\beta_z (g - y) - \tau \dot{y}) + f) \]

Harmonic oscillator

For $ \alpha_z = 4 \beta_z $ the system is critically damped

where

$ y $ is the state variable (position)
$ \dot{y} $ and $ \ddot{y} $ are velocity and acceleration, respectively
$ \alpha_z $ and $ \beta_z $ are parameters that govern the attractor behavior
$ \tau $ is used to control the speed of the trajectory
$ f $ is the forcing term
$ g $ is an attractor

\[ \tau \ddot{y} = \alpha_z (\beta_z (g - y) - \tau \dot{y}) + f) \] Remind you of anything?

Consider:

$ \tau = 1 $
$ P = \alpha_z \beta_z $
$ D = \alpha_z $
$ f = 0 $

Maybe now?

\[ \ddot{y} = P (g - y) - D \dot{y} \]

Demo of attractor behavior (2 DMPs, $ f = 0 $)

What about the forcing term?

$ f \neq 0 $ allows interesting trajectories

Sinusoidal forcing terms:

Vanishing forcing terms

$ f \rightarrow 0 $ as $ t \rightarrow \infty $ to guarantee convergence

Vanishing sinusoidal forcing terms:

Adjustable forcing term (first version)

\[ f(t) = \frac{\sum_i^N \Psi_i(t) w_i}{\sum_i^N \Psi_i(t) } \]

Each $ \Psi_i $ is a basis function (e.g. Gaussian):

E.g. 20 Gaussian basis functions ( $ N = 20 $)

The canonical system

"Recoding" of time into phase variable x

\[ x = e^{-\frac{\alpha_x}{\tau} t} \]

Why?

Time: $ 0 \rightarrow T $ (trajectory duration), phase: $ 1 \rightarrow 0 $
Provides DOFs coordination

E.g. 20 Gaussian basis functions w.r.t. phase $ x $

Ijspeert, Nakanishi, Hoffman, Pastor, and Schaal (2013)

Adjustable forcing term (second version)

\[ f(x) = \frac{\sum_i^N \Psi_i(x) w_i}{\sum_i^N \Psi_i(x) } x ( g - y_0 ) \]

What's different?

Depends on phase
Vanishes as $ x \rightarrow 0 $
Scaled by $ (g - y_0) $

How do we learn the parameters?

Assume demonstration $y_\text{demo}$, $\dot{y}_\text{demo}$, $\ddot{y}_\text{demo}$ with $ P $ data points
Calculate \[ f_\text{target} = \tau^2 \ddot{y}_\text{demo} - \alpha_z(\beta_z(g-y_\text{demo}) - \tau \dot{y}_\text{demo}) \]
Minimize weighted local regression loss function \[ J_i = \sum_{t=i}^P \Psi_i(x(t))(f_\text{target}(t) - w_i \xi(t)), \] where $\xi(t) = x(t)(g - y_0)$

Turns out there's analytical solution! \[ w_i = \frac{\boldsymbol{s}^T \Gamma_i \boldsymbol{f}_\text{target}}{\boldsymbol{s}^T \Gamma_i \boldsymbol{s}}, \] where \[ \boldsymbol{s} = \begin{pmatrix} \xi(1)\\ \xi(2)\\ \dotsc\\ \xi(P) \end{pmatrix} \quad \Gamma_i = \begin{pmatrix} \Psi_i(x(1)) & & & 0 \\ & \Psi_i(x(2)) & & \\ & & \dotsc & \\ 0 & & & \Psi_i(x(P)) \end{pmatrix} \quad \boldsymbol{f}_\text{target} = \begin{pmatrix} f_\text{target}(1)\\ f_\text{target}(2)\\ \dotsc\\ f_\text{target}(P)\\ \end{pmatrix} \]

Evolution of "force field"

Ijspeert, Nakanishi, Hoffman, Pastor, and Schaal (2013)

About rhytmic DMPs

Not very different. Achieved by...
... periodic phase
... or linear phase and periodic basis function (Von-Mises)
Example of usage: gait generation

Demonstration #1: drawing

Demonstration #2: airhockey

Example #1: kinestetic teaching

Example #2: kinestetic and mocap

Example #3: teaching in VR

Some practical considerations

Allows obstacle avoidance and error coupling
Usable in joint and cartesian space
Special care when $ g = y0 $

Conclusions

One-shot learning of trajectories (both + and -)
Can learn both rhythmic and discrete patterns
ONLY kinematics!!
Very nice adaptability (scale and temporal).
Supports moving targets!
Learns movement styles, trajectories converges into a target

Credit: https://air-hockey-challenge.robot-learning.net/