Apply DPC to more complex dynamical systems?

[FenglongSong]

Hi all,

First of all, thank you for providing interesting ideas and this great software package.

In your paper and the examples provided in this repo, the numerical experiments are conducted on low-dimensional system (nx<=2). I'm wondering if you have ever tested how well Differentiable Predictive Control can perform when applied to more complex dynamical systems (e.g., a quadrotor or something with a larger state space). Any insight will be much appreciated!

[drgona]

Hi @FenglongSong, thanks for the inquiry.

We did apply and test the DPC algorithm to systems with larger state space.

For instance, we trained DPC to control a multi-zone building thermal dynamics model, with 6 controlled states, and 7 inputs rolled over 32 prediction steps, resulting in a control policy with overall 224 control variables.
https://www.sciencedirect.com/science/article/pii/S2405896321012933
very old version of the implementation can be found here:
https://github.com/pnnl/neuromancer/tree/building_ctrl

Another example can be found in our paper
https://arxiv.org/abs/2004.11184
where we trained a control policy for reference tracking of a 12 dimensional linearized model of a quadcopter system
old implementation can be found here:
https://github.com/pnnl/neuromancer/tree/DPC/examples/control_ODE/Example_3

We are currently working on deploying DPC for a quadcopter model in the mujoco environment.

As with any DPC application you want to make sure that your model of the system used to train the policy accurately represents the controlled system, otherwise you will end up with plant-model mismatch that would need to be compensated upon deployment, for instance with low level PID controllers (common practice in robotics).

Another issue to be mindful is sampling of the parametric space. For large scale problem you would need to train the policy on more samples to obtain robust results. Here clever sampling methods (as opposed to uniform sampling) could help with convergence and task coverage.

Hope this helps.

[FenglongSong]

Thanks for the detailed explanation and nice insights! I'm also interested in using the DPC method on quadrotors to see if it brings performance improvements. Looking forward to seeing your future works!

[drgona]

@FenglongSong we will open source the code on applying DPC to the quadcopter model together with preprint by the end of this calendar year.