Authors: Soohyuk Cho, Dun-Ming Huang, Ryan Liu
Welcome to code repository for Model-Based State-Diffuser for Sample Efficient Online Reinforcement Learning
The implementation of diffusion is based on labml.ai and SynthER. The infrastructe and SAC agent is modified from UC Berkeley CS 285 HW5.
git clone [email protected]:ryanliu30/StateDiffusion.git
cd StateDiffusion
conda create -n StateDiffusion python=3.10
conda activate StateDiffusion
pip install -r requirements.txt
pip install -e .
To begin with, run the following command:
python cs285/scripts/run.py --dynamics_config_file experiments/dynamics/halfcheetah.yaml --sac_config_file experiments/sac/vanilla.yaml
This will run the baseline vanilla SAC in the halfcheetah envoronment. For MBPO:
python cs285/scripts/run.py --dynamics_config_file experiments/dynamics/halfcheetah.yaml --sac_config_file experiments/sac/mbpo.yaml
For SynthER:
python cs285/scripts/run.py --dynamics_config_file experiments/dynamics/halfcheetah.yaml --sac_config_file experiments/sac/mbpo.yaml --diffusion_config experiments/diff/all.yaml
For State Diffusion:
python cs285/scripts/run.py --dynamics_config_file experiments/dynamics/halfcheetah.yaml --sac_config_file experiments/sac/mbpo.yaml --diffusion_config experiments/diff/state.yaml
To reproduce our result, we recommend to run the following command:
python run_all.py --name halfcheetah;
python run_all.py --name hopper;
python run_all.py --name walker;
To produce the plots, please run the make_plots.ipynb notebook. The logs we used to write the report is also provided in the repo.