llama-factory only supports single gpu when using its GUI. See https://github.com/hiyouga/LLaMA-Factory?tab=readme-ov-file#llama-board-gui