Shadow-FT

Official code for paper "Shadow-FT: Tuning Instruct via Base"

[📜 Paper] • [🤗 HF Models] • [🐱 GitHub]

This repo contains the code for our paper: Shadow-FT: Tuning Instruct via Base by Taiqiang Wu* Runming Yang*, Jiayi Li, Pengfei Hu, Ngai Wong and Yujiu Yang.

* for equal contributions.

There is an explanation blog for this paper (in Chinese).

Overview

Observation:

• Directly tuning the INSTRUCT (i.e., instruction tuned) models often leads to marginal improvements and even performance degeneration.

• Paired BASE models, the foundation for these INSTRUCT variants, contain highly similar weight values (i.e., less than 2% on average for Llama 3.1 8B).

$\Rightarrow$ We propose the Shadow-FT framework to tune the INSTRUCT models by leveraging the corresponding BASE models. The key insight is to fine-tune the BASE model, and then directly graft the learned weight updates to the INSTRUCT model.

Quick start

The training codes are basically built on LLaMA-Factory. For evaluation, we employ the OpenCompass framework. Both are Tremendous projects, and you can find nearly everything there, thanks to their great framework and beautiful code!

Environment

The env for LLaMA-Factory is quite strict, please check the official repo for more details.

git clone https://github.com/wutaiqiang/Shadow-FT
cd Shadow-FT
pip install -e ".[torch,metrics]"
pip install importlib_metadata omegaconf
pip install torch==2.6.0 transformers==4.52.1 torchvision  deepspeed -U

Please refer to LLaMA Factory for more details.

Training data

We select 2000 samples from BAAI Infinity-Instruct and save it at data/Shadow_2k.parquet

For the custom dataset, remember to add information at data/dataset_info.json.

For Train

Set USE_LORA MODEL_DIR BASE_MODELS, and then bash run.sh

set MODEL_DIR='' to download the model from Huggingface, rather than a local file.

After that, you will get an automatically generated bash script for training, merging, and evaluating, such as:

##### Auto-generated 2025-05-22 13:54:08 #####
# Model     : Qwen2.5-14B
# LoRA mode : true
# Template  : qwen

##### Environment #####
export VLLM_WORKER_MULTIPROC_METHOD=spawn

##### Training #####
###### I  max=2000  lr=1e-5 ######
llamafactory-cli train \
  --model_name_or_path "${MODEL_ROOT}/Qwen2.5-14B-Instruct" \
  --finetuning_type lora --lora_rank 128 \
  --dataset "Shadow_2k" \
  --output_dir "${OUTPUT_ROOT}/instruct_lora" ...

##### LoRA delta‑merge #####
llamafactory-cli export \
  --base_model "${MODEL_ROOT}/Qwen2.5-14B-Instruct" \
  --lora_dir   "${OUTPUT_ROOT}/delta" \
  --output_dir "${OUTPUT_ROOT}/shadow_instruct"

##### Evaluation list #####
# ('short_name', 'model_path')

Use this bash file to start training!

For Evaluation

Please refer to OpenCompass for evaluation.

Future Plan

• Introduce evaluation scripts in this repo.

License

We use the Apache‑2.0 license. Please also comply with the licenses of any upstream models and datasets.

☕️ Citation

If you find this repository helpful, please consider citing our paper:

@article{wu2025shadow,
  title={Shadow-FT: Tuning Instruct via Base},
  author={Wu, Taiqiang and Yang, Runming and Li, Jiayi and Hu, Pengfei and Wong, Ngai and Yang, Yujiu},
  journal={arXiv preprint arXiv:2505.12716},
  year={2025}
}

For any questions, please pull an issue or email at [email protected]