FusionBench-LLM

Description

A Collection of examples for large language model fusion, based on FusionBench.

Installation

Pip

# clone project
git clone https://github.com/fusion-bench/fusion-bench-llm
cd fusion-bench-llm

# [OPTIONAL] create conda environment
conda create -n fbllm python=3.12
conda activate fbllm

# install pytorch according to instructions
# https://pytorch.org/get-started/

# 1. install fusion-bench
pip install fusion-bench
# or install fusion-bench in edit mode (recommended for development)
git clone https://github.com/tanganke/fusion_bench third_party/fusion_bench
pip install -e third_party/fusion_bench

# 2. install fusion-bench-llm
cd .. # go back to fusion-bench-llm directory
pip install -e .

How to run

fusion_bench_llm \
    # method=...
    # method.option_1=...
    # modelpool=...
    # ...