KTransformers

First LoRA SFT Run

KTransformers fine-tuning is driven through LLaMA-Factory. Install LLaMA-Factory first, then install the KT SFT extra from its KT requirements file:

cd /path/to/LLaMA-Factory
pip install -e .
pip install -r requirements/ktransformers.txt

The KT requirements file should contain:

ktransformers[sft]

Run an Example

Use the LLaMA-Factory KT examples:

CUDA_VISIBLE_DEVICES=0,1,2,3 accelerate launch \
  --config_file examples/ktransformers/accelerate/fsdp2_kt_int8.yaml \
  src/train.py \
  examples/ktransformers/train_lora/qwen3_5moe_lora_sft_kt.yaml

The current public KT SFT backends are:

BackendMeaning
AMXBF16BF16 expert backend.
AMXINT8INT8 expert backend with prepared KT weights.
AMXINT4INT4 expert backend with prepared KT weights.

SkipLoRA variants exist for advanced experiments but are not the default quick-start path.

Next Steps