DeepSeek Models
DeepSeek is a primary KTransformers model family for both inference and SFT. Inference and fine-tuning use different methods and package paths, so their statuses are listed separately.
Inference
| Model | Entry | Method | Status |
|---|---|---|---|
| DeepSeek V4-Flash | 0.7.2 two-RTX-5090 TP2 adapter generation | Native MXFP4 | Verified for this smoke; no Prefill/Decode TPS yet. |
| DeepSeek V3.2 | kt run deepseek-v3.2 or manual tutorial | Registry uses FP8; older tutorial uses AMXINT4 | Needs reconciliation and smoke. |
| DeepSeek V3-0324 / R1-0528 | kt run deepseek-v3 / kt run deepseek-r1 | Registry default AMXINT4 | Current / Needs docs. |
Fine-Tuning
Use DeepSeek SFT. DeepSeek V4-Flash 0.7.2 completed a two-RTX-5090, 20-step LoRA train–save–load smoke with native MXFP4 routed experts. Training TPS was not measured. DeepSeek V3 public FP8 checkpoints follow separate AMXBF16, AMXINT8, or AMXINT4 preparation paths.
Source Material
Relevant GitHub sources:
doc/en/DeepSeek-V4-Flash.mddoc/en/kt-kernel/deepseek-v3.2-sglang-tutorial.mddoc/en/SFT/KTransformers-Fine-Tuning_Quick-Start.mddoc/en/SFT/KTransformers-Fine-Tuning_User-Guide.md- DeepSeek V4 Flash LoRA quickstart