KTransformers

DeepSeek V4 Flash Docker Quick Start

KTransformers provides a prebuilt Docker image for DeepSeek V4 Flash. You can start the service without cloning the repository, compiling from source, or replacing dependency versions manually.

Prerequisites

  • DeepSeek V4 Flash model files are downloaded locally.
  • NVIDIA driver, Docker, and NVIDIA Container Toolkit are installed.
  • At least 200 GB of system memory.
  • A CPU with AVX2 or AVX-512 support.

Pull the Docker Image

Pull the image with the DSV4-specific tag from approachingai/ktransformers:

sudo docker pull approachingai/ktransformers:DSV4-specific

Start the Service

Enter the model directory after downloading the model:

cd /path/to/DeepSeek-V4-Flash-0731

Run the service:

sudo docker run --gpus all \
  --ipc host \
  --cap-add SYS_NICE \
  -p 30000:30000 \
  -v "$PWD":/model:ro \
  approachingai/ktransformers:DSV4-specific

The service starts at http://localhost:30000 and exposes an OpenAI-compatible API.

Verify the API

After the service logs show that it is ready, verify the endpoint:

curl http://localhost:30000/v1/models

More Tuning

For more tuning options, including layerwise prefill and source builds, see the KTransformers tutorial.