Model Recipes

Image Selector

Configure your GPUStack version, GPU type, framework, and inference backends to get the exact docker pull commands.

Configuration

GPU Type
Framework Version
Inference Backend
Optional Images

Required Images

GPUStack Version
Architecture
Registry
# GPUStack Image - GPUStack core service, required for both Server and Worker nodes
docker pull --platform linux/amd64 gpustack/gpustack:v2.2.0
 
# Inference Backend Images
docker pull --platform linux/amd64 gpustack/runner:cuda13.0-sglang0.5.12.post1
docker pull --platform linux/amd64 gpustack/runner:cuda13.0-vllm0.22.1
docker pull --platform linux/amd64 gpustack/runner:cuda13.0-vllm0.21.0
docker pull --platform linux/amd64 gpustack/runner:cuda13.0-vllm0.20.2
docker pull --platform linux/amd64 gpustack/runner:cuda13.0-vllm0.19.1
docker pull --platform linux/amd64 gpustack/runner:cuda13.0-vllm0.18.1
 
# Pause Image - Provides shared network and IPC environment for model instance containers, required for Docker environment only
docker pull --platform linux/amd64 gpustack/runtime:pause
 
# Benchmark Image - Used for running model performance benchmarks
docker pull --platform linux/amd64 gpustack/benchmark-runner:v0.0.3
 
# Gateway Images - Required for Kubernetes deployment only
docker pull --platform linux/amd64 gpustack/higress-plugins:0.2.3.post5
docker pull --platform linux/amd64 gpustack/mirrored-higress-higress:2.1.9
docker pull --platform linux/amd64 gpustack/mirrored-higress-pilot:2.1.9
docker pull --platform linux/amd64 gpustack/mirrored-higress-gateway:2.1.9
docker pull --platform linux/amd64 gpustack/gpustack-operator:v0.5.1
docker pull --platform linux/amd64 gpustack/ssh-server:v1.1.0
docker pull --platform linux/amd64 gpustack/mirrored-kueue:v0.18.0
docker pull --platform linux/amd64 gpustack/mirrored-node-feature-discovery:v0.18.3
docker pull --platform linux/amd64 gpustack/mirrored-csi-nfs-driver:v4.13.0
docker pull --platform linux/amd64 gpustack/mirrored-csi-s3-driver:v0.43.7
docker pull --platform linux/amd64 gpustack/mirrored-csi-provisioner:v6.1.0
docker pull --platform linux/amd64 gpustack/mirrored-csi-resizer:v2.0.0
docker pull --platform linux/amd64 gpustack/mirrored-csi-snapshotter:v8.4.0
docker pull --platform linux/amd64 gpustack/mirrored-csi-livenessprobe:v2.17.0
docker pull --platform linux/amd64 gpustack/mirrored-csi-node-driver-registrar:v2.15.0
Need offline installation?

Suitable for completely air-gapped environments, importing images via files.

  1. Pull images on a machine with internet access (refer to commands above).
  2. Export images to tar files:
    docker save -o gpustack-server-images.tar gpustack/gpustack:v2.2.0 gpustack/higress-plugins:0.2.3.post5 gpustack/mirrored-higress-higress:2.1.9 gpustack/mirrored-higress-pilot:2.1.9 gpustack/mirrored-higress-gateway:2.1.9 gpustack/gpustack-operator:v0.5.1 gpustack/ssh-server:v1.1.0 gpustack/mirrored-kueue:v0.18.0 gpustack/mirrored-node-feature-discovery:v0.18.3 gpustack/mirrored-csi-nfs-driver:v4.13.0 gpustack/mirrored-csi-s3-driver:v0.43.7 gpustack/mirrored-csi-provisioner:v6.1.0 gpustack/mirrored-csi-resizer:v2.0.0 gpustack/mirrored-csi-snapshotter:v8.4.0 gpustack/mirrored-csi-livenessprobe:v2.17.0 gpustack/mirrored-csi-node-driver-registrar:v2.15.0

docker save -o gpustack-worker-images.tar gpustack/gpustack:v2.2.0 gpustack/runner:cuda13.0-sglang0.5.12.post1 gpustack/runner:cuda13.0-vllm0.22.1 gpustack/runner:cuda13.0-vllm0.21.0 gpustack/runner:cuda13.0-vllm0.20.2 gpustack/runner:cuda13.0-vllm0.19.1 gpustack/runner:cuda13.0-vllm0.18.1 gpustack/runtime:pause gpustack/benchmark-runner:v0.0.3
  3. Copy files to the offline machine.
  4. Import images by node role:
    Server Node:
    docker load -i gpustack-server-images.tar
    Worker Node:
    docker load -i gpustack-worker-images.tar
    Server + Worker Node:
    docker load -i gpustack-server-images.tar
docker load -i gpustack-worker-images.tar
  5. When running GPUStack Server and Worker containers, specify the container registry using the --system-default-container-registry parameter:
    sudo docker run -d --name gpustack \
    --restart unless-stopped \
    -p 80:80 \
    -p 10161:10161 \
    --volume gpustack-data:/var/lib/gpustack \
    docker.io/gpustack/gpustack:v2.2.0 \
    --system-default-container-registry docker.io
View full air-gapped docs →