# Deploying DeepSeek V4 Flash 0731 with vLLM: Verified GPU Pods, Quants, and Serving Recipes

DeepSeek V4 Flash `0731` is the current official Flash release:

```text
deepseek-ai/DeepSeek-V4-Flash-0731
```

It keeps the same underlying architecture as the earlier V4 Flash preview, but DeepSeek re-trained the model for much stronger agentic performance. The published results are a big jump: **82.7 on Terminal Bench 2.1 versus 61.8 for the preview**, and **54.4 on DeepSWE versus 7.3**.

The model is MIT-licensed and ungated.

Under the hood, V4 Flash is a **284B-parameter Mixture-of-Experts model** with roughly **13B parameters active per token**. It supports a native context window of **1,048,576 tokens** and uses a hybrid attention design combining Compressed Sparse Attention (CSA) with Heavily Compressed Attention (HCA).

The checkpoint is already aggressively compressed. Routed experts are stored in **MXFP4 with FP8 scales**, while linear, attention, and router layers use FP8. At serving time, the KV cache can also be stored in FP8.

DeepSeek reports that V4 needs roughly **27% of V3.2's inference FLOPs per token** and about **10% of its KV-cache footprint at 1M context**.

If Hugging Face shows roughly 304–305B parameters for the repository, that does not contradict the 284B model size. The fused `0731` repository also contains the DSpark speculative-decoding module.

The checkpoint is around **167 GB on disk**, compared with roughly 160 GB for the preview. Do not use those figures as direct VRAM requirements. vLLM's MI325X validation, for example, measured about **148.66 GiB of resident model memory** for the preview checkpoint.

That difference is partly units: Hugging Face reports decimal GB, while GPU memory is normally discussed in GiB. More importantly, loading the weights is only part of the memory budget. KV cache, runtime buffers, kernels, and concurrency all need room too.

## Which checkpoint should you use?

For most users, start with:

```text
deepseek-ai/DeepSeek-V4-Flash-0731
```

The other variants are useful when a particular GPU or runtime path requires them:

| Checkpoint | What it gives you |
| --- | --- |
| `deepseek-ai/DeepSeek-V4-Flash-0731` | Current official release with DSpark |
| `deepseek-ai/DeepSeek-V4-Flash` | Older preview with MTP |
| `deepseek-ai/DeepSeek-V4-Flash-DSpark` | Preview weights with DSpark |
| `nvidia/DeepSeek-V4-Flash-NVFP4` | NVIDIA NVFP4 conversion of the preview |
| `MJPansa/DeepSeek-V4-Flash-0731-NVFP4` | Community NVFP4 conversion of `0731` |
| `yiminyuan/DeepSeek-V4-Flash-0731-INT4-W4A16` | Community INT4 transcode |
| Unsloth `0731` GGUF variants | llama.cpp / CPU+GPU setups |

One detail matters if you are choosing NVIDIA's NVFP4 checkpoint: **it is based on the preview weights, not** `0731`. You get the NVIDIA NVFP4 execution path, but not the newer post-training that produced the large agentic gains above.

The community W4A16 conversion is also not proof that stock `0731` runs cleanly on A100. Its published end-to-end validation used Radeon Pro W6800X Duo hardware with CPU offload.

## Pin vLLM before you start

The minimum runtime depends on the path you are using:

| What you need | vLLM version |
| --- | --- |
| DeepSeek V4 model support | `0.20.0+` |
| DSpark on NVIDIA | `0.25.0+` |
| DSpark on ROCm | `0.26.0+` |

For the documented NVIDIA `0731` setup:

```bash
docker pull vllm/vllm-openai:v0.25.0
```

Once you have a working deployment, keep the version pinned. A model this dependent on specialized kernels is not a good place to discover that `latest` changed underneath you.

## Which GPUs can you actually use?

The current DeepSeek and vLLM recipes give you these reproducible paths:

| Hardware | Published deployment |
| --- | --- |
| **4× GB300** | DeepSeek's own `0731` command |
| **4× B300** | vLLM Blackwell path |
| **4× B200** | vLLM Blackwell path |
| **4× H200** | vLLM Hopper path |
| **8× H200** | Prefill/decode deployment |
| **8× RTX PRO 6000 96GB** | Explicit `0731` validation |
| **1× MI325X 256GB** | Validated single-GPU, 4K context |
| **4× MI355X 288GB** | Validated ROCm setup |
| **2× DGX Spark / GB10** | Specialized two-node recipe |
| **DGX Station GB300** | Single-GPU profile |

For ordinary multi-GPU serving on H200, B200, and B300, vLLM's recommended pattern is:

```text
Data Parallel = 4
Expert Parallel = enabled
```

H100, A100, 2× H200, 2× B200, 1× B200, and MI300X are not included here because the current recipe does not provide the same level of end-to-end deployment evidence.

## 4× GB300: the cleanest `0731` reference

DeepSeek publishes this command for a single 4× GB300 node:

```bash
vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --data-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend deep_gemm_mega_moe \
  --attention-config '{"use_fp4_indexer_cache": true}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'
```

This is specifically a **Blackwell** recipe.

These two options:

```bash
--moe-backend deep_gemm_mega_moe
--attention-config '{"use_fp4_indexer_cache": true}'
```

should not be copied onto H200.

You may also see the FP4 indexer setting written upstream as:

```bash
--attention_config.use_fp4_indexer_cache True
```

Use the syntax supported by the vLLM build you pinned.

## B200 and B300

For 4× B200 or 4× B300, a useful starting point is:

```bash
vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --data-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend deep_gemm_mega_moe \
  --attention-config '{"use_fp4_indexer_cache": true}' \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice
```

To add DSpark:

```bash
--speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'
```

There is a small upstream disagreement here: DeepSeek's model card uses `greedy`, while the current vLLM recipe generator uses `probabilistic`.

If speculative decoding matters to your workload, benchmark both rather than treating either value as universally best.

## H200 uses a different path

H200 is Hopper, so the Blackwell FP4 indexer and `deep_gemm_mega_moe` overrides are intentionally absent.

Start with:

```bash
vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --data-parallel-size 4 \
  --enable-expert-parallel \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --max-model-len auto
```

If you have a full **8× H200 node**, vLLM also documents a more advanced layout with four GPUs handling prefill and four handling decode. KV state is transferred using Mooncake or NIXL, with `vllm-router` in front.

Those published prefill/decode examples currently use the **preview checkpoint**, not `0731`, so keep that distinction if you are reproducing them exactly.

## RTX PRO 6000 Blackwell needs two extra setup steps

vLLM has an explicit `0731` validation for:

```text
8× RTX PRO 6000 Blackwell
TP8
Expert Parallel
```

using:

```text
vllm/vllm-openai:v0.25.0
```

Before starting the model, install the required FlashInfer build:

```bash
/usr/bin/python3.12 -m pip install \
  --no-deps \
  flashinfer-python==0.6.14

export FLASHINFER_DISABLE_VERSION_CHECK=1
```

The `--no-deps` is important. Installing FlashInfer normally can replace PyTorch and break vLLM's compiled extensions.

Then run:

```bash
vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --block-size 256 \
  --enable-expert-parallel \
  --tensor-parallel-size 8 \
  --tokenizer-mode deepseek_v4 \
  --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_v4
```

On this particular profile, **do not enable MTP or DSpark**. Both currently fail during warmup on the `sm_120` sparse-MLA path.

## AMD: one MI325X can run it, with limits

The single-GPU MI325X validation is one of the more useful upstream tests because vLLM publishes actual memory observations rather than just saying the model fits.

The validated setup is:

```text
1× MI325X 256GB
TP1
4K context
10 GB explicit KV cache
```

Run:

```bash
export VLLM_ROCM_USE_AITER=1

vllm serve deepseek-ai/DeepSeek-V4-Flash \
  --host 0.0.0.0 \
  --port 8002 \
  --tensor-parallel-size 1 \
  --kv-cache-dtype fp8_e4m3 \
  --max-model-len 4096 \
  --enable-chunked-prefill \
  --max-num-batched-tokens 256 \
  --kv-cache-memory-bytes 10000000000 \
  --distributed-executor-backend mp \
  --trust-remote-code \
  --tokenizer-mode deepseek_v4 \
  --moe-backend triton_unfused \
  --enforce-eager
```

vLLM measured about **148.66 GiB of model memory after loading** and reports successful chat, non-thinking, and Think High tests over a 24-hour allocation.

This is a useful single-GPU option if 4K context is enough. It should not be interpreted as a one-GPU million-token deployment.

The published AMD validation uses the preview checkpoint. If you want `0731` with DSpark on ROCm, use **vLLM 0.26.0 or newer**.

## 4× MI355X

The other documented AMD path uses TP4 across four MI355X GPUs:

```bash
export VLLM_ROCM_USE_AITER=1

vllm serve deepseek-ai/DeepSeek-V4-Flash \
  --host localhost \
  --port 8001 \
  --dtype auto \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 4 \
  --max-num-seqs 512 \
  --max-num-batched-tokens 8192 \
  --distributed-executor-backend mp \
  --trust-remote-code \
  --gpu-memory-utilization 0.9 \
  --tokenizer-mode deepseek_v4 \
  --reasoning-parser deepseek_v4 \
  --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice \
  --compilation-config '{"mode":3,"cudagraph_mode":"FULL_DECODE_ONLY"}'
```

The published validation also includes 8-shot GSM8K results:

```text
Flexible-extract exact match: 0.9439
Strict-match exact match:     0.9431
```

TP8 is available when lower latency matters more than reproducing the published TP4 setup.

## DGX Spark is a special case

A single DGX Spark GB10 has 128 GB of unified memory, so the documented path uses **two machines at TP2**, one GB10 per node.

It also uses a specialized image:

```text
eugr/spark-vllm-b12x:latest
```

rather than the standard vLLM container, plus ConnectX/RoCE network configuration between the nodes.

If you are using DGX Spark, follow that recipe as its own deployment path. It is not equivalent to a normal two-GPU Blackwell pod.

vLLM also carries a single-GPU **DGX Station GB300** profile. The currently published command uses the preview checkpoint with MTP rather than the `0731` DSpark release.

## Don't size every server for 1M context

DeepSeek V4 Flash supports:

```text
1,048,576 tokens
```

but that does not mean every deployment should reserve enough memory to serve a million-token request.

The MI325X profile makes the distinction obvious: the model supports 1M context, while that particular validated server configuration supports 4K.

Several vLLM recipes therefore use:

```bash
--max-model-len auto
```

instead of forcing the model's architectural maximum.

V4 also has a hybrid CSA+HCA cache, so generic dense-transformer calculations such as "X GB per 100K tokens" are not reliable across every configuration.

Use the memory profile vLLM prints at startup. If you need to reserve a fixed cache budget, use:

```bash
--kv-cache-memory-bytes
```

Most published configurations use:

```bash
--kv-cache-dtype fp8
```

or, on ROCm:

```bash
--kv-cache-dtype fp8_e4m3
```

to reduce cache memory.

If you are using `--data-parallel-size 4`, remember that scheduling settings such as `--max-num-seqs` and `--max-num-batched-tokens` apply **per data-parallel rank**, not once for the entire four-GPU server.

## Reasoning and tool calling

DeepSeek V4 Flash supports three reasoning levels:

```text
low
high
max
```

DeepSeek's recommended sampling depends on what you are doing:

| Workload | Temperature | Top-p |
| --- | --- | --- |
| Normal chat / reasoning | 1.0 | 1.0 |
| Agentic work | 1.0 | 0.95 |

If you are serving Think High or Think Max, leave room for very long outputs. DeepSeek recommends allowing up to **384K output tokens** for the deeper reasoning modes, and the vLLM recipe notes that Think Max needs:

```text
--max-model-len >= 393216
```

to avoid truncating the reasoning budget.

For the OpenAI-compatible vLLM endpoint, start the server with:

```bash
--tokenizer-mode deepseek_v4 \
--reasoning-parser deepseek_v4 \
--tool-call-parser deepseek_v4 \
--enable-auto-tool-choice
```

A normal request looks like:

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepseek-ai/DeepSeek-V4-Flash-0731",
    "messages": [
      {
        "role": "user",
        "content": "Explain optimistic and pessimistic locking."
      }
    ],
    "temperature": 1.0,
    "top_p": 1.0,
    "max_tokens": 2048
  }'
```

For an agentic request with Think High:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://YOUR_SERVER_IP:8000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V4-Flash-0731",
    messages=[
        {
            "role": "user",
            "content": "Find the race condition in this worker design."
        }
    ],
    temperature=1.0,
    top_p=0.95,
    extra_body={
        "chat_template_kwargs": {
            "thinking": True,
            "reasoning_effort": "high"
        }
    },
)

print(response.choices[0].message)
```

## Keep the model on persistent storage

Current Hugging Face tooling uses the `hf` CLI:

```bash
export HF_HOME=/workspace/huggingface
mkdir -p "$HF_HOME"

hf download deepseek-ai/DeepSeek-V4-Flash-0731
```

At roughly 167 GB, repeatedly downloading the checkpoint every time a pod is recreated gets expensive in both time and bandwidth. Persistent model storage is worth setting up from the beginning.

## Which DeepSeek V4 Flash setup should you choose?

If you are choosing GPUs specifically for `DeepSeek-V4-Flash-0731`, the current upstream paths are fairly clear.

**4× GB300** is the cleanest reference because DeepSeek publishes the exact `0731` command.

**4× B200 or B300** gives you the equivalent Blackwell DP4 + expert-parallel route.

**4× H200** is the documented Hopper setup. Do not copy the Blackwell FP4-specific flags into it.

If you have **8× H200**, vLLM also has a more advanced prefill/decode design, although the published commands currently use the preview checkpoint.

**8× RTX PRO 6000 Blackwell** is explicitly validated for `0731`, but MTP and DSpark need to stay off in the current profile.

On AMD, **1× MI325X** is a real validated option if a 4K context limit is acceptable, while **4× MI355X** gives you the stronger multi-GPU ROCm path.

H100, A100, 2× H200, 2× B200, 1× B200, and MI300X are intentionally not recommended here. The current upstream material does not provide equivalent deployment recipes for them.

With V4 Flash, the useful question is not simply whether the checkpoint can fit in aggregate memory. You need enough headroom for the actual runtime, a supported kernel path for the GPU architecture, enough KV cache for the context you plan to serve, and a configuration that stays up under real concurrent traffic.

## Sources

### DeepSeek

*   **DeepSeek V4 Flash 0731 model card**  
    https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
    
*   **DeepSeek V4 Flash preview — architecture and precision**  
    https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash
    
*   **DeepSeek V4 technical report**  
    https://arxiv.org/abs/2606.19348
    

### vLLM

*   **DeepSeek V4 Flash deployment recipe**  
    https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4-Flash
    
*   **Raw hardware profiles and validations**  
    https://raw.githubusercontent.com/vllm-project/recipes/main/models/deepseek-ai/DeepSeek-V4-Flash.yaml
    
*   **FP8 KV-cache documentation**  
    https://docs.vllm.ai/en/latest/features/quantization/quantized\_kvcache/
    

### Alternative checkpoints

*   **NVIDIA DeepSeek V4 Flash NVFP4**  
    https://huggingface.co/nvidia/DeepSeek-V4-Flash-NVFP4
    
*   **Community** `0731` **NVFP4 conversion**  
    https://huggingface.co/MJPansa/DeepSeek-V4-Flash-0731-NVFP4
    
*   **Community** `0731` **INT4/W4A16 conversion**  
    https://huggingface.co/yiminyuan/DeepSeek-V4-Flash-0731-INT4-W4A16
    
*   **Unsloth DeepSeek V4 GGUF documentation**  
    https://unsloth.ai/docs/models/deepseek-v4