NVIDIA Serves GLM-5.3 at 607.8 Tokens a Second on Two DGX Stations

NVIDIA says two DGX Stations, each holding a single GB300 GPU, serve the 753-billion-parameter GLM-5.3 at a peak of 607.8 tokens per second across all users. That peak arrives at 16 concurrent requests, and throughput falls to 385.0 at 32, according to a developer post by NVIDIA technical marketing engineer Fatima Fall.

Every figure below is NVIDIA's own, from one workload on one setup.

The setup NVIDIA used

Each DGX Station has 252 GB of HBM3e memory, and the two are linked over ConnectX-8 RDMA. The post runs vLLM v0.30.0 with tensor parallelism of 2, so neither GPU holds a full copy of the model and every token needs both to exchange results.

The model is GLM-5.3 in NVFP4, a quantised build with 753 billion total parameters and 40 billion active per token. The checkpoint is about 464 GB, and each GPU holds roughly 218 GiB of weights. Settings include multi-token prediction with four draft tokens, a 98,304-token context limit and prefix caching.

The throughput numbers

The test used AIPerf 0.12.0 on 80 coding conversations, 89 turns in all, from the SPEED-Bench dataset, with an 8,192-token output cap.

Concurrent requests	Total tokens/s	Per-user tokens/s
1	107.4	115.3
8	386.0	53.0
16	607.8	42.4
32	385.0	13.0

1. 1 request107.4 tokens/s
2. 8 requests386 tokens/s
3. 16 requests607.8 tokens/s
4. 32 requests385 tokens/s

Source: NVIDIA developer blog, accessed 2026-11-14

At 32 requests, mean inter-token latency rose to 77.9 ms from 23.8 ms at 16, and mean latency per request reached 453.5 seconds. The post's token counts include reasoning tokens, run at the default maximum reasoning effort.

What the licence allows

Z.ai published the GLM-5.3 weights on Aug. 25, according to the noze.it analysis, under its own licence named glm-5.3. That analysis describes it as close to MIT with one condition: a model-as-a-service business with more than $10 billion in revenue over any 12 months must pass a Z.ai security review before commercial use. The Hugging Face card lists the licence name and does not include the text.

The NVIDIA post adds its own gate: running the setup requires accepting the NVIDIA Open Model Agreement and the GLM-5.3 terms, and the dataset preparation needs gated access to a Humanity's Last Exam dataset.

Caveats the post itself states

The launcher binds the API to 0.0.0.0 with no authentication, and the post advises readers to firewall the port and add authentication and TLS before wider access. It also says disabling prefix caching may change performance and does not quantify the effect.

No independent party has reproduced the 607.8 figure. The same Hugging Face card cites a 50% gain over GLM-5.2 on Z.ai's internal code benchmark, a separate vendor figure. On NVIDIA's other business dealings, see its startup investments; for another open-weights card read closely, see Qwen3.8-27B.