GLM-5.3-NVFP4

Model Overview

• Model Architecture: GlmMoeDsaForCausalLM
  • Input: Text
  • Output: Text
• Model Optimizations:
  • Weight quantization: FP4
  • Activation quantization: FP4
• Format: compressed-tensors
• Base model: zai-org/GLM-5.3

This is an NVFP4 checkpoint derived from zai-org/GLM-5.3. It is optimized for Blackwell inference with NVFP4 weights and activations while retaining higher precision where needed for model quality.

Model Optimizations

This model uses NVFP4 FP4 (E2M1) weights and activations and two-level scaling. Layers outside of routed experts are left unquantized. The checkpoint uses LLM Compressor's compressed-tensors NVFP4 format.

Deployment

vLLM Serving

vllm serve RedHatAI/GLM-5.3-NVFP4 \
  --served-model-name GLM-5.3-NVFP4 \
  --tensor-parallel-size 4 \
  --kv-cache-dtype fp8_e4m3 \
  --reasoning-parser glm45 \
  --chat-template-content-format string \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --trust-remote-code

For optimal peformance, consider using the DSpark draft model RedHatAI/GLM-5.3-speculator.dspark for speculative decoding, shown below.

vllm serve RedHatAI/GLM-5.3-NVFP4 \
  --served-model-name GLM-5.3-NVFP4 \
  --tensor-parallel-size 4 \
  --kv-cache-dtype fp8_e4m3 \
  --reasoning-parser glm45 \
  --chat-template-content-format string \
  --speculative-config '{"model":"RedHatAI/GLM-5.3-speculator.dspark","num_speculative_tokens":8,"method":"dspark"}' \
  --trust-remote-code

Creation

The source model was compressed using an LLM-Compressor with experimental layerwise decompression/compression along with an NVFP4 scheme using perfectblend calibration samples.

Evaluation

This model was evaluated on GSM8K Platinum, IFEval, MATH-500, AIME 2025, and GPQA Diamond using lighteval. Evaluation was run on a vLLM server hosted on 4xB200. Generation used temperature 1.0, the glm45 reasoning parser, FP8 E4M3 KV cache, and the same serving stack for the NVFP4 and local FP8 baseline. GSM8K, IFEval, MATH-500, and GPQA used three seeds; AIME used eight seeds. Recovery is relative to the base model run in the same evaluation harness.

Accuracy

Performance Evaluation

Performance evaluation for RedHatAI/GLM-5.3-NVFP4 with speculative decoding.

In the graph, interactivity is the median per-request output-token rate, aggregate throughput is total completed output tokens divided by measurement duration, and each label is completed requests divided by measurement duration. The math_reasoning dataset was used for this benchmarking.