Swift-Qwen3.8-27B-NVFP4
NVFP4/FP8 mixed-precision quantization of Swift-Qwen3.8-27B, UkisAI's reasoning-efficient derivative of Qwen3.8-27B. This update was created directly from the original BF16 source weights using NVIDIA Model Optimizer, following the verified precision mapping of NVIDIA's Qwen3.8-27B-NVFP4. The model architecture, vision components, tokenizer, and BF16 MTP weights are preserved.
Weight-memory fix
The previous checkpoint retained the 144 large projections across 48 Gated DeltaNet
layers and lm_head in BF16. This update reduces model-weight memory, leaving more
memory for the existing KV cache. KV-cache precision is unchanged and the KV cache
was not quantized.
What is in this checkpoint
| Component | Precision |
|---|---|
192 MLP projections (gate_proj, up_proj, down_proj) and lm_head | NVFP4, block size 16 |
144 large DeltaNet projections (in_proj_qkv, in_proj_z, out_proj) | FP8 E4M3 |
64 full-attention projections (q_proj, k_proj, v_proj, o_proj) | FP8 E4M3 |
| Embeddings, norms, small DeltaNet projections/controls, convolutions, vision, MTP | BF16 |
| Format | NVIDIA ModelOpt mixed precision |
| Checkpoint size | Approximately 21.945 GB |
All 798 retained BF16 tensors, including all 15 MTP tensors, were checked byte for byte against the original source. The precision inventory lists every exported tensor.
Calibration
Source: ukisai/Swift-Qwen3.8-27b at revision 1b30aaaf753fe5c1cb51ada2ea0367a53445359c.
ModelOpt commit: 022767c7ab3d7d36211affd85e5c496770cde768 (installed version 0.47.0rc0).
Calibration used 2,048 examples from NVIDIA's seven-source Nemotron post-training v3
collection, a 4,096-token maximum, seed 42, and 6,459,330 actual tokens. The original
source model was split across both H100s in one process. Both full calibration passes
completed and all 193 NVFP4 modules had Local-Hessian coverage.
NVFP4 uses ModelOpt Local-Hessian calibration with block size 16 and the full FP8 scale sweep. FP8 projections use max calibration. Quantizers are enabled only for the explicit projection list; KV-cache quantizers remain disabled.
See the exact recipe, calibration manifest, and package versions. This reproduces NVIDIA's published approach with documented representative data; NVIDIA's exact calibration sample IDs and sampling parameters are not public.
Requirements and hardware validation
Validated with vLLM 0.28.0, using a loader that supports ModelOpt mixed-precision
checkpoints. This update changes the checkpoint serialization from the previous
compressed-tensors format to ModelOpt.
H100 inference uses Marlin weight-only NVFP4 kernels. Native Blackwell W4A4 execution has not been tested for this new checkpoint. The reported memory and quality results are H100 measurements. FP4/FP8 activation quantization was also exercised separately through ModelOpt fake-quantized likelihood evaluation.
Evaluation scope
These results compare the previous NVFP4 checkpoint with this release on H100.
Benchmarks
Fresh paired tests used both H100s, with one H100 per inference measurement and identical
settings for old/new models. Full budget means gpu_memory_utilization=0.90; constrained
budget means 0.40. Both use BF16 model dtype and kv_cache_dtype=auto.
The 3.17x constrained-context increase comes from lower weight memory. Cache-entry specifications are identical. Full-budget tests reach the unchanged 262,144-token architecture ceiling. Long-context checks retrieved three independent values placed at 10%, 50%, and 90% of the prompt and completed a real 256-token generation at the capacity boundary. Strict loading, numeric precision checks, MTP and synthetic image tests passed. These are sanity checks, not a full production benchmark suite or a claim of improved inference speed.
See validation details. The previous checkpoint and its original Blackwell benchmark remain available at their pinned revision; those Blackwell results describe the old weights.
How to use
vLLM
vllm serve ukisai/Swift-Qwen3.8-27B-NVFP4 \
--tensor-parallel-size 1 \
--dtype bfloat16 \
--kv-cache-dtype auto \
--max-model-len 262144 \
--reasoning-parser qwen3 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--port 8000
Adjust tensor parallelism and context length to your GPU memory. See the base model's vLLM recipe for installation and hardware-specific settings.
Optional MTP decoding
The checkpoint includes the base model's MTP head in BF16. To enable self-speculative decoding, append to the server command above:
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'
UkisAI API
Swift is also served through an OpenAI-compatible API at https://ukisai.com/api/swift/v1.
It is free for research purposes and needs no API key. The model id is swift.
License and access
Swift weights are distributed through gated access under the Swift Open License v1.0. Personal, research, educational, evaluation, and commercial use are free for individuals and organizations with annual recurring revenue, including affiliates, of up to US$1,000,000. Above that threshold, commercial use requires a separate Swift Enterprise License. Contact UkisAI for terms.
Citation
@misc{swift-qwen3.8-27b,
title = {Swift-Qwen3.8-27B},
author = {UkisAI},
year = {2026},
url = {https://huggingface.co/ukisai/Swift-Qwen3.8-27b}
}
Acknowledgements
We acknowledge the NVIDIA Innovation Lab for providing access to 8× NVIDIA H100 GPUs to train Swift.