Qwen3.8-27B Uncensored NVFP4 RTX 5090
RTX 5090 optimized ModelOpt NVFP4 conversion of orcarouter/Qwen3.8-27B-Uncensored, produced directly from the original BF16 safetensors.
This checkpoint preserves the Qwen3.8 vision-language architecture and native MTP tensors. It is paired with the target-matched fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090, now released in ModelOpt NVFP4.
Combined release
| Component | Disk size | GPU weight allocation |
|---|---|---|
| NVFP4 target | 20.59 GB | 18.80 GB |
| NVFP4 DSpark drafter | 1.64 GB | 1.70 GB |
| Combined pair | 22.24 GB | 20.50 GB |
The validated Hermes serving profile launches with a 122,880-token context limit. SGLang automatically allocated an 87,798-token active pool and left 2.83 GB of GPU memory free after cache allocation and CUDA graph capture on one RTX 5090.
| Context field | Tokens |
|---|---|
| Architecture maximum | 262,144 |
| Validated launch context | 122,880 |
| Automatically allocated active pool | 87,798 |
Measured performance
The current NVFP4 target and NVFP4 drafter profile reached 217.73 tokens per second median decode across 18 successful requests on one RTX 5090.
| Metric | Result |
|---|---|
| Requests | 18 of 18 successful |
| Median decode | 217.73 tokens per second |
| Median time to first token | 0.113 seconds |
| P95 time to first token | 0.149 seconds |
| Generated tokens per request | 26 to 640 |
| Total generated tokens | 3,633 |
Test conditions were concurrency 1, temperature 0, two warmups, three repeats across six prompts, FP8 E4M3 KV cache, and a fixed request seed. Decode throughput excludes prompt processing and time to first token. The 16,000-token request setting was a ceiling. This result is not a sustained 16,000-token generation measurement.
The earlier deterministic BF16 drafter profile measured 139.32 tokens per second with a 16,384-token active pool. Its full standard decoding, native MTP, public DSpark, and trained DSpark comparison remains available in benchmarks/benchmark-summary.json.
Checkpoint
| Field | Value |
|---|---|
| Source revision | 404ea47aaa5d8a8b00049c9e9750089aca011ab2 |
| Format | ModelOpt NVFP4 safetensors |
| Export size | 20,593,147,464 bytes |
| Architecture | Qwen3_5ForConditionalGeneration |
| Native MTP tensors | 15 |
| Recommended runtime | SGLang Qwen3.8 image |
| Recommended hardware | NVIDIA RTX 5090 32 GB |
Run with the matched NVFP4 DSpark drafter
docker run --rm --gpus all --ipc=host -p 30000:30000 \
lmsysorg/sglang:qwen38-27b \
sglang serve \
--model-path fleetml/Qwen3.8-27B-Uncensored-NVFP4-RTX5090 \
--quantization modelopt_fp4 \
--trust-remote-code \
--speculative-algorithm DSPARK \
--speculative-draft-model-path fleetml/Qwen3.8-27B-Uncensored-DSpark-RTX5090 \
--speculative-draft-model-quantization modelopt_fp4 \
--speculative-dspark-block-size 7 \
--speculative-num-steps 1 \
--speculative-eagle-topk 1 \
--context-length 122880 \
--max-mamba-cache-size 8 \
--kv-cache-dtype fp8_e4m3 \
--attention-backend flashinfer \
--chunked-prefill-size 1024 \
--mamba-radix-cache-strategy extra_buffer_lazy \
--mamba-ssm-dtype bfloat16 \
--mm-feature-transport cpu \
--cuda-graph-max-bs-decode 1 \
--mem-fraction-static 0.86 \
--max-running-requests 1 \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder \
--host 0.0.0.0 \
--port 30000
The active token pool is allocated automatically. Available capacity can vary slightly with the runtime build and GPU state.
Provenance
- Base model: Qwen/Qwen3.8-27B
- BF16 source: orcarouter/Qwen3.8-27B-Uncensored
- Quantization: NVIDIA ModelOpt
0.47.0.dev70, commitc4129b6e - Calibration: 126 tokenized UltraChat conversations
- Validation: complete shard, tensor, dtype, MTP, load, API, Hermes, and benchmark checks
Use
This checkpoint is intended for controlled local research, evaluation, and agent development. Use it responsibly and comply with the Apache 2.0 license and applicable law.