SLIMDER Qwen3.8 REAM-288 N-gram 50% — Mixed NVFP4/FP8 v2
This is the corrected mixed-precision runtime checkpoint derived from the promoted 50%-compact REAM-288 checkpoint. Expert tensors are stored in NVFP4, while the compact PLE n-gram table is stored in FP8 with per-row FP16 scales.
The v2 exporter uses one shared global scale for each fused gate/up expert tensor, matching the vLLM Marlin loader's contract. This corrects the scale mismatch present in the superseded first mixed export.
Size and structure
- Logical tensor size: 59.78 GiB
- BF16 compact source size: 140.17 GiB
- Compact PLE rows: 160,000,768
- Trainable parameters removed from the unpruned source: approximately 25.60B
- Selection manifest SHA-256:
24a0cf8360699362b6a99da498a4138b6a390918bf9cfc9df5b15bcdc3397983
Verified gates
- All 8,640 fused gate/up expert scale pairs match exactly
- Structural checkpoint validation: passed
- PLE FP8 sample cosine versus BF16: 0.999665
- Finite load, forward pass, and generation under the included vLLM runtime patch
- Generation throughput: 68.1–79.8 tokens/s across the measured prompt profiles
- Five basic quality probes: 5/5 correct
- Sealed 32-prompt mixed-versus-BF16 comparison: passed
- loss delta: +0.03287
- token KL: 0.14940
- router top-10 agreement: 0.92783
- PLE cosine: 0.999667
- catastrophic failures: 0
Runtime notes
This model uses the experimental qwen4_exp architecture, a compact PLE remap,
and FP8 PLE storage. Use the included custom model/runtime files and a vLLM
build with the required Qwen4 experimental and Marlin NVFP4 support. The
checkpoint is intended for research and recovery fine-tuning, not as a generic
drop-in Transformers model.
The recovery QLoRA adapter and its held-out validation evidence are published separately after the guarded recovery run completes.
Provenance
- Original base:
sjakek/slimder-qwen38-ream288-depth32-agentic - BF16 compact parent:
sjakek/slimder-qwen38-ream288-depth32-agentic-ngram50-compact - Selection/holdout evidence:
sjakek/slimder-qwen38-ngram-activation-refine-20260901
Limitations
The runtime and sealed corpus gates passed, but this remains an experimental pruned and quantized checkpoint. Broad task-level evaluation is still required before production use.
Qwen3.8 Perian project lineage
This repository is retained in the Qwen3.8 Perian checkpoints collection. Its exact position in the lineage is: Mixed-precision training base after depth, expert-width, and PLE n-gram reductions, before the final rank-32 QLoRA.
The final Qwen3.8 Perian GGUF release combines three reductions and one post-training stage:
- depth: 48 to 32 transformer layers;
- routed-expert width: 384 to 288 experts per layer;
- PLE n-gram capacity: 320,001,446 to 160,000,768 rows (50%, about 25.60B parameters removed), using activation-aware bigram and frequency-ranked trigram selections validated on a document-disjoint 5M-token holdout;
- rank-32 QLoRA on 12,558 normalized traces spanning math/STEM reasoning, coding/debugging, agentic tool use, retrieval, and general multi-step reasoning. The trace mixture draws from several frontier-model families, including Fable 5, GLM 5.2, Kimi K3, Claude Opus 4.7, Qwen3.8-Max, and GPT-5.6-Sol. The final merged milestone was trained through 9,336,692 supervised assistant tokens.
Earlier checkpoints in this collection do not inherit later stages merely by being listed beside them; the stage statement above is authoritative for this artifact.