REAM-288 compact model — packed NVFP4 routed experts
This is the production-candidate packed checkpoint derived from
sjakek/slimder-qwen38-ream288-depth32-agentic-ngram50-compact.
What is compressed
- All 32 layers' fused routed-expert
gate_up_projanddown_projtensors are stored as packed NVFP4 E2M1 values. - Each block of 16 weights has an FP8 E4M3 scale and each expert projection has an FP32 global scale.
- The compact PLE n-gram table remains BF16 and is byte-identical to the validated base checkpoint.
- All non-routed-expert tensors remain at their source precision.
Logical tensor payload is 85,394,969,008 bytes, down from 150,511,416,232 bytes: a 43.26% whole-checkpoint reduction (1.7625× compression). The packed routed-expert payload is 25,480,617,984 bytes, down from 90,596,966,400 bytes.
The frozen n-gram selection manifest SHA-256 is
24a0cf8360699362b6a99da498a4138b6a390918bf9cfc9df5b15bcdc3397983.
Validation
- Structure and hashes validated for all 976 tensors.
- Thirty routed-expert slices sampled across layers 0, 7, 15, 23, and 31: minimum cosine similarity 0.995475 and maximum relative mean absolute error 0.090817 after dequantization.
- Two-GPU full load, three finite forward passes, and deterministic generation passed without meta tensors or disk offload.
- Fresh two-step LoRA smoke passed directly against this packed checkpoint: train losses 5.06963 and 4.97441, eval loss 5.05745, finite nonzero gradients, and 5,079,040 trainable adapter parameters.
Runtime requirement
This checkpoint includes transformers_compact_runtime.py. Install its patch before loading:
import importlib.util
from pathlib import Path
checkpoint = Path("/path/to/checkpoint")
spec = importlib.util.spec_from_file_location(
"ream_compact_runtime", checkpoint / "transformers_compact_runtime.py"
)
runtime = importlib.util.module_from_spec(spec)
spec.loader.exec_module(runtime)
runtime.install()
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
checkpoint, dtype="bfloat16", device_map="auto", low_cpu_mem_usage=True
)
The bundled compatibility path dequantizes only the routed expert slices selected for a token and uses the stock eager PyTorch expert computation. This proves the checkpoint is genuinely packed and trainable, but it is not yet a fused NVFP4 serving kernel. A serving engine needs explicit support for Qwen4Exp fused experts plus the compact PLE runtime to realize NVFP4 inference throughput.
Provenance files
nvfp4_packing_manifest.json: exact sizes, shard hashes, and format identifier.hf_quant_config.json: quantized and excluded module classes.PACKED_SHA256SUMS: full top-level file hashes.- The validation and QLoRA receipts are published alongside the model as small JSON artifacts.
Qwen3.8 Perian project lineage
This repository is retained in the Qwen3.8 Perian checkpoints collection. Its exact position in the lineage is: Experimental quantized checkpoint after depth, expert-width, and PLE n-gram reductions, before the final rank-32 QLoRA.
The final Qwen3.8 Perian GGUF release combines three reductions and one post-training stage:
- depth: 48 to 32 transformer layers;
- routed-expert width: 384 to 288 experts per layer;
- PLE n-gram capacity: 320,001,446 to 160,000,768 rows (50%, about 25.60B parameters removed), using activation-aware bigram and frequency-ranked trigram selections validated on a document-disjoint 5M-token holdout;
- rank-32 QLoRA on 12,558 normalized traces spanning math/STEM reasoning, coding/debugging, agentic tool use, retrieval, and general multi-step reasoning. The trace mixture draws from several frontier-model families, including Fable 5, GLM 5.2, Kimi K3, Claude Opus 4.7, Qwen3.8-Max, and GPT-5.6-Sol. The final merged milestone was trained through 9,336,692 supervised assistant tokens.
Earlier checkpoints in this collection do not inherit later stages merely by being listed beside them; the stage statement above is authoritative for this artifact.