Leanstral-1.5-119B-A6B — NVFP4
NVFP4 (4-bit) quantization of Leanstral-1.5-119B-A6B for Blackwell-class NVIDIA
GPUs (native FP4 tensor cores). Derived from the BF16 HF conversion
sahilchachra/Leanstral-1.5-119B-A6B-BF16,
itself an unofficial community conversion of
mistralai/Leanstral-1.5-119B-A6B.
Architecture
Composed Mistral3
model = DeepSeek-V3 text backbone (MLA attention + shared-expert MoE: 128
routed experts, 4 active/token, 1 shared) + Pixtral vision encoder. (llama.cpp
also detects the text stack as deepseek2 = DeepSeek-V3.)
What was quantized
- NVFP4 (W4A4),
nvfp4-pack-quantized, group 16, FP8-E4M3 scales: all language-modelLinearlayers and all MoE experts — the fused DeepSeek-V3 experts are auto-linearized by llm-compressor and packed. 13,932 expert tensors packed (verified — experts were NOT skipped). - Kept full precision: vision tower, multimodal projector, MoE router
(
mlp.gate),lm_head, token embeddings. - Calibration:
HuggingFaceH4/ultrachat_200k, 128 samples, seq 512, text-only.
Provenance & precision
Pipeline: original FP8 consolidated → dequant to BF16 (lossless) → NVFP4. The 4-bit weights derive from an already-8-bit source, not full-precision weights.
✅ Verified / ⚠️ Not yet verified (tested 2026-07-04 on 1× B200, CUDA 13)
- ✅ Conversion is structurally exact: strict key+shape match against a real
Mistral3(deepseek_v3+pixtral)model (801 tensors). - ✅ NVFP4 produced with all experts packed (13,932 tensors); valid
compressed-tensors format; loads in
transformers. - ✅ vLLM 0.23 resolves the architecture (
Mistral3ForConditionalGeneration). - ⚠️ End-to-end generation not yet confirmed on a single B200. Two engine-side
gaps for this new composed arch (deepseek_v3 text inside Mistral3):
- vLLM 0.23 raises
KeyError: ...mlp.gate.e_score_correction_biaswhen loading weights (its DeepSeek-V3 loader in the Mistral3 wrapper expects a different key path). Needs a vLLM build that supports this composition. - transformers loads it but dequantizes NVFP4 to BF16 (no FP4 compute path) → ~159 GB + the naive MoE loop → OOM/too slow on one 183 GB B200.
- vLLM 0.23 raises
- ⚠️ Vision path is converted (Pixtral tensors +
preprocessor_config.jsonpresent) but not runtime-validated. - ⚠️ RoPE scaling is approximate (YARN mapped; source also carries
llama_4_scaling). Long-context (>8k) not guaranteed.
In short: the artifact is correct; serving it needs engine support for the composed arch that isn't in the pinned stack yet. Track vLLM support for DeepSeek-V3-backed Mistral3.
Intended usage (mirrors the base model)
Tokenizer is Tekken (tekken.json, use mistral_common); chat format is the
Mistral instruct format (handled by mistral_common, no manual [INST]). The base
model recommends temperature 1.0. Once engine support lands, the intended path:
vllm serve sahilchachra/Leanstral-1.5-119B-A6B-NVFP4 \
--tokenizer-mode mistral --reasoning-parser mistral \
--tool-call-parser mistral --enable-auto-tool-choice
Files included: config.json, weights + index, tekken.json, params.json,
chat_template.jinja, preprocessor_config.json.
License
Derives from mistralai/Leanstral-1.5-119B-A6B; upstream license/terms apply.
Community artifact, not affiliated with Mistral AI.