Mixed-precision NVFP4 weights of Accio-Lab/occamy-1.0.
The routed MoE experts use NVFP4 weights and activations through NVIDIA Model Optimizer 0.44.0 NVFP4_EXPERTS_ONLY_CFG. Attention, routers, shared experts, embeddings, output head, and vision weights retain BF16. The original Occamy tokenizer and chat template are preserved. This checkpoint does not include a donor MTP head.
Calibration
128 records from the train split of CNN/DailyMail, configuration 3.0.0, truncated to 256 tokens. Sixteen separate test records were held out. The exact dataset revision and selection are in data-provenance.json; the quantization configuration is in quantization-recipe.json. No training or fine-tuning was performed.
For expert weights whose observers were not activated, ModelOpt used weight-derived or fused-tensor scaling during export. This limited calibration set does not establish coverage of every expert on every workload. The runtime checks below test the exported checkpoint.
Evaluation
| Check | BF16 | NVFP4 |
|---|---|---|
| Text requests ending normally | 8/8 | 8/8 |
| Strict JSON fixtures | 3/4 | 3/4 |
| Tool-call JSON fixtures | 2/2 | 2/2 |
| Structured tool calls through the API | 2/2 | 2/2 |
| Local add-tool execution and result follow-up | True | True |
| Generated code with independent tests | 2/2 | 2/2 |
| Held-out token NLL | 2.252837 | 2.262056 |
| Synthetic red-image recognition | pass | pass |
Full scalar results are in VALIDATION.json. These are small regression checks, not general benchmark results. The add tool was executed locally and its result returned to the model; the weather case validates the structured call only. Code fences were removed for code execution; JSON checks require raw JSON. No broad vision, audio, video, long-context, or throughput claim is made. The validation context length was 2048.
Tested serving configuration
SGLang 0.5.13.post1, FlashInfer 0.6.12, PyTorch 2.11.0+cu130. Native NVFP4 execution requires compatible hardware and kernels.
python -m sglang.launch_server \
--model-path Accio-Lab/occamy-1.0-NVFP4 \
--quantization modelopt_fp4 --dtype bfloat16 \
--moe-runner-backend flashinfer_cutlass \
--attention-backend triton --mamba-backend triton \
--mm-attention-backend sdpa \
--tool-call-parser qwen3_coder --reasoning-parser qwen3 \
--context-length 2048 --max-total-tokens 4096 \
--max-running-requests 1 --max-mamba-cache-size 4 \
--disable-cuda-graph --disable-radix-cache --disable-overlap-schedule
The tests used greedy decoding and chat_template_kwargs={"enable_thinking": false}.
vLLM compatibility
On a single H200, this checkpoint completed all 9 requests and passed all 9 semantic checks, covering text, code, strict JSON, tool calls and result follow-up, and images. The BF16 reference also passed 9/9. There were no request errors or truncated responses. This used the Marlin W4A16 path, not native FP4 compute. MTP was disabled. This is a small compatibility check, not a broad quality or speed benchmark.
Attribution and license
Original model: Accio-Lab, based on Qwen3.6-35B-A3B. Original Apache-2.0 license is included.
See Technical details for reproduction settings.
Occamy checkpoints
BF16 · GGUF · FP8 · NVFP4 · MLX 8-bit · MLX 6-bit · MLX 4-bit · MLX 3-bit · MTP head
Compare file sizes, validation scope and deployment commands in the checkpoint explorer.