MiMo-V2.6-Flash-RL-JANG_2L
Calibrated mixed 2-bit-class JANG · approximately 2.73 effective bits/parameter in the main shards · 95.87 GiB text weights · approximately 103.19 GiB complete download.
Iteration 2 of XiaomiMiMo/MiMo-V2.6-Flash-RL, calibrated with imatrix + AWQ + per-expert GPTQ on 258,507 tokens. It protects ten early gates in source-native MXFP4 and 22 later gates at 3-bit. The native tokenizer/chat template and image, video and audio-input weights are preserved.
Runtime: use MLX Studio / vMLX 1.6.65 or newer with JANG 2.5.48 or newer, which include the MiMo-V2.6 mixed affine/MXFP4 and multimodal runtime. Older MiMo-V2.5 loaders and generic uniform-bit loaders are incompatible. The bounded Python results and quality limitations below remain separate from broad benchmark parity.
JANGQ-AI · JANG source · Source model
Measured quality
KL is KL(source || quantized) in nats, against the exact source checkpoint decoded in its original FP8/MXFP4 formats. Lower is better. Top-1 agreement measures matching next-token predictions, not task accuracy. Calibration, validation and the final sealed set are separate. Results on different sets must not be compared as if paired.
| Validation metric | Previous GPTQ allocation | This JANG_2L |
|---|---|---|
| Mean KL | 0.942919 | 0.914656 |
| Median KL | 0.061255 | 0.055000 |
| 90th percentile | 2.703788 | 2.616918 |
| 99th percentile | 13.062842 | 12.778649 |
| Top-1 agreement | 76.5213% | 77.0381% |
| Positions | 32,506 | 32,506 |
| Final sealed metric | Previous GPTQ allocation | This JANG_2L |
|---|---|---|
| Mean KL | 0.905232 | 0.866514 |
| Median KL | 0.072926 | 0.063002 |
| 90th percentile | 2.584622 | 2.413376 |
| 99th percentile | 12.561462 | 12.266034 |
| Worst 1% mean | 15.539492 | 15.194221 |
| Maximum KL | 26.005333 | 29.807011 |
| Top-1 agreement | 75.7578% | 76.6198% |
| Positions | 32,365 | 32,365 |
The final sealed set was scored once after selecting this recipe; it was not used to tune allocation. Machine-readable reports are in evaluation/. The mean remains much higher than the median, reflecting substantial distortion on sensitive positions. This is an aggressive memory-budget quant, not a claim of source benchmark parity or globally optimal quality. No matched whole-model stock-MLX RTN score was measured.
A small frozen public subset using native thinking On, temperature 1 and top-p 0.95 passed 5/6 BFCL tool tasks, versus 4/6 previously. Both allocations passed 0/3 adapted Aider/Exercism JavaScript tasks under the same bounded budgets; reasoning consumed the output budget on several attempts. A separate Off branch preserved the failed trajectory and repaired linked-list tests 20/23 → 23/23; grade-school remained 9/10 and REST 6/9. These are subset diagnostics, not official leaderboard scores or a replacement for the On result.
The source's GRPO, groupwise grading, self-correction and multi-prefix distillation describe training methods. They are not inference switches or online learning from SSD cache hits. The observed correction example does not establish that every source RL capability survived quantization.
Precision and layout
| Component | Stored format |
|---|---|
| 109 routed expert projections | 2-bit affine, group 128 |
| 22 protected later gates | 3-bit affine, group 128 |
| Ten protected early gates | Source-native MXFP4, group 32 |
| Attention, embedding, output head, dense and MTP matrices | 8-bit affine, group 64 |
| Routers | FP32 |
| Vision/audio components | Native BF16 where supplied |
| Affine scales and biases | BF16 |
Calibration provenance is recorded in jang_config.json under quantization.awq, quantization.imatrix and quantization.gptq, each with applied: true. AWQ transformations are already folded into the saved weights; loaders must not apply them a second time.
Honor every per-module quantization override. The top-level 8-bit default is not the expert precision. JANG_2L describes a mixed-precision tier; effective bits include scales, biases and protected components.
The root contains 24 naturally aligned model shards, one index and 1,477 tensors. The untied head is bare lm_head.weight, measured from packed bytes as q8/g64 affine. All 72 MTP tensors stay in the main shards. Audio-tokenizer and DFlash sidecars remain in subdirectories. All tensor files, including sidecars, pass dtype-alignment checks. Main-shard tensor payload is 98.69 GiB; loaded text parameters are 95.868648 GiB. Activations, media towers and active KV need additional memory; tests used an M5 Max with 128 GiB unified memory.
Reasoning, tools and generation
- Reasoning defaults On. Disable with
chat_template_kwargs={"enable_thinking": false}; the native template adds a closed empty thinking block. - Use reasoning parser
think_xmland tool parserxml_function, with automatic tool choice enabled. Preservereasoning_content, tool-call IDs, parsed JSON arguments and complete tool-result batches in history. - Low/high reasoning effort do not select distinct native levels.
reasoning_effort="none"selects Off. Output-token caps include reasoning. generation_config.jsonuses source-recommended temperature 1.0, top-p 0.95, do_sample=true. EOS IDs: 151643, 151645, 151672. Tokenizer:Qwen2Tokenizer; embedded and standalone native templates agree.- Tool syntax is
<tool_call><function=NAME><parameter=ARG>VALUE</parameter></function></tool_call>. Partial or malformed generated arguments must not be executed as complete calls.
Multimodal and SSD cache limits
Python vMLX exercises image, video and audio-input preprocessing, reasoning/tool parsing, immediate tool continuations, multimodal cache reuse and process restart. Functional quality is mixed: a horizontal-motion video is answered incorrectly as downward in Off mode, including cache bypass; On answers right in the bounded control. Some combined image/audio prompts omit or misidentify requested details. These are model behavior failures, not evidence that a cache hit fixes perception. Audio output and synchronized video/audio understanding are not qualified. MTP and DFlash execution remain disabled.
The qualified SSD format stores native BF16 KV losslessly as U16 payloads with typed metadata, not FP32 expansion: nine full-attention slots plus 39 rotating sliding-window slots. Exact reconstruction/logits were verified for the native format; optional q4/q8 cache experiments were not selected. Active request KV and OS page cache still consume RAM. Reusable application cache tiers retain zero bytes under the tested SSD-only policy.
Immediate tool-result continuations wait for the server's terminal SSD persistence barrier; no client sleeps were needed. The paired candidate public run persisted all 52 responses and restored all 31 tool continuations. Stored coverage is the prompt checkpoint through N−1; later assistant/tool suffixes are prefetched on the next request.
At the stock GPU limit, a 32,036-token prompt took 54.916 s cold, 1.045 s on SSD reuse (32,035 tokens restored), and 55.846 s bypassed, with matching answers. Peak allocation was about 104.47 GiB with media towers loaded. A short visible UI correction measured about 46.7–46.8 tokens/s; neither timing is a steady-state benchmark. The tested runtime window is 32K. The source declares 1M context; 1M is not qualified for this bundle. Set prompt/window and SSD capacity controls to fit the machine.
한국어 안내
258,507개 토큰으로 imatrix/AWQ 보정과 전문가별 GPTQ를 적용한 JANG_2L 혼합 정밀도 모델입니다. 대부분의 전문가 행렬은 2비트이며, 초기 게이트 10개는 원본 MXFP4, 이후 게이트 22개는 3비트로 보호합니다. 텍스트 가중치는 약 95.87 GiB입니다.
검증 세트 평균 KL은 0.914656 nats, 다음 토큰 일치율은 **77.0381%**입니다. 별도 최종 평가의 평균 KL은 0.866514 nats, 일치율은 **76.6198%**입니다. 원본 성능과 동일하다는 의미는 아닙니다. 추론은 기본 활성화되며 enable_thinking=false로 끌 수 있습니다. 추론 파서는 think_xml, 도구 파서는 xml_function입니다.
새 MiMo-V2.6 Python 런타임이 필요합니다. 일부 영상 방향 판단과 복합 멀티모달 지시는 실패하며, Swift·1M 문맥·오디오 출력·MTP/DFlash 실행은 검증되지 않았습니다. 기본 SSD 캐시는 네이티브 BF16 데이터를 손실 없이 저장합니다.
License
The source checkpoint declares MIT. Source revision: 5711b268169967567844e1e560e8a3966da959b1. Quantization does not change the source license.