OsaurusAI/Kanana-2-30B-A3B-Instruct-JANG_4M
A Korean-first JANG_4M MLX bundle of Kakao's Kanana-2 30B-A3B Mixture-of-Experts model, tuned to run fast on Apple Silicon via Osaurus.
Converted from kakaocorp/kanana-2-30b-a3b-instruct-2601.
Why this model
Kanana-2 is one of the strongest open Korean LLMs — Kakao trained it with a tokenizer ~30% more efficient on Korean and a 6-language mix (Korean, English, Japanese, Chinese, Thai, Vietnamese). It is a 30.7B-parameter MoE that activates only ~3B parameters per token (top-6 of 128 routed experts + 2 shared), so you get large-model Korean quality at small-model decode speed. JANG_4M compresses it to ~17 GB so it fits comfortably in unified memory on a MacBook.
Bundle
| Field | Value |
|---|---|
| Source | kakaocorp/kanana-2-30b-a3b-instruct-2601 |
| Architecture | deepseek_v3 / DeepseekV3ForCausalLM (MLA attention) |
| Total / active params | 30.7B / ~3B per token |
| Decoder layers | 48 (layer 0 dense, 1–47 MoE) |
| Hidden size | 2048 |
| Experts | 128 routed, top-6, + 2 shared (moe_intermediate_size 768) |
| Attention | MLA — kv_lora_rank 512, no query LoRA, qk_nope 128 / qk_rope 64, v_head_dim 128 |
| Context | 32,768 tokens |
| Weight format | jang_affine (JANG v2, instant mmap load) |
| Tier bits | attention 8-bit · router/gate fp16 · MoE expert + MLP bulk 4-bit · embed/lm_head bookends high-precision |
| Average bits/weight | 4.26 |
| Shards | 32 safetensors |
| Bundle size | ~17 GB |
| Capabilities | text (family deepseek_v3, reasoning qwen3, tools qwen, cache mla) |
Quantization policy (JANG_4M)
JANG_4M is a mixed-precision affine scheme designed for MoE: the precision-sensitive pieces stay wide while the redundant expert bulk is compressed.
- 8-bit affine: self-attention projections (MLA q/kv compression + decompression, o_proj).
- fp16 passthrough: MoE router/gate weights (routing is extremely precision-sensitive), all RMSNorms.
- 4-bit affine: routed-expert and dense-MLP gate/up/down projections (the bulk of the weights).
- High-precision bookends: token embeddings and
lm_head.
The per-tensor bit widths are recoverable directly from the weight/scale shapes; jang_config.json is authoritative for the JANG runtime.
Modalities
| Modality | Status |
|---|---|
| Text | supported |
| Vision / Audio / Video | not present in source — text-only model |
Languages
Korean (primary), English, Japanese, Chinese, Thai, Vietnamese.
Reasoning & tools
The source chat template is ChatML-style (<|im_start|>role) with <think>…</think> reasoning and <tool_call>{json}</tool_call> tool calls. The bundle stamps reasoning_parser: qwen3, tool_parser: qwen. Thinking is off by default for the instruct variant; enable it through your runtime's thinking toggle.
Runtime
Built for the Osaurus / vMLX Apple-Silicon runtime. Loading requires a deepseek_v3-aware MLX loader that honors jang_config.json and the bundle's mixed per-tensor bit widths. tokenizer_config.json, generation_config.json, and chat_template.jinja are preserved from the source.
Notes
- This is a freshly converted bundle. The base model's published Korean benchmarks (e.g. KMMLU) reflect the unquantized weights; JANG_4M-level evaluation for this bundle is pending and will be added here when complete.
- MTP: the source ships an MTP layer; speculative decoding is not enabled in this bundle (
mtp: nonefor autoregressive decode).
Credits
- Base model: Kakao Corp. — Kanana-2, released under the Kanana License.
- JANG conversion & Osaurus packaging: Jinho Jang — eric@osaurus.ai