Qwen3.5-397B-A17B-JANG_2L

This model uses reasoning/thinking mode. The model reasons inside <think>...</think> tags before answering. This dramatically improves accuracy on hard questions (abstract algebra: 50% → 80%, math: 50% → 85%). MLX Studio is required to run this model — it handles JANG format, bfloat16 compute, and thinking mode natively.

LM Studio, Ollama, oMLX, Inferencer do NOT support JANG format. Use MLX Studio or pip install "jang[mlx]".

JANG is fully open-source. Quantization engine, research, and full commit history: github.com/jjang-ai/jangq. Created by Jinho Jang.

Key Features

• 92.0% MMLU (200 questions, reasoning mode) — 397B intelligence on Apple Silicon
• 36 tok/s generation speed on M4 Ultra 256 GB
• Reasoning mode: <think>...</think> for step-by-step problem solving
• Vision (VLM): 333 vision tensors, processes images and video
• 187 GB on disk, 197 GB peak GPU RAM
• bfloat16 compute: auto-detected by JANG loader for 512-expert models

Results: JANG_2L vs MLX 4-bit (200-question MMLU)

Per-subject comparison across all modes. Both JANG and MLX 4-bit tested with and without reasoning.

Summary

JANG_2L is 22 GB smaller than MLX 4-bit. MLX 4-bit with reasoning reaches 94.0%, while JANG_2L reaches 92.0% at significantly smaller size. MLX 2-bit and 3-bit produce NaN -- cannot run (float16 overflow on 512-expert models). JANG solves this with bfloat16.

Specs

Requirements

• Apple Silicon Mac with 256 GB unified memory (M3/M4 Ultra)
• MLX Studio (recommended) or pip install "jang[mlx]>=2.1.5"
• Python 3.11+ for CLI usage

Quick Start (Python)

pip install "jang[mlx]>=2.1.5"

from jang_tools.loader import load_jang_model
from mlx_lm import generate

model, tokenizer = load_jang_model("JANGQ-AI/Qwen3.5-397B-A17B-JANG_2L")
# bfloat16 is auto-applied for 512-expert models

# With reasoning (recommended for hard questions)
messages = [{"role": "user", "content": "Prove that sqrt(2) is irrational."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False,
    add_generation_prompt=True, enable_thinking=True)
result = generate(model, tokenizer, prompt=prompt, max_tokens=2048)

# Without reasoning (faster for simple questions)
prompt = tokenizer.apply_chat_template(messages, tokenize=False,
    add_generation_prompt=True, enable_thinking=False)
result = generate(model, tokenizer, prompt=prompt, max_tokens=100)

VLM Usage

from jang_tools.loader import load_jang_vlm_model
from mlx_vlm import generate as vlm_generate

model, processor = load_jang_vlm_model("JANGQ-AI/Qwen3.5-397B-A17B-JANG_2L")

# Format prompt with image tokens
messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": "Describe this image."},
]}]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
result = vlm_generate(model, processor, prompt=prompt, image=["photo.jpg"], max_tokens=200)

What is JANG?

JANG (Jang Adaptive N-bit Grading) is a mixed-precision quantization format for Apple Silicon that classifies every weight tensor by sensitivity:

• CRITICAL (8-bit): Full attention Q/K/V/O, MoE routers, output head
• IMPORTANT (6-bit): Embeddings, GatedDeltaNet (linear attention)
• COMPRESS (2-bit): MoE expert MLP (512 experts provide redundancy)
• MLP Asymmetry: gate_proj=4-bit (SiLU amplifier), down_proj=3-bit (residual projection)

This gives 397B-level intelligence at 187 GB — fitting on a single M4 Ultra Mac Studio.

Technical Notes

• bfloat16 compute: 512-expert models with hidden_size=4096 overflow float16 (max 65,504) at the shared expert down_proj. The JANG loader auto-detects this and uses bfloat16 (max 3.4×10^38). Zero quality impact — quantization noise dominates.
• Reasoning mode: The model uses <think>...</think> tags for step-by-step reasoning. On hard questions (math, physics, algebra), this improves accuracy by 25+ percentage points.
• Chat template: Includes enable_thinking toggle. Set to False for fast answers, True for reasoning.

한국어 안내

JANG은 Apple Silicon을 위한 혼합 정밀도 양자화 형식입니다. Qwen3.5-397B를 Mac Studio 한 대에서 36 tok/s로 실행할 수 있습니다.

• 92.0% MMLU (추론 모드)
• 187 GB 디스크 용량, M4 Ultra 256 GB에서 실행
• MLX Studio 필요

pip install "jang[mlx]>=2.1.5"