Qwen3.5-397B-A17B-JANG_1L

397B intelligence on a 128 GB Mac. This model fits in 112 GB — the first 397B quantization that runs on M4 Pro/Max 128 GB machines. Uses reasoning mode for 86.5% MMLU accuracy.

LM Studio, Ollama, oMLX do NOT support JANG format. Use MLX Studio or pip install "jang[mlx]>=2.1.5".

JANG is fully open-source. Quantization engine, research, and full commit history: github.com/jjang-ai/jangq. Created by Jinho Jang.

Key Features

• 86.5% MMLU (200 questions, reasoning mode) — 397B on 128 GB Macs
• 36.1 tok/s generation, 96 tok/s prefill
• 112 GB on disk, 110 GB GPU RAM (peak 120 GB)
• Reasoning mode: <think>...</think> step-by-step problem solving
• Vision (VLM): 333 vision tensors, 31.6 tok/s image processing
• bfloat16 compute: auto-detected for 512-expert models

Results: JANG_1L vs MLX 4-bit (200-question MMLU)

Per-subject comparison across all modes. Both JANG and MLX 4-bit tested with and without reasoning.

Summary

JANG_1L is 97 GB smaller than MLX 4-bit and fits on 128 GB Macs where MLX 4-bit (209 GB) cannot run. MLX 2-bit and 3-bit produce NaN -- cannot run (float16 overflow on 512-expert models). JANG solves this with bfloat16.

Specs

Requirements

• Apple Silicon Mac with 128+ GB unified memory
• MLX Studio (recommended) or pip install "jang[mlx]>=2.1.5"

Quick Start

pip install "jang[mlx]>=2.1.5"

from jang_tools.loader import load_jang_model
from mlx_lm import generate

model, tokenizer = load_jang_model("JANGQ-AI/Qwen3.5-397B-A17B-JANG_1L")

# With reasoning
messages = [{"role": "user", "content": "Prove that sqrt(2) is irrational."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False,
    add_generation_prompt=True, enable_thinking=True)
result = generate(model, tokenizer, prompt=prompt, max_tokens=2048)

# Without reasoning (faster)
prompt = tokenizer.apply_chat_template(messages, tokenize=False,
    add_generation_prompt=True, enable_thinking=False)
result = generate(model, tokenizer, prompt=prompt, max_tokens=100)

VLM Usage

from jang_tools.loader import load_jang_vlm_model
from mlx_vlm import generate as vlm_generate

model, processor = load_jang_vlm_model("JANGQ-AI/Qwen3.5-397B-A17B-JANG_1L")
messages = [{"role": "user", "content": [
    {"type": "image"},
    {"type": "text", "text": "Describe this image."},
]}]
prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
result = vlm_generate(model, processor, prompt=prompt, image=["photo.jpg"], max_tokens=200)

Technical Notes

• bfloat16: 512-expert models overflow float16 (max 65,504) at the shared expert down_proj. JANG auto-detects and uses bfloat16 (max 3.4x10^38). Zero quality impact.
• JANG_1L profile: 8-bit for all attention + routers + embeddings, 2-bit for expert MLP. On MoE models, expert MLP is 97.9% of params — so 2-bit covers almost everything while critical components get maximum precision.

한국어

JANG_1L은 Qwen3.5-397B를 128 GB Mac에서 실행할 수 있는 최초의 양자화입니다. 112 GB, 36 tok/s, 86.5% MMLU.

pip install "jang[mlx]>=2.1.5"