Qwen3.8-Flash-Next Uncensored — MLX-Serve 4-bit: How to Run Locally

The uncensored Qwen3.8-Flash-Next (125B total / 6B active MoE) packed for mlx-serve, so it actually fits and flies on a 128 GB Apple Silicon Mac:

• ~68 GB resident — the 51B-parameter n-gram table lives in ngram_table.bin and is memory-mapped from disk, not held in RAM
• ~56 tok/s decode serial, ~60 tok/s with MTP measured on an M5 Max 128 GB (custom Metal kernels for the MoE gather, Gated-DeltaNet decode, hyper-connections, and sparse attention)
• Vision tower and MTP speculative head included
• Full 262,144-token context
• Base weights are orcarouter's abliteration of Qwen/Qwen3.8-Flash-Next — refusals removed, everything else intact

The other ways to run this model uncensored on a Mac either don't fit 128 GB (the original MLX repo is 163 GiB) or run slower (GGUF IQ4_XS ≈ 26 tok/s on the same hardware; generic Python MLX paths ≈ 27–36 tok/s).

About the base model

This is a 4-bit MLX-Serve pack of orcarouter/Qwen3.8-Flash-Next-Uncensored — the full-precision BF16 abliterated (refusal-removed) build of Qwen's Qwen3.8-Flash-Next. See orcarouter's card for the full abliteration methodology and evaluation; this section summarizes it.

⚙️ Requirements

📦 What's in the pack

Built with mlx-serve's own converter (tests/convert_qwen38_flash_next.py) from the bf16 uncensored weights:

The architecture's x·(1+w) RMSNorm offset is folded at convert time.

🚀 Run it

1. Get mlx-serve (binary release, or brew install ddalcu/mlx-serve/mlx-serve):

curl -LO https://github.com/ddalcu/mlx-serve/releases/download/v26.8.11-pre-release.1/mlx-serve-bin-macos-arm64.tar.gz
tar xzf mlx-serve-bin-macos-arm64.tar.gz

2. Download this pack (installing hf_transfer first makes this run at full line rate):

pip install -U huggingface_hub hf_transfer
HF_HUB_ENABLE_HF_TRANSFER=1 hf download ARC4NUM/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit \
  --local-dir ~/.mlx-serve/models/ARC4NUM/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit

3. Chat right in the terminal:

./mlx-serve --model ~/.mlx-serve/models/ARC4NUM/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit \
  --prompt "Hello!" --max-tokens 512 --temp 0.7

4. Or serve it — OpenAI, Anthropic, and Ollama APIs on one port:

./mlx-serve serve --host 127.0.0.1 --port 11234
# OpenAI:    http://127.0.0.1:11234/v1/chat/completions
# Anthropic: http://127.0.0.1:11234/v1/messages
# Ollama:    http://127.0.0.1:11234/api/chat

Works out of the box with Claude Code, pi, OpenCode, Codex, aider, Open WebUI, Raycast — anything that speaks one of those wires. mlx-serve launch <agent> preconfigures and starts your coding agent against it.

MTP speculative decoding (the model's own draft head) is opt-in: pass --mtp at launch or "enable_mtp": true per request. Fastest on code and echo-heavy agent loops; a touch slower on freeform prose.

🎛️ Recommended Settings

Qwen's official sampling settings for this model:

Thinking effort defaults to xhigh and this model will happily ruminate; for agent work, --reasoning-budget 2048 (or the reasoning_effort request field: low / medium / xhigh) keeps it moving. The model supports preserve thinking — keeping prior turns' traces in context — which costs tokens but can help multi-turn accuracy.

📊 Measured performance

M5 Max, 128 GB, this pack, --temp 0 300-token generations:

The Python paths are kernel-bound, not bandwidth-bound — the n-gram gathers, hyper-connections, and GDN decode run as generic graphs there. mlx-serve's fused kernels are the whole difference. The author reports ~60 serial / 78 MTP on an M4 Max for the base-model pack; MTP wiring is still marked partial in this pre-release, so expect free speed from future engine updates.

Context memory: only 12 of 48 layers are full attention (2 KV heads × 256 head-dim), so KV cache costs ~24 KB/token — ~2.4 GB at 100k context. Long agent sessions fit comfortably inside 128 GB.

🧪 Reproduce this pack

python3 tests/convert_qwen38_flash_next.py \
  --src /path/to/Qwen3.8-Flash-Next-Uncensored-bf16 \
  --dst ./Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit
python3 tests/convert_qwen38_flash_next.py --add-vision \
  --src /path/to/Qwen3.8-Flash-Next-Uncensored-bf16 \
  --dst ./Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit

(~10 minutes total; streams shard-by-shard, never needs the full 335 GB in RAM.)

⚠️ Disclaimer — read before use

Reproduced from the source model card and applying equally to this quantized redistribution, which performs format conversion only.

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-Flash-Next would refuse. It has no meaningful built-in guardrails.
• It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
• You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm. Its outputs do not reflect the views of the uploaders or of Qwen / Alibaba.

By downloading or using this model you acknowledge and accept the above.

Attribution & lineage

This repository changes precision and file format only — no weights were trained, merged, or otherwise modified beyond quantization. The chain:

License: Qwen Community 1.0, inherited unchanged from the base model and applying to this derivative. Review it before commercial use.