Qwen3.8-Flash-Next-Uncensored · oQ3e-fp16-mtp — a Novaeon.Studio build
A 99 B-A5B uncensored, vision-capable MoE that actually runs — and runs fast — on a single 128 GB Mac. This is an oMLX oQ3e quantization (error-compensated 3-bit base, sensitivity-guided 5/6/8-bit and fp16 for the tensors that matter) of orcarouter/Qwen3.8-Flash-Next-Uncensored, with the native MTP head preserved and working. Built, tuned, and benchmarked on an Apple M5 Max (128 GB) by Novaeon.Studio.
Why this build exists: Flash-Next is a 512-expert MoE with a PLE/n-gram embedding stack that most quantizers either refuse or destroy. At bf16 it is ~335 GB; the canonical oQ4e is ~106 GB and leaves almost no headroom on a 128 GB box. oQ3e lands at 85 GB on disk / ~79.5 GB in oMLX, which is the difference between "technically loads" and "this is my daily driver." Everything under Measured is from our hardware.
| Base model | orcarouter/Qwen3.8-Flash-Next-Uncensored |
| Architecture | qwen4_exp (Qwen4ExpForConditionalGeneration) — 48 layers, hidden 2560, 512 experts / 10 active, GatedDeltaNet linear attention + full attention every 4th layer, PLE n-gram embedding stack, vision tower (27-layer, 1152-dim) |
| Quantization | oQ3e — 3-bit affine base (group 64) with 475 sensitivity-selected tensor overrides (198 @ 8-bit, 129 @ 5-bit, 17 @ 4-bit, 3 @ 6-bit) + fp16 for norms/router/embeddings |
| MTP head | Native, mtp_num_hidden_layers: 1 · 76 mtp.* tensors preserved — and it delivers +67 % decode |
| Alignment | Uncensored (inherited from the base; verified on these quantized weights) |
| Context | 262,144 native · vision preserved |
| Size | 85.2 GB on disk · 17 shards · 3,748 tensors · 79.5 GB reported resident by oMLX |
| Engine | oMLX (Apple MLX) — VLM engine. Requires an oMLX build with the fixed qwen4_exp loader (0.7.0.dev2+). |
Best for & why it's here
Best for: uncensored, long-context work on big-RAM Apple Silicon (128 GB+) for anyone who specifically wants the Flash-Next frontier MoE brain in oMLX format. Experimental.
Why we published it: an aggressive oQ3e (≈3-bit) pack that squeezes the ~99B-class uncensored Flash-Next MoE onto a single 128 GB Mac at all. Decode is heavier and more memory-bound than our A3B seat, so it's a specialist pick — reach for it when you want this specific frontier brain uncensored, not as a general agent workhorse. Published for completeness of the Novaeon oMLX line-up.
Where the bytes go
| Group | Size |
|---|---|
| Transformer body (MoE + attention) | 55.6 GB |
| PLE / embedding tables (largely fp16) | 26.3 GB |
| Vision tower | 1.8 GB |
| MTP head | 1.5 GB |
Measured on our hardware (Apple M5 Max 128 GB · oMLX, VLM engine)
MTP speculative decoding
Decode tok/s, median of 3 runs, 320-token generation, temp 0, 8 k KV, TurboQuant-KV 8-bit:
| Configuration | Decode (tok/s) | vs MTP off |
|---|---|---|
| MTP off | 42.3 | — |
MTP on, draft-4 | 70.2 | +66 % |
MTP on, draft-6 ⭐ | 70.8 | +67 % |
MTP on, draft-8 | 67.9 | +61 % |
draft-6 is the shipped default. draft-8 regresses — the extra drafts stop paying for their verification cost.
Prefill and long context
| Prompt | Input tokens | Wall | Prefill throughput |
|---|---|---|---|
| 4 k-class | 11,268 | 8.2 s | ~1,384 tok/s |
| 16 k-class | 44,869 | 23.6 s | ~1,914 tok/s |
| 32 k-class | 90,068 | 34.8 s | ~2,605 tok/s |
Prefill throughput rises with prompt length — the MoE prefill path amortizes well. A needle-in-a-haystack retrieval at 80 k tokens was answered exactly and verbatim, and a warm-cache repeat of that 80 k prompt returned in 3.4 s (99.7 % prefix cache hit).
Memory behaviour
Loaded via oMLX's qwen4_exp path under a 108 GB memory-guard ceiling: 79.5 GB reported model size, qwen4_ple_ssd_offload off (no SSD offload forced), ~62.5 GB of the PLE stack served through mmap rather than wired pages. Under sustained decode the machine stayed at ~21 % free system memory with no paging stalls and no throughput cliff — i.e. the mmap-backed PLE is not an SSD bottleneck on this hardware. Cold load: ~17 s.
Capability probes
| Probe | Result |
|---|---|
| General coherence / explanation | Pass — fluent, accurate, well-structured |
| Reasoning (bat-and-ball trap) | Pass — $0.05 with a correct one-line check |
| Refusal probes (profanity, security mechanics) | Pass — zero refusals, engaged directly with both |
| Vision (chart reading) | Pass — correctly identified chart type, background, title, and axis category labels from a PNG |
| Long-context retrieval @ 80 k | Pass — exact needle recall |
| Tool calling — single-tool select | Pass — correct function, correct arguments |
| Tool calling — "no tool needed" | Pass — answered directly, no spurious call |
| Tool calling — parallel multi-city | Fail — emitted one call instead of two |
| Tool calling — chained (weather → email) | Fail — emitted reasoning prose instead of a call |
Be honest about the weak spot: this build is strong at prose, reasoning, vision, and long context, and it is genuinely uncensored — but multi-step and parallel tool orchestration is its soft edge. If you are wiring it into an agent loop that depends on parallel or chained tool calls, test that path before committing. Single-tool selection is reliable.
Recommended oMLX settings
{
"mtp_enabled": true,
"mtp_num_draft_tokens": 6,
"turboquant_kv_enabled": true,
"turboquant_kv_bits": 8.0,
"turboquant_skip_last": true,
"qwen4_ple_ssd_offload": false,
"qwen35_ane_prefill_enabled": false,
"qwen35_oq_a8_enabled": false,
"max_context_window": 262144
}
Notes:
preserve_mtp: trueis not the default when quantizing with oMLX'soqpipeline — pass it explicitly or you will silently ship a model with no MTP head.- Keep
qwen4_ple_ssd_offloadoff on a 128 GB box; forcing SSD offload converts a 70 tok/s model into a disk-bound one. - ANE prefill measured as a net regression for this class on our hardware — left off.
Usage
# oMLX
omlx serve --model-dir ~/.omlx/models
curl http://127.0.0.1:8000/v1/chat/completions \
-H "Authorization: Bearer $OMLX_API_KEY" -H 'Content-Type: application/json' \
-d '{"model":"Qwen3.8-Flash-Next-Uncensored-oQ3e-fp16-mtp","messages":[{"role":"user","content":"Hello"}]}'
Text + image works through the standard OpenAI image_url content block (base64 data URIs supported).
Intended use & limitations
This model is uncensored: it will follow instructions that aligned models decline, and it applies no content filtering of its own. You own what you generate with it, and you are responsible for putting appropriate safeguards around any deployment that faces other people. It is intended for local research, creative work, and agentic experimentation by people who want an unfiltered local assistant — not for unsupervised public-facing serving.
Other limitations: 3-bit base quantization will cost some accuracy against the bf16 original, particularly on tight factual recall; parallel/chained tool calling is unreliable (see probes above); and it requires an oMLX build with the working qwen4_exp loader.
Credits
Base model by orcarouter. Architecture by the Qwen team. Quantization stack: oMLX on Apple MLX. Quantized, tuned, benchmarked, and documented by Novaeon.Studio.
License inherited from the base model (Apache-2.0).