Qwen3.8-Flash-Next-Uncensored — MLX-Serve mixed 4/8-bit, verified mirror
This is a verified mirror of existing weights, with a reproducible local mlx-serve runtime patch and measurements on a Mac Studio M3 Ultra / 96GB. It is not a newly trained model, a new abliteration, or a new quantization recipe.
All 102 weight/N-gram files (107,303,406,192 bytes) were compared by SHA-256
against ARC4NUM's existing pack
at revision 9ebf9993b1eaec96aec938bf883601b51a90393b: 102/102 matched.
The model configuration also matched. The original pack deserves the credit;
this mirror adds the local runtime work and validation notes below.
The MTP fix changes the inference engine, not the model weights. Downloading these weights alone does not install the fix. See engine/README.md.
Which variant should I use?
Two MLX-Serve-specific mixed-precision variants are available:
| Variant | Positioning | Routed MoE experts | Other eligible projections | Token embeddings | N-gram table | Approx. pack size |
|---|---|---|---|---|---|---|
| 4/8-bit | Recommended default / quality-oriented | 4-bit | 8-bit | 4-bit | 4-bit | ~107 GB |
| 3/8-bit | Lower-memory experimental variant | 3-bit | 8-bit | 4-bit | 4-bit | ~86 GB |
Choose the 4/8-bit variant as the default starting point when memory capacity is sufficient. It uses less aggressive quantization for the routed MoE expert weights.
Choose the 3/8-bit variant when reducing model memory or storage requirements is more important. Only the routed MoE expert weights are reduced to 3-bit; the token embeddings and N-gram table remain 4-bit, and the other eligible projections remain 8-bit.
These labels describe the intended trade-off between the two packs. They are not a claim that the 4/8-bit variant is universally more accurate, or that the 3/8-bit variant has universally equivalent quality or speed. Measured quality and performance are workload-, context-, runtime-, and hardware-dependent.
Runtime compatibility
These are specialized checkpoints for the patched
mlx-serveruntime used by this repository. They are not generic Transformers,mlx-lm, ormlx-vlmcheckpoints. Do not rely on Hugging Face's automatically generatedmlx-lm/mlx-vlmusage examples for these packs; follow this repository'sengine/README.mdand runtime instructions instead.
Validation notes
Results reported for each variant should be interpreted only under the conditions documented on that variant's model card. Scores obtained with different task sets, runtime revisions, generation limits, or evaluation procedures should not be compared directly.
The 3/8-bit variant has additionally been validated on a Mac Studio M3 Ultra with 96 GB unified memory using its matching patched MLX-Serve V2 runtime. In that workload, it reduced measured MLX peak memory relative to the tested 4/8-bit reference. This is a workload-specific observation, not a general memory or quality guarantee.
Lineage and contents
- Base model: Qwen/Qwen3.8-Flash-Next.
- Uncensored source: orcarouter/Qwen3.8-Flash-Next-Uncensored,
local conversion source revision
8336e613ea508b13c2159bd0f68965d97a606b95. - Existing byte-identical pack: ARC4NUM.
- Engine and converter: ddalcu/mlx-serve.
- This repository: mirror, local runtime modifications, and validation by yomie4343.
| Component | Storage |
|---|---|
| Routed MoE experts | 4-bit affine, group 64 |
| Other eligible projections | 8-bit affine, group 64 |
| Token embeddings | 4-bit affine, group 64 |
| N-gram table | 4-bit, group 32, separate SSD-mapped file |
| MTP head | Included, following the same mixed policy |
| Vision | Source vision weights preserved |
This is an mlx-serve-specific pack with a separate ngram_table.bin.
Do not assume generic Transformers, mlx-lm, or mlx-vlm can load it directly.
Keep the complete pack on a fast SSD and allow roughly 108GB plus working space.
Download and run
hf download yomie4343/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-mixed-4-8bit \
--local-dir ./qwen-flash-next
Build the pinned, patched runtime using engine/README.md. Then, from that engine checkout, a conservative text-only starting configuration is:
MLX_SERVE_CACHE_LIMIT=2147483648 MLX_SERVE_NGRAM_WARM=0 \
./zig-out/bin/mlx-serve --model /absolute/path/to/qwen-flash-next \
--serve --host 127.0.0.1 --port 11234 --ctx-size 32768 \
--kv-quant 8 --prefill-chunk 1024 --no-mtp --no-pld
The starting configuration above is deliberately not the full benchmark profile. For the exact tested MTP-auto settings, see evaluation/README.md. No wired-memory-limit increase or memory-admission bypass is recommended. Only expose the loopback endpoint externally after adding authentication and appropriate access controls.
Measured results, 2026-09-08 JST
Same patched engine, same 25 TypeScript tasks, temperature 0, thinking off, maximum output 3072, 12 agent iterations. Both models were independently regraded.
| Model | Passed | Task execution + normal grading |
|---|---|---|
| Original model | 22/25 (88%) | 38m 13.478s |
| This Uncensored model | 21/25 (84%) | 36m 50.870s |
The Uncensored run took about 3.6% less wall time, but solved one fewer task. This is one trial per model, not evidence of general superiority, and wall time includes tool use and revisions rather than measuring decode speed alone. All 50 initial test outcomes matched independent grading.
The patched runs completed 27 MTP use/reload cycles without the reproduced quantization-shape crash. A separate captured-history replay failed on request 19 with the old engine and passed 19/19 with the fix, including 14 MTP reloads. An actual Hermes tool round trip at about 19.7K input tokens passed in 29.6s; its cached follow-up decoded at 53.5 tok/s. These are workload-specific observations.
Limitations and safety
- Advertised/native context is not a guarantee of usable context on 96GB. The benchmark configured 204800, but its coding tasks did not exercise that full length.
- Earlier local image-plus-200K follow-up testing failed memory admission; one image-plus-180K test passed. Image allocations persisted during a 60-second idle check. Those cases were not rerun as part of this MTP fix and are not capacity guarantees.
- Greedy AR and MTP outputs have not been established as universally identical.
- The local patch includes experimental QSA/cache/lifecycle work; it is not an upstream release or a promise of support on other hardware.
- This source model is uncensored. Outputs can be unsafe, incorrect, or offensive. Use sandboxing, least-privilege tools, human review, and application-level safeguards. Do not give it unrestricted access to credentials, personal files, or external actions.
License and attribution
The weights retain Qwen Community License 1.0, not Apache-2.0. Read its commercial-use provisions, including the separate-license condition for qualifying Model-as-a-Service / AI Work Assistant businesses. The runtime patches retain mlx-serve's MIT terms and applicable third-party notices; see engine/LICENSE, engine/NOTICE, and engine/LICENSE-APACHE-2.0. Powered by MTPLX by Youssof Altoukhi — https://github.com/youssofal/mtplx — for upstream-derived portions identified in the engine notices.
Runtime update: JSON-number argument preservation
A focused engine follow-up fixes numeric-looking string IDs such as 007
being converted into invalid JSON and losing all tool arguments. Weights are
unchanged. See patch and verification notes.