Quantizations in this repo
| Quant | Location | Size | Bits/weight (effective) |
|---|---|---|---|
| 4-bit (default) | repo root | 163 GB | ~7.85 |
| 6-bit | 6-bit/ | 192 GB | ~9.27 |
| 8-bit | 8-bit/ | 221 GB | ~10.68 |
The 4-bit weights are at the repo root (loads by default); 6-bit and 8-bit are in the 6-bit/ and 8-bit/ subfolders. mlx-vlm's current qwen4_exp path quantizes attention / projection Linears; the fused-3D experts and n-gram table stay higher-precision, so effective bits/weight are above the nominal.
An abliterated and MLX 4-bit build of
Qwen/Qwen3.8-Flash-Nextfor Apple Silicon (Metal). Converted withmlx-vlm(which supportsqwen4_exp).
Derived releases: •
Qwen3.8-Flash-Next-Uncensored(BF16 source) •Qwen3.8-Flash-Next-Uncensored-FP8(block-FP8, mirrors official) •Qwen3.8-Flash-Next-Uncensored-MLX(4 / 6 / 8-bit, Apple Silicon).
⚠️ Disclaimer — read before use
This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:
- It will comply with harmful, unethical, offensive, or illegal requests that the original
Qwen3.8-Flash-Nextwould refuse. It has no meaningful built-in guardrails. - It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
- You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
- Use must comply with the Apache 2.0 License inherited from the base model, and all laws and regulations that apply to you.
- The authors and uploaders accept no liability for any misuse or harm. Its outputs do not reflect the views of the uploaders or of Qwen / Alibaba.
By downloading or using this model you acknowledge and accept the above.
🐋 Run it with OrcaCode Review
Models are only half the system.
OrcaCode Review turns every model listed on OrcaRouter into a production code-review agent:
- reviews every PR
- finds security + correctness issues
- posts inline findings
- P0/P1 can block merges
- swap models anytime
Open model. Open harness. Open bill.
- Website — https://www.orcarouter.ai
- GitHub — https://github.com/Continuum-AI-Corp/Orca-Code-Review
Model details
| Base model | Qwen/Qwen3.8-Flash-Next |
| Architecture | Qwen4ExpForConditionalGeneration (qwen4_exp, Qwen4 preview) — 48 layers, hidden 2560, hybrid attention (36 Gated-DeltaNet linear + 12 full-attention, interval 4), 512 fused experts, top-10 + shared expert, 51B-param PLE n-gram embedding, Hyper-Connections residual, native vision + video tower, and an MTP speculative-decoding head |
| Modification | Abliteration (refusal-direction removal) then MLX 4-bit quantization |
| Quantization | MLX affine 4-bit (group size 64); MoE router gates kept at 8-bit |
| Format | safetensors (MLX), 163 GB, 35 shards (~7.85 bits/weight effective) |
| Note | mlx-vlm's current qwen4_exp path quantizes attention / projection Linears; the fused-3D experts and the n-gram table remain higher-precision, so the effective footprint is larger than a uniform 4-bit. |
| Context | 262,144 tokens |
Abliteration
Refusal-direction removal following Arditi et al. (2024), Refusal in Language Models Is Mediated by a
Single Direction. A single refusal direction r (k = 1) is estimated from the block-input residual
stream (the 2560-d Hyper-Connections GR-Read output, where the refusal direction is linearly separable —
the widened 4-branch output_hidden_states smears it) as the massive-activation-masked mean-difference of
harmful − harmless activations, selected at layer 24 by a full 9-layer quality sweep
(harmful 0.00 / KL 0.085). r is then orthogonalized out of every residual-writing matrix —
W' = W − r(rᵀW) — computed in float32:
| Component | matrices edited |
|---|---|
self_attn.o_proj (12 full-attention layers + MTP) | 13 |
linear_attn.out_proj (36 GDN linear-attention layers) | 36 |
mlp.experts.down_proj (fused 3D, all 512 experts × 49 layers) | 49 |
mlp.shared_expert.down_proj | 49 |
ple.value_proj + embed_tokens (row space) | 2 |
| Total residual-writer tensors | 149 |
Preserved (never touched): the full vision + video tower (333 visual.* tensors), the MoE
router (mlp.gate), the fused experts.gate_up_proj reader, all Hyper-Connection mixers, the
QSA sparse-attention indexer, the n-gram embedding table, mtp.fc_*, norms, and lm_head. The MTP
head's residual writers are abliterated consistently so speculative decoding keeps working. Max residual
leakage after the edit: 0.0755 (float32 projection → bf16 storage).
This is a surgical weight edit — it changes ~0 general capability (see Evaluation) while collapsing refusal behaviour.
Usage — Apple Silicon (MLX)
pip install mlx-vlm
python -m mlx_vlm.generate --model Qwen3.8-Flash-Next-Uncensored-MLX-4bit \
--prompt "Prove that sqrt(2) is irrational." --max-tokens 512
Requires a Mac with enough unified memory for the 163 GB weights (e.g. M-series Ultra). Refusal /
capability behaviour is inherited from the BF16 source Qwen3.8-Flash-Next-Uncensored (see its card's Evaluation); runtime
verification requires Apple-Silicon hardware.
Bias, risks, and limitations
- Safety guardrails removed — see the disclaimer.
- Inherits biases / limitations of the base
Qwen3.8-Flash-Next.
License
Apache 2.0, inherited from Qwen/Qwen3.8-Flash-Next.