Spark-X2.5-4B-Abliterated-Uncensored — MLX oQ4e (fp16 source)

Model Details

Model Description

oQ4e-quantized MLX build (created by xunlinkx with the omlx pipeline) of Mondk/Spark-X2.5-4B-Abliterated-Uncensored for first-class, all-local serving on Apple Silicon via omlx (the MLX OpenAI-compatible server) and mlx-lm.

Attribution chain: XHToken/Spark-X2.5-4B (official base, Apache-2.0) → Mondk's abliterated/uncensored adaptation → xunlinkx's fp16 MLX conversion + oQ4e quantization (this repo).

This is not a new model — it is a quantization of the existing checkpoint, produced with omlx's quantize_oq_streaming(..., oq_level=4, enhanced=True) pipeline:

• Why we made it: run a full 4B Spark-X2.5-class model locally in ~2.5 GB of memory at usable decode rates (≈44 tok/s on an M3 Pro), with the attention gates and projection heads kept at higher precision so reasoning quality survives 4-bit compression. The -fp16 suffix records that the quantizer consumed full-precision float16 source weights (best-fidelity input), as opposed to bf16 or already-quantized sources.

• Quantization: method oQ4e (omlx enhanced streaming imatrix calibration, 128×512 blocks), mixed-bit per-layer layout embedded in config.json (quantization / quantization_config) so the exact layout is restored automatically on load.

• Developed by: xunlinkx (oQ4e quantization + MLX build, using omlx)

• Base checkpoint by: Mondk (abliterated/uncensored adaptation of XHToken/Spark-X2.5-4B)

• Shared by: xunlinkx

• Model type: spark2_5 text-generation decoder (MLX quantized, model_type: spark2_5)

• Language(s) (NLP): multilingual (base model; Spark-X2.5 pretraining, including English/Chinese coverage)

• License: the Spark-X2.5 series is Apache 2.0; this is a derived quantization / adaption build of the gated Mondk abliterated/uncensored checkpoint — see Mondk/Spark-X2.5-4B-Abliterated-Uncensored for those terms.

• Finetuned from model: Spark-X2.5-4B (base), abliterated/uncensored adaptation by Mondk; no additional fine-tuning performed here.

Model Sources

• Repository (quantization tooling): omlx — quantize/serve/stream via quantize_oq_streaming
• Base model: Mondk/Spark-X2.5-4B-Abliterated-Uncensored

Quantization Details

Per-layer overrides are written into config.json, so mlx-lm/omlx reproduce this exact layout at load time — no manual flags needed. Calibration report: oq_imatrix_report.json (included in this repo).

5.179 weight-bits per parameter average (group_size 64, affine; embeddings/norms fp16, per-group scales add ~0.25 bpw on disk). Head gates are lifted to 5–8 bit on all 36 layers; q_k_v_proj/down_proj to 5–6 bit on sensitive layers.

Uses

Direct Use

Local and private inference on macOS (Apple Silicon):

# omlx — OpenAI-compatible server with native spark2_5 support
omlx serve -m xunlinkx/Spark-X2.5-4B-Abliterated-Uncensored-MLX-oQ4e-fp16

# or plain mlx-lm
python -m mlx_lm.generate \
  --model xunlinkx/Spark-X2.5-4B-Abliterated-Uncensored-MLX-oQ4e-fp16 \
  -p "Explain why attention gates matter under extreme quantization." \
  -m 256 --temp 1.0 --top-p 0.95  # official Spark-X2.5 sampling: temp 1.0, top_p 0.95, top_k -1

Compatibility — Spark support required (patch it yourself until mainline)

Requires an mlx-lm with native spark2_5 model support (module name matches the checkpoint's model_type). Neither upstream has merged native support yet (as of 2026-09-15), although omlx main carries a vendored patch (#3606, 2026-09-14) that no tagged release includes yet — until releases catch up, apply the PRs:

• mlx-lm users: checkout ml-explore/mlx-lm#1891 (gh pr checkout 1891 inside a mlx-lm clone, or pip install "mlx-lm @ git+https://github.com/xunlinkx/mlx-lm@spark2_5-native"; ab1806e-era alternative for omlx-scheduler compatibility: .../xunlinkx/mlx-lm@spark2_5-backport).
• omlx users: spark support is on main since jundot/omlx#3606 (2026-09-14, vendored patch, self-disabling once the native module is present) — but no tagged release includes it yet (latest stable v0.6.4 and the 0.7.0.dev pre-releases all predate it). Run omlx from source main, or apply jundot/omlx#3671 for the oQ rules + mlx-lm pin (gh pr checkout 3671).

Without spark2_5 support, loading fails with an unsupported-model-type error.

Recommended Sampling Settings

The official Spark-X2.5 card (XHToken/Spark-X2.5-4B) recommends:

• temperature = 1.0
• top_p = 0.95
• top_k = -1 (disabled)

These match the checkpoint's own generation_config.json (do_sample: true, penalties 0). All official benchmarks were evaluated in thinking mode (enabled by default by the chat template). Use temperature 0 only for deterministic arithmetic/code checks (as in the upstream MLX quickstart).

Long-context (windowed attention, sliding window 512, RoPE base 5e6 for full-attention layers) works up to max_position_embeddings = 1,048,576.

Out-of-Scope Use

• Do not fine-tune or continue-training from this quantized checkpoint — it is for inference only.
• Not a replacement for full-fidelity fp16/bf16 serving when maximum math/code accuracy is required (4-bit quantization is lossy; gates/projections are retained at 6–8 bit to bound the loss).
• This is an abliterated/uncensored checkpoint: it does not refuse or moderate. Do not deploy where unmoderated model output is unacceptable.

Bias, Risks, and Limitations

• Unmoderated output: the base model was abliterated to remove refusals and uncensored. Users get exactly what the weights produce — including content the upstream model would normally refuse. Deploy with guardrails or user-facing disclaimers where appropriate.
• Quantization loss: 4-bit group-64 affine quantization plus 6-bit projections can degrade formatting, long arithmetic, and rare-token recall versus fp16. The 8-bit g_proj and 6-bit projections are a deliberate mitigation.
• Windowed attention: sliding-window (512) layers mean very long prompts rely on repeating, rolling context; cross-window dependencies are best-effort.
• Verify outputs in high-stakes use; the model has no built-in safety tooling.

Recommendations

Users (both direct and downstream) should be aware of the above limitations, especially the uncensored nature of the adaptation. Provide content warnings in any public-facing wrapper.

How to Get Started with the Model

See Uses → Direct Use. A minimal client example (OpenAI-compatible via omlx):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="omlx")
resp = client.chat.completions.create(
    model="Spark-X2.5-4B-Abliterated-Uncensored-MLX-oQ4e-fp16",
    messages=[{"role": "user", "content": "Write a haiku about quantization."}],
    temperature=1.0, top_p=0.95, max_tokens=128,
)
print(resp.choices[0].message.content)

Training Details

Training Data

No training data used here — this is a quantization artifact. See the base model card for the pretraining and abliteration/uncensoring data details.

Training Procedure

Not trained. Produced by omlx quantize_oq_streaming, oQ4e mode, fp16 source, imatrix calibration at 128×512 streaming blocks.

Evaluation

Measured on Apple M3 Pro (36 GB):

• Load: strict single-file load ≈ 0.5 s
• Prefill: ≈ 145 tok/s (long prompts crossing the 512 sliding window handled correctly)
• Decode: ≈ 44 tok/s (200-token steady state), peak memory 2.5 GB
• Fidelity: 100 % argmax agreement with the fp32 reference over a 1,170-token prefill; cached-decode parity across the sliding-window boundary.

Technical Specifications

Model Architecture and Objective

Spark-X2.5 4B decoder (from config.json): 36 layers (mixed sliding-window/full attention), hidden size 2560, intermediate 10240, 16→4 grouped-query attention, head dim 256, vocab 131072, sliding window 512, max position 1048576. RoPE: layer-type-specific partial dims/bases (SWA base 1e4 / full base 5e6). Head-wise sigmoid attention output gating (the layer kept at 8-bit). Tied embeddings unprojected for MLX weight naming.

Compute Infrastructure

Hardware

Apple Silicon (developed on Apple M3 Pro, 36 GB). No cloud compute, no remote training.

Software

omlx (quantize + serve), mlx-lm (native spark2_5 module), MLX, Python 3.x.

Environmental Impact

Quantization and evaluation ran entirely on local Apple Silicon hardware; no cloud instances or dedicated compute were used. Estimated carbon contribution of this artifact is negligible (single workstation, hours).

Model Card Contact

xunlinkx — questions/issues via the model repo discussion tab.