Hermes-4-14B-abliterated-4bit-mlx

Uncensored Hermes 4 14B, running locally on a Mac. Abliterated, quantized to 4-bit MLX for Apple Silicon — 8.3 GB, comfortable on a 16 GB Mac. No cloud, no API key, no refusals.

A 4-bit MLX quantization of Babsie/Hermes-4-14B-BF16-abliterated, tuned for fast on-device inference on Apple Silicon.

• Base model: Babsie/Hermes-4-14B-BF16-abliterated (BF16 abliterated, ~28 GB)
• Architecture: Qwen3 (Hermes 4 post-training)
• Quantization: 4-bit affine, group size 64
• Format: MLX safetensors
• Footprint: ~8 GB on disk (two safetensors shards: 5.35 GB + 2.95 GB), runs comfortably on a 16 GB Mac and flies on 32 GB+
• Context: 40,960 tokens (~40 K) — inherited unchanged from upstream Hermes 4 14B

Why this exists

Hermes 4 is NousResearch's instruction-tuned family. The 14B variant is built on Qwen3, so it inherits Qwen3's tokenizer and chat template — but the post-training is pure Hermes, which means it's tuned for tool use, structured output, and not refusing benign-but-edgy questions.

Babsie's abliteration applies refusal-direction projection (Arditi et al., 2024) to the BF16 weights. The model still has all its capabilities — it just doesn't pre-empt with a refusal on questions that the upstream Hermes would politely decline. You become the moderator.

This MLX 4-bit conversion makes it usable on Apple Silicon at the speed the hardware was designed for. 14B at 4-bit fits in ~8 GB, which means it runs even on a 16 GB MacBook — and on a 64+ GB machine you can keep multiple models loaded simultaneously.

Usage

Python (mlx-lm)

from mlx_lm import load, generate

model, tokenizer = load("divinetribe/Hermes-4-14B-abliterated-4bit-mlx")

messages = [{"role": "user", "content": "Write a haiku about local inference."}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False
)

response = generate(model, tokenizer, prompt=prompt, max_tokens=256)
print(response)

Local OpenAI-compatible server

pip install mlx-lm
mlx_lm.server --model divinetribe/Hermes-4-14B-abliterated-4bit-mlx --port 8080

That gives you a local http://localhost:8080/v1/chat/completions endpoint that any OpenAI SDK client can hit. No tokens, no API bills, no telemetry.

Inside claude-code-local

MLX_MODEL=divinetribe/Hermes-4-14B-abliterated-4bit-mlx \
  bash scripts/start-mlx-server.sh

This routes Claude Code (or any Anthropic-API client) to the local model. See the claude-code-local repo for the full setup.

Where it fits in the lineup

Abliteration

"Abliteration" suppresses the model's built-in refusal direction so it doesn't refuse benign-but-edgy requests. It is not a general capability upgrade — use responsibly, and you remain bound by the upstream Hermes 4 / Qwen3 licenses.

Credits

• Hermes 4 post-training by NousResearch
• Qwen3 base by Alibaba
• Abliteration by Babsie
• MLX 4-bit conversion by divinetribe using mlx-lm 0.31.2

License

Apache 2.0, inherited from the upstream Hermes 4 / Qwen3 family.

About the author

This model was built by Matt Macosko (@nicedreamzapp) for the claude-code-local stack — run Claude Code 100% on-device with local AI on Apple Silicon (⭐ 2,664 on GitHub).

• 🤗 All my models: nicedreamzwholesale.com/software/huggingface/
• 💻 Software portfolio: nicedreamzwholesale.com/software/
• 🔒 AirGap AI (legal / healthcare / NDA workflows): nicedreamzwholesale.com/airgap/

More abliterated MLX models

Part of the Abliterated MLX for Apple Silicon collection — 11 uncensored models converted for Macs, from Gemma 4 12B up to Llama 3.3 70B, plus Qwen3, Qwen3-VL, Hermes 4 and Muse Glimmer 30B.

Part of Claude Code Local

This model is one of the fighters in Claude Code Local (3.2k★), which runs Claude Code 100% on-device on Apple Silicon through an MLX-native Anthropic-API server. Not sure which local model to run as an agent? Check the Agent-12 local agent leaderboard: real agent tasks, judged by the filesystem, same hardware for every row.

Built by Matt Macosko in Arcata, CA. Open to work on local-AI and Apple Silicon inference: matt@ineedhemp.com.