🐘 Pink Elephant 48B-S — GGUF

Dense-14B brain. Sparse 48B reach. Aligned to be helpful. Run it on your own machine.

From a single dense core to an SFT + DPO–aligned sparse Mixture-of-Experts flagship — converted, quantized, and Ollama-ready so you can run the entire success story locally.

🚀 At a glance

Pink Elephant 48B-S is the culmination of a six-stage lineage: a dense 14B foundation, upcycled into an 8-expert sparse MoE, refined through a second training epoch (48B-S), aligned with supervised fine-tuning and direct preference optimization, then merged bit-exact and converted to GGUF — ready to run in Ollama / llama.cpp on a single consumer GPU.

✅ Tested — corrected build runs clean

We download the corrected Q4_K_M GGUF (pe_48b_s_dpo_fix_Q4_K_M.gguf) fresh from this repo and ran it end-to-end in llama.cpp / Ollama: it offloads to GPU, and produces coherent, on-topic, non-degenerate output. (The earlier 8-expert build lost the router bias and could garble; the shipped files are the corrected 2-expert build — see docs/VERIFICATION.md and tools/conversion/pinkelephant.py.) See the 📦 GGUF files table for exactly which hardware each file needs.

Verified aligned benchmarks (bf16, greedy, single NVIDIA RTX PRO 6000 Blackwell — our own infrastructure):

• GSM8K: 68.5%
• MMLU (5-shot): 60.0%

🚀 Start here for clean output: docs/HOW_TO_RUN.md — correct ChatML template + sampler settings. 📘 Read the whole story: docs/ULTIMATE_REPORT.md — full traceability for investors, engineers, and users.

💡 Why a sparse MoE?

Pretraining a 48B model from scratch is expensive — and wasteful, because most of that compute goes to reconstructing what a good dense 14B already knows. Pink Elephant took the smarter route:

• Start with a proven dense brain. A 14B foundation, instruction-tuned on code + math, scoring HumanEval 71.34% and MATH-500 63.40%.
• Upcycle, don't retrain. Split the feed-forward weights into 8 complementary experts, exactly preserving parent behavior at initialization — so you inherit the dense model's knowledge instead of relearning it.
• Sparse compute, dense capacity. Only top-2 of 8 experts activate per token (~30.7%), giving you 47.7B parameters of knowledge with 3.2× fewer FLOPs than a dense 48B.

The result: more capability per flop, per watt, per dollar — on hardware you can actually afford to own.

🏆 The Success Journey

One lineage, six milestones: a proven dense core → upcycled to sparse scale → refined → supervised fine-tuned → preference-aligned → bit-exact merged and converted to GGUF.

🔄 The Lineage

🎬 The Story, Act by Act

Act I — The Dense Core: 🐘 Pink Elephant 14B

pink-elephant-llm-14b

A dense 14.66B decoder-only model, instruction-tuned on 99,661 curated code + mathematics examples (bigcode/self-oss-instruct + open-r1/OpenR1-Math). It proved the thesis:

• HumanEval pass@1: 71.34%
• MATH-500: 63.40%

Act II — The Upcycle: Dense 14B → Sparse 48B MoE

pinkelephant-llm-48b

Rather than train 48B from scratch, Pink Elephant upcycled the 14B foundation into an 8-expert Mixture-of-Experts — feedforward weights split and replicated into eight complementary experts per layer, exactly preserving the parent's behavior at initialization. A sparse brain with dense-scale capacity at ~30.7% activation per token.

Act III — The Refinement: 48B-S

pinkelephant-llm-48b-s

A second full epoch (3,114 steps) on the same curriculum with a freshly restarted cosine schedule. Training loss fell 32% (0.0616 → 0.0419) with validation loss steady near 0.50 — refinement, not overfitting.

Act IV — Supervised Fine-Tuning (SFT)

pinkelephant-llm-48b-s-sft

An industrial-standard QLoRA recipe (frozen 4-bit NF4 base purely as a memory-efficient training vehicle; all math in native bf16; adapters full precision) fine-tuned the whole 47.7B on a single GPU. Training loss fell 0.65 → ~0.44; held-out CE identified step 90,000 as the best adapter (CE 0.5392).

Act V — Direct Preference Optimization (DPO)

pinkelephant-llm-48b-s-dpo

An alignment campaign using a frozen reference + trainable policy (QLoRA-for-DPO). The defining engineering decision: an evidence-based turning-point early stop — we watched the loss curve, found its true minimum 0.4763 at step 11,579, then stopped 1,500 steps later. A clean 2,000-pair held-out preference set shows ~0.65 preference accuracy.

Act VI — The Merge & Conversion (This Repo) 🚀

pinkelephant-llm-48b-s-gguf — you are here

We fused the best SFT adapter (step 90,000) + best DPO adapter (min-loss step 11,579) into the 48B-S base — bit-exact — then converted the non-standard fused-layout MoE to standard GGUF and quantized it.

base → moe → train → sft → dpo → verification → release

🎯 Use cases

Because it is a compact, aligned, MIT-licensed sparse MoE that fits on consumer hardware, Pink Elephant 48B-S GGUF is ideal for:

📌 Specifications

📦 GGUF files — pick the one that fits your machine

✅ Tested (corrected build): the Q4_K_M file below (pe_48b_s_dpo_fix_Q4_K_M.gguf) was downloaded fresh from this repo and run end-to-end in llama.cpp / Ollama — created as a model, offloaded to GPU, and produced coherent, non-degenerate output. It is the corrected 2-expert build (the earlier 8-expert build dropped the router bias and could garble; superseded). See docs/VERIFICATION.md and tools/conversion/pinkelephant.py for the fix.

How to read this:

• Choose Q4_K_M to run on a typical modern machine. It's the file we verified consistently.
  • 8–12 GB GPU → full GPU offload, interactive speed. Best choice for most users.
  • No GPU / 16 GB RAM → runs on CPU, slower but works.
• Choose f16 only if you have a 32 GB-class unified host; otherwise use Q4_K_M.

Note on quality/speed and size: these are the corrected 2-expert (effective) build of the SFT+DPO-aligned 48B-S. It ships only the live experts {0,4} (the 8-expert source has 6 dead experts that were never routed-to), which is why the files are far smaller than the old 8-expert GGUF. Q4_K_M trades a small amount of precision for a much smaller file; f16 preserves full fidelity. Both fix the router-bias bug from the earlier release.

🚀 Run it with Ollama

Prerequisite

Install Ollama from https://ollama.com and confirm it's running:

ollama --version

Option A — Pull directly from Hugging Face

ollama pull hf.co/pinkelephantlimited/pinkelephant-llm-48b-s-gguf:Q4_K_M

Option B — Build from the bundled Modelfile

The repo ships a ready-made Modelfile with the correct ChatML template and safe sampler settings (this matters — see ⚙️ Sampler & format notes below):

FROM ./pe_48b_s_dpo_fix_Q4_K_M.gguf

# Lower temperature + repeat_penalty 1.0 avoid degenerate/garbled output on this MoE.
PARAMETER temperature 0.3
PARAMETER repeat_penalty 1.0
PARAMETER top_p 0.9
PARAMETER num_ctx 4096

# ChatML (OpenHermes-2.5) template — REQUIRED for correct output.
TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

Download pe_48b_s_dpo_fix_Q4_K_M.gguf from the Files tab into this folder, then:

ollama create pe-48b -f Modelfile
ollama run pe-48b "Write a Python function that checks if a number is prime."

Chat via the API

curl http://localhost:11434/api/chat -d '{
  "model": "pe-48b",
  "messages": [{"role": "user", "content": "What is the capital of France?"}],
  "stream": false
}'

Python:

import requests
r = requests.post("http://localhost:11434/api/chat", json={
    "model": "pe-48b",
    "messages": [{"role": "user", "content": "Explain MoE in one sentence."}],
    "stream": False,
})
print(r.json()["message"]["content"])

⚙️ Sampler & format notes (read this before you run)

This model is ChatML / OpenHermes-2.5 aligned (SFT + DPO). Getting clean output depends on two things: the chat template, and conservative sampling.

1. It must be prompted in ChatML form. The bundled Modelfile already sets the correct ChatML TEMPLATE, so plain ollama run pe-48b "..." and /api/chat are handled for you. If you bypass the template and send a bare/raw prompt (e.g. via /api/generate), you must wrap it yourself, exactly like this (note the closing assistant tag):

   <|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n
   

   A bare prompt such as "The capital of France is" will not auto-complete correctly.

2. Use a low temperature and keep repeat_penalty at 1.0. The default Ollama sampler (temperature ~0.7, repeat_penalty ~1.1) makes this model garble / repeat words. The bundled Modelfile pins temperature 0.3, repeat_penalty 1.0, top_p 0.9, which gives stable, non-degenerate output. For the most deterministic results you may set temperature 0. Do not raise temperature much above 0.3 without re-testing.

3. Known robustness note (verified). The raw-completion path with an explicit ChatML prompt (point 1) is the most reliable on this custom MoE — it produced the model's exact correct answer deterministically. The Ollama /api/chat template path generally works, but in some Ollama versions it can be less stable for this architecture; if you ever see odd output via /api/chat, switch to the raw-completion form in point 1 with "temperature": 0 (deterministic) as a reliable fallback.

Tip: prefer the chat template (or the explicit ChatML raw-completion form) rather than bare completion for natural, instruction-following output from this SFT/DPO-aligned model.

🖥️ Hardware fit

• Q4_K_M (8.3 GB): runs fully offloaded on an 8–12 GB GPU (or CPU with ≥ 16 GB RAM). The corrected 2-expert build is small enough for consumer-range cards.
• f16 (29.3 GB): needs ~32 GB VRAM / unified memory.

🔬 Engineering → fully reproducible in this repo

🧪 Full verification report

See docs/VERIFICATION.md for the complete, measurement-by-measurement evidence: bit-exact merge, all-403-tensor integrity, router top-2 routing, conversion fidelity, and the HF → Ollama round-trip.

✅ Verification highlights (measured)

• Bit-exact merge: merged == base + 2·SFT + 2·DPO with max|diff| = 0.000e+00.
• Tensor integrity: all 403 tensors (3 global + 40×10 per-layer) with exact shapes — incl. 3D stacked expert blocks ffn_{gate,up,down}_exps[·,·,8].
• Router check: top-2 selected experts identical to source (overlap 1.0).
• Weights through conversion: o_proj, norms, qkv→q match source within f16 rounding.
• HF → Ollama round-trip: SHA-256 of the downloaded Q4_K_M matches the artifact that generated valid answers in Ollama.

📚 The Pink Elephant Family

🎉 Get started now

# Fastest path — pull straight from Hugging Face
ollama pull hf.co/pinkelephantlimited/pinkelephant-llm-48b-s-gguf:Q4_K_M
ollama run pinkelephant-llm-48b-s-gguf "Explain mixture of experts like I'm five."

Or watch the whole story come alive in the Ultimate Project Report.

License & Commercial Use

MIT License — free for commercial and research use, modification, and redistribution. Use at your own risk.

🦾 Pink Elephant Limited · Pink Elephant 48B-S GGUF · 2026 — from a 14B dense core to a runnable, aligned MoE flagship, in one lineage.