This is a Harbinger-24B fine-tune, produced through P-E-W's Heretic (v1.1.0) abliteration engine merged with the Magnitude-Preserving Orthogonal Ablation PR.
Note: I am also unsure if there was any point to abliterating this model. The original model does not include a jinja chat template.
Heretication Results
| Score Metric | Value | Parameter | Value |
|---|---|---|---|
| Refusals | 4/100 | direction_index | 17.41 |
| KL Divergence | 0.0210 | attn.o_proj.max_weight | 1.43 |
| Initial Refusals | 98/100 | attn.o_proj.max_weight_position | 33.59 |
| attn.o_proj.min_weight | 0.91 | ||
| attn.o_proj.min_weight_distance | 22.15 | ||
| mlp.down_proj.max_weight | 1.12 | ||
| mlp.down_proj.max_weight_position | 23.83 | ||
| mlp.down_proj.min_weight | 0.92 | ||
| mlp.down_proj.min_weight_distance | 22.75 |
Degree of Heretication
The Heresy Index weighs the resulting model's corruption by the process (KL Divergence) and its abolition of doctrine (Refusals) for a final verdict in classification.
| Index Entry | Classification | Analysis |
|---|---|---|
 | Absolute Heresy | Less than 10/100 Refusals and 0.10 KL Divergence |
 | Tainted Heresy | Around 25-11/100 Refusals and/or -0.20-0.11 KL Divergence |
 | Impotent Heresy | Anything above 25/100 Refusals and 0.21 KL Divergence |
Note: This is an arbitrary classification inspired by Warhammer 40K, having no tangible indication towards the model's performance.
Harbinger-24B
Like our Wayfarer line of finetunes, Harbinger-24B was designed for immersive adventures and other stories where consequences feel real and every decision matters. Training focused on enhancing instruction following, improving mid-sequence continuation, and strengthening narrative coherence over long sequences of outputs without user intervention. The same DPO (direct preference optimization) techniques used in our Muse model were applied to Harbinger, resulting in polished outputs with fewer clichés, repetitive patterns, and other common artifacts.
If you want to easily try this model, you can do so at https://aidungeon.com. Note that Harbinger requires a subscription while Muse and Wayfarer Small are free.
We plan to continue improving and open-sourcing similar models, so please share any and all feedback on how we can improve model behavior. Below we share more details on how Muse was created.
Quantized GGUF weights can be downloaded here.
Model details
Harbinger 24B was trained in two stages, on top of Mistral Small 3.1 Instruct.
SFT - Various multi-turn datasets from a multitude of sources, focused on Wayfarer-style text adventures and general roleplay, each carefully balanced and rewritten to be free of common AI cliches. A small single-turn instruct dataset was included to send a stronger signal during finetuning.
DPO - Reward Model User Preference Data, detailed in our blog - This stage refined Harbinger's narrative coherence while preserving its unforgiving essence, resulting in more consistent character behaviors and smoother storytelling flows.
Inference
Mistral Small 3.1 is sensitive to higher temperatures, so the following settings are recommended as a baseline. Nothing stops you from experimenting with these, of course.
"temperature": 0.8,
"repetition_penalty": 1.05,
"min_p": 0.025
Limitations
Harbinger was trained exclusively on second-person present tense data (using “you”) in a narrative style. Other styles will work as well but may produce suboptimal results.
Prompt Format
ChatML was used during all training stages.
<|im_start|>system
You're a masterful storyteller and gamemaster. Write in second person present tense (You are), crafting vivid, engaging narratives with authority and confidence.<|im_end|>
<|im_start|>user
> You peer into the darkness.
<|im_start|>assistant
You have been eaten by a grue.
GAME OVER
Credits
Thanks to Gryphe Padar for collaborating on this finetune with us!