This is a Maginum-Cydoms-24B fine-tune, produced through P-E-W's Heretic (v1.1.0) abliteration engine merged with the Magnitude-Preserving Orthogonal Ablation PR.
Heretication Results
| Score Metric | Value | Parameter | Value |
|---|---|---|---|
| Refusals | 4/100 | direction_index | 23.52 |
| KL Divergence | 0.0307 | attn.o_proj.max_weight | 1.96 |
| Initial Refusals | 62/100 | attn.o_proj.max_weight_position | 32.76 |
| attn.o_proj.min_weight | 0.31 | ||
| attn.o_proj.min_weight_distance | 18.25 | ||
| mlp.down_proj.max_weight | 1.23 | ||
| mlp.down_proj.max_weight_position | 27.53 | ||
| mlp.down_proj.min_weight | 0.84 | ||
| mlp.down_proj.min_weight_distance | 19.87 |
Degree of Heretication
The Heresy Index weighs the resulting model's corruption by the process (KL Divergence) and its abolition of doctrine (Refusals) for a final verdict in classification.
| Index Entry | Classification | Analysis |
|---|---|---|
 | Absolute Heresy | Less than 10/100 Refusals and 0.10 KL Divergence |
 | Tainted Heresy | Around 25-11/100 Refusals and/or -0.20-0.11 KL Divergence |
 | Impotent Heresy | Anything above 25/100 Refusals and 0.21 KL Divergence |
Note: This is an arbitrary classification inspired by Warhammer 40K, having no tangible indication towards the model's performance.
Maginum-Cydoms-24B
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Methods
This model was merged using the following merge methods:
Models Merged
The following models were included in the merge:
- anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
- TheDrummer/Magidonia-24B-v4.3
- TheDrummer/Precog-24B-v1
- zerofata/MS3.2-PaintedFantasy-v3-24B
- TheDrummer/Cydonia-24B-v4.3
- ReadyArt/4.2.0-Broken-Tutu-24b
- zerofata/MS3.2-PaintedFantasy-v2-24B
Configuration
The following YAML configurations were used to produce this model:
Maginum-Cydoms-S001:
models:
- model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
- model: TheDrummer/Magidonia-24B-v4.3
parameters:
density: 1.0
weight: 1.0
- model: TheDrummer/Precog-24B-v1
parameters:
density: 0.4
weight: 0.6
- model: zerofata/MS3.2-PaintedFantasy-v3-24B
parameters:
density: 0.4
weight: 0.4
merge_method: ties
base_model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
parameters:
normalize: false
int8_mask: false
dtype: float32
tokenizer:
source: union
Maginum-Cydoms-S002:
models:
- model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
- model: TheDrummer/Cydonia-24B-v4.3
parameters:
density: 1.0
weight: 1.0
epsilon: 0.0
- model: ReadyArt/4.2.0-Broken-Tutu-24b
parameters:
density: 0.4
weight: 0.6
epsilon: 0.2
- model: zerofata/MS3.2-PaintedFantasy-v2-24B
parameters:
density: 0.4
weight: 0.4
epsilon: 0.2
merge_method: della
base_model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
parameters:
normalize: false
int8_mask: false
dtype: float32
tokenizer:
source: union
models:
- model: Maginum-Cydoms-S001
- model: Maginum-Cydoms-S002
merge_method: slerp
base_model: Maginum-Cydoms-S001
parameters:
t:
- filter: self_attn
value: [0.3, 0.4, 0.6, 0.4, 0.3, 0.4, 0.6, 0.4, 0.3]
- filter: mlp
value: [0.7, 0.6, 0.4, 0.6, 0.7, 0.6, 0.4, 0.6, 0.7]
- value: 0.5
dtype: float32
out_dtype: bfloat16
tokenizer:
source: union