Model Summary
Darkhn-M3.2-36B-Animus-V12.0-Heretic-Absolute is a fine-tuned language model resulting from the Heretic repository and optimization methodology, built upon Darkhn's highly capable M3.2-36B-Animus-V12.0 architecture and maintained by the Silicone-Moss repository. It utilizes a targeted vector intervention technique (orthogonalization) tuned via Optuna to suppress refusal responses while maintaining exceptional coherence and probability distribution (KL Divergence).
This specific checkpoint represents Trial 177 KL Divergence of 0.0200. indicating high adherence to the base model's probability distribution. The trade-off is a non-zero refusal rate (8 refusals logged in the test set), making it a highly intelligent but with enough friction to keep the prose high quality.
Run Configuration: "Trial 177"
The following parameters define the intervention vector applied to the model. This configuration was discovered during a deep hyperparameter search, revealing an aggressive scaling strategy on the attention outputs while maintaining a light touch on the MLPs.
Optimization Results
| Metric | Value | Description |
|---|---|---|
| Refusals | 8/100 | The model refused a minimal subset of prompts in the Heretic test set, trading absolute compliance for high coherence. |
| KL Divergence | 0.0200 | Measures deviation from the base model's probability distribution. A score this low indicates exceptional preservation of general knowledge. |
| Trial ID | 177 | Specific Optuna trial identifier. |
| Direction Scope | Per Layer | Intervention vectors were calculated and applied on a per-layer basis. |
Intervention Parameters
Interventions were applied to two primary distinct layers: the Attention Output Projection (attn.o_proj) and the MLP Down Projection (mlp.down_proj). The data shows Optuna heavily penalizing the attention mechanism deep in the network while barely whispering to the MLP.
| Parameter Scope | Setting | Value |
|---|---|---|
| Attention Output | attn.o_proj.max_weight | 3.407 |
| (attn.o_proj) | attn.o_proj.max_weight_position | 42.08 (Layer Depth) |
| attn.o_proj.min_weight | 2.578 | |
| attn.o_proj.min_weight_distance | 21.90 | |
| MLP Down Proj | mlp.down_proj.max_weight | 0.509 |
| (mlp.down_proj) | mlp.down_proj.max_weight_position | 35.70 (Layer Depth) |
| mlp.down_proj.min_weight | 0.139 | |
| mlp.down_proj.min_weight_distance | 21.75 |
Methodology & Definitions
To ensure uniform understanding of the Heretic run data, the following definitions apply to the parameters listed above:
- Direction Scope: Defines whether the refusal vector is calculated once for the entire model ("Global") or recalculated individually for each layer ("Per Layer"). "Per Layer" allows for precise removal of refusal mechanisms without damaging general knowledge.
- Max Weight: The maximum scaling factor applied to the intervention vector. A higher weight indicates a stronger "push" against the targeted concept (refusal) at the peak layer. Note the massive 3.4x weight applied to the attention projection in this run.
- Max Weight Position: The specific layer index (depth) where the intervention is strongest.
- Observation: The intervention peak sits deep in the model (Layers 35-42), confirming that higher-order reasoning and refusal circuitry coalesce late in the Animus architecture.
- Min Weight: The baseline scaling factor applied to the intervention vector at the periphery of the target zone.
- Distance: The "spread" or bandwidth of the intervention. It determines how many layers around the "Max Weight Position" are affected by the vector modification.
Usage & Limitations
- Intended Use: Research into model alignment, vector arithmetic, and uninhibited creative writing.
- Risks: As an "Absolute" variant, this model has had most of its safety guardrails stripped via heretic intervention. It may still hallucinate or diverge from logical consistency.
Credits & References
This research builds upon the excellent work of the open-source AI community:
-
Repository / Distribution: Silicone-Moss
-
Base Model: M3.2-36B-Animus-V12.0 by the skilled mind of Darkhn.
-
Methodology: Heretic by p-e-w.
-
Original Model card and support links below