gemma-4-12B-uncensored-opus4.7-cot

A QLoRA fine-tune of an uncensored gemma-4-12B-it (abliteration-derived), distilled from Claude Opus 4.7 chain-of-thought traces. The idea was to see how much of the capability loss caused by abliteration could be recovered by training the model to reason in a more structured, deliberative style, without restoring refusal.

The merged model is provided here in fp16 safetensors.

Benchmarks

Evaluated with lm-evaluation-harness. MMLU and GSM8K use the chat template (multi-turn few-shot), since the bare loglikelihood mode noticeably underrates models with a thinking template on this architecture. By community request the table now includes the abliterated pre-SFT model, so the full base → abliterated → SFT trajectory is visible.

Every metric tells the same story: abliteration degrades capability and the CoT fine-tune recovers it. On MMLU the SFT closes ~73% of the gap abliteration opened (0.635 → 0.739); on GSM8K abliteration roughly halves math ability and the fine-tune nearly fully restores it (0.496 → 0.920); on WikiText-2 perplexity the SFT model even edges below the clean base (1.717 vs 1.834 bits/byte).

Notes:

• Run in bfloat16 (gemma overflows in fp16, producing degenerate output); gemma4_unified requires a recent transformers.
• GSM8K is reported as flexible-extract. The abliterated model writes prose answers rather than the #### N string and degenerates without the chat template + a generous generation budget, so strict-match understates it (abliterated strict-match = 0.216).
• The three perplexity figures use identical methodology (lm-eval wikitext, rolling loglikelihood); bits/byte is the tokenizer-independent, cross-model comparable number.
• base/SFT MMLU & GSM8K are the original eval; the abliterated row and all perplexity numbers were added in the strengthened run.

Usage

Trained to think out loud. A useful system prompt:

You are a reasoning assistant. Think step by step, then give your final
answer on a clearly marked last line beginning with "Final answer:".

Allow at least 768 generation tokens — shorter budgets cut off chains of thought mid-derivation and make the model look worse than it is.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

repo = "Rangle2/gemma-4-12B-uncensored-opus4.7-cot"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16,
                                             device_map="auto")

Limitations

Trained on STEM-style verbal reasoning traces, so gains are concentrated there. Code generation regresses a little compared to the clean base — the model's outputs got more verbose, which is the wrong shape for code. Tool use, long-context retrieval and non-English usage were not in the training set and are unevaluated. The underlying abliterated direction is inherited: the model is overconfident and rarely defers.

Safety-style phrases ("the safe answer is to explain…") still show up inside chains of thought, but the model proceeds to answer anyway. This is the expected deliberate-then-comply pattern of abliterated models, not real alignment — don't read those phrases as a guardrail.

Disclaimer

This model has had its refusal behavior aggressively removed and will attempt to answer prompts that a standard instruction-tuned model would correctly decline. It is released for research, red-teaming and interpretability work.

It is provided as is, with no warranty of any kind, and the author disclaims all liability for any direct or indirect damage arising from its use, misuse or redistribution. You are solely responsible for the prompts you send to it, the outputs it produces for you, and any downstream use of those outputs. You must comply with all laws applicable to you and to any users you expose this model to, and with the Gemma Terms of Use of the upstream Google model.

Do not deploy this model to end users without your own safety layer (input filtering, output classification, human review). Outputs may be wrong, biased, offensive or unsafe; do not rely on them for medical, legal, financial or safety-critical decisions.

By downloading or using this model, you accept all of the above.

Training

• Base: uncensored gemma-4-12B-it (abliteration-derived).
• Teacher data: Claude Opus 4.7 chain-of-thought traces (eddieran/opus-4.7-reasoning-cot).
• QLoRA, r=16, α=32 on q_proj/v_proj, bf16 compute, 4-bit NF4 base, 2 epochs, max_len=3072, paged-AdamW-8bit, single A100-80GB.
• Adapter (~40 MB) merged into the base in fp16; this repo carries the merged weights.