K2-Horizon-32B Heretic
BF16 weights of a refusal-attenuated derivative of the IFM/K2-Horizon-32B model.
The pipeline is: Heretic refusal attenuation (trial 124: 3/100 refusals, KL divergence 0.056) → LoRA merge into the base BF16 weights. This release is the merged full-precision model with no further training.
The modeling files in this repository include a small fix that returns
hidden states from the model forward pass (required by Heretic).
It does not change inference behavior. trust_remote_code=True is required.
Files
model-00001-of-00064.safetensors … model-00064-of-00064.safetensors
config.json / tokenizer.json / chat_template.jinja / modeling_k2_horizon.py …
| Property | Value |
|---|---|
| Architecture | K2Horizon (custom code, trust_remote_code required) |
| Parameters | 34.8B dense |
| Format | safetensors BF16, 64 shards (~69.6 GiB total) |
| Refusals | 3/100 on harmful_behaviors eval (base model: ~52/100) |
| License | Apache-2.0, inherited from the base model |
Quantized GGUF versions (BF16/Q8_0/Q6_K/Q4_K_M/Q2_K) are available at soyaakinohara/K2-Horizon-32B-heretic-gguf.
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"soyaakinohara/K2-Horizon-32B-heretic",
dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
tok = AutoTokenizer.from_pretrained(
"soyaakinohara/K2-Horizon-32B-heretic",
trust_remote_code=True,
)
Full-precision weights need ~70 GiB of memory. For 2x16 GiB VRAM setups, use the Q4_K_M GGUF with a K2-compatible llama.cpp build instead.
日本語
概要
IFM/K2-Horizon-32B をベースに、Heretic で無検閲化したモデルの BF16 重みです。 trial 124(拒否 3/100、KL 0.056)をマージしたもので、追加学習はしていません。
同梱の modeling ファイルには hidden states を返す小さな修正が入っています。
Heretic 用のもので、推論の動作は変わりません。
使うときは trust_remote_code=True が必要です。
使い方
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"soyaakinohara/K2-Horizon-32B-heretic",
dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
フル精度だと約 70GB 要ります。 VRAM が足りないときは GGUF 版を使ってください。