atomic9b-mix30x
Merged BF16 checkpoint of atomic9b after two SFT passes on petruhonk/Qwen3.8-9B-Distill-uncensored-heretic (Qwen3.5 9B, hidden_size=4096, 32 layers, 262k context).
Trained on an RTX 4090 (loki) with Unsloth QLoRA (r=16, alpha=32), then LoRA-merged to BF16.
Double train
| Pass | Data | Steps / notes |
|---|---|---|
| 1 | PME synthetic mix (~2570 rows) | Adapter at lora-mac-pme-2570 |
| 2 | mix30x ChatML corpus (252,401 train / 21,941 eval), 1 epoch | Adapter at lora (checkpoint-31602), train loss 0.4249 |
This repo is the pass-2 LoRA merged onto the heretic 9B base (merged16-cpu). It is a causal LM (Qwen3_5ForCausalLM / qwen3_5_text), not the vision tower.
Files
model.safetensors— 17 GB BF16 mergeconfig.json,generation_config.json,tokenizer.json,tokenizer_config.json,chat_template.jinja
A Q8_0 GGUF sibling was also exported on Loki as Qwen3.8-9B-Distill-Heretic-Uncensored-Q8_0-mix30x.gguf (sha256 c755603a8d85516b70ac55675b36cdf41f760b4a4a3a07c98b461c3b63222cd6).
Training hyperparams (pass 2)
- seq_len 2048, batch 1, grad_accum 8, packing on
- lr 2e-4, warmup 200, 1 epoch
- mask_think, upweight_tools, think_frac 0.75 / direct_frac 0.25
- vision layers frozen
Load
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "pressatojump/atomic9b-mix30x"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto", trust_remote_code=True)
License
Apache-2.0, same family as the Qwen 3.5 / heretic base. Uncensored / heretic lineage — no extra safety fine-tune was added.