Gemma 4 12B IT Roleplay SFT — Epoch 2 (Merged BF16)
Text roleplay SFT derivative of Google Gemma 4 12B IT, released by ChatoyantAI. This is checkpoint-6126, not checkpoint-3600.
Release variants
- Standalone BF16 model: base weights plus the merged LoRA; no adapter is needed.
- LoRA adapter: requires the original
google/gemma-4-12B-itbase.
Training and scope
| Item | Value |
|---|---|
| Architecture | Gemma4UnifiedForConditionalGeneration |
| Global step / completed epoch | 6126 / 2.0 |
| Adaptation | LoRA rank 32, alpha 64, dropout 0 |
| Target modules | Language-model attention and MLP projections |
| Training framework | Unsloth with PEFT |
| Training data | Internal filtered roleplay conversations; not included in this release |
| Supervised target | Final target assistant response; earlier conversation is context |
| Final logged validation loss | 1.06754386 |
Validation loss is training-run telemetry, not a public benchmark or a claim of human preference improvement. No external benchmark accuracy is claimed. This release targets text conversations. Multimodal modules are retained from the base architecture, but multimodal quality has not been evaluated after SFT. Training data may include mature themes; outputs require appropriate review. The model may generate inaccurate, biased, inappropriate, or memorized content.
Context and template
The checkpoint tokenizer and chat template are included. Preserve the order
system → assistant greeting → user → assistant history … → user as applicable.
Do not replace this with another Gemma generation's template.
The deployed adapter service is configured for 65,536 total input plus output tokens.
The base architectural context setting is preserved; long-context quality after
fine-tuning has not been independently benchmarked.
Usage
Requires a Transformers release with Gemma4Unified support. Export environment: Transformers 5.10.1, PEFT 0.18.1, PyTorch 2.11.0+cu130. Use suitable CUDA wheels for your machine.
import torch
from transformers import Gemma4UnifiedForConditionalGeneration, AutoTokenizer
model_id = "ChatoyantAI/gemma-4-12b-it-roleplay-sft-epoch2-bf16"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = Gemma4UnifiedForConditionalGeneration.from_pretrained(
model_id, dtype=torch.bfloat16, device_map="auto"
)
messages = [
{"role": "system", "content": "You are a friendly cafe host. Reply in English."},
{"role": "assistant", "content": "Welcome! Would you like a table by the window?"},
{"role": "user", "content": "Yes, please. Could I have some tea?"},
]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt"
).to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
For the standalone BF16 variant, an example vLLM launch is:
vllm serve ChatoyantAI/gemma-4-12b-it-roleplay-sft-epoch2-bf16 --dtype bfloat16 --max-model-len 65536 --served-model-name roleplay
This command requires sufficient GPU memory and a vLLM version supporting the architecture. A 64K configuration is not a guarantee of any particular concurrency. No FP8/INT4 quantization is applied to this release.
Provenance and checks
release-manifest.json records source weight hashes, adapter configuration,
export versions, merge validation and file hashes. Training examples, optimizer
states, raw run logs and credentials are deliberately excluded.
License and attribution
Apache License 2.0, following the original base model license. See LICENSE,
NOTICE, and BASE_MODEL_README.md for the original model attribution.
The model weights were modified by merging the roleplay LoRA (standalone variant)
or are provided as the corresponding separately trained adapter (LoRA variant).
This derivative is not an official Google release.