gemma-4-12B-it-OnlyText
Text-only causal language model derived from google/gemma-4-12B-it by removing the vision and audio stacks (their towers/embedders and projector weights) and the multimodal special tokens. The text backbone and LM head are preserved.
Details
- Base model: google/gemma-4-12B-it
- Architecture:
Gemma4UnifiedForCausalLM - Parameters: 11.91B
- Layers: 48 · Hidden size: 3840
- MTP head: not present
- Weights:
bfloat16
Attribution
This model is a derivative of google/gemma-4-12B-it by the Google team, released under the apache-2.0 license. All credit for the underlying weights and capabilities belongs to the original authors; this repository only removes modalities, it does not add new training.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("OnlyTextLLMs/gemma-4-12B-it-OnlyText")
tokenizer = AutoTokenizer.from_pretrained("OnlyTextLLMs/gemma-4-12B-it-OnlyText")