gemma-4-e2b-ltx-2-5-prompt
Full bf16 fine-tune of google/gemma-4-E2B-it that converts a story into a single highly detailed LTX 2.5 cinematic video prompt — the most emotionally intense, visually compelling scene of the story, rendered as a shot-by-shot prompt (camera, lighting, motion, lens, atmosphere).
Model description
- Base model:
google/gemma-4-E2B-it(Gemma 4 E2B IT). - Fine-tuning method: full bf16 fine-tune (no quantization, no
adapter). The base model loads as
Gemma4ForConditionalGeneration(multimodal); the vision/audio towers are frozen and only the text decoder (language_model+lm_head) is trained — this is a text-only model despite the multimodal checkpoint. - Compute dtype: bf16.
- Task: text generation — story → LTX 2.5 prompt.
Training data
Chat-formatted examples (data/dataset.jsonl), one conversation per
example: a leading instruction + the story as the user turn, and the
target LTX 2.5 prompt as the final assistant turn. The prompt
(instruction + story, joined as {instruction}\n\n### CONTEXT\n{story})
is masked to -100, so the loss is computed only on the assistant
completion. Examples are pre-tokenized by prepare_dataset.py; the
assistant turn is located by its start marker (a special token derived
from the generation-prompt diff), which sidesteps TRL's fragile
prompt/completion prefix check.
- Train examples: 8958
- Validation examples: 497
Training hyperparameters
| Parameter | Value |
|---|---|
| Epochs | 1.0 |
| Max length | 4096 |
| Micro batch | 16 |
| Gradient accumulation | 1 |
| Effective batch | 16 |
| Learning rate | 0.0001 |
| LR schedule | cosine (~3% warmup) |
| Weight decay | 0.01 |
| Max grad norm | 0.3 |
| Optimizer | adamw_torch_fused |
| Gradient checkpointing | True |
| Precision | bf16 |
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "autopostflow/gemma-4-e2b-ltx-2-5-prompt"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo,
dtype=torch.bfloat16,
device_map="auto",
attn_implementation="sdpa",
).eval()
instruction = "<your system instruction — You are an expert cinematographer…>"
story = "<the story>"
user_content = f"{instruction}\n\n### CONTEXT\n{story}"
messages = [{"role": "user", "content": user_content}]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt",
return_dict=True,
)
inputs = {k: v.to("cuda") for k, v in inputs.items()}
with torch.inference_mode():
out = model.generate(
**inputs,
max_new_tokens=4096,
do_sample=True,
temperature=0.9,
top_p=0.95,
top_k=64,
repetition_penalty=1.05,
pad_token_id=tokenizer.pad_token_id,
)
prompt = tokenizer.decode(
out[0, inputs["input_ids"].shape[1]:],
skip_special_tokens=True,
).strip()
print(prompt)
The prompt layout (### CONTEXT separator) must match what the model
was trained on — keep it consistent at inference.
Intended use
Generate LTX 2.5 video prompts from narrative text. Feed the model's output to an LTX 2.5 video generator as the text prompt.
License
Derivative of google/gemma-4-E2B-it, distributed under the
Gemma Terms of Use. See the Gemma license for terms and acceptable
use policies.