Gemma-2-2B Temporal Link Classification
Full fine-tuned version of
google/gemma-2-2b-it
on the
fahmidiqbal/tlink-classification
dataset for temporal relation classification.
Task
Given an already-formatted task prompt containing two marked spans
(such as <e1>, <e2>, <t1>, or <t2>), the model generates one
temporal-relation label:
BEFORE— first span occurs earlier than the secondAFTER— first span occurs later than the secondOTHER— overlap or another non-ordering relationNONE— no clear temporal relation
Training Details
| Parameter | Value |
|---|---|
| Base model | google/gemma-2-2b-it |
| Fine-tuning | Full (all parameters) |
| Epochs | 3 |
| Batch size | 8 |
| Learning rate | 2e-5 |
| Weight decay | 0.01 |
| Max length | 2048 |
| Precision | bfloat16 |
| Prompt format | Gemma chat template |
| Dataset | fahmidiqbal/tlink-classification |
Results
Test metrics have not been added yet. Run the evaluation script first to
generate test_metrics.json, then run this upload script again.
Usage
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
MODEL_ID = "AnirbanSaha/gemma2-2b-tlink"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
MODEL_ID,
dtype=torch.bfloat16,
device_map="auto",
)
model.eval()
prompt = "Your already-formatted temporal relation prompt here..."
messages = [
{"role": "user", "content": prompt}
]
formatted_prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
enc = tokenizer(
formatted_prompt,
add_special_tokens=False,
return_tensors="pt",
).to(model.device)
with torch.inference_mode():
out = model.generate(
**enc,
max_new_tokens=8,
do_sample=False,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id,
)
new_tokens = out[0][enc["input_ids"].shape[1]:]
prediction = tokenizer.decode(
new_tokens,
skip_special_tokens=True,
).strip()
print(prediction)
Base Model License
The base checkpoint uses the Gemma license. Users of this fine-tuned model should review and comply with the applicable Gemma terms.