GPT-2 fine-tuned for excuses (Excusify)
Part of Excusify. Given a short prompt, the model writes an excuse for a situation, conditioned on scenario, urgency and believability.
Prompt format
work | medium | high | forgot to submit the report :
The model writes the completion after the colon and stops with an end-of-text token.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Sohamb2005/gpt2-finetuned-excuses"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
inputs = tokenizer("work | medium | high | forgot to submit the report :", return_tensors="pt")
out = model.generate(**inputs, do_sample=True, top_p=0.92, temperature=0.8, max_new_tokens=60,
pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Training
| Base model | gpt2 (124M parameters) |
| Data | data/excuses.csv: Scenario (work/school/family/social), Situation, Urgency (low/medium/high), Believability (low/medium/high), Excuse |
| Split | 777 train / 87 held-out examples |
| Objective | causal LM loss on the completion and its end-of-text token (prompt masked) |
| Settings | learning rate 5e-05, 5 epochs, batch 8, cosine schedule; chosen from a sweep (best situation match, then lowest loss) |
Evaluation
| Value | |
|---|---|
| Held-out loss (GPT-2 before → this model) | 3.469 → 1.75 |
| Held-out perplexity (GPT-2 before → this model) | 32.1 → 5.8 |
| Situation match, held-out prompts | 100% (hand-written answers: 100%) |
| Situation match, situations not in the data | 100% |
Situation match is the share of sampled answers that mention at least one content word of the situation in the prompt.
Limitations
Made for entertainment and practice messages. The training set is small and hand-written, so the model can repeat phrasings from it, and situations far from the training data may get generic answers.