HeadlineGPT
HeadlineGPT is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct specialized for generating concise, engaging titles from source content.
It is designed for:
- News and article headlines
- Research and academic content
- Talks and presentations
- Social media posts
- Other short-form content that needs an attention-grabbing title
Model Details
| Property | Value |
|---|---|
| Base model | Qwen2.5-1.5B-Instruct |
| Fine-tuning | LoRA / PEFT |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Language | English |
| Training objective | Reward-weighted supervised fine-tuning |
Training
The model was trained on content–title pairs.
Instead of treating every training example equally, examples were weighted according to their associated engagement score. Higher-scoring examples therefore contribute more strongly to the training loss.
This approach is intended to encourage title characteristics associated with higher engagement while retaining the broader writing patterns present in the training data.
The model was trained for 2 epochs on a randomly sampled 25,000-example subset of the larger dataset.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"csankalp21/headlinegpt",
torch_dtype=torch.float16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
"csankalp21/headlinegpt"
)
messages = [
{
"role": "system",
"content": "You are an expert at writing highly engaging titles."
},
{
"role": "user",
"content": (
"Generate a high-engagement title for the following content:\n\n"
"<your content here>"
)
}
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer(
text,
return_tensors="pt"
).to(model.device)
with torch.no_grad():
output = model.generate(
**inputs,
max_new_tokens=40,
temperature=0.7,
do_sample=True,
top_p=0.9,
repetition_penalty=1.1
)
generated_tokens = output[0][inputs["input_ids"].shape[1]:]
print(
tokenizer.decode(
generated_tokens,
skip_special_tokens=True
)
)