SciHigh 2026 Task 2 — PEGASUS-XSum Title Generation Model
This repository contains a fine-tuned PEGASUS-XSum model for Task 2: Title Generation from Abstracts of the SciHigh 2026 shared task at FIRE 2026.
The model takes a scientific paper abstract as input and generates a concise research-paper title.
Base Model
- Model:
google/pegasus-xsum - Architecture: PEGASUS encoder-decoder Transformer
- Parameters: approximately 570 million
- Framework: Hugging Face Transformers
Task
Input: Scientific paper abstract Output: Generated scientific paper title
PEGASUS is designed for abstractive summarization, making it suitable for highly compressed generation tasks such as generating a paper title from an abstract.
Dataset
The model was fine-tuned on the SpringerSSAT dataset released for SciHigh 2026 Task 2.
Dataset split used:
- Training: 2,778 abstract-title pairs
- Validation: 347 abstract-title pairs
- Test: 348 abstracts with masked reference titles
Only the training split was used for gradient updates. The validation set was used for evaluation and best-checkpoint selection.
Preprocessing
- Maximum abstract length: 512 tokens
- Maximum target-title length: 64 tokens
- Input truncation: enabled
- Target truncation: enabled
- Dynamic batch padding using
DataCollatorForSeq2Seq - No task-specific instruction prefix was added
- Abstracts were provided directly to PEGASUS
Fine-Tuning Configuration
- Epochs: 3
- Learning rate:
3e-5 - Weight decay:
0.01 - Per-device training batch size: 1
- Per-device evaluation batch size: 1
- Gradient accumulation steps: 8
- Effective training batch size: 8
- Mixed-precision training: FP16
- Gradient checkpointing: enabled
- Evaluation strategy: once per epoch
- Checkpoint saving strategy: once per epoch
- Generation beam size during validation: 4
- Maximum generation length: 64
- Best-model selection metric: ROUGE-L
- Best model automatically restored after training
Validation Results
| Epoch | Validation Loss | ROUGE-1 | ROUGE-2 | ROUGE-L | ROUGE-Lsum |
|---|---|---|---|---|---|
| 1 | 2.632621 | 42.9633 | 20.7970 | 36.6800 | 36.5897 |
| 2 | 2.586069 | 43.4483 | 21.3048 | 36.9716 | 36.8742 |
| 3 | 2.579055 | 43.6198 | 21.2562 | 37.0629 | 36.9731 |
Best Checkpoint
The best model was obtained at Epoch 3.
- ROUGE-1: 43.6198
- ROUGE-2: 21.2562
- ROUGE-L: 37.0629
- ROUGE-Lsum: 36.9731
- Validation loss: 2.579055
A separate evaluation over all 347 validation examples reproduced these results.
These are validation-set results and are not official hidden-test scores.
Example Usage
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
model_name = "kckrish21/SciHigh2026-Task2-PEGASUS-XSum"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
abstract = """
Insert a scientific paper abstract here.
"""
inputs = tokenizer(
abstract,
return_tensors="pt",
max_length=512,
truncation=True
)
generated_ids = model.generate(
**inputs,
max_length=64,
num_beams=4,
early_stopping=True
)
title = tokenizer.decode(
generated_ids[0],
skip_special_tokens=True
)
print(title)
Test-Set Generation
For SciHigh 2026 test inference:
- Test examples: 348
- Generation batch size: 4
- Beam size: 4
- Maximum generation length: 64
- Early stopping: enabled
- Missing predictions: 0
- Empty predictions: 0
The reference test titles are masked by the task organizers, so no test-set metric is reported here.
Intended Use
This model is intended for:
- scientific paper title generation from abstracts
- research on scientific text generation
- participation and reproducibility for SciHigh 2026 Task 2
Generated titles should be treated as model suggestions rather than authoritative paper titles.
Limitations
The model was fine-tuned on a relatively small dataset of social-science research abstracts. Performance may differ for scientific domains or writing styles that are substantially different from the SpringerSSAT training distribution.
Like other neural text-generation systems, the model may generate titles that are incomplete, overly generic, or not fully supported by the source abstract.
Shared Task
SciHigh 2026 — Research Highlight Generation from Scientific Papers FIRE 2026 Subtask 2: Title Generation from Abstracts
Ranking for the shared task is primarily based on ROUGE-L F1.