T5-CNN-25K
A fine-tuned T5-small model for abstractive text summarization, trained on 25,000 examples from the CNN/DailyMail dataset.
Model Description
This model is designed to generate concise abstractive summaries from input text.
It is used as the summarization model in the Text Summarizer application, which supports PDF and text document summarization.
Base Model
- Base model:
google-t5/t5-small - Task: Abstractive text summarization
- Training dataset: CNN/DailyMail
- Training examples: 25,000
- Maximum input length: 512 tokens
- Model format: Safetensors
Intended Use
The model is intended for:
- Summarizing documents and articles
- Generating short, medium, and long summaries
- Educational and research projects
- Demonstrations of transformer-based text summarization
Usage
from transformers import T5Tokenizer, T5ForConditionalGeneration
model_name = "vaibhavvanshu/t5-cnn-25k"
tokenizer = T5Tokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)
text = "summarize: Your text goes here."
inputs = tokenizer(
text,
return_tensors="pt",
max_length=512,
truncation=True
)
outputs = model.generate(
**inputs,
max_length=250,
num_beams=4,
early_stopping=True
)
summary = tokenizer.decode(
outputs[0],
skip_special_tokens=True
)
print(summary)
Training
The model was fine-tuned from T5-small using the CNN/DailyMail summarization dataset.
The project contains multiple experiments using different training dataset sizes. This repository contains the 25K training version selected for the final application.
The final application uses hierarchical summarization for longer documents by:
- Splitting large documents into manageable chunks.
- Generating intermediate summaries for each chunk.
- Combining the intermediate summaries.
- Performing additional summarization passes when necessary.
- Producing the final summary according to the requested summary length.
Summary Lengths
The application supports three output lengths:
| Length | Target Maximum |
|---|---|
| Short | 150 tokens |
| Medium | 250 tokens |
| Long | 400 tokens |
These settings are implemented by the application rather than being separate models.
Limitations
Like other abstractive summarization models, this model may:
- Omit important information.
- Produce inaccurate statements.
- Occasionally hallucinate information.
- Perform differently depending on the input domain and writing style.
Generated summaries should therefore be reviewed when accuracy is important.
Dataset
The model was trained using the CNN/DailyMail summarization dataset.
Dataset:
cnn_dailymail
The dataset contains news articles paired with human-written reference summaries.
License
This model is released under the Apache License 2.0.
The model is a fine-tuned version of T5-small. Users should also review the licenses and terms associated with the base model and training dataset when using the model.
Related Project
This model is part of the Text Summarizer application, which consists of:
- React frontend
- FastAPI backend
- T5-based summarization model
- PDF text extraction
- OCR support
- Hierarchical document summarization
The trained model is hosted on Hugging Face and the application can use this repository for model inference.