📝 Dialogue Summarizer using FLAN-T5
This model is a fine-tuned version of google/flan-t5-base (replace with flan-t5-small if that's what you used) for the task of dialogue summarization. It generates concise summaries from conversational dialogues while preserving the key information.
🚀 Model Details
- Base Model: google/flan-t5-base
- Task: Dialogue Summarization
- Framework: PyTorch
- Library: Hugging Face Transformers
- Fine-tuning Dataset: SAMSum Dialogue Summarization Dataset
📊 Dataset
The model was fine-tuned on the SAMSum Dataset, which contains messenger-style conversations paired with human-written summaries.
Example:
Input Dialogue
Amanda: Are we still meeting tomorrow?
Jerry: Yes, let's meet at 10 AM.
Amanda: Perfect. See you then!
Generated Summary
Amanda and Jerry confirm their meeting for tomorrow at 10 AM.
🧠 Training Details
- Epochs: 10
- Batch Size: 8
- Weight Decay: 0.01
- Warmup Steps: 500
- Optimizer: AdamW (Transformers default)
- Maximum Input Length: 512 tokens
- Maximum Summary Length: 150 tokens
💻 Usage
Install dependencies:
pip install transformers torch
Load the model:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
model_name = "Ayush5605/dialogue-summarizer"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
Generate a summary:
dialogue = """
Amanda: Are we still meeting tomorrow?
Jerry: Yes, let's meet at 10 AM.
Amanda: Perfect. See you then!
"""
inputs = tokenizer(
dialogue,
return_tensors="pt",
max_length=512,
truncation=True
)
summary_ids = model.generate(
**inputs,
max_length=150,
num_beams=4,
early_stopping=True
)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
print(summary)
📈 Intended Use
This model is intended for:
- Dialogue summarization
- Chat transcript summarization
- Meeting summarization
- Customer support conversation summarization
- Educational and research purposes
⚠️ Limitations
- Performance depends on the quality and style of the input dialogue.
- Very long conversations (>512 tokens) are truncated.
- The model may omit minor conversational details while generating concise summaries.
- It has been fine-tuned primarily on English dialogue data.
📚 Training Framework
- Python
- PyTorch
- Hugging Face Transformers
- Hugging Face Datasets
👨💻 Author
Ayush Auti
- GitHub: https://github.com/Ayush5605
- Hugging Face: https://huggingface.co/Ayush5605
📄 Citation
If you use this model in your work, please cite the original FLAN-T5 paper and the SAMSum dataset.
@article{flan2022,
title={Scaling Instruction-Finetuned Language Models},
author={Chung, Hyung Won and others},
year={2022}
}
@inproceedings{samsum,
title={SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization},
author={Gliwa et al.},
year={2019}
}