ANNOUNCEMENT:
!!! The previous pipeline has been destroyed by a system update of hugging face. !!! In the updated pipeline, the generated text may show like:
Today was a beautiful day, I have done all my homework. I wonder is this I need? Thank you so much for your help.TL;DRThank You
please filter the result by TL;DR.
Model Description
This is the project for the course Multimedia System in Leiden University 2023 fall. The model is part of self-designed software for the final project that summarizes short texts. The original summarization model is from https://towardsdatascience.com/text-summarization-with-gpt2-and-layer-ai-599625085d8e .
Training Datasets
- Amazon review dataset https://www.kaggle.com/datasets/kritanjalijain/amazon-reviews?select=amazon_review_polarity_csv.tgz
- Twitter crawler dataset https://www.kaggle.com/datasets/tripathiharsh/training
- Emotion analysis dataset https://huggingface.co/datasets/dair-ai/emotion
- Kindle review dataset https://www.kaggle.com/datasets/meetnagadia/amazon-kindle-book-review-for-sentiment-analysis/data
TODO
- Improve training data quality
- Model analysis
Notification
Due to the technique support of hugging face, this project can only functioning with Interface code below. When you first call the API, it may take more than 1 minutes to download the model and functioning. Please wait carefully, thank you!
Fine-tune Code
You can check "text_summary.py" in the uploaded documents, and download the model to test and modify this task.
Interface(legacy)
please see the announcement above.
import requests
API_URL = "https://api-inference.huggingface.co/models/Lin0He/text-summary-gpt2-short"
headers = {"Authorization": "Bearer <Your API tokens>"}
def query(payload):
response = requests.post(API_URL, headers=headers, json=payload)
return response.json()
output = query({
"inputs": "Today was a beautiful day. I have down all my homework, and baked some cookies with my friends. What a lovely day."
})
print(output)