Indonesian GPT-2 finetuned on Indonesian poems
This is the Indonesian gpt2-small model fine-tuned to Indonesian poems. The dataset can be found in here All training was done on Google Colab Jupyter Notebook (soon).
The dataset is splitted into two subset with details belows:
| split | count (examples) | percentage |
|---|---|---|
| train | 7,358 | 80% |
| validation | 1,890 | 20% |
Evaluation results
The model evaluation results after 10 epochs are as follows:
| dataset | train/loss | eval/loss | eval perplexity |
|---|---|---|---|
| id puisi | 3.324700 | 3.502665 | 33.20 |
The logs can be found in wandb page here or tensorboard here