gpt2-imdb-sentiment-classifier
This model is a fine-tuned version of gpt2 on the imdb dataset. It achieves the following results on the evaluation set:
- Loss: 0.1703
- Accuracy: 0.9394
Model description
More information needed
Intended uses & limitations
This is comparable to distilbert-imdb and trained with exactly the same script
It achieves slightly lower loss (0.1703 vs 0.1903) and slightly higher accuracy (0.9394 vs 0.928)
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.1967 | 1.0 | 1563 | 0.1703 | 0.9394 |
Framework versions
- Transformers 4.18.0
- Pytorch 1.13.1+cu117
- Datasets 2.9.0
- Tokenizers 0.12.1