GAMEBERT
This model is a fine-tuned version of google/bert_uncased_L-2_H-128_A-2 on a custom game commands dataset. It achieves the following results on the evaluation set:
- Loss: 0.0409
- Precision: 0.9692
- Recall: 0.9702
- F1: 0.9697
- Accuracy: 0.9900
Model description
A fine tuned tiniest available BERT meant for low latency tagging of text for purposes such as identifying target and action to give to a game from a sentence.
Intended uses & limitations
This model is made to be used by the PyPi package voice-speak-up - [https://pypi.org/project/voice-speak-up/]. It is part of the second stage of the pipeline and aims to identify actions, targets, and corrections in user spoken sentences transcribed by whisper.
Training and evaluation data
Custom chatette based dataset - [https://github.com/SimGus/Chatette]
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 4
Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|---|---|---|---|---|---|---|---|
| 0.2531 | 1.0 | 2250 | 0.1332 | 0.9098 | 0.9078 | 0.9088 | 0.9707 |
| 0.1138 | 2.0 | 4500 | 0.0631 | 0.9592 | 0.9552 | 0.9572 | 0.9853 |
| 0.0812 | 3.0 | 6750 | 0.0443 | 0.9683 | 0.9698 | 0.9690 | 0.9895 |
| 0.0733 | 4.0 | 9000 | 0.0409 | 0.9692 | 0.9702 | 0.9697 | 0.9900 |
Framework versions
- Transformers 5.14.1
- Pytorch 2.13.0+cu130
- Datasets 5.0.0
- Tokenizers 0.22.2