Facial Expression Recognition — Fine-tuned ViT
This model is a fine-tuned version of motheecreator/vit-Facial-Expression-Recognition.
It achieves the following results on the evaluation set:
- Loss: 0.3936
- Accuracy: 0.8675 (86.75%)
Model description
This model is based on a Vision Transformer (ViT) architecture fine-tuned for facial expression recognition.
The associated research, Facial Expression Recognition with CNNs and Vision Transformers: Full Images, Landmarks, and a Hybrid Method on FER-2013, FER+ and RAF-DB, investigates different approaches to facial expression recognition using Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs).
The study evaluates full-face images, facial landmarks, and hybrid representations across three benchmark datasets: FER-2013, FER+, and RAF-DB.
Intended uses & limitations
The model is intended for research and experimentation in facial expression recognition.
Performance may vary depending on image quality, lighting conditions, facial orientation, occlusions, and differences between datasets.
Training and evaluation data
The specific training dataset was not identified in the automatically generated model card.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 512
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 1000
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 0.5969 | 0.34 | 100 | 0.3997 | 0.8652 |
| 0.6090 | 0.67 | 200 | 0.3994 | 0.8644 |
| 0.6038 | 1.01 | 300 | 0.3969 | 0.8677 |
| 0.5819 | 1.35 | 400 | 0.3947 | 0.8674 |
| 0.5864 | 1.69 | 500 | 0.3936 | 0.8675 |
| 0.5819 | 2.02 | 600 | 0.3925 | 0.8661 |
| 0.5694 | 2.36 | 700 | 0.3961 | 0.8656 |
| 0.5618 | 2.70 | 800 | 0.3994 | 0.8650 |
Framework versions
- Transformers 4.36.0
- PyTorch 2.0.0
- Datasets 2.1.0
- Tokenizers 0.15.0
Citation
If you use this model in your research, please cite the following publication:
Casanova, D., Hoffmann, A. B. G., Candido Junior, A., Rodrigues, P. J., & de Paula Filho, P. L. (2026). Facial Expression Recognition with CNNs and Vision Transformers: Full Images, Landmarks, and a Hybrid Method on FER-2013, FER+ and RAF-DB. In VI International Conference on Optimization, Learning Algorithms and Applications (OL2A 2026): Book of Abstracts (p. 36). Instituto Politécnico de Bragança.
Publication: OL2A 2026 — Book of Abstracts
BibTeX
@inproceedings{casanova2026facial,
author = {Casanova, Daniel and
Hoffmann, Alessandra B. G. and
Candido Junior, Arnaldo and
Rodrigues, Pedro Jo{\~a}o and
de Paula Filho, Pedro Luiz},
title = {Facial Expression Recognition with {CNNs}
and Vision Transformers: Full Images,
Landmarks, and a Hybrid Method on
{FER-2013}, {FER+} and {RAF-DB}},
booktitle = {VI International Conference on Optimization,
Learning Algorithms and Applications
({OL2A} 2026): Book of Abstracts},
year = {2026},
pages = {36},
publisher = {Instituto Polit{\'e}cnico de Bragan{\c{c}}a},
isbn = {978-972-745-373-3},
url = {http://hdl.handle.net/10198/36787},
note = {Conference abstract}
}