BERTimbau-base-text-filter
📣 UPDATE: If you like/use Tucanos or GigaVerbo for anything, know that version 2 is already out. Go to our new organization (Polyglot) to check out all the improvements the Tucano 2 series brings!
BERTimbau-base-text-filter is a BERT model that can be used to score the quality of a given Portuguese text string. This model was trained on the GigaVerbo-Text-Filter dataset.
Details
- Size: 109,038,209 parameters
- Dataset: GigaVerbo-Text-Filter
- Language: Portuguese
- Number of Training Epochs: 3
- Batch size: 128
- Optimizer:
torch.optim.AdamW - Learning Rate: 4e-5
This repository has the source code used to train this model.
Usage
Here's an example of how to use the BERTimbau-base-text-filter:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from transformers import TextClassificationPipeline
import torch
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained("TucanoBR/BERTimbau-base-text-filter")
model = AutoModelForSequenceClassification.from_pretrained("TucanoBR/BERTimbau-base-text-filter")
model.to(device)
classifier = TextClassificationPipeline(model=model, tokenizer=tokenizer, device=device)
result = classifier("Os tucanos são aves que correspondem à família Ramphastidae, vivem nas florestas tropicais da América Central e América do Sul. A família inclui cinco gêneros e mais de quarenta espécies diferentes. Possuem bicos notavelmente grandes e coloridos, que possuem a função de termorregulação para as muitas espécies que passam muito tempo na copa da floresta exposta ao sol tropical quente.")
Cite as 🤗
@misc{correa2024tucanoadvancingneuraltext,
title={{Tucano: Advancing Neural Text Generation for Portuguese}},
author={Corr{\^e}a, Nicholas Kluge and Sen, Aniket and Falk, Sophia and Fatimah, Shiza},
year={2024},
eprint={2411.07854},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2411.07854},
}
@article{correa2025tucanoadvancingneuraltext,
title={{Tucano: Advancing Neural Text Generation for Portuguese}},
author={Corr{\^e}a, Nicholas Kluge and Sen, Aniket and Falk, Sophia and Fatimah, Shiza},
journal={Patterns},
publisher={Elsevier},
year={2025},
doi={10.1016/j.patter.2025.101325},
url={https://doi.org/10.1016/j.patter.2025.101325},
issn={2666-3899}
}
Aknowlegments
We gratefully acknowledge the granted access to the Marvin cluster hosted by University of Bonn along with the support provided by its High Performance Computing & Analytics Lab.
License
BERTimbau-base-text-filter is licensed under the Apache License, Version 2.0. For more details, see the LICENSE file.