GECToR-eus

Basque grammatical error correction and real-word error detection via GECToR (seq2edit), fine-tuned on RoBERTa-eus-base.

This is the PyTorch checkpoint. For browser/edge deployment, use the ONNX int4 export (83 MB).

Overview

This model corrects real-word grammar errors in Basque — cases where every word is a valid dictionary word but the inflection is wrong in context (e.g. dio → zaio, zaidalaren → zaidalako, etortzen → etorriko). It also provides per-word error detection via a dedicated detect head (P(INCORRECT) per token).

Performance

Evaluated on the Elhuyar Dem_single (221 errorful sentences) and Dem_none (250 clean sentences) benchmarks:

For comparison, GECToR-2024 (English, RoBERTa-large 300M) scores F0.5=72.9 on BEA-dev.

Usage

This model uses the gotutiyan/gector framework. Install it and use the CLI:

pip install gector

# Predict (correct grammar)
python -m gector.cli.predict \
  --input input.txt \
  --restore_dir itzune/gector-eus \
  --verb_file verb-form-vocab.txt \
  --out output.txt \
  --min_error_prob 0.5

Or load in Python:

from gector import GECToR

model = GECToR.from_pretrained("itzune/gector-eus")
# model.bert           → RoBERTa encoder
# model.label_proj_layer  → label head (→ 5000 classes)
# model.d_proj_layer      → detect head (→ 2 classes: CORRECT, INCORRECT)

How it works

GECToR uses an edit-based approach — instead of generating text, it predicts a label per token:

• $KEEP — leave this word unchanged
• $DELETE — remove this word
• $REPLACE_xxx — replace this word with xxx
• $APPEND_xxx — append xxx after this word

The model runs iteratively (up to 5 passes): each pass applies the predicted edits, then re-runs on the corrected text until no more changes are needed.

The detect head provides an independent P(INCORRECT) per token — a confidence score for whether each word is wrong. This is used as a threshold (min_error_prob) to control the precision/recall tradeoff, and also powers error detection visualizations.

Training data — Elhuyar/Orai GEC corpus

The model was trained on the Elhuyar GEC corpus, created by the Elhuyar Foundation (now Orai NLP) and distributed under CC-BY-NC-SA.

• Source: Correct sentences were extracted from a Basque news corpus compiled from Berria.eus, Argia.eus, and the Tokikom.eus proximity-media network (500,015 news items, 4,927,748 sentences, ~66M words).
• Error generation: Synthetic grammar errors (R1–R4: tense, verb agreement/argument, case/agreement, suffix) were introduced by rule application.
• Scale: 9.3M sentence pairs total (4.71M with errors, 5.29M clean). We trained on 1M pairs.
• Distribution: Orai.eus resources page / HiTZ corpus page

ℹ️ This is distinct from the Elhuyar web corpus (186M tokens, Leturia 2014), a separate general web-crawl resource on Orai's resources page that is not the source of the GEC data.

Training

• Base model: ixa-ehu/roberta-eus-euscrawl-base-cased
• Training data: 1M sentence pairs from the Elhuyar GEC corpus (see above)
• Epochs: 5 (1 cold epoch at lr=0.001, 4 fine-tune epochs at lr=1e-5)
• Batch size: 64
• Max length: 128 tokens
• GECToR implementation: gotutiyan/gector
• Training code: itzune/gector-eus

Related

• itzune/gector-eus-onnx — ONNX int4 export (83 MB) for browser/edge deployment
• itzune/gector-eus — Training code and scripts
• txukun — Browser-based Basque text corrector using this model

Credits

• RoBERTa-eus-base was pre-trained by the IXA NLP Group (UPV/EHU) — see Artetxe et al. (2022)
• GECToR was introduced by Omelianchuk et al. (2020) (Grammarly)
• Elhuyar GEC corpus was created by the Elhuyar Foundation (now Orai NLP) and HiTZ Zentroa (UPV/EHU)
• GECToR implementation: gotutiyan/gector (MIT)

License

CC-BY-NC-SA 4.0

This model's weights are a derivative work of the Elhuyar GEC corpus, which is licensed under CC-BY-NC-SA. Under the ShareAlike clause, the trained model weights are distributed under the same license:

• NC (NonCommercial) — You may not use this model for commercial purposes.
• SA (ShareAlike) — If you remix, transform, or build upon this model, you must distribute your contributions under the same license.

The base encoder (RoBERTa-eus-base, Apache 2.0) and the GECToR architecture (gotutiyan/gector, MIT) are separately licensed, but the trained weights inherit the CC-BY-NC-SA restriction from the training data.

The training scripts and ONNX export code (itzune/gector-eus) are MIT licensed.

Citation

If you use this model, please cite the original works:

@inproceedings{beloki2020gec,
  title     = {Grammatical Error Correction for Basque through a seq2seq neural architecture and synthetic examples},
  author    = {Beloki, Zuhaitz and Saralegi, Xabier and Ceberio, Klara and Corral, Ander},
  booktitle = {Proceedings of the 36th Conference of the Spanish Society for Natural Language Processing (SEPLN 2020)},
  address   = {Granada, Spain},
  year      = {2020}
}

@article{omelianchuk2020gector,
  title={GECToR--Grammatical Error Correction: Tag, Not Rewrite},
  author={Omelianchuk, Kostiantyn and Atrasevych, Vitaly and Chernodub, Artem and Skurzhanskyi, Oleksandr},
  journal={arXiv preprint arXiv:2005.12592},
  year={2020}
}

@article{artetxe2022roberteus,
  title={Roberteus: a monolingual basque language model},
  author={Artetxe, Mikel and Adebisi, Iyanuoluwa and Azkarate, Mikel and Ceberio, Itziar and Campos, Jon Ander and Esnal, Gorka and Fernandez de Landa, Oscar and Goikoetxea, Joseba and Gutierrez, Aitor and Igondi, Maite and others},
  journal={Procesamiento del Lenguaje Natural},
  volume={69},
  pages={27--34},
  year={2022}
}