This is the BSC-TeMU/roberta-large-bne model (source) trained on the squad_es v2.0.0 dataset (source).
Current achievement: em=60.21, f1=68.61
Results:
{
"epoch": 4.0,
"eval_HasAns_exact": 48.44804318488529,
"eval_HasAns_f1": 65.24520506718169,
"eval_HasAns_total": 5928,
"eval_NoAns_exact": 71.97301854974705,
"eval_NoAns_f1": 71.97301854974705,
"eval_NoAns_total": 5930,
"eval_best_exact": 60.22094788328555,
"eval_best_exact_thresh": 0.0,
"eval_best_f1": 68.6181122987237,
"eval_best_f1_thresh": 0.0,
"eval_exact": 60.2125147579693,
"eval_f1": 68.60967917340695,
"eval_samples": 12203,
"eval_total": 11858
}
Training script:
python -m torch.distributed.launch --nproc_per_node=3 ./run_qa.py \
--model_name_or_path BSC-TeMU/roberta-large-bne \
--dataset_name squad_es \
--dataset_config_name v2.0.0 \
--do_train \
--do_eval \
--learning_rate 3e-5 \
--num_train_epochs 4 \
--max_seq_length 384 \
--doc_stride 128 \
--output_dir ./models/roberta-large-bne-finetuned-squad-es/ \
--per_device_eval_batch_size=24 \
--per_device_train_batch_size=12 \
--version_2_with_negative \
--ddp_find_unused_parameters=False \