This is the BSC-TeMU/roberta-base-bne model (source) trained on the squad_es v2.0.0 dataset (source).
Current achievement: em=58.80, f1=67.40
Results:
{
"epoch": 4.0,
"eval_HasAns_exact": 48.51551956815115,
"eval_HasAns_f1": 65.70745010262016,
"eval_HasAns_total": 5928,
"eval_NoAns_exact": 69.0893760539629,
"eval_NoAns_f1": 69.0893760539629,
"eval_NoAns_total": 5930,
"eval_best_exact": 58.804182830156854,
"eval_best_exact_thresh": 0.0,
"eval_best_f1": 67.39869828034618,
"eval_best_f1_thresh": 0.0,
"eval_exact": 58.804182830156854,
"eval_f1": 67.39869828034568,
"eval_samples": 12211,
"eval_total": 11858
}
Training script:
python -m torch.distributed.launch --nproc_per_node=3 ./run_qa.py \
--model_name_or_path BSC-TeMU/roberta-base-bne \
--dataset_name squad_es \
--dataset_config_name v2.0.0 \
--do_train \
--do_eval \
--learning_rate 3e-5 \
--num_train_epochs 4 \
--max_seq_length 384 \
--doc_stride 128 \
--output_dir ./models/roberta-base-bne-squad-2.0-es/ \
--per_device_eval_batch_size=8 \
--per_device_train_batch_size=8 \
--version_2_with_negative \
--ddp_find_unused_parameters=False \
--overwrite_output_dir \