XLNet Fine-tuned on SQuAD / Quoref Dataset
XLNet jointly developed by Google and CMU and fine-tuned on SQuAD / SQuAD 2.0 and Quoref for question answering down-stream task.
Evaluation Result on Quoref
{
"exact_match": 73.65591397848462,
"f1": 77.9981532789881
}
Results Comparison on Quoref
| Metric | XLNet Base Line | Model FT on SQuAD |
|---|---|---|
| EM | 61.88 | 73.66 (+11.78) |
| F1 | 70.51 | 78.00 (+7.49) |
How to Use
from transformers import XLNetForQuestionAnswering, XLNetTokenizerFast
model = XLNetForQuestionAnswering.from_pretrained('jkgrad/xlnet-base-cased-squad-quoref)
tokenizer = XLNetTokenizerFast.from_pretrained('jkgrad/xlnet-base-cased-squad-quoref')