bert-base-uncased-squad-qa
bert-base-uncased fine-tuned for extractive question answering on 15,000 examples of SQuAD v1.1.
| Model | Exact Match | F1 |
|---|---|---|
| Frozen BERT + Logistic Regression | 14.35 | 24.57 |
| This model (full fine-tuning) | 69.25 | 79.69 |
Evaluated on the full SQuAD v1.1 validation set (10,570 questions).
Files in this repository
-
Model 2 (full fine-tuning):
BertForQuestionAnsweringweights, config and tokenizer. -
Model 1 (feature-based baseline): two scikit-learn Logistic Regression models (
StandardScaler+LogisticRegressionpipelines), hosted as.joblibfiles:model1_start_classifier.joblib: scores each context word as the answer startmodel1_end_classifier.joblib: scores each context word as the answer end
They take as input the
last_hidden_state(768-d) of the frozen, originalbert-base-uncased, not of the fine-tuned model, at the first sub-token of each context word.
Training
- AdamW, encoder lr 2e-05, QA head lr 0.001, weight decay 0.01, linear schedule with 10% warm-up
- 2 epochs, batch size 16, max length 384, stride 128,
fp16=True, seed 42
Usage
from transformers import pipeline
qa = pipeline("question-answering", model="Giobbva/bert-base-uncased-squad-qa")
qa(question="Where is the Eiffel Tower?", context="The Eiffel Tower is in Paris.")
Loading the baseline classifiers:
import joblib
from huggingface_hub import hf_hub_download
start_classifier = joblib.load(hf_hub_download("Giobbva/bert-base-uncased-squad-qa", "model1_start_classifier.joblib"))
end_classifier = joblib.load(hf_hub_download("Giobbva/bert-base-uncased-squad-qa", "model1_end_classifier.joblib"))