Model Card for SwahBERT-KenSwQuAD-baseline

Model Summary

This model is a SwahBERT-base-cased model fine-tuned on the KenSwQuAD dataset for extractive question answering in Swahili.

Unlike sequence-to-sequence approaches, this model uses a span extraction architecture (BERT-style) where the model predicts the start and end positions of the answer within the context.

Model Details

Intended Use

Primary Use Cases

• Swahili Question Answering: Extract answer spans from Swahili text given a question
• Research: Baseline for Swahili extractive QA experiments
• Comparison: Benchmark against generative QA approaches (mT5-based models)

How to Use

from transformers import pipeline

# Load the QA pipeline
qa_pipeline = pipeline("question-answering", model="kikwaib/SwahBERT-KenSwQuAD-baseline")

# Example usage
context = """
Mji wa Dar es Salaam ni mji mkubwa na wenye watu wengi nchini Tanzania.
Ni bandari muhimu na kitovu cha kiuchumi cha nchi.
Lugha kuu zinazozungumzwa ni Kiswahili na Kiingereza.
"""

question = "Lugha kuu zinazozungumzwa ni upi?"

result = qa_pipeline(question=question, context=context)
print(f"Answer: {result['answer']}")
print(f"Score: {result['score']:.4f}")
# Expected Output: "Kiswahili na Kiingereza"

Alternative Usage (Manual)

from transformers import AutoTokenizer, AutoModelForQuestionAnswering
import torch

model_name = "kikwaib/SwahBERT-KenSwQuAD-baseline"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForQuestionAnswering.from_pretrained(model_name)

context = "Kenya ni nchi ya Afrika Mashariki. Nairobi ni mji mkuu wa Kenya."
question = "Mji mkuu wa Kenya ni upi?"

inputs = tokenizer(question, context, return_tensors="pt")
outputs = model(**inputs)

# Get the answer span
answer_start = torch.argmax(outputs.start_logits)
answer_end = torch.argmax(outputs.end_logits) + 1
answer = tokenizer.convert_tokens_to_string(
    tokenizer.convert_ids_to_tokens(inputs["input_ids"][0][answer_start:answer_end])
)
print(f"Answer: {answer}")

Limitations

• Extractive only: Can only return text that appears verbatim in the context
• Answer must be contiguous: Cannot combine information from multiple locations
• Context length limit: Maximum 384 tokens with 128 token stride for long documents

Training Data

Dataset: KenSwQuAD

The model was fine-tuned on the full KenSwQuAD (Kenya Swahili Question Answering Dataset).

Data Processing

• Questions and answers extracted from CSV
• Context loaded from corresponding .txt files
• Answer start positions computed by finding answer text in context
• QA pairs where answer not found in context were skipped
• Long contexts handled with sliding window (stride=128 tokens)

Training Procedure

Hardware

Hyperparameters

Training Strategy

• Evaluation Strategy: Per epoch
• Save Strategy: Per epoch
• Best Model Selection: Based on validation loss
• Hub Push: Enabled during training

Evaluation Results

The model was evaluated using the official SQuAD metrics:

Comparison with Generative Approaches

This model represents a baseline extractive approach for Swahili QA. For comparison with generative (seq2seq) approaches, see:

Key Differences

Framework Versions

Citation

If you use this model, please cite the KenSwQuAD research:

Related Models

Training Date: December 2025