AfrimBert-QA - W4A8 QAT

amidblue/AfrimBert-QA fine-tuned with quantization-aware training on a language-balanced subset of AfriQuAD (weights: symmetric int4, group size 128; activations: dynamic per-token int8).

Files

• model.safetensors: standard checkpoint (weights adapted to the quantization grid), loadable with AutoModelForQuestionAnswering.
• quant_model.safetensors + quant_meta.json + quant_utils.py: real integer weights (two int4 values packed per byte) with scales; word embeddings stored as per-row int8.

import sys
from huggingface_hub import snapshot_download
p = snapshot_download("theophilusowiti/AfrimBert-QA-int4-qat", allow_patterns=["config.json", "quant_meta.json", "quant_model.safetensors", "quant_utils.py"])
sys.path.insert(0, p)
from quant_utils import load_quantized_model
model = load_quantized_model(p)

Note: weights are dequantized on the fly, so this saves storage; int4 compute speedups need a kernel backend (e.g. torchao, ONNX Runtime, TensorRT). The base model is gated; check its terms before making this repo public.