Balochi POS Tagger
A Part-of-Speech tagger for Balochi language written in Arabic script,
fine-tuned from distilbert-base-multilingual-cased on the Balochi CoNLL-U dataset.
Test Set Results
| Metric | Score |
|---|---|
| Accuracy | 0.7002 |
| Precision | 0.5994 |
| Recall | 0.5289 |
| F1 Macro | 0.5360 |
| F1 Weighted | 0.6541 |
Labels
{ "ADJ": 0, "ADP": 1, "ADV": 2, "AUX": 3, "CCONJ": 4, "DET": 5, "INTJ": 6, "NOUN": 7, "NUM": 8, "PART": 9, "PRON": 10, "PROPN": 11, "PUNCT": 12, "SCONJ": 13, "VERB": 14, "X": 15 }
Usage
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("shahbakhsh/balochi-pos-tagger")
model = AutoModelForTokenClassification.from_pretrained("shahbakhsh/balochi-pos-tagger")
sentence = 'ایش انت ملّاہانی حکومت ءِ سوگات'
words = sentence.split()
inputs = tokenizer(words, is_split_into_words=True, return_tensors='pt')
with torch.no_grad():
outputs = model(**inputs)
predictions = outputs.logits.argmax(-1)[0]
word_ids = inputs.word_ids()
for idx, word_id in enumerate(word_ids):
if word_id is not None:
print(words[word_id], model.config.id2label[predictions[idx].item()])
Built by
shahbakhsh — AI/ML Engineer, Balochistan, Pakistan