wangchanberta-ud-thai-pud-upos

This model is a fine-tuned version of airesearch/wangchanberta-base-att-spm-uncased on the universal_dependencies dataset. It achieves the following results on the evaluation set:

• Loss: 0.0442
• Macro avg precision: 0.9221
• Macro avg recall: 0.9178
• Macro avg f1: 0.9199
• Weighted avg precision: 0.9883
• Weighted avg recall: 0.9883
• Weighted avg f1: 0.9883
• Accuracy: 0.9883

Model description

This model is train on thai UD Thai PUD corpus with Universal Part-of-speech (UPOS) tag to help with pos tagging in Thai language.

Example

from transformers import AutoModelForTokenClassification, AutoTokenizer, TokenClassificationPipeline

model = AutoModelForTokenClassification.from_pretrained("Pavarissy/wangchanberta-ud-thai-pud-upos")
tokenizer = AutoTokenizer.from_pretrained("Pavarissy/wangchanberta-ud-thai-pud-upos")

pipeline = TokenClassificationPipeline(model=model, tokenizer=tokenizer, grouped_entities=True)
outputs = pipeline("ประเทศไทย อยู่ใน ทวีป เอเชีย")
print(outputs)
# [{'entity_group': 'NOUN', 'score': 0.419697, 'word': '', 'start': 0, 'end': 1}, {'entity_group': 'PROPN', 'score': 0.8809489, 'word': 'ประเทศไทย', 'start': 0, 'end': 9}, {'entity_group': 'VERB', 'score': 0.7754166, 'word': 'อยู่ใน', 'start': 9, 'end': 16}, {'entity_group': 'NOUN', 'score': 0.9976932, 'word': 'ทวีป', 'start': 16, 'end': 21}, {'entity_group': 'PROPN', 'score': 0.97770107, 'word': 'เอเชีย', 'start': 21, 'end': 28}]

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Framework versions

• Transformers 4.34.1
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1