bge-m3-thai-intent
BAAI/bge-m3 fine-tuned to pick a customer's intent from natural-language intent descriptions.
Intents are data, not weights: embed the message, embed each intent description, softmax the similarities.
Adding an intent means writing one line of description — no retraining.
Usage
import numpy as np
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("SimonY0s/bge-m3-thai-intent")
intents = {
"track_order": "Asking where a parcel is, delivery status, or when an order will arrive",
"refund_return": "Wants to return an item, cancel an order, or get money back",
"talk_to_human": "Explicitly asks to speak with a human agent instead of a bot",
}
msg = model.encode(["ของยังไม่มาเลยค่ะ สั่งไปตั้งแต่วันจันทร์"], normalize_embeddings=True)
lab = model.encode(list(intents.values()), normalize_embeddings=True)
logits = 30.0 * (msg @ lab.T)[0] # scale 30 was used in training
probs = np.exp(logits - logits.max()); probs /= probs.sum()
print(dict(zip(intents, probs.round(3))))
Descriptions were written in English during training; messages were Thai (some Thai-English mix).
Results
Synthetic Thai customer messages, 30 intents across e-commerce, banking and telecom.
seen = 24 intents present in training (173 test messages); unseen = 6 held-out intents known only by description (241 messages).
| Model | seen | unseen |
|---|---|---|
| Jev 1.13 (TypeSafe, via OpenRouter) | 93.6% | 87.6% |
| this model | 93.6% | 88.8% |
BAAI/bge-m3 zero-shot | 81.5% | 80.9% |
intfloat/multilingual-e5-small, same recipe | 86.1% | 75.9% |
Differences under ~5 points are within noise at these test-set sizes.
Training
741 (message, intent description) pairs, 4 epochs, batch 16, lr 1e-5, AdamW, CLS pooling, cross-entropy over the intents of the message's own domain with similarity scale 30. About 5 minutes on an Apple M4 Pro.
Limitations — read before relying on it
- Synthetic data. Messages and labels were written by an LLM, not by real customers, and the training split kept only messages where a second model (Jev) agreed with the intended label. Expect lower accuracy on real traffic; validate on a human-labelled set.
- Probabilities are not calibrated. The softmax scores were not calibrated on held-out data; tune any confidence threshold yourself.
- Romanised Thai ("karaoke" spelling), numbers and dates are weak spots.
- It maps short messages to intents. It does not reason over structured state (for example "order number known, ticket not yet opened").