Multilingual Banking Intent Classifier (EN + BN + Banglish)
Overview
This model is a fine-tuned MuRIL-based multilingual intent classifier designed for production-grade banking chatbots.
- Model Name: Banking Multilingual Intent Classifier
- Base Model: google/muril-base-cased
- Task: Multilingual Intent Classification
- Intents: 14
- Languages: English, Bangla (Bengali script), Banglish (Romanized Bengali), Code-Mixed
The model performs 14-way intent classification for banking conversational systems.
Base Model
google/muril-base-cased
MuRIL was selected for:
- Strong multilingual support
- Excellent performance on Indic languages
- Stable tokenization for Bangla + English
- Robust handling of code-mixed inputs
Supported Intents (14)
ACCOUNT_INFO
ATM_SUPPORT
CARD_ISSUE
CARD_MANAGEMENT
CARD_REPLACEMENT
CHECK_BALANCE
EDIT_PERSONAL_DETAILS
FAILED_TRANSFER
FALLBACK
FEES
GREETING
LOST_OR_STOLEN_CARD
MINI_STATEMENT
TRANSFER
Dataset Summary
Total Samples: 100,971
Languages (Balanced)
| Language | Count |
|---|---|
| English (en) | 33,657 |
| Bangla (bn) | 33,657 |
| Banglish (bn-latn) | 33,657 |
Additional 500 code-mixed examples included.
Final Training Dataset
| Split | Samples |
|---|---|
| Train | 91,051 |
| Test | 20,295 |
Class Distribution (Final Train)
- All intents are within a safe 4–10% range.
- FALLBACK is controlled at ~9.4%, preventing dominance.
- This distribution avoids class collapse and overconfidence bias.
Evaluation Metrics
Overall Performance
- Accuracy: 99.12%
- F1 Micro: 99.12%
- F1 Macro: 99.08%
- Validation Loss: 0.046
Per-Intent Accuracy
| Intent | Accuracy |
|---|---|
| ACCOUNT_INFO | 99.14% |
| ATM_SUPPORT | 99.70% |
| CARD_ISSUE | 99.25% |
| CARD_MANAGEMENT | 99.43% |
| CARD_REPLACEMENT | 99.08% |
| CHECK_BALANCE | 99.05% |
| EDIT_PERSONAL_DETAILS | 100.00% |
| FAILED_TRANSFER | 98.75% |
| FALLBACK | 97.86% |
| FEES | 99.76% |
| GREETING | 97.41% |
| LOST_OR_STOLEN_CARD | 99.59% |
| MINI_STATEMENT | 98.80% |
| TRANSFER | 99.78% |
Strengths
- Strong multilingual support
- Balanced dataset distribution
- Robust fallback handling
- Stable across operational banking intents
- High macro F1 ensures no minority intent collapse
- Performs well on code-mixed queries
Intended Use
- Banking chatbot intent routing
- Customer support automation
- Financial conversational AI
- Multilingual banking assistants
Out of Scope
- Fraud detection
- Sentiment analysis
- Financial advisory decisions
- Regulatory or legal compliance automation
Production Recommendations
- Apply confidence thresholding
- Route low-confidence predictions to human fallback
- Use softmax entropy monitoring
- Normalize numeric expressions before inference
- Log confusion pairs in production
Example Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
# Load model and tokenizer
model_name = "learn-abc/banking-multilingual-intent-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
model.eval()
# Prediction function
def predict_intent(text):
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)
inputs = {k: v.to(device) for k, v in inputs.items()}
with torch.no_grad():
outputs = model(**inputs)
prediction = torch.argmax(outputs.logits, dim=-1).item()
confidence = torch.softmax(outputs.logits, dim=-1)[0][prediction].item()
predicted_intent = model.config.id2label[prediction]
return {
"intent": predicted_intent,
"confidence": confidence
}
# Example usage - English
result = predict_intent("what is my balance")
print(f"Intent: {result['intent']}, Confidence: {result['confidence']:.2f}")
# Output: Intent: CHECK_BALANCE, Confidence: 0.99
# Example usage - Bangla
result = predict_intent("আমার ব্যালেন্স কত")
print(f"Intent: {result['intent']}, Confidence: {result['confidence']:.2f}")
# Output: Intent: CHECK_BALANCE, Confidence: 0.98
# Example usage - Banglish (Romanized)
result = predict_intent("amar balance koto ache")
print(f"Intent: {result['intent']}, Confidence: {result['confidence']:.2f}")
# Output: Intent: CHECK_BALANCE, Confidence: 0.97
# Example usage - Code-mixed
result = predict_intent("আমার last 10 transaction দেখাও")
print(f"Intent: {result['intent']}, Confidence: {result['confidence']:.2f}")
# Output: Intent: MINI_STATEMENT, Confidence: 0.98
Limitations
- Does not handle multi-turn conversational context
- Extremely ambiguous short inputs may require thresholding
- Synthetic data may introduce stylistic bias
- No speech-to-text robustness included
Version
- Version: 2.0
- Status: Production-Ready
- Architecture: MuRIL Base
- Language Coverage: EN + BN + Banglish
License
This project is licensed under the Apache 2.0 License.
Contact Me
For any inquiries or support, please reach out to:
- Author: Abhishek Singh
- LinkedIn: My LinkedIn Profile
- Portfolio: Abhishek Singh Portfolio