job-title-normalizer-e5-base
A sentence encoder fine-tuned to normalize messy, multilingual job titles to a canonical occupation taxonomy (ESCO + O*NET), framed and evaluated as retrieval: embed a noisy or foreign-language title (the query) and retrieve the closest canonical occupation label (the passage) from a fixed corpus of 4,055 occupations.
Live demo: https://job-title-normalizer-525186107937.us-central1.run.app (Cloud Run; may cold-start ~1 min) · Lighter variant: job-title-normalizer-e5-small
Why retrieval, not classification? Occupation taxonomies have thousands of classes and the label set evolves constantly. A nearest-neighbour retriever over embeddings generalizes to unseen labels and lets you swap the corpus without retraining a softmax head.
Results
Held-out test set of 15,248 real ESCO/O*NET titles against a 4,055-occupation corpus. The split is by occupation (zero overlap between train/val/test occupations, asserted at build time), so every test occupation is unseen during training.
| Slice | Method | Recall@1 | Recall@5 | Recall@10 | MRR |
|---|---|---|---|---|---|
| Overall | BM25 (lexical) | 0.095 | 0.163 | 0.183 | 0.124 |
| Overall | zero-shot e5-base | 0.237 | 0.378 | 0.437 | 0.298 |
| Overall | this model | 0.381 | 0.572 | 0.645 | 0.463 |
| FR→EN | this model | 0.548 | 0.774 | 0.847 | 0.643 |
| DE→EN | this model | 0.536 | 0.792 | 0.852 | 0.642 |
Cross-lingual context: BM25 scores MRR 0.034 on the FR/DE→EN slice (a French query shares almost no tokens with an English canonical label); this model reaches 0.643 — lexical search structurally cannot do this task, and fine-tuning adds ~35% over the zero-shot base.
Training
- Base:
intfloat/multilingual-e5-base(mean pooling,query:/passage:prefixes, 768-dim). - Objective: in-batch-negatives InfoNCE (
MultipleNegativesRankingLoss, scale 20 ≈ temperature 0.05), wrapped in MatryoshkaLoss (dims 768/512/256/128/64) so truncated embeddings stay usable. - Data: 160,240 positive pairs (synonyms, alternate titles, and cross-lingual label pairs of the same occupation) built from ESCO (EN/FR/DE) and O*NET alternate titles; ≤50 pairs per occupation.
- Setup: 2 epochs, batch 64, lr 2e-5, warmup 10%, max_seq_len 64, fp16,
NoDuplicatesDataLoaderto reduce in-batch false negatives. Trained on an RTX 4060 Laptop (8 GB) in ~46 min. - Prefixes are baked into the training text exactly as applied at inference, and the preset is persisted (
tn_preset.json) so downstream tooling recovers the correct behavior.
How to use
The e5 family needs asymmetric prefixes: encode input titles as query: … and canonical
labels as passage: …. Forgetting them silently degrades accuracy.
from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim
model = SentenceTransformer("Misbahuddin/job-title-normalizer-e5-base")
canonicals = [
"passage: software developer",
"passage: data scientist",
"passage: nurse responsible for general care",
]
query = "query: Ingénieur logiciel" # French → English canonical
q = model.encode(query, normalize_embeddings=True)
c = model.encode(canonicals, normalize_embeddings=True)
scores = cos_sim(q, c)[0]
print(canonicals[int(scores.argmax())], float(scores.max()))
# passage: software developer ~0.79
Vectors are L2-normalized, so cosine == dot product — index canonical embeddings with FAISS
IndexFlatIP for production search. Matryoshka: you may truncate embeddings to 256/128/64
dims (then re-normalize) for cheaper indexes without re-encoding.
Calibrate an abstention threshold. Out-of-taxonomy queries still return a nearest neighbour — but at tellingly low scores (e.g. "RevOps", which has no ESCO/O*NET occupation, scores ~0.31 vs ~0.8 for true matches). Reject below a threshold tuned on your data.
Intended use & limitations
- In scope: normalizing titles from resumes, postings, CRM/ATS/HRIS records to occupation IDs; semantic occupation search; FR/DE→EN cross-lingual lookup.
- Not a hiring/screening/compensation decision system. Similarity of role names says nothing about people. Keep a human in the loop for consequential uses.
- Coverage is bounded by ESCO + O*NET (with a European resp. U.S. framing); en/fr/de only is verified. ESCO and O*NET overlap conceptually, so near-duplicate occupations exist across the two namespaces.
- Tuned for short titles (max 64 tokens); long descriptions are out of distribution.
Acknowledgements
- ESCO — © European Union; reused under Commission Decision 2011/833/EU. https://esco.ec.europa.eu/
- O*NET — by the National Center for O*NET Development for USDOL/ETA; CC BY 4.0. https://www.onetcenter.org/
ONET® is a trademark of USDOL/ETA. This model was produced using ONET data but is not endorsed by USDOL/ETA. ESCO is a service of the European Commission; this model is not endorsed by the Commission.