Vietnamese Legal Embedding (With Word Segmentation)

Model: bqbbao6/vietnamese-legal-embedding-wsgm
Base model: intfloat/multilingual-e5-base

Model Description

vietnamese-legal-embedding-wsgm is a text embedding model fine-tuned specifically for Vietnamese legal document retrieval leveraging prior word segmentation. Built on top of multilingual-e5-base, this model is optimized to capture deep semantic relationships and precise legal contexts by processing text where compound words are explicitly segmented (e.g., connected via underscores _).

This model is ideal for advanced RAG pipelines and legal semantic search systems that already utilize a dedicated Vietnamese NLP pre-processing step (such as PyVi) to maximize retrieval accuracy.

Model Details

Note on Prefixes & Text Processing: * This model follows the E5 instruction format. You must use the prefix query: for user questions and passage: for the legal corpus documents to achieve optimal retrieval performance.

• Word Segmentation: This is the word-segmented version of the core bqbbao6/vietnamese-legal-embedding model. Please ensure all input text undergoes word segmentation before being fed into this model. If your pipeline handles raw text without a tokenizer setup, use the raw text version instead.

Usage

First, install the Sentence Transformers library and the PyVi tokenizer:

pip install -U sentence-transformers pyvi

from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim
from pyvi import ViTokenizer
import torch

Then you can load this model and run inference.

# Load the model
model = SentenceTransformer("bqbbao6/vietnamese-legal-embedding-wsgm", trust_remote_code=True)

# Helper function for legal formatting with word segmentation
def process_text(text, prefix="passage: "):
    # Segment words (e.g., "Người lao động" -> "Người lao_động")
    segmented_text = ViTokenizer.tokenize(text)
    return prefix + segmented_text

# Define raw inputs
raw_query = "Người lao động được nghỉ bao nhiêu ngày phép mỗi năm?"
raw_corpus = [
    "Người lao động làm việc đủ 12 tháng được nghỉ 12 ngày phép năm có hưởng lương.",
    "Doanh nghiệp phải đóng thuế thu nhập doanh nghiệp hàng năm.",
    "Cán bộ công chức được nghỉ 12 ngày phép năm theo quy định."
]

# Format with proper E5 prefixes and word segmentation
query = process_text(raw_query, prefix="query: ")
corpus = [process_text(p, prefix="passage: ") for p in raw_corpus]

# Encode text
q_emb = model.encode(query, normalize_embeddings=True)
c_emb = model.encode(corpus, normalize_embeddings=True)

# Calculate similarity
scores = cos_sim(q_emb, c_emb)[0]

for i, score in enumerate(scores):
    print(f"Score: {score:.4f} | {raw_corpus[i]}")

Evaluation

Eval 1: ZacLegalTextRetrieval benchmark using the MTEB framework, and compared against the base model and a Vietnamese-specific embedding model.

Notice: This evaluation dataset does not utilize word segmentation | Metric | bqbbao6/vietnamese-legal-embedding-wsgm | intfloat/multilingual-e5-base | bkai-foundation-models/vietnamese-bi-encoder | |---|---|---|---| | NDCG@10 | 0.8059 | 0.6030 | 0.6160 | | MRR@10 | 0.7543 | 0.5482 | 0.5579 | | MAP@10 | 0.7546 | 0.5491 | 0.5588 | | Recall@1 | 0.6269 | 0.4467 | 0.4442 | | Recall@5 | 0.9124 | 0.6916 | 0.7170 | | Recall@10 | 0.9613 | 0.7722 | 0.7951 | | Precision@1 | 0.6282 | 0.4480 | 0.4454 | | Hit Rate@10 | 0.9632 | 0.7728 | 0.7970 |

Eval 2: another-symato/VMTEB-Zalo-legel-retrieval-wseg and compared against the base model and a Vietnamese-specific embedding model.

The fine-tuned model significantly outperforms both the base model and the Vietnamese-specific bi-encoder across all metrics, demonstrating the effectiveness of domain-specific fine-tuning for Vietnamese legal retrieval.

Citation

@inproceedings{10.1007/978-981-95-1746-6_17,
  address = {Singapore},
  author = {Pham, Bao Loc
and Hoang, Quoc Viet
and Luu, Quy Tung
and Vo, Trong Thu},
  booktitle = {Proceedings of the Fifth International Conference on Intelligent Systems and Networks},
  isbn = {978-981-95-1746-6},
  pages = {153--163},
  publisher = {Springer Nature Singapore},
  title = {GN-TRVN: A Benchmark for Vietnamese Table Markdown Retrieval Task},
  year = {2026},
}


@article{enevoldsen2025mmtebmassivemultilingualtext,
  title={MMTEB: Massive Multilingual Text Embedding Benchmark},
  author={Kenneth Enevoldsen and Isaac Chung and Imene Kerboua and Márton Kardos and Ashwin Mathur and David Stap and Jay Gala and Wissam Siblini and Dominik Krzemiński and Genta Indra Winata and Saba Sturua and Saiteja Utpala and Mathieu Ciancone and Marion Schaeffer and Gabriel Sequeira and Diganta Misra and Shreeya Dhakal and Jonathan Rystrøm and Roman Solomatin and Ömer Çağatan and Akash Kundu and Martin Bernstorff and Shitao Xiao and Akshita Sukhlecha and Bhavish Pahwa and Rafał Poświata and Kranthi Kiran GV and Shawon Ashraf and Daniel Auras and Björn Plüster and Jan Philipp Harries and Loïc Magne and Isabelle Mohr and Mariya Hendriksen and Dawei Zhu and Hippolyte Gisserot-Boukhlef and Tom Aarsen and Jan Kostkan and Konrad Wojtasik and Taemin Lee and Marek Šuppa and Crystina Zhang and Roberta Rocca and Mohammed Hamdy and Andrianos Michail and John Yang and Manuel Faysse and Aleksei Vatolin and Nandan Thakur and Manan Dey and Dipam Vasani and Pranjal Chitale and Simone Tedeschi and Nguyen Tai and Artem Snegirev and Michael Günther and Mengzhou Xia and Weijia Shi and Xing Han Lù and Jordan Clive and Gayatri Krishnakumar and Anna Maksimova and Silvan Wehrli and Maria Tikhonova and Henil Panchal and Aleksandr Abramov and Malte Ostendorff and Zheng Liu and Simon Clematide and Lester James Miranda and Alena Fenogenova and Guangyu Song and Ruqiya Bin Safi and Wen-Ding Li and Alessia Borghini and Federico Cassano and Hongjin Su and Jimmy Lin and Howard Yen and Lasse Hansen and Sara Hooker and Chenghao Xiao and Vaibhav Adlakha and Orion Weller and Siva Reddy and Niklas Muennighoff},
  publisher = {arXiv},
  journal={arXiv preprint arXiv:2502.13595},
  year={2025},
  url={https://arxiv.org/abs/2502.13595},
  doi = {10.48550/arXiv.2502.13595},
}

@article{muennighoff2022mteb,
  author = {Muennighoff, Niklas and Tazi, Nouamane and Magne, Loïc and Reimers, Nils},
  title = {MTEB: Massive Text Embedding Benchmark},
  publisher = {arXiv},
  journal={arXiv preprint arXiv:2210.07316},
  year = {2022}
  url = {https://arxiv.org/abs/2210.07316},
  doi = {10.48550/ARXIV.2210.07316},
}

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}