Ows2-MiniLM — Ontology Alignment Model for Agri-Food Domains

Ows2-MiniLM is a sentence embedding model fine-tuned for semantic alignment between scientific text and agri-food ontology terms. It is part of the OWS² (Ontology-based Workflow for Semantic Specialisation) project developed at INRAE (French National Research Institute for Agriculture, Food and Environment).

Given a sentence extracted from a scientific abstract or publication, the model retrieves the most semantically relevant ontology terms from a reference vocabulary of ~14 600 concepts covering plant science, food science, and agronomy domains.

The reference vocabulary incorporates concepts from:

• TransformON: Nature Publication
• Planteome: PubMed Publication

Acknowledgement: This work was supported by INRAE Metaprogramme DIGIT-BIO within the framework of the SEED project, and was carried out within the Plateforme de Profilage Métabolique et de Métabolomique (P2M2) — UMR 1349 IGEPP, INRAE.

Model Details

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'architecture': 'BertModel'})
  (1): Pooling({'pooling_mode': 'mean'})
  (2): Normalize({})
)

Intended Use

This model is designed for ontology-based semantic search in scientific literature, specifically:

• Automatic annotation of scientific abstracts with ontology concepts
• Semantic retrieval of the most relevant agri-food ontology terms given a sentence
• Alignment between free-text descriptions and controlled vocabularies (e.g., PPDO, AgroVoc-related thesauri)

Out-of-scope use: This model is specialized for agri-food science. Performance on general-purpose or non-scientific text may be significantly lower.

Usage

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("tcolombu/Ows2-MiniLM_1")

# A sentence from a scientific abstract
query = "Fruit-based sauces exhibited significant variations in antioxidant activity among different cultivars."

# Ontology terms to rank
ontology_terms = [
    "fruit sauce",
    "antioxidant activity",
    "barley powdery mildew disease response",
    "rootless",
]

# Encode and compute cosine similarities
query_embedding = model.encode(query, convert_to_tensor=True)
term_embeddings = model.encode(ontology_terms, convert_to_tensor=True)

scores = util.cos_sim(query_embedding, term_embeddings)[0]
ranked = sorted(zip(ontology_terms, scores.tolist()), key=lambda x: x[1], reverse=True)

for term, score in ranked:
    print(f"{score:.4f}  {term}")
# Expected top result: fruit sauce

Evaluation

Two distinct evaluation protocols were used, each measuring different aspects of model performance. Both are computed on the same 100-query expert goldtest, but differ in corpus construction and scoring logic.

Evaluator 1 — Standard IR (InformationRetrievalEvaluator)

Called during training via python main.py finetune --config <yaml>

This is the standard InformationRetrievalEvaluator from the sentence-transformers library. Each ontology term is registered with a composite key "{ontology_id}_{term}", meaning the same surface form can appear multiple times in the corpus if it has multiple ontology IDs. Relevance is checked by exact doc_id match only.

• Corpus size: ~14 600 entries (with duplicates)
• Scoring: Single exact doc_id match required
• Stored in: metrics_comparison.json

Evaluator 2 — Combined Goldtest Benchmark (CombinedGoldtestEvaluator)

Called post-training via python main.py eval_model <model_path>

A custom three-component evaluator. Terms are deduplicated to 14 544 unique surface forms. Synonymous ontology IDs sharing the same term are merged: a prediction is correct if any valid ID for the target term is retrieved. Queries are split into two subsets:

• Exact Match: the target term appears verbatim in the query sentence
• Taboo Match: the target term is absent from the query — pure semantic understanding required

Stored in: goldtest_eval/combined_goldtest_metrics.json

Reading the two scores: Evaluator 1 (66.63% MRR) is stricter because duplicated corpus entries act as additional distractors. Evaluator 2 (73.94% MRR) is more semantically faithful because synonyms of the correct answer are also accepted as valid.

Baseline Comparison — all-MiniLM-L6-v2 (no fine-tuning) vs Ows2-MiniLM

All metrics below are computed on our agri-food ontology corpus (not the standard sentence-transformers benchmarks). The base model has never seen this vocabulary.

Validation set — InformationRetrievalEvaluator (2 238 queries, ~14 400 corpus terms)

Goldtest — Expert benchmark (100 queries, ~14 600 corpus terms)

Fine-tuning on the OWS² mixed dataset (real + synthetic pairs) provides a ~+32 points absolute gain on Acc@1 and ~+32 points on MRR@10 on the expert goldtest, demonstrating the strong impact of domain specialisation even with limited real annotated data (~491 expert examples).

Training Details

Training Data

The model was trained on a mixed dataset combining real annotated data and LLM-generated synthetic pairs:

• Format: (anchor_sentence, positive_term, hard_negative_term) triplets
• Loss: MultipleNegativesRankingLoss — uses all other items in the batch as implicit negatives
• Synthetic generation LLM: Qwen — run locally (Qwen2+ is Apache 2.0; Qwen1.x Tongyi license is compatible with domain-specific fine-tuned model publication)
• Dataset file: mixed_dataset_v0_full.jsonl

Training Hyperparameters

Note on hyperparameter selection: Batch size 16 with LR 2e-5 over 15 epochs was identified as the optimal configuration after a systematic grid search across 44 experiments. Increasing epochs beyond 15 showed no further gain (convergence plateau confirmed).

Training Logs (validation NDCG@10 over 15 epochs)

Training Time

• Training: ~10.7 minutes (NVIDIA GPU, SLURM cluster)
• Evaluation: ~7.8 minutes
• Total: ~18.5 minutes

Limitations

• Domain specificity: The model is fine-tuned on agri-food science vocabulary. Performance on out-of-domain text (e.g., clinical, legal) is expected to be significantly lower.
• Ontology coverage: The reference ontology contains ~14 600 terms. Terms absent from this vocabulary cannot be retrieved.
• Real data scarcity: Only ~491 real annotated examples were available. Performance is expected to improve as the real annotation corpus grows.
• Taboo match gap: The 19-point gap between Exact Match MRR (73.94%) and Taboo Match MRR (54.48%) indicates the model still relies partially on lexical overlap. Improving semantic generalization is an identified next step.

Framework Versions

• Python: 3.10.20
• Sentence Transformers: 5.6.0
• Transformers: 5.14.1
• PyTorch: 2.10.0+cu128
• Accelerate: 1.14.0
• Datasets: 5.0.0

Citation

If you use this model, please cite:

@misc{ows2-minilm-2025,
  title        = {Ows2-MiniLM: An Ontology Alignment Model for Agri-Food Science},
  institution  = {INRAE, P2M2},
  note         = {OWS\textsuperscript{2} -- Ontology-based Workflow for Semantic Specialisation},
  howpublished = {\url{https://huggingface.co/tcolombu/Ows2-MiniLM_1}}
}

@inproceedings{reimers-2019-sentence-bert,
    title     = {Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks},
    author    = {Reimers, Nils and Gurevych, Iryna},
    booktitle = {Proceedings of EMNLP 2019},
    year      = {2019},
    url       = {https://arxiv.org/abs/1908.10084}
}

Related Poster

If you refer to this work, please also cite the related poster:

• HAL Id: hal-05681918
• URL: https://hal.science/hal-05681918v1