RetrievaEmbedding-01: AMBER

The AMBER (Adaptive Multitask Bilingual Embedding Representations) is a text embedding model trained by Retrieva, Inc. This model is primarily designed for Japanese, but it also supports English. We trained this model on various datasets related to Japanese and English.

This model size is 132M parameters (base size).

Model Details

Model Description

The AMBER model is a text embedding model based on the sbintuitions/modernbert-ja-130m architecture, designed for Japanese text. This model was trained on a variety of datasets related to Japanese, and also includes English datasets. The model can be used for English text as well. During training, prompts (instructions) in natural language were included, allowing the model to generate embeddings tailored to specific tasks.

• Developed by: Retrieva, Inc.
• Model type: Based on the ModernBERT Architecture.
• Language(s) (NLP): Primarily Japanese (optional support for English).
• License: Apache 2.0
• Finetuned from model: sbintuitions/modernbert-ja-130m
• Model Type: Sentence Transformer
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 512 dimensions
• Similarity Function: Cosine Similarity

Uses

How to Get Started with the Model

Install Library

First install the python library using pip:

pip install sentence-transformers sentencepiece

Run Inference

Then you can load this model and run inference.

You can specify the prompt at inference time by adding an argument called prompt to model.encode. The prompts used in the Japanese benchmark are described in jmteb/tasks, and the prompts used in the English benchmark are described in mteb/models/retrieva_en.py.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("retrieva-jp/amber-base")
# Run inference
queries = [
    "自然言語処理とはなんですか？",
    "株式会社レトリバについて教えて",
]
documents = [
    "自然言語処理（しぜんげんごしょり、英語: Natural language processing、略称：NLP）は、人間が日常的に使っている自然言語をコンピュータに処理させる一連の技術であり、人工知能と言語学の一分野である。",
    "株式会社レトリバは、自然言語処理と機械学習を核としたAI技術で組織の課題解決を支援するテクノロジー企業である。",
]

queries_embeddings = model.encode(queries, prompt_name="Retrieval-query")
documents_embeddings = model.encode(documents, prompt_name="Retrieval-passage")

similarities = model.similarity(queries_embeddings, documents_embeddings)
print(similarities.shape)

Training Details

Training Data

We used multiple datasets to train this model. We selected datasets from llm-jp-eval, llm-japanese-dataset, and hpprc/emb for Japanese datasets. For English datasets, we mainly used some of the datasets utilized in Asai et al. (2023). Additionally, we partially used the English datasets at the sentence-transformers repository and kilt-tasks. To consider cross-lingual between Japanese and English, we also used translation datasets between Japanese and English.

For Japanese, we used synthetic data created by LLM to prepare a sufficient amount of training data.

Evaluation

We evaluated the model on the following benchmarks:

• Japanese Benchmark: JMTEB
• Japanese Retrieval Tasks: JQaRA, JaCWIR, MLDR Japanese Subset
• English Benchmark: MTEB(eng, v2).

The scores in the table are all calculated by us unless otherwise noted.

Japanese Benchmark: JMTEB

Note that the Mean (TaskType) in the following leaderboard is the same as the Avg. in the original JMTEB leaderboard.

The files used for evaluation are stored in the jmteb directory.

Japanese Retrieval Tasks: JQaRA, JaCWIR, MLDR Japanese Subset

The files used for MLDR are stored in the mldr directory.

The prompts used in JQaRA and JaCWIR are Retrieval-query and Retrieval-passage described in config_sentence_transformers.json.

English Benchmark: MTEB(eng, v2)

The files used for evaluation are stored in the mteb directory.

Citation

BibTeX:

@inproceedings{amber2025,
    title = {インストラクションと複数タスクを利用した日本語向け分散表現モデルの構築},
    author = {勝又智 and 木村大翼 and 西鳥羽二郎},
    booktitle = {言語処理学会第31回年次大会発表論文集},
    year = {2025},
}

More Information

https://note.com/retrieva/n/n4ee9d304f44d (in Japanese)

Model Card Authors

Satoru Katsumata, Daisuke Kimura, Jiro Nishitoba

Model Card Contact

pr[at]retrieva.jp