use-token-shuffle-pt
This model is a fine-tuned version of paraphrase-multilingual-mpnet-base-v2, trained on the Ukrainian text corpus UberText 2.0 with token-shuffling augmentation with pool targets. It is part of the Ukrainian Sentence Embeddings collection, which explores the effect of different training strategies on sentence embedding quality for Ukrainian.
Model Description
The model was fine-tuned using a contrastive objective on UberText 2.0, token-shuffling augmentation.
Collection Overview
| Model | Description |
|---|---|
| use-natural-no-pt | Raw UberText 2.0, no augmentation, no pool targets |
| use-natural-pt | Raw UberText 2.0, no augmentation, pool targets |
| use-generated-no-pt | Generated augmentation, no pool targets |
| use-generated-pt | Generated augmentation, pool targets |
| use-translation-no-pt | Back-translation augmentation, no pool targets |
| use-translation-pt | Back-translation augmentation, pool targets |
| use-mask-no-pt | Masking augmentation, no pool targets |
| use-mask-pt | Masking augmentation, pool targets |
| use-dropout-no-pt | Dropout augmentation, no pool targets |
| use-dropout-pt | Dropout augmentation, pool targets |
| use-token-shuffle-no-pt | Token-shuffling augmentation, no pool targets |
| use-token-shuffle-pt | Token-shuffling augmentation, pool targets |
| use-combined-no-pt | Combined augmentation strategies, no pool targets |
| use-combined-pt | Combined augmentation strategies, pool targets |
| use-stochastic-no-pt | Markov-based stochastic augmentation, no pool targets |
| use-stochastic-pt | Markov-based stochastic augmentation, pool targets |
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("victormuryn/use-token-shuffle-pt")
sentences = [
"Проводжає сина мати захищати рідний край",
"Хоч би малесеньку хатину він мріяв мати над Дніпром",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
Training Details
- Base model: paraphrase-multilingual-mpnet-base-v2
- Training corpus: UberText 2.0
- Augmentation: Token-shuffling
- Pool targets: Yes
Citation
To be added
License
Apache 2.0