This Model2Vec model was created by using Tokenlearn, with nomic-embed-text-v2-moe as a base.
The output dimension is 384.
The evaluation in the model card was executed using this distilled model, not the original.
This model was trained in streaming mode over large precomputed feature shards with incremental PCA (384d), vocabulary quantization capped at 32k effective tokens, and fine-tuning optimizations for large-scale data.
This is smaller but better model than cnmoro/nomic-embed-text-v2-moe-distilled-high-quality
Usage
Load this model using model2vec library:
from model2vec import StaticModel
model = StaticModel.from_pretrained("cnmoro/static-nomic-384-pten")
# Compute text embeddings
embeddings = model.encode(["Example sentence"])
Or using sentence-transformers library:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('cnmoro/static-nomic-384-pten')
# Compute text embeddings
embeddings = model.encode(["Example sentence"])