all-MiniLM-L6-v2-multi-qa-dare-ties-merge
A demonstration model produced by the native model-merging feature (SentenceTransformer.merge)
added to Sentence Transformers. It randomly drops 30% of each task delta and rescales, then applies sign election (dare_ties), relative to sentence-transformers/all-MiniLM-L6-v2 as the base.
Merged checkpoints:
sentence-transformers/all-MiniLM-L6-v2sentence-transformers/multi-qa-MiniLM-L6-cos-v1- Base model (delta reference):
sentence-transformers/all-MiniLM-L6-v2
How it was created
from sentence_transformers import SentenceTransformer
merged = SentenceTransformer.merge(
models=["sentence-transformers/all-MiniLM-L6-v2",
"sentence-transformers/multi-qa-MiniLM-L6-cos-v1"],
weights=[0.5, 0.5],
densities=[0.7, 0.7],
method="dare_ties",
base_model="sentence-transformers/all-MiniLM-L6-v2",
output_path="all-MiniLM-L6-v2-multi-qa-dare-ties-merge",
dtype="float32",
)
Usage
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("yjoonjang/all-MiniLM-L6-v2-multi-qa-dare-ties-merge")
emb = model.encode(["Model merging combines fine-tuned checkpoints.",
"It often beats each individual model."])
print(model.similarity(emb, emb))
License
Derivative of the two base models above; their licenses apply. See each base model's card.