Custom Contextual Embedding Model (v1.0-FineTuned)
This is a specialized, fine-tuned dense text embedding model engineered for production Retrieval-Augmented Generation (RAG), context-aware semantic search, and document reranking.
This model has undergone custom contrastive instruction tuning to improve cross-domain query-to-document matching and handling of nuanced contextual semantics.
Key Improvements & Features
- Custom Contrastive Fine-Tuning: Trained using Multiple Negatives Ranking Loss (MNRL) paired with hard-negative mining for high-precision retrieval.
- Enhanced Context Window: Retains structural context for long-form passages (up to 512–8192 tokens depending on sequence truncation limits).
- Low-Latency Retrieval: 0.6B parameter scale balances embedding quality with fast query-side inference on standard GPU infrastructure.
- Optimized Cosine Space: Specifically calibrated for Cosine Similarity metric evaluation, eliminating the need for expensive vector recalibration.
Usage (Sentence-Transformers)
Using this model becomes easy when you have sentence-transformers installed:
pip install -U sentence-transformers
from sentence_transformers import SentenceTransformer, util
model = SentenceTransformer(
"Layasaran/text_embed_0.5b",
trust_remote_code=True
)
texts = [
"Scientists explore the universe driven by curiosity.",
"Children learn through curious exploration.",
"Historical discoveries began with curious questions.",
"Animals use curiosity to adapt and survive.",
"Philosophy examines the nature of curiosity.",
]
doc_embeddings = model.encode(texts, convert_to_tensor=True)
query = "How do children acquire knowledge?"
query_embedding = model.encode(query, convert_to_tensor=True)
similarity_scores = util.cos_sim(query_embedding, doc_embeddings)[0]
top_k = 3
top_indices = similarity_scores.argsort(descending=True)[:top_k]
print(f"Query: '{query}'\n")
print("Top Retrieved Contexts for RAG Prompt:")
print("-" * 50)
retrieved_context = []
for idx in top_indices:
score = float(similarity_scores[idx])
text = texts[idx]
retrieved_context.append(text)
print(f"Score: {score:.4f} | Text: {text}")
rag_context_str = "\n".join([f"- {doc}" for doc in retrieved_context])
rag_prompt = f"""Use the following context to answer the question:
Context:
{rag_context_str}
Question: {query}
Answer:"""
print("\n" + "=" * 50)
print("Final RAG Prompt structure:")
print("=" * 50)
print(rag_prompt)