SentenceTransformer based on KBLab/sentence-bert-swedish-cased

This is a sentence-transformers model finetuned from KBLab/sentence-bert-swedish-cased. It maps inputs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Training By Me

Any weights that have been trained by me fall under the following license:

The Non-Commercial Public License (NCPL)

Copyright (c) 2026 LilyPersilly

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software, associated datasets, and any future fine-tuned model weights

(collectively, the "Software"), to run, execute, demonstrate, and study the

Software for public or private purposes (including live streaming, content creation,

and educational use, even if the hosting platform displays advertisements or generates platform-native creator revenue), subject to the following conditions:

CODE RESTRICTION: The source code, associated datasets, and any fine-tuned models derived from the Software may NOT be incorporated into, bundled with, or used to power any commercial software products, SaaS platforms, paid APIs, for-profit business operations, or standalone revenue-generating services.

INCLUSION: Any distribution of modified versions, derivative works, or fine-tuned model weights must also include this exact license text and remain strictly non-commercial.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.

Model Description

• Model Type: Sentence Transformer
• Base model: KBLab/sentence-bert-swedish-cased
• Maximum Sequence Length: 384 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Nej. Lagstadgad arbetstidsförkortning vore ett dråpslag mot Sveriges tillväxt, företagens konkurrenskraft och välfärdens finansiering.',
    'Vi avvisar dekadent konsumtionskultur såväl som godtyckliga globalistiska livsstilsdirektiv.',
    'RUT och ROT ska finnas kvar; de förenklar vardagen, skapar jobb för utrikesfödda och minskar den svarta sektorn.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.0068,  0.0584],
#         [-0.0068,  1.0000, -0.0380],
#         [ 0.0584, -0.0380,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 291 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1	label
  type	string	string	float
  modality	text	text
  details	min: 19 tokensmean: 27.61 tokensmax: 40 tokens	min: 13 tokensmean: 27.62 tokensmax: 41 tokens	min: 0.0mean: 0.28max: 1.0

• Samples:

  sentence_0	sentence_1	label
  Ett absolut krav på att merparten av alla utomnordiska invånare ska deporteras och återvandra till sina ursprungsområden.	Huvudregeln ska vara att icke-svenska medborgare som begår allvarliga brott alltid ska utvisas på livstid.	0.0
  Vi förespråkar att folkets vilja ska styra bidragssystemets utformning direkt genom digitala folkomröstningar.	Svenska arbetstagares trygghet på arbetsmarknaden ska värnas, men kollektivavtal och arbetsmarknad regleras bäst genom nationella och trygga avtal fria från EU-direktiv.	0.0
  Socialförsäkringar och arbetslöshetsersättningar ska höjas kraftigt och knytas till löneutvecklingen. Alla former av bidragstak och utförsäkringar ska avskaffas.	Grundtryggheten ska stärkas. Högerpolitikens straffande 'bidragstak' och utförsäkringar skapar bara fattigdom; vi vill ha starka socialförsäkringar man kan lita på.	1.0

• Loss: CosineSimilarityLoss with these parameters:

  {
      "loss_fct": "torch.nn.modules.loss.MSELoss",
      "cos_score_transformation": "torch.nn.modules.linear.Identity"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• num_train_epochs: 4
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• per_device_train_batch_size: 8
• num_train_epochs: 4
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• dataloader_multiprocessing_context: None
• dataloader_in_order: True
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}
• warmup_ratio: None

Training Time

• Training: 25.7 seconds

Framework Versions

• Python: 3.14.7
• Sentence Transformers: 6.0.0
• Transformers: 5.15.1
• PyTorch: 2.13.0+cu126
• Accelerate: 1.14.0
• Datasets: 5.0.1
• Tokenizers: 0.22.2

Additional Resources

• Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
• Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}