SentenceTransformer based on intfloat/multilingual-e5-base

This is a sentence-transformers model finetuned from intfloat/multilingual-e5-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/multilingual-e5-base
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("meet136/gst-legal-embeddings-v1")
# Run inference
sentences = [
    'Is reverse charge applicable on import of services?',
    'Section 3: 3. The Government shall, by notification, appoint the following classes of officers for Officers under\n\n3. The Government shall, by notification, appoint the following classes of officers for Officers under\nthe purposes of this Act, namely:–– this Act.\n(a) Principal Chief Commissioners of Central Tax or Principal Directors General of\nCentral Tax,\n(b) Chief Commissioners of Central Tax or Directors General of Central Tax,\n(c) Principal Commissioners of Central Tax or Principal Additional Directors\nGeneral of Central Tax,\n(d) Commissioners of Central Tax or Additional Directors General of Central Tax,\n12 THE GAZETTE OF INDIA EXTRAORDINARY [PART II—\n(e) Additional Commissioners of Central Tax or Additional Directors of\nCentral Tax,\n(f) Joint Commissioners of Central Tax or Joint Directors of Central Tax,\n(g) Deputy Commissioners of Central Tax or Deputy Directors of Central Tax,\n(h) Assistant Commissioners of Central Tax or Assistant Directors of\nCentral Tax, and\n(i) any other class of officers as it may deem fit:\nProvided that the officers appointed under the Central Excise Act, 1944 shall be deemed 1 of 1944.\nto be the officers appointed under the provisions of this Act.\nAppointment 4. (1) The Board may, in addition to the officers as may be notified by the Government\nof officers. under section 3, appoint such persons as it may think fit to be the officers under this Act.\n\n(7) Notwithstanding anything contained in sub-section (1), where the goods being\nsent or taken on approval for sale or return are removed before the supply takes place, the\ninvoice shall be issued before or at the time of supply or six months from the date of removal,\nwhichever is earlier.\nExplanation.––For the purposes of this section, the expression “tax invoice” shall\ninclude any revised invoice issued by the supplier in respect of a supply made earlier.\nProhibition of 32. (1) A person who is not a registered person shall not collect in respect of any\nunauthorised supply of goods or services or both any amount by way of tax under this Act.\ncollection of\ntax. (2) No registered person shall collect tax except in accordance with the provisions of\nthis Act or the rules made thereunder.\nAmount of 33. Notwithstanding anything contained in this Act or any other law for the time being\ntax to be in force, where any supply is made for a consideration, every person who is liable to pay tax\nindicated in\nfor such supply shall prominently indicate in all documents relating to assessment, tax\ntax invoice\ninvoice and other like documents, the amount of tax which shall form part of the price at\nand other\ndocuments. which such supply is made.\nCredit and debit 34. (1) Where a tax invoice has been issued for supply of any goods or services or\nnotes. both and the taxable value or tax charged in that tax invoice is found to exceed the taxable\nvalue or tax payable in respect of such supply, or where the goods supplied are returned by\nthe recipient, or where goods or services or both supplied are found to be deficient, the\nregistered person, who has supplied such goods or services or both, may issue to the\nrecipient a credit note containing such particulars as may be prescribed.\nSEC. 1] THE GAZETTE OF INDIA EXTRAORDINARY 31',
    'Section 3: 3. The Government shall, by notification, appoint the following classes of officers for Officers under\n\n3. The Government shall, by notification, appoint the following classes of officers for Officers under\nthe purposes of this Act, namely:–– this Act.\n(a) Principal Chief Commissioners of Central Tax or Principal Directors General of\nCentral Tax,\n(b) Chief Commissioners of Central Tax or Directors General of Central Tax,\n(c) Principal Commissioners of Central Tax or Principal Additional Directors\nGeneral of Central Tax,\n(d) Commissioners of Central Tax or Additional Directors General of Central Tax,\n12 THE GAZETTE OF INDIA EXTRAORDINARY [PART II—\n(e) Additional Commissioners of Central Tax or Additional Directors of\nCentral Tax,\n(f) Joint Commissioners of Central Tax or Joint Directors of Central Tax,\n(g) Deputy Commissioners of Central Tax or Deputy Directors of Central Tax,\n(h) Assistant Commissioners of Central Tax or Assistant Directors of\nCentral Tax, and\n(i) any other class of officers as it may deem fit:\nProvided that the officers appointed under the Central Excise Act, 1944 shall be deemed 1 of 1944.\nto be the officers appointed under the provisions of this Act.\nAppointment 4. (1) The Board may, in addition to the officers as may be notified by the Government\nof officers. under section 3, appoint such persons as it may think fit to be the officers under this Act.\n\n(3) The certificate of registration issued to a person under sub-section (1) shall be\ndeemed to have not been issued if the said registration is cancelled in pursuance of an\napplication filed by such person that he was not liable to registration under section 22 or\nsection 24.\n140. (1) A registered person, other than a person opting to pay tax under section 10, Transitional\nshall be entitled to take, in his electronic credit ledger, the amount of CENVAT credit carried arrangements\nfor input tax\nforward in the return relating to the period ending with the day immediately preceding the\ncredit.\nappointed day, furnished by him under the existing law in such manner as may be prescribed:\nProvided that the registered person shall not be allowed to take credit in the following\ncircumstances, namely:—\n(i) where the said amount of credit is not admissible as input tax credit under this\nAct; or\n(ii) where he has not furnished all the returns required under the existing law for\nthe period of six months immediately preceding the appointed date; or\n(iii) where the said amount of credit relates to goods manufactured and cleared\nunder such exemption notifications as are notified by the Government.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.8032, 0.7968],
#         [0.8032, 1.0000, 0.9908],
#         [0.7968, 0.9908, 1.0000]])

Evaluation

Metrics

Information Retrieval

• Dataset: gst-eval
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 510 training samples

• Columns: sentence_0 and sentence_1

• Approximate statistics based on the first 510 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 10 tokensmean: 13.68 tokensmax: 20 tokens	min: 512 tokensmean: 512.0 tokensmax: 512 tokens

• Samples:

  sentence_0	sentence_1
  Import of services par RCM applicable hai kya?	Section 3: 3. The Government shall, by notification, appoint the following classes of officers for Officers under

  3. The Government shall, by notification, appoint the following classes of officers for Officers under the purposes of this Act, namely:–– this Act.

  (a) Principal Chief Commissioners of Central Tax or Principal Directors General of Central Tax,

  (b) Chief Commissioners of Central Tax or Directors General of Central Tax,

  (c) Principal Commissioners of Central Tax or Principal Additional Directors General of Central Tax,

  (d) Commissioners of Central Tax or Additional Directors General of Central Tax, 12 THE GAZETTE OF INDIA EXTRAORDINARY [PART II—

  (e) Additional Commissioners of Central Tax or Additional Directors of Central Tax,

  (f) Joint Commissioners of Central Tax or Joint Directors of Central Tax,

  (g) Deputy Commissioners of Central Tax or Deputy Directors of Central Tax,

  (h) Assistant Commissioners of Central Tax or Assistant Directors of Central Tax, and

  (i) a... | | Can ITC be claimed on health insurance premium for employees? | Section 3: 3. The Government shall, by notification, appoint the following classes of officers for Officers under

  3. The Government shall, by notification, appoint the following classes of officers for Officers under the purposes of this Act, namely:–– this Act. (a) Principal Chief Commissioners of Central Tax or Principal Directors General of Central Tax, (b) Chief Commissioners of Central Tax or Directors General of Central Tax, (c) Principal Commissioners of Central Tax or Principal Additional Directors General of Central Tax, (d) Commissioners of Central Tax or Additional Directors General of Central Tax, 12 THE GAZETTE OF INDIA EXTRAORDINARY [PART II— (e) Additional Commissioners of Central Tax or Additional Directors of Central Tax, (f) Joint Commissioners of Central Tax or Joint Directors of Central Tax, (g) Deputy Commissioners of Central Tax or Deputy Directors of Central Tax, (h) Assistant Commissioners of Central Tax or Assistant Directors of Central Tax, and (i) any other ... | | Office ka kiraya par ITC claim kar sakte hain? | Section 110: 110. (1) A person shall not be qualified for appointment as— President and

  4. (1) A person shall not be qualified for appointment as— President and Members of (a) the President, unless he has been a Judge of the Supreme Court or is or has Appellate been the Chief Justice of a High Court, or is or has been a Judge of a High Court for a Tribunal, period not less than five years; their qualification, (b) a Judicial Member, unless he— appointment, conditions of (i) has been a Judge of the High Court; or service, etc. (ii) is or has been a District Judge qualified to be appointed as a Judge of a High Court; or 68 THE GAZETTE OF INDIA EXTRAORDINARY [PART II— (iii) is or has been a Member of Indian Legal Service and has held a post not less than Additional Secretary for three years; (c) a Technical Member (Centre) unless he is or has been a member of Indian Revenue (Customs and Central Excise) Service, Group A, and has completed at least fifteen years of service in Group A; (d... |

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 5
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

• do_predict: False
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Training Time

• Training: 1.1 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.4.1
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}