hi-di-hi-base

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("carsondial/slinger20241231-1")
# Run inference
sentences = [
    'What is article marketing and how does it work?\nHow can I use article marketing to increase traffic to my website?\nWhat are some effective strategies for an article marketing campaign?',
    "Advertising An Article Marketing\nAs every website owner knows, content is essential to raising your rankings in the search engines. There's another way to use content to bring even more visitors to your site. It's called article marketing. You can use one article in dozens of venues, to multiply your exposure exponentially. More traffic means more sales. Let's look at some strategies you can use in your article marketing campaign.Let's say you sell seeds, gardening supplies and gardening books on your site. You're facing stiff competition, as there are many websites marketing these products.\nHow do you get your share of attention? The answer is simple: exposure, as much as you can generate! Article marketing is a natural for this purpose. Write one article, and use it to advertise your site, over and over again.\nYou know your products and their uses best. Write an article on how to start seeds. Give detailed and useful information such as equipment required, temperature and seasonal concerns, thinning and transplanting. There are many topics you can write about.\nWrite an article telling your reader how to choose a good pair of garden gloves, and the advantages of a leather glove for one task, or a sturdy cotton glove for light work.\nOr, tell your reader how a quality bulb planter makes short work of a big planting project. I'm sure you get the idea. Do not write an advertisement!\nWhen your articles are prepared, your related subjectsarticle marketing campaign is ready to launch. Do a search using the term ”article directories gardening”. You'll get hundreds or even thousands of results. Visit some of these article directories and browse their gardening category. Take a look at the quality of their articles. Unfortunately, there are many article directories with poorly written articles, so be choosy when selecting those you want to be associated with, as quality-conscious directories get the most readers.\nHere's how it works: you submit your article to the directory. You are not paid money, but are rewarded with a byline and a link to your site. Every person who reads your article has the opportunity to click through to your site. Give your reader useful information in an engaging style, and chances are good they'll come to your site to see what else they can learn. You can submit the same article to other directories, and watch your traffic and sales grow.\nYour article marketing strategy should include contacts with other website owners, especially those with an e-newsletter. Offer your article as newsletter content in exchange for a byline. See if you can negotiate reciprocal links. Of course you don't want to contact direct competition. A florist, greenhouse or organic gardening site might be good choices.\nYour article marketing campaign's success is directly related to the amount of exposure you create through your effort. One indirect result of a good article marketing campaign is establishing yourself as an expert in your business. Success is measured by traffic and sales. Be prolific and diligent!\nAbout the Author: InsightsOnMarketing provides readers with the latest reviews, articles, commentaries and write-ups on all article marketing, article directories, article reader related subjects",
    'Alandari Gray Loveseat\nSporting a pleasing roll arm for classic flair, the Alandari loveseat in soft gray exudes an easy elegance sure to please. Feel-good upholstery with linen texturing is made for everyday luxury. Muted-tone toss pillows are a soothing, sophisticated complement.\nFrame constructions have been rigorously tested to simulate the home and transportation environments for improved durability. Frame components are secured with combinations of glue, blocks, interlocking panels and staples. Seats and back spring rails are cut from mixed hardwood and engineered lumber. Stripes and patterns are match cut. All fabrics are pre-approved for wearability and durability against AHFA standards. Cushions are constructed of low melt fiber wrapped over high quality foam. Constructed with a platform seat foundation.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Datasets: dim_768, dim_512, dim_256, dim_128 and dim_64
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

json

• Dataset: json
• Size: 45,000 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 12 tokensmean: 30.74 tokensmax: 112 tokens	min: 51 tokensmean: 382.61 tokensmax: 512 tokens

• Samples:

  anchor	positive
  How to split the Join form into multiple partsAdding HTML and text to the Join form pageCustomizing the login logout message	This tutorial will show you how to split the Join form/page into two or three parts. Whether you want a cleaner looking join form or you want to reduce the number of spam bots this should help.This Tutorial will show you how to add HTML and text to the Join form page. If you have ever wanted to add a few more details or spruce up the Join form then this might be what you have been looking for.This tutorial will show you how to spruce up the login - logout message people see. If you want to add a special message or just change the standard default phrase this one might be for you. It's very simple and only takes a few moments to apply this one.This tutorial will show you how to take advantage and optimize your site using the Page Block feature available in your Administration panel. With a little exploring and tweaking it's possible to boost your site's performance.If your Forum RSS feed links are redirecting to your homepage when clicked on you might need to update the forum module...
  Failed mobile technologies of 2011What happened to SiriDisappointing mobile innovations	Last year we all got excited about mobile-computing products that failed to deliverIt's an awesome time to be a gadget-happy consumer electronics freak. Multi-touch user interfaces. Huge advances in miniaturization and battery life. Cloud-based storage. Mobile computing has never been better.But sometimes, when companies announce incredible new products or technologies, and everybody proclaims that a new era has dawned, and that culture-shifting transformations are about to take place -- nothing happens.Here are five mobile technologies from last year that were supposed to change the world, but didn't.Apple seemed to do everything right with its voice assistant strategy.The company acquired the leading app maker with the best technology. It spent two years perfecting and integrating the technology, and bulking up on servers to handle the number-crunching required to deliver human-like voice interaction.Siri was then launched to huge fanfare.Overnight, people changed how they int...
  What are the different types of web usage for voice over campaigns?What are the standard types of web usage for voice over?What are the different ways to use voice over in web campaigns?	Different Types of Web UsageClearly, nothing has upended the voice over industry quite like the Internet. On all ends of the spectrum, from the basic sourcing of voice over talent to right on down to hyper-targeted, increasingly personalized messaging, it can feel a little “Wild West” to navigate if you’re not going directly through an agent. And one of the biggest stumblers when sourcing for voice talent? The concept of “web usage.”Usage — a.k.a., how the finished voice over is being used — will cause the greatest fluctuation in a voice talent’s quote. As we all know, there are myriad ways any material can be used and distributed online… and not all distribution is created equally.Below, we’ve got a handy little guide detailing all the different, standard types of web usage we tend to come across here at Blue Wave Voiceover (Heck, the way things are going, we might have to update this once more before you finish reading it).Feel free to refer to this when putting together any kind...

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 4
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: True
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.1
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.1
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}