SentenceTransformer based on BAAI/bge-m3

This is a sentence-transformers model finetuned from BAAI/bge-m3. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-m3
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("pameydorke/arcanum-cross-platform-retriever")
# Run inference
queries = [
    'loot wolf cubs fallout',
]
documents = [
    'Wolf Cave. Wolf Cave. Location. The cave is located at 1230W 1530S, roughly midway between Shrouded Hills, Dernholm, and Black Root. It is marked on your map when you explore the immediate area. Area. The cave entrance is at Point A. On the east side (point 1) are a Wolf and eight Wolf Cubs guarding a lootable skeleton and some containers with random items and some coin. On the west side (point 2) are three Putrid Rodents guarding another lootable skeleton with lockpicks and a barrel. Point 3 has four sewer rats guarding a single barrel with some health potions. Point 4 is unguarded and has a barrel with dynamite and other random tech items.',
    'Original Post: Unusual Playthroughs?. Thinking along the lines of my [other topic](https://www.reddit.com/r/arcanum/comments/4tcgiz/meta_abuse_dominate_will/?st=iqrbwiyg&sh=5dcf12db), what are some of the more unusual/cool ways that you\'ve been able to play through the game? \n\nI\'d definitely recommend that new players do a regular run or two first (preferably with a really persuasive character to be able to get some of the "best" plot points/outcomes), but I love the range of options provided by the rich, open-ended world. Some of my favorites include:\n\n* Evil necromancer playthrough. You can kill pretty much all of the "important" NPCs and use the level 2 black necromancy power to torture their spirits into giving up whatever plot information you need for the next step. In my case I played a half-orc who picked a fight with anyone and everyone who insulted him for being a half-orc. \n\n* Mind control/meta abuse: by exploiting the reflection shield bug to make mental domination permanent, you can go through the game permanently enslaving just about everyone in the world. In addition to reducing entire cities to mindless servants who stand in place waiting for your next command you can add most people (and monsters!) to your retinue instead of the regular companions. \n\n* Low-intelligence playthrough: maybe it\'s not *that* unusual, but a run with a character who uses the dumb dialog options is worth it for comedic effect alone. Your journal entries are often hilarious, and I never fail to laugh out loud when Raven explains the Falcon\'s Ache quest by telling you "don\'t hit! OK? Don\'t hit!" User 1: I\'ve made a few gunfighters... in a game where magic is so ridiculously imbalanced I felt like the odd man out to be sure User 2: My favorite has been The Mad Bomber. Pyromaniac background and only do explosive tech, throwing skills and a few other things to make it fun, (planting bombs on people is fun). You use bombs as weapons. User 3: The one time I managed to finish the story was as a half-ogre, with the rare half-ogre birth (lowers your strength and constitution to boost intelligence). Maxed intelligence and charisma, played as a diplomatic wizard. The writing for the dialogues were particularly clever for this one. 90% of the npcs would initially have a negative reaction modifier and their opening statement would be something condescending or insulting because they expected me to be an idiot like all other half-ogres, and then my initial responses usually included something like "Excuse me sir, but I would advise you to rephrase that before I rip your arms off." Cue the npc shitting their pants followed by an immediate large boost to the npc\'s reaction and them apologizing for being an asshole. It was kind of like the opposite of your half-orc necromancer. Instead of killing everyone that insulted me, I persuaded them to bend over backwards for me so that they could keep living. I talked my way through almost everything, including the final boss. ',
    "Original Post: Arcanum on a Lenovo Legion Go 2. Sorry, if I make any mistakes. English is not my first language and I'm posting for the first time.\n\nYesterday I wrestled with my Legion go 2 and the unofficial patch. It started but the screen was black and would only show an image, if I moved the mouse. The trick was to not install the high res patch. For some reason it doesn't communicate well with the hardware. I don't even need a controller support mod, because I can use the trackpad or the Legion go's own mode, where you can control the mouse with the left stick.\n\nI hope this helps.  User 1: Have you tried [Arcanum Universal Setup for UAP](https://www.nexusmods.com/arcanumofsteamworksandmagickobscura/mods/4)?",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4995, 0.0530, 0.0133]])

Evaluation

Metrics

Information Retrieval

• Dataset: arcanum-test
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 4,612 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative
  type	string	string	string
  modality	text	text	text
  details	min: 5 tokensmean: 8.88 tokensmax: 14 tokens	min: 22 tokensmean: 398.81 tokensmax: 951 tokens	min: 13 tokensmean: 428.8 tokensmax: 999 tokens

• Samples:

  anchor	positive	negative
  compound bow explosives requirement	Pyrotechnic Bow. Pyrotechnic Bow. The Pyrotechnic Bow Schematic can be found on the second level of the Black Mountain Clan. Expertise. Requires an expertise of 45 in Explosives Requires an expertise of 20 in Mechanical. Ingredients. Fire Obstruction + Compound Bow Fire Obstruction is the sixth degree in Explosvies The Compound Bow schematic can be purchased from Blacksmiths. Effects. One unit can be obtained per unit of both components Stats: 1-10 HD, 2-5 FT, 5-20 FD, 8 SP, 20 RNG, 5 WE, 80 QU. Used as an ingredient for:. None. Notes. The Mistress of Fire in Tulla carries a Pyrotechnic Bow. Item Page	Charged Sword. Charged Sword. The Charged Sword Schematic can be purchased from Blacksmiths. Expertise. Requires an expertise of 5 in Smithy Requires an expertise of 40 in Electric. Ingredients. Balanced Sword + Large Capacitor Balanced Sword is the second degree in Smithy Large capacitors are sold by Inventor Shops. Effects. One sword can be obtained per unit of both components 3-12 HD, 3-9 FT, 1-20 ED, 10 SP, 80 WE, 80 QU (Tech). Used as an ingredient for:. None. Notes. None. Item Page
  main choices arcana	Original Post: What aspects of Arcanum do you wish other games would adopt?. For me, I really liked the way you could just bash doors down. It always bugged me in other games, where you'd have a giant hammer and godlike strength, or a rocket launcher, yet perfectly simple doors would deter you. Case in pointNot Arcanum! The only fault I can find with the system is Virgil's overenthusiastic response to the idea of breaking down doors. What aspects of Arcanum would you like to see "Bleed out" into more mainstream RPGs? U...	Original Post: New Arcanum player, Looking for mod recommendations.. Hey y'all. I just downloaded this game and I'm looking for some good mods to download, preferably with links to where I can. I'm mainly looking for quality of life and bug fixing stuff. User 1: For a base build, use the official patch to 1.0.7.4 and then the unofficial patch to 2.02 for best results on modern systems. In addition to the UAP mod, multiverse mod has a lot of support, but it has heavily re-balanced the game. And then there are misc mods and downloads galore at Terra Arcanum that you can pick and choose through. User 2: first off, ya need drog blacktooth's unofficial patch. it restores and fixes literally thousands of issues with arcanum including content.
  how do i solve tulla puzzle	Tulla. Tulla. "I can't STAND Tulla..everyone is so bloody boring. You'd think it was a damned graveyard with all of the walking dead around here..". —Geoffrey Tarellond-Ashe. Tulla Exterior. Area Note. You can find a lot of good magical loot in the city, but if you attack any magicians to get it, the entire city will turn again you. Also, if you joined the dark elves in T'sen-Ang, the gate guard won't let you in, and fighting him will turn the city against you, giving you little reason to go in. 1 - Entrance Gate. The only normal way into Tulla is to use the teleportation gate outside the city. When you talk to the elf next to it, he'll inform you that Jorian is expecting you, and he'll allow you to use the gate.  Early access is possible by using a pathing glitch, however. 2 - Study Room. Inside you'll meet Herzod Munk, who is something less than an eager student. He'll ask you to take his final Phantasm College exam for him. If you agree, he'll give you Fa'al Kin's Coin, the key to t...	Shrouded Hills. Shrouded Hills. Shrouded Hills Type: Town Coordinates: West 1410, South 1317 Inhabitants: Constable OwensLloyd GurloesDoc RobertsJongle DunneGaylinGypsySogg Mead MugPercival TooneBessie Toone (deceased) Quests: Destroy the Town's Steam EngineDestroy the construction materials for the new bridgeFind the Amulet of N'TalaFind Pure Ore for LloydFree the Ghost of Bessie TooneFix the Town's Steam EnginePickup Jongle's PackageRistrezze: find a Camera OR some Bessie Toone paraphenaliaRob the BankStop The Bank RobberyThe Thieves and the Bridge. Shrouded Hills is a small mining town east of the Stonewall Range, south-west of Tarant and north-west of Dernholm. It is well-known for the Bessie Toone Mine and its agriculture, where a majority of the towns production comes from. It is also a part of the Unified Kingdom, and is governed by Constable Owens. The town is primitive in that it has been barely touched by either magick or technology. Services. Shrouded Hills is small, yet pro...

• Loss: GISTEmbedLoss with these parameters:

  {
      "guide": "SentenceTransformer('BAAI/bge-m3')",
      "temperature": 0.07,
      "margin_strategy": "absolute",
      "margin": 0.0,
      "contrast_anchors": true,
      "contrast_positives": true,
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 5
• num_train_epochs: 1
• learning_rate: 1e-05
• lr_scheduler_type: cosine
• warmup_steps: 0.1
• weight_decay: 0.01
• gradient_accumulation_steps: 2
• per_device_eval_batch_size: 5
• batch_sampler: no_duplicates

All Hyperparameters

• per_device_train_batch_size: 5
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 1e-05
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 2
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 5
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Training Time

• Training: 30.7 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.6.0
• Transformers: 5.13.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

GISTEmbedLoss

@misc{solatorio2024gistembed,
    title={GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning},
    author={Aivin V. Solatorio},
    year={2024},
    eprint={2402.16829},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}