SentenceTransformer based on allenai/specter2_base

This is a sentence-transformers model finetuned from allenai/specter2_base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: allenai/specter2_base
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jagadeesh/zeiss-re-1757443344")
# Run inference
sentences = [
    'We previously demonstrated that neural stem/progenitor cells (NSPCs) were induced within and around the ischemic areas in a mouse model of ischemic stroke. These injury/ischemia-induced NSPCs (iNSPCs) differentiated to electrophysiologically functional neurons in vitro, indicating the presence of a self-repair system following injury. However, during the healing process after stroke, ischemic areas were gradually occupied by inflammatory cells, mainly microglial cells/macrophages (MGs/MΦs), and neurogenesis rarely occurred within and around the ischemic areas. Therefore, to achieve neural regeneration by utilizing endogenous iNSPCs, regulation of MGs/MΦs after an ischemic stroke might be necessary. To test this hypothesis, we used iNSPCs isolated from the ischemic areas after a stroke in our mouse model to investigate the role of MGs/MΦs in iNSPC regulation. In coculture experiments, we show that the presence of MGs/MΦs significantly reduces not only the proliferation but also the differentiation of iNSPCs toward neuronal cells, thereby preventing neurogenesis. These effects, however, are mitigated by MG/MΦ depletion using clodronate encapsulated in liposomes. Additionally, gene ontology analysis reveals that proliferation and neuronal differentiation are negatively regulated in iNSPCs cocultured with MGs/MΦs. These results indicate that MGs/MΦs negatively impact neurogenesis via iNSPCs, suggesting that the regulation of MGs/MΦs is essential to achieve iNSPC-based neural regeneration following an ischemic stroke.',
    "ZEISS Airyscan is an advanced imaging technology that enhances traditional confocal microscopy by using a 32-channel detector to capture more light with higher resolution and sensitivity. Unlike standard confocal systems that rely on a single pinhole, Airyscan collects the entire Airy disk pattern and reconstructs images for super-resolution clarityâ down to 120 nm laterally. This results in significantly improved signal-to-noise ratio and reduced photodamage, making it ideal for detailed imaging of live cells and biological samples. It's compatible with ZEISS LSM systems like the LSM 880 and 900, offering researchers a powerful tool for high-precision fluorescence microscopy",
    "ZEISS Airyscan is an advanced imaging technology that enhances traditional confocal microscopy by using a 32-channel detector to capture more light with higher resolution and sensitivity. Unlike standard confocal systems that rely on a single pinhole, Airyscan collects the entire Airy disk pattern and reconstructs images for super-resolution clarityâ down to 120 nm laterally. This results in significantly improved signal-to-noise ratio and reduced photodamage, making it ideal for detailed imaging of live cells and biological samples. It's compatible with ZEISS LSM systems like the LSM 880 and 900, offering researchers a powerful tool for high-precision fluorescence microscopy",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6498, 0.6498],
#         [0.6498, 1.0000, 1.0000],
#         [0.6498, 1.0000, 1.0000]])

Evaluation

Metrics

Information Retrieval

• Dataset: ir-eval
• Evaluated with InformationRetrievalEvaluator

Information Retrieval

• Dataset: ir-eval
• Evaluated with InformationRetrievalEvaluator

Information Retrieval

• Dataset: ir-eval
• Evaluated with InformationRetrievalEvaluator

Information Retrieval

• Dataset: ir-eval
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 17,793 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 2 tokensmean: 283.9 tokensmax: 512 tokens	min: 91 tokensmean: 465.57 tokensmax: 512 tokens

• Samples:

  anchor	positive
  Nutrition and resilience are linked, though it is not yet clear how diet confers stress resistance or the breadth of stressors that it can protect against. We have previously shown that transiently restricting an essential amino acid can protect Drosophila melanogaster against nicotine poisoning. Here, we sought to characterize the nature of this dietary-mediated protection and determine whether it was sex, amino acid and/or nicotine specific. When we compared between sexes, we found that isoleucine deprivation increases female, but not male, nicotine resistance. Surprisingly, we found that this protection afforded to females was not replicated by dietary protein restriction and was instead specific to individual amino acid restriction. To understand whether these beneficial effects of diet were specific to nicotine or were generalizable across stressors, we pre-treated flies with amino acid restriction diets and exposed them to other types of stress. We found that some of the diets th...	the zeiss stemi 508 is an apochromatic stereo microscope with an 8:1 zoom range, designed for high-contrast, color-accurate three-dimensional observation and documentation of diverse samples. its ergonomic design and robust mechanics support demanding applications in laboratory and industrial settings. key research and application areas: - biological research: suitable for observing the development and growth of model organisms like spider crabs, chicken, mouse, or zebrafish, including the evaluation, sorting, selection, or dissection of eggs, larvae, or embryos. it is also used in botany to observe changes in plant organs, diseases, and root development, in entomology for insect observation, documentation, and identification, in marine biology to study the life and reproduction of fish, and in parasitology for detecting and identifying the spread of parasites. the microscope is valuable for forensic analysis of ammunition parts, tool marks, documents, fibers, coatings, glass, textiles...
  The controlled supply of bioactive molecules is a subject of debate in animal nutrition. The release of bioactive molecules in the target organ, in this case the intestine, results in improved feed, as well as having a lower environmental impact. However, the degradation of bioactive molecules' in transit in the gastrointestinal passage is still an unresolved issue. This paper discusses the feasibility of a simple and cost-effective procedure to bypass the degradation problem. A solid/liquid adsorption procedure was applied, and the operating parameters (pH, reaction time, and LY initial concentration) were studied. Lysozyme is used in this work as a representative bioactive molecule, while Adsorbo ® , a commercial mixture of clay minerals and zeolites which meets current feed regulations, is used as the carrier. A maximum LY loading of 32 mg LY /g AD (LY(32)-AD) was obtained, with fixing pH in the range 7.5-8, initial LY content at 37.5 mg LY /g AD , and reaction time at 30 min. A ful...	the zeiss evo family of scanning electron microscopes offers a modular and versatile platform for a wide range of scientific and industrial investigations, combining high-performance imaging and analysis with intuitive operation for users of varying experience levels. key research and application areas: - materials science: characterizing the morphology, structure, and composition of diverse materials, including metals, composites, polymers, ceramics, and coatings, for research and development. this includes investigating surface structures, fractures, inclusions, and grain boundaries. the evo supports advanced material analysis through techniques like energy dispersive spectroscopy (eds) and electron backscatter diffraction (ebsd). - life sciences: enabling the examination of biological specimens in their native or near-native hydrated states using variable and extended pressure modes. applications include imaging cells, tissues, plants, and microorganisms for structural and morpholog...
  Amorphous potassium sodium niobate (KNN) films were synthesized at 300 °C through the radio frequency magnetron sputtering method and subsequently crystallized by post-annealing at 700 °C in various alkali element atmospheres (Na and K). The as-deposited film is notably deficient in alkali metal elements, particularly K, whereas the loss of alkali elements in the films can be replenished through annealing in an alkali element atmosphere. By adjusting the molar ratio of Na and K in the annealing atmosphere, the ratio of Na/K in the resultant film varied, consequently suggesting the efficiency of this method on composition regulation of KNN films. Meanwhile, we also found that the physical characteristics of the films also underwent differences with the change of an annealing atmosphere. The films annealed in a high Na atmosphere exhibit large dielectric losses with limited piezoelectric vibration behavior, while annealing in a high K atmosphere reduces the dielectric losses and enhances...	the zeiss sigma family of field emission scanning electron microscopes (fe-sems) offers versatile solutions for high-quality imaging and advanced analytical microscopy across a multitude of scientific and industrial domains. these instruments are engineered for reliable, high-end nano-analysis, combining fe-sem technology with an intuitive user experience to enhance productivity. key research and application areas: - advancing materials science: facilitating the development and understanding of novel materials by enabling the investigation of micro- and nanoscale structures. this includes characterizing metals, alloys, polymers, catalysts, and coatings for various applications such as electronics and energy. - driving innovation in nanoscience and nanomaterials: providing capabilities for the analysis of nanoparticles, thin films, 2d materials (like graphene and mos2), and other nanostructures to understand their properties and potential applications. - supporting energy research: enab...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 5.1.0
• Transformers: 4.56.1
• PyTorch: 2.8.0.dev20250319+cu128
• Accelerate: 1.10.1
• Datasets: 3.6.0
• Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}