wizard-tulu-dolphin-70b-v1.0-slerp

I used this blend as part of Midnight Rose v2.0.3. I haven't tested it much on its own, but I created this blend to be good at instruction following. It's nothing fancy. I just figured that WizardLM, Tulu, and Dolphin 2.2 together would be a heavy hitter for smarts.

Merge Details

Merge Method

This model was merged using the SLERP merge method.

Models Merged

The following models were included in the merge:

• /home/llm/mergequant/models/BASE/ehartford_dolphin-2.2-70b-32000vocab
• /home/llm/mergequant/models/wizard-tulu-70b-v1.0

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: /home/llm/mergequant/models/wizard-tulu-70b-v1.0
  - model: /home/llm/mergequant/models/BASE/ehartford_dolphin-2.2-70b-32000vocab
merge_method: slerp
base_model: /home/llm/mergequant/models/wizard-tulu-70b-v1.0
parameters:
  t:
    - value: 0.5
dtype: float16

models:
  - model: /home/llm/mergequant/models/BASE/NousResearch_Llama-2-70b-hf
    # no parameters necessary for base model
  - model: /home/llm/mergequant/models/BASE/allenai_tulu-2-dpo-70b
    parameters:
      density: 0.35
      weight: 0.75
  - model: /home/llm/mergequant/models/BASE/WizardLM_WizardLM-70B-V1.0
    parameters:
      density: 0.35
      weight: 0.5
merge_method: dare_ties
base_model: /home/llm/mergequant/models/BASE/NousResearch_Llama-2-70b-hf
parameters:
  normalize: true
  int8_mask: true
dtype: float16
tokenzer_source: union