merge
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Model Stock merge method using Qwen/Qwen2.5-7B-Instruct as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: Qwen/Qwen2.5-7B-Instruct
layer_range: [0, 6]
- model: Qwen/Qwen2.5-7B-Instruct
layer_range: [7, 13]
- model: Qwen/Qwen2.5-7B-Instruct
layer_range: [14, 20]
- model: Qwen/Qwen2.5-7B-Instruct
layer_range: [21, 27]
merge_method: model_stock
base_model: Qwen/Qwen2.5-7B-Instruct
dtype: bfloat16