Mistral-12B
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the Arcee Fusion merge method using mistralai/Mistral-Nemo-Instruct-2407 as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
merge_method: arcee_fusion
base_model: mistralai/Mistral-Nemo-Instruct-2407
dtype: bfloat16
models:
# First, it merges Mistral-Nemo with Lumimaid
- model: mistralai/Mistral-Nemo-Instruct-2407
- model: NeverSleep/Lumimaid-v0.2-12B
parameters:
weight: 0.5
# Then, it passes that result into a second Arcee Fusion layer with absolute-heresy
# This is handled dynamically by Mergekit when you save it as a nested config.