Winter Garden 7B - Γ
It was mentioned that we are in the open ai dark winter; so I thought I would make myself a nice winter garden.
An experiment
This time I did something a bit different.
- Mistral-7B-v0.1
and merged in
- Yarn-Mistral-7b-128k
- Thespis-Balanced-7b-v1
- ZySec-7B-v1
- LemonadeRP-4.5.3
- Noromaid-7B-0.4-DPO
- Prima-LelantaclesV6-7b
- West-Hermes-7B
- Capricorn-7B-DPO
- kun-kunoichi-v1-DPO-v2-SLERP-7B
- Kunoichi-DPO-v2-7B
- WestLake-7B-v2-laser-truthy-dpo
- StrangeMerges_6-7B-dare_ties
- NeuralMarcoro14-7B
- multi_verse_model
- Multi-Verse-RP-7B
- MonarchLake-7B
- AlphaMonarch-7B
in an iterative DARE-TIES tree merge, ordering the merge order by tensor-relative cosine similarity until the merge branches resolve to a single value.
Chat Template
Basic Mistral <s>[INST][/INST] works pretty well. It seems smart, but we will see.
Scores
| Metric | Score |
|---|
Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |