RiverCub-Gemma-3-27B
A slerp merge of what I believe to be the two best gemma 3 27b models after extensive testing of many. Unfortunately most finetunes of this model seem to make it come out worse than google's official instruct trained model, hence why I am using it in this slerp merge to keep some of it's magic. It really is quite good. Big tiger gemma v3 was surprisingly pretty good too, and seemed much less lobotomized compared to a lot of the other models I tested.
Quants
GGUFs
iMatrix
Static
- https://huggingface.co/mradermacher/RiverCub-Gemma-3-27B-GGUF
- https://huggingface.co/lemon07r/RiverCub-Gemma-3-27B-Q4_K_S
Special Thanks
Big thanks to everyone over at the KoboldAI discord. The members there have helped me a ton with various things over the long while I've been there, even letting me borrow GPU hours on runpod for some testing at some point. ɛmpti gets today's special thanks in particular for helping me figure out how to get rid of the extra head that was carried over from drummer's model.. which seems to have been caused by an issue with axolotl.
Merge Details
Merge Method
This model was merged using the SLERP merge method.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
modules:
text_decoder:
slices:
- sources:
- model: unsloth/gemma-3-27b-it
layer_range: [0, 62]
- model: TheDrummer/Big-Tiger-Gemma-27B-v3
layer_range: [0, 62]
vision_tower:
models:
- model: unsloth/gemma-3-27b-it
multi_modal_projector:
models:
- model: unsloth/gemma-3-27b-it
merge_method: slerp
base_model: unsloth/gemma-3-27b-it
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: bfloat16