SahabatAI-Lion-9B-TIES-v1
formerly gemma2-9b-cpt-sahabatai-v1-instruct-BaseTIES (model name too long :D )
Based on some research, when a finetuned model is merged with its base model with TIES method, there is possibility the merged model will achieve better output.
UPDATE!!! as 20 November 2024, this model is third best model (number one for Gemma2-9B based model) on HF's Open LLM Leaderboard (with Merge/MoErges hide model unchecked) for LLM model below 10B parameters.
gmonsoon/SahabatAI-Lion-9B-TIES-v1 is a merge of the following models:
DEMO Spaces: HERE
🧩 Configuration
models:
- model: GoToCompany/gemma2-9b-cpt-sahabatai-v1-instruct
parameters:
weight: 1
density: 1
- model: GoToCompany/gemma2-9b-cpt-sahabatai-v1-instruct
parameters:
weight: 1
density: 1
merge_method: ties
base_model: aisingapore/gemma2-9b-cpt-sea-lionv3-instruct
parameters:
density: 1
normalize: true
int8_mask: true
dtype: bfloat16
💻 Usage
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "gmonsoon/SahabatAI-Lion-9B-TIES-v1"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 33.70 |
| IFEval (0-Shot) | 73.78 |
| BBH (3-Shot) | 43.40 |
| MATH Lvl 5 (4-Shot) | 19.34 |
| GPQA (0-shot) | 9.40 |
| MuSR (0-shot) | 19.13 |
| MMLU-PRO (5-shot) | 37.19 |