Introduction
We introduce Llama-3-Motif, a new language model family of Moreh, specialized in Korean and English.
Llama-3-Motif-102B-Instruct is a chat model tuned from this model.
Training Platform
- Llama-3-Motif-102B is trained on MoAI platform, refer to link for more information.
Quick Usage
base model is not served directly. Instead, you can chat directly with Llama-3-Motif-102B-Instruct through our Model hub.
Details
More details will be provided in the upcoming technical report.
Release Date
2024.12.02
Benchmark Results
| Provider | Model | kmmlu_direct score | |
|---|---|---|---|
| Moreh | Llama-3-Motif-102B | 64.74 | + |
| Meta | Llama3-70B-instruct | 54.5* | |
| Meta | Llama3.1-70B-instruct | 52.1* | |
| Meta | Llama3.1-405B-instruct | 65.8* | |
| Alibaba | Qwen2-72B-instruct | 64.1* | |
| OpenAI | GPT-4-0125-preview | 59.95* | |
| OpenAI | GPT-4o-2024-05-13 | 64.11** | |
| gemini pro | 50.18* | ||
| LG | exaone 3.0 | 44.5* | + |
| Naver | HyperCLOVA X | 53.4* | + |
| Upstage | SOLAR-10.7B | 41.65* | + |
* : Community report
** : Measured by Moreh
+ : Claimed to have better capability in Korean
How to use
We do not recommend using base model directly!
Use with vLLM
- Refer to this link to install vllm
from transformers import AutoTokenizer
from vllm import LLM, SamplingParams
# Change tensor_parallel_size to GPU numbers you can afford
model = LLM("moreh/Llama-3-Motif-102B", tensor_parallel_size=4)
tokenizer = AutoTokenizer.from_pretrained("moreh/Llama-3-Motif-102B")
messages = [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "유치원생에게 빅뱅 이론의 개념을 설명해보세요"},
]
messages_batch = [tokenizer.apply_chat_template(conversation=messages, add_generation_prompt=True, tokenize=False)]
# vllm does not support generation_config of hf. So we have to set it like below
sampling_params = SamplingParams(max_tokens=512, temperature=0, repetition_penalty=1.0, stop_token_ids=[tokenizer.eos_token_id])
responses = model.generate(messages_batch, sampling_params=sampling_params)
print(responses[0].outputs[0].text)
Use with transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "moreh/Llama-3-Motif-102B"
# all generation configs are set in generation_configs.json
model = AutoModelForCausalLM.from_pretrained(model_id).cuda()
tokenizer = AutoTokenizer.from_pretrained(model_id)
messages = [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "유치원생에게 빅뱅 이론의 개념을 설명해보세요"},
]
messages_batch = tokenizer.apply_chat_template(conversation=messages, add_generation_prompt=True, tokenize=False)
input_ids = tokenizer(messages_batch, padding=True, return_tensors='pt')['input_ids'].cuda()
outputs = model.generate(input_ids)