PlatYi-34B-LoRA
Model Details
Model Developers Kyujin Han (kyujinpy)
Input Models input text only.
Output Models generate text only.
Model Architecture
PlatYi-34B-LoRA is an auto-regressive language model based on the Yi-34B transformer architecture.
Blog Link
Blog: [Coming soon...]
Github: [Coming soon...]
Base Model
01-ai/Yi-34B
Training Dataset
garage-bAInd/Open-Platypus.
Notice
While training, I used LoRA.
The lora_r values is 16.
Model Benchmark
Open leaderboard
- Follow up as link.
| Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
|---|---|---|---|---|---|---|---|
| PlatYi-34B-Q | 69.86 | 66.89 | 85.14 | 77.66 | 53.03 | 82.48 | 53.98 |
| PlatYi-34B-LoRA | 68.1 | 67.15 | 85.37 | 78.46 | 53.32 | 83.66 | 40.64 |
| 01-ai/Yi-34B | 69.42 | 64.59 | 85.69 | 76.35 | 56.23 | 83.03 | 50.64 |
Implementation Code
### KO-Platypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "kyujinpy/PlatYi-34B-LoRA"
OpenOrca = AutoModelForCausalLM.from_pretrained(
repo,
return_dict=True,
torch_dtype=torch.float16,
device_map='auto'
)
OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 68.10 |
| AI2 Reasoning Challenge (25-Shot) | 67.15 |
| HellaSwag (10-Shot) | 85.37 |
| MMLU (5-Shot) | 78.46 |
| TruthfulQA (0-shot) | 53.32 |
| Winogrande (5-shot) | 83.66 |
| GSM8k (5-shot) | 40.64 |