Hermes-2.5-Yi-1.5-9B-Chat

This model is a fine-tuned version of 01-ai/Yi-1.5-9B-Chat on the teknium/OpenHermes-2.5 dataset. I'm very happy with the results. The model now seems a lot smarter and "aware" in certain situations (first look, so I might change my opinion with more usage). It got quite an big edge on the AGIEval Benchmark for models in it's class. I plan to extend its context length to 32k with POSE.

Model Details

• Base Model: 01-ai/Yi-1.5-9B-Chat
• chat-template: chatml
• Dataset: teknium/OpenHermes-2.5
• Sequence Length: 8192 tokens
• Training:
• Epochs: 1
• Hardware: 4 Nodes x 4 NVIDIA A100 40GB GPUs
• Duration: 48:32:13
• Cluster: KIT SCC Cluster

Benchmark n_shots=0

For detailed benchmark results, including sub-categories and various metrics, please refer to the full benchmark table at the end of this README.

GGUF and Quantizations

• llama.cpp b3166
• juvi21/Hermes-2.5-Yi-1.5-9B-Chat-GGUF is availabe in:
• F16 Q8_0 Q6_KQ5_K_M Q4_K_M Q3_K_M Q2_K

Usage

To use this model, you can load it using the Hugging Face Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("juvi21/Hermes-2.5-Yi-1.5-9B-Chat")
tokenizer = AutoTokenizer.from_pretrained("juvi21/Hermes-2.5-Yi-1.5-9B-Chat")

# Generate text
input_text = "What is the question to 42?"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs)
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(generated_text)

chatml

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
Knock Knock, who is there?<|im_end|>
<|im_start|>assistant
Hi there! <|im_end|>

License

This model is released under the Apache 2.0 license.

Acknowledgements

Special thanks to:

• Teknium for the great OpenHermes-2.5 dataset
• 01-ai for their great model
• KIT SCC for FLOPS

Citation

If you use this model in your research, consider citing. Although definetly cite NousResearch and 01-ai:

@misc{
  author = {juvi21},
  title = Hermes-2.5-Yi-1.5-9B-Chat},
  year = {2024},
}

full-benchmark-results