The "microsoft/Orca-2-13b" model fully fine-tuned on HuggingFaceH4/no_robots, totally-not-an-llm/EverythingLM-data-V3, mlabonne/guanaco-llama2-1k, and OpenAssistant/oasst_top1_2023-08-25. This model achieved a test loss of 0.18.
Make sure to comply with the microsoft research license. Please read it before using this model.
This model was trained on the ChatML prompt template.
The responses seen in the inference API were generated using the following sampling parameters:
temperature = 0.1
top_p = 0.14
top_k = 41
repetition_penalty = 1.176
Updates:
12/18/23 - 🔥 This model holds the #5 position on the Open LLM Leaderboard among llama2-13b models. 🔥