GGUF STATIC/IMATRIX made available by mradermacher
Chat Template:
<start_of_turn>user
{system}
{prompt}<end_of_turn>
<start_of_turn>model
{response}<end_of_turn>
ST Settings:
- CONTEXT
- INSTRUCT (The model was trained with diverse system prompts, make sure to write the system prompt based on your preference.)
ls=LoraConfig(
r = 128,
target_modules = ['q_proj', 'down_proj', 'up_proj', 'o_proj', 'v_proj', 'gate_proj', 'k_proj'],
lora_alpha = 32,
lora_dropout = 0.1,
bias = "none",
use_rslora = True,
)