Outdated! Move to Lyra-v4
Outdated
Mistral-NeMo-12B-Lyra-v4, layered over Lyra-v3, which was built on top of Lyra-v2a2, which itself was built upon Lyra-v2a1.
Model Versioning
Lyra-v1 [Merge of Custom Roleplay & Instruct Trains, on Different Formats]
|
| [Additional SFT on 10% of Previous Data, Mixed]
v
Lyra-v2a1
|
| [Low Rank SFT Step + Tokenizer Diddling]
v
Lyra-v2a2
|
| [RL Step Performed on Multiturn Sets, Magpie-style Responses by Lyra-v2a2 for Rejected Data]
v
Lyra-v3
|
| [Backmerge to v2a1 + LoRA Extraction + Low Rank SFT Step for Coherency]
v
Lyra-v4
This uses ChatML, or any of its variants which were included in previous versions.
<|im_start|>system
This is the system prompt.<|im_end|>
<|im_start|>user
Instructions placed here.<|im_end|>
<|im_start|>assistant
The model's response will be here.<|im_end|>
--------------------------------------------------
[INST]system
This is another system prompt.[/INST]
[INST]user
Your instructions placed here.[/INST]
[INST]assistant
The model's response will be here.[/INST]
Recommended Samplers:
Temperature: 0.6 - 1 # I recommend lowering it a litte for v4?
min_p: 0.1 - 0.2 # Crucial for NeMo
Recommended Stopping Strings:
<|im_end|>
</s>
[/INST]
Introduces run-off generations at times, as seen in v2a2. It's layered on top of older models, so eh, makes sense. Easy to cut out though.
Notes
- Some people have been having issues with run-on generations for Lyra-v3. Kind of weird, when I never had issues. - Anyway, make sure not to skip special tokens, or ban EOS tokens. I think this is the main problem that happens when v3 was to be quanted. The special tokens map config is fucked in v3, Quantizing tools likely spazzed out seeing it. I blame llamafactory for it. It ran fine unquantised. - I like long generations, though I can control it easily to create short ones. If you're struggling, prompt better. Fix your system prompts, use an Author's Note, use a prefill. They are there for a reason. - Lyra passes my internal benchmark suite, hence why I'm releasing it. Do I like it? Yes? it's out. that's it. They are my models for my personal enjoyment first. - Issues like roleplay format are what I consider worthless, as it follows few-shot examples fine. This is not a priority for me to 'fix', as I see no isses with it. Same with excessive generations. Its easy to cut out. - If you don't like it, just try another model? Plenty of other choices. Ymmv, I like it.