Llama-3-8B-Instruct-DADA
Warning: This model is experimental and thus potentially unpredictable.
This model employs the same strategy as Mixtral Instruct ITR DADA
I trained Llama-3-8B-Instruct on the Alpaca-DADA dataset for 10 epochs at 1e-6 learning rate. I then did a 50/50 SLERP merge of the resulting model back onto Llama-3-8B-Instruct
This model may require custom stopping strings to tame due to current issues surrounding Llama-3 EOS tokens and various back-ends. It certainly gives some interesting answers using an assistant template/card in SillyTavern, though.
The below answer is one of the more interesting answers I've gotten out of an LLM on the same query, although there was an indentiation error (indicated by the red circle)
Training was done using qlora-pipe