lilchat
lilbase (297m params, trained from scratch) finetuned into a chat model on smol-smoltalk.
it follows the chat format, answers in full sentences, stops when its done, and does lists and code blocks. it does not know things. expect confident wrong answers on facts and maths, and weak memory across turns.
run it
ollama run navthings/lilchat
or try it in your browser: https://navthings.github.io/playground/
files
| file | size | notes |
|---|---|---|
model.safetensors + config + tokenizer | 1.19gb | transformers LlamaForCausalLM, chat template included |
lilchat-q8_0.gguf | 379mb | |
lilchat-q4_k_m.gguf | 274mb | smallest |
prompt format
llama-2 style, with </s> (id 2) as the start token and after every assistant reply:
</s>[INST] hi [/INST] Hello! How can I help you today?</s>[INST] next message [/INST]
system prompts go in <<SYS>>\n...\n<</SYS>>\n\n before the first [INST].
finetune
full sft in mlx on a macbook air, 1,136 steps of 32k tokens (~37m tokens, about 12% of smol-smoltalk), loss only on assistant turns. held-out loss 1.337 vs 2.230 for lilbase on the same data.
temperature 0.4 works best. greedy decoding loops, keep some temperature.