Microllm
-- Use the non GGUF version for now. --
A small (768-dim, 22-layer, ~50260 vocab) decoder-only transformer, pretrained from scratch on streaming FineWeb-Edu and instruction fine-tuned on Dolly-15k + No Robots. This is an independent hobbyist project, not affiliated with any AI lab - trained end-to-end on a single rented GPU.
Architecturally this is a standard Llama-style model (RMSNorm, RoPE, SwiGLU,
tied embeddings), so it loads directly with transformers:
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("MLVXN/microllm")
model = AutoModelForCausalLM.from_pretrained("MLVXN/microllm")
prompt = "<|user|>What's your name?<|assistant|>"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=100, do_sample=True, temperature=0.8, top_k=50)
print(tok.decode(out[0]))
Chat format
This model was fine-tuned on a simple turn structure, not a full chat template - wrap each user message like this:
<|user|>{message}<|assistant|>
Generation should stop at <|end|> (id 50259).
Checkpoint info
- Fine-tuning phase reached:
no_robots - Fine-tuning global step:
4845 - seq_len: 1024
Known limitations
This is a ~150M-parameter model trained on a modest compute budget. Expect coherent grammar and conversational fluency, but unreliable facts and no real multi-step reasoning - that's the honest ceiling for this size/budget, not a bug. It reliably knows its own identity (name/creator) because that was explicitly trained in, separately from general knowledge quality.
Running in LM Studio / llama.cpp / Ollama
If a .gguf file is included in this repo, download it directly in LM Studio
via its Hugging Face search, or point llama.cpp / Ollama at the file. If no
.gguf is present, convert it yourself:
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
pip install -r requirements.txt
python convert_hf_to_gguf.py /path/to/microllm --outfile microllm.gguf --outtype f16
# optional: quantize for a smaller file
./llama-quantize microllm.gguf microllm.Q4_K_M.gguf Q4_K_M