Quartz Micro Preview V2
A tiny language model trained from scratch by Vertex AGI on a single GTX 1660 Ti: a 100.09M-parameter dense Llama-architecture decoder (12 layers, hidden 768, 12 heads / 4 KV heads, SwiGLU 2048, RoPE, tied 32K byte-level BPE, 1,024 context). This is Quartz Micro Preview V2, the instruction-tuned chat model. The raw pretrained model is Quartz Micro Preview V2 Base (VertexAGI/quartz-micro-preview-v2-base).
Honest summary: at ~100M parameters this model writes short, fluent, on-topic text but often states wrong facts with confidence, is weak at maths, reasoning and code, and can repeat itself. Treat it as a small research / hobby model, not a source of truth.
Formats (one repo, three formats)
| Where | Format |
|---|---|
| repo root | stock Hugging Face LlamaForCausalLM (fp32 safetensors), loads with plain transformers and plain mlx_lm |
mlx/fp16, mlx/q8, mlx/q4 | stock MLX (fp16, 8-bit, 4-bit) |
gguf/ | stock llama.cpp GGUF: f16, q8_0, q4_k_m |
Every format was checked against the original training code on held-out text (perplexity; lower is better):
| Format | Perplexity | Difference from original |
|---|---|---|
| transformers fp32 (this repo root) | 27.844 | 0.000% |
| MLX fp16 | 27.842 | 0.007% |
| MLX 8-bit | 27.857 | 0.045% |
| MLX 4-bit | 28.457 | 2.202% |
| GGUF f16 | 27.844 | 0.000% |
| GGUF Q8_0 | 27.850 | 0.020% |
| GGUF Q4_K_M | 27.888 | 0.157% |
Usage
# stock transformers (no custom code, no trust_remote_code)
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("VertexAGI/quartz-micro-preview-v2")
model = AutoModelForCausalLM.from_pretrained("VertexAGI/quartz-micro-preview-v2")
messages = [{"role": "user", "content": "Why do cats purr?"}]
prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
ids = tok(prompt, add_special_tokens=False, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=400, do_sample=True, temperature=0.6, top_p=0.9, repetition_penalty=1.1)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# stock MLX (Apple silicon)
from mlx_lm import load, generate
model, tok = load("VertexAGI/quartz-micro-preview-v2")
print(generate(model, tok, prompt=tok.apply_chat_template([{"role": "user", "content": "Why do cats purr?"}], add_generation_prompt=True), max_tokens=400))
# stock llama.cpp (GGUF in the gguf/ folder)
hf download VertexAGI/quartz-micro-preview-v2 gguf/quartz-micro-preview-v2-q8_0.gguf --local-dir .
llama-cli -m gguf/quartz-micro-preview-v2-q8_0.gguf -cnv
Instruction tuning
- Base: Quartz Micro Preview V2 Base (VertexAGI/quartz-micro-preview-v2-base).
- Data: 24,170 English conversations (9,258,981 tokens): filtered OpenAssistant OASST2 (reviewed, non-synthetic, top-ranked replies, follow-up turns kept) plus short conversational / rewriting / summarizing / constraint-following slices of smol-smoltalk. Long reasoning, maths, code and URLs were removed.
- Method: full-parameter fine-tune, loss on assistant tokens only, 2 epochs, lr 1e-4 cosine, fp32.
- Chat format:
<|user|>\n{message}\n<|assistant|>\n{reply}<eos>(built into the tokenizer's chat template; no system role - put instructions in the user message). - Suggested sampling: temperature 0.6, top-p 0.9, repetition penalty 1.1.
Evaluation (hand-written prompts, not copied from the training sets)
- 20/20 prompts answered, 17/20 replies ended cleanly with
<eos>, mean distinct-4-gram ratio 0.79. - Simple keyword fact checks: 5/15 (a deliberately easy test; misses are normal for this size).
- Assistant-token loss on held-out chat data: base 2.856 -> tuned 2.280.
- All prompts and replies are in
eval_outputs.md, including the wrong ones. Machine-readable results:eval_results.json(per-prompt replies, gate checks) andeval_formats.json(all shipped formats).
Same prompts, every shipped format (greedy, repetition penalty 1.1, 400 new tokens)
| Format | Ended with | Keyword fact checks | Distinct-4-gram |
|---|---|---|---|
| MLX fp16 | 13/21 | 6/15 | 0.57 |
| MLX 8-bit | 12/21 | 6/15 | 0.57 |
| MLX 4-bit | 9/21 | 5/15 | 0.54 |
| GGUF f16 | 15/21 | 6/15 | 0.74 |
| GGUF Q8_0 | 16/21 | 5/15 | 0.77 |
| GGUF Q4_K_M | 14/21 | 5/15 | 0.75 |
The reference run above (stock transformers fp32) scored 17/20 and 5/15. Quantizing to 8-bit changes little; MLX 4-bit loses the most (fewer replies end cleanly). The 21 prompts are the 20 evaluation prompts plus one follow-up question. The MLX columns detect a clean ending by length (an approximation), the GGUF columns by the real end-of-text marker, so compare formats within a family rather than across the two. Raw numbers: eval_results.json, eval_formats.json.
Limitations
Frequent factual mistakes and invented details; weak maths and logic; can loop on longer answers; 1,024-token context; English only; no safety tuning or refusal training, so do not deploy it unsupervised.
License
Apache-2.0 for the weights and code. Training data carries its own terms (see the dataset cards; Wikipedia and Stack Exchange are share-alike).