Vertex-0.6-35M-Instruct
The instruction-tuned chat version of Vertex-0.6-35M-Base — a ≈34M-parameter Qwen3-architecture model trained from scratch on a single RTX 4060 Laptop GPU. Uses standard ChatML formatting, so it works out of the box in LM Studio, llama.cpp, Ollama, and MLX.
Model
| Architecture | Qwen3 (Qwen3ForCausalLM) |
| Parameters | 33,924,992 (≈34M), tied embeddings |
| Context length | 1024 |
| Vocab | 32002 (32000 BPE + `< |
| Chat format | ChatML |
| EOS | `< |
Chat format
Standard ChatML, embedded as a chat_template:
<|im_start|>user
Hello!<|im_end|>
<|im_start|>assistant
Hi there!<|im_end|>
Training
SFT on top of Vertex-0.6-35M-Base:
- Data: 269,072 conversations — smol-smoltalk (conversations ≤1024 tokens) interleaved with Vertex-0.6-35M-self-identification (derived from SupraLabs/LLM-self-identification)
- ≈127M conversation tokens, 2 epochs, assistant-only loss masking
- bf16 + torch.compile, fused AdamW, lr 1e-3 cosine, 5,605 steps (~1.9h)
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "VertexResearch/Vertex-0.6-35M-Instruct"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
enc = tok.apply_chat_template(
[{"role": "user", "content": "Who are you?"}],
add_generation_prompt=True, return_tensors="pt", return_dict=True,
)
out = model.generate(enc["input_ids"], max_new_tokens=100)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
# I am Vertex 0.6 35M. I was created by VertexResearch.
Limitations
These models are not the most coherent yet and need more tuning: expect rambling, repetition, and inconsistent answers, especially over longer generations.
- 34M parameters: expect simple conversational ability, not reasoning, factual reliability, or long-form coherence.
- English + Python centric; 1024-token context; no safety tuning.