🖤 Sable-Mini-30M (Preview)
Sable-Mini is a 30M parameter causal language model trained on approximately 3–4B tokens. It is the flagship of the Sable tiny model family, balancing compact size with capable text generation.
⚠️ This is a preview release — expect improvements in future versions.
📊 Benchmark Results
Evaluated with the LM Evaluation Harness:
| Benchmark | Accuracy | Metric |
|---|---|---|
| BoolQ | 57.5% | acc |
| PIQA | 57.8% | acc_norm |
| WinoGrande | 53.3% | acc |
| ARC-Easy | 39.0% | acc_norm |
| HellaSwag | 28.1% | acc_norm |
| Lambada | 11.7% | acc |
🚀 Quick Start
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("yourorg/sable-mini-30m-preview")
model = AutoModelForCausalLM.from_pretrained("yourorg/sable-mini-30m-preview")
inputs = tokenizer("The future of tiny models is", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
🏗️ Model Details
| Property | Value |
|---|---|
| Parameters | 30M |
| Architecture | Llama-style decoder |
| Training tokens | ~3–4B |
| Vocab size | 24k |
⚠️ Limitations
- Preview release — not recommended for production use
- Limited knowledge due to small parameter count
- May produce biased or incorrect outputs
📜 License
Apache 2.0 — free for commercial and research use.
Part of the Sable family 🖤 — tiny models, sharp minds.