BananaMind 2 Pico — Chinchilla Test
Freshly initialized 900,002-parameter Pico model trained to
the Chinchilla budget of 20 tokens per parameter. This is a controlled short-run
comparison, not a continuation of Banaxi-Tech/pico-test.
| Field | Value |
|---|---|
| Parameters | 900,002 |
| Training tokens | 18,000,040 |
| Tokens per parameter | 20.000000 |
| Context / vocabulary | 4,096 / 384 |
| Architecture | 6L, 96D, XSA refresh on layers 4 and 6 |
| Data | 70% FineWeb-HQ, 30% Cosmopedia v2 |
| Optimizers | Muon 0.07 + AdamW 0.004 |
| Precision / hardware | bfloat16 autocast / NVIDIA RTX PRO 6000 Blackwell Server Edition |
| Template revision | 5fa6920cf5ae031f03bd044a4de12564ec453837 |
Load with trust_remote_code=True.