DumbMath Preview 1 0722
Today, Small56.AI releases DumbMath Preview 1 0722. Despite its extremely small size of 665K parameters, this model outperforms a Atom-2.7M four times its size; although it is a preview version, it demonstrates arithmetic performance comparable to Qwen/Qwen2.5-0.5B.
- Parameters: 665K
- Training tokens: 0.52B
- Training time: Under 2 hours on Colab T4
What the model cannot do
- Conversation (no alphabet tokens exist)
- Complex mathematics
- Arithmetic outside its supported scope
arithmark
| eval | model | score |
|---|---|---|
| ArithMark 2.0 | mathbananamind-1.1 | 90.2% |
| our model | 62.5% | |
| qwen2.5-0.5B | 63.0% | |
| Parameters | mathbananamind-1.1 | 2900K |
| our model | 665K | |
| qwen2.5-0.5B | 494000k |