Tabuada
Tiny TabPFN-style classifier for small tables, pre-trained from scratch on synthetic data only. Give it a labelled table and some unlabelled rows and it predicts their classes in one forward pass, with no fitting. tabuada.py in this repo holds the model and doubles as the loader.
🚀 Usage
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from tabuada import load, predict
model = load("jgalego/tabuada")
X, y = load_wine(return_X_y=True)
train_x, test_x, train_y, test_y = train_test_split(X, y, train_size=100, random_state=0)
print((predict(model, train_x, train_y, test_x) == test_y).mean())
Run it with tabuada.py on the path: download it from this repo or uv run it. Inputs are numeric. At most 32 features and 10 classes.
🏋️ Training
| Model | Transformer over table rows, 4.8M parameters, 6 layers, width 256, 8 heads |
| Data | A fresh batch of 64 synthetic tables of 256 rows at every step, drawn from random two-layer networks. No real data. |
| Steps | 20,000, AdamW, lr 0.0003 with warmup and cosine decay |
| Final train batch | loss 1.1876, accuracy 51.2% on held-out rows of synthetic tables |
| Hardware | NVIDIA L4, 0.26 h |
📊 Results
Mean test accuracy on small real datasets from scikit-learn, over random splits. 192 labelled rows at most, 10 splits.
| Dataset | Tabuada | Logistic regression | Random forest |
|---|---|---|---|
| iris | 94.7% | 95.3% | 94.7% |
| wine | 96.9% | 97.4% | 97.9% |
| breast_cancer | 96.9% | 97.3% | 95.4% |
| mean | 96.1% | 96.7% | 96.0% |
⚠️ Limitations
- Numeric features only. Encode categories and fill missing values first.
- At most 32 features and 10 classes; the labelled rows must fit in memory as one sequence.
- The synthetic prior has no categorical, missing or heavy-tailed features, so real tables can fall outside it.
- Not compared on a large benchmark. Three small datasets say little about tables in general.