LightPFN
LightPFN is a small tabular foundation model for classification: a 4,603,088-parameter in-context learner
pretrained only on synthetic data. fit stores the training set as context and predict_proba answers in one
forward pass, with no training on your data and no hyperparameters to tune. It runs on CPU, CUDA, ROCm and, through
its own Vulkan kernels, on AMD, Intel and NVIDIA GPUs.
- Code, documentation and training pipeline: github.com/GioOtto/LightPFN
- Package: pypi.org/project/LightPFN
- Technical report: LightPFN_report.pdf
- License: Apache 2.0, code and weights
Usage
pip install LightPFN
from lightpfn import LightPFNClassifier
clf = LightPFNClassifier(n_estimators=4, random_state=0)
clf.fit(X_train, y_train) # downloads these weights at a pinned commit on the first fit
proba = clf.predict_proba(X_test)
X can be a NumPy array or a pandas DataFrame with numeric, categorical, string and boolean columns and missing
values. device="auto" uses a CUDA or ROCm GPU, else a Vulkan GPU (pip install "LightPFN[vulkan]"), else the CPU.
The user guide covers every option.
Model
| Task | classification, 2 to 10 classes |
| Parameters | 4,603,088 (18 MB, float32) |
| Architecture | cell embedding (value, rank, missing flag), two induced column stages, row refinement with four summary tokens, row compression, seven in-context blocks, retrieval decoder |
| Pretraining data | synthetic only: 90% structural causal graph prior, 10% rule prior (XOR, parity, lookup, trees); 7.68 million task draws from 4.03 million distinct tasks |
| Training | 120,000 steps of 64 tasks on two RTX 5090 (tables up to 2,048 rows), then 39,250 steps on tables up to 60,000 rows |
| Not used in training | real datasets, distillation, weights or outputs of other tabular foundation models |
Evaluation
The official TabArena leaderboard includes default, tuned and ensembled baselines. Other evaluations use default baselines; AUC differences have 95% paired bootstrap intervals.
| Benchmark | Result |
|---|---|
| Official TabArena leaderboard, 38 classification datasets, default configuration (four estimators), run and verified by the TabArena maintainers, 11 October 2026 | all splits: 21st of 92, Elo 1441 (+69 / -79); Lite: 20th of 92, Elo 1426 (+71 / -70); none imputed. Above every tree-based method, tuned and ensembled ones included (intervals overlap default CatBoost and the tuned entries); behind TabICLv2 (1575) and larger foundation models |
| 55 OpenML-CC18 datasets outside TabArena (at most 1,000 rows, one estimator) | mean AUC 0.911, +0.86 [0.42, 1.40] over CatBoost, +1.8 to +2.1 over LightGBM, XGBoost and random forest |
| TabArena, 38 classification tasks, official splits, first repeat (our harness), four estimators | mean AUC 0.858, rank 2.50 of 7, behind TabICLv2 (0.864), lower error than CatBoost on 76% of tasks |
| 13 OpenML datasets of 50,000 to 2.2M rows, 10,000 to 100,000 training rows | mean AUC difference from CatBoost between -0.34 and +0.28 points, intervals include zero |
The leaderboard view is Classification with imputed models excluded: the default view also covers the 13 regression datasets, which LightPFN does not support. Full tables, timings and the official TabArena results: docs/en/RESULTS.md.
Limitations
- Classification only (2 to 10 classes); regression is planned for version 2.
- Categorical columns are read as ordinal codes. On tables dominated by high-cardinality categorical columns CatBoost is ahead; native categorical handling is planned for version 2.
- Above 20,000 training rows each estimator reads a stratified subsample (
max_context). - CPU time grows with the context length; on large tables a GPU is much faster.
Files
| File | Content |
|---|---|
model.safetensors, config.json | weights (float32) and architecture of the released model |
provenance.json | hashes of the weights and of the source checkpoint |
LightPFN_report.pdf | technical report |
LICENSE, NOTICE | Apache License 2.0 and attribution notice |
Citation
@techreport{ottoboni2026lightpfn,
title = {A Sling Against Giants: {LightPFN}, a 4.6M-parameter tabular in-context classifier designed to stay small},
author = {Ottoboni, Giorgio},
year = {2026},
url = {https://github.com/GioOtto/LightPFN}
}