LinearPFN
Bayesian variable selection for linear models with interactions, in one forward pass.
A transformer pretrained on a spike-and-slab prior over main effects and pairwise interactions. Given a dataset, it returns posterior inclusion probabilities and posterior-mean coefficients, with no MCMC and no per-dataset fitting.
Code and documentation: github.com/schiekiera/LinearPFN
Use
git clone https://github.com/schiekiera/LinearPFN.git
pip install -e LinearPFN
from linearpfn import LinearPFN
model = LinearPFN.from_pretrained() # downloads this model's weights
result = model.fit(X, y)
print(result.summary())
Model
| Input | n rows, p predictors, outcome y (trained on p = 2 to 30, n = 20 to 2,000) |
| Output | a PIP and a posterior mean for each of the p + p(p-1)/2 effects, and a predictive distribution |
| Architecture | nanoTabPFN-style transformer, 8 layers, 31.5M parameters, fp32 |
| Training data | synthetic only, drawn from the prior in config.yaml |
Limitations
The outputs are the posterior under the training prior: linear main effects, pairwise interactions, strong heredity, Gaussian noise. Missing values and categorical predictors must be handled before fitting.
Files
linearpfn_strong.pt | weights and configuration (torch.load(..., weights_only=True)) |
config.yaml | prior, architecture and training settings |
SHA256SUMS | checksums of the two files above |
Citation
@article{schiekiera2026linearpfn,
title = {{LinearPFN}: Amortized Variable Selection for Linear Models with Interactions},
author = {Schiekiera, Louis and Zimmer, Max and Roux, Christophe and Arnold, Manuel
and Pokutta, Sebastian and G{\"u}nther, Fritz},
journal = {arXiv preprint},
year = {2026}
}
License: Apache 2.0.