Tabnetics Diakrino

An in-context transformer that ranks features in high-dimensional, low-sample-size (HDLSS) tables.

Diakrino — from the Greek διακρίνω, "to discern" — is the neural feature-selection oracle of the Tabnetics toolkit. Give it a small labelled support set and it reads the entire table at once, returning per-feature relevance evidence, query-row predictions, redundancy structure, conformal selection signals, and reconstructed per-feature population statistics in a single forward pass. There is no per-dataset gradient training: the model is pre-trained once and then applied in context.

This is a research checkpoint. It is not a promoted production component of Tabnetics, and it has documented failure modes on real HDLSS data (see Limitations). We publish it for reproducibility and for research on in-context models for wide tables, on the understanding that it is used as one opt-in source of evidence in a portfolio rather than as a stand-alone selector. Please read the whole card before using it.

A peer-reviewed article describing this work is in preparation.

EuroHPC acknowledgement

Part of Tabnetics Diakrino was pre-trained on the EuroHPC supercomputer MareNostrum 5 within the scope of a EuroHPC Joint Undertaking project. The following acknowledgement is mandatory and must be reproduced in any publication or dissemination that uses these weights:

This work was granted access to the EuroHPC supercomputer MareNostrum 5, hosted by the Barcelona Supercomputing Center (BSC-CNS, Spain), through a EuroHPC AI Factory Fast Lane Access call (project EHPC-AIF-2026FL01-314, "Neural HDLSS Oracle for High-Dimensional Tabular AI"). Part of the Tabnetics Diakrino model was pre-trained on MareNostrum 5 within the scope of this EuroHPC project.

We thank the EuroHPC Joint Undertaking and the Barcelona Supercomputing Center for the allocation and for their support.

Model summary

What it does

Diakrino targets problems where features vastly outnumber samples (p >> n): transcriptomics, proteomics, metabolomics, spectroscopy, and similar wide tables. Rather than being fitted per dataset, it is pre-trained once on a large corpus of synthetic HDLSS "worlds" together with feature-selection-ready slices of real data. At inference you hand it a support set of labelled rows plus the query rows, and it returns feature-relevance logits and predictions in one pass.

Inside Tabnetics it is wired up as an opt-in sidecar that contributes relevance evidence to the portfolio (MNPO) selection process. It is designed to complement the classical feature-selection stack, not to replace it.

Architecture

Diakrino is a swiglu_fusion_v2 in-context transformer with several task heads sharing one trunk:

• Context trunk — 10 transformer layers, d_model = 1024, 16 attention heads, SwiGLU-fusion FFN (expansion 2), dropout 0.1, QK-norm attention, up to 25 classes.
• Joint full-row channel — induced-point set attention over the combined support and query rows (joint_sample_mode = induced, 128 induced points, 3 layers, 8 heads), scale-init warm-started.
• Feature-selection head — listwise and pairwise-rank relevance objectives over per-feature screening statistics and distribution summaries.
• Query in-context-learning head — a causal in-context classifier over query rows (8 heads, 2 layers, up to 512 rows).
• Redundancy head — a bilinear pairwise redundancy estimator (rank 128).
• Conformal selection head — a risk-controlled soft-selection signal.
• Population reconstruction heads — per-feature marginal-distribution family, parameters, and dependency structure. These are auxiliary training targets and are never inference inputs.
• Refiner and V-REx emitter — deep-supervision / equilibrium refinement plus a variance-risk extrapolation term that penalises inconsistency across synthetic environments.

The published weights are a LAWA average (latest-weight averaging over the final 10 saved checkpoints), which is noticeably more stable than any single step.

Training data and procedure

The training corpus combines the Tabnetics feature-selection-ready slices of real datasets with large-scale synthetic HDLSS worlds in which the relevant features and the per-feature population distributions are known by construction. Training consumed hundreds of thousands of in-context episodes.

The objective is multi-task: feature-relevance ranking, query classification and in-context learning, redundancy, conformal selection, population-statistics reconstruction, and the V-REx invariance penalty across synthetic environments.

Pre-training ran data-parallel (DDP) on 4× NVIDIA H100 GPUs in bf16 mixed precision on the EuroHPC MareNostrum 5 ACC partition at BSC (see the acknowledgement above). The released checkpoint is the LAWA average over the last 10 checkpoints of the main 40,000-step run.

Evaluation

The numbers below come from a held-out synthetic HDLSS validation set of 2,304 records, measured on the published LAWA checkpoint. They are synthetic-validation figures rather than real-data benchmark scores, and they should be read alongside the Limitations.

The relevance heads clearly carry signal on synthetic worlds (ROC-AUC ≈ 0.72). Linked precision is much higher than direct precision because the linkage-aware metric also credits correlated proxies of truly relevant features. The query-classification heads, by contrast, sit at or below chance on this validation set, so any trust signal derived from them should be treated as non-functional for now.

Intended use and scope

Use it for:

• Contributing feature-relevance evidence as an opt-in sidecar inside the Tabnetics portfolio, alongside the classical selectors.
• Research on in-context and prior-data-fitted models for HDLSS tabular data.
• Reproducing the Tabnetics 1.5.0 Diakrino experiments.

Do not use it for:

• Stand-alone or sole feature selection. On real HDLSS data it under-performs the classical Tabnetics stack when used in isolation.
• Production defaults, or as the basis for a state-of-the-art claim.
• Clinical, diagnostic, or any other high-stakes decision-making.

Limitations and known failure modes

We publish this checkpoint with its limitations stated plainly. It is not a validated selector.

1. Not production-promoted. Diakrino is an opt-in research component. It has not cleared the checkpoint-qualification bar that Tabnetics requires before selector weights are promoted. The portfolio profile that includes Diakrino passed only a weak protected-augmentation gate, and that gain was not traced back to the ranker itself.
2. Relevance-blindness on real HDLSS data. In the paired closeout campaign, real Diakrino ranks did not beat a control in which the same ranks were randomly permuted (−0.0032 balanced accuracy, Holm-adjusted p = 0.9383), and adding Diakrino to the classical feature-selection union was not additive on real HDLSS panels (≈ −0.026, p ≈ 0.16). The model tends to pick features near the median univariate Fisher score (top-50 Fisher recall ≈ 0) and to over-decorrelate.
3. Query heads at chance. Mean query-classification accuracy stays at or below chance, so query-derived trust signals are not usable yet.
4. Cross-chunk calibration is required. Raw per-feature logits are not on a shared cross-chunk scale, and a naive global top-k can collapse onto a single chunk. Apply a within-chunk (z-score) calibration step before ranking globally. Calibration and representation are not fully disentangled here — at least one failure mode is representational, not merely a calibration artefact.
5. Headline metrics are synthetic-only. The evaluation table above comes from a synthetic HDLSS validation harness, not from real benchmark datasets.
6. No trust sidecars for this checkpoint. The Tabnetics native loader can verify a checkpoint against artifact_manifest.json / classifier_trust_record.json trust sidecars. Those qualification artifacts are not produced for this research checkpoint, so loading it requires allow_untrusted_checkpoint=True (see below).

How to use

Install Tabnetics (>= 1.5.0) into a PyTorch environment:

pip install "tabnetics>=1.5.0" torch huggingface_hub

Request access to the gated repo, authenticate with huggingface-cli login, then download the weights:

from huggingface_hub import snapshot_download

local_dir = snapshot_download(repo_id="klokedm/tabnetics-diakrino", repo_type="model")
checkpoint = f"{local_dir}/checkpoint_lawa10_public.pt"

This is a feature-ranking (FS-teacher) checkpoint. Load and inspect it:

from tabnetics.classification import load_tabentics_diakrino_fs_classifier

model, report = load_tabentics_diakrino_fs_classifier(checkpoint, map_location="cuda")
print(report["checkpoint_format"])                                     # 'fs_teacher'
print(report["loaded_count"], report["loaded_classifier_head_count"])  # 646, 0

Its value is feature relevance, not query classification. Within Tabnetics it is consumed as an opt-in feature-selection sidecar: the FS-teacher writes per-feature relevance logits to a persisted sidecar, and the opt-in diakrino_prior, diakrino_screening_prior, and diakrino_conformal_selection selectors (all default_enabled=False) read that sidecar during portfolio selection. To rank features directly from such a sidecar, with cross-chunk calibration:

from tabnetics.classification import select_tabentics_diakrino_features

# sidecar_path -> a parquet of the model's persisted per-feature relevance logits
selected, info = select_tabentics_diakrino_features(
    X_train, max_features=64, sidecar_path="diakrino_sidecar.parquet",
    calibrate="chunk_zscore",
)

Classification is not available on this checkpoint. It is an FS-teacher with no trained native classification head (checkpoint_comparable_for_classification = False, loaded_classifier_head_count = 0), and its query heads are at chance. The native classifier path (TabenticsDiakrinoNativeClassifier / predict_tabentics_diakrino_proba) therefore fails closed for this checkpoint; it will only become meaningful for a future, qualified FS-classifier bundle.

The Tabnetics documentation and the Diakrino results page cover the sidecar emission workflow, the portfolio integration path, and the full evidence discussion.

Two things are worth repeating: always calibrate within chunks before taking a global top-k, and combine Diakrino evidence with the classical Tabnetics selectors instead of using it on its own.

Provenance

License

Released under the Apache License 2.0, matching the core tabnetics package. Third-party components and any datasets referenced during training keep their own upstream licenses and terms.

Citation

If you use Tabnetics Diakrino, please cite the Tabnetics repository and include the EuroHPC acknowledgement above. A peer-reviewed article is in preparation and this entry will be updated once it appears.

@software{kokol_tabnetics_2026,
  author = {Kokol, Marko},
  title  = {Tabnetics},
  year   = {2026},
  url    = {https://github.com/klokedm/tabnetics-public}
}

Contact

Marko Kokol — marko@tabnetics.org · tabnetics.org · Discussions