OrdinalICL for TabDPT-Turbo

OrdinalICL predicts ordered categories such as ratings, stages, or survey responses from small tabular datasets. It adds an ordinal target interface to the unchanged TabDPT-Turbo 1.2 backbone and returns a probability for every level in a user-declared scale.

The release contains the final three-seed ensemble evaluated in OrdinalICL: Making Tabular Foundation Models Ordinal-Native by Gautier Marti.

What is different from ordinary classification?

An ordinary classifier treats low, medium, and high as unrelated names. OrdinalICL knows their declared order. Its correction module uses rank coordinates and ordered boundary decisions, produces a coherent class distribution, and averages predictions in both orientations of the scale for exact order-reversal symmetry.

The 63.5M-parameter TabDPT backbone is unchanged. The release adds one shared 424,129-parameter nominal path and three independently trained ordinal correction modules of 424,129 parameters each.

Install

Install the packaged inference client directly from this model repository:

python -m pip install "https://huggingface.co/gmarti/ordinalicl-tabdpt-turbo/resolve/main/ordinalicl_tabdpt-0.1.0-py3-none-any.whl"

Alternatively, clone the repository and install from source:

git clone https://huggingface.co/gmarti/ordinalicl-tabdpt-turbo
cd ordinalicl-tabdpt-turbo
python -m pip install .

Python 3.10 or newer is required. The dependency on tabdpt==1.2.0 is intentional: it is the exact released backbone used for the primary model.

The paper also reports a cross-release replication on TabDPT-Turbo v1.3. This repository packages the primary v1.2 ensemble because it has the slightly lower real-benchmark mean and the two released ordinal models are statistically tied. The v1.3 replication uses a different upstream API and is retained in the paper's reproducibility material rather than silently substituted here.

Quick start

The order supplied in classes is part of the task definition.

import numpy as np
from ordinalicl_tabdpt import OrdinalICLClassifier

X_context = np.array([
    [-1.2, 0.1], [-0.8, 0.0], [-0.2, 0.2],
    [0.2, 0.4], [0.7, 0.8], [1.2, 1.0],
], dtype=np.float32)
y_context = np.array(["low", "low", "medium", "medium", "high", "high"])
X_query = np.array([[-0.5, 0.1], [0.5, 0.6]], dtype=np.float32)

model = OrdinalICLClassifier.from_pretrained(
    "gmarti/ordinalicl-tabdpt-turbo",
    classes=["low", "medium", "high"],
    device="cuda",  # omit this argument to select CUDA when available
)
model.fit(X_context, y_context)

probabilities = model.predict_proba(X_query)
predicted_labels = model.predict(X_query)
expected_ranks = model.predict_expected_rank(X_query)

The first call downloads this small model repository and the exact upstream TabDPT-Turbo checkpoint. The checkpoint is pinned by revision and SHA-256. Later calls use the Hugging Face cache.

fit stores the in-context examples; it does not optimize parameters. Probability columns follow the order given in classes. A level may be absent from the context and must nevertheless remain in classes if it belongs to the declared scale. See examples/missing_level.py for this case.

Input and output contract

• X must contain finite numeric values and 1 to 12 columns.
• classes must declare 3 to 10 unique levels from lowest to highest.
• Every context label must occur in classes.
• predict_proba returns one row per query and one column per declared level.
• The rows are finite, nonnegative, and sum to one.
• CPU inference works, but CUDA is substantially faster.

The reported experiments use 32 labeled context rows. Behavior with different context sizes is not yet characterized.

Results

Lower normalized ranked probability score (nRPS) is better. Within each matched experiment, the ordinal and nominal modules have the same backbone, parameter count, training tables, initialization, optimizer, schedule, and compute.

On the 40 real families, mean nRPS is 0.146894 for OrdinalICL, 0.147059 for the matched nominal adaptation, and 0.148968 for released TabDPT-Turbo. The family-level interval for OrdinalICL minus released TabDPT is [-0.003376, -0.000974].

The synthetic tests isolate the effect of target geometry under controlled training. The real benchmark measures transfer across 40 external dataset families. The paper and supplementary material contain the full protocols, per-family results, model survey, and statistical analysis.

Training summary

The shared nominal path and each released ordinal correction module saw 128,000 synthetic ordered tables. Training varied the number of declared levels from 3 to 10, the number of features from 1 to 12, ordered mechanisms, imbalance, noise, missing context ranks, and distribution shift. Each table had 32 context rows. The TabDPT-Turbo backbone remained fixed. No Yelp, ESS, or other real benchmark row was used to train the released parameters.

Files

• weights/common_nominal.safetensors: shared nominal context path.
• weights/ordinal_residual_seed_*.safetensors: the three ordinal correction modules used by the released ensemble. The historical filenames preserve the experiment artifact names.
• config.json: architecture, inference defaults, provenance, and the exact upstream revision.
• MANIFEST.json: hashes and tensor inventories.
• src/ordinalicl_tabdpt: inference implementation.
• VERIFICATION.md: conversion, equivalence, and GPU smoke-test record.

Only tensor-only safetensors files are distributed. The 254 MB TabDPT base checkpoint remains in its official repository and is downloaded separately.

Reproduce and verify

From a clone of this repository:

python -m pip install ".[test]"
pytest
python scripts/smoke_test.py --device cuda

smoke_test.py downloads and verifies the pinned base checkpoint unless --base-checkpoint points to an existing local copy.

Intended use

This model is intended for research on numeric tabular tasks with a known, ordered categorical target. Suitable examples include ratings, disease stages, educational attainment, and ordered survey responses.

Limitations and responsible use

• The controlled improvement is consistent and statistically resolved, but small in absolute nRPS.
• Training and evaluation cover 3 to 10 levels, at most 12 numeric features, and a 32-row context. Other settings are untested.
• Categorical encoding and missing-value handling must be completed before inference.
• Evaluation focuses on one foundation-model backbone and nRPS as the main endpoint.
• TabDPT reports pretraining overlap with three sources represented in the real benchmark. This can affect absolute scores against released TabDPT. It does not favor either side of the primary matched comparison because both sides use the same pretrained backbone.
• The model is not validated for clinical, employment, credit, or other consequential decisions about individuals. Such uses require domain-specific validation and governance.

This repository contains no raw dataset rows, direct identifiers, row-level predictions, or restricted-use data.

Licenses

• First-party model weights: CC BY 4.0 (WEIGHTS_LICENSE.md).
• Original inference code: BSD-3-Clause (LICENSE).
• Required TabDPT 1.2 code and checkpoint: Apache-2.0, obtained from the upstream repository and not redistributed here (THIRD_PARTY_NOTICES.md).

Citation

@article{marti2026ordinalicl,
  title   = {OrdinalICL: Making Tabular Foundation Models Ordinal-Native},
  author  = {Marti, Gautier},
  year    = {2026},
  url     = {https://huggingface.co/gmarti/ordinalicl-tabdpt-turbo}
}

This citation will be updated with the assigned arXiv identifier when it is available.