RT-J — FP8 E4M3 checkpoints
Native FP8 quantizations of RelativeDB/rt-j-fp16,
the RT-J Relational Transformer foundation model for prediction over related
data. These checkpoints are produced and consumed by
relational-transformers.
Sibling repos: rt-j-fp16 · rt-j-int8 · rt-j-int4
| File | Task head | Size |
|---|---|---|
classification/model.fp8.safetensors | classification / ranking logits | 82 MiB |
regression/model.fp8.safetensors | regression / forecasting normalized values | 82 MiB |
FP8 Format
Every two-dimensional matrix weight is stored directly as PyTorch
float8_e4m3fn. Biases, normalization scales, and mask embeddings retain the
source checkpoint dtype. The format is ordinary safetensors with
quantization=fp8_e4m3fn metadata and requires no packed custom container.
Portable PyTorch expands FP8 weights while loading. The Triton CUDA backend keeps them FP8-resident and uses supported GPU tensor-core paths.
Accuracy
Measured on the RelativeDB golden relational batch (B=5, S=16) against
RelativeDB/rt-j-fp16 using the portable PyTorch backend:
| Task | Maximum absolute logit error | Mean absolute logit error |
|---|---|---|
| classification / ranking | 0.0542 | 0.0435 |
| regression / forecasting | 0.0286 | 0.0213 |
Target-score signs and ranking are preserved on this batch. Validate task metrics against FP16 on representative application contexts before deployment.
Usage
from relational_transformers import RelationalTransformer
classifier = RelationalTransformer("RelativeDB/rt-j-fp8")
probabilities = classifier.predict(relational_batch)
regressor = RelationalTransformer("RelativeDB/rt-j-fp8", task="regression")
predictions = regressor.predict(relational_batch)
For native FP8 CUDA inference:
model = RelationalTransformer("RelativeDB/rt-j-fp8", backend="triton")
Reproduce
pip install relational-transformers
relational-transformers-quantize RelativeDB/rt-j-fp16 ./rt-j-fp8
License and Attribution
Derivative of stanford-star/rt-j (Stanford STAR lab), redistributed under the same CC-BY-NC-SA-4.0 license. Architecture and training details are described in the upstream model card; only the weight storage format differs here.