RT-J — fp16 checkpoints
Half-precision conversions of stanford-star/rt-j,
the Relational Transformer foundation model for in-context learning over
relational databases. Produced and consumed by the
RelativeDB native inference engine
(cpp/rt_quantize --type f16). Weights stay f16-resident at inference:
the engine's CPU (Accelerate / portable SIMD) and Metal/MPS kernels convert
to fp32 inside the GEMM, halving weight memory and DRAM traffic vs. fp32
with no measurable accuracy loss (drift vs. the PyTorch reference is
identical to the fp32 checkpoint on the golden batch).
Sibling repos: rt-j-int8 · rt-j-int4
| File | Task head | Size |
|---|---|---|
classification/model.f16.safetensors | classification / ranking (logits — apply sigmoid) | 172 MB |
regression/model.f16.safetensors | regression / forecasting (normalized values) | 172 MB |
Format
Plain safetensors: every transformer-block projection (wq/wk/wv/wg, wo,
ffn.w1/w2/w3 — ~99% of parameters) is stored as F16 (IEEE half,
round-to-nearest). The value/col-name encoders, decoder head, norms, biases
and mask embeddings stay fp32. Any safetensors reader can load these files.
Accuracy
Golden batch (B=5, S=16) vs. the PyTorch reference, identical on CPU and
Metal/MPS: yhat max abs. error 3.9e-3 — the same as the fp32 checkpoint
(upstream weights are bf16, so f16 storage adds no error above fp32
op-ordering drift).
Usage (RelativeDB native engine)
./build/rt_test testdata classification/model.f16.safetensors --quantized --device mps
# via the Java / Python / Rust bindings: place the .f16 file next to the fp32
# checkpoint (or point at a directory containing it) and opt in with
export RELATIVEDB_RT_QUANTIZED=f16
The C ABI (rt_model_load) accepts these files directly — the format is
auto-detected from the tensor dtypes. f16 checkpoints currently run on the
CPU and Metal/MPS backends (CUDA is fp32-only).
Reproduce
cmake -B build -S cpp && cmake --build build -j
./build/rt_quantize <rt-j>/classification/model.safetensors classification/model.f16.safetensors --type f16
./build/rt_quantize <rt-j>/regression/model.safetensors regression/model.f16.safetensors --type f16
License & attribution
Derivative of stanford-star/rt-j (Stanford STAR lab), redistributed under the same CC-BY-NC-SA-4.0 license. Architecture and training details are described in the upstream model card; only the weight storage format differs here.