VTXAI/VTX-JEV-1

VTX-JEV-1 is a 12.66M-parameter, non-autoregressive System 1 decision model. It reads one state and any number of typed questions, then returns calibrated distributions for choice, score, and noul in one forward pass. It never generates text.

The model is built on the dequantized VTXAI/vtx-embed-7M embedding table, then fine-tuned on all 655,806 rows of SargeDev/jev-distill-corpus-v3 for two epochs. The released weights use native blockwise LF4 storage.

Highlights

• 8.0 MB model file, 6.33x smaller than the FP32 checkpoint.
• 73.1% LF4 accuracy on 3,000 held-out JEV cases versus 72.6% FP32.
• 2.86x faster evaluation than FP32 on a T4 in the included implementation.
• CPU and CUDA support; no API key or remote inference service.
• Up to 255 Choice options, 2-10 Score levels, and binary Noul questions.
• Typed response helpers: response.choices, response.scores, and response.nouls.

Jev-style inference

from inference import JevClient, Choice, Noul, Score

client = JevClient.from_pretrained("VTXAI/VTX-JEV-1")

response = client.system_one(
    state="I was charged twice and production is unavailable.",
    questions={
        "refund": Noul("Does the customer request a refund?"),
        "team": Choice(
            "Which team should handle this?",
            {"billing": "Payments", "technical": "Production outage"},
        ),
        "severity": Score(
            "How severe is the impact?",
            ["Minor", "Major", "Critical"],
        ),
    },
)

print(response.nouls["refund"].noul)
print(response.choices["team"].choice)
print(response.scores["severity"].score)
print(response.to_dict())

After cloning/downloading this repository, import inference.py from its root. From a local checkout:

from inference import JevClient
client = JevClient.from_pretrained(".")

Transformers architecture loading

The repository contains its own architecture and config code, so no project package is required:

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "VTXAI/VTX-JEV-1",
    trust_remote_code=True,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
)

The model forward signature is:

logits, act_logits = model(
    input_ids,       # [batch, sequence]
    attention_mask,  # [batch, sequence]
    marker_pos,      # [batch, options], position of each [MASK]
    marker_mask,     # [batch, options]
    qtype,           # [batch], choice=0, score=1, noul=2
)

act_logits is retained for architecture compatibility but is not used by the Jev-style client.

Quantization

All 2D weights are packed as unsigned 4-bit nibbles with one FP16 scale and minimum per 32-value row block. LayerNorm, bias vectors, and temperatures remain floating point. CUDA computes in FP16; CPU computes in FP32.

import json
from safetensors import safe_open

with open("config.json") as handle:
    config = json.load(handle)

assert config["quantization"] == {
    "format": "lf4",
    "bits": 4,
    "block_size": 32,
    "scheme": "asymmetric_minmax",
    "compute_dtype": "float16_cuda_float32_cpu",
}

Evaluation

On 3,000 fixed held-out JEV cases:

The LF4 model is the default deployment artifact. The original FP32 checkpoint remains preserved in the training workspace and can be reconstructed from this release with python training/prepare_fp32.py --model . --output ./vtx-jev-fp32.

Training and quantization code

Complete training and LF4 conversion code is included under training/. See training/README.md for the full-corpus fine-tuning and quantization commands.

Intended use

VTX-JEV-1 is designed for routing, triage, policy gates, classification, ordinal rating, and other bounded decision tasks where application code needs probabilities rather than generated prose. Probabilities are model estimates and should be calibrated or monitored on the deployment distribution.

License

Apache-2.0. The base embedding model is MIT licensed; see its model card for provenance.