openjev-rlcd-qwen3-4b-v1

Status: training in progress (v1, single-category checkpoint) — this card will be updated with final metrics once training completes.

A Qwen3-4B checkpoint fine-tuned with RLCD (Reinforcement Learning for Calibrated Decisions) via GRPO, using XiaomiMiMo/verl (a fork of verl-project/verl) as the training framework.

What this is

This model is trained to produce typed decisions with calibrated confidence — given a short piece of text and a small set of labeled options (2-20), it outputs a single-line JSON object naming its chosen label and a confidence score for that choice:

{"answer": "B", "confidence": 0.94}

No chain-of-thought, no free-form generation — one short structured decision per input, matching a "System One" inference style rather than a reasoning model.

This is an independent, non-affiliated reproduction of the training methodology described by TypeSafe AI's Jev model (RLCD instead of plain RLHF / correctness-only reward). It is not TypeSafe's Jev model, not derived from their weights, and not endorsed by them — named openjev (not jev) for exactly this reason, mirroring why the original open-source clone this project descends from (workszop/openjev) was itself renamed to SemIf to avoid trademark confusion with the commercial product.

Why RLCD instead of plain correctness reward

A binary correct/incorrect reward (the default in most GRPO recipes, e.g. verl's own GSM8K example) only trains for accuracy — a model can learn to always claim high confidence and still score well on average. RLCD instead scores every rollout with a Brier score proper scoring rule:

reward = 1 - (confidence - is_correct)^2

This is maximized only when the model is both correct and confident, and penalizes confident-but-wrong answers harder than unsure-but-wrong ones — the calibration property that lets downstream software act automatically on high-confidence decisions and route low-confidence ones to human review, which is the whole point of a typed-decision model.

Training data (v1)

• Dataset: benayas/snips — SNIPS intent classification, 7 intents (AddToPlaylist, BookRestaurant, GetWeather, PlayMusic, RateBook, SearchCreativeWork, SearchScreeningEvent), 13,084 train / 1,400 test examples.
• Each example is reframed as a lettered multiple-choice question with the intent options shuffled per-row (seeded, deterministic) to avoid positional bias.
• v1 scope note: this checkpoint only covers one of the ~14 typed-decision use-case categories the wider Jev-style research surveyed. Broader categories (content moderation, severity classification, tool routing, prompt-injection detection, etc.) are a planned v2, not yet included here.

Training setup

Important inference note: Qwen3's chat template defaults to "thinking mode." This model was trained with it explicitly disabled — at inference time you must pass the same template kwarg (e.g. chat_template_kwargs={"enable_thinking": false} on an OpenAI-compatible vLLM server, or enable_thinking=False to tokenizer.apply_chat_template), otherwise the model will spend its whole (short) response budget on <think>...</think> reasoning and never emit the JSON decision.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("dknguyen2304/openjev-rlcd-qwen3-4b-v1")
tokenizer = AutoTokenizer.from_pretrained("dknguyen2304/openjev-rlcd-qwen3-4b-v1")

prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": (
        "You are a typed-decision engine (System One). Given the input, choose exactly ONE "
        "label from the options below and report your confidence that this label is correct.\n"
        "Respond with ONLY a single-line JSON object in this exact format, no other text, no "
        'reasoning, no markdown fences:\n{"answer": "<LETTER>", "confidence": <number between 0 '
        "and 1>}\n\n"
        "Input: add this song to my road trip playlist\n\n"
        "Options:\nA) BookRestaurant\nB) AddToPlaylist\nC) GetWeather\nD) PlayMusic\n"
        "E) RateBook\nF) SearchCreativeWork\nG) SearchScreeningEvent"
    )}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

Limitations

• v1 covers a single, relatively easy intent-classification domain (SNIPS) — generalization to harder/more ambiguous typed-decision domains is untested.
• Confidence is a single scalar attached to the chosen answer, not a full probability distribution over all options.
• Not affiliated with, endorsed by, or derived from TypeSafe AI's Jev product.