FRIDA-Decisions

FRIDA-Decisions makes structured decisions over Russian text in a single encoder pass: pick one of K options, place a text on an ordinal scale, answer yes / no, or rank candidates. The options are written as text inside the request, so a new label set is a new JSON, not a new training run. No generation, no output tokens, no parsing: every answer is one of the declared options, with its confidence.

It is built on ai-forever/FRIDA (T5 encoder, 823M parameters) and runs on a consumer GPU. Developed by УЭСМО, SberAI.

• razvilka. 0.893 on razvilka (735 items); TypeSafe Jev, a commercial API, scores 0.897 on the same items (paired McNemar p = 0.84). The highest among the open models we ran on razvilka.
• Fast. 28–34 ms per request on an RTX 5060 Ti (a ≈400-token text, 1–3 questions), in process; conditions in the latency table below.
• Light. 1.8 GiB allocated by PyTorch at peak over the whole razvilka run, plus the CUDA context; an int8 ONNX build runs on CPU, and an MLX build on Apple Silicon.
• Packing. All options of all questions share one sequence and the text is encoded once; with the state cache a follow-up question about the same text costs only its own tokens. A catalog of 243 intents is answered in 0.44 s, against 4.65 s for one sequence per option.
• Serving. A vLLM server batches requests from many users and keeps the texts it has read in its prefix cache: about 60 requests/s on one RTX 5060 Ti with 8 requests in flight.

Quickstart

pip install "frida-decisions[torch] @ git+https://github.com/ai-forever/FRIDA-Decisions@v0.4.0"

from frida_decisions import Judge

judge = Judge.from_pretrained("ai-forever/FRIDA-Decisions", device="cuda")

request = {
    "state": "Здравствуйте, у меня не приходит код подтверждения уже час.",
    "questions": {
        "topic": {
            "type": "choice",
            "instructions": "К какой теме относится обращение?",
            "criteria": {
                "login": "вход в аккаунт, коды подтверждения, пароли",
                "payment": "оплата, списания, возвраты",
                "delivery": "доставка заказа",
            },
        },
        "urgency": {
            "type": "score",
            "instructions": "Оцени срочность обращения.",
            "criteria": ["низкая", "средняя", "высокая"],
        },
        "spam": {
            "type": "noul",
            "instructions": "Является ли сообщение спамом?",
            "criteria": {"true": "реклама, чужие ссылки, просьба перевести деньги",
                         "false": "вопрос или жалоба по нашему сервису"},
        },
    },
}
print(judge.judge(request)["answers"])

CPU without PyTorch: pip install "frida-decisions[onnx] @ git+..." and OnnxJudge.from_pretrained("ai-forever/FRIDA-Decisions") — int8 weights and per-token int8 activations; it scores 0.891 on razvilka (the same decision as the GPU model on 726 of 735 items), and a 384-token request with 3 questions takes about 0.9 s on 6 CPU threads, roughly 2.5x faster than fp32.

Notebooks: quickstart · evaluation on razvilka

vLLM server

For many users at once (Linux, GPU). The server batches requests together and keeps the texts it has read in vLLM's prefix cache, so a follow-up question about a text costs only its own tokens.

pip install "frida-decisions[vllm] @ git+https://github.com/ai-forever/FRIDA-Decisions@v0.4.0"
vllm serve ai-forever/FRIDA-Decisions \
  --hf-overrides '{"architectures": ["FridaDecisionsModel"]}' \
  --io-processor-plugin frida_decisions \
  --no-enable-chunked-prefill --enforce-eager --max-model-len 2048

POST /pooling with the request under data returns the same response as Judge (client example).

Measured with the command above plus --gpu-memory-utilization 0.45 (the card also drives a display), client on the same machine, after warm-up; the first request after a start takes about 0.3 s. vLLM and PyTorch differ on 2 razvilka items, both near-ties in fp32.

Async API, no server. await judge.judge(request) inside your own program: AsyncJudge wraps the PyTorch (or ONNX, or MLX) judge on any OS, VllmJudge runs the vLLM engine in your process (Linux, GPU) with the same batching and cache as the server (example). Which is faster depends on the load: on the same card, a burst of 256 short tickets runs at ≈187 requests/s with AsyncJudge over PyTorch and ≈126 with VllmJudge; razvilka's mix of lengths with 32 requests in flight, at ≈58 and ≈69 (VllmJudge with 45 % of the GPU, medians of alternating runs).

Apple Silicon. MlxJudge (frida-decisions[mlx], contributed by @akolotov) runs the model in MLX, without torch. On an Apple M4 Pro, in float32 it makes the same 735 razvilka decisions as PyTorch float32 on CPU; in bfloat16, 733 of them, with the margin error of bfloat16 in PyTorch (example). PyTorch itself also runs on a Mac with device="mps".

Question types

Filtering ranking candidates by a threshold

probabilities split one unit among the candidates, so they order but cannot filter. To keep only the relevant candidates, threshold scores against the request's own list (each score minus the mean of its candidates), not as raw values. The recipe, its numbers on razvilka and its limits are in the repository README; examples/quickstart.py and the Colab notebook filter a retriever's output.

Input length

• The 512-token range is for the text (state) only. Questions and options do not count against it: each question's instructions take up to 96 tokens and each option up to 256. Options are packed into rows of up to 16 options / 1,024 tokens next to the text, and a request with more options simply gets more rows, so the number of options is limited only by time and memory.
• Default cut: 384 tokens. Judge.from_pretrained(..., state_max=512) uses the whole trained range (the razvilka numbers above use 512). A longer text is cut from the end, and the response says so in usage.state_truncated. On the vLLM server the cut is set with FRIDA_DECISIONS_STATE_MAX.
• Longer texts. T5's relative positions accept a longer state_max, but the model was trained on texts up to 512 tokens, and on longer ones accuracy may drop, especially for questions about details deep in the text. For long documents, split them into fragments of up to 512 tokens (by paragraph or section) and ask the questions per fragment: the fragments go in one judge_batch call, and a yes/no question like "is there X anywhere" becomes the maximum over fragments.
• Cost grows with text + options, not text × options: the text is encoded once per request, and with the state cache a follow-up question about the same text costs only its own tokens.

Raising the limits. The caps are settings, not part of the weights:

judge = Judge.from_pretrained("ai-forever/FRIDA-Decisions", state_max=1024)  # text cut; the same for OnnxJudge
judge.config.option_max_tokens = 512        # one option (default 256)
judge.config.instruction_max_tokens = 192   # one question's instructions (default 96)
judge.config.max_row_tokens = 2048          # one packed row (default 1024)
judge.config.max_options_per_row = 32       # options per row (default 16)

The defaults ship with the weights in decisions_config.json; edit it in a local copy of the model folder to change them for every backend. For the vLLM server, point vllm serve at that folder, set FRIDA_DECISIONS_STATE_MAX for the text cut, and keep --max-model-len above the longest row (the server says so when a row does not fit). Larger values cost time and memory, and the model was trained within the defaults, so check quality on your own data before relying on them.

Benchmarks

razvilka — 735 Russian items, 15 tasks (routing, intents, topic and sentiment classification, moderation, relevance ranking), all four question types, gold from published datasets. Every model answers the same items, each in its own input format, and all answers are scored by the same rule (razvilka_eval.py).

¹ 140 of 735 texts exceed open-jev's input window and get no answer from it and count as ties; on the other 595 it scores 0.565.

² Zero-shot through OpenRouter: one call per question at temperature 0, the text, question and options in the prompt, the chosen option key returned as JSON; reasoning off for Qwen3.5, low for gpt-oss, which cannot switch it off. Against FRIDA-Decisions (paired McNemar): Qwen3.5-35B-A3B p = 0.84, gpt-oss-20b p = 0.02.

Latency, one request, single stream, in process:

GPU and Apple rows: median of 30 requests after warm-up, timing parsing, tokenisation, packing and the forward pass (Apple: tools/mlx_check.py, one run; MLX float32 takes 126 and 156 ms, PyTorch on MPS in bf16 139 and 168 ms). CPU rows were measured on a machine with background load; the ratio between them (about 2.5x) is the stable part.

Packing, RTX 5060 Ti, bf16, same model, one sequence per option vs packed with the state cache:

Training

• 1.42M examples (1.5M questions) over 151 question types; by our grouping they fall into eight domains: relevance and RAG grounding, NLI and fact checking, moderation and safety, agents and tool choice, topic classification, sentiment and emotion, LLM request routing, intents and customer support.
• 71% Russian, 29% English; mostly human or naturally labelled data, roughly a quarter (our estimate) with LLM-generated text or model labels; instruction wordings augmented with paraphrases.
• LoRA rank 16 on the attention projections (q, k, v, o) of all 24 layers plus a scalar head, 4.7M trainable parameters; merged into the weights in this release. Listwise softmax for choice / ranking, pairwise BCE for noul.
• One epoch, with a compute budget comparable to about 50 hours of a single consumer GPU (RTX 5060 Ti class).

Files

• model.safetensors — encoder weights, bf16
• head.safetensors — readout head, fp32
• decisions_config.json — packing limits and instruction suffixes
• onnx/model_int8_pertoken.onnx — CPU build

License

MIT. Based on ai-forever/FRIDA (MIT).

Citation

@misc{frida_decisions_2026,
  title  = {FRIDA-Decisions},
  author = {{УЭСМО, SberAI}},
  year   = {2026},
  url    = {https://github.com/ai-forever/FRIDA-Decisions}
}