MiniJEV · General

A small decision model for English and Italian. You give it a state (optional text), a question, and any list of choices. It returns the chosen option and a calibrated probability for each choice.

It is not a chatbot and never generates text. It only scores the choices you give it, so it cannot answer with something outside the list. This makes it fast, local and predictable, and its confidence has a meaning.

dec.decide("", "Is a hotdog food?", ["yes", "no"])            # -> yes   0.79
dec.decide("", "Is a tomato a fruit?", ["yes", "no"])          # -> yes   0.78
dec.decide("chair", "What is this?", ["animal", "furniture", "vehicle"])   # -> furniture
dec.boolean("balena", "È un tipo di mammifero?", labels=("sì", "no"))

How it works

state ─┐
question ─┼─ multilingual MiniLM-L12 (fine-tuned) ─► MiniJEV core ─► score per choice ─► softmax / T
choices ─┘   (each choice is encoded by meaning, so any list of 2..N choices works)

• Text encoder: paraphrase-multilingual-MiniLM-L12-v2, fine-tuned end to end. It has 117.7M parameters, 96M of which are the multilingual vocabulary.
• MiniJEV core: 0.83M parameters. It uses a bi-encoder scorer: each choice's meaning is compared with the state + question.
• Choice handling: choice order does not matter (the model is permutation-equivariant), and the number of choices is dynamic.
• Calibration: a temperature is fitted on validation data (T = 3.51).

This encoder was selected by an ablation study, summarised below. All variants were trained on the same data and scored on validation, with unseen entities:

On the earlier, smaller dataset, a fixed output head (which scores positions, not the meaning of the choices) scored 0.37. It was not permutation-invariant.

Training data: distilled, not scraped

Qwen2.5-3B-Instruct was used only offline, as a teacher, to build the dataset. It is never used at inference.

1. The teacher listed examples for 100 categories: food, animals, software, places, people, science and more.
2. For each (entity, category) pair, the teacher's next-token probability of "yes" vs "no" was stored as a soft label.
3. Entities the teacher itself did not place in their category were dropped. 2,221 entities remained, giving about 24k decisions: yes/no questions and 2–10-way category choices, in English and Italian, with many phrasings.

Training used 50% hard labels and 50% KL distillation on the teacher distribution. Splits are by entity: validation and test entities are never seen in training.

Results (unseen entities)

Latency on a laptop CPU, one decision including the encoder:

• 1 thread: p50 31 ms, p95 41 ms, p99 45 ms
• 4 threads: p50 26 ms

Limits (read before using)

• Knowledge is shallow. It confuses close categories. For example, it answers "is a whale a fish?" with yes, and it misplaces some less common technical names (such as Redis). Treat it as a fast, calibrated first-pass classifier, not as a knowledge base.
• Italian is weaker than English.
• The dataset is synthetic (template questions). Unusual phrasings work less well.
• Use the confidence. Low-confidence answers (under about 0.6) are right much less often. Route them to a human or a larger model.

Usage

git clone https://huggingface.co/markuz89/minijev-general && cd minijev-general/code
pip install -e ".[train,text]"

from minijev.inference.runtime import MiniJEV

dec = MiniJEV.load("..")                       # the cloned repo folder (or the repo id)
d = dec.decide("", "Is Python a programming language?", ["yes", "no"])
print(d.choice, d.confidence, d.scores)        # calibrated distribution over your choices
print(d.raw_scores)                            # uncalibrated, for comparison

Files

• model.safetensors: MiniJEV weights
• text_encoder/: the fine-tuned MiniLM, with its tokenizer
• config.json: architecture, calibration and metrics
• code/: the minijev package

Everything is safetensors and JSON, with no pickle.