jul-decision-e5-small
multilingual-e5-small trained to answer
jul decisions: Choice, yes/no (Noul) and Score. Small enough to run
inside an AWS Lambda function. ONNX 8-bit build: usejul/jul-decision-e5-small-onnx.
jul models add jul-decision-e5-small --repo usejul/jul-decision-e5-small-onnx --backend onnx
Two sets of weights, same size (e5-small):
- the root: vectors. Text and options are encoded apart and compared; reads Choice, and every
question tuned with
autotune. Inputs are prefixed withquery:. cross/: a cross model. The question and the text are read together; reads Noul and Score with no labeled examples.jul models addattaches it by itself;jul packships only the part a deployment needs.
Results
Jev bench (AG News, Banking77, Emotion; Choice only, so the vectors), ONNX 8-bit, 4 ms per text:
| e5-small | jul-decision-e5-small | |
|---|---|---|
| zero-shot | 0.543 | 0.557 |
autotune, hybrid head (1,000 labels) | 0.790 | 0.780 |
Kev's typed decisions (transfer-v9 development split, clean questions, never trained on), through jul:
| yes/no | Choice | Score | all | |
|---|---|---|---|---|
| vectors only | 0.579 | 0.401 | 0.275 | 0.460 |
| with the cross model | 0.726 | 0.401 | 0.500 | 0.524 |
The cross model: paraphrase 0.50 → 0.79, inference (QNLI) 0.55 → 0.70, offensive posts 0.725 → 0.80; "is the customer angry?" 0.84–0.99 on angry support messages, 0.00–0.02 on calm ones.
Not good at: urgency, hard but polite complaints (read as offensive), sentences with the same words in another order, knowledge questions. With few labels per task, tune: zero-shot is a starting point.
Training
jul's own score as the loss (vectors), and a cross-entropy over pairs with one head per question type (cross model), on public English and French datasets under commercial-use licenses: classification, tool routing, moderation, paraphrase, inference, compositions, dates and tone. Embeddings frozen for the vectors. One GPU.
Versions
v2.0 (this one): stage-2 vectors and the cross model. v1.0: the first vectors, without cross/
(revision="v1.0").
License
Apache-2.0. Based on multilingual-e5-small (MIT).