kushalpatil/jevify-gemma4-e4b
google/gemma-4-E4B-it fine-tuned (LoRA, merged) to give honest probabilities when asked typed questions
about a piece of state — the model behind jevify, a local,
Jev-compatible probabilistic decision API.
Nothing is generated: one prefill, read the next-token distribution over the answer labels, done.
Use
pip install "jevify[transformers] @ git+https://github.com/kushalpatil07/jevify"
jevify serve --model kushalpatil/jevify-gemma4-e4b # POST /v1/systemone, Jev wire format
from jevify import Jevify, Noul, Choice, Score
jev = Jevify.from_transformers("kushalpatil/jevify-gemma4-e4b") # or serve it with vLLM and use Jevify.from_runtime("vllm", "kushalpatil/jevify-gemma4-e4b")
jev.system_one("Help! My payouts have been failing for 3 days.", {
"urgent": Noul("Does this convey urgency?"),
"team": Choice("Which team should handle this?", {"billing": None, "technical": None, "sales": None}),
"mood": Score("How frustrated is the customer?", ["calm", "frustrated", "furious"]),
})
Training
LoRA r=64 on attention projections, 2 epochs over ~47k (state, question, target-distribution)
items from 16 sources: hard-labeled classification sets with randomized option subsets and
order, multi-annotator sets with real human label distributions, and constructed long states
(up to 24k tokens) with answers known by construction. Loss = KL(target || label distribution).
No teacher model. Recipe: train/ in the jevify repo.
Results
Held-out, out-of-distribution (6 datasets not used in training; 307 items), no post-hoc scaling:
ood_massive n= 50 acc=0.900 nll=0.308 brier=0.140 ece=0.065 conf=0.932
ood_trec n= 50 acc=0.940 nll=0.201 brier=0.095 ece=0.097 conf=0.874
ood_paws n= 50 acc=0.540 nll=0.915 brier=0.622 ece=0.262 conf=0.792
ood_sms_spam n= 50 acc=0.980 nll=0.121 brier=0.056 ece=0.046 conf=0.934
ood_app_reviews n= 50 acc=0.740 nll=0.901 brier=0.427 ece=0.126 conf=0.705
ood_subj n= 50 acc=0.940 nll=0.218 brier=0.109 ece=0.128 conf=0.841
spike n= 7 acc=1.000 nll=0.051 brier=0.021 ece=0.044 conf=0.956
ALL n= 307 acc=0.844 nll=0.435 brier=0.237 ece=0.043 conf=0.849
In-distribution held-out (800 items), raw -> trained:
step 0 acc=0.745 conf=0.962 ece=0.217 nll=1.827 brier=0.412
step 1209 acc=0.823 conf=0.823 ece=0.028 nll=0.438 brier=0.193