kushalpatil/jevify-gemma4-26b-a4b
google/gemma-4-26B-A4B-it fine-tuned (LoRA, merged) to give honest probabilities when asked typed questions
about a piece of state — the model behind jevify, a local,
Jev-compatible probabilistic decision API.
Nothing is generated: one prefill, read the next-token distribution over the answer labels, done.
Use
pip install "jevify[transformers] @ git+https://github.com/kushalpatil07/jevify"
jevify serve --model kushalpatil/jevify-gemma4-26b-a4b # POST /v1/systemone, Jev wire format
from jevify import Jevify, Noul, Choice, Score
jev = Jevify.from_transformers("kushalpatil/jevify-gemma4-26b-a4b") # or serve it with vLLM and use Jevify.from_runtime("vllm", "kushalpatil/jevify-gemma4-26b-a4b")
jev.system_one("Help! My payouts have been failing for 3 days.", {
"urgent": Noul("Does this convey urgency?"),
"team": Choice("Which team should handle this?", {"billing": None, "technical": None, "sales": None}),
"mood": Score("How frustrated is the customer?", ["calm", "frustrated", "furious"]),
})
Training
LoRA r=64 on attention projections, 2 epochs over ~47k (state, question, target-distribution)
items from 16 sources: hard-labeled classification sets with randomized option subsets and
order, multi-annotator sets with real human label distributions, and constructed long states
(up to 24k tokens) with answers known by construction. Loss = KL(target || label distribution).
No teacher model. Recipe: train/ in the jevify repo.
Results
Held-out, out-of-distribution (6 datasets not used in training; 307 items), no post-hoc scaling:
ood_massive n= 50 acc=0.860 nll=0.349 brier=0.179 ece=0.102 conf=0.939
ood_trec n= 50 acc=0.940 nll=0.202 brier=0.102 ece=0.064 conf=0.922
ood_paws n= 50 acc=0.580 nll=0.971 brier=0.599 ece=0.288 conf=0.855
ood_sms_spam n= 50 acc=0.960 nll=0.112 brier=0.056 ece=0.038 conf=0.946
ood_app_reviews n= 50 acc=0.700 nll=0.939 brier=0.451 ece=0.166 conf=0.717
ood_subj n= 50 acc=0.940 nll=0.183 brier=0.090 ece=0.149 conf=0.862
spike n= 7 acc=1.000 nll=0.064 brier=0.024 ece=0.057 conf=0.943
ALL n= 307 acc=0.834 nll=0.450 brier=0.241 ece=0.061 conf=0.875
In-distribution held-out (800 items), raw -> trained:
step 0 acc=0.757 conf=0.991 ece=0.234 nll=3.179 brier=0.425
step 1215 acc=0.821 conf=0.829 ece=0.032 nll=0.422 brier=0.188