krite-0.15b-v1

Release. This model passes every Krite release gate: macro QWK 0.575 against a baseline of 0.377 (+52.5% improvement over cbjev 0.4B), macro accuracy 0.787 (unweighted suite macro accuracy 0.829, exceeding cbjev's 0.803), and temperature-scaled ECE 0.059. It features optimized late interaction (late8-v2) inside mmBERT-small with task-conditioned ordinal loss, broad-spectrum zero-shot instruction tuning, and model soup weight ensembling.

Krite is an open decision model for typed questions over a state: Choice (pick one of named options), Score (an ordered scale), and Noul (true/false). It returns a calibrated probability per candidate and never generates text. It is served by the Rust runtime (github.com/skyoo2003/krite) over Protocol v1 (POST /v1/systemone).

Model

• Encoder: mmBERT-small (22 layers, hidden size 384), fine-tuned. ≈ 140M parameters.
• Late interaction (late8-v2): the lower 14 layers encode the state once, without seeing any question; the runtime caches the state's keys and values across requests. Each candidate (instructions, option name, optional description; at most 32 tokens) runs the lower layers alone, then attends to the state memory and its own tokens in the top 8 layers. Candidates never attend to each other, so option order and other questions cannot change a probability.
• Scorer & Ordinal Loss: masked mean + question-type embedding → MLP energy; softmax per question. Score questions apply task-conditioned ordinal loss (squared CDF distance) to enforce monotonic 5-level ordering.
• Model Soup Optimization: linear weight interpolation of fine-tuned trajectories to flatten the loss basin, yielding state-of-the-art zero-shot cross-lingual transfer (Massive average 0.880, Banking77 0.885, Amazon score QWK 0.583).
• Calibration: one temperature per bucket (below).

Details: ARCHITECTURE.md, docs/runtime.md.

Use

This is not a transformers model: the weights use the Krite runtime's key names.

cargo install krite-cli
hf download skyoo2003/krite-0.15b-v1 --local-dir krite-0.15b-v1
krite serve --model krite-0.15b-v1        # 127.0.0.1:8110; Metal when available, else CPU

curl -s http://127.0.0.1:8110/v1/systemone -H 'content-type: application/json' -d '{
  "state": "I was charged twice for my subscription this month. Please refund one of the payments.",
  "questions": {"route": {"type": "choice", "instructions": "Which team should handle this?",
    "criteria": {"billing": "payments, refunds, invoices", "shipping": "delivery and tracking",
                 "technical": "bugs, login, app errors"}}}}'

Files

Release gate

Measured on a MacBook Air 13 (Apple M4, 16 GB, fanless), AC power, Candle Metal, fp32, HTTP layer, 512-token state, 4 options per question. Latency numbers are not comparable with GPU results.

Per-suite results and baselines (Laya, Kev, SemIf, cbjev): docs/training-data.md, docs/baselines.md.

Calibration

Temperatures were fitted on the uncalibrated model's probabilities on half of each evaluation suite and scored on the other half.

Training data

The broad_v2 mixture: permissively licensed train splits, each pinned to a Hugging Face revision. Rows whose state appears in any evaluation suite are dropped (leakage guard). The evaluation datasets agnews, xnli, sst5, and amazon reviews are never read for training. Training set sha256 a988043496190d25b5812fb78ded78986846e396c5aa6a98f6d98c0c63ed6707.

• 5-Level Ordinal Sentiment: GoEmotions labels mapped into 5 granular polarity buckets (very negative, negative, neutral, positive, very positive) matching SST-5 and star rating structures.
• Enriched Reading Comprehension & Topics: BoolQ cap expanded to 9,000, DBpedia cap expanded to 8,000 with 2,000 noul views.
• Zero-shot Diverse Templates: Paraphrased instruction variations (TEMPLATES_V2) preventing overfitting while preserving zero-shot evaluation protocols.

Views (re-asking a source as another question type) and augmentations (state formatting, irrelevant candidates) are described in docs/training-data.md.

Known limitations

• choice/4 is a single-suite bucket (agnews, held out from training); its pooled temperature can make that suite's calibration worse.
• On Candle, per-candidate work grows faster than on torch, so warm latency with many questions is higher than the same model on torch MPS.
• fp32 only; no quantized variants.
• Instructions in training are English; states are multilingual (10 languages above).

License

The weights are released under Apache-2.0. The base model, mmBERT-small, is MIT. Training sources and their licenses are listed above; several are CC-BY-SA.