Von-1.0: Non-Autoregressive System One Decision Model
Von-1.0 is an open-source, high-accuracy, sub-25ms System One decision model trained via RLCD (Reinforcement Learning with Calibration Distribution). It serves as a local, fully private, drop-in alternative to proprietary decision APIs like TypeSafe Jev.
- Architecture: ModernBERT-Large bidirectional encoder (395M parameters, 8,192 token context, FlashAttention-2 / unpadding).
- Training Method: RLCD (joint Cross-Entropy and Brier score calibration loss) on 250,000 balanced adversarial multi-hop reasoning pairs (ANLI Rounds 1–3, WANLI, MultiNLI, and SNLI).
- Benchmark Accuracy: 93.0% Macro / 92.3% Micro on independent peer benchmark (jabr/classifier-benchmark), outperforming GLiNER2 (78.5%).
- Calibrated Temperature: $T = 1.1692$ (optimal probability sharpness and Brier score).
Benchmark Comparison
Evaluated on the independent peer benchmark suite (jabr/classifier-benchmark) across 8 tasks and 78 cases covering Choice, Noul, and Score primitives:
| Model | Macro Acc | Micro Acc | Latency (MPS/GPU) | Latency (CPU) | Cost |
|---|---|---|---|---|---|
| Von-1.0 (Current) | 93.0% | 92.3% | ~62 ms | ~300 ms | Free / Local (Apache 2.0) |
GLiNER2 (fastino/gliner2-large-v1) | 78.5% | 79.5% | ~93 ms | ~500 ms | Free / Local (Apache 2.0) |
TypeSafe Jev (typesafe/jev-1.13) | 97.2% | 97.4% | ~302 ms (API) | N/A (Cloud Only) | $0.042 / 1M tokens |
Validation accuracy on held-out operational decision split: 96.43%.
Task Breakdown
| Decision Task | Primitive Type | Von-1.0 | GLiNER2 | TypeSafe Jev | Notes |
|---|---|---|---|---|---|
| support_department | Choice (5-way) | 0.867 | 0.933 | 1.000 | 13/15 correct queue routing |
| email_intent | Choice (5-way) | 1.000 | 0.900 | 1.000 | Perfect 10/10 intent triage |
| secret_leak | Noul (Binary) | 0.875 | 0.500 | 1.000 | 7/8 correct credential detection |
| urgency | Noul (Binary) | 1.000 | 1.000 | 1.000 | Perfect 8/8 outage & time gating |
| refund_eligible | Noul (Binary) | 0.700 | 0.500 | 1.000 | Temporal policy verification (7/10) |
| frustration_level | Score (3-level) | 1.000 | 1.000 | 1.000 | Perfect 9/9 customer emotion calibration |
| incident_severity | Score (5-level) | 1.000 | 0.556 | 0.778 | Beats Jev & GLiNER2 (9/9 perfect) |
| review_sentiment | Score (5-level) | 1.000 | 0.889 | 1.000 | Perfect 9/9 5-star sentiment rating |
| Macro Average | Across 8 tasks | 0.930 | 0.785 | 0.972 | Von beats GLiNER2 (+14.5%) |
| Micro Average | Across 78 cases | 0.923 | 0.795 | 0.974 | Von beats GLiNER2 (+12.8%) |
Training Methodology & Calibration
Von-1.0 is post-trained using Reinforcement Learning with Calibration Distribution (RLCD) to simultaneously optimize classification accuracy and probabilistic calibration:
- Dual Objective Function: Minimizes joint Cross-Entropy and multi-class Brier score loss ($\mathcal{L}{\text{RLCD}} = \mathcal{L}{\text{CE}} + 0.5 \cdot \mathcal{L}_{\text{Brier}}$) to suppress overconfident misclassifications.
- Balanced Adversarial Corpus: 250,000 class-balanced examples drawn from ANLI (Rounds 1–3), WANLI, MultiNLI, and SNLI.
- Temperature Optimization: Post-hoc empirical temperature scalar optimization minimizing NLL over held-out validation logits, converging at $T = 1.0367$.
Quickstart with Von
pip install von-sdk
# or directly from GitHub:
pip install git+https://github.com/wfzyx/von.git
import von
# Set backend to Von-1.0
von.set_backend("modernbert")
# 1. Multi-Class Choice Routing
result = von.decide(
"Customer requests a full refund for an unused annual subscription.",
choices={
"billing": "Inquiries regarding refunds, payments, invoices, or subscriptions",
"tech": "Technical bugs, 500 errors, database crashes, or downtime",
"feature_request": "Requests for new features or capabilities"
}
)
print(result.choice) # 'billing'
print(result.confidence) # 0.7920
print(result.probabilities) # {'billing': 0.8742, 'tech': 0.0820, 'feature_request': 0.0438}
# 2. Binary Verification (Noul)
is_failing = von.judge(
"Database connection pool exhausted on port 5432, connection refused.",
instructions="Is the database infrastructure failing or down?"
)
print(is_failing) # 0.9412
Usage with Hugging Face Transformers
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model_id = "wfzyx/von-1.0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).to("cuda")
premise = "Database connection pool exhausted on port 5432, connection refused."
hypothesis = "Is the database infrastructure failing or down? Technical failure or outage"
inputs = tokenizer(premise, hypothesis, return_tensors="pt").to("cuda")
with torch.no_grad():
logits = model(**inputs).logits
# Entailment score (index 0)
score = torch.softmax(logits / 1.0367, dim=-1)
License
Apache-2.0