OCF typed-decisions CE smoke (ModernBERT-base)
Not a foundation classifier. Not RLCD. No Jev comparison.
Open Classification Foundation (OCF) specialist smoke: non-autoregressive System-1
decision head on answerdotai/ModernBERT-base,
fine-tuned with cross-entropy on LocalLLaMA/typed-decisions train, temperature-calibrated
on a held-out train slice (cal_frac=0.1).
Architecture
ModernBERT-base (bidirectional)
+ 2-layer decision Transformer head
+ OptionScorer at each option [MASK]
→ softmax(/T) over options
| Knob | Value |
|---|---|
| Params | ~164M |
max_len | 640 |
| Option token budget | 48 / option |
| Primitives | choice / score / noul |
Training
| Field | Value |
|---|---|
| Objective | CE on teacher-argmax labels (soft teachers in dataset) |
| Epochs / eff. batch | 4 / 8 |
| LR encoder / head | 2.5e-5 / 1e-4 |
| Seed (this upload) | 1 (best of 0/1/2; mean acc 0.731, SD 0.028) |
| Temperatures (shipped) | choice 1.5 · score 1.25 · noul 1.5 |
Evaluation (LocalLLaMA/typed-decisions test, 2,000 decisions)
| Arm | acc | NLL | ECE as-run | ECE after equal crossfit |
|---|---|---|---|---|
| Laya typed-decisions (reference) | 0.767 | 0.707 | 0.214 | 0.021 |
| This checkpoint (s1) | 0.751 | 0.629 | 0.044 | 0.030 |
| OCF stub (prior, ~8M) | ≈0.61 | ≈0.90 | ≈0.04 | — |
Latency (warm, RTX 3050 Laptop): p50 59.6 ms / p95 87.8 ms per question (n=200). Not a cross-vendor claim.
Full board: see source-repo snapshot results-snapshot-2026-09-28-ocf-typed-decisions-mbert.md.
Files
config.json—ClassifierConfig(includes temperatures)temperatures.json— fitted per-type Tmodel.pt— fullClassifierBundlestate_dictrecipe.json— train hyperparams + acc
How to run
Requires the OCF stack from the source repository (packages.classify + models_ai.classify):
from classify import load
agent = load("path/to/this/repo") # config.json + model.pt + temperatures.json
out = agent.predict(
{"subject": "Refund", "body": "Billed twice"},
{
"department": {
"type": "choice",
"instructions": "Which department?",
"criteria": {"billing": "invoices", "tech": "bugs", "other": "else"},
}
},
)
Not a drop-in for the external laya package.
Concurrent baseline
Laya — open System-1 concurrent (ModernBERT-large). Matched estimand only; not a dependency.
Non-claims
- Not a foundation / zero-shot System-1 model
- Not RLCD-trained
- Does not claim to beat TypeSafe Jev or Laya
- Soft teacher labels; teachers disagree on ~41% of test decisions