DiffusionGemma 26B DJev v10 — step 160
This is the selected one-step DJev checkpoint from the v10 continuation of
google/diffusiongemma-26B-A4B-it. It is tuned for typed probabilistic
decisions through the Jev/System One request schema. The selected operating
point uses no generated reasoning block (think=0), four samples, and a
64-token answer canvas.
This is a research release. It returns calibrated decision distributions but should not be used as the sole decision-maker in high-impact settings.
Intended serving path
The evaluated path uses the structured DiffusionGemma runtime and the
/v1/systemone endpoint in the accompanying
Turbo-dLLM repository.
The checkpoint is a full Hugging Face export, not an adapter.
Recommended DJev settings:
{
"think": 0,
"samples": 4,
"canvas_size": 64
}
The upstream generation_config.json remains in the export for general
DiffusionGemma generation. The DJev scores below were measured with the
structured one-step serving path, not generic generate() defaults.
Evaluation
JevBench public v1.3-format set
All 231 public requests completed without error. This public set was reported, not used to select step 160; selection used held-out gates.
| Overall accuracy | Easy | Standard | Judge | Hard | Hard ECE | C13 | I13 estimate |
|---|---|---|---|---|---|---|---|
| 0.8831 | 1.000 | 0.986 | 0.882 | 0.745 | 0.044 | 82.9 | 83.2 |
In a paired run, this checkpoint exceeded Mapika/decider-4b v2 on the public
set (0.8831 versus 0.8355) and on all five internal held-out sets. See the
comparison report
for the pinned revisions, confidence intervals, latency methodology, and raw
artifact hashes.
LocalLLaMA/typed-decisions test
This result is in-task/specialist, not zero-shot: the model's training lineage includes the dataset's disjoint train/development rows. The 400-case test split was evaluation-only.
| Accuracy | Soft accuracy | KL | TV | Brier | ECE | Score MAE | Within 1 |
|---|---|---|---|---|---|---|---|
| 0.687 | 0.580 | 0.421 | 0.247 | 0.166 | 0.111 | 0.436 | 0.925 |
The exact dataset revision, test hash, scorer, per-type results, and prediction artifact hashes are in the evaluation report.
Training and selection
- Base:
google/diffusiongemma-26B-A4B-itat revisionf7f5b7f5fa82ffc52addd066915886d497f5517b. - Warm start: DJev v9 step 343.
- Update: BF16 all-text continuation with FSDP2 across eight H100s, a 4,096 token training maximum, preservation distillation, and mixed teacher/gold targets.
- Optimizer: AdamW, peak learning rate
1e-6, cosine decay, batch size one per rank with four gradient-accumulation steps. - Candidate checkpoints: steps 160, 240, and 320. Step 160 was selected on held-out hard/generalization gates rather than the public score.
- No JevBench public evaluation row or expected answer was used as training data. The public-development and synthetic/replay construction is recorded in the repository's recipes and run scripts.
Reproduction entry points:
dllm_parallel/recipes/runs/diffusiongemma-26b-djev-distill-v10.yamlruns/data-v10/run_v10.shruns/decider-v2-comparison/run_comparison.shruns/typed-decisions-eval/evaluate_endpoint.py
Limitations
- The checkpoint is specialized for structured decisions; the fine-tune was not evaluated as a general chat or multimodal assistant.
- Typed Decisions Choice and ordinal Score remain weaker than the published TypeSafe Jev 1.13.0 reference, while Noul accuracy is higher.
- H100 latency measurements are serving-stack and hardware specific.
- The model inherits the capabilities, failure modes, and license obligations of the DiffusionGemma base model.