DreamReasoner-8B FineCode any-order CPT
This is the final Hugging Face-format checkpoint from continued pretraining of
the released post-trained Dream-org/DreamReasoner-8B model on the sealed
FineCode B64 dataset. The run used the any-order (empty dependency graph)
projected objective.
Training configuration
- Run:
dream-fc-ao-g64-m1-a1-16n-s24256-20260920-a1-r3 - Base revision:
ed62b1d2c82ccd234b05ed2463b4c0ee640f2068 - Source commit:
f8e89e543dee0d164e3faaee89599f4a0e2eb24f - Final optimizer step: 24,256
- Context length: 2,048
- Global batch: 64 sequences (microbatch 1, gradient accumulation 1)
- Hardware: 16 nodes x 4 GB300 GPUs
- Precision/distribution: BF16, FSDP full shard
- Learning rate: cosine decay from
5e-7to5e-8, warmup ratio 0.05 - Weight decay: 0.1
- Objective:
projected_eq73_v1 - Graph: empty
- Teacher parameters:
epsilon=0,rollin_epsilon=0,action_epsilon=0 - Terminal-EOS supervision: enabled
The complete resolved training configuration is included as
resolved_config.yaml.
Evaluation snapshot
Greedy B64/S64 confidence decoding at threshold 0.95 and seed 17:
| Benchmark | Generation cap | Pass@1 |
|---|---|---|
| HumanEval | 4,096 | 141/164 (85.98%) |
| MBPP sanitized | 4,096 | 214/257 (83.27%) |
| LiveCodeBench Pro | 4,096 | 106/706 (15.01%) |
Integrity
The original 14-file checkpoint payload (before this model card was added) has checkpoint fingerprint:
617bf0f99914c4fd094a3da3386896752d2c758b191c29b00bcbeacbaafc8581
The source training-state manifest has SHA-256:
c347708d3e5e8eaaa244673a86b71f198fabb263133255d223f16f5a3f47afb1
The weight file model.safetensors has SHA-256:
71167ba6b94d18062b071d5068b9ed682f0176725259ebaea8405fbff7a8593d
Loading
This repository contains custom DreamReasoner modeling code. Review it and use
trust_remote_code=True when loading:
import torch
from transformers import AutoModel, AutoTokenizer
repo_id = "zimplex/dllm-dreamreasoner-8b-finecode-any-order-b64-s24256-v1"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModel.from_pretrained(
repo_id,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
)
This is a research checkpoint. It has not received a separate safety or deployment evaluation.