14B ReadAll stage-2 RL: local step15 / cumulative step30
这是用户指定的本轮续训 local step15，父模型为已发布的 ReadAll step15，因此累计标记为 global step30。本断点已完成60次optimizer更新、960道训练题；完整计划为2560题、一轮40个outer step。
Root files contain the complete BF16 inference model (Qwen2.5-14B, 579 parameter tensors). training_checkpoint/ contains the exact FP32 four-rank FSDP model, Adam optimizer, scheduler, CPU/CUDA/NumPy/Python RNG, generation RNG and driver/data-order state. It is a full-parameter ReadAll RL checkpoint.
Load model weights
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "Xirui1208/readall-readtwice-14b-stage2-localstep15-globalstep30-20260926"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype=torch.bfloat16, device_map="auto"
)
Use the ReadTwice SKIM → sequential complete-memory UPDATE → FINAL protocol from training_code.tar.gz for long-context document QA. A direct single-prompt call does not reproduce the reported ReadTwice workflow. Generation budgets are512/1024/1024 and the native context window is32768. Long documents are chunked losslessly into5000-token sections and processed serially with bounded memory.
Training and continuation
- Parent model: ReadAll14B step15, pinned revision
ebdc012fadb9a18eb74cc8adb5822b4bc42c7d66. - Data/reward: DocQA + HQA2560, pinned revision
fcd8d755f07710413b58665c2445290ede092e17. - User explicitly disabled online group filtering:64 raw questions per outer step,8 independent trajectories/question,4 optimizer updates of16 questions. All-correct/all-wrong groups remain included; their centered reward advantage is zero, while KL still applies.
- LR1e-6, constant schedule, AdamW weight decay0.01, grad clip1, PPO clip0.2, KL0.001 against initial SFT A230. Phase weights SKIM/UPDATE/FINAL are0.10/0.45/0.45. Full configuration is in
training_config.yaml. - Resume from
training_checkpoint/, not from the rounded BF16 inference export, to preserve the optimizer and FP32 master parameters. The saved cursor is960, the optimizer count is60, and the next local outer step is16. Preserve the saved shuffle order and all four rank states. - The included runtime targets Python3.12, PyTorch2.6.0+cu124, Transformers4.50.3, vLLM0.8.2 and Ray2.55.1. It uses4 GPUs with CPU parameter/optimizer offload.
training_code.tar.gz contains the exact training entrypoints, patched runtime, protocol and rewards. Source paths in config/driver are from the original machine. On a new machine, materialize the model/reference/dataset locally and update path fields consistently in both config and driver metadata; preserve all non-path settings and saved state. Only publish checkpoints/latest.json locally after all shards have downloaded and verified against checksums.json.
Validation
All exported tensors were reconstructed from the four FP32 source shards, cast to BF16 and compared element-by-element with the written safetensors. Source model hashes were checked before and after export. checksums.json covers all uploaded payload files except itself.
This local-step15 checkpoint has not been benchmark-evaluated. The earlier local-step10 HQA results belong to a different checkpoint and are not attributed to this model. Training continued separately after saving this immutable checkpoint.