ReadAll / ReadTwice RL step36
This repository contains the exact merged BF16 actor checkpoint at global step 36
of readall_sft_hqa_rl_step18_pass8_20260919_r1.
It is the initialization checkpoint for the later September 21 RL run.
Four safetensors shards, the weight index, tokenizer, and original model/generation
configuration are included. See checkpoint_manifest.json for SHA-256 hashes.
Evaluation
Use greedy decoding for the primary comparison: temperature=0, top_p=1
(do_sample=False in Transformers). The stored generation_config.json is
preserved from the source checkpoint and contains sampling defaults; pass the
explicit greedy overrides when evaluating.
- HQA 224k: 128/128 examples, EM 57.81%, F1 72.15%, bidirectional SubEM 74.22%, zero recorded evaluation error events.
- LongBench QA: 1,000 examples across NarrativeQA, Qasper, HotpotQA, 2WikiMQA, and MuSiQue; greedy macro EM 32.20%, F1 44.27%.
These are fixed development-set results. They do not establish performance on
an unseen test set. The HQA 224k number requires the matching paged ReadTwice
pipeline: balanced page-local outlines, 5,000-token chunks, 24 sections per page,
serial memory updates with <KEEP_MEMORY>, UPDATE cap 1,024, and seed 801.
The model's native context is 32,768 tokens; 224k is processed recurrently,
not passed to a single model call. Direct full-context chat is a different setup.
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "Xirui1208/readall-readtwice-step36-20260920"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
repo_id, torch_dtype="bfloat16", device_map="auto"
)
# For generation, explicitly pass do_sample=False for greedy decoding.
This repository is public and can be downloaded without authentication.