14B selective SFT — epoch 1, step 2944
This is the completed selective gate SFT checkpoint on Xirui1208/readall-readtwice-14b-rl-step15-20260925, revision ebdc012fadb9a18eb74cc8adb5822b4bc42c7d66.
The repository packages the full BF16 frozen ReadAll reader in the root and the trained selective LoRA + gate bias in gate_adapter/. Load the adapter with the gate implementation to obtain selective reading. Loading only the root model gives the frozen ReadAll model. Keep the reader weights frozen; merging the gate adapter into the reader changes the intended pipeline.
Completed training
One epoch: 19,456 training views from 16,384 distinct questions; 23,552 gate pages; 2,944 optimizer updates. SFT views include 16,384 short documents and 1,024 each at 56K, 112K and 224K. These SFT navigation outlines are the original verified 7B teacher data from the transfer dataset. The adapter and optimizer were initialized fresh on the 14B base.
Four-GPU DDP, two pages per GPU, global batch eight pages. LoRA rank 16 / alpha 32 on q/k/v/o projections; trainable gate bias; AdamW 5e-5, weight decay .01, gradient clipping 1. The loss preserves true evidence labels, balances positions and classes by source length, and gives each document equal weight.
The 704 calibration views come from 512 distinct questions; test questions do not select the checkpoint. Calibration document-normalized class-balanced BCE, averaged over four lengths and three permutations: 0.2344866580. This is a gate calibration metric, not a QA accuracy. No new 56K/112K/224K QA benchmark is claimed for this SFT adapter. base_provenance/evaluation.json concerns the frozen ReadAll base only.
The accompanying RL data uses new frozen-14B outlines. The user stopped preparation after 18,446 complete views, including only 14 training views at 224K. That cutoff applies to the subsequent RL dataset; this SFT checkpoint completed its original full epoch.
Download and use
from huggingface_hub import snapshot_download
snapshot_download('Xirui1208/selective-readtwice-14b-sft-epoch1-step2944-20260925', local_dir='selective_14b')
Use an authenticated HF account if the repository is private. Configure machine-local paths:
python selective_14b/code/configure.py
CUDA_VISIBLE_DEVICES=0 python selective_14b/code/start_reader.py
# In another shell, use a separate GPU for the gate:
CUDA_VISIBLE_DEVICES=1 python selective_14b/code/infer.py --context-file document.txt --question 'Your question' --threshold 0.5
The reader protocol matches this 14B base: 5,000-token source chunks; at most 24 sections/page and 8,192 gate prompt tokens; SKIM/UPDATE/FINAL budgets 512/1024/1024. A complete <KEEP_MEMORY> response preserves memory; otherwise the generated memory replaces it, including length-capped output. Native context is 32,768 tokens, with recurrent pages for longer documents.
gate_adapter/trainer_state.pt contains the final four-rank SFT optimizer/RNG state. Original training paths are retained as provenance; code/configure.py rewrites inference paths locally. release_manifest.json records SHA256 checksums for the full release. All adapter tensors are byte-identical to the finished local SFT checkpoint.