Matched GPT-2 baseline

Shared reference for Generations 2 and 3. A plain transformer: the gradient-convention change that supersedes Gen 2 does not touch it, and the same checkpoint anchors both ladders.

One arm of Generation 2 of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering isolates one component by construction rather than by attribution.

This arm removes: the reference architecture.

The ladder

Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result.

Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget.

What each gap compares

Every value below was measured under this generation's detached-source convention. Two of them were once read as prices of a mechanism; those readings are withdrawn and marked.

Each row compares the two arms it names: the value is the right-hand arm's settled PPL minus the left-hand arm's, both shown. It is not measured against the matched GPT-2; that ratio is the last column of the ladder table above.

Why the conservative exchange field did so badly — resolved

The rows for the arm attention_potential were this generation's least expected result: 27.4% worse than the arm attention, and 20.8% worse than no exchange field at all. The cause was the training signal, not conservativity.

• To keep its force a gradient, the conservative field held its sources and routing fixed, and it did so by detaching them. Detaching changes no forward value, but the field then sent no gradient into any hidden state.
• With the identical forward force written out over live tensors, the same arm settled at 61.11, below the arm attention's 63.51, with a 0.8% clip-hit rate against 29.2% (single seed).
• The explanation previously given here, expressivity (the gradient ties the output projection to the query projection), is withdrawn as the cause of the gap.

The same detach applies to Vφ and ξ in every arm of this collection, which is why the collection is superseded rather than corrected in place.

What the ladder is measuring

The models below integrate a damped mechanical system in semantic space. With the register mechanism on, the equation of motion carries a non-conservative term and is Lagrangian only in the Lagrange–d'Alembert sense:

$$m \ddot{h} = -\nabla V_\theta(h) - \gamma m \dot{h} + F_{\mathrm{rc}}(h, r) + F_\phi(h)$$

\(V_\theta\) is the pointwise scalar potential, \(V_\phi\) the pairwise one, and \(F_{\mathrm{rc}}\) the reverse channel through which the register bank acts on the token state. \(F_{\mathrm{rc}}\) is not the gradient of anything in \(h\), so it is the term that makes the trajectory non-geodesic.

The damping \(-\gamma m \dot{h}\) is not neutral either. It points along the velocity, but it does change the path, not only the speed: a trajectory bends with curvature \(|F_\perp| / (m v^2)\), so slowing down lets the potential bend it more sharply. A damped trajectory is therefore not a geodesic of a fixed Jacobi metric, which needs conserved energy. What these cards call a damped geodesic step is one integrated step of \(m \ddot{h} = -\nabla V - \gamma m \dot{h}\): the geodesic equation of the layer-dependent metric \(\Omega^2_\ell = 2 T_\ell m\) with a friction term added, which is the object E1 measures.

Setting \(F_{\mathrm{rc}} = 0\) leaves every force a gradient, so each layer step is a damped geodesic step in that sense, in the total potential with the context frozen for the layer, followed by the LayerNorm projection. It holds per step only: the trained models read MAPS, not FLOW, under refinement (6b-7), so the steps are not samples of one continuous curve. One arm in this collection is fully conservative in this sense: the arm conservative-only, with \(V = V_\theta + V_\phi\) and the register-to-token path off. The multi-ξ SPLM, the arm splm-multixi (planned for Gen 3, not yet published), with \(V = V_\theta\) alone, is the other. This generation measured the arm conservative-only at +31.3% against the arm no-exchange; that gap is withdrawn as a price, because its only inter-token channels had their sources detached. The arm attention_potential is not fully conservative: its exchange field enters as a potential, but its reverse channel is still on, so \(F_{\mathrm{rc}} \neq 0\).

This model

Protocol

• Corpus OpenWebText (GPT-2 BPE, 50,257); 32,500 steps × 32 × 512 = 532,480,000 tokens.
• Settled PPL is the mean of the last three 500-step evals at block 512 — not the best checkpoint.
• Shared across all arms: tokenizer, corpus, validation batches, width d=384, block 512, effective batch 32, token budget (532,480,000 tokens, identical on every arm), WSD schedule (warmup 5%, stable to 65%, decay to 100%, floor 6e-05).
• Not shared: depth (GPT-2 L=8 vs Fock L=2) and the learning rate of the GPT-2 arm (6e-04, separately tuned).
• Full protocol, pre-registered bands and run health: Depth_Ladder_and_Matched_Baseline_Protocol.md §5.4.

Reproduction

Set in Cell 0 of colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb and run Cell 0 → 1 → 1b → 1c → 1d → 2 → 3 → 4 → 5 → 5b → 6:

separate notebook: colab_matched_gpt2_baseline_openwebtext.ipynb

Training artefacts were written to semsimula_matched_gpt2_baseline_owt/.

Companion notes

Every number on this card is derived in one of these. They are the working notes, not a paper: pre-registrations sit next to the results that scored them, and withdrawn claims are marked withdrawn rather than deleted.

Theoretical foundations of the ladder

The design and theory behind every arm in this collection: the integrator they share, the potential they share, and the mechanisms the ladder compares. The same list appears on every card.

Files

The code, and why this subset

code/ mirrors the layout of the companion repository, so the notebook's own sys.path lines resolve unchanged.

Caveats

• This is a reference, not a tuned transformer. It exists to give the ladder a scale, and it is the only arm whose learning rate was tuned separately (6e-04). Read it as "what a conventional architecture reaches on this budget", not as a claim about GPT-2.
• Depth is not matched. L=8 here against L=2 on every Fock arm. The ratio column elsewhere in the collection is for orientation, not a like-for-like efficiency claim.
• The second checkpoint is not the same kind as the other arms'. The Fock arms each ship step 500, essentially untrained. This arm has no step-500 checkpoint; its second file is step 25,000 at PPL 52.87, already within 6% of the endpoint. Do not read a cross-arm comparison of "first" checkpoints as like-for-like.
• gpt2_baseline_summary.json carries a stale cross-reference. Its fock_* fields and ratio_fock_to_gpt2_settled are not a ladder arm: the file was written before any L=2 arm existed and those numbers come from a pre-ladder L=8 run read mid-schedule. The GPT-2 fields in it are correct. NOTE_summary_json_fock_fields.md sits beside it.
• OpenWebText only.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Non-Conservative Memory, and the Shared-Potential Separator},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v22 (Oct 5, 2026).
               Companion code repository (DOI 10.5281/zenodo.23152203):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-10-05.