Fock-PARFLM, no exchange field, L=4, live gradients (Gen 3)

Generation 3: current. Every gradient path that applies is live. This model is the Fock-PARFLM no-exchange arm at L=4, with exactly the forward pass of its Gen 2 twin (run 4, the same model with detached sources, not published). The only difference is that the loss gradient now reaches earlier tokens through Vφ (as pair sources) and through ξ (the context channels). In Gen 2 both detached their sources, so earlier tokens were never trained as context. The Gen 3 collection holds the live-gradient ladder as its arms land.

Result

• Settled is the mean of the last three evals (50.05, 50.77, 49.48); best and final 49.48 at step 32,500. The run was still improving at the end.
• 30.2% lower than its Gen 2 twin from the gradient convention alone, and 13.3% below the L=2 Gen 3 model.
• Parity with the matched GPT-2 at equal tokens, within eval noise: the gap is 0.29 PPL, against a step-to-step eval bounce of 1–3 PPL. The parameter count is not matched (2.3×; about 19M of the difference is the untied output head), the baseline is untuned and twice as deep (L=8), and this is one seed. A parameter-matched baseline has not been run.
• The depth inversion is gone. With detached sources, L=4 (71.75) was worse than L=2 (66.98). With live gradients this L=4 model is below every L=2 model in the programme.
• At step 3,000: 119.77 against run 4's 138.94 (−13.8%); a move, and a hit on the pre-registered band 112–135.
• Pre-registration: point 57, band 52–63, frozen at step 3,000 before the extension; actual 50.10, a miss on the good side. Below run 4 (called near-certain) and below the L=2 Gen 3 model's 57.76 (called even odds): both yes.
• Run health: 0 watchdog, 0 spikes, 0.5% clip-hit (3 of 650 logged steps, max norm 1.96); trained across two Colab sessions, resumed from the step-27,000 checkpoint with optimizer state.
• Depth at matched mechanism and convention: the L=2 Fock-PARFLM live model settled at 53.12, so this model is 5.7% better at L=4, at 1.9× the per-step cost (3.12 against 1.64 s/step).

This model sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — the same budget, corpus, tokenizer and validation batches as every Gen 2 arm and the matched GPT-2 in the table above. No model is trained longer than another, so a gap in the table is a gap between models and never a difference in budget.

What changed from Gen 2

Both channels keep their forward force exactly: the force on a token is still the gradient of Vθ(ξ, h) + Vφ in that token's own coordinates, with the context held fixed, plus the reverse channel's register-to-token force. What changes is the backward pass (vphi_grad_path='live', xi_grad_path='live' in model_parf_multixi.py):

• Vφ. Its sources and its score head's source input are live tensors. The force is taken with respect to an alias of the token's own state, so no reaction force appears and the forward value is bit-identical.
• ξ. The context channels are built from live states. The force is taken at a point downstream of them, so no path through ξ enters it.

Verified at L=4 before training, from initialisation through the notebook's own cells: forward pass bit-identical (max |Δlogit| = 0), and the gradient from a layer step into earlier tokens goes from exactly 0 at every layer to non-zero.

Causality

SCAF leak monitor CLEAN at all seven audits (5k to 30k, 32.5k; future perturbation exactly 0, leak tax ≤ 2e-4 nats; Tier A cosine deviation 0 at every layer, and from step 30,000 Tier B basin-crossing rate 0). Independent check on the final weights: future perturbation at five cut points exactly 0, batch independence exactly 0, prefix-only leak tax −1.1e-4 nats. The prefix-only logits differ by up to 0.07 from the full-sequence ones: shortening the input changes the reduction order, and the cumulative register readout and four layers let that float difference grow. It carries no information, since any dependence on future content would have broken the bit-exact future-perturbation test, and the leak tax shows no unearned advantage. Output: causality_check_output.txt.

Geometry and forecastability

Terms. The three gates of the flow-or-maps probe (6b-7, CG2) ask whether the layer stack behaves like one continuous flow, or like a sequence of separate maps that only work at the step size they were trained at.

• Gate 1, inertia. The velocity entering each layer is reset to zero. The rise in perplexity is what the momentum is worth to the trained model.
• Gate 2, extension. More layers at the trained step Δt, so the trajectory is integrated past the trained time T = L·Δt.
• Gate 3, refinement. The same total time T, cut into more and smaller steps (Δt = T/N, N > L). A flow reads about the same (FLOW); separate maps degrade (MAPS). Refinement invariance is the precondition for reading the trajectory as a geodesic of the learned metric, rather than of the step map at one step size.

Defined and analysed in the book, Semantic Simulation (Zenodo, latest version): §8.9 Settling, refinement and depth extension, with Proposition 44 (phase-sampled dissipation: the θ/sin θ amplification, θ = ω·Δt) and Proposition 45 (the exact damped-mode flow is refinement-invariant); Remark 62 Refinement invariance is a testable precondition; and §27.15, the replay instrument and the CG experiments.

The register-to-token force is not the gradient of anything in h, so this model's layer step is not a geodesic step of any potential on the tokens; the replay instrument measures how far from one it is, and which term carries the difference. Against the L=2 Gen 3 model:

• Vφ is inert in this model. With the register path available, the pair potential carries almost none of the step (−0.0035), where in the conservative-only model it carries most of the departure from the Vθ geodesic. The register path and Vφ behave as substitutes. The L=2 Fock-PARFLM live model reads the same (−0.0002). These are attributions, not a trained-without result; the factorial runs that test it (the same model without Vφ) are planned.
• Inertia is worth more here than in any other arm: resetting the velocity costs 55% in perplexity.
• Per step only. Gate 3 still reads MAPS; extra layers at the trained dt degrade more gently than in any L=2 arm, but they do degrade.
• Far more refinement-ready than the L=2 Fock-PARFLM live model. Cutting the same time into 1.5× the steps costs +216% here, against +1,274% at L=2. A mechanism consistent with the stiff-mode readings: the phase-sampled dissipation of a split step is amplified by θ/sin θ (θ = ω·dt), most sharply near θ = π. This model trains its stiff modes at θ ≈ 2.35 (amplification 3.3), and refinement keeps them below π (θ ≈ 1.57). The L=2 model trains them at θ ≈ 3.40, just past π (amplification −13.3), and refinement carries them back across π (θ ≈ 2.27), flipping the sign of the term the network learned to rely on. Not the whole story: the L=2 conservative-only model also crosses π and fails much less (+143%), so the register path amplifies the effect. One seed each; the prediction that follows (refinement cost falls as the trained θ moves below π) is pre-registered for the settling and refinement experiments.
• Stiff, and integrated exactly. ω·dt ≈ 2.35, past the explicit-integrator wall for 69% of readings; the closed-form propagator integrates those modes exactly.

Forecastability against the matched GPT-2 (6b-10), the first clean test. At L=2 the test cannot be read, because the first step only projects the embedding onto the LayerNorm sphere. At L=4 the later steps are clean: the radial fraction is 0.86 at layer 1 and 0.00 at layers 2 and 3. Scored on layers 2–3 by a rule fixed before the cell was run:

By that rule the trajectory is forecastable: consecutive steps keep their direction, and the integrator's own velocity predicts the next state, better than the position sequence alone. The effect is concentrated at layer 2; layer 3, the large output step, is close to the stay-put null. The paired control, the same cell on the Gen 2 twin, has not run, so the result is not yet attributable to the live gradients.

Reproduction

Set in Cell 0 of colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb and run Cell 0 → 1 → 1b → 1c → 1d → 2 → 3 → 4 → 5 → 5b → 6:

LADDER_MECHANISM='none', REVERSE_CHANNEL=True, LADDER_L=4, LADDER_LR=1.2e-3,
VPHI_GRAD_PATH='live', XI_GRAD_PATH='live'   # tag components 'vplive', 'xilive'; dt = T/L = 2

The run was a 3,000-step probe (PROBE_MAX_STEPS = 3_000), extended in place to 32,500 steps on the same schedule, across two Colab sessions: the second resumed from the step-27,000 checkpoint with its optimizer state. Cell 5b prints a SOURCE-GRADIENT PROBE banner for this setting.

Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_vplive_xilive_L4probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt2_lr0p0012_noattn/.

Glossary

Theoretical foundations of the ladder

The design and theory behind every arm of the ladder, in Gen 2 and Gen 3 alike: the integrator they share, the potential they share, and the mechanisms the ladder compares. The same list appears on every ladder card.

Files

Companion notes

Caveats

• One seed.
• Parameters are not matched to the GPT-2 baseline (76.8M against 33.7M), the baseline is untuned, and its depth is twice this model's. The ratio is for orientation, and parity is stated only at equal tokens.
• The conservative family is not tuned either: no learning-rate or width search for this architecture, Vφ still in the explicit kick, and constant damping. The number is a point on a developing design, not its ceiling.
• Depth rests on one seed per depth: the margin over the L=2 Fock-PARFLM live model is larger than either model's last-three spread, but has no seed variance behind it.
• OpenWebText only.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Non-Conservative Memory, and the Shared-Potential Separator},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v22 (Oct 5, 2026).
               Companion code repository (DOI 10.5281/zenodo.23152203):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-10-06.