Fock-PARFLM, no exchange field, L=2, live gradients (Gen 3)

Generation 3: current. Every gradient path that applies is live. This model is the Fock-PARFLM no-exchange arm at L=2, with exactly the forward pass of its Gen 2 twin, the arm no-exchange. The only difference is that the loss gradient now reaches earlier tokens through Vφ (as pair sources) and through ξ (the context channels). In Gen 2 both detached their sources, so earlier tokens were never trained as context. The Gen 3 collection holds the live-gradient ladder as its arms land.

Result

• Settled is the mean of the last three evals (53.01, 53.29, 53.07); best 51.27 at step 31,000, final 53.07.
• 20.7% lower than its Gen 2 twin from the gradient convention alone.
• What the register mechanism adds when trained: 8.0% against the conservative-only model at the same depth and convention.
• What depth adds, at matched mechanism and convention: the L=4 model is 5.7% better. That margin is larger than either model's last-three spread. It costs 1.9× per step (3.12 against 1.64 s/step). This is the first clean depth comparison in the programme.
• At step 3,000: 123.59 against the Gen 2 twin's 140.61 at the same step (−12.1%); a move, against the pre-registered criterion of ≤ 137.1.
• Pre-registration: point 53, band 50–57, frozen before the run; actual 53.12, a hit on the point. Below the L=2 conservative-only live model's 57.76 (called at 80%) and above the L=4 live model's 50.10 (called at 75%): both yes. Vφ inert, |share| < 0.05 (called): −0.0002, a hit. ω·dt median in [3.3, 4.2]: 3.40, a hit.
• Run health: 0 watchdog, 0 spikes, 0.2% clip-hit (1 of 650 logged steps, max norm 1.01); one Colab session, fresh run, 1.64 s/step.

This model sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — the same budget, corpus, tokenizer and validation batches as every Gen 2 arm and the matched GPT-2 in the table above. No model is trained longer than another, so a gap in the table is a gap between models and never a difference in budget.

What changed from Gen 2

Both channels keep their forward force exactly: the force on a token is still the gradient of Vθ(ξ, h) + Vφ in that token's own coordinates, with the context held fixed, plus the reverse channel's register-to-token force. What changes is the backward pass (vphi_grad_path='live', xi_grad_path='live' in model_parf_multixi.py):

• Vφ. Its sources and its score head's source input are live tensors. The force is taken with respect to an alias of the token's own state, so no reaction force appears and the forward value is bit-identical.
• ξ. The context channels are built from live states. The force is taken at a point downstream of them, so no path through ξ enters it.

The parameter count is identical to the Gen 2 twin's. The switches were verified on this configuration before training: forward pass bit-identical (max |Δlogit| = 0), and the gradient from a layer step into earlier tokens goes from exactly 0 to non-zero.

Causality

SCAF leak monitor CLEAN at all seven audits (5k to 30k, 32.5k; future perturbation exactly 0, leak tax ≤ 1.2e-4 nats; Tier A cosine deviation 0 and Tier B basin-crossing rate 0 at every audit). The step-32,500 audit printed its honest and standard perplexities as nan; its future-perturbation test read exactly 0. Independent check on the final weights: future perturbation at five cut points exactly 0, batch independence exactly 0, prefix-only leak tax +1.5e-4 nats. The prefix-only logits differ by up to 0.04 from the full-sequence ones: shortening the input changes the reduction order. It carries no information, since any dependence on future content would have broken the bit-exact future-perturbation test. Output: causality_check_output.txt.

Geometry

Terms. The three gates of the flow-or-maps probe (6b-7, CG2) ask whether the layer stack behaves like one continuous flow, or like a sequence of separate maps that only work at the step size they were trained at.

• Gate 1, inertia. The velocity entering each layer is reset to zero. The rise in perplexity is what the momentum is worth to the trained model.
• Gate 2, extension. More layers at the trained step Δt, so the trajectory is integrated past the trained time T = L·Δt.
• Gate 3, refinement. The same total time T, cut into more and smaller steps (Δt = T/N, N > L). A flow reads about the same (FLOW); separate maps degrade (MAPS). Refinement invariance is the precondition for reading the trajectory as a geodesic of the learned metric, rather than of the step map at one step size.

Defined and analysed in the book, Semantic Simulation (Zenodo, latest version): §8.9 Settling, refinement and depth extension, with Proposition 44 (phase-sampled dissipation: the θ/sin θ amplification, θ = ω·Δt) and Proposition 45 (the exact damped-mode flow is refinement-invariant); Remark 62 Refinement invariance is a testable precondition; and §27.15, the replay instrument and the CG experiments.

The register-to-token force is not the gradient of anything in h, so this model's layer step is not a geodesic step of any potential on the tokens; the replay instrument measures how far from one it is, and which term carries the difference.

• Vφ is inert here, as at L=4. With the register path available, the pair potential carries essentially none of the step (−0.0002, the pre-registered |share| < 0.05), where in the conservative-only model it carries most of the departure from the Vθ geodesic. The register path and Vφ behave as substitutes, now at two depths. These are attributions, not a trained-without result; the factorial runs that test it (the same model without Vφ) are planned.
• Refinement fails nine times worse than in the conservative-only model, at the same reliance on velocity. Gate 1 is +33% here against +34% there; Gate 3 is +1,274% against +143%. In the four L=2 models measured before, the two were perfectly rank-ordered. This one breaks the order: the non-conservative register path drives refinement failure independently of momentum.
• The L=4 model is far more refinement-ready (+216% at 1.5× the steps, against +1,274% here), and the stiff-mode readings suggest why. The phase-sampled dissipation of a split step is amplified by θ/sin θ (θ = ω·dt), most sharply near θ = π. This model trains its stiff modes at θ ≈ 3.40, just past π (amplification −13.3), and refinement carries them back across π (θ ≈ 2.27), flipping the sign of the term the network learned to rely on. The L=4 model trains them at θ ≈ 2.35 (amplification 3.3) and stays below π when refined (θ ≈ 1.57). Not the whole story: the conservative-only model also crosses π and fails much less (+143%), so the register path amplifies the effect. One seed each; the prediction that follows (refinement cost falls as the trained θ moves below π) is pre-registered for the settling and refinement experiments.
• Switching the register path off costs +300% at inference (51.17 → 204.84, 6b-8), yet the model trained without it is only 8.7% worse (57.76 against 53.12). An inference-time ablation overstates a component a model was trained to use, here by a factor of about 35. Freezing the register bank at initialisation costs +24%, so the accumulation itself does real work.
• Uniform forcing. Every token at the last layer is substantially redirected from the geodesic step (100% above 0.75); there is no near-geodesic population.
• Stiff, and integrated exactly. ω·dt ≈ 3.40, past the explicit-integrator wall for 99.6% of readings; the closed-form propagator integrates those modes exactly.

Forecastability (6b-10) cannot be read at L=2: the first step only projects the embedding onto the LayerNorm sphere, which leaves one dynamical step. The L=4 model gives the first clean test.

Reproduction

Set in Cell 0 of colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb and run Cell 0 → 1 → 1b → 1c → 1d → 2 → 3 → 4 → 5 → 5b → 6:

LADDER_MECHANISM='none', REVERSE_CHANNEL=True, LADDER_L=2, LADDER_LR=1.2e-3,
VPHI_GRAD_PATH='live', XI_GRAD_PATH='live'   # tag components 'vplive', 'xilive'; dt = T/L = 4

One Colab session, a fresh run of 32,500 steps (PROBE_MAX_STEPS = None). Cell 5b prints a SOURCE-GRADIENT PROBE banner for this setting.

Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_vplive_xilive_L2probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt4_lr0p0012_noattn/.

Glossary

Theoretical foundations of the ladder

The design and theory behind every arm of the ladder, in Gen 2 and Gen 3 alike: the integrator they share, the potential they share, and the mechanisms the ladder compares. The same list appears on every ladder card.

Files

Companion notes

Caveats

• One seed. The depth and register-mechanism margins have no seed variance behind them.
• Parameters are not matched to the GPT-2 baseline (76.8M against 33.7M), the baseline is untuned, and its depth is four times this model's. The ratio is for orientation only.
• The conservative family is not tuned either: no learning-rate or width search for this architecture, Vφ still in the explicit kick, and constant damping. The number is a point on a developing design, not its ceiling.
• OpenWebText only.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Non-Conservative Memory, and the Shared-Potential Separator},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v22 (Oct 5, 2026).
               Companion code repository (DOI 10.5281/zenodo.23152203):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-10-06.