Fock-PARFLM with a conservative exchange field, L=2, everything live (Gen 3)

Generation 3: current. Every gradient path that applies is live. This model is the Fock-PARFLM attention_potential arm at L=2: a conservative exchange field (an attention-like interaction written as a potential) on top of the no-exchange model, with exactly the forward pass of its Gen 2 twin, the arm attention_potential. The loss gradient now reaches earlier tokens through Vφ, through ξ, and through the exchange field's sources and routing. In Gen 2 all three were detached. The Gen 3 collection holds the live-gradient ladder as its arms land.

Result

• Settled is the mean of the last three evals (54.17, 54.35, 54.12); best 52.44 at step 31,000, final 54.12.
• 33.0% lower than its Gen 2 twin from the gradient convention alone, and 11.3% below its parent probe, in which only the exchange field's gradient was live.
• What the conservative exchange field adds, as trained: nothing. This model is 2.1% worse than the same model without the field. It led that model by 5–7 PPL between steps 3,000 and 8,000, and lost the lead from about step 10,000 as the field's gradient grew. A hardened version with QK-normalised routing has since been trained: it settled at 52.90, 2.4% better than this model but only 0.4% better than the model without the field, within eval noise, so the routing was the problem here and the field still adds nothing measurable once fixed.
• The price of conservativity is not measured here. That needs the non-conservative attention arm retrained with live gradients.
• At step 3,000: 118.14 against its parent probe's 133.74 (−11.7%) and the L=2 no-exchange live model's 123.59 at the same step.
• Pre-registration: point 51, band 46–56, frozen before the run; actual 54.21, in the band. The key line, beating the L=2 no-exchange live model (53.12) by more than 2% (about 52.1 or lower): no, it is 2.1% worse. ω·dt median in the L=2 band [3.3, 4.2]: 2.07, a miss.
• Run health: 0 watchdog, 0 spikes, 1.4% clip-hit (9 of 650 logged steps, max norm 1.14, all between steps 20,000 and 27,500); the global gradient norm rose from 0.43 to 0.91 through the constant-learning-rate phase and fell to 0.75 in the decay; one Colab session, fresh run, 1.68 s/step.

This model sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — the same budget, corpus, tokenizer and validation batches as every Gen 2 arm and the matched GPT-2 in the table above. No model is trained longer than another, so a gap in the table is a gap between models and never a difference in budget.

What changed from Gen 2

All three channels keep their forward force exactly. What changes is the backward pass (relax_grad_path, vphi_grad_path and xi_grad_path all 'live' in model_parf_multixi.py):

• The exchange field. Its conservative force, the gradient of the attention potential with the routing held fixed, is written out over live tensors, so the loss reaches the field's queries, keys and values. In Gen 2 the field sent no gradient into any hidden state.
• Vφ. Its sources and its score head's source input are live tensors.
• ξ. The context channels are built from live states.

The parameter count is identical to the Gen 2 twin's. The forward pass of this configuration was verified bit-identical (logits and every parameter gradient) to the committed code before and during the run.

Causality

SCAF leak monitor CLEAN at all seven audits (5k to 30k, 32.5k; future perturbation exactly 0, leak tax ≤ 3.3e-4 nats; Tier A cosine deviation 0 and Tier B basin-crossing rate 0 at every audit). The step-32,500 audit printed its honest and standard perplexities as nan; its future-perturbation test read exactly 0. Independent check on the final weights: future perturbation at five cut points exactly 0, batch independence exactly 0, prefix-only leak tax −9.5e-5 nats. The prefix-only logits differ by up to 0.06 from the full-sequence ones: shortening the input changes the reduction order. It carries no information, since any dependence on future content would have broken the bit-exact future-perturbation test. Output: causality_check_output.txt.

The exchange field: load-bearing, but substituting

• Switching the field off costs +56% (52.35 → 81.81; half strength, +8%). The model has reorganised around it, yet the same model trained without it is 2.1% better. The field substitutes for work the model otherwise does through its registers and Vφ, rather than adding to it: the clearest case in this programme of an inference-time ablation overstating a component's trained value.
• Its routing weights grew without bound. The routing is unnormalised (scores q·k/√d_k, with no QK normalisation and no per-group gradient clip), so the projections grew 5–6× and the attainable logit scale about 28×. The routing sharpened only moderately and did not collapse. The field's gradient became most of the model's gradient through the constant-learning-rate phase. The hardened arm, with QK-normalised routing and a clamped logit scale as already used in the creation gate, bounds the logits; see the result table.
• The register path still dominates the step. Its increment is 1.16× the conservative step at the first layer and 2.75× at the output layer, so the field dominates the conservative part, the smaller share of the step.

Geometry

Terms. The three gates of the flow-or-maps probe (6b-7, CG2) ask whether the layer stack behaves like one continuous flow, or like a sequence of separate maps that only work at the step size they were trained at.

• Gate 1, inertia. The velocity entering each layer is reset to zero. The rise in perplexity is what the momentum is worth to the trained model.
• Gate 2, extension. More layers at the trained step Δt, so the trajectory is integrated past the trained time T = L·Δt.
• Gate 3, refinement. The same total time T, cut into more and smaller steps (Δt = T/N, N > L). A flow reads about the same (FLOW); separate maps degrade (MAPS). Refinement invariance is the precondition for reading the trajectory as a geodesic of the learned metric, rather than of the step map at one step size.

Defined and analysed in the book, Semantic Simulation (Zenodo, latest version): §8.9 Settling, refinement and depth extension, with Proposition 44 (phase-sampled dissipation: the θ/sin θ amplification, θ = ω·Δt) and Proposition 45 (the exact damped-mode flow is refinement-invariant); Remark 62 Refinement invariance is a testable precondition; and §27.15, the replay instrument and the CG experiments.

• Extension fails far worse than without the field: 1.5× the steps at the trained step size costs +325%, against +51%.
• Refinement fails as badly as without it, although the stiff modes here train below π (θ ≈ 2.07, 3.00 at the output layer) and do not cross π when the step is refined. The π-crossing account of the L=4 model's refinement readiness therefore does not explain this model: the register path and the field drive refinement failure in their own right.
• The field takes stiffness off Vθ: ω·dt ≈ 2.07, against 3.40 without it.

Forecastability (6b-10) cannot be read at L=2: the first step only projects the embedding onto the LayerNorm sphere, which leaves one dynamical step.

Refinement readiness: what we are studying

A model is refinement-ready if cutting the same integration time into more, smaller steps leaves its output about the same (Gate 3 above). That is the precondition for reading its layer stack as a continuous flow rather than a fixed sequence of maps. This pair is the sharpest contrast in the programme. This model and the hardened model differ only in the exchange field's routing hardening, yet at 1.5× the steps:

Ruled out so far, each by a pre-registered test:

• the stiff modes crossing π under refinement (the hardened model's still cross);
• the size of the register push (the hardened model's is 7.8× the state, as large as its siblings');
• the exchange field itself (the no-exchange model fails just as badly);
• the repeated register bookkeeping alone (holding it back on the no-exchange model recovers only about 13% of the penalty).

The hypothesis under test. A Fock model is refinement-ready when its register state is accumulated: a memory built up over the layers, so more steps add more of the same and the update behaves like a flow. It fails when its register state is reset and rewritten at each layer: refinement then inserts rewrite events the model never trained with, and the update behaves like a sequence of separate maps. The conservative-only model, which has no registers, refines well (+143%).

Evidence so far (2026-10-06):

• The no-exchange and unhardened models empty nearly every register at the first layer and refill it from the current input. The hardened model empties about half and keeps the rest, retaining about 90× more of its initial register state.
• Holding the per-step register updates back under refinement helps the no-exchange model slightly but breaks the hardened one. In the hardened model those updates are part of why it refines well: its register state integrates smoothly with more steps.
• Not yet established: every model here is one seed. The decisive test, a second seed of the hardened and of the no-exchange model, is queued. Until it runs, the routing hardening is the cause only within this pair.

Pre-registration and results: Depth_Ladder_and_Matched_Baseline_Protocol.md, §5.18.

Reproduction

Set in Cell 0 of colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb and run Cell 0 → 1 → 1b → 1c → 1d → 2 → 3 → 4 → 5 → 5b → 6:

LADDER_MECHANISM='attention_potential', REVERSE_CHANNEL=True, LADDER_L=2, LADDER_LR=1.2e-3,
RELAX_GRAD_PATH='live', VPHI_GRAD_PATH='live', XI_GRAD_PATH='live'   # tag 'rglive', 'vplive', 'xilive'; dt = T/L = 4

One Colab session, a fresh run of 32,500 steps (PROBE_MAX_STEPS = None). Cell 5b prints a GRADIENT-PATH PROBE banner for the exchange field and a SOURCE-GRADIENT PROBE banner for Vφ and ξ.

Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_rglive_vplive_xilive_L2probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt4_lr0p0012_attnpot/.

Glossary

Theoretical foundations of the ladder

The design and theory behind every arm of the ladder, in Gen 2 and Gen 3 alike: the integrator they share, the potential they share, and the mechanisms the ladder compares. The same list appears on every ladder card.

Files

Companion notes

Caveats

• One seed. The 2.1% gap to the model without the field has no seed variance behind it.
• The exchange field is untuned under live gradients: no routing normalisation and no gradient clip of its own. This result is for the field as trained, not its ceiling.
• Parameters are not matched to the GPT-2 baseline (77.4M against 33.7M), the baseline is untuned, and its depth is four times this model's.
• OpenWebText only.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Non-Conservative Memory, and the Shared-Potential Separator},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v22 (Oct 5, 2026).
               Companion code repository (DOI 10.5281/zenodo.23152203):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-10-06.