Fock-PARFLM, exchange field as a potential (arm 'attention_potential')

Generation 2 of 3: superseded, kept as the paired control. Every model in this collection was trained with the sources of Vφ and ξ detached (for the arm attention_potential, those of its exchange field too). That keeps each force causal, but no loss gradient ever reached an earlier token through those channels. Retraining with the same forward pass and live source gradients changes the results substantially: the arm attention_potential goes from 80.90 to 61.11 (the gradient-path probe; 54.21 with every path live), the arm no-exchange from 66.98 to 53.12, and the arm conservative-only from 87.93 to 57.76. The perplexities here are correct measurements of the detached convention; their readings as the price of the Fock mechanism or of conservativity are withdrawn. Current models: the Gen 3 collection. Details: Gradient_Starvation_Investigation.md.

One arm of Generation 2 of the SemSimula mechanism ladder — OpenWebText, d=384, L=2: five models that differ by exactly one mechanism each, so that every gap in the ordering isolates one component by construction rather than by attribution.

This arm removes: the exchange field's non-conservative part — the same ξ-routed attention, entering as a scalar potential so its force is a gradient; the reverse channel is still on, and is not the gradient of anything, so the model as a whole remains non-conservative.

The ladder

Italic entries are pre-registered predictions, recorded before the run. The ordering is itself the prediction; any inversion is a result.

Every arm above sees exactly 532,480,000 training tokens — 32,500 steps × 32 × 512 = 532,480,000 tokens — on the same corpus, tokenizer and validation batches. No arm is trained longer than another, so a gap in the table is a gap between mechanisms and never a difference in budget.

What each gap compares

Every value below was measured under this generation's detached-source convention. Two of them were once read as prices of a mechanism; those readings are withdrawn and marked.

Each row compares the two arms it names: the value is the right-hand arm's settled PPL minus the left-hand arm's, both shown. It is not measured against the matched GPT-2; that ratio is the last column of the ladder table above.

Why the conservative exchange field did so badly — resolved

The rows for the arm attention_potential were this generation's least expected result: 27.4% worse than the arm attention, and 20.8% worse than no exchange field at all. The cause was the training signal, not conservativity.

• To keep its force a gradient, the conservative field held its sources and routing fixed, and it did so by detaching them. Detaching changes no forward value, but the field then sent no gradient into any hidden state.
• With the identical forward force written out over live tensors, the same arm settled at 61.11, below the arm attention's 63.51, with a 0.8% clip-hit rate against 29.2% (single seed).
• The explanation previously given here, expressivity (the gradient ties the output projection to the query projection), is withdrawn as the cause of the gap.

The same detach applies to Vφ and ξ in every arm of this collection, which is why the collection is superseded rather than corrected in place.

What the ladder is measuring

The models below integrate a damped mechanical system in semantic space. With the register mechanism on, the equation of motion carries a non-conservative term and is Lagrangian only in the Lagrange–d'Alembert sense:

$$m \ddot{h} = -\nabla V_\theta(h) - \gamma m \dot{h} + F_{\mathrm{rc}}(h, r) + F_\phi(h)$$

\(V_\theta\) is the pointwise scalar potential, \(V_\phi\) the pairwise one, and \(F_{\mathrm{rc}}\) the reverse channel through which the register bank acts on the token state. \(F_{\mathrm{rc}}\) is not the gradient of anything in \(h\), so it is the term that makes the trajectory non-geodesic.

The damping \(-\gamma m \dot{h}\) is not neutral either. It points along the velocity, but it does change the path, not only the speed: a trajectory bends with curvature \(|F_\perp| / (m v^2)\), so slowing down lets the potential bend it more sharply. A damped trajectory is therefore not a geodesic of a fixed Jacobi metric, which needs conserved energy. What these cards call a damped geodesic step is one integrated step of \(m \ddot{h} = -\nabla V - \gamma m \dot{h}\): the geodesic equation of the layer-dependent metric \(\Omega^2_\ell = 2 T_\ell m\) with a friction term added, which is the object E1 measures.

Setting \(F_{\mathrm{rc}} = 0\) leaves every force a gradient, so each layer step is a damped geodesic step in that sense, in the total potential with the context frozen for the layer, followed by the LayerNorm projection. It holds per step only: the trained models read MAPS, not FLOW, under refinement (6b-7), so the steps are not samples of one continuous curve. One arm in this collection is fully conservative in this sense: the arm conservative-only, with \(V = V_\theta + V_\phi\) and the register-to-token path off. The multi-ξ SPLM, the arm splm-multixi (planned for Gen 3, not yet published), with \(V = V_\theta\) alone, is the other. This generation measured the arm conservative-only at +31.3% against the arm no-exchange; that gap is withdrawn as a price, because its only inter-token channels had their sources detached. The arm attention_potential is not fully conservative: its exchange field enters as a potential, but its reverse channel is still on, so \(F_{\mathrm{rc}} \neq 0\).

This model

Protocol

• Corpus OpenWebText (GPT-2 BPE, 50,257); 32,500 steps × 32 × 512 = 532,480,000 tokens.
• Settled PPL is the mean of the last three 500-step evals at block 512 — not the best checkpoint.
• Shared across all arms: tokenizer, corpus, validation batches, width d=384, block 512, effective batch 32, token budget (532,480,000 tokens, identical on every arm), WSD schedule (warmup 5%, stable to 65%, decay to 100%, floor 6e-05).
• Not shared: depth (GPT-2 L=8 vs Fock L=2) and the learning rate of the GPT-2 arm (6e-04, separately tuned).
• Full protocol, pre-registered bands and run health: Depth_Ladder_and_Matched_Baseline_Protocol.md §5.8.

Reproduction

Set in Cell 0 of colab_fock_cfc_baoab_lowrank_depth_ladder_openwebtext_d384.ipynb and run Cell 0 → 1 → 1b → 1c → 1d → 2 → 3 → 4 → 5 → 5b → 6:

LADDER_MECHANISM='attention_potential', REVERSE_CHANNEL=True, LADDER_L=2, LADDER_LR=1.2e-3

Training artefacts were written to semsimula_fock_cfc_baoab_owt_xi5long_topk16_dt32da16_mh4_aniso_dcvt5x8_vtjoint_cgqk_L2probe_ob_untied_wsd_e5c_plgate_rep0.05_fockreg0.005_g0.1_baoab_cfc_lowrank_idt4_lr0p0012_attnpot/.

Companion notes

Every number on this card is derived in one of these. They are the working notes, not a paper: pre-registrations sit next to the results that scored them, and withdrawn claims are marked withdrawn rather than deleted.

Theoretical foundations of the ladder

The design and theory behind every arm in this collection: the integrator they share, the potential they share, and the mechanisms the ladder compares. The same list appears on every card.

Files

The code, and why this subset

code/ mirrors the layout of the companion repository, so the notebook's own sys.path lines resolve unchanged.

The closure was computed by walking the imports, not chosen by hand, so the set is reproducible and minimal: drop any file and an import fails.

Caveats

• L=2 only. Every trajectory-level claim is scoped to two layers; the L=4 rung is queued.
• OpenWebText only. How the work divides between the conservative potential and the register mechanism may be a property of the corpus; a TinyStories replication is designed but not run.
• Depth is not matched between the GPT-2 arm (L=8) and the Fock arms (L=2). The ratio column is reported for orientation, not as a like-for-like efficiency claim.
• Ablations overstate. Removing this mechanism from a trained model at inference indicated 3.75× to 3.91×; training without it gives 1.31× (both under this generation's detached-source convention). Inference-time ablation overstated the mechanism by about 2.9× in PPL ratio and 4.9× in nats. Only trained-without numbers appear here.

Citation

@misc{Gueorguiev2026SemSim,
  author    = {Gueorguiev, Dimitar P.},
  title     = {Semantic Simulation: A Prescriptive Lagrangian Framework
               for Semantic Inference --- Conservative Potentials,
               Non-Conservative Memory, and the Shared-Potential Separator},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19712427},
  url       = {https://doi.org/10.5281/zenodo.19712427},
  note      = {Version v22 (Oct 5, 2026).
               Companion code repository (DOI 10.5281/zenodo.23152203):
               \url{https://github.com/dimitarpg13/semsimula-paper}}
}

Card generated from _ladder/ladder.json on 2026-10-05.