Modular Residual Reasoning Core with G6.5 Attachment Demo

A research checkpoint of an attention-free reasoning core that integrates typed external elements through commutative residual assembly.

External element types developed in the associated experiments include:

• frozen reconstructive document memory;
• deterministic bounded virtual-machine execution;
• positional tokenizer-aware result buffers.

Checkpoint

Important distinction

When loaded as a standard causal LM, this repository runs the reasoning core without external memory or VM elements.

The G6.5 attachment result requires the typed external-element protocol demonstrated in attachment_demo/.

Therefore standard HellaSwag/MMLU/WikiText evaluation and the attachment demo measure different operating modes.

Loading the core

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

path = "Bochkov/modular-reasoning-0p5b-g6p5-demo"

tokenizer = AutoTokenizer.from_pretrained(path)

model = AutoModelForCausalLM.from_pretrained(
    path,
    trust_remote_code=True,
    dtype=torch.bfloat16,
).to("cuda").eval()

inputs = tokenizer(
    "The capital of France is",
    return_tensors="pt",
    add_special_tokens=False,
).to("cuda")

with torch.inference_mode():
    output = model(
        input_ids=inputs["input_ids"],
        num_iterations=5,
        return_dict=True,
    )

print(output.logits.shape)

External consistency assembly

For proposal $z_e$ attached to reasoning node $i$:

$$ r_e = z_e - Z_i, $$

$$ R_i = \frac{\sum_e w_e(z_e-Z_i)} {\epsilon+\sum_e w_e}. $$

Element order is discarded by segment summation.

A deterministic consistency step moves the reasoning field toward the weighted proposal mean before a bounded trainable correction.

Zero-shot G1.7 → G6.5 attachment result

A fixed reasoning checkpoint was evaluated on a provenance-controlled delta suite whose required elements were absent from G1.7 and present in G6.5.

Best reported checkpoint:

The reasoning weights were unchanged during attachment.

Fault controls detached 10 required elements. Reattachment restored the baseline exactly under the deterministic evaluation protocol:

reattach_loss_difference = 0

Scope

This is not open-domain QA. The suite contains 417 provenance-controlled tasks referencing 105 newly attached elements. The experiment measures compatibility, coverage growth and fault locality.

Demo artifacts

attachment_demo/
├── qa/
├── cache/
├── growth_results.json
└── run_attachment_demo.py

Run:

python3.11 attachment_demo/run_attachment_demo.py   --model .   --device cuda:0   --iterations 5

Limitations

• The core was trained on a specialized mixture of language, memory-QA, and VM tasks.
• Standard benchmark scores do not include automatic semantic memory retrieval.
• The demonstrated memory is reconstructive rather than lossless.
• The G6.5 result uses provenance-controlled retrieval tasks.
• No claim is made that stored memory values equal dense trainable parameters.
• No KV cache is implemented.
• Use batch size 1 for generic harness evaluation.

Audited standard language-model evaluation — core only

No external memory or VM is enabled in this table. The low standalone LAMBADA/WikiText performance is material to interpreting this artifact; it is not an attachment result.

Bounded attachment demonstration — not a standard LM benchmark

The names G1p7 and G6p5 identify the source configurations used to select evidence. They do not state that billions of memory values are loaded by this downloadable demonstration.

Evaluated: 417 examples, 2396 target tokens. Cache ID: da0ab720f71cc2eb103983b2ae3cc439b354876b85c566406192a6bc5bb796fc.

Observed NLL difference: 3.7643 nats/token. Because coverage changes from one condition to the other, this difference is not an isolated estimate of memory-scale benefit at fixed answerability.

The physical size of the delivered attachment artifact was not supplied to this update script; no physical-capacity claim is made here.

This is a selected-evidence demonstration. Report the number of selected records, the file size, and the full snapshot footprint separately after auditing them. The source snapshot is not bundled merely because its name appears here.

The QA tasks have controlled provenance. Some candidate-generation experiments used source-evidence token spans and answer offsets provided by the dataset builder; they must not be interpreted as retrieval from an arbitrary user question without those fields.

🧑‍🔬 Citation & Concept

If you use this model or the underlying concepts in your research, please cite our work:

@misc{bochkov2026parametermonolithreconstructivememories,
      title={Beyond the Parameter Monolith: Reconstructive Memories, Executable Skills, and Residual Assembly for Language Models}, 
      author={A. Bochkov},
      year={2026},
      eprint={2610.04012},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2610.04012}, 
}