Prajna CRN by @eulogik · Repo: github.com/eulogik/prajna · Model: huggingface.co/eulogik/prajna

🧠 Prajna CRN — Cognitive Resonance Network adapter for Gemma 4 E2B

Open weights. A 6.7M-parameter trainable "cortex" injected into the hidden states of a frozen Gemma 4 E2B. On its training-distribution text it cuts perplexity from 106.85 → 6.02 (≈18× lower) versus the frozen base — using only 0.3% of the base's parameters, trained CPU-only on a Mac Mini M4.

Prajna is not a standalone model and not a general reasoning upgrade. It is a parameter-efficient hidden-state correction network: a small module that reads the base model's intermediate hidden states and adds a gated correction back into the residual stream. The base stays frozen; only the CRN trains. This repo ships the trained weights and the minimal loader.

What it does (honestly)

Read this carefully: the ≈18× is a corpus-compression result, not general intelligence. On text outside its training domain the same adapter increases loss, and it does not perform implicit-goal reasoning (a reworded car-wash probe scored 0/8; the widely-circulated "76% of models fail" test is not passed). We publish this openly because the architecture is the interesting part, and the honest limitations are part of the experiment.

Why the architecture is the innovation

The CRN is a small module injected at intermediate hidden states (here: Gemma layers 7, 15, 23, 31) that computes a correction from four cooperating sub-modules:

• Resonance Attention — frequency-modulated attention over hidden states using a learned spectral filter bank (interpretable "cognitive bands", top-k selected).
• Skill Composer — a library of low-rank composable skills (top-k selection).
• Reflective Loop — a latent-space self-correction operator.
• Episodic Memory — a small differentiable key-value memory with read/write gates and temporal decay, giving a persistent context channel without retraining the base.

A per-injection sigmoid gate (crn_mix, learned) controls correction strength. Because corrections are additive and gated, each injection is ablatable — we measured that disabling injection @layer 7 alone raises in-distribution ppl from 6.02 to 13.93, while @layer 31 barely matters. This makes the method interpretable in a way LoRA/adapters are not.

Key properties

• Base model is fully frozen (no_grad); zero gradient flow into it.
• Backbone-agnostic: operates on extracted hidden states.
• 0.3% trainable params; trains on CPU in ~22 h.

Files

Training pipeline and data generation are private; this is an open-weights release. The loader above is sufficient to run inference.

Quick start

import torch
from crn_components import PrajnaStudentMultiLayer

student = PrajnaStudentMultiLayer(
    device="cpu", inject_every=8, max_length=96,
    num_frequencies=8, top_k=2, num_skills=32, skill_rank=4,
    num_corrections=8, mem_size=256, mem_dim=64,
)
ckpt = torch.load("dpo_final.pt", map_location="cpu", weights_only=False)
student.load_state_dict(ckpt["crn"], strict=False)
student.load_memory("memory_dpo_final.json")   # required
student.eval()
tok = student.tok

ids = tok("Explain why the sky is blue.\n", return_tensors="pt").input_ids
with torch.no_grad():
    out = student._collect_hidden(ids)
    logits, _ = student._apply_crn(out, training=False)
print(tok.decode(logits.argmax(-1).flatten()))

The base google/gemma-4-E2B is downloaded automatically from HuggingFace on first load (~10 GB). Set HF_HOME to an external disk if space is tight. Note: the wrapped model runs on CPU; MPS is not supported for this loader (a Gemma-4 embedding allocation issue).

Training (summary)

Hardware: Mac Mini M4, 16 GB, CPU only.

Limitations & future work

• The released checkpoint is a domain specialist, not a general model. It overfits its training corpus and degrades out-of-distribution.
• No implicit-goal / common-sense reasoning is demonstrated (reworded car-wash probe: 0/8). The earlier "reasoning" framing was a keyword-match artifact in evaluation and has been retracted.
• Active research directions (not yet demonstrated):
  • Make the correction generalize beyond the training domain.
  • Validate reasoning transfer on reworded/held-out prompts.
  • Scale injections and memory; explore larger bases.

We are publishing this as a credible, reproducible experiment — a genuinely novel parameter-efficient architecture with an honest account of where it works and where it doesn't. Contributions and critiques welcome.

🔗 Reference & author

Author: @eulogik · GitHub · Model hub

Cite / reference:

@misc{prajna-crn-2026,
  title        = {Prajna: Cognitive Resonance Network — a parameter-efficient hidden-state
                  correction adapter for frozen LLMs},
  author       = {eulogik},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/eulogik/prajna}},
  note         = {Open-weights release (training code private)}
}

Follow-up research (different system): a follow-up study explores retrieval-free error correction on a frozen base (CRN v2). Paper: arXiv:2609.16145. Note this paper studies a different system from the model on this page.

If you build on Prajna or reproduce the in-distribution perplexity result, a link back to huggingface.co/eulogik/prajna is appreciated. Feedback and collaborations welcome via the repo.

License

Weights: Apache 2.0. Loader (crn_components.py): Apache 2.0. Training code: private.