URCHIN Strict-Small
URCHIN (Unified Recurrent Connectome with Horizontal Integrate-and-fire Neurons) is a spiking, Dale-constrained recurrent language model for the BabyLM 2026 challenge (Strict-Small track, 10M words), built with the Parallelized Hierarchical Connectome Spiking State-space Model (PHCSSM) as its core architecture. A single cortex region of LIF neurons with a Dale-masked recurrent synapse is iterated K=24 lateral transmission steps per token to a fixed point; a linear head reads the cortex voltage. 4.23M parameters, no attention.
- Architecture: A single horizontal layer of Dale's-law excitatory/inhibitory LIF spiking neurons joined only by lateral (horizontal) connections: the same population receives the input current and is read out by a linear voltage head, with no separate input/output population; each token is resolved by a K-iteration fixed-point recurrence over the lateral synapse. No attention, no depth. Custom code (
trust_remote_code=True). - Track: BabyLM 2026 Strict-Small (10M words).
- Tokenizer: official BabyLM 10M GPT-BERT baseline tokenizer (16384).
- Serial (event-driven) variant:
serial_urchin.py+configuration_serial_urchin.pyprovide an RSNN single-time-scan forward (UrchinSerialForCausalLM) reproducing the parallel outputs (score-equivalent), event-driven in O(T), using the SAME model.safetensors weights.
Usage
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("strict_small", trust_remote_code=True).eval()
tok = AutoTokenizer.from_pretrained("strict_small")
ids = tok("The quick brown fox", return_tensors="pt").input_ids
with torch.no_grad():
logits = model(ids).logits
Intermediate checkpoints
Intermediate training checkpoints are provided as git revisions named chck_<N>M for the BabyLM challenge fast-eval.
License and citation
Released under CC BY-NC 4.0 (attribution required, non-commercial only). If you use this model or code, please cite (see CITATION.cff):
@misc{anonymous2026urchin,
title = {URCHIN: A Horizontal Spiking Language Model for Data-Constrained Pretraining},
author = {Anonymous},
year = {2026},
howpublished = {under review (anonymized)},
note = {URCHIN spiking recurrent language model, BabyLM 2026}
}
Provenance and integrity fingerprints (canary + weight SHA-256) are documented in PROVENANCE.md.