Tessera-1B-Nano-Base
The plain half of a controlled experiment by Paragon Intelligence Labs: the
untouched HuggingFaceTB/SmolLM2-360M backbone, continued-pretrained with no
concept path, no extra parameters, no tricks. This is the matched NTP-only baseline of the comparison - every difference from its sibling is the concept path's doing, and nothing else.
*60-second demo of the comparison Space
- the left column is this model. The metals rows were captured live from the Space; the photosynthesis row was generated with the Space's code against the published checkpoints on CPU.*
TL;DR
This arm is the boring half on purpose: same tokens, same order, same initialization as the concept arm, no restarts, no NaNs. Science needs a control before it needs a result.
- Read this card as the yardstick: the final numbers below are what the concept arm is measured against.
- The comparison outcome and the intervention readings live in the whitepaper at
docs/whitepaper.mdin the ncp-smol repository.
Numbers
The raw record this narrative is built from; per-arm READMEs and the whitepaper hold the full analysis.
| Metric | Value |
|---|---|
| Held-out NTP loss | 2.5135 |
| Held-out perplexity | 12.3485 |
| Training tokens | 999,948,288 |
| Tracked compute estimate | $24.88 |
Intervention deltas are increases in held-out NTP loss relative to normal predicted concept feedback, evaluated on identical batches.
Architecture
- chunk size: 4
- product code: 15 segments x 64 entries
- causal concept blocks: 2
- injection point: before token decoder block 2
- NCP target: next continuous concept
- loss:
L_ntp + 1 L_ncp + 1 L_vq
This is a compact ConceptLM-style implementation, not an 8.9B NCP-ArchPreview replica. It omits iterative residual coding, cross-scale residual connections, and the large-scale training recipe.
Training data and provenance
The matched corpus and the training record are pinned in the ncp-smol repository:
packed-cache SHA256 hashes, the complete metric log, trainer state, and the raw
intervention evaluation JSON (also shipped in this repository as eval.json and
metrics.jsonl).
Loading
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("yava-code/Tessera-1B-Nano-Base")
model = AutoModelForCausalLM.from_pretrained("yava-code/Tessera-1B-Nano-Base")
The Tessera family
Three checkpoints, one story, in reading order:
- Tessera-135M-Gate - 135M architecture gate (overfit, published for the scale story)
- Tessera-1B-Nano-Base - 1B-token matched NTP-only control (this model)
- Tessera-1B-Nano - 1B-token matched concept arm
All cards are generated from the run artifacts by the same build_card; the
study repository holds the
whitepaper and full records.
References
- Code and study: https://github.com/yava-code/Tessera-1B-Nano
- ConceptLM: https://arxiv.org/abs/2602.08984
- NCP-ArchPreview: https://arxiv.org/abs/2609.10715 By @yava-code and @el1zabethcodes