Stoicheia -- literary fold 2
Stoicheia is a 405M-parameter character-level masked-diffusion encoder for Ancient Greek
(d_model 1024, depth 32, banded attention: three of every four blocks attend within a
256-character window, the fourth globally). Its input is factored into five aligned planes --
letters, word/sentence boundaries, diacritics, capitalization, punctuation -- each of which can
be masked independently to an explicit unknown state at inference. That is what lets one model
read an edited text, scriptio continua, and a lacuna of unknown length without changing
anything but its input.
Anonymous release accompanying a paper under review.
This is the backbone for fold 2: the tenth of the literary corpus assigned to fold 2's test zone was excluded from its training data, so this checkpoint has provably never read it.
What this checkpoint has not read
Ten literary folds rotate an 80/10/10 split built by a 13-stage pipeline that clusters records into editions of the same work -- by canonical identifier where one exists, by duplicate-aware content matching where it does not -- and then excises from each fold's training data every sentence colliding with its held-out zones (exact and reordered duplicates, word 5-grams, document-level near-duplicates). The guarantee is exact at the fold level: for any passage of the open corpus, at least one released checkpoint has provably never seen it, which is what makes it possible to interrogate a transmitted text with a model that cannot merely be recalling it.
Stoicheia-doc_clean extends the same discipline to documentary text: every inscription and
papyrus is excluded, along with anything a contamination screen flags as quoting one.
Usage
import torch
from transformers import AutoModel
from huggingface_hub import hf_hub_download
REPO = "anonymous-stoicheia/Stoicheia-fold-2"
model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()
# `trust_remote_code` loads the model classes, but the processor is a separate helper:
# fetch it into the working directory before importing it.
hf_hub_download(repo_id=REPO, filename="processing_char_bert.py", local_dir=".")
from processing_char_bert import CharBertProcessor
proc = CharBertProcessor()
# a gap of uncertain width in unaccented scriptio continua: "[N±M]" scores every width
# in N-M..N+M by the model's own confidence, and restores accents and word division too
text = "εναρχηηνο[5±3]καιολογοςηνπροστονθεον"
best, width, candidates = proc.restore_elastic(model, text, mask_dia_boundary=True)
print(best) # ἐν ἀρχῇ ἦν ὁ λόγος, καὶ ὁ λόγος ἦν πρὸς τὸν θεόν.
print(width) # 5 -- the width the model judged most likely
Checkpoint provenance
This is the step with the best held-out bits-per-character (step 112,001), not the last step reached: pretraining is dev-driven, so the eleven backbones stop at different points (78K--122K). training_metadata.json in this repo carries the training configuration.