StoryPatch v3 — contextual ranking
Select a few words in a short English story, compare alternatives, keep the original or apply and undo an edit. Every character outside the selection is preserved by string composition. Text is not sent to a remote generation API.
Two models, one project
The generator has 13,439,680 parameters. The context ranker has 13,645,761 parameters and transfers its backbone from this project's own generator, then learns a new classification head and span markers. All weight lineage originates in DiffuThink's random initialization. The ranker is not newly random-initialized end to end. No external pretrained weights or teacher model are used. Both use the project's own BPE and a 512-token context. This is custom PyTorch code, not a Transformers AutoModel checkpoint.
Run
python -m pip install -r requirements.txt
python -m diffuthink.v2 serve --model . --ranker ranker --device cpu --port 7862
Open http://127.0.0.1:7862/. Select Classement contextuel · expérimental
to try the new ranker. Causal NLL exploration remains the default. The HTTP
rewrite API accepts ranking: "learned". User edits remain manual.
from diffuthink.v2.inference import load
from diffuthink.v2.editor import rewrite_span
from diffuthink.storypatch.ranker import Reranker
model, tokenizer = load('.', 'cpu')
ranker = Reranker.load('ranker', tokenizer)
text = 'Lily lost her favorite toy. She felt happy. Tears ran down her face.'
a = text.index('happy')
result = rewrite_span(model, tokenizer, text, a, a+5, ranker=ranker)
print(result['recommendation'])
Training and evidence
24,000 training contexts: 12,000 procedural contrasts and 12,000 varied TinyStories excerpts with mechanical corruptions. TinyStories is synthetic; there are no human preference labels. The own-generator backbone is adapted for up to 3,200 updates. Selected checkpoint: update 3000, selected by validation BCE on 1,200 contexts.
The final preference score combines causal NLL and bounded learned compatibility:
-NLL + 0.05 * clip(learned_score, -8, 8).
The weight is selected on validation with no top1 loss in either domain.
Separate validation thresholds determine when to recommend preserving the
original. These scores are not probabilities of correctness.
The paired test has 1,200 inputs across 600 contexts. Half the inputs are clean and half altered. Each method receives identical generated candidates.
| Top1 reference match on altered inputs | NLL | Combined |
|---|---|---|
| Corpus excerpts (300 contexts) | 34.7% | 36.3% |
| Procedural templates (300 contexts) | 33.3% | 50.7% |
| Legacy probes (36, reused) | 75.0% | 75.0% |
Original corpus spans are reconstruction references, not exhaustive lists of valid edits. Exact reference matching can reject good paraphrases. The procedural and legacy tests are reused regression diagnostics, not new independent evidence. No general grammar or reasoning claim is made. Complete results and failures: report. All candidates, hashes and validation trials are included.
The keep/replace policy is conservative: it recommends an accepted repair on only 8.5% of altered inputs, versus 33.2% for NLL alone, while preserving 99.8% of clean inputs. Better ranking does not imply more successful replacement recommendations. All proposals remain available for manual review.
Limitations and licensing
This remains a small English story model. It can suggest incorrect, awkward or contradictory edits, or preserve an original mistake. It cannot recover a reference that the generator did not propose. Ranker training excerpts are shorter than the maximum inference context; long-context generalization is not established. No blind human evaluation has been performed.
Source text and derived TinyStories examples retain CDLA-Sharing-1.0 terms;
see DATA_LICENSE.md and the pinned source revision in
training_info.json. Code, weights and procedural templates use Apache-2.0.
Development, experiments and documentation were substantially AI-assisted.
Reproduction: French protocol.
Published model: https://huggingface.co/OthmaneW/StoryPatch-context-v3