calculator_scratchpad, max_digits=3 (torchwright)
A compiled transformer: the
torchwright compiler emitted
these weights directly from a computation graph — nothing was trained. This
bundle is the calculator_scratchpad example built with max_digits=3: a computation graph for integer arithmetic (A op B with op in + - *) that streams its serial carry/borrow work as visible thinking tokens before emitting the answer.
The bundle uses the stock Phi-3 architecture and loads through transformers
without custom model code or trust_remote_code.
Run it in fp32 with greedy decoding (do_sample=False). Other
precisions and decoding modes are outside the supported contract.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = 'physicsrob/torchwright-calculator-scratchpad-max-digits-3'
model = AutoModelForCausalLM.from_pretrained(repo_id).eval()
tok = AutoTokenizer.from_pretrained(repo_id)
enc = tok('12*34\n', return_tensors="pt")
out = model.generate(enc["input_ids"], max_new_tokens=96, do_sample=False,
eos_token_id=tok.eos_token_id, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
Input and output
Prompts are A op B terminated by a newline: two non-negative decimal
operands of up to 3 digits, with op one of +, -, *.
Subtraction may produce a negative result. Wider operands, or any character
outside the model's small vocabulary, are outside the contract — the output
is undefined.
| prompt | output |
|---|---|
12*34 | <THINKING>…</THINKING>408 |
7+8 | <THINKING>…</THINKING>15 |
999*999 | <THINKING>…</THINKING>998001 |
999+1 | <THINKING>…</THINKING>1000 |
999-123 | <THINKING>…</THINKING>876 |
123-999 | <THINKING>…</THINKING>-876 |
The answer is preceded by a visible scratchpad between <THINKING> and
</THINKING>: superscript glyphs (⁰¹²) stream the per-column carry /
borrow / comparison state, plain digits are the unnormalized scratch answer,
and subscript glyphs (₀₁₂) stream the leading-zero cleanup count. The
final answer follows the closing tag — e.g. 999+1 ends
…</THINKING>1000.
Intended use and limitations
This model is a demonstration of a computation graph compiled into transformer weights. It is not a general language model or a general-purpose calculator; only the input contract above is supported.
Verification
The examples above are exact reference outputs. The Modal publishing path
reloads the emitted checkpoint through stock transformers, checks those
examples plus additional width-limit cases against Python integer arithmetic,
and refuses to upload on a mismatch. This is a functional smoke test, not
exhaustive verification of every allowed expression.
Size
The checkpoint stores 2.04 GB of dense fp32 weights (511,043,584 entries) at compile width d=2048. 99.96% of those entries are exactly zero: the vast majority of the model is unused canvas, so size reflects the compile geometry rather than stored knowledge.
The zero entries are not compressed, and dense transformers execution still
pays their memory and compute cost. CPU execution is supported; allow
additional RAM beyond the checkpoint size.
Family
One example of many compiled with torchwright. Calculator siblings —
calculator-simple (serial arithmetic, depth grows with the digit count),
calculator-advanced (carry-lookahead, near-flat depth),
calculator-scratchpad (flat depth; the serial work streams out as visible
thinking tokens), and calculator-memorize (no arithmetic at all: a fact
table, exponential in the digit count) — are published at several digit
widths. Browse the
torchwright calculator models
on Hugging Face.