Where is Shakespeare? — Final D

Final D is a frozen Qwen3-0.6B-Base model fine-tuned to retrieve prompt-supplied facts and execute formal lookup programs through depth 10. The title asks where an answer comes from inside a transformer. This model studies in-context retrieval, not Shakespeare facts recalled from pretraining.

The owner has released this model package openly under Apache-2.0, retaining the upstream model terms. The accompanying LICENSE is byte-identical to the original Qwen license. This repository contains the exact frozen checkpoint, tokenizer, configuration and checksum manifest; it does not contain the separate original training archive or private research code. Public access requires no invitation.

Intended use

Research on prompt-fact retrieval, multi-step program control and premature stopping. It is a Base-derived formal-task model, not a conversational assistant. Load it without a chat template or remote code. Its failures on longer programs are part of the research, rather than a solved capability.

Architecture and training

• Base revision: da87bfb608c14b7cf20ba1ce41287e8de496c0cd.
• 28 transformer blocks, indexed 0–27; residual width 1,024.
• 16 query heads, eight key/value groups; head dimension 128.
• Synthetic, acyclic prompt-fact worlds; curriculum maximum depths 2, 4, 8, 8, 10.
• 35,136 original examples, 549 optimizer steps, 595,357 supervised target tokens.
• Turn/category-balanced cross entropy; original effective batch size 64.
• Original training used eight H100 GPUs. A new single-device run is a replica experiment, not a bit-identical recreation of these weights.

Training targets count supervised trace tokens, not all prompt tokens. Qwen pretraining is inherited; see the upstream model card.

Archived evaluation

The archived native benchmark contains 2,560 complete greedy traces, 40 per depth/family cell across depths 1–16 and four alternating/repeated relation families. Within trained depths 1–10, exact accuracy is 1,548/1,600 (96.8%). Beyond them, it is 128/960 (13.3%). Of 832 extrapolation errors, 823 (98.9%) first fail by premature DONE, four by a wrong result and five by unnecessary continuation. These are archived results for one checkpoint and specified panels, not a new replication or a general reasoning benchmark.

In one actual depth-12 repeated-IN trace, the model returns OSCAR16 correctly at turn 10 but announces DONE. It omits the last two lookups. The stopping algorithm remains unresolved. Copying tested late object-token keys and values can transfer an answer, and a block-11 direction can rescue individual action fields; neither finding establishes a complete circuit or safe autonomous execution.

Download and verify

python -m pip install "huggingface-hub==0.35.3" "transformers==4.57.1" "torch==2.7.1" "safetensors==0.6.2"
hf download mundetr1/where-is-shakespeare-final-d --revision final-d-v1 --local-dir models/final-d

Choose an appropriate CUDA build of torch for GPU use. The three safetensors shards total about 2.4 GB. MANIFEST.json gives exact byte counts and SHA-256 hashes for every weight, tokenizer, configuration and license file.

import hashlib
import json
from pathlib import Path

root = Path("models/final-d")
manifest = json.loads((root / "MANIFEST.json").read_text())
for name, expected in manifest["files"].items():
    h = hashlib.sha256()
    with (root / name).open("rb") as stream:
        for chunk in iter(lambda: stream.read(8 * 1024 * 1024), b""):
            h.update(chunk)
    assert (root / name).stat().st_size == expected["bytes"], name
    assert h.hexdigest() == expected["sha256"], name
print("MODEL_CHECKSUMS_PASS")

Run a small example

This cleaned symbolic example teaches the task; it is not a newly measured model result. Real prompts also contain distractor facts. Read the nested query from the inside out. Each line returns a result and announces the next operation; depth four means four lookups and four lines, including the final DONE line.

NORA16 -> JACK20 | JACK20 IN NORA17 | NORA17 -> ALICE23 | ALICE23 IN ALICE19 |  ?IN ?-> ?IN ?-> NORA16 |

Expected oracle trace:

JACK20 | IN
NORA17 | ->
ALICE23 | IN
ALICE19 | DONE

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

path = "models/final-d"
device = "cuda" if torch.cuda.is_available() else "cpu"
torch.backends.cuda.matmul.allow_tf32 = False
torch.backends.cudnn.allow_tf32 = False
tokenizer = AutoTokenizer.from_pretrained(path, local_files_only=True, trust_remote_code=False)
model = AutoModelForCausalLM.from_pretrained(
    path, local_files_only=True, trust_remote_code=False, torch_dtype=torch.float32,
    attn_implementation="sdpa",
).to(device).eval()
prompt = 'NORA16 -> JACK20 | JACK20 IN NORA17 | NORA17 -> ALICE23 | ALICE23 IN ALICE19 |  ?IN ?-> ?IN ?-> NORA16 |'
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to(device)
with torch.inference_mode():
    output = model.generate(**inputs, do_sample=False, max_new_tokens=128,
                            pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

CPU inference is possible but slower. Scientific interventions used FP32 with TF32 disabled and a numerical-equivalence gate. Basic generation above is a usage example, not that full gate or causal assay. Exact outputs can depend on the implementation and numerical environment.

Limitations and provenance

Longer programs often stop early. Other assays can produce overruns and malformed traces. A successful intervention on one field can harm complete execution. The model is not a general reliable controller, and these results cover one seed and checkpoint. The complete minimal retrieval circuit and definitive stop rule remain open. No new training or GPU scientific campaign was run to publish this package. Cite the model repository and exact revision or tag used in your work.

The associated research repository is currently private: Where-is-Shakespeare. Model access here is public and independent of access to that code repository.