GPT-2 (ANEForge)
An unmodified duplicate of openai-community/gpt2, tagged for use with
ANEForge so the weights load and run directly on the
Apple Neural Engine (no CoreML). Weights are byte-identical to the source; see the original repo for
the model details. Licensed mit from the source.
Use with ANEForge
# pip install "aneforge[models]"
import aneforge as af
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("aneforge/gpt2")
model = af.load_llm("aneforge/gpt2") # prefill + resident-KV-cache decode on the ANE
ids = tok.encode("The Neural Engine is")
print(tok.decode(model.generate(ids, max_new_tokens=20)))
ANEForge compiles the model's graph into a single ANE program and streams the weights from this repo
via huggingface_hub. See the docs and the
paper.