Tiny random GPT-2 for testing

This model is randomly initialized and has not been trained. It is for testing upload, download, tokenization, and model loading only. Its output is not meaningful and it is not suitable for real language tasks or benchmarking. No pretrained model weights or training datasets were used.

Architecture: 1 GPT-2 layer, 1 attention head, 16 hidden dimensions, 32 vocabulary tokens, and a maximum context length of 64 tokens. Parameter count: 3792.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "ruhook/test-ruhook"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
inputs = tokenizer("hello world", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=5, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Validated locally with Python 3, torch 2.2.2 and transformers 4.46.3. The toy word-level tokenizer maps words outside its small vocabulary to UNK.