AAIE 8K GFT — 5M checkpoint
This is the selected 5-million-token general-instruction-tuning checkpoint from the AAIE research project. It is a standalone, full-weight Hugging Face model: there is no LoRA adapter to attach or merge.
Lineage
AAIE 50M YaRN 8K base -> selective SmolTalk V2 general instruction tuning -> selected 5M checkpoint
- Architecture: Llama-compatible dense decoder, 20 layers, hidden size 512, eight attention heads / two KV heads, tied embeddings.
- Context configuration: 8,192 tokens using YaRN (
factor: 8, original position length 1,024). - Tokenizer vocabulary: 151,936 tokens.
- Training: full-weight, completion-masked instruction tuning; selected after a
bounded 5M-token schedule at learning rate
1e-4. - Model-weight SHA-256:
4b1c4cc74f0f0dadd9923b4381151444c04e00682a39301b812a2116ec7c22b7.
The generation configuration corrects the checkpoint package's end-token
settings: <|im_end|> (151645) and <|endoftext|> (151643) both stop
generation.
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "moinsaj/aaie-8k-gft-5m"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
messages = [{"role": "user", "content": "Explain database indexes in two sentences."}]
inputs = tokenizer.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
)
output = model.generate(inputs, max_new_tokens=160, do_sample=False)
print(tokenizer.decode(output[0, inputs.shape[-1]:], skip_special_tokens=True))
Important limitations
This is an AI-reviewed research checkpoint, not an educational assessment tool or a replacement for the AAIE product's hosted model. Its bounded evaluation showed better stopping behaviour than the unchanged base, but it did not demonstrate dependable instruction correctness, extraction, or criterion-level feedback. It requires further human/educator validation before any applied use.
The 8K context setting is an experimental YaRN extension. It should not be read as evidence of reliable general long-context reasoning.