AAIE Dense GFT — 2K YaRN Feedback Generation
This is a standalone, merged Llama-compatible model for experimental structured feedback generation. It includes the 2K YaRN configuration and can be downloaded and run directly with Transformers; the original LoRA adapter is retained in the adapter/ subfolder.
Model lineage
- Base:
namquangstudy/aaie-ddense-gft-llama - Pinned base revision:
7dbc72d720237bd6abaaa3e7701ca69544a03e4a - Fine-tuning: LoRA rank 8, alpha 20, dropout 0
- Context configuration: 2,048 tokens with YaRN factor 2.0 from the 1,024-token native context
- Export: adapter merged into the base with
PeftModel.merge_and_unload
Inference
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "moinsaj/aaie-ddense-gft-yarn2-fg-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
messages = [{"role": "user", "content": "Your structured feedback prompt here"}]
inputs = tokenizer.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
)
output = model.generate(inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(output[0], skip_special_tokens=True))
The model is configured for a maximum of 2,048 tokens including the prompt and generated response. The tokenizer EOS token must be used as the stopping token.
Training scope
The LoRA adapter was trained for two epochs on 450 feedback-generation candidate examples that fit within the 2K window. A separate family-disjoint validation split of 65 records and held-out evaluation split of 74 records were used for this diagnostic experiment. Epoch 1 was selected by validation-generation structure.
This repository contains model artefacts only. It does not include training data, prompts, student submissions, candidate targets, or model predictions.
Status and limitations
Experimental research artefact. It is not a production release and has not been human-reviewed for educational correctness, grounding, fairness, privacy, or deployment suitability. Do not use it to make student-assessment decisions.
The 2K YaRN setting was tested only for this targeted feedback-generation experiment. It is not evidence of general long-context capability.