GPT-2 Large - Alpaca (Merged)

This is an instruction-tuned version of openai-community/gpt2-large (774M parameters) fine-tuned on the yahma/alpaca-cleaned dataset using Low-Rank Adaptation (LoRA) and subsequently merged back into the base weights.

The model is exported in 16-bit precision (float16) in safetensors format, ready for inference, evaluation, or further quantization (e.g. GGUF for llama.cpp).

Model Details

• Model Name: GPT-2 Large Alpaca Merged
• Developed by: Trained locally with TRL & PEFT
• Base Model: openai-community/gpt2-large (~774M parameters, 36 layers, 20 heads, 1280 hidden dimension)
• Language: English
• License: MIT (Base model) / Follows Alpaca dataset terms
• Fine-Tuning Framework: Hugging Face transformers, peft, and trl (SFTTrainer)
• Precision: FP16 (torch.float16)
• Format: safetensors

Prompt Template

To achieve optimal performance, use the standard Alpaca prompt template:

Without Context / Input

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:

With Context / Input

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Input:
{input}

### Response:

Quickstart / How to Use

Using transformers pipeline

from transformers import pipeline

generator = pipeline(
    "text-generation",
    model="americansquid/gpt2-large-alpaca",
    torch_dtype="auto",
    device_map="auto"
)

prompt = (
    "Below is an instruction that describes a task. "
    "Write a response that appropriately completes the request.\n\n"
    "### Instruction:\nExplain quantum computing in simple terms.\n\n"
    "### Response:\n"
)

output = generator(
    prompt,
    max_new_tokens=150,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.15,
    do_sample=True,
    pad_token_id=generator.tokenizer.eos_token_id,
)

print(output[0]["generated_text"])

Using AutoModelForCausalLM and AutoTokenizer

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name_or_path = "americansquid/gpt2-large-alpaca"

tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
model = AutoModelForCausalLM.from_pretrained(
    model_name_or_path,
    torch_dtype=torch.float16,
    device_map="auto"
)

instruction = "Give 3 tips for productive remote work."
prompt = (
    "Below is an instruction that describes a task. "
    "Write a response that appropriately completes the request.\n\n"
    f"### Instruction:\n{instruction}\n\n"
    "### Response:\n"
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=True,
        temperature=0.7,
        top_p=0.9,
        repetition_penalty=1.15,
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response[len(prompt):].strip())

Training Details

Dataset

• Dataset: yahma/alpaca-cleaned
• Data Size: 51,760 examples
  • Train split: 50,760 examples
  • Eval split: 1,000 examples (stratified holdout)
• Formatting: Alpaca instruction prompt template with EOS token termination.

Training Configuration

Training Results

• Final Train Loss: 1.3819
• Final Validation Loss: 1.3191
• Validation Token Accuracy: 67.40%
• Validation Entropy: 1.3064

Quantization & Deployment

Because the LoRA weights have been merged directly into the base weights, this model can be:

1. Converted to GGUF using llama.cpp (convert_hf_to_gguf.py) for local execution in LM Studio or Ollama.
2. Served with vLLM, Hugging Face TGI, or ONNX Runtime.

Limitations and Bias

• Base Model Limitations: GPT-2 was released in 2019 and has known architectural constraints (1024 maximum context window, smaller capacity relative to modern LLMs).
• Hallucinations & Factuality: The model may generate plausible-sounding but incorrect or fabricated statements. Outputs should be verified for factual accuracy.
• Safety & Toxicity: GPT-2 was pre-trained on open web crawls and may reflect societal biases or inappropriate content if prompted in certain ways. Safety filtering should be implemented for public deployment.