bestie-genz-qwen2.5-1.5b-lora-gguf 🫶 — a Gen Z persona LLM

bestie-genz-qwen2.5-1.5b-lora-gguf is a small, local chat model that answers normal questions correctly but talks like your chronically-online Gen Z best friend. It's Qwen2.5-1.5B-Instruct with a lightweight LoRA adapter that reshapes tone, not facts.

Built to run fully offline on a MacBook Air M2 (16GB) via llama.cpp with Metal acceleration.

⚠️ Slang snapshot: This model reflects Gen Z slang and tone as of 2026. Internet language moves fast — some phrasing will feel dated over time. This is a point-in-time snapshot, not a living style guide.

What's in this repo

The LoRA is intentionally not merged into the base weights. llama.cpp applies it at inference time via --lora, so you can toggle the persona on/off with the same base model.

Base model & why

• Base: Qwen2.5-1.5B-Instruct (Apache-2.0)
• Why: Best-in-class reasoning/factuality for a sub-2B model, permissive license for redistribution, first-class llama.cpp conversion support, and small enough to LoRA-fine-tune and run on 16GB unified memory.

Quick start (llama.cpp)

Single merged model (no adapter needed):

llama-cli \
  -m bestie-merged-q4_k_m.gguf \
  -st -n 200 --temp 0.7 \
  -p "Why is the sky blue?"

CLI (single response):

llama-cli \
  -m qwen2.5-1.5b-instruct-q4_k_m.gguf \
  --lora bestie-lora.gguf \
  -st -n 200 --temp 0.7 \
  -p "Why is the sky blue?"

Server (OpenAI-compatible API):

llama-server \
  -m qwen2.5-1.5b-instruct-q4_k_m.gguf \
  --lora bestie-lora.gguf \
  --host 127.0.0.1 --port 8080 -c 2048 --alias bestie

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"bestie","messages":[{"role":"user","content":"How do I stay motivated to work out?"}]}'

Example outputs (base vs. bestie)

Prompt: Why is the sky blue?

• Base: "The sky appears blue because of a phenomenon called Rayleigh scattering. This happens when light from the sun travels through the Earth's atmosphere and is scattered..."
• bestie: "Bet, quick breakdown: Sunlight splits into colors, blue gets scattered more. No fluff, just signal."

Prompt: Explain what a black hole is.

• Base: "A black hole is a region of spacetime where gravity is so strong that nothing, not even light, can escape from it..."
• bestie: "Real talk, no BS: A black hole is a region of spacetime with such strong gravitational pull that nothing, not even light, can escape from it."

Prompt: How do I stay motivated to work out?

• bestie: "Real talk, no BS: Break your workout into manageable 30-minute sessions, track your progress, and always focus on form over intensity. Progress over perfection, always."

Intended use

• Casual Q&A, study help, explanations, and everyday assistance with a fun, informal Gen Z voice.
• Great for demos, personal projects, and learning the LoRA → GGUF → llama.cpp pipeline.

Out of scope / limitations

• Small model (1.5B): can hallucinate or oversimplify. Verify anything important.
• Persona artifacts: trained on a small (~165-example) dataset, so it occasionally over-uses signature phrases ("real talk, no BS", "drop your context") or leaks a stray token. Larger/cleaner data would smooth this out.
• English only, informal register — not suitable for formal/professional writing.
• Not safety-tuned beyond the base model's alignment.

Training details

• Method: LoRA (PEFT) — the only step done in transformers/peft, since llama.cpp is inference-only and can't train.
• Everything else via llama.cpp: GGUF conversion (convert_hf_to_gguf.py), LoRA→GGUF export (convert_lora_to_gguf.py), quantization (llama-quantize), inference (llama-cli), and serving (llama-server).
• Data: ~165 hand-written (prompt, response) pairs — normal questions answered correctly in an authentic Gen Z voice — plus ~20 validation examples.
• Hyperparameters: low rank (small-dataset appropriate), few epochs, tuned for 16GB unified memory on Apple Silicon (MPS).

License

Apache-2.0, inherited from the Qwen2.5-1.5B-Instruct base model.