⚡ GPT-neo-Custom (7B) GGUF ⚡

Next-Generation Polish & English AI Assistant

[!TIP] 💬 Found a bug or have an idea to improve the model?
Join the conversation in the Community / Discussions tab! All feedback, bug reports, and feature requests are actively monitored and incorporated into future releases.

📖 Model Overview

GPT-neo-Custom 7B is a high-performance conversational language model designed to bridge the gap in high-quality Polish language generation while maintaining top-tier software engineering and mathematical problem-solving skills.

Built upon the state-of-the-art Qwen 2.5 7B Instruct architecture, this model underwent a multi-stage QLoRA fine-tuning process specifically balanced for conversational coherence and zero-shot reasoning.

✨ Key Strengths

• 🇵🇱 Natural Polish Fluency: Full mastery of Polish grammar, complex sentence structures, declensions, and cultural nuances without translation artifacts.
• 💻 Software Engineering: Proficient in writing, debugging, and explaining code in Python, C++, JavaScript, TypeScript, SQL, and shell scripting.
• 🧮 Structured Reasoning: Excels at chain-of-thought mathematical reasoning, logic puzzles, and technical documentation.
• 🚀 Universal Compatibility: High-performance GGUF quants tailored for local execution from low-end laptops to dedicated RTX workstations.

📊 Quantization Matrix

🚀 Quickstart Deployment Guide

🦙 1. Ollama (Instant Run)

Run directly from Hugging Face:

ollama run hf.co/Szymon-Wachelko/GPT-neo-Custom-7B-GGUF

Or configure locally with a Modelfile:

FROM ./"GPT-neo-Custom [7B] [Q8_0].gguf"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER stop "<|im_end|>"
SYSTEM "Jesteś polskim asystentem AI GPT-neo-Custom. Zawsze odpowiadaj po polsku."

ollama create gpt-neo-custom -f Modelfile
ollama run gpt-neo-custom

⚡ 2. llama.cpp

Interactive Terminal Chat

llama-cli -m "GPT-neo-Custom [7B] [Q8_0].gguf" -ngl 99 -cnv --color -p "Jesteś polskim asystentem AI GPT-neo-Custom. Zawsze odpowiadaj po polsku."

Standalone ChatGPT-like Web UI

llama-server -m "GPT-neo-Custom [7B] [Q8_0].gguf" -ngl 99 --port 8080 -c 4096

Open your browser at http://localhost:8080 for a full offline web interface.

🖥️ 3. LM Studio

1. Install LM Studio.
2. Download any .gguf file from the Files tab above.
3. Drag and drop the .gguf file into LM Studio.
4. Set GPU Acceleration to MAX in the right panel.
5. Click New Chat and start conversation!

🐍 4. Python Integration

from llama_cpp import Llama

# Load model with full GPU offloading
llm = Llama(
    model_path="GPT-neo-Custom [7B] [Q8_0].gguf",
    n_gpu_layers=-1, # -1 offloads all layers to GPU
    n_ctx=4096
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "Jesteś asystentem AI GPT-neo-Custom. Zawsze odpowiadaj po polsku."},
        {"role": "user", "content": "Cześć! W czym możesz mi dzisiaj pomóc?"}
    ]
)

print(response["choices"][0]["message"]["content"])

💬 Prompt Template (ChatML)

The model is trained on the standard ChatML format:

<|im_start|>system
Jesteś asystentem AI GPT-neo-Custom. Zawsze odpowiadasz w języku polskim.<|im_end|>
<|im_start|>user
Napisz skrypt w Pythonie do pobierania kursów walut z API NBP.<|im_end|>
<|im_start|>assistant

💡 Community, Bug Reports & Feature Requests

The GPT-neo-Custom project is developed with an open-source mindset, and community feedback is vital for future iterations!

🛠️ How to contribute feedback or report issues:

1. Navigate to the **[Community / Discussions](https://huggingface.