EngGPT2-16B-A3B GGUF
This repository contains a GGUF conversion of
engineering-group/EngGPT2-16B-A3B.
The model inherits the original EngGPT Non-Commercial License and is intended for research and non-commercial use only.
Files
enggpt2-bf16.gguf: BF16 GGUF conversion.enggpt2-repro-notes.md: conversion and runtime notes.enggpt2-llama-cpp-support.patch: patch used for llama.cpp support.enggpt2-config-for-conversion.json: local conversion config.
Runtime Requirement
This GGUF requires a patched llama.cpp runtime. An unpatched runtime may fail
with:
wrong number of tensors; expected 291, got 243
The required runtime changes add support for optional attention Q/K RMSNorm tensors in the LLaMA loader and apply those norms before RoPE. The converter patch also maps EngGPT2 MoE expert tensor names into a Mixtral-compatible GGUF layout.
Build
For a static local build:
cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF
cmake --build build --target llama-cli llama-server llama-completion -j 8
Quick Test
./build/bin/llama-cli \
-m enggpt2-bf16.gguf \
-p "Spiega in italiano cos'e un trasformatore elettrico." \
-n 300 \
--temp 0
Ollama
Stock Ollama does not run this GGUF unless its bundled llama.cpp runner includes the same runtime patch.
Related Repositories
- Original model:
engineering-group/EngGPT2-16B-A3B - llama.cpp fork:
robertobissanti/llama.cpp - MLX workflow:
robertobissanti/enggpt2-mlx-manager