Emma-5 GGUF
Unofficial GGUF conversion of eldavoo/emma-5, archived from the original Egomnia Emma-5 release.
The conversion was validated by reconstructing the ONNX attention graph in PyTorch and comparing logits against the original ONNX Runtime output before exporting to GGUF.
The original ONNX model remains the reference runtime. This GGUF is intended for raw llama.cpp completion use, not chat-template frontends.
Files
| File | Description |
|---|---|
emma-5-f16.gguf | FP16 GGUF, ~1.2 GB |
Conversion Details
The original model is distributed as ONNX only. Direct conversion to GGUF required matching the exported attention graph exactly.
Pipeline summary:
- ONNX weights and rotary constants were loaded.
- The exported attention graph was matched in PyTorch for validation.
- Logit parity was checked against ONNX Runtime.
- Q/K rows were permuted from split-half RoPE layout to llama.cpp interleaved layout.
- The model was exported as FP16 GGUF with
rms_norm_eps=1e-6.
Validation result:
max absolute logit error: ~4e-5
Architecture
| Parameter | Value |
|---|---|
| Parameters | ~550M |
| Layers | 28 |
| Hidden size | 1280 |
| Q heads | 16 |
| KV heads | 4 |
| Head dim | 80 |
| GQA ratio | 4:1 |
| FFN | SwiGLU |
| Context | 2048 |
| Vocab | 50000 |
| RoPE theta | 10000 |
| Embeddings | Tied |
Prompt Format
Use a single-line Italian instruction format:
### Istruzione: Qual e la capitale d Italia? ### Risposta:
Use raw completion mode only.
Do not use ChatML, Llama 3 chat templates, OpenAI-style messages, or frontend "conversation" wrappers. In particular, prompts containing markers like <|im_start|> or <|im_end|> are not supported and may cause repeated marker output.
Avoid multiline templates in llama.cpp. The original ONNX SentencePiece path normalizes whitespace differently, while llama.cpp may preserve newline tokens.
For the most faithful behavior, compare against or use the original ONNX Runtime model.
Example
./llama-cli \
-m emma-5-f16.gguf \
--no-conversation \
-p "### Istruzione: Qual e la capitale d Italia? ### Risposta:" \
-n 64 \
--temp 0.7 \
--repeat-penalty 1.2
Expected style:
La capitale d'Italia è Roma...
Quantization
This repo provides the FP16 GGUF. Other quantizations can be generated with llama.cpp:
./llama-quantize emma-5-f16.gguf emma-5-Q4_K_M.gguf Q4_K_M
Compatibility
Converted and smoke-tested with a recent llama.cpp build.
Credits
- Original model by Egomnia S.p.A.
- Archived by eldavoo
- GGUF conversion by Spadav
This is an unofficial conversion.