embeddinggemma-300m GGUF
GGUF format of google/embeddinggemma-300m for use with CrispEmbed and Ollama-compatible runtimes.
Google EmbeddingGemma 300M. Lightweight multilingual embedding model based on Gemma 3, optimized for search, retrieval, and semantic similarity across 100+ languages.
Model details
- Architecture: Gemma 3 transformer (300M params), bidirectional attention
- Pooling: Mean pooling + Dense projection (768→3072→768) + L2 normalize
- Embedding dimension: 768
- Languages: 100+ languages
- Context length: 2,048 tokens
- License: Gemma
Files
| File | Quantization | Size | Parity (cos vs HF) |
|---|---|---|---|
| embeddinggemma-300m.gguf | F32 | ~1.2 GB | 1.0000 |
| embeddinggemma-300m-q8_0.gguf | Q8_0 | ~327 MB | 0.9998 |
| embeddinggemma-300m-q5_k.gguf | Q5_K | ~289 MB | 0.9954 |
| embeddinggemma-300m-q4_k.gguf | Q4_K | ~277 MB | 0.9834 |
| embeddinggemma-300m-qat-q8_0-dense.gguf | Q8_0 (QAT) | ~347 MB | 0.984 vs vanilla HF |
Q4_K shows mild degradation (~1.7%) which is typical for 4-bit quantization on embedding models. Use Q8_0 or Q5_K if highest fidelity is needed.
embeddinggemma-300m-qat-q8_0-dense.gguf (community / gemma-embedding arch)
This is the official llama.cpp QAT export
(ggml-org/embeddinggemma-300m-qat-q8_0-GGUF,
general.architecture = gemma-embedding) with the SentenceTransformers Dense
head (2_Dense/3_Dense from
google/embeddinggemma-300m)
baked in as dense.0/1.weight. The upstream gemma-embedding GGUF omits the
Dense modules (llama.cpp applies them from an external
--sentence-transformers-dense-modules file); without them the output is the
raw backbone mean-pool, orthogonal to real EmbeddingGemma. With Dense baked,
CrispEmbed reproduces the HF SentenceTransformer output at cos 0.984 (the
QAT checkpoint's weights differ slightly from vanilla google/embeddinggemma-300m,
so this is measured vs vanilla — not a defect). Reproduce with
models/add-st-dense-to-gguf.py.
Quick Start
# With CrispEmbed
crispembed -m embeddinggemma-300m.gguf "Hello world"
See CrispEmbed for full documentation.
Notes
These GGUFs use the Ollama-compatible format with CrispEmbed extension keys:
gemma3.is_bidirectional = 1— bidirectional (no causal mask)gemma3.pooling_type = 1— mean poolinggemma3.rope.freq_base_local = 10000.0— sliding-window RoPE theta- Dense projection weights stored in F32 for correctness across all quant levels
Provenance and EU AI Act Art. 53 note
- Upstream model: google/embeddinggemma-300m — published by
google. - Upstream licence:
gemma. This repository redistributes under the same terms; it grants no rights the upstream licence does not. - What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
- Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
- Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.