embeddinggemma-300m-qat-q8_0 GGUF
Recommended way to run this model:
llama-server -hf ggml-org/embeddinggemma-300m-qat-q8_0-GGUF --embeddings
Then the endpoint can be accessed at http://localhost:8080/embedding, for
example using curl:
curl --request POST \
--url http://localhost:8080/embedding \
--header "Content-Type: application/json" \
--data '{"input": "Hello embeddings"}' \
--silent
Alternatively, the llama-embedding command line tool can be used:
llama-embedding -hf ggml-org/embeddinggemma-300m-qat-q8_0-GGUF --verbose-prompt -p "Hello embeddings"
embd_normalize
When a model uses pooling, or the pooling method is specified using --pooling,
the normalization can be controlled by the embd_normalize parameter.
The default value is 2 which means that the embeddings are normalized using
the Euclidean norm (L2). Other options are:
- -1 No normalization
- 0 Max absolute
- 1 Taxicab
- 2 Euclidean/L2
- >2 P-Norm
This can be passed in the request body to llama-server, for example:
--data '{"input": "Hello embeddings", "embd_normalize": -1}' \
And for llama-embedding, by passing --embd-normalize <value>, for example:
llama-embedding -hf ggml-org/embeddinggemma-300m-qat-q8_0-GGUF --embd-normalize -1 -p "Hello embeddings"