Qwen3-Embedding-4B
This repository contains Qwen/Qwen3-Embedding-4B together with a Furiosa Executable Bundle (FXB) for running it on FuriosaAI RNGD with Furiosa-LLM. The same model also runs on other frameworks (such as Sentence Transformers, vLLM, and Transformers); for usage with those, see the upstream Qwen/Qwen3-Embedding-4B model card.
Overview
Qwen3-Embedding-4B is the 4B text-embedding model in the Qwen3-Embedding
series, built on the Qwen3 dense transformer backbone. It maps text into dense
vector representations for semantic search, retrieval, and similarity matching,
with strong multilingual coverage. Embeddings can be requested at a reduced
dimensionality through the dimensions parameter, thanks to the model's
Matryoshka Representation Learning (MRL) support. Its intended use is the same as
the upstream Qwen/Qwen3-Embedding-4B,
and it is released under the Apache 2.0 License.
- Architecture: Qwen3 (dense)
- Input / Output: Text / Embeddings (vector)
- Supported Inference Engine: Furiosa LLM
- Supported Hardware: FuriosaAI RNGD
Quantization
No quantization — the model runs in its native BF16 precision.
Parallelism Strategy
On RNGD, Qwen3-Embedding-4B runs with a tensor-parallel size of 8 PEs, which maps to a single RNGD card (8 PEs per card).
Usage
To run this model with Furiosa-LLM, follow the examples below after installing Furiosa-LLM and its prerequisites. You can use the model either online through the OpenAI-compatible server or offline through the Furiosa-LLM Python API.
Launch the server
The simplest way to serve the model is:
# Launch the server, listening on port 8000 by default
furiosa-llm serve furiosa-ai/Qwen3-Embedding-4B
When the server is ready, you will see:
INFO: Started server process [27507]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Basic Usage
The server exposes an OpenAI-compatible /v1/embeddings endpoint. You can send a
request with curl:
curl http://localhost:8000/v1/embeddings \
-H "Content-Type: application/json" \
-d '{
"model": "furiosa-ai/Qwen3-Embedding-4B",
"input": ["Hello, world!", "How are you?"]
}' \
| python -m json.tool
Because the endpoint is OpenAI-compatible, you can also use the OpenAI Python client:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.embeddings.create(
model="furiosa-ai/Qwen3-Embedding-4B",
input=["Hello, world!", "How are you?"],
)
for data in response.data:
print(f"Index {data.index}: {len(data.embedding)} dimensions")
Advanced Usage
For offline use, load the model with the LLM constructor (the FXB shipped in
the repo is discovered automatically) and call embed to obtain dense vectors:
from furiosa_llm import LLM
with LLM("furiosa-ai/Qwen3-Embedding-4B") as llm:
outputs = llm.embed(["Hello, world!", "How are you?"])
for output in outputs:
print(f"{len(output.outputs.embedding)} dimensions")
Learn more
- Furiosa-LLM Server (
furiosa-llm serve) — full OpenAI-compatible API reference, including the Embeddings API - Furiosa-LLM — Furiosa-LLM documentation and API reference
- Qwen/Qwen3-Embedding-4B — upstream model card