SMITH-Exp
SMITH-Exp is an instruction-tuned Mixture-of-Experts checkpoint based on GigaChat 3.1 Lightning (10B total parameters, 1.8B active, BF16). It is optimized for multi-hop retrieval through GigaChat 3 tool calls: the model iteratively invokes search_index, then returns an answer with supporting_corpus_ids.
For reproducible results, use the checkpoint with the SMITH retrieval harness or an equivalent implementation of the same tool-calling loop. The evaluation results below assume this execution protocol; behavior and quality are not guaranteed in general-purpose chat frameworks or integrations that use different prompts, tools, parsing, or stopping logic.
The included gigachat3_guided_decoding.py plugin integrates with vLLM's GigaChat 3 tool parser and constrains tool calls to the expected structure. This reduces malformed calls and allows the harness to parse search_index and final answer actions reliably. Use the plugin with the supplied chat template, as shown below.
Evaluation
MuSiQue answerable validation (bdsaglam/musique), 100 queries mixing the three four-hop topologies: 50 4hop1, 25 4hop2, and 25 4hop3. Ranking is against gold supporting paragraphs at k=10 on a text-deduplicated train+validation FAISS index. Agent runs use temperature 0, retriever_k=100, agent_max_k=10, and at most six searches.
Citation scores the supporting_corpus_ids in the final answer. Search scores the union of documents retrieved over the trajectory. Searches is the mean number of search_index calls per question.
The embedder-only row is a comparison baseline (single top-10 search of the original question, no agent). Agents still call search_index as a tool; that tool uses Qwen3-Embedding-4B (2,560-d).
Usage
Serving dependencies
uv venv --python python3.11 --seed .venv
uv pip install -r requirements.txt \
--python .venv/bin/python \
--torch-backend=cu130 \
--index-strategy unsafe-best-match
Package versions and license information are listed in requirements.txt. The model fits on a single 80 GB GPU.
transformers
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
model_name = "ai-forever/SMITH-Exp"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model.generation_config = GenerationConfig.from_pretrained(model_name)
messages = [
{"role": "user", "content": "Which corpus passages support the answer?"}
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt")
inputs = {k: v.to(model.device) for k, v in inputs.items()}
outputs = model.generate(**inputs, max_new_tokens=512)
prompt_len = inputs["input_ids"].shape[1]
print(tokenizer.decode(outputs[0][prompt_len:], skip_special_tokens=True))
vLLM
vllm serve ai-forever/SMITH-Exp \
--trust-remote-code \
--enable-auto-tool-choice \
--tool-call-parser gigachat3 \
--tool-parser-plugin ./gigachat3_guided_decoding.py \
--chat-template ./chat_template.jinja \
--tensor-parallel-size 1 \
--dtype bfloat16
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "ai-forever/SMITH-Exp",
"temperature": 0,
"tool_choice": "required",
"messages": [
{"role": "user", "content": "Which corpus passages support the answer?"}
],
"tools": [
{
"type": "function",
"function": {
"name": "search_index",
"description": "Search the local semantic index and return relevant text snippets.",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "The search query string."},
"k": {"type": "integer", "description": "Optional number of top results to return."}
},
"required": ["query"]
}
}
}
]
}'