Abhishek0219/qwen-fine-tuned
Qwen/Qwen3-4B-Instruct-2507 fine-tuned on Amazon's Shopping Queries Dataset (ESCI, KDD Cup 2022)
to classify query–product relevance as one letter: Exact, Substitute,
Complement, Irrelevant. QLoRA (r=16, alpha=32, all projection modules),
12k balanced training pairs, 2 epochs; adapter merged into bf16 weights.
| model | ESCI test micro-F1 (5k rows) |
|---|---|
| this model | 0.642 |
| gpt-4o-mini, 0-shot | 0.612 |
| gpt-4o-mini, 4-shot | 0.543 |
Usage (vLLM, OpenAI-compatible)
The model answers with exactly one token, so read the label and its confidence from the first-token logprobs — no decoding needed:
resp = client.chat.completions.create(model="esci", messages=build_messages(query, title, brand, color),
max_tokens=1, temperature=0, logprobs=True, top_logprobs=20)
build_messages and the exact system/user prompt live in prompts.py of the
source repo. Serve with --generation-config vllm (the bundled
generation_config.json is the base model's sampling default).
Data: Shopping Queries Dataset (Amazon Science, Apache-2.0). Adapter:
Abhishek0219/qwen-fine-tuned-lora. Source and deployment YAML: https://github.com/AbhishekG-1plus/esci-distill.