Swift1.5-Qwen3.8-Flash-Next-W4A16-Merlin

Run UkisAI's Swift 1.5 post-train of Qwen3.8-Flash-Next at its full 262K context on four RTX 3090s, with vision, tool calls and fast agent follow-ups.

A 4-bit build of ukisai/Swift1.5-Qwen3.8-Flash-Next (125B MoE): the Merlin checkpoint with the 300 dense tensors Swift 1.5 changed swapped in, made for consumer Ampere cards and served by the patched vLLM engine halt95/qwen38-flash-next-3090s through the companion repository, halt95/Swift1.5-qwen38-flash-next-3090s. The whole KV cache stays in VRAM: long conversations are never paged out to system RAM.

This is an independent derivative made by halt95. It is not made, reviewed or endorsed by UkisAI or by Qwen.

Why use it

• Swift 1.5 at the Merlin checkpoint's speed. On v2.2.0 the two stepped at the same speed within run-to-run noise, and at xhigh Swift 1.5 scored GSM8K-200 198/200 against the Merlin checkpoint's 197 (unpaired runs) with 27 % fewer completion tokens (19 % without one Merlin answer that hit the token cap).
• Long context that actually fits. A 924,993-token FP8 KV pool, measured on this checkpoint; the 131K and 262K needles were found exactly.
• Built for agents. Follow-up turns resume from cache: in one conversation the next turn starts in 0.96 s; with 8 agents at once (temperature 0), 93.0 % of follow-up tokens come from cache and the median follow-up starts in 1.28 s.
• Fast. 5,206–5,473 tok/s prefill from 10K to 100K tokens; decode is the engine's (not re-measured on this checkpoint).
• Quality kept. Experts in Intel's INT4 AutoRound, attention in BF16, xhigh reasoning by default; GSM8K 197/200 (thinking off), structured output 80/80.
• Vision and tools. Up to 42 images per request (4 MP each), Qwen3 tool calling and reasoning parsers, one OpenAI-compatible endpoint.

What you need

• 4× RTX 3090 (24 GB, sm_86), headless, with no other CUDA process on them. PCIe peer-to-peer is recommended (it also runs without, with slower prefill); no NVLink needed.
• 96 GB of host RAM (the model's n-gram embedding table lives there), /dev/shm of at least 1 GB, and about 250 GB of free disk (the checkpoint is 115 GiB).
• Linux x86-64 with glibc 2.34 or newer, an NVIDIA driver 580 or later (CUDA 13.0), Python 3.13 with venv and headers, a C/C++ compiler, ninja, git and curl. The build installs its own CUDA 13.0 toolchain into the venv.

Quick start

The hf command is the Hugging Face CLI (pipx install huggingface_hub).

git clone --branch v2.5.1-swift1.5 https://github.com/halt95/Swift1.5-qwen38-flash-next-3090s.git
cd Swift1.5-qwen38-flash-next-3090s
release/install-env.sh                      # the Flash-Next v2.5.1 engine, pinned and hash-checked, into ./engine
hf download halt95/Swift1.5-Qwen3.8-Flash-Next-W4A16-Merlin \
  --revision 3c9a1159eac1282463b06313de7dd9262edf8b58 --local-dir ~/models/Swift1.5-Merlin   # 115 GiB
(cd ~/models/Swift1.5-Merlin && sha256sum -c -) < release/checkpoint.sha256  # the 41 checkpoint files

serve/serve.sh ~/models/Swift1.5-Merlin

It is ready when the log shows Application startup complete. It serves on 127.0.0.1:8000; set VLLM_API_KEY before exposing it with HOST=0.0.0.0. Then talk to it like any OpenAI endpoint:

curl http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model": "flash-next", "messages": [{"role": "user", "content": "Hello!"}]}'

The first start compiles kernels (about 5–6 minutes); later starts take about 2–3 minutes. serve/serve.sh makes xhigh the default reasoning effort; a request can still send its own. A container recipe, a health check and troubleshooting are in the companion repository's reference.

New in v2.5.1

v2.5.1 is an engine release: the weights are unchanged since 2026-09-28. Downloaded before 2026-10-06? Re-fetch tokenizer.json, README.md, NOTICE and SHA256SUMS. tokenizer.json is now the upstream file. v2.5.0 was not published, so this is everything since v2.2.0.

• Agent follow-up turns resume on a 64-token grid instead of a 4,096-token block, and a conversation's cached prefix is kept until its next turn: a follow-up starts in 0.96 s.
• A larger KV pool (924,993 tokens, up from 806,792) on the same weights.
• Cold prefill at 5,206–5,473 tok/s (10K–100K), correct tokenization of combining marks (transformers 5.18.0), up to 42 images per request.
• Single-stream decode is about 3 % slower than v2.2.0 (the engine's measurement; not re-measured on this checkpoint).

Full measurements: the v2.5.1-swift1.5 bench card.

What's inside

We compared Swift 1.5 (revision 0bd4fe22431372cdad1979267d3ab45aa7e6150a) with Qwen/Qwen3.8-Flash-Next (revision de4b8e4d43b917e7706784d8bb445c9af86a3540) tensor by tensor: only the attention q/k/v/o_proj (12 layers), the GDN in_proj_qkv / in_proj_z / out_proj (36 layers) and the shared experts (48 layers) changed, 300 tensors. The MTP head was checked with a full SHA-256 of every tensor; the other unchanged groups with whole-shard or sampled hashes (a sampled match is strong evidence, not proof). The KV scales were recalibrated: Swift 1.5 changed k_proj / v_proj, so the Merlin sidecar does not carry over.

This checkpoint needs the companion repository's engine; stock vLLM was not tested with it.

Good to know

• Tested on one setup: 4× RTX 3090 at 220 W, PCIe Gen4 x16. Without peer-to-peer it works, with lower prefill.
• Only if you shrink the KV pool yourself: keep it larger than --max-model-len plus a few blocks. With the shipped settings (a 924,993-token pool, requests up to 262,144 tokens) this cannot occur.
• With 8 long agent sessions and a full pool, a few follow-ups per 5 minutes still re-read their whole conversation.
• Quality evidence is GSM8K-200 and a repetition-loop check at xhigh (0 of 72, on v2.2.0); UkisAI's published evaluations are of the BF16 model, not of this quantised checkpoint. We recommend xhigh, as UkisAI's evaluations use.

Licences

• The Swift contribution is under the Swift Open License v1.0 (LICENSE). Commercial Use by a Legal Entity whose gross revenue meets or exceeds US$1,000,000 (over the most recently completed fiscal year, across that entity and all entities controlling, controlled by, or under common control with it) is not licensed under it; a Qualified Non-Profit Organization's Non-Commercial or Research Purposes are exempt from the threshold, and an entity above it may obtain a separate written Swift Enterprise License from UkisAI. See Swift Open License §§1 and 5.
• The base-model portions are under the Qwen Community License 1.0 (LICENSE-QWEN). A commercial product or service with more than 100,000,000 monthly active users or more than US$20,000,000 monthly revenue must display the model name prominently in its user interface. A licensee or affiliate conducting a Model as a Service or AI Work Assistant business must obtain a separate Qwen licence before any commercial use, except for qualifying internal use that makes neither the software, its outputs, nor its model capabilities available to a third party. See Qwen Community License §§1–2.
• NOTICE carries UkisAI's notice, the credits and the changes made in this derivative, file by file.
• "UkisAI" and "Swift" are UkisAI's names, used here only to describe where the weights come from. This section summarises the licences; it is not legal advice, and the licence texts govern.

Credits

Qwen (model), UkisAI (Swift 1.5 post-train), Intel (AutoRound experts), RadixArk (FP8 PLE table), DominikBucko (MTP packing recipe), the vLLM community PLE-offload work, and aikitoria (peer-to-peer on consumer Ampere). Upstream fixes are credited by PR number in the engine repository's commit messages and by author in its NOTICE.