Swift-Qwen3.8-27B — MTPLX 4-bit

MTPLX / MLX conversion of ukisai/Swift-Qwen3.8-27b, built for fast local inference on Apple Silicon.

This build was created with MTPLX Forge and keeps Swift's native MTP head in BF16 for self-speculative decoding.

The model body is quantized to 4-bit affine, group size 64.

Highlights

• Base model: ukisai/Swift-Qwen3.8-27b
• Runtime: MTPLX / MLX
• Body quantization: 4-bit affine
• Group size: 64
• MTP head: BF16
• Maximum MTP depth: D3
• Vision weights: included
• Tested hardware: Apple M5 Max
• Tested memory: 128 GB unified memory
• MTPLX version used for Forge verification: 2.11.2
• Published artifact size: ~16.9 GB

Performance

Forge verification was run locally on an Apple M5 Max with 128 GB unified memory.

Recommended depth on this machine: D3.

The D3 result improves measured decode throughput from 29.4 tok/s to 65.2 tok/s, a 2.22× speedup over autoregressive decoding.

Verification sampler

temperature = 1.0
top_p      = 0.95
top_k      = 20

The complete Forge verification data is included in:

mtplx_runtime.json

Benchmark scope

These numbers measure decode throughput for this MTPLX conversion.

They should not be interpreted as:

• an end-to-end Swift-vs-Qwen3.8 task-speed benchmark;
• a quality comparison against the original BF16 checkpoint;
• a claim that every prompt will decode at 65.2 tok/s.

Actual throughput depends on prompt length, context size, generation length, sampling parameters, thermal conditions, and workload.

The Forge verification reported successful quality checks for AR, D1, D2 and D3.

Quantization recipe

This checkpoint was built with:

Body bits:       4-bit
Group size:      64
Quantization:    affine
Body dtype:      auto
MTP strategy:    keep BF16
MTP depth max:   3

In compact form:

Swift-Qwen3.8-27B
       │
       ├── model body → MLX 4-bit affine / g64
       │
       ├── MTP head   → BF16
       │
       └── vision     → preserved
       │
       ▼
Swift-Qwen3.8-27B-MTPLX

MTP

Swift retains the native multi-token prediction head from its Qwen3.8 architecture.

MTPLX can use this head as a self-speculative drafter: candidate tokens are predicted ahead and verified by the target model.

On the tested M5 Max, Forge selected MTP depth 3 as the fastest configuration.

The MTP head is intentionally kept in BF16 in this build rather than quantized.

Vision

The Forge conversion contains the vision weights in:

model-vision.safetensors

The published artifact therefore preserves the multimodal components of the source checkpoint.

The performance numbers above are text-generation decode measurements. Multimodal quality and image-processing throughput have not been independently benchmarked for this release.

Installation

Install MTPLX on Apple Silicon:

brew install youssofal/mtplx/mtplx

Download this model:

mtplx pull arsis-dev/ukisai-Swift-Qwen3.8-27b-MTPLX

Then start MTPLX:

mtplx start

Select this model when prompted.

You can also inspect the checkpoint before running it:

mtplx inspect arsis-dev/ukisai-Swift-Qwen3.8-27b-MTPLX

Retune for your Mac

The optimal MTP depth can depend on the Apple Silicon generation, memory bandwidth, thermal state and workload.

To benchmark AR vs D1/D2/D3 on your own machine:

mtplx tune \
  --model arsis-dev/ukisai-Swift-Qwen3.8-27b-MTPLX \
  --retune

Do not assume D3 will necessarily be optimal on every Mac.

API usage

MTPLX can expose the model through an OpenAI-compatible local API.

Start the server:

mtplx serve --port 8000

The default local endpoint is:

127.0.0.1:8000

This can be used with clients supporting OpenAI-compatible chat-completion APIs.

Files

Important files in this repository include:

model-00001-of-00003.safetensors
model-00002-of-00003.safetensors
model-00003-of-00003.safetensors
model-vision.safetensors
mtp.safetensors
model.safetensors.index.json
mtplx_runtime.json
config.json
generation_config.json
chat_template.jinja
tokenizer.json

mtp.safetensors contains the BF16 MTP sidecar used for speculative decoding.

mtplx_runtime.json contains Forge provenance and the local verification results.

Provenance

Source model:

ukisai/Swift-Qwen3.8-27b

Source revision used by Forge:

1b30aaaf753fe5c1cb51ada2ea0367a53445359c

Conversion:

MTPLX Forge 2.11.2

This repository is a quantized runtime conversion. It is not a new fine-tune and does not alter the training performed by UkisAI.

About Swift

Swift-Qwen3.8-27B is an efficient-thinking derivative of Qwen3.8-27B developed by UkisAI.

The purpose of this repository is only to provide an Apple-Silicon-oriented MTPLX/MLX conversion of that model.

For model architecture, training methodology, evaluation results, intended use and limitations, refer to the original model card:

ukisai/Swift-Qwen3.8-27b

License

This conversion inherits the licensing conditions of the original Swift checkpoint.

Swift weights are distributed under the Swift Open License v1.0.

Please review the License and access section of the original:

ukisai/Swift-Qwen3.8-27b

before downloading, redistributing or using these weights commercially.

The MTPLX software itself is a separate project and has its own software license.

Validation status

Validated:

• model loads through the MTPLX Forge verification pipeline;
• autoregressive decoding works;
• MTP D1 works;
• MTP D2 works;
• MTP D3 works;
• speculative acceptance remains high;
• D3 outperforms the AR baseline on the tested M5 Max;
• vision weights are present in the converted checkpoint.

Not yet independently evaluated for this conversion:

• full benchmark-suite parity with upstream BF16 Swift;
• long-context quality;
• multimodal benchmark quality;
• quantization-induced quality loss;
• end-to-end Swift-vs-Qwen3.8 task completion time.

Contributions and independent benchmark results are welcome.

Credits

Original model:

UkisAI — Swift-Qwen3.8-27B

Apple Silicon runtime and Forge tooling:

MTPLX — Youssofal

MTPLX conversion and benchmark:

arsis-dev

Thanks to the original model authors and the MTPLX project for making this conversion possible.