🪐 Qwopus3.8-27B-Flash (W4A16)
[!NOTE] Qwopus3.8-27B-Flash W4A16 is a 4-bit weight / 16-bit activation quantized version of Qwopus3.8-27B-Flash, using AutoRound with GPTQ group size 128. It preserves the Flash model's efficiency characteristics while reducing memory footprint.
[!TIP] W4A16 Quantization · ~65% Smaller · Same Efficiency Characteristics · FP16 Parity on ARC-Easy
📊 Benchmark Results
ARC-Easy
| Model | Accuracy | Acc Norm |
|---|---|---|
| Qwopus3.8-27B-Flash (FP16) | 83.71% | 77.06% |
| Qwopus3.8-27B-Flash (W4A16) | 84.13% | 78.54% |
The W4A16 quantized version matches or slightly exceeds the FP16 original on ARC-Easy.
[!NOTE] Additional benchmarks (MMLU, HumanEval, BBH, WikiText2 PPL) are in progress. Results will be added to this README when complete.
💾 Quantization Details
| Parameter | Value |
|---|---|
| Algorithm | AutoRound + GPTQ |
| Weight bits | 4 |
| Activation bits | 16 |
| Group size | 128 |
| Version | AutoRound 0.12.3 |
Model size: ~19GB (vs ~54GB FP16) — 65% reduction
The quantization preserves the following layers in FP16 for quality:
- Linear attention projection matrices (
in_proj_a,in_proj_b) across all 64 layers - MTP (Multi-Token Prediction) module
- Visual encoder
🚀 Usage
Works with any framework supporting GPTQ/AutoRound checkpoints:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Jackrong/Qwopus3.8-27B-Flash-W4A16/Jackrong_Qwopus3.8-27B-Flash-w4g128"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype="auto"
)
Or via vLLM for high-throughput serving:
vllm serve Jackrong/Qwopus3.8-27B-Flash-W4A16/Jackrong_Qwopus3.8-27B-Flash-w4g128 \
--dtype auto --tensor-parallel-size 1
🎯 Recommended Use Cases
- Local MTP speculative decoding where decoder throughput and draft acceptance matter.
- Long-running agent workflows with repeated tool calls and iterative edits.
- Resource-conscious workloads where wall-clock time and generated-token cost are operational constraints.
- Memory-constrained environments where the 65% size reduction enables local deployment on smaller GPUs.
⚠️ Limitations
- Accuracy–Efficiency Trade-off: The underlying Flash fine-tuning prioritizes practical inference efficiency. On the reported 2,500-question MMLU-Pro mixed set, Qwopus scores 91.28% versus 92.73% for the Qwen3.8 base comparison.
- Reasoning Stability: Although the long-tail behavior is improved in the reported evaluation, edge cases may still exhibit reasoning drift, loops, or inefficient trajectories.
- Agent Dependence: Tool-use quality and task completion remain dependent on the prompt, tool environment, orchestration layer, and feedback loop.
- Experimental Release: This is an independent experimental release for research, local evaluation, and technical exploration.
🙏 Acknowledgements
Special thanks to:
- Qwen for the Qwen3.8-27B base-model foundation.
- Jackrong for the Qwopus3.8-27B-Flash fine-tuning.
- Unsloth for practical, efficient large-model fine-tuning tooling.
- The open-source community for training tools, evaluation methods, and technical discussion.