Qwen3.5-2B-SpeedX

Qwen3.5-2B-SpeedX is an experimental, text-only recurrent conversion of Qwen/Qwen3.5-2B, designed for fast autoregressive inference and constant-size recurrent memory.

The original Qwen3.5-2B text backbone mixes native Gated DeltaNet (GDN) layers with full-attention layers. SpeedX replaces the six full-attention layers with distilled native GDN layers, producing a 24/24 recurrent token-mixing stack.

This is an experimental research checkpoint, not an official Qwen release.

Highlights

• ~1.92B parameters
• 24 / 24 recurrent Gated DeltaNet layers
• Full-attention layers 3, 7, 11, 15, 19, 23 replaced with native GDN
• No token-wise full-attention KV cache in the converted text stack
• Recurrent decode state is constant-size with respect to context length
• Hugging Face Transformers compatible through custom remote code
• Designed for native Qwen GDN kernels
• Experimental Triton, CUDA Graph, and FP8 inference runtime
• Text-generation only; upstream multimodal/vision components are not included

Architecture

Original Qwen3.5-2B text pattern:

GDN → GDN → GDN → Full Attention

SpeedX:

GDN → GDN → GDN → GDN

repeated across all 24 text layers.

The recurrent state follows

[ S_t = F(S_{t-1}, x_t) ]

so decode-state memory is approximately

[ M_{\mathrm{state}} = O(1) ]

with respect to context length, instead of a token-wise KV cache whose memory grows approximately as

[ M_{\mathrm{KV}} = O(T). ]

NVIDIA L4 benchmark

Measured on one NVIDIA L4 22.03 GiB, batch size 1, greedy decoding.

Exact BF16 CUDA Graph

Experimental FP8 runtime

These results use the project's optimized runtime. Standard Hugging Face model.generate() should not be assumed to reproduce the CUDA-Graph / FP8 numbers above.

At 16K context, an optimized exact recurrent path measured approximately:

• Official Qwen3.5-2B prefill: 10.85k tok/s
• SpeedX prefill: 13.79k tok/s
• Ratio: ~1.27×

Quick start

Custom model code is included, so trust_remote_code=True is required.

pip install -U \
  "transformers @ git+https://github.com/huggingface/transformers.git@e453228ef83ce0d756f7621ef8607220ebf5da6a" \
  "kernels>=0.16.0,<0.17" \
  "accelerate>=1.1.0" \
  "safetensors>=0.8.0"

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL_ID = "summerMC/Qwen3.5-2B-SpeedX"

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="cuda",
).eval()

inputs = tokenizer(
    "Explain recurrent memory in neural networks.",
    return_tensors="pt",
).to("cuda")

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Chat example

messages = [
    {"role": "system", "content": "You are a precise technical assistant."},
    {"role": "user", "content": "Compare recurrent state memory with a Transformer KV cache."},
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(text, return_tensors="pt").to("cuda")

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
    )

print(tokenizer.decode(output[0], skip_special_tokens=True))

Conversion

The original text stack contains 18 native GDN layers and six full-attention layers.

SpeedX preserves the native GDN layers and replaces full-attention layers:

3, 7, 11, 15, 19, 23

with native Qwen3_5GatedDeltaNet modules.

The replacement layers are initialized from nearby native GDN layers and distilled against the original full-attention outputs.

The experimental layer-wise objective combines normalized MSE and cosine distance:

[ \mathcal{L}

\frac{|y_S-y_T|_2^2} {\mathbb{E}[y_T^2]+\epsilon} + \lambda \left(1-\cos(y_S,y_T)\right). ]

Limitations

Experimental distillation

This checkpoint was created primarily to validate the recurrent conversion and high-speed runtime. The smoke-distillation schedule is short, so it should not be assumed to preserve all reasoning, factual, multilingual, or benchmark quality of the original Qwen3.5-2B.

Longer distillation or continued pretraining is recommended for serious deployment.

Text only

The upstream Qwen3.5-2B is multimodal. SpeedX contains the converted text-generation stack and does not provide the original vision stack.

Custom code

Review the repository code before enabling:

trust_remote_code=True

Optimized runtime vs checkpoint

The ~60 tok/s exact and ~79 tok/s FP8 figures rely on runtime-specific optimizations including:

• native GDN kernels
• Triton fusion
• persistent CUDA Graph greedy decoding
• selective precision / FP8 execution

The normal Hugging Face loading path is primarily a compatibility path.

Intended use

SpeedX is intended for research on:

• recurrent language models
• fixed-state decoding
• KV-cache-free decoding
• long-context recurrent memory
• Gated DeltaNet architectures
• CUDA Graph inference
• recurrent-model quantization

Demo

Hugging Face Space:

summerMC/Qwen-speed

Base model

Derived from:

Qwen/Qwen3.5-2B

Please review the upstream model card and license as well.

License

Apache-2.0, following the upstream Qwen3.5-2B model metadata.

Citation

If you use SpeedX in experiments, cite the upstream Qwen work and identify this checkpoint as an experimental recurrent conversion of Qwen3.5-2B.

A dedicated SpeedX technical report has not yet been published.

SpeedX 1T Logical Context

This revision introduces an experimental 1,000,000,000,000-token logical context architecture.

Context specification

The 1T value describes a logical recurrent/streaming context space.

It does not claim that the current weights were trained using a single one-trillion-token dense sequence.

Architecture

Input stream
     |
     v
L0 Active Tokens
     |
     v
L1 GDN24 recurrent state
     |
     v
L2 compressed chunk memory
     |
     v
L3 semantic retrieval memory
     |
     v
L4 persistent archive

Long-context design

• 64-layer Gated DeltaNet recurrent architecture
• bounded-memory streaming prefill
• 64-bit absolute token positions
• recurrent-state checkpointing
• resumable context processing
• hierarchical-memory-ready runtime
• semantic/external retrieval ready
• logical context target: 1 trillion tokens

Validation progression

262K
  |
  v
1M
  |
  v
16M
  |
  v
256M
  |
  v
1B
  |
  v
16B
  |
  v
1T

A 1T native-context capability should only be claimed after end-to-end training, calibration and validation at that scale.