SlimKV-Qwen2.5-14B-r4
A SlimKV variant of Qwen2.5-14B-Instruct with 4× feature-wise compression of beacon KV representations, corresponding to SlimKV-f4 in the paper.
Part of the SlimKV collection.
Compression settings
SlimKV jointly compresses the token dimension and the feature dimension of the KV cache. The r2/r4/r8 suffixes in these repository names specify feature-wise compression of beacon KV representations, corresponding to SlimKV-f2/f4/f8 in the paper.
| Repository suffix | Paper variant | Feature-wise compression |
|---|---|---|
| r2 | SlimKV-f2 | 2× |
| r4 | SlimKV-f4 | 4× |
| r8 | SlimKV-f8 | 8× |
Token-wise compression is configured separately through the beacon interval.
In the paper's tnfm notation, n is the token-wise compression factor and m is the feature-wise compression factor.
The nominal total compression factor is token-wise factor × feature-wise factor. Examples include:
| Total compression | Token-feature settings |
|---|---|
| 8× | t4f2 or t2f4 |
| 16× | t8f2, t4f4, or t2f8 |
| 32× | t16f2, t8f4, or t4f8 |
Each model fixes its feature factor; the token factor is configurable through the Beacon interval. The product describes compressed historical Beacon memory; full-dimensional raw-token tails and sinks affect the realized cache-memory reduction.
Layer-specific latent ranks are allocated adaptively under a global feature budget (Section 3.3). SlimKV applies low-rank compression to the beacon branch and performs reconstruction-free latent-space attention over beacon memory during decoding.
Files and usage
Model weights, configuration, and tokenizer files are included in this repository. Use the corresponding SlimKV inference implementation to load the custom model components. Loading with an unmodified Transformers implementation has not been verified.
License and modifications
This model is distributed under the Apache License 2.0, following the upstream Qwen2.5-14B license. The model weights and configuration have been modified for SlimKV; this repository is not the unmodified upstream Qwen release. See MODEL_MODIFICATIONS.md.
Paper
SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention. See Sections 3.3 and 5.1 and Table 2 for compression definitions.