Llama-3.1-8B-SlimKV-r4
A SlimKV variant of Llama-3.1-8B-Instruct with 4× feature-wise compression of beacon KV representations, corresponding to SlimKV-f4 in the paper.
Part of the SlimKV collection.
Compression settings
SlimKV jointly compresses the token dimension and the feature dimension of the KV cache. The r2/r4/r8 suffixes in these repository names specify feature-wise compression of beacon KV representations, corresponding to SlimKV-f2/f4/f8 in the paper.
| Repository suffix | Paper variant | Feature-wise compression |
|---|---|---|
| r2 | SlimKV-f2 | 2× |
| r4 | SlimKV-f4 | 4× |
| r8 | SlimKV-f8 | 8× |
Token-wise compression is configured separately through the beacon interval.
In the paper's tnfm notation, n is the token-wise compression factor and m is the feature-wise compression factor.
The nominal total compression factor is token-wise factor × feature-wise factor. Examples include:
| Total compression | Token-feature settings |
|---|---|
| 8× | t4f2 or t2f4 |
| 16× | t8f2, t4f4, or t2f8 |
| 32× | t16f2, t8f4, or t4f8 |
Each model fixes its feature factor; the token factor is configurable through the Beacon interval. The product describes compressed historical Beacon memory; full-dimensional raw-token tails and sinks affect the realized cache-memory reduction.
Layer-specific latent ranks are allocated adaptively under a global feature budget (Section 3.3). SlimKV applies low-rank compression to the beacon branch and performs reconstruction-free latent-space attention over beacon memory during decoding.
Files and usage
Model weights, configuration, and tokenizer files are included in this repository. Use the corresponding SlimKV inference implementation to load the custom model components. Loading with an unmodified Transformers implementation has not been verified.
License and attribution
Built with Llama.
This SlimKV variant contains modified Llama 3.1 model weights and remains subject to the Llama 3.1 Community License and Acceptable Use Policy. The upstream Llama materials are not licensed under Apache 2.0. See NOTICE for the required upstream attribution.
Paper
SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention. See Sections 3.3 and 5.1 and Table 2 for compression definitions.