Llama-2-70b-QuIP-HARP-2Bit
2-bit quantized version of meta-llama/Llama-2-70b-hf, produced with HARP (Hadamard-Preconditioned Adaptive Rotations), a learnable structured orthogonal incoherence processor for extreme LLM quantization.
- Paper: HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization (EMNLP 2026)
- Code: brain-lab-research/HARP
- Backend: QuIP# (
harp_quip/), HARP as a drop-in replacement for the fixed randomized Hadamard/RHT incoherence-processing stage - Collection: brain-lab/harp-quantized-models
What is HARP?
Ultra-low-bit PTQ methods such as QuIP# rely on a randomized Hadamard transform (RHT) to make weights more incoherent before quantization. HARP replaces this fixed transform with a learned structured orthogonal processor. It starts from a Hadamard-style initialization and is fit only on calibration data, so no model retraining is needed. Stored in int8, it adds negligible bits-per-parameter overhead. Everything else in the QuIP# pipeline (codebooks, LDLQ solver, HF export, inference) stays unchanged.
Results
WikiText2 / C4 perplexity at ~2 bits, context length 4096 (BPP includes HARP processor storage):
| Method | BPP↓ | Wiki2↓ | C4↓ |
|---|---|---|---|
| QuIP# (fixed RHT) | 2.00 | 4.16 | 6.01 |
| QuIP# + HARP (this model) | 2.02 | 4.01 | 5.82 |
Usage
This checkpoint uses QuIP#-style packed weights and requires the harp_quip inference code (a fork of QuIP#) rather than plain transformers.AutoModelForCausalLM:
git clone https://github.com/brain-lab-research/HARP
cd HARP/harp_quip
pip install -r requirements.txt
cd quiptools && python setup.py install && cd ..
from lib.utils.unsafe_import import model_from_hf_path
model, model_str = model_from_hf_path("brain-lab/Llama-2-70b-QuIP-HARP-2Bit")
See harp_quip/eval/eval_ppl.py and harp_quip/eval/eval_zeroshot.py for perplexity / zero-shot evaluation scripts, and the main repo README for the full quick-start guide.
License
Governed by the Llama 2 license. This is a derivative of Meta's Llama 2 70B; use is subject to Meta's Llama 2 Community License Agreement.
Citation
@article{zagitov2026harp,
title={HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization},
author={Artur Zagitov and Gleb Molodtsov and Aleksandr Beznosikov},
year={2026},
eprint={2605.29843},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2605.29843},
}