SPLADEX — best_proxy checkpoint
This is the best_proxy checkpoint from an inference-free SPLADE-v3-doc layer-pruned training run
pruned from Cdn13/splade-multi-static-doc.
Model description
- Architecture: SPLADE-v3-doc (MLM head,
log1p(ReLU(logits)).max()document pooling) - Query encoder: Inference-free — static per-token weights (
static_query_weights.pt) initialized from IDF and learned during training. - Document encoder: Full SPLADE document side (pruned transformer layers).
- Selection criterion: Best window-average selection proxy during training (top-1 ranking score − sparsity budget penalty).
Files
| File | Description |
|---|---|
config.json | Model config (HF format) |
model.safetensors / pytorch_model.bin | Document-encoder weights |
tokenizer* | Tokenizer files |
static_query_weights.pt | Learned static query token weights |
trainer_state.pt | Optimizer / scheduler state + training metrics at best step |
Usage
import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM
repo = "Cdn13/static-splade-trained-pruned"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForMaskedLM.from_pretrained(repo)
# Load static query weights
sqw = torch.load("static_query_weights.pt", map_location="cpu")
query_weights = sqw["query_weights"] # shape: [vocab_size]
Note: The query representation is
presence(token) * query_weights[token], computed without any forward pass through the model.