Referential Dangling Dependency Detector
This is the sentence-pair dependency detector released with Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression.
Project page: https://cslikai.cn/Referential-Dangling/
The model scores whether a candidate sentence supplies a necessary dependency for a retained sentence, conditioned on the question. It is used by the repository's automatic context-restoration experiments.
Authors
Zhengpei Hu1,∗, Kai Li2,∗, Dapeng Fu3, Xuechao Zou2, Yuanhao Tang1, Yue Li1, Tengfei Cao1, and Jianqiang Huang1,†
- 1School of Computer Technology and Application, Qinghai University
- 2Tsinghua University
- 3Ant Group Security and Intelligence Laboratory (SIL)
- ∗Equal contribution; †corresponding author
Model details
- Architecture: BERT sequence classifier with two labels
- Base model:
google-bert/bert-base-uncased - Labels:
NOT_DEPENDENCY(0),DEPENDENCY(1) - Maximum training input length: 256 tokens
- Input format:
retained sentence [SEP] candidate support [SEP] question
Training data
Training pairs were constructed from the HotpotQA training split. Positive pairs contain a retained sentence and a missing gold-support sentence that share a discriminative entity. Negatives include entity-overlapping hard negatives and unrelated deleted sentences. Splitting is grouped by source example to prevent sentence pairs from the same example appearing in both the training and validation partitions.
See src/build_train_tight.py and src/train_detector.py in the
Referential-Dangling repository
for the data construction and training code.
Usage
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
model_id = "JusperLee/referential-dangling-detector"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).eval()
retained = "The film was directed by Jane Smith."
candidate = "Jane Smith is a Canadian filmmaker."
question = "What nationality is the film's director?"
text = f"{retained} [SEP] {candidate} [SEP] {question}"
inputs = tokenizer(text, truncation=True, max_length=256, return_tensors="pt")
with torch.no_grad():
probability = model(**inputs).logits.softmax(dim=-1)[0, 1].item()
print(probability)
For the paper's restoration pipeline, use BertDependencyDetector from
src/beaver2_bert.py.
Intended use and limitations
This checkpoint is intended for research on dependency loss and automatic support restoration in compressed English QA contexts. It is not a general factuality, entailment, or coreference model. Its predictions depend on the candidate-generation procedure and may not transfer reliably to other domains, languages, or substantially different compression settings without evaluation.
Citation
@misc{referentialdangling,
title={Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression},
author={Zhengpei Hu and Kai Li and Dapeng Fu and Xuechao Zou and Yuanhao Tang and Yue Li and Tengfei Cao and Jianqiang Huang},
note={Research code and model release}
}