Eklav-4B-Reranker-CotGen (CotGen baseline)

Eklav trains a model to pick up a teacher's reasoning mid thought rather than imitate it end to end. The student sees a partial reasoning trace from the teacher, with the answer revealing tail removed, and learns to continue reasoning and produce the answer on its own. The model's own reasoning is conditioned on the teacher's partial trace during training rather than trained to reproduce it word for word. Same base model, same training data as standard full trace CoT distillation, only the training objective changes.

Highlights

• Standard full trace CoT SFT baseline used to measure Eklav's improvement at this scale

Model details

Results

nDCG@10 on BRIGHT, single evaluation run per domain.

Use as a reranker

This is a pointwise reranker, the same style as Rank1 (jhu-clsp/rank1-7b, which this checkpoint's training recipe reproduces): the model generates a reasoning trace ending in </think> true or </think> false, and relevance is scored from the logits at that final token rather than by parsing generated text, which avoids depending on the model reliably stopping on its own. This checkpoint can generate a long reasoning trace before reaching </think>, so use vLLM with a stop string rather than a fixed transformers.generate token budget, the same setup Rank1's own card recommends and the one used to produce the results on this page.

from vllm import LLM, SamplingParams
import math

model_id = "AdarshSingh7647/Eklav-4B-Reranker-CotGen"
model = LLM(model=model_id, max_model_len=16000)
tokenizer = model.get_tokenizer()

def create_prompt(query: str, passage: str) -> str:
    return (
        "Determine if the following passage is relevant to the query. "
        "Answer only with 'true' or 'false'.\n"
        f"Query: {query}\n"
        f"Passage: {passage}\n<think>"
    )

sampling_params = SamplingParams(
    temperature=0,
    max_tokens=8192,
    logprobs=20,
    stop=["</think> true", "</think> false"],
)

def score(query: str, passage: str) -> float:
    prompt = create_prompt(query, passage)
    output = model.generate([prompt], sampling_params)[0].outputs[0]
    # the answer token is usually the second to last logprob step (vLLM's stop
    # string match can consume one extra token, e.g. <|im_end|>, after it), but
    # scan from the end so this is robust to that off by one
    for step in reversed(output.logprobs or []):
        true_lp = next((v.logprob for k, v in step.items() if tokenizer.decode([k]).strip().lower() == "true"), None)
        false_lp = next((v.logprob for k, v in step.items() if tokenizer.decode([k]).strip().lower() == "false"), None)
        if true_lp is not None and false_lp is not None:
            true_score, false_score = math.exp(true_lp), math.exp(false_lp)
            return true_score / (true_score + false_score)
    return 0.5

query = "What causes seasons on Earth?"
passages = [
    "Seasons are caused by the tilt of Earth's axis relative to its orbit around the Sun.",
    "The Great Wall of China is visible from space, according to popular belief.",
]
ranked = sorted(passages, key=lambda p: score(query, p), reverse=True)
for p in ranked:
    print(p)

We tested a plain transformers.generate loop with a fixed token budget on this checkpoint and saw it occasionally run past a few hundred tokens without closing </think>, degenerating into repetition instead of answering, so we recommend the stop-string based vLLM setup above rather than a fixed max_new_tokens cutoff.