JevCodeLocator-0.6B

A 0.6B code localizer. Given a repository and a natural-language query (an issue, a symptom, a symbol, an error message, or a described behavior), it reranks candidate code locations and returns a ranked list of path:start-end chunks together with a file-level ranking.

The model is a reranker, not a retriever: a cheap sparse retriever (BM25 plus symbol/path channels) produces a shortlist of K candidates and the model scores all of them in a single forward pass. There are no embeddings and no vector index. On an entry-level 6 GB GPU it runs in roughly 0.4 s at K=8 and 0.9 s at K=16, which makes it usable as a local first-stage localizer for coding agents.

Architecture

state (task / repo / candidate_count / query)  +  candidate texts ("path:start-end: summary")
        │
        ├─ one leaf-token path per candidate  ──►  Qwen3-0.6B backbone
        │                                          (last hidden state at the EOS of each path)
        ├─ LayerNorm → Linear(d, 1)                    per-candidate logit z_i
        └─ set-attention over the K candidates:
             u     = Linear(d + 1, 128)([h_i ; log K])
             mixed = MultiheadAttention(u, u, u)       (key-padding mask for empty slots)
             z_i  += Linear(128, 1)(tanh(u + mixed))
        │
        └─ softmax over the K candidates

The scalar head alone can rank candidates independently; the zero-initialized set-attention correction is what makes a candidate's score depend on the whole shortlist (for example, suppressing a cluster of look-alike test files in favour of the implementation file).

Training

Three consecutive fine-tuning stages on top of Qwen/Qwen3-0.6B:

• Objective: cross-entropy against the gold candidate (gold_distribution) plus the file-level auxiliary loss (weight 0.30).
• Candidate text is identical at training and serving time: a compact summary consisting of the path, the signature and the first lines of the chunk.
• A Python replay set (2,286 rows) is kept in every round so that Python behaviour does not regress.
• Training queries are generated programmatically from doc comments, symbol names and constants. No human-written query set was used for training.

Evaluation

Frozen held-out pools: 200 queries per repository, K=8, gold candidate forced into the pool. file@1 is the share of queries whose gold file is ranked first. The baseline is the same model before the multilingual stages (stage 1 only).

Restricted to semantic (doc-comment) queries, file@1: Go 0.524 → 0.913, C++ 0.629 → 0.814, TypeScript 0.667 → 0.758, Rust 0.833 → 0.875.

On a set of five hand-checked, real reverse-proxy questions about a large Go service (K=16, FP32): gold-file recall@1 = 1.000, recall@3 = 1.000, recall@5 = 1.000. Latency on an entry-level 6 GB GPU: 0.40–0.44 s (K=8), ~0.83 s (K=16).

Usage

pip install torch transformers safetensors
python inference_example.py           # runs on this directory

import torch

from modeling_jev import load_jev_model, score_candidates

model, tok = load_jev_model(".", device="cuda" if torch.cuda.is_available() else "cpu",
                            dtype=torch.float32)

candidates = {
    "src/auth/session.py:41-88": "def create_session(user, password) | validates credentials ...",
    "src/db/pool.py:12-60": "def connect(dsn, max_size) | opens the database pool ...",
}

for key, prob in score_candidates(model, tok, "where are credentials validated?", candidates):
    print(f"{prob:.4f}  {key}")

score_candidates returns the candidates ranked by probability. Summing the probabilities of all candidates that belong to the same file gives the file-level ranking.

Files

Intended use and limitations

• Use it as a reranker. It selects among the candidates it is given; if the retriever's shortlist does not contain the gold file, the model cannot recover it. Pool recall is the ceiling.
• Trained on programmatically generated queries (doc comments, symbols, constants) with a small amount of issue-style data mixed in. Free-form questions work but are the hardest distribution.
• Language coverage: Python, Go, TypeScript/JavaScript, C, C++, Rust, C#, PHP, Java, Ruby, Bash. The chunker used to build candidates is tree-sitter based; other languages are untested.
• Candidate text is a compact summary, so very long functions are effectively truncated.
• Not instruction-tuned, not a chat model, no safety tuning. Do not use it to make decisions about code it cannot see.

Training data and licensing

The weights in this repository are released under Apache-2.0, the same license as the base model Qwen/Qwen3-0.6B (Apache-2.0, Copyright 2025 Alibaba Cloud). The inference code here (modeling_jev.py, inference_example.py, patch_tokenizer_for_transformers4.py) is original work under the same license; see LICENSE and NOTICE.

Training rows are generated programmatically: a tree-sitter chunker extracts code locations, queries are derived from doc comments, symbol names and constants, and the candidate text is a compact summary of the chunk. No source file from any repository is redistributed in this repository.

Licenses of the source repositories used for training and evaluation data:

A detailed list of the source repositories is available on request.

Notes:

• The weights are not a redistribution of those repositories, and the model does not reproduce training code verbatim: it only scores candidate locations supplied by the caller. Whether copyleft-licensed training data extends to model weights is legally unsettled in most jurisdictions.
• Commercial use: the training mix includes one AGPL-3.0 project and one non-commercial project. For a commercially clean release, retrain without those two (and preferably without the GPL/LGPL projects); the data pipeline is deterministic and reproducible.
• You remain responsible for complying with the licenses of the code you run the model on.
• This section is provenance information, not legal advice.