depguard-jev-style-1.5b
Local System One model for dependency security review. JEV-compatible typed verdicts, no API key required.
Drop-in local alternative to TypeSafe Jev for supply-chain review pipelines. Returns safe/caution/block per package update — swap the endpoint, keep the agent.
Overview
depguard-jev-style-1.5b is a fine-tuned Qwen2.5-1.5B-Instruct that produces typed dependency security verdicts — matching the output contract of TypeSafe System One / Jev for supply-chain review use cases.
It outputs a single JSON object per package update:
{
"verdict": "safe | caution | block",
"reason": "<one sentence>",
"cve_count": 0,
"recommendation": "upgrade | pin | replace | accept | reject"
}
Feed it Dependabot PRs, Renovate changelogs, or npm audit output. Get a typed decision the agent can act on — auto-merge safe updates, block risky ones, queue caution for human review.
Why local?
| Jev (TypeSafe API) | depguard-jev-style-1.5b | |
|---|---|---|
| Typed output | yes | yes |
| Latency | ~125ms (network) | ~82ms (local, MPS) |
| Cost | per-request billing | free after download |
| Data leaves org | yes | no |
| Fine-tune on internal allowlist | no | yes (LoRA) |
| Accuracy (OSV-Bench) | 96.1% F1 | 95.7% F1 |
| Block precision (malicious pkg holdout) | 94.3% | 95.1% |
Dependency metadata often contains internal package names and version strings that reveal your technology stack. Run the classifier locally.
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch, json, re
tok = AutoTokenizer.from_pretrained("enterprise-ai-lab/depguard-jev-style-1.5b")
model = AutoModelForCausalLM.from_pretrained(
"enterprise-ai-lab/depguard-jev-style-1.5b", dtype=torch.float32
).eval()
SYS = (
"You are a dependency security review assistant. For each package update output ONE "
"JSON object with keys: verdict (safe|caution|block), reason (one sentence), "
"cve_count (integer), recommendation (upgrade|pin|replace|accept|reject). "
"Output only the JSON."
)
def review_dep(name, registry, old_ver, new_ver, description, license):
content = (f"Package: {name}\nRegistry: {registry}\n"
f"Current: {old_ver} -> Proposed: {new_ver}\n"
f"Description: {description}\nLicense: {license}")
msgs = [{"role": "system", "content": SYS}, {"role": "user", "content": content}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt")
with torch.no_grad():
out = model.generate(**ids, max_new_tokens=120, do_sample=False,
pad_token_id=tok.eos_token_id)
text = tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True)
m = re.search(r"\{.*\}", text, re.DOTALL)
return json.loads(m.group(0)) if m else {}
result = review_dep("lodash", "npm", "4.17.15", "4.17.21",
"Utility library", "MIT")
# {"verdict": "safe", "reason": "Security patch for CVE-2021-23337.", "cve_count": 1, "recommendation": "upgrade"}
Renovate / Dependabot integration
Wire it as an OpenAI-compatible endpoint and drop it into your existing approval agent:
# Before (Jev):
llm = ChatOpenAI(base_url="https://api.typesafe.ai/v1", api_key=JEV_KEY, model="jev-latest")
# After (DepGuard local):
llm = ChatOpenAI(base_url="http://localhost:8000/v1", api_key="not-needed", model="depguard-jev-style-1.5b")
Auto-merge pattern:
verdict = review_dep(name, registry, old_ver, new_ver, desc, lic)
if verdict["verdict"] == "safe" and verdict["cve_count"] == 0:
merge_pr(pr_id)
elif verdict["verdict"] == "block":
close_pr(pr_id, reason=verdict["reason"])
else:
assign_for_review(pr_id)
Training
Fine-tuned with LoRA (r=16, alpha=32) on a curated dependency security corpus:
- 92,000 package updates from PyPI, npm, Maven, and Go modules (2018-2024)
- Ground truth: OSV database, GitHub Advisory Database, manual security team labels
- Includes: known malicious packages (event-stream, ua-parser-js, colors 1.4.44-liberty, etc.)
- Held-out validation: 4,800 packages stratified by verdict
- Training: 12 epochs, AdamW lr=2e-4, MPS/CUDA
Benchmarks
OSV-Bench (CVE-annotated package updates, n=4,800)
| Model | Precision | Recall | F1 |
|---|---|---|---|
| depguard-jev-style-1.5b | 96.0% | 95.4% | 95.7% |
| Jev (TypeSafe API) | 96.4% | 95.9% | 96.1% |
| GPT-4o-mini (zero-shot) | 90.2% | 89.6% | 89.9% |
| OSV-Scanner (rule-based) | 98.1% | 71.3% | 82.7% |
Malicious Package Detection (supply-chain attack samples)
| Model | Block Rate | False Positive Rate |
|---|---|---|
| depguard-jev-style-1.5b | 95.1% | 4.9% |
| Jev (TypeSafe API) | 94.3% | 5.7% |
| OSV-Scanner (rule-based) | 61.2% | 1.1% |
Note: rule-based scanners miss novel malicious packages not yet in the CVE database. The LLM classifier catches behavioral and metadata signals that signature-based tools miss.
Intended use
- Dependabot / Renovate PR auto-approval pipelines
- Supply-chain security gates in CI/CD
- Package allowlist enforcement
- Air-gapped environments (no external CVE API calls needed)
Limitations
- Verdict is based on metadata and description — not static analysis of package contents
- Novel malicious packages with clean metadata may score safe
- Combine with OSV-Scanner for defense in depth
License
Apache 2.0. Base model (Qwen2.5-1.5B-Instruct) is subject to its own Qwen license.