Gemma-4-31B: Cybersecurity Safety Repair

A safety repair of google/gemma-4-31B-it that reduces attacker-useful cybersecurity output. Released gated for evaluation on your own workloads, not certified for deployment; see Results for the full safety/utility trade-off.

What we did

Lexsi turns a known safety failure into a controlled model change. Instead of retraining the whole model or wrapping it in a guard, router, or inference-time filter, we locate the specific behaviour in the weights, apply a bounded, circuit-restricted edit, and verify the fix holds without eroding the capabilities you depend on. You get one drop-in checkpoint: the safety change lives in the model and is served the way you serve it today, with nothing extra to run or maintain.

Every repair is a measured trade-off. We report the safety gain and its collateral effects on legitimate use and capability side by side (see Results), so you can decide whether to deploy from the evidence.

This checkpoint applies that approach to cybersecurity: reducing attacker-useful assistance while keeping the model useful for legitimate, defensive security work.

Built using the Lexsi Alignment and Safety Stack:

The repair workflow behind this checkpoint (Figure 2 of the paper). We fix the specification before intervention; verification tests both the intended gain and the behavioural blast radius before any deployment decision. The four libraries below implement it.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Lexsi/Gemma-4-31B-it-Cybersecurity-Safety-Repair"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "How do I configure fail2ban to block repeated SSH login attempts?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to(model.device)

out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

We evaluated with greedy decoding in bfloat16; the results below use that setting.

Results

Base checkpoint (google/gemma-4-31B-it) vs. this repaired checkpoint, under the cybersecurity study's harness. The repair is a genuine safety gain on attacker-facing tasks, offset by a cost on one class of legitimate requests: a trade-off to weigh for your use case.

Safety: attacker-useful responses (%, lower is safer)

Harmful-request refusal (%, higher is safer):

Multi-turn attacks (%, lower is safer):

Cost: refusal of legitimate requests (%, lower is better)

This is the trade-off to weigh. Refusal of everyday legitimate security requests (CyberSecEval-FRR) roughly doubles, and OR-Bench-Hard also rises, while FORTRESS-benign and XSTest barely move; a general-capability sweep misses the cost. Weigh this against the large attacker-useful reduction above when deciding where the checkpoint fits your use case.

Capability preserved (base → this checkpoint)

The accompanying report gives the full multi-benchmark picture, the judge protocol, and why domain-matched benign evaluation, not general-capability checks, surfaces the legitimate-request cost.

Intended use

Research and evaluation of attacker-useful output suppression. It is a candidate for general-purpose assistants whose users are not security professionals, once the operator has run its own benign-workload tests.

License

This is a derivative work of google/gemma-4-31B-it with selected weights modified. The NOTICE file states what was changed. Google releases the base model under the Apache License 2.0; a copy is included as LICENSE-APACHE-2.0.txt.

Lexsi's modifications (the weight correction, the recipe, and this card) are offered under the Lexsi Source Available License (LSAL) v1.2 (source-available, noncommercial). See the NOTICE file for details.

Please also follow Google's Gemma Prohibited Use Policy when using this model or anything derived from it.

Released gated for teams to evaluate and use. Dual-use: review the Results before deploying in your setting. Not independently audited.

Contact

Questions, errors in this card, and organizational-use acknowledgements (LSAL Section 1A): support@lexsi.ai

Citation

If you use this checkpoint, please cite it and the accompanying paper:

@misc{lexsi2026gemma4cyberrepair,
  title        = {Gemma-4-31B: Cybersecurity Safety Repair},
  author       = {Seth, Pratinav and Dhor, Ashim and Sadhu, Saisab and Bhattacharjee, Soham and Gosalia, Hem and
            Sankarapu, Vinay Kumar},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Lexsi/Gemma-4-31B-it-Cybersecurity-Safety-Repair}}
}

@misc{lexsi2026lastmile,
  title  = {The Last Mile of Model Safety: Targeted Model Repair},
  author = {Seth, Pratinav and Dhor, Ashim and Sadhu, Saisab and Bhattacharjee, Soham and Gosalia, Hem and
            Sankarapu, Vinay Kumar},
  year   = {2026},
  note   = {Lexsi Labs white paper}
}