Surogate-3.5-2B
Surogate-3.5-2B is a Romanian-first bilingual assistant developed by Invergent. It is optimized for Romanian instruction following, reasoning, math, and orthographic correctness. English remains supported.
Thinking support: explicit <think>...</think> reasoning traces in both
Romanian and English, alongside direct-answer mode.
The model is compatible with the Qwen3.5 chat template and tooling.
Evaluation
| Benchmark | Surogate-3.5-2B |
|---|---|
| Romanian text quality | |
| Invented word forms / 1k ↓ | 1.016 |
| English leakage / 1k ↓ | 0.731 |
| Missing diacritics / 1k ↓ | 0.587 |
| Knowledge & STEM | |
| MMLU-Pro | ~63.8 |
| MMLU-Redux | ~76.4 |
| GPQA | ~49.5 |
| SuperGPQA | ~36.0 |
| Instruction following | |
| RO IFEval prompt / instruction strict | 44.75 / 70.05 |
| EN IFEval prompt / instruction strict | 67.84 / 75.78 |
| IFEval | ~74.7 |
| IFBench | ~39.2 |
| MultiChallenge | ~32.0 |
| Math & reasoning | |
| RO GSM8K direct / thinking strict | 49.77 / 47.88 |
| EN GSM8K strict | 68.92 |
| HMMT Feb 25 / Nov 25 | ~22.9 / ~19.6 |
| PolyMATH | ~26.1 |
| Agents & long context | |
| BFCL-V4 | ~41.9 |
| TAU2-Bench | ~46.8 |
| LongBench v2 | ~37.2 |
| Multilingual & translation | |
| Translation EN to RO / RO to EN (chrF2) | 47.42 / 56.17 |
| WMT24++ | ~44.0 |
| MMMLU | ~60.6 |
| MMLU-ProX | ~50.2 |
| INCLUDE | ~53.2 |
| Global PIQA | ~66.5 |
Plain values are direct measurements. Values marked ~ are estimates derived
from the source foundation model's published results and measured capability
retention.
Bilingual Thinking Example
With enable_thinking=True, the reasoning trace follows the prompt language.
Romanian prompt: Un tren parcurge 180 km în 3 ore. Care este viteza sa medie?
<think>
Viteza medie este distanța împărțită la timp: 180 km / 3 h = 60 km/h.
</think>
Viteza medie este 60 km/h.
English prompt: A train travels 180 km in 3 hours. What is its average speed?
<think>
Average speed is distance divided by time: 180 km / 3 h = 60 km/h.
</think>
The average speed is 60 km/h.
Usage
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "surogate/Surogate-3.5-2B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
model_id,
device_map="auto",
torch_dtype="auto",
)
messages = [
{"role": "user", "content": "Explică pe scurt de ce cerul este albastru."}
]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
enable_thinking=False,
return_tensors="pt",
return_dict=True,
).to(model.device)
output = model.generate(**inputs, max_new_tokens=512)
answer = processor.decode(
output[0][inputs.input_ids.shape[1]:],
skip_special_tokens=False,
)
print(answer)
Set enable_thinking=True to request an explicit reasoning trace. The shipped
generation defaults are temperature 0.6, top-p 0.95, and top-k 20.
Limitations
- Romanian is the primary optimization target.
- English performance may vary by task.
- A 2B model has limited factual depth and reliability on complex reasoning.
- The model can produce spelling, reasoning, and factual errors.
- Generated facts, calculations, and high-impact advice require independent verification.
Use is subject to the repository's license terms.
Contact Us
Contact us at sales@invergent.ai