Sokol — Slovenian Assistant LLM (4B, on-device)

Trained by MediaAtlas — LLM fine-tuning on your own data, trained in the EU, weights delivered. Pricing · All our models

Na kratko: Majhen slovenski jezikovni model (4B) za popolnoma lokalno rabo na računalniku: pogovor v slovenščini, prevajanje EN ⇄ SL, razmišljanje in klici orodij. Nadaljevano predučenje na 1,78 milijarde slovenskih žetonov je slovensko perpleksnost znižalo za 51 %. Sprejme tudi slike.

Vision / slike: takes images too. The safetensors checkpoint keeps the base model's vision encoder; for GGUF load gguf/mmproj-sokol-4b-slo-F16.gguf next to the model (llama.cpp --mmproj; LM Studio pairs it automatically). Our training was text-only, so image understanding is the base model's and was not benchmarked; in a spot check it read Slovenian text on an image correctly and described a photo in Slovenian.

Sokol (falcon) is a small 4B Slovenian assistant built for fully-local, on-device use (desktop via GGUF / llama.cpp / LM Studio). It chats in fluent Slovenian, translates EN↔SL, reasons through tasks (<think>), and makes native tool / function calls (Qwen3.6 XML format).

It is a text LLM, not a speech model — pair it with your own ASR + TTS to build a local voice assistant (it routes <think> to reasoning_content, so reasoning is never spoken).

This single repo ships both the full-precision weights (safetensors) and quantized GGUFs (under gguf/), so you can run it with transformers, or fully local with llama.cpp / LM Studio.

What it is

• Base: SicariusSicariiStuff/Qwen3.5-4B_Abliterated (dense, qwen3_5 arch, natively supported in transformers ≥ 5.6; multimodal-capable).
• Phase 2 — CPT (full-parameter): continued pretraining on ~1.78 B tokens of Slovenian text for language adaptation (full-FT, DeepSpeed ZeRO-2, 1 epoch).
• Phase 3 — SFT (LoRA): chat + translation + tool-calling + reasoning + identity (~250 k examples).
• Deployment: GGUF (desktop, llama.cpp / LM Studio) and full safetensors (transformers).

Files in this repo

GGUFs are exported with --no-mtp (the multi-token-prediction head is dropped), which is required for the qwen3_5 GDN-hybrid architecture to load cleanly in current llama.cpp.

Run it (GGUF, llama.cpp)

# thinking + native tool calls; --reasoning-budget routes <think> to reasoning_content
llama-server -m gguf/sokol-4b-slo-Q5_K_M.gguf --jinja --reasoning-budget 8192 \
             -c 8192 --temp 0.4 --top-p 0.9
# with image input:
#   add  --mmproj gguf/mmproj-sokol-4b-slo-F16.gguf

Or open the .gguf directly in LM Studio. For clean, non-thinking output pass --reasoning-budget 0 (useful when driving TTS in a voice stack, so <think> is never spoken).

Capabilities

• Slovenian chat — fluent, formal Slovenian conversation.
• Translation — bidirectional EN↔SL.
• Reasoning — emits <think> traces (distilled from a large Qwen3.8-Max teacher); pass enable_thinking=False for direct answers.
• Tool / function calling — native Qwen3.6 XML, e.g. <tool_call><function=get_weather><parameter=city>Ljubljana</parameter></function></tool_call>.
• Identity — presents as Sokol, a local Slovenian assistant.

Training data (provenance)

CPT — Slovenian text (~1.68 M docs / ~1.78 B tokens)

SFT — ~250 k examples

Nothing is truncated: all SFT examples are ≤ 7942 content tokens (cutoff 8192).

Evaluation

Slovenian-LLM-Eval — acc_norm (cjvt/slovenian-llm-eval, 200/task)

Sokol (CPT + SFT) vs the abliterated base, same eval, via flywheel-sl:

Wins 6/7 tasks, ties 1, no regressions. CPT also halved Slovenian perplexity (6.996 → 3.438 on held-out SL text, −50.9%).

Judge track — chat quality (1–5)

Open-ended answers to 32 general Slovenian questions (8 categories), judged by a stronger 27B Slovenian judge model on correctness / helpfulness / language:

Sokol clearly wins language fluency and tool-calling (per-category +0.67) and the objective acc_norm; the small aggregate correctness dip comes almost entirely from two n=4 categories (grammar, instruction-format), i.e. 1–2 questions each — noise at this eval size, not a systematic regression. (v1 is 32 questions; per-category deltas need a larger v2 to be reliable.)

⚠️ Disclaimers & license

• General assistant. Outputs may be incorrect or fabricated; not professional advice. Verify anything important.
• Mixed data-license provenance (Wikipedia CC BY-SA, FineWeb2 ODC-BY, GaMS-Nemotron-Chat, OPUS, Hermes/APIGen/ToolACE, Qwen3.8-Max distillation). Marked license: other; resolve data terms before public/commercial use. Derivative of Qwen3.5-4B — the upstream Qwen license also applies.
• Provided AS IS, no warranty. See LICENSES.md.

Citation

@misc{fius2026sokol4b,
  title        = {Sokol: a small on-device Slovenian assistant LLM (4B)},
  author       = {Fius, Tadej},
  year         = {2026},
  publisher    = {MediaAtlas Ltd},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/texdata/Sokol-4B-SLO}
}