Anrn 12B R1

Anrn 12B R1 is a 12B dense, natively multimodal Gemma 4 derivative distributed as a byte-identical BF16 base plus a project LoRA adapter applied at a tested scale of 0.45. The release supports text, image, audio, and video inputs through the native Transformers service and includes a ready-to-run llama.cpp text deployment.

Release layout: this repository intentionally keeps the original BF16 base and the trained adapter separate rather than destructively merging them. The included launchers compose both parts at runtime.

Benchmark results

The Anrn values above come from the project's August 2026 evaluation run. The comparison values shown in the chart are reproduced from the published benchmark results used by that report; values marked with ~ are approximate.

Model details

Repository structure

model/           Original BF16 Gemma 4 12B weights and processor files
adapter/         Anrn R1 PEFT/LoRA adapter
llama-model/     Q5_K_M GGUF base, Q8_0 GGUF LoRA, and embedded chat template
llama-runtime/   Verified Windows CUDA llama.cpp runtime
assets/          Benchmark figure
start.ps1        Native multimodal OpenAI-compatible service
serve.py         Native service implementation
Start-Llama-Service.bat
                 One-click llama.cpp text Web UI

Quick start: native multimodal service

Requirements: Windows, Python, a recent CUDA-capable NVIDIA GPU, and PowerShell.

python -m pip install -r requirements.txt
.\start.ps1 -Port 8091

The launcher loads model/, mounts adapter/ at scale 0.45, injects system_prompt.md, uses PyTorch SDPA for long-context inference, and automatically offloads language layers to system memory when required. The tested configuration used approximately 14 GB of GPU memory.

OpenAI-compatible endpoint:

POST http://127.0.0.1:8091/v1/chat/completions

Example health check:

Invoke-RestMethod http://127.0.0.1:8091/health

Quick start: llama.cpp Web UI

On Windows, double-click:

Start-Llama-Service.bat

The bootstrap checks the bundled llama.cpp runtime and, if it is missing, downloads a matching official Windows CUDA/CPU build. It then starts the text Web UI with:

• 131,072-token context
• LoRA scale 0.45
• Flash Attention
• Q8_0 K/V cache
• the project system prompt embedded in the Jinja chat template

Web UI: http://127.0.0.1:8091

Use Stop-Llama-Service.bat to stop it. The llama.cpp route is text-only; use start.ps1 for native image, audio, and video inputs.

Prompt behavior

system_prompt.md contains the release's default project constitution. The native server injects it before caller-provided system content. A request may set:

{"use_default_system_prompt": false}

to disable the default prompt for that request. The llama.cpp prompt is embedded in llama-model/chat_template.prompt30k.jinja.

Verification

The release contains machine-readable verification artifacts covering:

• native text, image, audio, and video loading
• prompt hash and injection behavior
• llama.cpp health, 128K context, and live LoRA scale
• identity smoke tests
• package checksums and rollback scripts

Primary records: verification.json, llama-bootstrap-verification.json, prompt30k-verification.json, and SHA256SUMS.txt.

Limitations

• Benchmark scores are sensitive to prompt templates, decoding parameters, judge versions, and harness revisions.
• The BF16 base is large; CPU offload improves accessibility but reduces generation speed.
• The included llama.cpp path is an auxiliary quantized text deployment and does not expose the native multimodal input path.
• Users should evaluate the model for their target language, domain, and deployment policy before production use.

License

Apache 2.0. See the Gemma 4 license information and the upstream model terms.