VectraYX-Vision-1B
A sub-2B vision–language model specialized for cybersecurity imagery —
disassembly/decompiler panes (IDA, Ghidra), packet captures (Wireshark),
scanner and post-exploitation terminals (Nmap, Metasploit), and memory-forensics
dumps (Volatility) — that answers in Spanish, emits structured visual reasoning
via native <|think|> tokens before its answer, invokes external tools through
the Model Context Protocol using native <|tool_call|> tokens, and exports
cleanly to llama.cpp's LLaVA mmproj format for fully on-premise, air-gapped
deployment.
This is the GGUF export (model + mmproj) for offline inference. For the text-only backbone see jsantillana/vectrayx-1b.
Architecture
| Component | Value |
|---|---|
| Visual encoder | SigLIP-so400m-patch14-384 (frozen, 400M, 1152-d, 729 patches) |
| Projector | MLP: 1152→4096→2048, GELU + LayerNorm (~13.1M params) |
| Language backbone | VectraYX-1B: 22 layers, 2048 d_model, 5504 d_ffn, GQA 16/4, SwiGLU |
| Backbone positional encoding | RoPE (θ=1e6) on 3/4 layers, NoPE on every 4th layer |
| Vocabulary | 32,768 (BPE) |
| Special tokens | `< |
| Total parameters | ~1.04B (backbone) + 400M (encoder) + 13M (projector) |
Files
model.gguf(2.22 GB, F16) — the LLM decoder (also usable standalone for text as in vectrayx-1b)mmproj.gguf(823 MB, F16) — the SigLIP encoder + MLP projector (the vision tower)
Both are needed for multimodal inference; load both in llama.cpp / Ollama.
Training
Four-phase curriculum (all phases complete):
- Phase 1 — 9.2B tokens general Spanish pretraining (backbone)
- Phase 2 — ≈50B tokens three-block curriculum (backbone)
- Phase 3 — ≈6B tokens tooling SFT establishing native
<|tool_call|>+<|think|>(backbone) - Vision phase — 4a (projector alignment, ~228 steps), 4b (instruct, projector+backbone), 4c (think+tools). This checkpoint is the SFT-v2 continuation at step 400, warm-started from the v3B run (step 1900 of a low-LR phase-4b).
Total vision-phase compute: ≈2.2h of 2×A100-40GB wall time; ≈16M tokens.
Honest status
B6/B7 (vision benchmark) scores are near zero at the current training volume (400–1900 vision steps). The model generates fluent Spanish but ignores image content — visual grounding at this scale requires more SFT steps or a denser annotated screenshot corpus. This is a template/infrastructure release, not a quality release. Baseline context:
| Model | B6 tool-id | B6 correct | B7 chain-quality |
|---|---|---|---|
| GPT-4o (frontier ceiling) | 0.940 | 0.000 | 0.940 |
| Phi-4-multimodal (4.5B) | 0.440 | 0.000 | 0.220 |
| VectraYX-Vision-1B (v3B, step 1900) | 0.080 | 0.060 | 0.025 |
| VectraYX-Vision-1B (SFT-v2, step 400) — this checkpoint | 0.020 | 0.000 | 0.010 |
| 1B backbone, no image (control) | 0.020 | 0.000 | 0.000 |
The text-only control confirms the benchmark is not trivially answerable from text alone — the with-image v3B row (0.08) exceeds the no-image control (0.02).
Usage (llama.cpp multimodal)
# Load model + vision tower
./llama-cli -m model.gguf --mmproj mmproj.gguf \
--image /path/to/screenshot.png \
-p "¿Qué herramienta de seguridad muestra esta captura y qué hace?" \
--temp 0.7 -n 512
Usage (Ollama)
# Register the model with its mmproj
ollama create vectrayx-vision-1b \
-f <(printf 'FROM ./model.gguf
PARAMETER temperature 0.7
PARAMETER num_predict 512
')
# (mmproj must be co-located for multimodal; see Ollama LLaVA docs)
ollama run vectrayx-vision-1b
Notes / known limitations
- Visual grounding is weak. The model is primarily an architecture/data/curriculum release pending a longer SFT run. Do not expect useful visual QA yet.
- NoPE export: NoPE layers export as RoPE layers with rotary application disabled
per layer. The GGUF RoPE permute fix uses
stack([w1,w2], dim=2)interleaved. - Export is GGUF-splittable: the decoder checkpoint contains no visual weights and
vice versa, matching the LLaVA
model.gguf/mmproj.ggufconvention. - This is the first released model to combine periodic NoPE layers with an injected visual-token block; the NoPE×vision interaction is an open research question (see the accompanying paper for the ablation design).
Citation
See the accompanying paper, VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use. Paper on Hugging Face.
Author
Juan S. Santillana — DevOps engineer (Globant). Institutional affiliation approval pending. Weights, benchmarks, and configs are released; the tool-calling MCP Modelfile will follow the visual-prompt-injection and over-triggering safety evaluation.