Tyness BOE 1.2B Instruct — Pre-Alpha 2 (RAG-style grounded SFT, Instruct base)

Second pre-alpha from Tyness Train, an agent-guided fine-tuning harness for small LLMs on low-VRAM GPUs. This model is RAG-style grounded: every prompt carries excerpts of the relevant norm, and the model is trained to answer only from that context.

⚠️ Status: PRE-ALPHA. Grounded answering works partially (35% of eval items judged partially correct; mean judge score 0.267, 53× better than the first pre-alpha), but structured outputs still degrade. Always verify against boe.es.

Model description

Why grounded SFT (the experiment behind this model)

Three iterations established that SFT alone teaches format, not facts on 1.2B models:

Full log: eval-report.md and finding-sft-format-not-facts.md.

Training data

Source: consolidated Spanish legislation (BOE open data) via legalize-es, with teacher-generated ChatML entries from es-legal-priv. 10 BOE articles, 1,148 train + 20 held-out eval items:

Methodology (RAG-style):

• Each entry cites specific precepts (articulos_citados); those sections are extracted from the norm markdown and embedded as a CONTEXTO block in the user turn (up to 9,000 chars).
• The system prompt enforces grounded behavior: answer only from the context; say «No consta en el contexto proporcionado» otherwise.
• Loss on assistant tokens only. Provenance metadata strictly separated from training.
• Held-out eval (2/article) with the same grounded format, judged by an LLM referee (glm5.3-flash): correct / partial / incorrect + score + reason.

Training configuration

Evaluation (teacher-generated paired set, n=200 per style)

Evaluated on a fresh, uncontaminated evaluation set: 200 questions generated by a teacher LLM across the 10 norms (never part of any training split), each grounded in a specific precept, asked under two paired context styles. Judged by glm5.3-flash (correct/partial/incorrect + 0–1 score).

For comparison, the previous pre-alpha (filtered contexts) scores 0.369 ± 0.314 on the same paired set — the relevance-ordered context construction is a confirmed improvement (+0.046, p≈0.046). Full methodology: eval-report.md.

What works: grounded reading of provided precepts, identifier exactness when grounded, legal register, norm identification. What fails: exact JSON ficha outputs, multi-precept synthesis, occasional repetition loops.

How to run

llama.cpp (all quantizations included)

# 4-bit
llama-cli -hf jasonjimnz/tyness-boe-1.2b-pre-alpha-2-instruct:Q4_K_M --jinja -c 16384

# OpenAI-compatible server (recommended: paste norm excerpts as context)
llama-server -m tyness-boe-1.2b-pre-alpha-2-instruct-q4_k_m.gguf -ngl 99 -c 16384 --port 8080 --jinja
curl http://127.0.0.1:8080/v1/chat/completions -d '{
  "messages": [
    {"role":"system","content":"Eres un jurista experto en el BOE. Responde ÚNICAMENTE con el CONTEXTO. Si un dato no consta: No consta en el contexto proporcionado."},
    {"role":"user","content":"CONTEXTO:\n<paste relevant norm paragraphs here>\n\nPREGUNTA: ¿…?"}
  ],
  "temperature": 0
}'

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("jasonjimnz/tyness-boe-1.2b-pre-alpha-2-instruct", dtype="auto")
tok = AutoTokenizer.from_pretrained("jasonjimnz/tyness-boe-1.2b-pre-alpha-2-instruct")
messages = [
    {"role": "system", "content": "Eres un jurista experto en el BOE. Responde ÚNICAMENTE con el CONTEXTO proporcionado; si un dato no consta, responde: No consta en el contexto proporcionado."},
    {"role": "user", "content": "CONTEXTO:\n...\n\nPREGUNTA: ..."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")

Files

Limitations

• Partially grounded at best (35%): expect incomplete or mangled structured answers.
• Trained on 10 norms; out-of-scope norms are untested.
• Base (non-instruct) lineage: chat behavior comes purely from SFT.
• fp32 compute only on Pascal; 6GB VRAM caps training context at 2048 tokens.
• Spanish only. Never use for real legal guidance.

Attribution

Base model: LiquidAI (LFM Open License v1.0). Data: Agencia Estatal Boletín Oficial del Estado, reused per its open-data conditions, via legalize-es.

Trained and published with Tyness Train v0.4.15 — Pre-Alpha 2, October 2026.