Tyness BOE 1.2B Instruct — Pre-Alpha (focused grounded SFT)

Third pre-alpha from Tyness Train. This one is fine-tuned from the Instruct variant and trained with focused contexts: instead of whole norm sections, each prompt carries only the paragraphs lexically relevant to the question — everything fits a 2048-token training window with zero truncation.

⚠️ Status: PRE-ALPHA. Best grounded results so far (mean judge score 0.35, 2/20 fully correct, 8/20 partially correct on the hardest eval set) but not production-grade. Verify against boe.es.

Model description

The experiment so far

Key insight: SFT on 1.2B models teaches format, not facts — grounding must live in the prompt, and the Instruct prior beats extra epochs on the Base model.

Training data

10 BOE articles (state + autonomous scope), 1,148 grounded train items + 20 held-out:

Methodology (focused contexts):

• Each entry cites precepts (articulos_citados); those sections are split into paragraphs and only the lexically relevant ones (term overlap with the question) enter the CONTEXTO block, plus the norm's frontmatter identifiers (grounds BOE-A-AAAA-NNNNN answers).
• System prompt: answer only from the context; «No consta en el contexto proporcionado» otherwise.
• Loss on assistant tokens only; provenance metadata never trained on.
• Evaluation: LLM referee (glm5.3-flash) on held-out items, judged against expected answers and key reasons — including the harder 9K-char-context eval set from pre-alpha 2 for comparability (this card's reported score).

Training configuration

Evaluation

On its native focused-context eval the model behaves consistently: grounded reading, correct norm identification, refusal calibration. Exact-structure outputs (JSON fichas) remain the weak spot.

How to run

llama.cpp

llama-cli -hf jasonjimnz/tyness-boe-1.2b-pre-alpha-instruct:Q4_K_M --jinja -c 16384

llama-server -m tyness-boe-1.2b-pre-alpha-instruct-q4_k_m.gguf -ngl 99 -c 16384 --port 8080 --jinja
curl http://127.0.0.1:8080/v1/chat/completions -d '{
  "messages": [
    {"role":"system","content":"Eres un jurista experto en el BOE. Responde ÚNICAMENTE con el CONTEXTO. Si un dato no consta: No consta en el contexto proporcionado."},
    {"role":"user","content":"CONTEXTO:\n<relevant norm paragraphs>\n\nPREGUNTA: ¿…?"}
  ],
  "temperature": 0
}'

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("jasonjimnz/tyness-boe-1.2b-pre-alpha-instruct", dtype="auto")
tok = AutoTokenizer.from_pretrained("jasonjimnz/tyness-boe-1.2b-pre-alpha-instruct")
messages = [
    {"role": "system", "content": "Eres un jurista experto en el BOE. Responde ÚNICAMENTE con el CONTEXTO proporcionado; si un dato no consta, responde: No consta en el contexto proporcionado."},
    {"role": "user", "content": "CONTEXTO:\n...\n\nPREGUNTA: ..."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")

Files

Limitations

• 10-norm scope; 60% of hardest-eval items still fail or are only partially grounded.
• Structured/JSON outputs degrade; prose answers are stronger.
• 1.2B class: cite-only behavior is trained, but complex multi-precept reasoning is out of reach.
• Spanish only. Never use for real legal guidance.

Attribution

Base model: LiquidAI (LFM Open License v1.0). Data: Agencia Estatal Boletín Oficial del Estado, reused per its open-data conditions, via legalize-es.

Trained and published with Tyness Train v0.4.17 — Pre-Alpha, October 2026.