laya-news-decisions
A fine-tuned Laya (multilingual variant, 322M parameters) that answers four typed questions about a news article in one forward pass:
| key | type | question |
|---|---|---|
topic | choice, 10 classes | which category does this news report belong to |
impact | score, 5 levels | how far do the consequences reach, from one person to global |
violence | yes/no | does the article describe physical violence |
civilian_harm | score, 5 levels | how severe are the consequences for civilians |
It was trained with laya-rlcd-training, the open training loop we wrote for Laya. This model is what that loop produces on a real corpus.
Use
import json, laya
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
repo = "InfinimindCreations/laya-news-decisions"
agent = laya.load("convaiinnovations/laya", device="cpu", subfolder="multilingual")
agent.model.load_state_dict(load_file(hf_hub_download(repo, "model.safetensors")), strict=False)
questions = json.load(open(hf_hub_download(repo, "questions.json")))
text = "Floods in Bangladesh displace 40,000 families"
print(agent.predict({"headline": text}, questions)["answers"])
The state key is headline because that is the field the model was trained on,
but we fed it the article body (first 2,800 characters) during training, and
that is what it expects. Titles alone work, less well.
Use questions.json exactly as it is. The model is sensitive to the wording
of the options it was trained with, not to the instruction sentence. Two option
descriptions start with the literal text NEU v2., a leftover editing note from
our label schema. It is part of the trained wording; removing it changes the
answers. Renaming the question keys does not (checked on four texts, identical
output).
Training
- About 92,000 news articles from 2026, many languages, labelled on the full text by DeepSeek v4.1 Flash.
- One pass, full fine-tune on one A100, batch 16, G=32, lr 1e-5 with cosine decay. Roughly 11 USD.
- Only four questions on purpose: a seven-question version of the same model was worse on four of six shared questions. Every extra question costs the others.
Evaluation
176 gold judgements, each made by three independent LLM annotators and kept by
majority vote. Accuracy is exact match; for the ordinal questions ±1 is the
share within one level and ρ the rank correlation.
| question | selected checkpoint | end of run | notes |
|---|---|---|---|
| topic | 0.784 | 0.756 | majority class 0.148 |
| impact | 0.614 | 0.557 | ±1 0.977 · ρ 0.775 |
| violence | 0.943 | 0.966 | AUC 0.977 |
| civilian harm | 0.659 | 0.642 | ±1 0.943 · ρ 0.707 |
Read the first column with care. The released checkpoint was selected as the best of 14 evaluations on this same gold set, so its numbers are the optimistic end. The second column is the last checkpoint of the run and is closer to what you should expect on unseen data. Adjacent evaluations moved by up to 4.5 points on topic, which is eight articles out of 176.
For scale, the labelling models on the same 176 articles and the same topic question: DeepSeek v4.1 Flash 0.835, Gemini 3 Flash 0.812, Claude Haiku 4.5 0.778. The fine-tuned 322M encoder sits in that range at a fraction of the cost and runs locally.
Where it is weak: politics. It gets half of those right, while its teacher got
three quarters. Better labels did not fix it (we tried, from scratch and as a
continuation, both worse), so it is capacity or data volume, not label quality.
Limits
- The gold set is small and curated. It over-represents high-impact articles compared to a real news stream, so precision in the field will be lower than these numbers suggest.
- Topic categories overlap in the real world (is a sanctions package politics or finance?). Even the large labelling models disagree with each other on about one article in six.
- The 0.5 cut-off is not a decision boundary for
violence. Fit your threshold on your own data.
License
Apache 2.0, following Laya itself.