Finnish Legal Citation-Grounded 12B
A gemma-4-12B-it model fine-tuned to answer Finnish legal questions grounded in
retrieved court-decision chunks: it cites every claim with a [doc:ID] tag, attributes
each citation to the court's holding rather than an overruled or dissenting passage, pins
the right paragraph (kohta), and refuses when the sources do not support an answer.
This is a collaboration: the Finnish corpus, dataset schema, and judge definitions are
Niko Hannolainen's (nikohann/fi-legal-sft, built from a 264M-triple Finnish/EU legal
knowledge graph); the teacher pass, judge stack, and student training were run by
overthelex on 8×H100. It ports the method behind
overthelex/ua-legal-citation-grounded-14b from Ukrainian to Finnish.
Method
Reverse-QA distillation with a multi-stage reject filter:
- Teacher:
google/gemma-4-31B-itwrites[doc:ID]-cited answers over retrieved chunks (served under vLLM with--reasoning-parser gemma4+enable_thinking=false). - Judge stack (only judge-passing answers survive to training):
- membership — every cited id must resolve to a chunk in context
- attribution — a citation on an overruled/dissent chunk must be marked as a losing position in the same sentence (turns same-case overruled/dissent passages into hard negatives)
- kohta-precision — a cited paragraph number must fall inside the chunk's official range
- factual-consistency — an LLM pass (31B) checks each claim against its cited chunk
- Student: LoRA (r=64) on
gemma-4-12B-it, seq 8192, 2 epochs, 8×H100 DDP, stratified to a real-usage court-family mix.
Evaluation
On 150 held-back examples (unseen question/context; see caveat), against the unfinetuned base:
| metric | base gemma-4-12B-it | this model |
|---|---|---|
| Citation coverage | 100.0% | 97.2% |
| In-context rate | 99.7% | 98.5% |
| Membership-clean | 97.9% | 97.2% |
| Attribution-clean (cites holding, not overruled/dissent) | 97.2% | 99.3% |
| Kohta-precision | 97.2% | 100.0% |
| Refusal recall | 62.5% | 100.0% |
| Over-refusal | 0.0% | 2.8% |
| Degenerate outputs | 0 | 0 |
The base model already cites in-context well, but it refuses correctly only 62.5% of the time (it answers even when the sources do not support it) and cites the losing position more often. Read as error rates, the fine-tune removes about three quarters of the attribution errors (2.8% → 0.7%) and every paragraph-precision error (2.8% → 0%), and takes refusal recall from 62.5% to 100% — better calibrated, not just more citation-happy, at a small cost in raw coverage because it now refuses when it should.
Caveat: these are in-distribution held-back examples (unseen question/context on decisions the model saw other examples from), not a separate held-out-decisions test set. Read them as in-distribution grounding quality, not cross-decision generalization.
Attribution-label caveat (provisional number): attribution-clean is scored against the
corpus kumottu (overruled) markers, which are being corrected. The current marker over-flags
lower instances — it flagged every lower instance in a document rather than only the one
named in the operative part (1,257 of 2,986 marked decisions carry more than one flag; 194
have an operative part that expressly leaves a lower judgment standing). So a share of the
attribution rejects are false rejects, and 99.3% is partly conformance to labels that are
wrong in places. This will be re-measured on the corrected labels; treat it as provisional.
Serving
Serve with vLLM and read the answer from the content field:
vllm serve <this-model> --reasoning-parser gemma4 --chat-template chat_template.jinja
Pass chat_template_kwargs={"enable_thinking": false} per request (Gemma-4 has a thinking
channel that must be suppressed). Inference guardrail: strip any [doc:ID] not present in the
context you passed (a few percent of citations can be out-of-context even after training).
License
Derivative of google/gemma-4-12B-it, governed by the Gemma Terms of Use.