coapi-voice-v3 — evidence, not product
This model is published as the artefact behind a negative result. It is a LoRA fine-tune of Qwen/Qwen3-4B that was measured to be worse than its untrained base on the task it was trained for. It is here so that the claim can be checked and the hypothesis attacked, not so that it can be used. If you want the working configuration, it is the untrained unsloth/Qwen3-4B-GGUF (Q4_K_M) behind retrieval and deterministic post-processing — see the write-up. Do not use this model for advice on Spanish property transactions.
- Write-up: We Trained It Three Times. Then We Stopped.
- Full experiment (pre-registration, judge files, version records): Chronos exp_035
What it is
An attempt to put the manner of a Spanish real-estate professional — answer in the question's language (EN/PL/ES), plain text, cite the tema and article, give the Spanish term, hand binding decisions to a professional — into a small model's weights, while the facts stay in retrieval over our own study notes. The thesis "facts in retrieval, form in the weights" was refuted at this scale: every configuration tried installed the form and lowered the model's use of the facts in front of it.
The model was trained on our study notes; the answers come from the regulation as summarised there. It does not know Spanish law and was never intended to answer without retrieved context in the prompt.
Training
| Base | Qwen/Qwen3-4B (safetensors), fused after training, converted with llama.cpp, quantised to Q4_K_M and Q8_0 |
| Adapter | LoRA rank 8, scale 1.0, dropout 0.05; keys: self_attn.q_proj, self_attn.v_proj only, last 16 of 36 layers; trainable params 1.31 M (0.033 %) |
| Schedule | lr 5e-5, 1 epoch(s) = 182 iterations, batch 2, max sequence 2560, prompt masked, seed 35 |
| Data | 363 train / 14 validation examples, sha256 3c854250bd2a5149… / b3658d7dccdee388…. Written by a frontier model to a brief, with the BM25-retrieved context inside the user turn, regex-gated (every number and citation in the answer exists in the context). Not published: the examples quote our study notes verbatim. |
| Toolchain | mlx-lm 0.31.3, mlx 0.32.2, llama.cpp 6f41ac59e0, Ollama 0.33.0; Mac Mini M4 Pro 64 GB |
| Built | 2026-09-22T11:08:56 |
Result on the sealed 60-row eval (same retrieval, same sampling, same frozen judge rubric)
| Untrained base | v3 | |
|---|---|---|
| Language / plain text | 82 % / 18 % | 98 % / 93 % |
| Answers citing / hand-off | — / 88 % | 78 % / 19 % |
| Truncated answers | — | 12/60 |
| Correctness (0–2) / fully correct | 1.13 / 38 % | 0.97 / 20 % |
| Correctness with substance in context (38 rows) | 1.50 | 1.21 |
| Native Polish / Spanish | 3/17 · 9/12 | 1/17 · 2/12 |
| Tail degradation | 10/60 | 27/60 |
Reading. 25× fewer trainable parameters, attention only, one epoch at a low learning rate: the shape barely installs and the substance damage is the same as v2. Adapter strength is not the variable.
What was ruled out as the cause, each by a pre-registered diagnostic: quantisation (Q8_0 same), epochs (val-loss-minimum checkpoint same), sequence truncation (fixed in v2), decoding (temperature 0 and 0.7 same profile), adapter strength (v2 33 M vs v3 1.3 M same damage). The surviving hypothesis is the objective: a few hundred answer-only sequences under prompt masking teach the base a distribution over answers that it follows into invented continuations. One confound is open — the training answers were written by a much stronger model, and a 4B imitating that register may learn to sound certain where it would have hedged. The falsifying experiment: train on the base's own outputs, shape-edited only. These weights are here so that someone can run it.
Files
| File | Size | sha256 |
|---|---|---|
coapi-voice-v3-Q4_K_M.gguf | 2.50 GB | 25244fd8eedb28a8627f595df7650e04e02e0a0013be1881d0b008f7efcfbb28 |
coapi-voice-v3-Q8_0.gguf | 4.28 GB | 50e59d93bf692cb399f2ef16b1646f1c2e17da86dbbd7f91917eee17c3eaab37 |
adapters/ | the mlx-lm LoRA adapter as trained | |
lora_config.yaml, version_record.json | exact configuration and environment |
Running it (to reproduce, not to use)
ollama run hf.co/miktam/coapi-voice-v3-gguf:Q4_K_M
Thinking off (/no_think), temperature 0 / seed 42 for the scored pass; shipped sampling was temperature 0.7, top_k 20, top_p 0.8, min_p 0.05, repeat 1.05, presence 0. The pre-registration, version records and judge files are in the experiment directory linked above, where the eval set is sealed by hash; the eval runner, scorer and retrieval code are not published.
Rights
Trained only on the authors' own study notes. No course materials, no client or buyer data, no conversation logs entered the training set, the eval, or the retrieval index. Base model Apache-2.0 (Qwen); this derivative is released under the same licence.