LFM2.5-8B-A1B text-to-SQL, bf16
LiquidAI/LFM2.5-8B-A1B fine-tuned on text-to-SQL, merged and left in bf16. It is the ceiling arm of a panel of 7 arms: every quantized arm below was made from this checkpoint and allocated the same byte budget, so their accuracies differ by method and not by size.
What this is
| base model | LiquidAI/LFM2.5-8B-A1B |
| fine-tune | lora r=32, 1.0 epoch over 49,905 text-to-SQL conversations |
| training data | gretelai/synthetic_text_to_sql, Salesforce/wikisql, b-mc2/sql-create-context |
| quantization | none -- this is the bf16 fine-tune every quantized arm was made from |
| size on disk | 15.773 GiB (16.0000 bits per parameter) |
| loads with | transformers |
Results
Execution match on 12,000 held-out text-to-SQL problems: the generated query is run against the schema and compared to the reference result set.
| arm | exec match | size | bits/param |
|---|---|---|---|
| bf16 | 84.29% | 15.773 GiB | 16.0000 |
| gptq_4b | 82.07% | 4.097 GiB | 4.1565 |
| awq_4b | 81.77% | 4.097 GiB | 4.1565 |
| dq_4b | 82.84% | 4.096 GiB | 4.1547 |
| gptq_3b | 60.76% | 3.104 GiB | 3.1488 |
| awq_3b | 57.92% | 3.104 GiB | 3.1488 |
| dq_3b | 79.89% | 3.103 GiB | 3.1475 |
This arm, by evaluation source:
| eval source | exec match | items |
|---|---|---|
gretel | 72.02% | 3,063 |
wikisql | 88.50% | 8,937 |
How this arm compares
McNemar exact over the per-item hits, so every row is a paired test on the same problems in the same order. p (Holm) is step-down corrected within the family the panel declared, not within this card.
| comparison | delta (pts) | 95% CI | p | p (Holm) | verdict |
|---|---|---|---|---|---|
| 4b GPTQ vs bf16 | -2.22 | [-2.66, -1.79] | 1.61e-23 | 3.23e-23 | separated |
| 4b AWQ vs bf16 | -2.52 | [-2.98, -2.07] | 2.52e-28 | 7.55e-28 | separated |
| 4b DynQuant vs bf16 | -1.45 | [-1.84, -1.06] | 2.09e-13 | 2.09e-13 | separated |
| 3b GPTQ vs bf16 | -23.53 | [-24.38, -22.69] | 0 | 0 | separated |
| 3b AWQ vs bf16 | -26.37 | [-27.25, -25.50] | 0 | 0 | separated |
| 3b DynQuant vs bf16 | -4.40 | [-4.96, -3.84] | 1.82e-53 | 7.29e-53 | separated |
What is not claimed
- Storage, measured; throughput, not. The number reported here is bytes on disk and execution match. This card makes no claim about decode speed or peak VRAM against an fp16 baseline, because this panel did not measure either.
- One task. Execution match on held-out text-to-SQL is what was scored. It says nothing about how this arm behaves on anything else, and a quantization that holds one task can lose another.
Install
This checkpoint is plain bf16 and loads with transformers alone. It is the ceiling arm of a DynQuant panel: the quantized arms in the table above are this same fine-tune at a fraction of the size, and those need the package.
pip install dynquant
Source, format spec, and the allocator that produced their bit maps: https://github.com/kambojvikram/dynquant
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("VikramPal/LFM2.5-8B-A1B-bf16", device_map="cuda")
tokenizer = AutoTokenizer.from_pretrained("VikramPal/LFM2.5-8B-A1B-bf16")
Provenance
- panel model:
/workspace/runs/s4/lfm25-8b-a1b.text2sql/merged - parameters counted: 8,467,856,128
- byte target: --
- fine-tune: 1560 steps, train loss 0.1114, 6.7 h
- fine-tune commit:
d0d33f3bce6f3f59359ce704b16040c7e9ba78f5 - evaluation: 12,000 problems in 59 min