BLUM Finance 4B — Transformers Benchmark Release
BLUM Finance 4B is an open, evidence-bound financial reasoning model. This
repository is the portable BF16 Transformers release used for public benchmark
submission. It contains the same trained LoRA delta as the audited MLX release,
mapped and fused into the exact Qwen/Qwen3-4B base architecture.
Intended Use
The model structures supplied point-in-time evidence into:
- balanced bull and bear cases;
- explicit risks and invalidation conditions;
- conservative confidence;
- monitoring conditions and abstention.
It is not a price oracle, broker, trade executor or proof of market alpha.
Load
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Italianhype/Blum-Finance-4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
device_map="auto",
)
No trust_remote_code=True is required.
Governed Continual Memory
This repository also ships the installable blum-finance package. It provides
schema-validated inference, auditable local memory and explicit opt-in community
contributions:
pip install "git+https://huggingface.co/Italianhype/Blum-Finance-4B"
blum-contribute example.json --output contribution.json --consent
blum-memory-add contribution.json
Local retrieval accepts only mature, source-verified outcomes observable before
the new request timestamp. Memories are labeled historical analogies and cannot
replace current evidence. Adding --push opens a pull request to the quarantine
dataset; inference sends no telemetry and anonymous input never mutates released
weights. See CONTRIBUTING.md for the evidence contract.
Lineage
- Base:
Qwen/Qwen3-4B - Base revision:
1cfa9a7208912126459214e8b04321603b3df60c - Dataset:
Italianhype/Blum-Finance-Reasoning - Dataset revision:
76ad77699d498fc930daf02e452fe3ec8b490f90 - MLX adapter revision:
ea297ba88ab008e97104b0c118103eef2f8f9ec1 - LoRA rank:
8 - MLX scale:
20 - Equivalent PEFT alpha:
160 - Adapted layers:
20–35
The conversion transposes MLX A/B matrices into PEFT orientation and preserves
the update 20 × Bᵀ × Aᵀ. All 112 expected modules were fused; none were
missing. The PEFT adapter and merged model produced identical deterministic
smoke-test output.
Evaluation Status
The original MLX release passed the 53-example BLUM temporal reasoning test with 96.26% aggregate task-contract score, 100% structured validity and 92.86% no-fabrication. These metrics measure BLUM schema adherence and grounding, not general intelligence or trading performance.
The portable release is published at immutable revision
ad6f5cec7f729370d2976d8c78983521cb37ca83 and tagged
benchmark-submission-v1.
The five-example portable conversion smoke test scored 97.32% on the internal task contract, including 100% structured validity and 92.86% no-fabrication. The sample is too small for a robust capability claim.
Automated submissions were attempted for the Hugging Face Open LLM Leaderboard
and the FinOS Open Financial LLM Leaderboard. Both external validators rejected
Qwen3 as requiring trust_remote_code=True; Transformers 4.57.6 loads this
repository without remote code. No official leaderboard score is claimed.
A deterministic stratified community evaluation on seven finance/business
subjects of canonical cais/mmlu scored 74.86% accuracy over 700 test
questions (100 per subject), with a Wilson 95% confidence interval of
71.51–77.93%. It used five-shot answer-token logit scoring.
| Subject | Accuracy | Samples |
|---|---|---|
| Business ethics | 77% | 100 |
| Econometrics | 65% | 100 |
| High-school macroeconomics | 70% | 100 |
| High-school microeconomics | 87% | 100 |
| Management | 87% | 100 |
| Marketing | 92% | 100 |
| Professional accounting | 46% | 100 |
This is an author-run community evaluation, not the full canonical MMLU suite
and not an official leaderboard result. The deterministic 100-row cap gives
each subject equal weight but can differ from a full-split result. Dataset
revision: c30699e8356da336a370243923dbaf21066bb9fe.
Evaluation summary · Per-example predictions · Evaluator
Independent evaluation packages are prepared for Vals AI CorpFin/Finance Agent and Scale Labs PRBench Finance. Those scores remain unavailable until the benchmark owners evaluate the immutable model revision.
Known Failure
A sparse-evidence smoke prompt incorrectly identified NVDA as Applied Materials. This is a documented fabrication failure. Callers must validate company identity and all market facts against BLUM Engine or another authoritative data source.
Safety
- Do not use this model as unattended financial advice.
- Do not let it authorize or execute transactions.
- Supply timestamped evidence and validate every factual statement.
- Confidence is bounded but not calibrated on mature trading outcomes.