sft_gene_pathway_v7_6_100k_relabel_9b
Qwen3.5-9B fine-tuned to reason mechanistically about genetic (CRISPR) perturbations and predict how a queried pathway responds — upregulated, downregulated, or unchanged — conditioned on a single-cell expression profile.
This is the genetic arm of BioReasonCell. It is not a plain text model: it consumes a 2058-d STATE cell embedding alongside the prompt (see Loading).
Where this sits in the project
The repo name is the run name, so it maps 1:1 onto the source tree:
| experiment leaf | experiments/sft_gene_pathway_v7_6_100k/9b_full_relabel/ |
| family | sft_gene_pathway_v7_6_100k — v7.6 pooled with v7.5, scale arm + row-composition A/Bs + the relabel repair |
| training run | sft_gene_pathway_v7_6_100k_relabel_9b_stage2 (stage 2; stage 1 is ..._9b_stage1) |
| checkpoint | sft_gene_pathway_v7_6_100k_relabel_9b_converted — these weights |
| dataset config | sft_gene_pathway_v7_6_full100k_relabel |
| scoring set | benchmark_genetic_v1 |
Results
Scored on benchmark_genetic_v1 — 27,067 rows / 1,319 perturbations, held out, with
(cell, perturbation) leakage asserted at build time. The metric is committed accuracy
(accuracy on rows where the model commits to a direction), reported as the per-perturbation
macro-average, never row-pooled.
| dataset | k | greedy | maj@10 | constant-direction control |
|---|---|---|---|---|
| xaira_orion | 598 | 69.7 | 74.0 | 61.0 |
| nadig | 292 | 55.6 | 59.7 | 58.6 |
| marson | 240 | 54.3 | 54.0 | 51.7 |
| xaira_pisces | 110 | 70.4 | 71.9 | 65.5 |
| kaufman | 77 | 61.9 | 66.9 | 51.3 |
| pooled | 1,317 | 63.4 | 66.6 | 58.6 |
The constant-direction control is the score from always answering the majority direction for that dataset. It is the number any model on this task has to clear, and it is high — 58.6 pooled — so read every result against it rather than against 50.
Against mechanistic baselines
Per-perturbation committed accuracy, greedy, with each baseline rate-matched onto the same footing:
| dataset | 9B (this model) | STATE few-shot(ii) | STATE zero-shot | Rhaister few-shot(ii) |
|---|---|---|---|---|
| xaira_orion | 69.7 | 58.6 | 52.3 ⁽¹⁷%⁾ | — |
| nadig | 55.6 | 35.0 | 47.0 ⁽⁹⁹%⁾ | 51.9 ⁽⁹⁴%⁾ |
| marson | 54.3 | 50.6 | — | 54.3 ⁽⁸⁴%⁾ |
| xaira_pisces | 70.4 | 71.4 | 60.2 ⁽³⁸%⁾ | 68.3 ⁽⁸⁴%⁾ |
| kaufman | 61.9 | 51.4 | 45.4 ⁽¹⁰%⁾ | 49.2 ⁽⁷¹%⁾ |
| pooled | 63.4 |
Superscripts are the share of the dataset each baseline covers; cells under 10% coverage are shown as
— because 8–14 perturbations is a number, not a measurement. Both zero-shot baselines are capped at
their released checkpoint's 2,024-gene perturbation vocabulary.
Things worth knowing before you quote a number
- maj@10 is worth more than doubling the base model. Voting over ten samples at temperature 1 buys +3.2 here, against +2.5 for 9B over 4B at greedy. The two stack.
nadigis the one dataset almost nothing clears. Only this model at maj@10 (59.7) beats its 58.6 constant call, and by about a point. The observed-data ceiling there is 84.0, so that is a real deficit, not a noise floor.kaufmanis the informative case for self-consistency: commit rate falls 37.0 → 31.5 while accuracy rises to 66.9 — voting makes the model abstain more and be right more often when it does answer.- Greedy
forcedaccuracy is degenerate at n=1 (an abstention cannot be pushed to a direction) and only becomes meaningful at maj@10.
Loading
The checkpoint carries a cell_projection.pt alongside the transformer weights. The model expects a
STATE 2058-d per-perturbation cell embedding projected into the token stream between
<|CELL_START|> and <|CELL_END|>; loading the weights with AutoModel.from_pretrained alone will
run, but will not reproduce the results above, because the cell channel will be empty.
from huggingface_hub import snapshot_download
path = snapshot_download("wanglab/sft_gene_pathway_v7_6_100k_relabel_9b")
# path/model.safetensors transformer weights
# path/cell_projection.pt the cell-embedding projector (required)
The full inference path — embedding construction, projection, prompt rendering and scoring — lives in the BioReasonCell repository. Rendering matters: a prompt belongs to a corpus version, and scoring this model under a different template measures something else.
Training
| base model | Qwen3.5-9B |
| method | two-stage SFT, LoRA r128 / α128 |
| stage 1 | cell-projector alignment, 1 epoch |
| stage 2 | full reasoning SFT, 10 epochs, warm-started from stage 1's projector |
| corpus | sft_gene_pathway_v7_6_full100k_relabel — 102,957 rows |
| context | max_length_text 16384 |
| hardware | 2 nodes × 4 GPUs, effective batch 64 |
| wall clock | 2:56:19 (stage 1) + 27:35 (stage 2) |
Training traces were generated by Claude Opus 5 at effort: medium over CRISPR perturbation
data, then repaired: 25,531 traces invalidated by a labelling error were regenerated. This checkpoint
is the arm trained on the repaired corpus, and the repair is worth +2.5 greedy / +2.8 maj@10 over the
4B on identical perturbations.
Intended use and limitations
Research use. This model predicts a direction of pathway response, not an effect size, and its outputs are hypotheses to be tested — not experimental evidence.
- The constant-direction control is 58.6 pooled. A model at 63.4 is meaningfully but not dramatically
above "always say up," and on
marsonit does not clear its control at all. - Performance is not uniform across datasets; quote the per-dataset row, not just the pooled number.
- The model was trained and scored on one template. Scoring under a different prompt measures a different thing.
- Reasoning traces are model-generated and can be fluent while wrong. Do not treat a trace's intermediate claims about individual genes as validated.
Related
- Generations:
wanglab/sft_gene_pathway_v7_6_100k_relabel_9b_generations - Reasoning corpus:
wanglab/BioReasonCell-ReasoningData
License
Inherits the license of the Qwen3.5-9B base model. Training data derives from Claude Opus 5 generations; downstream use is subject to the terms governing those outputs.