decosa-cell-pointer-modernbert-base (v2, value-blind)

Given a paragraph with its numbers tagged and one or more tables, the model says which table cell each number reports: the column for the group the sentence names and the row for the endpoint, statistic, timepoint and population it describes. It can also answer "no cell", and for a computed number (difference, sum, percentage, ratio, relative reduction) it names the operation and its operand cells.

It never sees the cells' values. Tables are written as {column label} under each row label, and "(N=...)" is removed from column labels, so a number can only be placed by what the sentence says, not by where the same value happens to be printed. That is the point: a pointer that sees values tends to cite the cell holding the written (wrong) value and so hides the very error a checker is looking for. Your code then compares the written number with the cell's value.

It was built as the pointer inside a clinical study report (CSR) number checker, where it replaced an LLM call. It runs on a CPU (ONNX Runtime, no GPU needed).

Two models in this repository

Results

Held-out sets, never trained on. The model is used inside the same CSR number checker end to end (code decides whether a number is wrong); only the pointer changes. The baseline is the same checker with Qwen3.8-27B as the pointer, values shown. All rows use BM25 table retrieval. "Right cell" counts checked numbers whose gold is one cell; "right cell, planted" is the same on the numbers with a planted error (does the pointer still point where the sentence claims when the value is wrong?).

Sets: synthetic test (12 fictional CSRs, 96 planted errors), rewritten test (the same reports with paragraphs rewritten by Qwen3.8, 96 planted errors), ClinicalTrials.gov (30 real phase 3 trials from conditions not in training, their posted results laid out as report tables with template narratives; 178 planted errors).

A cascade (this model first, Qwen3.8 with values shown when the model's calibrated confidence is under 0.6, threshold set on dev) did not help: 95/96, 93/96 and 176/178, because the value-aware fallback again cited the cell holding the written value on the numbers it was handed. Use the model alone.

Transfer outside clinical reports is weak. On 10-K MD&A text (two FY2025 annual reports not trained on, 69 figures; gold from a code tie-out that sees values, which favours value-aware pointers):

Not ready for financial filings: most misses pick the consolidated line where the sentence names a segment.

Speed: about 0.2-0.45 s of CPU time per checked number on a shared server (ONNX Runtime, 4 threads); a 12-page synthetic report took 25-35 s end to end with no LLM call. Not measured on a dedicated CPU.

Intended use

• Locating the table cell a number in running text refers to, in clinical study reports and similar results documents (efficacy, safety, baseline and disposition tables), so code can check the number against the cell.
• As the pointing step of a number-consistency checker, where the separation between "which cell?" (model, value-blind) and "does it match?" (code) is what makes wrong numbers visible.

Not for: deciding by itself whether a number is correct (it points; it does not check), financial filings or other out-of-domain tables without your own evaluation, or any clinical, regulatory or patient-care decision. It is not a validated system and must not be used inside GxP or regulated processes without your own validation and human review.

Limitations (measured, see Results)

• Trained mostly on clinical report tables; weak on financial segment and note tables (56.5% right cell on 10-K MD&A).
• The synthetic test reports come from the same generator templates as the training reports (different seeds, drugs, arms and numbers); the rewritten and ClinicalTrials.gov sets are the fairer measures. The ClinicalTrials.gov narratives also use the training template. It has not been tested on a real sponsor CSR.
• Numbers inside names ("Group 1", "Chondroitin 4&6 Sulfate") get "no cell"; a checker then reports them as untraceable. This is most of the false flags on ClinicalTrials.gov (38 on 30 clean documents).
• For computed numbers it gives the operands, not their order.
• One test false-flag fix in the surrounding checker (sums kept within one table) was prompted by errors seen on test, so the test false-flag number is not a fully clean held-out figure; dev showed the same pattern.
• Latency was measured on a shared machine.
• model.safetensors (both variants) was rebuilt from the ONNX export after the original training checkpoints were lost; the rebuilt encoder matches ONNX Runtime to a max hidden-state difference of 3e-5. The ONNX files are the ones the results above were measured with.

Training data (every source and its licence)

Not used: FinQA (MIT; its operands are mostly computed, not cell lookups), EMA Policy 0070 clinical data (non-commercial research only). No human labels. The generator, the training data and the evaluation sets are not released.

Held-out discipline: nothing from the evaluation sets (synthetic test seeds, their rewrites, the 30 evaluation trials) or the transfer filings was trained on. Model choice, the temperature and the cascade threshold were set on the model's own dev split.

Model

ModernBERT-base (answerdotai/ModernBERT-base, Apache-2.0) reads [tagged sentences] [SEP] [tables], up to 2,048 tokens. Heads (modeling_cellpointer.py, class Pointer): a pair scorer over (number, cell) plus a "no cell" score, an operation classifier over none / diff / sum / pct / ratio / reduction, and an operand pair scorer. A number is the mean of its tokens and its [#k] tag; a cell is the mean of its {...} tokens. Softmax with temperature 1.4 (in pointer.json) gives calibrated probabilities. v1 was fine-tuned for one epoch on Apple silicon (MPS) on synthetic CSRs, their rewrites, ClinicalTrials.gov tables and TAT-QA; v2 continues v1 with SEC face-statement sentences and a replay of the v1 mix.

Files

• model.safetensors, config.json, tokenizer.json, tokenizer_config.json, special_tokens_map.json: encoder plus heads (fp32), for PyTorch.
• onnx/model.onnx (the encoder; output hidden) and onnx/heads.npz (head weights, applied in numpy): the CPU runtime.
• cellpointer_input.py + csr_tables.py: the exact input rendering. Use them; the model expects this format.
• modeling_cellpointer.py: the PyTorch module. usage.py: a runnable example.
• pointer.json: view, max length, temperature, the dev-tuned cascade threshold and the sha256 of every weight file.
• aware/: the value-aware twin, same layout.
• eval_summary.json: the numbers above. SHA256SUMS, LICENSE, NOTICE.

Usage

pip install torch transformers safetensors
python usage.py                     # the value-blind model
MODEL_DIR=aware python usage.py     # the value-aware twin

import cellpointer_input as CI
table = CI.table_from_rows("T1", "14.2.1", "Primary endpoint: PASI 75 at Week 16 (full analysis set)", [
    ["", "Placebo (N=103)", "Drug X 40 mg (N=102)"],
    ["Responders, n (%)", "18 (17.5)", "64 (62.7)"],
], header_rows=1)
r = CI.render([{"text": text, "mentions": mentions}], [table], blind=True)
# r.text_a / r.text_b go to the tokenizer as a pair; r.mentions and r.cells give the spans the heads pool over.
# See usage.py for the full forward pass. Output, for the sentence in usage.py:
#   64 [#1] -> T1 R1C2 p=1.00   62.7 [#2] -> T1 R1C2 p=0.99   18 [#3] -> T1 R1C1 p=0.88   17.5 [#4] -> T1 R1C1 p=0.81

Licence

Apache-2.0 for the weights and code in this repository. The base model is Apache-2.0 (ModernBERT, Answer.AI and LightOn). The training data sources above ask for attribution; it is given here and in NOTICE.

Citation and contact

Decosa, decosa.ai. Evaluation: docs/evals/cell-pointer.md in the Decosa API repository (numbers reproduced above).