Model Card for quinex-quantity-v0-124M

quinex-quantity-v0-124M is based on the NASA/IBM INDUS model (also known as nasa-smd-ibm-v0.1), which is an encoder-only transformer model based on RoBERTa that was pre-trained on scientific literature and Wikipedia. We further fine-tuned this model to identify quantities (i.e., a number and, if applicable, a unit) in text. For more details, please refer to our paper "Quinex: Quantitative Information Extraction from Text using Open and Lightweight LLMs" (published soon).

Uses

This model is intended for detecting quantity mentions in text using sequence labeling. Please note that quantity modifiers (e.g., 'approximately', 'about', 'more than', 'not', etc.) are not considered part of quantity spans in this work.

Example output

Model details

• Base Model: INDUS
• Tokenizer: INDUS
• Parameters: 124M

Fine-tuning data

The model was first fine-tuned on non-curated examples from a filtered variant of Wiki-Quantities and subsequently on a combination of datasets for quantity span identification, including:

• Wiki-Quantities (tiny variant, curated examples only)
• SOFC-Exp (relabeled)
• Grobid-quantities (relabeled)
• MeasEval (relabeled)
• Custom quinex data

Evaluation results

Evaluation results on the test set as described in the paper:

Note that here we report the scores of this specific checkpoint, which slightly differ from the scores averaged over multiple seeds reported in the paper.

Also, note that these scores do not account for alternative correct answers (e.g., '1.2 kW and 1.4 kW' could be labeled as a list or individually) or debatable cases (e.g., whether 'bi-layer' or 'quartet' should be considered a quantity). Counting these as correct results in higher scores.

Furthermore, we evaluated this model on four scientific publications from different domains to assess its generalization capabilities. We report entity-level accuracy, i.e., the number of correctly identified quantities divided by the total number of quantities in the text. The numbers in parentheses represent the scores for strict and relaxed evaluation. These are calculated by setting all judgements of low confidence to 'incorrect' and 'correct', respectively.

Of the 839 quantities in the documents, 41 were false negatives, 11 were inaccurately labeled, and there were an additional 8 false positives. Most of the 60 incorrect quantity predictions concern quantities with "single" as the value, such as "single-gene" (22%), followed by quantities with "many", "multiple", or "multi-" as the value (17%). Excluding errors involving imprecise quantities, the overall micro-averaged accuracy is 93.98%. Excluding errors involving quantities with the value "single", the accuracy is 94.87%. Excluding errors involving quantities with noun or adjectival units, the accuracy is 96.71%.

Citation

If you use this model in your research, please cite the following paper:

@article{quinex2025,
    title = {{Quinex: Quantitative Information Extraction from Text using Open and Lightweight LLMs}},	
    author = {Göpfert, Jan and Kuckertz, Patrick and Müller, Gian and Lütz, Luna and Körner, Celine and Khuat, Hang and Stolten, Detlef and Weinand, Jann M.},
    month = okt,
    year = {2025},
}

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2
• Datasets 2.16.1
• Tokenizers 0.15.0