ONNX Model for NuNerZero This repository hosts the ONNX version of NuNerZero, a zero-shot named entity recognition (NER) model optimized for fast inference using ONNX Runtime. This conversion aims to provide efficient and production-ready performance while maintaining the original capabilities of the NuNerZero model.
This model is a part of Zink. Zink is a zero-shot anonymizer and currently uses onnx NuNERZero model for anonymization.
Repository Contents:
- model.onnx: The primary ONNX model file used for inference.
- gliner_config.json: Configuration settings for the model.
- added_tokens.json: Additional tokens required by the tokenizer.
- special_tokens_map.json: Mapping for special tokens.
- tokenizer.json and tokenizer_config.json: Tokenizer vocabulary and configuration files.
- spm.model: SentencePiece model file used by the tokenizer.
Requirements:
- Python 3.7 or higher
- GLiNER – the package that provides the interface to load and run the model. I created the onnx version on "gliner==0.2.3".
Usage Here’s a quick example on how to load and use the ONNX model with GLiNER:
from gliner import GLiNER
import time
# Load the ONNX model and tokenizer
model_name="deepanwa/NuNerZero_onnx"
model = GLiNER.from_pretrained(model_name,load_onnx_model=True, load_tokenizer=True)
text = "Dr. Michael, a cardiologist from Canada, was born on 07/04/1970. John Doe dialled his mother at 992-234-3456 and then went out for a walk."
labels = ("person", "profession", "location", "date", "phone number", "relationship", "medical condition", "age")
start = time.time()
result = model.predict_entities(text, labels)
end = time.time()
print("Predicted entities:", result)
print("Time taken:", end - start)
Important Notes Large Model Size: The model file is approximately 1.85 GB, so please ensure you have sufficient bandwidth and disk space when downloading.
Inference Efficiency: Leveraging the ONNX format can significantly accelerate inference compared to the original PyTorch implementation.
Citation: 10.57967/hf/4902
Contributing Contributions, suggestions, or bug reports are welcome. Please open an issue or submit a pull request if you have improvements.