Multilingual SpanMarker NER: mDeBERTa-v3 on MultiCoNER 2023 (33 Fine-Grained Classes, 10 Languages)

Developed by: Rishabh Kumar
Model Name: Rishabh157/spanmarker-multiconer-mdeberta
Base Architecture: microsoft/mdeberta-v3-base

This repository provides an enterprise-grade, state-of-the-art multilingual Named Entity Recognition (NER) model developed by Rishabh Kumar, combining microsoft/mdeberta-v3-base with the SpanMarker candidate span-classification framework, fine-tuned on MultiCoNER 2023 (SemEval-2023 Task 2) across 10 languages for 6 full epochs (117,756 steps).

Model Overview

• Developer / Creator: Rishabh Kumar (@Rishabh157)
• Base Backbone: microsoft/mdeberta-v3-base (12 layers, 768 hidden dimension, disentangled relative positional attention).
• Task Formulation: SpanMarker (Candidate phrase classification with boundary marker tokens [S_i, E_j] and Greedy Non-Maximum Suppression).
• Tuning Strategy: Full-Parameter Fine-Tuning (all 278.8M parameters active).
• Sequence Parameters:
  • model_max_length: 384 tokens (100% empirical coverage across multi-sentence titles).
  • entity_max_length: 16 words (captures 99.99% of complex, multi-word entities).
  • marker_max_length: 128 candidate tokens.
• Supported Languages (10 Languages):
  • Bangla (bn), German (de), English (en), Spanish (es), French (fr), Hindi (hi), Italian (it), Multilingual (multi), Portuguese (pt), Swedish (sv).
• Entity Taxonomy: 33 Fine-Grained Classes (67 BIO tags):
  • Person Subtypes: Artist, Athlete, Cleric, Politician, Scientist, SportsManager, OtherPER
  • Location Subtypes: Facility, HumanSettlement, Station, OtherLOC
  • Organization Subtypes: AerospaceManufacturer, CarManufacturer, MusicalGRP, PublicCorp, PrivateCorp, SportsGRP, ORG
  • Creative & Visual Works: ArtWork, MusicalWork, VisualWork, WrittenWork
  • Products & Technology: Clothing, Drink, Food, Software, Vehicle, OtherPROD
  • Biomedical & Healthcare: AnatomicalStructure, Disease, MedicalProcedure, Medication/Vaccine, Symptom

Empirical Benchmark Performance

Evaluated on the full official multilingual MultiCoNER 2023 validation set (15,578 sentences across all 10 language splits) using strict exact-span seqeval matching:

Overall 6-Epoch Validation Results

Comparison with Prior Benchmarks

Per-Class Detailed Performance Breakdown (33 Categories)

Note on "Support Count": In Named Entity Recognition benchmarks, Support indicates the exact number of ground-truth entity spans belonging to that category in the validation dataset (15,578 sentences). It represents the sample size over which Recall and F1 are computed, providing transparency into entity frequency across the corpus.

Epoch-by-Epoch Training Trajectory

Usage Example

from span_marker import SpanMarkerModel

# Load the model directly from Hugging Face Hub
model = SpanMarkerModel.from_pretrained("Rishabh157/spanmarker-multiconer-mdeberta")

# Multilingual test sentences
sentences = [
    "Boeing and Airbus compete directly in the commercial aircraft market.",
    "Der FC Bayern München gewann das Finale der UEFA Champions League.",
    "El presidente de Francia anunció nuevas medidas en París.",
    "डॉ. एपीजे अब्दुल कलाम भारत के राष्ट्रपति और एक प्रसिद्ध वैज्ञानिक थे।"
]

# Run span-level entity inference
entities = model.predict(sentences)

for sent, ent_list in zip(sentences, entities):
    print(f"\nSentence: {sent}")
    for ent in ent_list:
        print(f"  [{ent['label']}] -> '{ent['char_span']}' (span: {ent['word']})")

Training Configuration & Reproducibility

• Optimizer: AdamW (β₁=0.9, β₂=0.999, ε=1e-8)
• Learning Rate: 3.0e-5 with Cosine Annealing Schedule
• Warmup Ratio: 10% (first 11,775 steps)
• Batch Size: 16 per device (Effective batch size = 16)
• Weight Decay: 0.01
• Gradient Clipping: Max norm 1.0
• Frameworks: transformers 4.41+, span-marker 1.5.0, PyTorch 2.3+

Author & Citation

This model was trained, evaluated, and published by Rishabh Kumar (@Rishabh157).

If you use this model or refer to this work, please cite:

@misc{rishabhkumar2026_spanmarker_multiconer,
  author = {Rishabh Kumar},
  title = {SpanMarker Fine-Grained Multilingual NER with mDeBERTa-v3 on MultiCoNER 2023},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Rishabh157/spanmarker-multiconer-mdeberta}}
}