/ Модель
Каталог

Характеристики

Контекст
512 токенов
Параметры
306 млн
Лицензия
MIT
Архитектура
deberta-v2 · dense
Базовая модель
mdeberta-v3-base
Библиотека
transformers
Опубликована
15 ноя 2023

Ссылки

Hugging Face

rured2-ner-microsoft-mdeberta-v3-base

denis-gordeev
Разметка токеновТекст → label7 профессий

Русскоязычная модель на базе DeBERTa V3 для распознавания и классификации именованных сущностей в тексте.

131скачиваний за 30 дней7лайковвеса обновлены24 ноя 2023

Бенчмарки

Как модель справилась с задачами пользователей ailist и что о ней публикуют открытые лидерборды

Публичные оценки

Из открытых лидербордов. Сравнивайте результаты одного теста при одинаковых условиях — отсутствие оценки не значит, что модель хуже.

В подключённых источниках результатов для этой модели пока нет.

Что умеет модель

Описание составлено по карточке модели и метаданным Hugging Face

Модель denis-gordeev/rured2-ner-microsoft-mdeberta-v3-base — русскоязычная модель распознавания именованных сущностей, разработанная Денисом Гордеевым путём дообучения microsoft/mdeberta-v3-base на RURED2. Она размечает токены текста несколькими возможными метками; среди классов указаны персоны, организации, страны, города, даты, денежные суммы, профессии, источники новостей и другие сущности. Модель построена на DeBERTa V3, содержит 306 млн параметров и принимает контекст длиной до 512 токенов.

Модель подходит для извлечения сущностей из русскоязычных текстов, например новостей и документов, а также для структурирования упоминаний людей, мест, организаций, событий и числовых значений. В README приведены результаты оценки: micro-F1 0,5837 в целом, при этом качество заметно различается между классами; например, для B-person F1 составляет 0,9639, а для некоторых меток показатели низкие или нулевые. Для применения требуется учитывать многозначные предсказания для отдельных токенов и проверять разметку на целевом наборе данных. Распространяется по лицензии MIT.

Сценарии применения

  • извлечение имён людей из русскоязычных новостей
  • распознавание организаций и источников новостей в статьях
  • извлечение дат, денежных сумм и процентов из документов
  • разметка географических названий и событий в тексте

Ограничения

Общий micro-F1 на оценочном наборе составляет 0,5837, а качество существенно различается между классами; для некоторых меток F1 близок к нулю. Контекст ограничен 512 токенами, а модель может выдавать несколько меток для одного токена.

Для каких профессий подходит7

Разделы каталога профессий, в задачах которых используется эта модель

Все профессии
I.3Универсальные способностиИзвлечение и структурирование данныхAI помогает превращать разрозненные данные в структуру, пригодную для поиска, аналитики и автоматизации.18 задач25 791модельII.8Разработка и данныеИнженерия данных и аналитикаAI помогает готовить надёжные данные и превращать их в понятную аналитику.22 задачи23 488моделейI.4Универсальные способностиОбработка документов и формAI помогает оцифровывать документы, проверять их комплектность и сокращать ручной ввод данных.16 задач7 180моделейI.1Универсальные способностиПоиск и навигация по информацииAI помогает находить нужные сведения и помогать человеку быстро переходить от вопроса к подтверждённому источнику.18 задач13 467моделей

Также подходит для

Карточка модели

README от автора на Hugging Face, как он его опубликовал

Открыть на Hugging Face

Russian NER model fine-tuned on RURED2. https://github.com/denis-gordeev/rured2

If you have any questions, message me at https://t.me/nlp_party This model outputs multiple possible labels for a single token. So for proper usage you can use it like in the following code:

import torch
from torch import nn
from transformers import (AutoTokenizer, AutoModelForTokenClassification, 
                          TrainingArguments, Trainer)

model_name = "denis-gordeev/rured2-ner-microsoft-mdeberta-v3-base"
model = AutoModelForTokenClassification.from_pretrained(
    model_name).to('cuda')

tokenizer = AutoTokenizer.from_pretrained(model_name)

def predict(text:str, glue_tokens=False, output_together=True, glue_words=True):
    sigmoid = nn.Sigmoid()
    tokenized = tokenizer(text)
    input_ids = torch.tensor(
            [tokenized["input_ids"]], dtype=torch.long
        ).to("cuda")
    token_type_ids = torch.tensor(
            [tokenized["token_type_ids"]], dtype=torch.long
        ).to("cuda")
    attention_mask = torch.tensor(
            [tokenized["attention_mask"]], dtype=torch.long
        ).to("cuda")
    preds = model(**{"input_ids": input_ids, "token_type_ids": token_type_ids, "attention_mask": attention_mask})
    logits = sigmoid(preds.logits)

    output_tokens = []
    output_preds = []
    id_to_label = {int(k): v for k, v in model.config.id2label.items()}
    for i, token in enumerate(input_ids[0]):
        if token > 3:
            class_ids = (logits[0][i] > 0.5).nonzero()
            if class_ids.shape[0] >= 1:
                class_names = [id_to_label[int(cl)] for cl in class_ids]
            else:
                class_names = [id_to_label[int(logits[0][i].argmax())]]
            converted_token = tokenizer.convert_ids_to_tokens([token])[0]
            new_word_bool = converted_token.startswith("▁")
            converted_token = converted_token.replace("▁", "")
            if glue_words and not(new_word_bool) and output_tokens:
                output_tokens[-1] += converted_token
            else:
                output_tokens.append(converted_token)
                output_preds.append(class_names)
        else:
            class_names = []
    if output_together:
        return [[output_tokens[t_i], output_preds[t_i]] for t_i in range(len(output_tokens))]
    return output_tokens, output_preds

denis-gordeev/rured2-ner-microsoft-mdeberta-v3-base

This model is a fine-tuned version of microsoft/mdeberta-v3-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0096
  • F1 Micro: 0.5837
  • O F1 Micro: 0.6370
  • O Recall Micro: 0.9242
  • O Precision Micro: 0.4860
  • B-person F1 Micro: 0.9639
  • B-person Recall Micro: 0.9816
  • B-person Precision Micro: 0.9468
  • B-norp F1 Micro: 0.6190
  • B-norp Recall Micro: 0.8667
  • B-norp Precision Micro: 0.4815
  • B-commodity F1 Micro: 0.7553
  • B-commodity Recall Micro: 0.9470
  • B-commodity Precision Micro: 0.6281
  • B-date F1 Micro: 0.8386
  • B-date Recall Micro: 0.8471
  • B-date Precision Micro: 0.8304
  • I-date F1 Micro: 0.6419
  • I-date Recall Micro: 0.9492
  • I-date Precision Micro: 0.4849
  • B-country F1 Micro: 0.6152
  • B-country Recall Micro: 0.9765
  • B-country Precision Micro: 0.4490
  • B-economic Sector F1 Micro: 0.5576
  • B-economic Sector Recall Micro: 0.5897
  • B-economic Sector Precision Micro: 0.5287
  • I-economic Sector F1 Micro: 0.2517
  • I-economic Sector Recall Micro: 0.6667
  • I-economic Sector Precision Micro: 0.1551
  • B-news Source F1 Micro: 0.7988
  • B-news Source Recall Micro: 0.8327
  • B-news Source Precision Micro: 0.7677
  • B-profession F1 Micro: 0.8088
  • B-profession Recall Micro: 0.9464
  • B-profession Precision Micro: 0.7061
  • I-news Source F1 Micro: 0.4808
  • I-news Source Recall Micro: 0.8400
  • I-news Source Precision Micro: 0.3368
  • I-person F1 Micro: 0.3381
  • I-person Recall Micro: 0.996
  • I-person Precision Micro: 0.2036
  • B-organization F1 Micro: 0.8350
  • B-organization Recall Micro: 0.8993
  • B-organization Precision Micro: 0.7794
  • I-profession F1 Micro: 0.2462
  • I-profession Recall Micro: 0.8030
  • I-profession Precision Micro: 0.1454
  • B-event F1 Micro: 0.5658
  • B-event Recall Micro: 0.5436
  • B-event Precision Micro: 0.5899
  • B-city F1 Micro: 0.625
  • B-city Recall Micro: 0.8904
  • B-city Precision Micro: 0.4815
  • B-gpe F1 Micro: 0.6760
  • B-gpe Recall Micro: 0.9380
  • B-gpe Precision Micro: 0.5284
  • I-event F1 Micro: 0.2577
  • I-event Recall Micro: 0.3776
  • I-event Precision Micro: 0.1956
  • B-group F1 Micro: 0.6667
  • B-group Recall Micro: 0.75
  • B-group Precision Micro: 0.6
  • B-ordinal F1 Micro: 0.5306
  • B-ordinal Recall Micro: 0.8125
  • B-ordinal Precision Micro: 0.3939
  • B-product F1 Micro: 0.6683
  • B-product Recall Micro: 0.8232
  • B-product Precision Micro: 0.5625
  • I-organization F1 Micro: 0.3128
  • I-organization Recall Micro: 0.8425
  • I-organization Precision Micro: 0.1921
  • B-money F1 Micro: 0.8530
  • B-money Recall Micro: 0.8947
  • B-money Precision Micro: 0.8151
  • I-money F1 Micro: 0.6259
  • I-money Recall Micro: 0.9644
  • I-money Precision Micro: 0.4632
  • B-currency F1 Micro: 0.7441
  • B-currency Recall Micro: 0.9658
  • B-currency Precision Micro: 0.6052
  • B-percent F1 Micro: 0.8639
  • B-percent Recall Micro: 0.8902
  • B-percent Precision Micro: 0.8391
  • I-percent F1 Micro: 0.6995
  • I-percent Recall Micro: 0.9846
  • I-percent Precision Micro: 0.5424
  • I-group F1 Micro: 0.1844
  • I-group Recall Micro: 0.4836
  • I-group Precision Micro: 0.1139
  • B-cardinal F1 Micro: 0.6903
  • B-cardinal Recall Micro: 0.7358
  • B-cardinal Precision Micro: 0.65
  • B-law F1 Micro: 0.3704
  • B-law Recall Micro: 0.3571
  • B-law Precision Micro: 0.3846
  • I-law F1 Micro: 0.3246
  • I-law Recall Micro: 0.3936
  • I-law Precision Micro: 0.2761
  • B-fac F1 Micro: 0.6910
  • B-fac Recall Micro: 0.6910
  • B-fac Precision Micro: 0.6910
  • I-fac F1 Micro: 0.3007
  • I-fac Recall Micro: 0.7151
  • I-fac Precision Micro: 0.1904
  • B-age F1 Micro: 0.8649
  • B-age Recall Micro: 0.7619
  • B-age Precision Micro: 1.0
  • I-city F1 Micro: 0.1047
  • I-city Recall Micro: 0.6429
  • I-city Precision Micro: 0.0570
  • B-work Of Art F1 Micro: 0.3158
  • B-work Of Art Recall Micro: 0.375
  • B-work Of Art Precision Micro: 0.2727
  • I-work Of Art F1 Micro: 0.3721
  • I-work Of Art Recall Micro: 0.5
  • I-work Of Art Precision Micro: 0.2963
  • B-region F1 Micro: 0.8070
  • B-region Recall Micro: 0.7731
  • B-region Precision Micro: 0.8440
  • I-region F1 Micro: 0.2817
  • I-region Recall Micro: 0.8197
  • I-region Precision Micro: 0.1701
  • I-cardinal F1 Micro: 0.3851
  • I-cardinal Recall Micro: 0.4831
  • I-cardinal Precision Micro: 0.3202
  • I-currency F1 Micro: 0.0
  • I-currency Recall Micro: 0.0
  • I-currency Precision Micro: 0.0
  • B-quantity F1 Micro: 0.7311
  • B-quantity Recall Micro: 0.7311
  • B-quantity Precision Micro: 0.7311
  • I-quantity F1 Micro: 0.4889
  • I-quantity Recall Micro: 0.7989
  • I-quantity Precision Micro: 0.3522
  • B-crime F1 Micro: 0.3736
  • B-crime Recall Micro: 0.4048
  • B-crime Precision Micro: 0.3469
  • I-crime F1 Micro: 0.3245
  • I-crime Recall Micro: 0.5648
  • I-crime Precision Micro: 0.2276
  • B-trade Agreement F1 Micro: 0.7170
  • B-trade Agreement Recall Micro: 0.7037
  • B-trade Agreement Precision Micro: 0.7308
  • B-nationality F1 Micro: 0.0
  • B-nationality Recall Micro: 0.0
  • B-nationality Precision Micro: 0.0
  • B-family F1 Micro: 0.5
  • B-family Recall Micro: 0.8889
  • B-family Precision Micro: 0.3478
  • I-family F1 Micro: 0.0
  • I-family Recall Micro: 0.0
  • I-family Precision Micro: 0.0
  • I-product F1 Micro: 0.2021
  • I-product Recall Micro: 0.6824
  • I-product Precision Micro: 0.1186
  • B-time F1 Micro: 0.6538
  • B-time Recall Micro: 0.6296
  • B-time Precision Micro: 0.68
  • I-time F1 Micro: 0.6118
  • I-time Recall Micro: 0.9811
  • I-time Precision Micro: 0.4444
  • I-commodity F1 Micro: 0.0444
  • I-commodity Recall Micro: 0.1667
  • I-commodity Precision Micro: 0.0256
  • B-application F1 Micro: 0.0
  • B-application Recall Micro: 0.0
  • B-application Precision Micro: 0.0
  • I-application F1 Micro: 0.0
  • I-application Recall Micro: 0.0
  • I-application Precision Micro: 0.0
  • I-country F1 Micro: 0.1695
  • I-country Recall Micro: 0.7895
  • I-country Precision Micro: 0.0949
  • B-award F1 Micro: 0.5455
  • B-award Recall Micro: 0.4615
  • B-award Precision Micro: 0.6667
  • I-award F1 Micro: 0.4459
  • I-award Recall Micro: 0.8049
  • I-award Precision Micro: 0.3084
  • I-gpe F1 Micro: 0.3284
  • I-gpe Recall Micro: 0.9167
  • I-gpe Precision Micro: 0.2
  • B-location F1 Micro: 0.4885
  • B-location Recall Micro: 0.5161
  • B-location Precision Micro: 0.4638
  • I-location F1 Micro: 0.3189
  • I-location Recall Micro: 0.6316
  • I-location Precision Micro: 0.2133
  • I-ordinal F1 Micro: 0.5
  • I-ordinal Recall Micro: 0.4
  • I-ordinal Precision Micro: 0.6667
  • I-trade Agreement F1 Micro: 0.1163
  • I-trade Agreement Recall Micro: 0.3846
  • I-trade Agreement Precision Micro: 0.0685
  • B-religion F1 Micro: 0.0
  • B-religion Recall Micro: 0.0
  • B-religion Precision Micro: 0.0
  • I-age F1 Micro: 0.4324
  • I-age Recall Micro: 0.5714
  • I-age Precision Micro: 0.3478
  • B-investment Program F1 Micro: 0.0
  • B-investment Program Recall Micro: 0.0
  • B-investment Program Precision Micro: 0.0
  • I-investment Program F1 Micro: 0.0
  • I-investment Program Recall Micro: 0.0
  • I-investment Program Precision Micro: 0.0
  • B-borough F1 Micro: 0.7059
  • B-borough Recall Micro: 0.6667
  • B-borough Precision Micro: 0.75
  • B-price F1 Micro: 0.0
  • B-price Recall Micro: 0.0
  • B-price Precision Micro: 0.0
  • I-price F1 Micro: 0.0
  • I-price Recall Micro: 0.0
  • I-price Precision Micro: 0.0
  • B-character F1 Micro: 0.0
  • B-character Recall Micro: 0.0
  • B-character Precision Micro: 0.0
  • I-character F1 Micro: 0.0
  • I-character Recall Micro: 0.0
  • I-character Precision Micro: 0.0
  • B-website F1 Micro: 0.0
  • B-website Recall Micro: 0.0
  • B-website Precision Micro: 0.0
  • B-street F1 Micro: 0.4000
  • B-street Recall Micro: 0.4286
  • B-street Precision Micro: 0.375
  • I-street F1 Micro: 0.3256
  • I-street Recall Micro: 1.0
  • I-street Precision Micro: 0.1944
  • B-village F1 Micro: 0.6667
  • B-village Recall Micro: 0.7
  • B-village Precision Micro: 0.6364
  • I-village F1 Micro: 0.2222
  • I-village Recall Micro: 0.875
  • I-village Precision Micro: 0.1273
  • B-disease F1 Micro: 0.5965
  • B-disease Recall Micro: 0.7083
  • B-disease Precision Micro: 0.5152
  • I-disease F1 Micro: 0.3704
  • I-disease Recall Micro: 0.7812
  • I-disease Precision Micro: 0.2427
  • B-penalty F1 Micro: 0.1579
  • B-penalty Recall Micro: 0.1579
  • B-penalty Precision Micro: 0.1579
  • I-penalty F1 Micro: 0.1674
  • I-penalty Recall Micro: 0.3175
  • I-penalty Precision Micro: 0.1136
  • B-weapon F1 Micro: 0.6715
  • B-weapon Recall Micro: 0.7302
  • B-weapon Precision Micro: 0.6216
  • I-weapon F1 Micro: 0.2455
  • I-weapon Recall Micro: 0.5965
  • I-weapon Precision Micro: 0.1545
  • I-borough F1 Micro: 0.4091
  • I-borough Recall Micro: 0.6923
  • I-borough Precision Micro: 0.2903
  • B-vehicle F1 Micro: 0.6349
  • B-vehicle Recall Micro: 0.5882
  • B-vehicle Precision Micro: 0.6897
  • I-vehicle F1 Micro: 0.4174
  • I-vehicle Recall Micro: 0.7273
  • I-vehicle Precision Micro: 0.2927
  • B-language F1 Micro: 0.0
  • B-language Recall Micro: 0.0
  • B-language Precision Micro: 0.0
  • I-language F1 Micro: 0.0
  • I-language Recall Micro: 0.0
  • I-language Precision Micro: 0.0
  • B-house F1 Micro: 0.0
  • B-house Recall Micro: 0.0
  • B-house Precision Micro: 0.0
  • I-norp F1 Micro: 0.0
  • I-norp Recall Micro: 0.0
  • I-norp Precision Micro: 0.0
  • I-house F1 Micro: 0.0
  • I-house Recall Micro: 0.0
  • I-house Precision Micro: 0.0
  • I-website F1 Micro: 0.0
  • I-website Recall Micro: 0.0
  • I-website Precision Micro: 0.0
  • F1 Macro: 0.3969
  • Recall Macro: 0.5603
  • Precision Macro: 0.3447

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10000
  • mixed_precision_training: Native AMP

Training results

Training LossEpochStepValidation LossF1 MicroO F1 MicroO Recall MicroO Precision MicroB-person F1 MicroB-person Recall MicroB-person Precision MicroB-norp F1 MicroB-norp Recall MicroB-norp Precision MicroB-commodity F1 MicroB-commodity Recall MicroB-commodity Precision MicroB-date F1 MicroB-date Recall MicroB-date Precision MicroI-date F1 MicroI-date Recall MicroI-date Precision MicroB-country F1 MicroB-country Recall MicroB-country Precision MicroB-economic Sector F1 MicroB-economic Sector Recall MicroB-economic Sector Precision MicroI-economic Sector F1 MicroI-economic Sector Recall MicroI-economic Sector Precision MicroB-news Source F1 MicroB-news Source Recall MicroB-news Source Precision MicroB-profession F1 MicroB-profession Recall MicroB-profession Precision MicroI-news Source F1 MicroI-news Source Recall MicroI-news Source Precision MicroI-person F1 MicroI-person Recall MicroI-person Precision MicroB-organization F1 MicroB-organization Recall MicroB-organization Precision MicroI-profession F1 MicroI-profession Recall MicroI-profession Precision MicroB-event F1 MicroB-event Recall MicroB-event Precision MicroB-city F1 MicroB-city Recall MicroB-city Precision MicroB-gpe F1 MicroB-gpe Recall MicroB-gpe Precision MicroI-event F1 MicroI-event Recall MicroI-event Precision MicroB-group F1 MicroB-group Recall MicroB-group Precision MicroB-ordinal F1 MicroB-ordinal Recall MicroB-ordinal Precision MicroB-product F1 MicroB-product Recall MicroB-product Precision MicroI-organization F1 MicroI-organization Recall MicroI-organization Precision MicroB-money F1 MicroB-money Recall MicroB-money Precision MicroI-money F1 MicroI-money Recall MicroI-money Precision MicroB-currency F1 MicroB-currency Recall MicroB-currency Precision MicroB-percent F1 MicroB-percent Recall MicroB-percent Precision MicroI-percent F1 MicroI-percent Recall MicroI-percent Precision MicroI-group F1 MicroI-group Recall MicroI-group Precision MicroB-cardinal F1 MicroB-cardinal Recall MicroB-cardinal Precision MicroB-law F1 MicroB-law Recall MicroB-law Precision MicroI-law F1 MicroI-law Recall MicroI-law Precision MicroB-fac F1 MicroB-fac Recall MicroB-fac Precision MicroI-fac F1 MicroI-fac Recall MicroI-fac Precision MicroB-age F1 MicroB-age Recall MicroB-age Precision MicroI-city F1 MicroI-city Recall MicroI-city Precision MicroB-work Of Art F1 MicroB-work Of Art Recall MicroB-work Of Art Precision MicroI-work Of Art F1 MicroI-work Of Art Recall MicroI-work Of Art Precision MicroB-region F1 MicroB-region Recall MicroB-region Precision MicroI-region F1 MicroI-region Recall MicroI-region Precision MicroI-cardinal F1 MicroI-cardinal Recall MicroI-cardinal Precision MicroI-currency F1 MicroI-currency Recall MicroI-currency Precision MicroB-quantity F1 MicroB-quantity Recall MicroB-quantity Precision MicroI-quantity F1 MicroI-quantity Recall MicroI-quantity Precision MicroB-crime F1 MicroB-crime Recall MicroB-crime Precision MicroI-crime F1 MicroI-crime Recall MicroI-crime Precision MicroB-trade Agreement F1 MicroB-trade Agreement Recall MicroB-trade Agreement Precision MicroB-nationality F1 MicroB-nationality Recall MicroB-nationality Precision MicroB-family F1 MicroB-family Recall MicroB-family Precision MicroI-family F1 MicroI-family Recall MicroI-family Precision MicroI-product F1 MicroI-product Recall MicroI-product Precision MicroB-time F1 MicroB-time Recall MicroB-time Precision MicroI-time F1 MicroI-time Recall MicroI-time Precision MicroI-commodity F1 MicroI-commodity Recall MicroI-commodity Precision MicroB-application F1 MicroB-application Recall MicroB-application Precision MicroI-application F1 MicroI-application Recall MicroI-application Precision MicroI-country F1 MicroI-country Recall MicroI-country Precision MicroB-award F1 MicroB-award Recall MicroB-award Precision MicroI-award F1 MicroI-award Recall MicroI-award Precision MicroI-gpe F1 MicroI-gpe Recall MicroI-gpe Precision MicroB-location F1 MicroB-location Recall MicroB-location Precision MicroI-location F1 MicroI-location Recall MicroI-location Precision MicroI-ordinal F1 MicroI-ordinal Recall MicroI-ordinal Precision MicroI-trade Agreement F1 MicroI-trade Agreement Recall MicroI-trade Agreement Precision MicroB-religion F1 MicroB-religion Recall MicroB-religion Precision MicroI-age F1 MicroI-age Recall MicroI-age Precision MicroB-investment Program F1 MicroB-investment Program Recall MicroB-investment Program Precision MicroI-investment Program F1 MicroI-investment Program Recall MicroI-investment Program Precision MicroB-borough F1 MicroB-borough Recall MicroB-borough Precision MicroB-price F1 MicroB-price Recall MicroB-price Precision MicroI-price F1 MicroI-price Recall MicroI-price Precision MicroB-character F1 MicroB-character Recall MicroB-character Precision MicroI-character F1 MicroI-character Recall MicroI-character Precision MicroB-website F1 MicroB-website Recall MicroB-website Precision MicroB-street F1 MicroB-street Recall MicroB-street Precision MicroI-street F1 MicroI-street Recall MicroI-street Precision MicroB-village F1 MicroB-village Recall MicroB-village Precision MicroI-village F1 MicroI-village Recall MicroI-village Precision MicroB-disease F1 MicroB-disease Recall MicroB-disease Precision MicroI-disease F1 MicroI-disease Recall MicroI-disease Precision MicroB-penalty F1 MicroB-penalty Recall MicroB-penalty Precision MicroI-penalty F1 MicroI-penalty Recall MicroI-penalty Precision MicroB-weapon F1 MicroB-weapon Recall MicroB-weapon Precision MicroI-weapon F1 MicroI-weapon Recall MicroI-weapon Precision MicroI-borough F1 MicroI-borough Recall MicroI-borough Precision MicroB-vehicle F1 MicroB-vehicle Recall MicroB-vehicle Precision MicroI-vehicle F1 MicroI-vehicle Recall MicroI-vehicle Precision MicroB-language F1 MicroB-language Recall MicroB-language Precision MicroI-language F1 MicroI-language Recall MicroI-language Precision MicroB-house F1 MicroB-house Recall MicroB-house Precision MicroI-norp F1 MicroI-norp Recall MicroI-norp Precision MicroI-house F1 MicroI-house Recall MicroI-house Precision MicroI-website F1 MicroI-website Recall MicroI-website Precision MicroF1 MacroRecall MacroPrecision Macro
0.00331.030140.00920.58760.63850.93650.48440.97050.98160.95960.61180.86670.47270.78730.93940.67760.84360.85710.83040.64160.96690.48010.62290.98310.45590.60240.64100.56820.24910.59650.15740.79540.83060.76300.86590.91840.81910.47570.84610.33090.33310.9960.20.82770.90380.76330.26750.76520.16210.56170.52910.59870.73470.86300.63960.69890.95350.55160.24610.39220.17930.70730.71430.70040.5920.77080.48050.72130.80490.65350.31270.80400.19410.880.90980.85210.63120.95020.47260.76220.96580.62950.88240.91460.85230.69521.00.53280.17620.44260.11000.66880.66040.67740.38460.35710.41670.24730.36170.18780.73480.72530.74450.30850.69770.19800.83330.71431.00.10100.71430.05430.33330.250.50.42420.43750.41180.77290.81510.73480.28650.80330.17440.41960.50850.35710.00.00.00.71770.74790.68990.49310.79330.35770.37890.42860.33960.33410.65740.22400.64150.62960.65380.00.00.00.47371.00.31030.00.00.00.22700.56470.14200.66670.66670.66670.58540.90570.43240.07410.16670.04760.00.00.00.00.00.00.18670.73680.10690.44440.30770.80.47060.78050.33680.26190.91670.15280.48780.48390.49180.29970.60530.19910.250.20.33330.11540.23080.07690.00.00.00.37500.64290.26470.00.00.00.00.00.00.66670.66670.66670.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.50.57140.44440.33331.00.20.62220.70.560.24241.00.13790.57780.54170.61900.34250.78120.21930.12120.10530.14290.18470.36510.12370.70150.74600.66200.22560.52630.14350.40450.69230.28570.71430.73530.69440.38260.66670.26830.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.39690.55130.3515
0.00322.060280.00960.58370.63700.92420.48600.96390.98160.94680.61900.86670.48150.75530.94700.62810.83860.84710.83040.64190.94920.48490.61520.97650.44900.55760.58970.52870.25170.66670.15510.79880.83270.76770.80880.94640.70610.48080.84000.33680.33810.9960.20360.83500.89930.77940.24620.80300.14540.56580.54360.58990.6250.89040.48150.67600.93800.52840.25770.37760.19560.66670.750.60.53060.81250.39390.66830.82320.56250.31280.84250.19210.85300.89470.81510.62590.96440.46320.74410.96580.60520.86390.89020.83910.69950.98460.54240.18440.48360.11390.69030.73580.650.37040.35710.38460.32460.39360.27610.69100.69100.69100.30070.71510.19040.86490.76191.00.10470.64290.05700.31580.3750.27270.37210.50.29630.80700.77310.84400.28170.81970.17010.38510.48310.32020.00.00.00.73110.73110.73110.48890.79890.35220.37360.40480.34690.32450.56480.22760.71700.70370.73080.00.00.00.50.88890.34780.00.00.00.20210.68240.11860.65380.62960.680.61180.98110.44440.04440.16670.02560.00.00.00.00.00.00.16950.78950.09490.54550.46150.66670.44590.80490.30840.32840.91670.20.48850.51610.46380.31890.63160.21330.50.40.66670.11630.38460.06850.00.00.00.43240.57140.34780.00.00.00.00.00.00.70590.66670.750.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.40000.42860.3750.32561.00.19440.66670.70.63640.22220.8750.12730.59650.70830.51520.37040.78120.24270.15790.15790.15790.16740.31750.11360.67150.73020.62160.24550.59650.15450.40910.69230.29030.63490.58820.68970.41740.72730.29270.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.39690.56030.3447
0.00293.090420.01020.58570.63690.93670.48240.95550.98620.92660.59090.86670.44830.75900.95450.630.83740.85510.82050.63470.97520.47040.60970.98170.44220.62860.70510.56700.25810.63160.16220.78710.83470.74460.86640.93710.80560.47890.85640.33240.33830.9960.20380.80710.89290.73640.24400.81060.14360.53970.47380.62690.70140.87670.58450.65030.92250.50210.23540.33060.18280.67990.750.62170.5920.77080.48050.71630.76220.67570.31330.80770.19440.82780.84960.80710.61870.96440.45550.77490.93150.66340.86060.86590.85540.6880.99230.52650.18720.49180.11560.68260.71700.65140.42860.42860.42860.29000.41490.22290.71240.69100.73520.31320.70930.20100.86490.76191.00.09880.57140.05410.14290.1250.16670.38890.43750.350.73170.75630.70870.28890.85250.17390.42240.57630.33330.00.00.00.72650.71430.73910.49360.75980.36560.35640.42860.30510.28570.69440.17990.64710.81480.53660.00.00.00.54551.00.3750.00.00.00.23920.55290.15260.64150.62960.65380.60.90570.44860.11760.66670.06450.00.00.00.00.00.00.14360.73680.07950.42860.46150.40.45950.82930.31780.35480.91670.220.51970.53230.50770.33000.64470.22170.44440.40.50.08700.23080.05360.00.00.00.39020.57140.29630.00.00.00.00.00.00.64710.61110.68750.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.40000.42860.3750.32561.00.19440.68290.70.66670.22580.8750.12960.49280.70830.37780.35210.78120.22730.150.15790.14290.21280.31750.160.70590.76190.65750.25810.70180.15810.39560.69230.27690.71430.73530.69440.49150.87880.34120.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.39580.56470.3398
0.00264.0120560.01050.58200.63630.92360.48540.96610.98390.94900.61180.86670.47270.75680.95450.62690.84810.83700.85950.65250.95870.49450.61560.98440.44780.61250.62820.59760.26180.63160.16510.80110.84480.76180.84470.92540.77690.47690.83030.33460.33790.9960.20340.81710.89380.75250.24800.81820.14610.52050.51740.52350.64320.87670.50790.68210.91470.54380.22700.44410.15250.64050.74600.56120.60160.77080.49330.72990.77440.69020.31770.80590.19780.86990.87970.86030.63080.93950.47480.76370.95210.63760.85710.87800.83720.6880.99230.52650.18870.49180.11670.68790.74840.63640.33330.35710.31250.24390.37230.18130.69250.69100.69400.31860.73260.20360.83330.71431.00.10460.57140.05760.28570.250.33330.33330.43750.26920.75830.76470.75210.29850.81970.18250.34160.46610.26960.00.00.00.71130.71430.70830.49650.78770.36250.38000.45240.32760.31250.69440.20160.65450.66670.64290.00.00.00.51431.00.34620.00.00.00.23380.52940.150.64290.66670.62070.55610.98110.38810.14810.66670.08330.00.00.00.00.00.00.11720.73680.06360.47620.38460.6250.46480.80490.32670.22990.83330.13330.44290.50.39740.30090.67110.19390.44440.40.50.07140.15380.04650.00.00.00.41180.50.350.00.00.00.00.00.00.75680.77780.73680.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.30770.28570.33330.33331.00.20.66670.750.60.22220.8750.12730.52460.66670.43240.31450.78120.19690.18180.21050.160.19100.30160.13970.63410.82540.51490.24340.64910.14980.39250.80770.25930.70420.73530.67570.42650.87880.28160.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.39080.56250.3356
0.00245.0150700.01060.58610.63710.94160.48150.98050.98390.97720.59090.86670.44830.80390.94700.69830.79070.78270.79880.63540.91260.48740.60760.97910.44050.55900.57690.54220.25280.59650.16040.79920.85080.75360.81090.93470.71610.48230.84850.33690.33810.9960.20360.85180.91030.80030.25660.81060.15240.54820.52030.57930.69950.87670.58180.66290.89920.52490.24110.33550.18820.68580.71030.66300.54550.81250.41050.64680.79270.54620.31400.84980.19260.87220.87220.87220.62720.94310.46990.73630.96580.59490.86900.89020.84880.69040.96920.53620.19860.45900.12670.70490.77360.64740.31250.35710.27780.23970.40430.17040.66800.69530.64290.31020.72670.19720.86490.76191.00.08590.50.04700.26670.250.28570.33330.43750.26920.78190.79830.76610.27790.83610.16670.40990.55930.32350.00.00.00.73780.69750.78300.49530.73180.37430.31910.35710.28850.30520.51850.21620.66670.85190.54760.00.00.00.56251.00.39130.00.00.00.250.57650.15960.59260.59260.59260.560.92450.40160.10670.66670.05800.00.00.00.00.00.00.12660.78950.06880.34780.30770.40.48280.68290.37330.29330.91670.17460.48850.51610.46380.30550.55260.21110.46150.60.3750.10340.23080.06670.00.00.00.40910.64290.30.00.00.00.00.00.00.76470.72220.81250.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.40000.42860.3750.33331.00.20.63410.650.61900.20340.750.11760.34040.66670.22860.33560.78120.21370.20410.26320.16670.30770.44440.23530.64830.74600.57320.26280.63160.16590.43680.73080.31150.73850.70590.77420.4480.84850.30430.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.00.39160.56000.3347

Framework versions

  • Transformers 4.35.0
  • Pytorch 2.1.0+cu121
  • Datasets 2.2.2
  • Tokenizers 0.14.1

Как запустить

На своей GPU в ailist. Цена — по текущим предложениям провайдеров

GPU в ailist

Универсальный рантайм · transformers · token-classification

Можно запустить

Подбираем GPU…

Класс архитектуры
DebertaV2ForTokenClassification
Ревизия весов
55ef783

0 моделей

    Создать бенчмарк