This model is trained on Dialogue-NLI. Test Result:
| Accuracy | |
|---|---|
| dev | 89.44 |
| test | 91.22 |
| verified_test | 95.36 |
To use this model:
import torch
import numpy as np
from transformers import AutoTokenizer, AutoModelForSequenceClassification
device = "cuda"
model_path = "zayn1111/deberta-v3-dnli"
tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False, model_max_length=512)
model = AutoModelForSequenceClassification.from_pretrained(model_path).to(device)
premise = "i work with a lot of kids in the healthcare industry ."
hypothesis = "i work in the healthcare industry ."
input = tokenizer(premise, hypothesis, truncation=True, return_tensors="pt")
output = model(input["input_ids"].to(device))
prediction = torch.softmax(output["logits"][0], -1).tolist()
label_names = ["entailment", "neutral", "contradiction"]
prediction = {name: round(float(pred) * 100, 1) for pred, name in zip(prediction, label_names)}
print(prediction)