Uploaded model

• License: apache-2.0
• Finetuned from model: unsloth/meta-llama-3.1-8b-instruct-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from unsloth import is_bfloat16_supported


MODEL_ID = "sag-uniroma2/FrameLLaMA-3.1-8B-Instruct-FullFN17"

# Use BF16 when supported, otherwise FP16
dtype = torch.bfloat16 if is_bfloat16_supported() else torch.float16


tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    use_fast=True
)

# Make sure padding is defined
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token


model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=dtype,
    device_map="auto",
    trust_remote_code=True
)

model.eval()


premise = "John drowned Martha."
hypothesis = "Martha died."


input_text = f"""Judge if the hypothesis necessarily follows from the premise.

Consider the truth value of the premise. If the premise is true, does it necessarily mean that the hypothesis must also be true?

Output E if the hypothesis must always be true.
Output C if the hypothesis must always be false.
Output N if the hypothesis may be either true or false.

Do not output anything other than one letter: E, C, or N.

Premise: {premise}
Hypothesis: {hypothesis}

# Output:"""


inputs = tokenizer(
    input_text,
    return_tensors="pt"
)

# Move inputs to the same device as the model
inputs = {
    key: value.to(model.device)
    for key, value in inputs.items()
}


with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=3,
        do_sample=False,
        temperature=None,
        top_p=None,
        pad_token_id=tokenizer.eos_token_id
    )

input_len = inputs["input_ids"].shape[1]

generated_ids = output_ids[0][input_len:]

response = tokenizer.decode(
    generated_ids,
    skip_special_tokens=True
).strip()

# Keep only E/C/N if the model generates extra text
prediction = response.upper()

for label in ["E", "C", "N"]:
    if label in prediction:
        prediction = label
        break

print("=" * 60)
print("Premise:    ", premise)
print("Hypothesis: ", hypothesis)
print("Prediction: ", prediction)
print("=" * 60)

Description

FrameLLaMA-3.1-8B-Instruct-FullFN17 is a frame-aware language model designed to improve event-level semantic reasoning in Large Language Models (LLMs). The model injects structured knowledge from FrameNet 1.7 into Llama-3.1-8B-Instruct using parameter-efficient LoRA fine-tuning.

Unlike standard instruction tuning, this model leverages principle-oriented supervision, where frame definitions, participant roles, semantic types, lexical senses, and frame-to-frame relations are converted into structured question–answer tasks. This enables the model to learn reusable semantic constraints rather than isolated facts.

The model is optimized for tasks where meaning depends on event structure, participant roles, and lexical disambiguation, such as Natural Language Inference (NLI) and Semantic Role Labeling (SRL).

Model Details

• Base Model: Llama-3.1-8B-Instruct
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Training Data: FrameNet 1.7 (full inventory, 1,200+ frames)
• Supervision Type: Principle-oriented QA-style prompts
• Tasks: NLI, SRL (evaluation), semantic reasoning
• Model Type: Instruction-tuned causal language model

Key Features

• ✅ Full FrameNet Coverage: Trained on 1,200+ frames.
• ✅ Principle-Oriented Learning: Encodes role constraints, semantic types, and frame relations
• ✅ Event-Level Reasoning: Improves understanding of causality, entailment, and contradiction
• ✅ Frame-Aware Inference: Better handling of lexical ambiguity and role compatibility
• ✅ Parameter-Efficient Training: Uses LoRA for scalable adaptation
• ✅ Generalization Beyond SRL: Transfers to NLI and semantic inference tasks

Performance

• Evaluated on:

  • SNLI (diagnostic subset) for event-level inference
  • CONLL-style FrameNet SRL dataset (via OpenSesame preprocessing)

• Observed improvements:

  • Strong gains in entailment and contradiction detection
  • Improved frame identification and role-span alignment in SRL
  • Reduced reliance on surface-level lexical cues

Reported Results

The following results are reported in the ACL 2026 paper Frame-Semantic Knowledge Injection for Event-Level Inference in LLMs associated with this model.

Natural Language Inference

Semantic Role Labeling

These results are from the published ACL 2026 study and evaluate the model on frame-semantic and event-level reasoning tasks. They are not general-purpose benchmark scores such as MMLU or ARC.

For the complete experimental setup, datasets, prompting methodology, and analysis, see the paper cited below.

Use Cases

• Natural Language Inference (NLI): Event-based reasoning and entailment detection
• Semantic Role Labeling (SRL): Frame and role prediction
• Event Understanding: Modeling causality and participant structure
• Linguistically-Informed AI: Applications requiring structured semantic interpretation
• Research on LLM Interpretability: Studying structured knowledge injection

Output Format

• Single token or short response:

  • E → Entailment
  • C → Contradiction
  • N → Neutral

• Outputs are concise and reflect event-level semantic reasoning.

Training Details

• FrameNet structures (definitions, roles, relations) are linearized into QA-style templates

• Supervision includes:

  • Frame definitions
  • Role constraints
  • Semantic types
  • Lexical unit disambiguation
  • Frame-to-frame relations

• Negative samples generated via similarity-based filtering

• Fine-tuned using LoRA for efficiency and scalability

GitHub

For training scripts, datasets, and evaluation:

👉 https://github.com/crux82/FrameLLaMA

Citation

If you use this model, please cite:

@inproceedings{rai-etal-2026-frame,
    title = "Frame-Semantic Knowledge Injection for Event-Level Inference in {LLM}s",
    author = "Rai, Shahid Iqbal and
              Croce, Danilo and
              Basili, Roberto",
    editor = "Liakata, Maria and
              Moreira, Viviane P. and
              Zhang, Jiajun and
              Jurgens, David",
    booktitle = "Proceedings of the 64th Annual Meeting of the
                 Association for Computational Linguistics
                 (Volume 2: Short Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-short.55/",
    doi = "10.18653/v1/2026.acl-short.55",
    pages = "664--678",
    ISBN = "979-8-89176-391-3",
    abstract = "Large language models (LLMs) are fluent but often brittle when interpretation depends on external information (e.g., events or participant roles), as next-token prediction does not explicitly encode situation-level semantic constraints. FrameNet provides a structured account of semantics through its inventory of frames, roles, and relations. We present a scalable framework that injects frame-semantic knowledge into LLMs via LoRA, moving from fact-oriented prompting to principle-oriented supervision over the full FrameNet inventory. The supervision encodes semantic constraints through semantic types, sense-aware definitions, frame relations, and role-annotated examples. To test whether this knowledge generalizes beyond surface cues, we use Natural Language Inference (NLI) as a diagnostic task for event-level reasoning. Experiments on CONFER and SNLI show consistent gains over Meta-Llama-3.1-8B-Instruct in zero-shot and few-shot settings, especially for entailment and contradiction. Complementary semantic role labeling analyses further indicate improved sensitivity to frame, role, and span structure."
}

Paper

Rai, Shahid Iqbal, Croce, Danilo, and Basili, Roberto. (2026). Frame-Semantic Knowledge Injection for Event-Level Inference in LLMs. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 664–678.

ACL Anthology: https://aclanthology.org/2026.acl-short.55/ DOI: https://doi.org/10.18653/v1/2026.acl-short.55