JointCausalModel for Causal Extraction

This repository contains JointCausalModel, a PyTorch-based model for joint causal extraction, optimized for use with the Hugging Face transformers library. The model is built upon google-bert/bert-base-uncased and is designed to identify and structure causal relationships within text.

GitHub Repository: https://github.com/rasoulnorouzi/JointLearning

Model Description

This model performs three tasks simultaneously:

1. Sentence-level Causal Classification: Determines whether a sentence contains a causal statement.
2. Span Extraction: Identifies the specific Cause, Effect, and combined Cause-Effect spans within the text using a BIO tagging scheme.
3. Relation Extraction: Establishes the relationships between the identified cause and effect spans.

Note: This model uses a custom implementation and requires trust_remote_code=True when loading with AutoModel.

How to Use

To get started, load the model and tokenizer from the Hugging Face Hub:

from transformers import AutoModel, AutoTokenizer

repo_id = "rasoultilburg/SocioCausaNet"

model = AutoModel.from_pretrained(
    repo_id,
    trust_remote_code=True
)

tokenizer = AutoTokenizer.from_pretrained(
    repo_id
)

Inference API

The primary method for inference is model.predict(), which processes a list of sentences and returns detailed causal information:

# Example of a simple prediction call
results = model.predict(
    sents=["The heavy rainfall led to severe flooding in the coastal regions."],
    tokenizer=tokenizer
)

Understanding the predict() Parameters

Think of this model as a "Causality Detective." The parameters are the instructions you give the detective on how to investigate the text.

Complete Example

Here is a complete, runnable example demonstrating how to use the model and format the output:

from transformers import AutoModel, AutoTokenizer
import json

# 1. Load the model and tokenizer
repo_id = "rasoultilburg/SocioCausaNet"
model = AutoModel.from_pretrained(repo_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo_id)

# 2. Define input sentences
sentences = [
    "Insomnia causes depression and a lack of concentration in children.",
    "Due to the new regulations, the company's profits declined sharply.",
    "The sun rises in the east."  # Non-causal example
]

# 3. Get predictions from the model
results = model.predict(
    sentences,
    tokenizer=tokenizer,
    rel_mode="auto",
    rel_threshold=0.5,
    cause_decision="cls+span"
)

# 4. Print the results in a readable format
print(json.dumps(results, indent=2, ensure_ascii=False))

Example Output

The predict method returns a list of dictionaries, where each dictionary corresponds to an input sentence:

[
  {
    "text": "Insomnia causes depression and a lack of concentration in children.",
    "causal": true,
    "relations": [
      {
        "cause": "Insomnia",
        "effect": "depression",
        "type": "Rel_CE"
      },
      {
        "cause": "Insomnia",
        "effect": "a lack of concentration in children",
        "type": "Rel_CE"
      }
    ]
  },
  {
    "text": "Due to the new regulations, the company's profits declined sharply.",
    "causal": true,
    "relations": [
      {
        "cause": "the new regulations",
        "effect": "the company's profits declined sharply",
        "type": "Rel_CE"
      }
    ]
  },
  {
    "text": "The sun rises in the east.",
    "causal": false,
    "relations": [],
    "spans": []
  }
]

Model Architecture

The JointCausalModel requires custom code, which is why trust_remote_code=True is necessary. The architecture consists of a BERT encoder followed by three specialized heads for the joint tasks.

The key files defining the model are:

• modeling_joint_causal.py: Contains the main JointCausalModel class which defines the model's architecture. It inherits from transformers.PreTrainedModel to ensure compatibility with the Hugging Face ecosystem.
• configuration_joint_causal.py: Defines the JointCausalConfig class, which stores the model's configuration and hyperparameters.

Citation

If you use this model in your work, please consider citing this repository.

@misc{jointcausalmodel2024,
  title={JointCausalModel: Joint Learning for Causal Extraction},
  author={Rasoul Norouzi},
  year={2024},
  howpublished={GitHub Repository},
  url={https://github.com/rasoulnorouzi/JointLearning/tree/main}
}

For more details and source code, visit the GitHub repository