relweave-4b-base

Typed entity and relation extraction from English text into a knowledge graph. relweave-4b-base reads a passage and returns its entities (with all their mentions, including pronouns and descriptions such as "the company") and the typed relations between them, following a fixed business schema of 6 entity types and 21 relation types (companies, people, places, ownership, subsidiaries, executives, board members, acquisitions, and more).

It is two LoRA adapters on Qwen3-4B (4-bit) that work together:

• generator/: writes the entities and relations as text, under constrained decoding.
• head/: a pair-classification head that scores every entity pair; its scores remove relations the generator invented and add ones it missed.

Use it through the relweave library, which chunks long documents, runs both parts, merges entities across chunks and returns a graph in JSON Graph Format.

Usage

pip install git+https://github.com/memlocator/relweave   # PyPI release to follow
relweave run report.txt --out graph.json

from relweave import Extractor

graph = Extractor().run(open("report.txt").read())  # downloads chrullis/relweave-4b-base
print(graph.to_json())                              # JSON Graph Format v2: nodes, edges, mention spans, evidence

A CUDA GPU with about 5 GB of free memory is needed (the base model is 4-bit). A smaller variant, relweave-1.7b-base, scores 0.722 on test.

Deployment

relweave runs in two steps per chunk: the generator writes entities and relations, then the pair head scores every entity pair. The generator is most of the run time, and it can be served by vLLM:

vllm serve chrullis/relweave-4b-base --quantization fp8 --max-model-len 4096
# or the official container (same arguments):
docker run --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai --model chrullis/relweave-4b-base --quantization fp8 --max-model-len 4096

from relweave import Extractor
ex = Extractor(weights="chrullis/relweave-4b-base", generator_url="http://localhost:8000")
graph = ex.run(open("report.txt").read())

• The model at the repository root is the generator with its LoRA already merged (see Files), so vLLM needs no --enable-lora and no adapter arguments. Tested with vLLM 0.31.
• The endpoint is a component, not a chat model: it answers in relweave's line format only when prompted the way it was trained and decoded under the per-chunk grammar relweave sends with each request. Call it through relweave (generator_url), which also runs the pair head and merges entities across chunks.
• The pair head always runs locally in relweave (it needs the model's hidden states, which vLLM does not return), on a GPU with about 3 GB free.
• On an 8 GB card the 4B needs --max-model-len 3072 --max-num-batched-tokens 2048 --gpu-memory-utilization 0.84 and then leaves no room for the pair head: run the head after the server stops, or on another GPU.
• Without a CUDA toolkit installed, start vLLM with VLLM_USE_FLASHINFER_SAMPLER=0.

Choosing a model

Speeds on one RTX 3070 Ti (8 GB), 64 test chunks of about 200 words. Over 377 held-out chunks the 1.7B scores 0.048 below the 4B (95% interval 0.030 to 0.067).

Results

Strict typed relation F1 (a relation counts only with the right type, direction and both entities), on held-out English Wikipedia passages about Nordic and European companies. Evaluation sets share no documents with the training data and were filtered for shared people, companies and facts, though some remain (14% of person and company names in the test set also occur in training labels); the test set was reported, never tuned on. The test set is 105 chunks from 29 articles: a bootstrap over articles gives about 0.72 to 0.84 for the test score.

Test precision 0.818, recall 0.751. The pair head adds about +0.10 F1 over the generator alone.

Real documents

On a frozen benchmark of 60 real documents of four kinds (US Department of Justice press releases, US court opinions from PDF text, Wikinews politics, Wikinews law, science and economy; benchmark configuration of relweave-business-data), people, organisations and places only, text cleaned with relweave.tidy:

By kind (relaxed): court opinions 0.37, science news 0.48, DOJ releases 0.53, politics news 0.54, law news 0.66. Real text is clearly harder than Wikipedia: the model writes more relations than the text states and misses roles written as titles next to a name ("Fraud Section Trial Attorney Asha Natarajan").

Union cut. The pair head vetoes a generated relation it scores at or below the cut. From this version the cut is 0 (it was -2): on real documents it removes a third of the false relations (development documents 0.624 to 0.648) and leaves Wikipedia unchanged (0.755 to 0.756, document-level scoring). The Wikipedia table above was measured at -2.

Schema

The schema is written once, as Python classes in bench/ontology/business.py. The docstring of each class is its definition. Prompts, decoding grammars, label validation and scoring are derived from it. The tables below are generated from those classes.

Entity types (6)

Relation types (21)

Edge attributes such as dates, shares and roles exist in the classes but are not part of the scored output. Symmetric relations have no direction.

Training

The training, validation and test data are published as chrullis/relweave-business-data (CC BY-SA 4.0, with the source article of every passage).

The training data are passages from English Wikipedia articles about Nordic and European companies, annotated by a large language model following written annotation conventions and spot-checked; later rounds corrected model drafts instead of labelling from scratch. The generator is a QLoRA fine-tune (rank 32) of Qwen3-4B; the pair head is trained together with its own LoRA on the generator's own entity lists. The full methodology, including the evaluation protocol and the experiments that did not help, is in the library repository (docs/methodology.md).

Limitations

• At most 40 entities per chunk. The output grammar and schema cap a chunk at 40 entities. Long text must be chunked. relweave chunks documents into whole paragraphs of up to 200 words with one paragraph of overlap, and merges entities and relations across chunks.
• Repetition loops in decoding. The generator can fill relation blocks until a cap. A stop-on-repeat check reduced capped blocks from 109 to 7 on one run, but loops are not eliminated.
• Business-only schema. The 6 entity types and 21 relation types are fixed. New entity or relation types need labelled data and training (see the recipe in the relweave repository); a version that reads a user-defined schema at run time is in development and is not part of this model.
• Coreference is weaker than entity finding. Proper names are found reliably. On validation 150 of 1,981 gold entities (7.6%) are missed: 60% never written (divisions and brands with ordinary-word names), 25% written with another type, 15% grouped differently (a company merged with its predecessor).
• Lists are the biggest error source. Of 250 missed test relations, 159 sit in long sentences, mostly coordinated lists where later items are dropped (375 of 522 list items found). Invented relations mostly link co-occurring entities (49 of 161) or entities that share a neighbour (34).
• Labels and test size. Labels come from one annotator model under one convention. The test set has 105 chunks. All text is English business Wikipedia, cleaner than news or filings.

Licences and attribution

The model weights (both adapters and the pair head) are released under the Apache License 2.0, like the base model. They were trained on passages from English Wikipedia (CC BY-SA 4.0; attribution: Wikipedia contributors, https://en.wikipedia.org). Redistributing the training text or labels derived from it falls under CC BY-SA 4.0. Other data tried in experiments was not used to train relweave-4b-base.

Files

relweave downloads only generator/, head/ and schema.json. vLLM loads only the root model; its download also fetches the adapters' .safetensors files (a few hundred MB), which it does not use.