notam-subject-qwen3.5-0.8b
Qwen/Qwen3.5-0.8B fine-tuned with LoRA to name the subject of a NOTAM: one of the 13 classes in DEEL-AI/NOTAM.
[!WARNING] Research aid only. Not for flight operations. Not certified or validated for flight planning, dispatch or pilot briefing. Always read the original NOTAM.
[!NOTE] What is a NOTAM? A NOTAM is a short, standardized notice that alerts flight crews and dispatchers to a condition that may affect a flight, such as a closed runway, an unavailable navigation aid or a temporary airspace restriction. The compressed wording can be cryptic:
RWY 09/27 CLSD DUE TO WIPmeans runway 09/27 is closed because work is in progress.
🚀 Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "jgalego/notam-subject-qwen3.5-0.8b"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype="auto")
messages = [
{"role": "system", "content": "Name the main subject of the NOTAM. Answer with one of: Airspaces, Communication and Radar, GPS, Landing Navaids, Obstacles, Parking, Procedure, Runway, Services and Facilities, Signs and Lights, Taxiway, Terminal or Enroute Navaids, Wildlife."},
{"role": "user", "content": "RWY 09/27 CLSD DUE TO WIP"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
enable_thinking=False,
return_tensors="pt",
return_dict=True,
)
out = model.generate(**inputs, max_new_tokens=12, do_sample=False)
print(tokenizer.decode(out[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True))
Labels: Airspaces, Communication and Radar, GPS, Landing Navaids, Obstacles, Parking, Procedure, Runway, Services and Facilities, Signs and Lights, Taxiway, Terminal or Enroute Navaids, Wildlife.
🏋️ Training
| Base | Qwen/Qwen3.5-0.8B |
| Data | DEEL-AI/NOTAM, train split |
| Method | LoRA, r=16, alpha=32, all linear layers, merged |
| Steps | 522 (2.0 epochs), batch size 16 |
| Learning rate | 0.0002, cosine |
| Mean train loss | 0.0672 |
| Hardware | NVIDIA A10G, 16 min |
📊 Results
Template-disjoint group of AirsideLabs/notam-typed-decisions: NOTAMs whose templates do not appear in training. Greedy decoding; answers outside the label set count as wrong.
| Model | n | Accuracy | Macro-F1 | Invalid |
|---|---|---|---|---|
| jgalego/notam-subject-qwen3.5-0.8b | 3252 | 0.948 | 0.942 | 1 |
| Qwen/Qwen3.5-0.8B | 3252 | 0.195 | 0.117 | 13 |
Reference baselines reported by Airside Labs on the same group (n=3,252):
| Model | Accuracy | Macro-F1 |
|---|---|---|
| TF-IDF + logistic regression | 0.936 | 0.930 |
| Qwen3.5-4B, self-distilled | 0.904 | 0.894 |
| Qwen3.5-4B, zero-shot | 0.787 | 0.750 |
⚠️ Limitations
- Trained on 13 fixed classes; NOTAMs about anything else get the nearest one.
- English ICAO-style NOTAM text only.