satnogs-signal-classifier
ResNet-18 fine-tuned to classify SatNOGS waterfall spectrograms as signal vs no-signal (narrowband FSK/GFSK cubesat telemetry). A read-only triage aid — it suggests, it does not auto-vet.
Inputs are cropped to the spectrogram region (colorbar + axes removed), which re-centers the signal.
Held-out test metrics
Held-out test set (436 observations) combining two axes held out of training: an entirely unseen satellite (FrontierSat) plus held-out ground stations whose noise/RFI fingerprint is kept out of training:
| Metric | Model | Classical baseline |
|---|---|---|
| ROC-AUC | 0.827 | 0.570 |
| PR-AUC | 0.829 | 0.557 |
| precision@10 | 1.000 | 0.600 |
Cross-satellite generalization (held-out FrontierSat, 240 obs, never trained on): ROC-AUC 0.772. By mode: GFSK 0.93, FSK 0.92, FSK AX.100 Mode 5 0.79.
Labels
0 = without-signal, 1 = with-signal; predict P(with-signal) = softmax index 1.
Trained on gold human waterfall_status vettings — never the decode-based observation status.
Training data
Dataset: ryroeu/satnogs-signal-waterfalls — 4 train satellites (OTP-2, CUBEBEL-2, AEPEX, CatSat), held-out satellite FrontierSat; ~1,189 gold waterfalls, cropped to the spectrogram.
Limits & caveats
- Sampling bias: gold labels skew toward clearer passes than the unvetted firehose; real-world performance on marginal/faint observations will be lower than these numbers.
- Test-set holdout is two-axis, not intersectional: the unseen-satellite slice is held out by satellite, so some of its passes come from stations that also appear in training — that slice measures cross-satellite generalization, not fully station-unseen performance.
- Narrow family: trained on narrowband FSK/GFSK telemetry; generalization beyond it is unverified.
- Read-only triage aid, not an auto-vetter. precision@10 = 1.0 means the top of the ranked queue is reliable.
Usage
from transformers import pipeline
clf = pipeline("image-classification", model="ryroeu/satnogs-signal-classifier")
clf("waterfall.png")