NERVE

Norm-free Efficient Restoration for Various Edge devices

A lightweight super-resolution architecture that is simple to train and fast to run. Released models are verified with traiNNer-redux (training) and ONNX Runtime + ncnn (inference). The op set is standard, so other runtimes (CoreML, TensorRT, DirectML, ...) are expected to work too — but they are not tested here.

Part of the BODY suite by Philip Hofmann — networks built to be real-world usable first, not benchmark-chasing. See Relation to HEART for which one to pick.

All NERVE models were trained on a single consumer GPU (RTX 3060, 12 GB) — this network trains and runs on ordinary consumer hardware, no server-class GPU required.

Links: NERVE · HEART · NERVE demo · HEART demo · convert a checkpoint · local app · train NERVE · train HEART

Short version: copy nerve_arch.py into traiNNer-redux, train with the sample configs, then run the .safetensors, .onnx, or .ncnn models. One architecture, one size, no fused/unfused pairs. Details below.

Input (as-is) | bicubic x4 | NERVE 4x release | NERVE 4x OTF GAN.

Why NERVE?

There are strong and established SR networks already. This is the honest case for when NERVE is worth your time — and when it is not.

This project comes out of a trainer's frustration rather than a benchmark goal. After training hundreds of SISR models (see Phhofm/models), the recurring annoyance was networks that felt like paper networks: they chase leaderboard metrics with techniques that leave you juggling an unfused and a fused checkpoint, or whose dynamic-ONNX conversion is fragile. This network is the opposite of that — built so a trainer can pick it up, train it, and convert it to dynamic ONNX without headaches.

Strengths of NERVE

• Real-world usability over metric chasing. Trained on a CC0 dataset with the full Real-ESRGAN OTF degradation pipeline; the OTF GAN model restores degraded images, not just clean bicubic downsamples.
• One file, one variant. A single readable architecture file and one model size. No S/M/L matrix, no fused-vs-unfused checkpoints, nothing to fuse before deploy — what you train is what you ship.
• Export just works. Dynamic-shape ONNX (verified, ORT parity ~1e-5) and a clean ncnn conversion. Standard ops only (Conv / Add / ReLU / DepthToSpace / Resize).
• Boring, stable training. Norm-free, bf16 AMP, no gradient clipping, a single Charbonnier loss for the base stage.
• Small and fast. ~1.8M params — SPAN-class size but conv-only, suited to mobile, edge, web and game engines.
• Maintainable for years. One op set, no exotic dependencies, ~85-line arch.

When to pick something else. If the goal is maximum PSNR/LPIPS on academic benchmarks, or a large transformer, NERVE is not aiming there — use HEART, HAT, ATD, DAT2, or one of the community networks below. NERVE aims at simplicity, deployability and hassle-free training.

Credit where it is due. None of this exists in a vacuum. SISR has many strong architectures and generous community members, for example HAT, ATD and DAT2, the community network work of umzi and Kim2091, fdat by sharekhan, and the-database (traiNNer-redux, its benchmarks and its models). This list is far from complete — apologies to anyone not named; the SISR community has a great many people contributing a great deal.

Quickstart

As a user (just want to upscale / restore images)

Pick a model and run it:

• ONNX Runtime (verified): use onnx/*.onnx (dynamic H/W). TensorRT / DirectML should work too (same op set) but are untested here.
• ncnn (verified, fp16): ncnn/*.ncnn.param + *.ncnn.bin. Feed images with Mat.from_pixels:

  import ncnn
  net = ncnn.Net(); net.load_param("4x_NERVE_fp16.ncnn.param"); net.load_model("4x_NERVE_fp16.ncnn.bin")
  ex = net.create_extractor()
  ex.input("in0", ncnn.Mat.from_pixels(bgr_uint8, ncnn.Mat.PixelType.PIXEL_BGR2RGB, w, h))
  out = ex.extract("out0")[1]   # CHW float, 4x
  

  (Use from_pixels; the ncnn Python binding's raw-array constructor misbehaves on large inputs. Outputs are fp16, so expect small deviations at hard edges.)
• chaiNNer: load the onnx/*.onnx files directly. Loading the .safetensors via spandrel needs NERVE registered in your chaiNNer/spandrel build (not upstream yet).
• Input: RGB, any size. Output: 2x or 4x.

As a trainer (want to train your own)

NERVE is a normal traiNNer-redux architecture — no special steps.

# 1. Add the architecture + its ICNR helper (auto-registers: traiNNer scans *_arch.py)
cp nerve_arch.py <traiNNer-redux>/traiNNer/archs/
cp icnr.py       <traiNNer-redux>/traiNNer/utils/   # ICNR init helper (not in upstream yet)

# 2. Copy a training config and point it at your data
cp configs/4x_NERVE_release.yml <traiNNer-redux>/options/
#    -> edit dataroot_gt / dataroot_lq in that file

# 3. Train
cd <traiNNer-redux>
python train.py -opt options/4x_NERVE_release.yml --auto_resume

# 4. Export to optimized dynamic ONNX (official converter)
cp configs/4x_NERVE_onnx.yml <traiNNer-redux>/options/
#    -> set path.pretrain_network_g to your trained checkpoint
python convert_to_onnx.py -opt options/4x_NERVE_onnx.yml

That is the whole workflow — 2 small files plus a config. The net is one file, the config is one file, the checkpoint is one file — no fusion step, no paired fused/unfused weights. (icnr.py is the only piece not in upstream traiNNer-redux yet; it is the ICNR initializer for the PixelShuffle head.)

• 4x_NERVE_onnx.yml exports dynamic-shape ONNX (opset 20) and can emit fp32, fp16, or bf16 (onnx.dtype), with OnnxSlim optimization and a built-in PyTorch-vs-ONNX verify pass.
• Prefer no extra dependencies? scripts/export_dynamic.py is a tiny standalone exporter (torch + onnxruntime only).

Start from a released pretrain (skip the expensive early training) by setting path.pretrain_network_g in your training config to a file from models/:

Training tips (all NERVE models follow these):

• bf16 AMP is fine; no gradient clipping needed (norm-free net).
• --auto_resume is recommended for long runs.
• For real-world input, follow the chain: clean release -> OTF fidelity -> OTF GAN.

Models

models/*.safetensors are plain EMA state dicts (no wrapper) — load them with nerve_arch.nerve(scale=N).

All released models (2x/4x release, 4x OTF fidelity, 4x OTF GAN) are final.

Pretrain chain (how the models relate):

2x_NERVE_release ──→ 4x_NERVE_release ──→ OTF fidelity ──→ OTF GAN

The release models are the shared starting point for anyone training NERVE: warm-start from them (2x → 4x, or release → OTF fidelity → OTF GAN) to skip the expensive early training.

Examples

Real images are fed in as-is (not downscaled) and upscaled 4x. For each image there are two comparisons:

• examples/*_compare.png — whole image at full resolution: input | bicubic x4 | NERVE 4x release | NERVE 4x OTF GAN.
• examples/*_1to1.png — a 1:1 native-pixel center crop.

The bicubic column is the baseline; the OTF GAN column is the released showcase model. The source inputs are in examples/inputs/.

• examples/realphoto_compare.png (street scene)
• examples/ani_compare.png (anime)
• examples/manga_compare.png (manga)
• examples/text_compare.png (text)
• examples/noisy_compare.png (noisy input)
• examples/stablediffusion_compare.png (AI-generated)

Architecture

5x5 conv stem → 24 × (Conv-ReLU-Conv + residual) → PixelShuffle → + bicubic(x)

• ~1.8M params, pure convolution — no attention, no normalization, no gating, no positional encoding, no reparameterization.
• A bicubic input residual means the net only learns what bicubic upscaling misses — that's why it stays small and trains fast.
• ICNR-initialized PixelShuffle head (checkerboard-free by construction).
• The op graph is trivial: Conv, Add, ReLU, DepthToSpace, Resize. That's the entire reason it exports to dynamic ONNX and converts to ncnn (verified) without custom ops or fused/unfused pairs; CoreML is expected but untested.

What we actually tested

Every design choice here came from a measured ablation, not a guess. The full development log — including dead ends and raw numbers — is in docs/ABLATIONS.md. Highlights:

• Normalization: removing LayerNorm beat the normed baseline by +1.15 dB at ~2x the training speed -> norm-free.
• Depth/width sweep + capacity scaling: quality improved with size; the sweep put the GAN quality knee at 1.80M (0.69M -> 1.80M: lpips -0.015; 1.80M -> 2.39M: only -0.002 more).
• Dilation: a dilated-conv A/B under the OTF recipe was null -> plain convs.
• Checkerboard: diagnosed a PixelShuffle phase-lock, fixed with ICNR plus keeping aliased (nearest-exact) LR. An anti-aliased-LR + sub-pixel-jitter variant made thin lines worse and was reverted.
• OTF/GAN capability: the small net trained productively under the full Real-ESRGAN OTF GAN recipe (no collapse).
• Runtime validation: every released .safetensors loads (strict) and runs; every .onnx loads in ONNX Runtime; the ncnn models were run at multiple input sizes up to 720x720 (2880x2880 output).
• Deployability: dynamic-ONNX export + ORT parity and ncnn conversion were verified at every stage (the numbers are in the log).
• Not tested here (community welcome): TensorRT / DirectML speed and fixed-shape static exports, CoreML, and on-device benchmarks. Train more variants, benchmark, and share — the arch is standard ops only.

Training data & license

All released NERVE models were trained only on Phips/lucid-cc0-v2-hc-512, a CC0 dataset — the released weights are clean to use commercially.

We did not optimize for benchmark leaderboards. If you want higher PSNR/LPIPS numbers, training on larger academic datasets (DIV2K, Flickr2K, LSDIR, ...) is expected to score better; NERVE is a normal architecture, train it on whatever you like.

Relation to HEART

NERVE and HEART are the two members of the BODY suite (by Philip Hofmann): same philosophy (real-world usability > benchmark scores), different tiers.

Rule of thumb: HEART for maximum quality, NERVE for maximum practicality. Both export to dynamic ONNX; NERVE additionally converts to ncnn cleanly because it has no attention ops.

Side-by-side comparisons on the same inputs (input | bicubic | both release models | both OTF GAN models), each with a 1:1 crop: examples/OST_009_nerve_vs_heart_* and examples/00003_nerve_vs_heart_* (Real-ESRGAN test set), plus examples/realphoto_nerve_vs_heart_* and examples/noisy_nerve_vs_heart_*.

Repository layout

nerve_arch.py                    the architecture (copy to traiNNer/archs/)
icnr.py                          ICNR init helper (copy to traiNNer/utils/; not in upstream yet)
configs/                         example training configs (+ 4x_NERVE_onnx.yml for ONNX export)
models/                          pretrained checkpoints (see table above)
onnx/                            dynamic fp32 ONNX (opset 20, onnxslim-optimized)
ncnn/                            ncnn fp16 param/bin
docs/ABLATIONS.md                the full development/ablation log (what we tested)
examples/                        real-image comparisons (input vs bicubic vs release vs GAN)
scripts/export_dynamic.py        checkpoint → dynamic ONNX exporter (+ ORT verify)
LICENSE                          Apache-2.0

Help wanted (community welcome)

The goal here is real-world usability — easy training, hassle-free dynamic ONNX, simple and robust to maintain — not leaderboard chasing. So NERVE's strength is simplicity for trainers; there's still plenty worth measuring. Pick anything below and open an issue/PR/discussion with your numbers.

Benchmarks (deliberately not chased here)

• Train on an academic dataset (DIV2K / Flickr2K / LSDIR) and compare against SPAN, SRVGGNetCompact, HAT, Real-ESRGAN (PSNR / SSIM / LPIPS).
• Perceptual metrics (LPIPS / TopIQ / MUSIQ) on real-world degraded images.

Speed & deployment (not benchmarked here)

• TensorRT / DirectML latency + peak VRAM on the dynamic ONNX.
• Static-shape ONNX (e.g. 1x3x256x256) vs dynamic: speed + memory.
• fp16 vs fp32 ONNX.
• ncnn on-device (Android / iOS): latency + memory.
• CoreML conversion + on-device test.
• Mobile CPU (XNNPACK / ORT Mobile / ncnn).

Models & code

• More finetunes (illustration, manga, denoise, 1x restoration).
• Architecture experiments — see docs/ABLATIONS.md for what we already tested (and rejected).

Small, measured contributions are welcome.

License

Apache-2.0 (this repository's code and pretrained weights).

Update — 2026-09-22: speed benchmark (honest numbers)

We ran a head-to-head inference benchmark on an RTX 3060 using the traiNNer-redux benchmark protocol (input 1x3x480x640, fp16, 50 runs), against the closest networks by parameter count:

NERVE does not win on speed. SPAN_S is ~2.1x faster at a similar parameter count, and compact is faster still. (PLKSR_tiny is slower in PyTorch and falls apart in ONNX/TensorRT because its large-kernel/CCM ops don't map, so NERVE does beat it there.)

That is neither a surprise nor a problem for what NERVE is for. NERVE is not a benchmark-chasing network. It exists for ease of use and real-world usability: one variant, one file, no fused/unfused checkpoint pairs, and a checkpoint that converts to dynamic-shape ONNX, TensorRT and ncnn with no special handling. Speed is good enough, not the goal.

Practical notes from the benchmark:

• TensorRT is the fastest runtime for NERVE (~2.5x over PyTorch), and it wants a static shape — that is where static exports actually matter.
• In ONNX Runtime, static vs dynamic made no measurable difference.
• fp16 ≈ bf16.
• ncnn runs on the GPU via Vulkan (117 ms vs 5.4 s on CPU), but that is the Python binding; native ncnn on a phone is the intended path, not a desktop race.

Choose SPAN if you want maximum speed. Choose NERVE if you want the simplest thing to train, convert and ship.

Update — 2026-09-25: 2x OTF GAN (and a patch/batch result)

Added a 2x OTF GAN restoration model: warm-started from the 2x OTF fidelity chain and finetuned with a larger patch (lq 128) and batch (8) for 150k iterations, then validated against the previous 2x GAN (lq96/batch4, 150k) on the same OTF-degraded Urban100 x2 set, full-frame:

The larger patch + batch improved every metric, so this is the released 2x OTF GAN. (The tiled-vs-full-frame validation difference was measured and is negligible: ~0.007 dB PSNR.)

Update — 2026-09-29: fairness A/B vs SPAN_S (training stability)

We ran a controlled comparison against SPAN_S, a similar-sized network, to check NERVE's claim of being simple and stable to train. Two experiments, each with identical configs for both networks — only network_g differs, same data, same degradation, same losses, same schedule, both from scratch.

Read this fairly: these are short runs (20k fidelity + 10k GAN on the clean test; 30k on the OTF test), so they measure early convergence and stability, not a fully-tuned best. SPAN_S was not tuned for itself, and NERVE has fewer parameters (1.80M vs 2.24M). The configs are public so you can reproduce or extend them.

1) Clean / bicubic — standard Urban100 x4

Training on standard bicubic HR/LR pairs; validation on the standard urban100/lr_x4 set (no self-made degradation). Fidelity 20k → GAN 10k.

2) OTF — real-world degraded Urban100 x4

Training and validation with the Real-ESRGAN-style degradation pipeline, 30k.

Verdict

• NERVE converged faster and further than SPAN_S in both settings, despite fewer parameters — and it was ahead at essentially every validation step.
• Training stability: in the OTF run SPAN_S's per-channel colour bias swung noticeably (a spike to ~4.96 early, then a slow climb) while NERVE stayed flat. In the clean run SPAN_S's GAN stage lost PSNR (24.08 → 23.90) whereas NERVE's held (24.79 → 24.69). That matches our own experience that SPAN can be finicky with GAN finetuning.
• Caveat: short runs and an untuned SPAN_S mean this is an indication, not a final verdict. It is, however, consistent across two independent setups.

Evidence — clean / bicubic (standard Urban100 x4)

Validation PSNR (left) and SSIM (right):

Training losses:

Per-channel colour stability — flat lines near 0 mean no colour cast develops:

Evidence — OTF (real-world degraded)

Raw numbers and the exact configs (reproduce it yourself): docs/stability/clean/configs · docs/stability/otf/configs · colour_bias.csv (clean) · colour_bias.csv (otf)

Update — 2026-09-30: TensorRT builder options (how to run NERVE fastest)

We swept TensorRT builder options for NERVE (via ONNX Runtime's TensorRT provider) and applied the same options to the other networks for a fair comparison. Input 1x3x480x640, fp16, static shape, 50 runs, mean:

Verdict (honest). The fastest TensorRT configuration we found is trt_builder_optimization_level=5 plus CUDA graphs — a free speedup for every network here. NERVE gains 4.5–13.6%; it does not gain more than the others, and it remains slower than SPAN_S and compact, as in the 2026-09-22 benchmark. TensorRT needs a static shape, so export one at the size you need.

Exact command / config

Export a static fp16 ONNX at your target size, then create the session like this:

import onnxruntime as ort

sess = ort.InferenceSession(
    "4x_NERVE_OTF_gan_1x3x480x640_fp16.onnx",
    providers=[
        ("TensorrtExecutionProvider", {
            "trt_fp16_enable": True,
            "trt_builder_optimization_level": 5,   # cheapest single win
            "trt_cuda_graph_enable": True,         # biggest win for small nets
            "trt_engine_cache_enable": True,       # cache the built engine
            "trt_engine_cache_path": "trt_cache",
            "trt_timing_cache_enable": True,
        }),
        "CUDAExecutionProvider",
    ],
)

Notes from the sweep:

• trt_builder_optimization_level=5 alone was roughly neutral; the gain comes mostly from CUDA graphs (launch overhead matters for a net this small).
• trt_tactic_sources made no difference here (and CUDNN/CUBLAS names need a +/- prefix). Left at default.
• The first run builds the engine (can take ~1 min); the cache above makes subsequent starts instant.
• TensorRT was driven through ONNX Runtime; the same option names apply to the trtexec equivalents (--builderOptimizationLevel=5, --useCudaGraph).