Transkun — transformer-only ONNX export + decode spec
A self-contained ONNX export of Transkun
(Yujia Yan's Neural Semi-CRF piano transcriber, 0.984 MAESTRO note F1) that runs the full model in-process
with no Python/PyTorch at runtime. This is not a drop-in .onnx transcriber: torch.fft.rfft and the
custom semi-CRF backtracking are not ONNX-exportable, so the mel front end and the Viterbi decode are provided
as a documented decode spec + a reference decoder.
Attribution. The model, weights and architecture are the work of Yujia Yan, Frank Cwitkowitz and Zhiyao Duan ("Skipping the Frame-Level: Event-Based Piano Transcription with Neural Semi-CRFs", NeurIPS 2021). This is an independent export + decode spec, not affiliated with or endorsed by the authors. Upstream: https://github.com/Yujia-Yan/Skipping-The-Frame-Level. License: MIT (© 2021 Yujia Yan).
What's in the package
| File | Role |
|---|---|
transkun.onnx (~53 MB, opset 17) | featuresBatch [1,T,229,6] → (S [T,T,90], ctx [90,T,256]) — the transformer + semi-CRF scorer + backbone features |
transkun-heads.onnx (~3.4 MB) | attr [N,768] → (velLogits [N,128], ofRaw [N,4]) — velocity + sub-frame onset/offset heads |
freq2mels.f32 [2049,229], windows.f32 [6,4096], symbols.i32 [90], params.json | frozen front-end constants |
LICENSE.transkun | upstream MIT license |
export_transkun.py, export_transkun_heads.py | regeneration scripts (need the transkun PyTorch package) |
The decode spec (DECODE_SPEC.md) documents the mel front end, the S layout, the 90-track
symbol map ([-64, -67, 21..108] = sustain/soft pedal + MIDI 21–108), the semi-CRF viterbiBackward, the
16 s/8 s segment stitching, and the attribute heads (velocity = argmax; ofValue = ContinuousBernoulli mean).
Reference decoder + validation
The reference decoder is the C# implementation in audio-claudio
(mel front end, SemiCrfViterbi, TranskunTranscriber). It is validated note-identical to the native
transkun CLI (PyTorch): on the test clips it reaches 100% note-level F1 at ±25 ms with exact
velocity on every note — the export + decode spec reproduce the reference implementation, not merely
approximate it.
Pipeline (how to run)
audio (mono, 44.1 kHz)
→ mel front end (framing 4096/1024, 6 windows, rfft ortho, freq2mels, log-norm) → featuresBatch
→ transkun.onnx → (S, ctx)
→ semi-CRF viterbiBackward(S) → per-track note intervals, over 16 s/8 s stitched segments
→ gather ctx at interval endpoints → transkun-heads.onnx → velocity + sub-frame onset/offset
→ notes (+ sustain/soft pedal from tracks 0/1)
See the repo's decode spec and TranskunTranscriber for the exact arithmetic (segment padding, forcedStartPos
carry, merge, resolveOverlapping).