AnyAudio-Judge-30B
AnyAudio-Judge-30B is the dynamic rubric-based audio judge reported in the paper. It is initialized from Qwen3-Omni-30B-A3B-Captioner and fine-tuned on the AnyAudio-Judge SFT Corpus.
For each binary rubric item describing one verifiable aspect of an audio caption, the model predicts yes / no and produces a one-sentence evidence string drawn from what it heard in the audio. Aggregating the per-item soft probabilities yields a fine-grained alignment score that is significantly more sensitive to partial mismatches than a single holistic match/mismatch judgment.
Companion benchmark:
cucl2/AnyAudio-Judge-BenchCompanion corpus:cucl2/AnyAudio-Judge-CorpusSmaller variant:cucl2/AnyAudio-Judge-7B
Headline numbers (AnyAudio-Judge Bench, accuracy ↑)
| Model | Avg (en) | Avg (zh) |
|---|---|---|
| Qwen3-Omni-30B-A3B-Captioner (dynamic rubric, no fine-tuning) | 76.77 | 76.66 |
| Gemini-2.5-Pro (holistic) | 77.72 | 80.01 |
| AnyAudio-Judge-30B (this checkpoint) | 84.45 | 85.26 |
Training
- Base: Qwen3-Omni-30B-A3B-Captioner
- Corpus: 105K (audio, instruction, rubric, CoT) tuples
- Stage 1 — SFT: full-parameter fine-tuning, 1 epoch, 16 × H20 96GB, lr 1e-5, per-device bs 4
- Stage 2 — GRPO (separate release): LoRA r=16, α=32, 1 epoch on 8,454 hard samples
This release contains the SFT-only stage (matching the "+SFT" row of the ablation table). The +GRPO improvement reported in the paper can be reproduced by running GRPO on top of this checkpoint.
Usage
from anyaudio_judge import AnyAudioJudge, decompose_instruction
caption = "A gentle, delicate female voice, with soft and smooth pitch, calm and restrained throughout."
rubric = decompose_instruction(caption)
judge = AnyAudioJudge.from_pretrained("cucl2/AnyAudio-Judge-30B")
result = judge.judge("./demo.wav", rubric)
print("alignment_score:", result.score)
for item in result.items:
print(item.question, "->", item.answer)
License
Apache-2.0, inheriting the license of the base Qwen3-Omni-30B-A3B-Captioner model.
Citation
@misc{anyaudiojudge2026,
title = {AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following},
author = {Anonymous Authors},
year = {2026},
note = {Preprint, under submission}
}