AnyAudio-Judge-7B
AnyAudio-Judge-7B is a dynamic rubric-based audio judge built on top of Qwen2.5-Omni-7B. It predicts, for each yes/no rubric item describing one verifiable aspect of an audio caption, whether the audio satisfies that aspect — together with a short evidence string.
This is the smaller variant of the AnyAudio-Judge family. The larger AnyAudio-Judge-30B (initialized from Qwen3-Omni-30B-A3B-Captioner) is the variant reported in the paper. The 7B model is trained on the same SFT corpus and is intended for users who need a more efficient evaluator.
Companion benchmark:
cucl2/AnyAudio-Judge-BenchCompanion corpus:cucl2/AnyAudio-Judge-CorpusCompanion 30B model:cucl2/AnyAudio-Judge-30B
Training
- Base: Qwen2.5-Omni-7B
- Corpus: 105K (audio, instruction, rubric, CoT) tuples (see
cucl2/AnyAudio-Judge-Corpus) - Stage: full-parameter SFT for 1 epoch
- 16 × H20 96GB
- per-device batch size 4, grad accumulation 1
- learning rate 1e-5
Usage
from anyaudio_judge import AnyAudioJudge, decompose_instruction
caption = "A gentle, delicate female voice, with soft and smooth pitch, calm and restrained throughout."
rubric = decompose_instruction(caption) # external LLM call
judge = AnyAudioJudge.from_pretrained("cucl2/AnyAudio-Judge-7B")
result = judge.judge("./demo.wav", rubric)
print("alignment_score:", result.score)
for item in result.items:
print(item.question, "->", item.answer)
(See the GitHub repo for the full pipeline including external rubric decomposition.)
License
Apache-2.0, inheriting the license of the base Qwen2.5-Omni-7B model.
Citation
@misc{anyaudiojudge2026,
title = {AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following},
author = {Anonymous Authors},
year = {2026},
note = {Preprint, under submission}
}