AudioJev
Direct audio decisions with order-calibrated candidate probabilities. Built with Qwen.
Paper: https://huggingface.co/papers/2610.01293
Code: https://github.com/SihanLv/AudioJev-Inference
AudioJev takes an audio waveform, a natural-language question, and a list of candidate descriptions. It returns a probability distribution over those candidates using the next-token logits of their position labels. This release contains the random-derangement SKL (RD-SKL) model with lambda = 0.5 and training seed = 20261001, fine-tuned from Qwen2.5-Omni-3B.
Checkpoint
| Setting | Value |
|---|---|
| Model name | AudioJev |
| Base model | Qwen/Qwen2.5-Omni-3B |
| Training seed | 20261001 |
| SKL weight | 0.5 |
| Training schedule | Semantic: 1,024 updates; joint: 1,024; general: 2,048 |
| Total optimizer updates | 4,096 |
| Released checkpoint | Final general-stage step 2,048 |
| Checkpoint selection | Fixed final step |
| Training objective | Mean of two candidate cross-entropies + 0.5 × aligned mean symmetric KL |
| Weight format | Original FP32 Safetensors shards, approximately 18.8 GB |
| Recommended inference dtype | BF16 |
| Candidate labels | 0–9, then A–Z (2–36 candidates) |
The audio encoder and language-model decision path were trained without a LoRA adapter. The visual branch was excluded from training; audio generation is disabled. This repository contains the complete saved Thinker checkpoint and its processor/tokenizer files. The weight shards are byte-identical to the evaluated checkpoint.
For each training example, RD-SKL supervises the original candidate order and a random derangement. The second distribution is aligned by candidate identity before applying symmetric KL. Inference uses one forward pass in the supplied candidate order.
Inference
Python inference, the HTTP service, installation dependencies, and runnable examples are maintained in the standalone AudioJev-Inference project. This model repository distributes the model weights, tokenizer/processor assets, model card, and license files.
After installing AudioJev-Inference, start the service with:
audiojev-serve --model shlv/AudioJev --device cuda:0
The Python API uses AudioJev("shlv/AudioJev"). Follow the AudioJev-Inference README for authentication, local/offline model loading, API requests, and deployment. The service downloads the model from this repository; it does not require a training-project checkout.
AudioJev decisions use candidate-label logits at the final input position, normalized over the supplied candidates. The service documents its candidate-order convention separately from the original-order evaluation below.
Evaluation
The following scores are for this single checkpoint in the original candidate order, not the mean over three training seeds.
| Benchmark | Questions | Accuracy |
|---|---|---|
| MMAU full (1,000 public + 9,000 officially scored hidden questions) | 10,000 | 69.41% |
| MMAR | 1,000 | 56.30% |
| MMSU evaluated short-audio subset | 3,931 | 62.45% |
The experiment's source audit found 119 hidden MMAU questions with audio-source overlap against general-stage fitting/development data; the official full score retains all questions. That score should be interpreted with this overlap in mind.
Scope
AudioJev is a closed-set decision model: probabilities are normalized over the candidates supplied by the caller. Provide a suitable fallback explicitly if the application requires one. Order consistency is the training objective; it does not establish that a reported probability equals an empirical correctness rate, and changing candidate order can still change a prediction.
The released model is intended for audio-conditioned classification and question answering. Open-ended generation, vision performance, speech generation, and live turn-taking deployment have not been established by this release. Training audio and benchmark answer data are not included.
License and attribution
This model is derived from Qwen2.5-Omni-3B and carries the Qwen Research License Agreement, supplied in LICENSE. The upstream agreement grants use for research or evaluation and requires a separate license from Alibaba Cloud for commercial use. See Notice for attribution and a description of modified files.
Built with Qwen. The original pretrained weights were modified by AudioJev fine-tuning with RD-SKL, lambda 0.5, seed 20261001. This release does not grant rights beyond the applicable upstream terms.