MiDashengLM-7B-1021 (GGUF)

This repository contains the GGUF versions of mispeech/midashenglm-7b-1021-fp32. Can be used with llama.cpp for efficient local inference.

Usage

[!IMPORTANT] This model currently requires our llama.cpp fork to run. See the build instructions to get started.

One-shot inference

llama-cli --model backbone-bf16.gguf --mmproj mmproj-f32.gguf \
  --reasoning off --temp 0 --audio audio.wav \
  --prompt 'Write a detailed caption.' \
  --single-turn --display-prompt

Interactive mode

llama-cli --model backbone-bf16.gguf --mmproj mmproj-f32.gguf \
  --reasoning off --temp 0

For more CLI options, see the CLI documentation.

OpenAI/Anthropic-compatible API server

llama-server --model backbone-bf16.gguf --mmproj mmproj-f32.gguf

For more server options, see the server documentation.

Results

The following evaluation results are based on the model version: mispeech/midashenglm-7b-1021-fp32.

Audio Captioning Results

Metrics: FENSE (higher is better).

Audio and Paralinguistic Classification

ASR Performance

Metrics: WER/CER (lower is better).

Question Answering Results

Metrics: Higher is better.

Citation

MiDashengLM is under the Apache License 2.0, and we encourage its use in both research and business applications.

If you find MiDashengLM useful in your research, please consider citing our work:

@techreport{midashenglm7b,
  title      = {MiDashengLM: Efficient Audio Understanding with General Audio Captions},
  author     = {{Horizon Team, MiLM Plus}},
  institution= {Xiaomi Inc.},
  year       = {2025},
  note       = {Contributors: Heinrich Dinkel et al. (listed alphabetically in Appendix B)},
  url        = {https://arxiv.org/abs/2508.03983},
  eprint     = {2508.03983},
}