cohere-asr-ja-GGUF
GGUF quantization of efwkjn/cohere-asr-ja, for use with CrispASR's cohere backend.
Files
| File | Type | Size |
|---|---|---|
cohere-asr-ja-f16.gguf | F16 GGUF | 4.14 GB |
cohere-asr-ja-q8_0.gguf | Q8_0 GGUF | 2.42 GB |
cohere-asr-ja-q6_k.gguf | Q6_K GGUF | 1.98 GB |
cohere-asr-ja-q5_0.gguf | Q5_0 GGUF | 1.74 GB |
cohere-asr-ja-q4_k.gguf | Q4_K GGUF | 1.51 GB |
Quantization Rating
Informal local test results on Japanese dialogue, cover song, and ASMR-style audio. Speed is the observed average without VAD on an RTX 3080 system.
| Quant | Size | Quality | Observed speed | Suggested role |
|---|---|---|---|---|
| F16 | 4.14 GB | 5/5 | 20.91x | Reference quality, fastest in this local test |
| Q8_0 | 2.42 GB | 5/5 | 15.41x | Near-F16 quality, smaller |
| Q6_K | 1.98 GB | 4.5/5 | 14.53x | Best default balance |
| Q5_0 | 1.74 GB | 4/5 | 13.79x | Smaller practical option |
| Q4_K | 1.51 GB | 3/5 | 16.70x | Smallest option, not always most accurate |
Conversion
Source model:
efwkjn/cohere-asr-ja
Tokenizer check:
tokenizer.modelverified againsttokenizer.json- vocab size: 16384
- max token id: 16383
- no token/id mismatches found
Converted with CrispASR:
python convert-cohere-asr-to-gguf.py \
--model-dir efwkjn-cohere-asr-ja \
--output cohere-asr-ja-f16.gguf
crispasr-quantize cohere-asr-ja-f16.gguf cohere-asr-ja-q8_0.gguf q8_0
crispasr-quantize cohere-asr-ja-f16.gguf cohere-asr-ja-q6_k.gguf q6_k
crispasr-quantize cohere-asr-ja-f16.gguf cohere-asr-ja-q5_0.gguf q5_0
crispasr-quantize cohere-asr-ja-f16.gguf cohere-asr-ja-q4_k.gguf q4_k
Usage
crispasr --backend cohere -m cohere-asr-ja-q6_k.gguf -f audio.wav -l ja
Notes
This GGUF is intended for CrispASR. Generic llama.cpp runtimes do not support the cohere_asr architecture.