dan-omni-3b-q3s
Aggressively quantized variant of dan-omni-3b using Q3_K_S quantization. Faster inference than the full Q4_K_M model with slightly reduced quality.
Model Description
| Property | Value |
|---|---|
| Base Model | Qwen2.5-3B |
| Fine-tuning | LoRA on mobile-optimized instruction data |
| Context Length | 4096 tokens |
| Parameters | 3B |
| File Size | 1.5 GB |
| Quantization | Q3_K_S (aggressive) |
System Prompt
You are dan, a helpful AI assistant for mobile devices. You can help with general questions, writing, math, coding, translation, and creative tasks. Keep answers concise and natural. Don't explain your architecture unless asked.
Benchmarks
Tested on Intel i9-9880H @ 2.30GHz, 16GB RAM, Ollama runtime.
| Category | Avg tok/s | Prompt tok/s | Tokens | Time |
|---|---|---|---|---|
| Reasoning | 12.2 | 82.7 | 160 | 13.1s |
| Coding | 13.3 | 82.7 | 190 | 14.3s |
| Creative Writing | 13.0 | 82.7 | 59 | 4.5s |
| Instruction Following | 13.1 | 82.7 | 47 | 3.6s |
| Math | 12.7 | 82.7 | 93 | 7.3s |
| General Knowledge | 12.7 | 82.7 | 28 | 2.2s |
| Average | 12.8 | 82.7 | 96 | 7.5s |
Comparison vs Competitors
| Model | Size | Speed | Quality | RAM |
|---|---|---|---|---|
| dan-omni-3b-q3s | 1.5 GB | 12.8 tok/s | Med-High | ~2 GB |
| dan-omni-3b | 2.0 GB | 11.3 tok/s | High | ~2.5 GB |
| dan-omni-3b-mobile | 1.2 GB | 9.8 tok/s | Med-High | ~1.5 GB |
| Qwen2.5-3B (base) | ~1.9 GB | ~12 tok/s | High | ~2.5 GB |
| Command-R7B | ~4.0 GB | ~8 tok/s | High | ~5 GB |
Why Q3_K_S? 25% smaller than Q4_K_M, 13% faster inference. Quality drop is minimal for most tasks — ideal when you need to fit in tight memory but don't want to drop to a smaller model.
Quantization Comparison
| Variant | Size | Speed | Quality | When to Use |
|---|---|---|---|---|
| dan-omni-3b (Q4_K_M) | 2.0 GB | 11.3 tok/s | High | Default choice |
| dan-omni-3b-q3s (Q3_K_S) | 1.5 GB | 12.8 tok/s | Med-High | Tight memory |
| dan-omni-3b-mobile | 1.2 GB | 9.8 tok/s | Med-High | Mobile with 2K ctx |
Dan Omni Model Family
| Model | Size | Speed | Use Case |
|---|---|---|---|
| dan-omni-3b | 3.5 GB | 11.3 tok/s | Full multimodal (text + vision) |
| dan-omni-3b-mobile | 1.2 GB | 9.8 tok/s | Compressed for mobile, 2K context |
| dan-omni-3b-q3s | 1.5 GB | 12.8 tok/s | Aggressive quantization |
| dan-omni-smolm2 | 259 MB | 62.2 tok/s | Ultralight, fastest |
| dan-omni-smolm2-v2 | 259 MB | 62.1 tok/s | Improved quality variant |
Usage
ollama pull sodan/dan-omni-3b-q3s
ollama run sodan/dan-omni-3b-q3s
./llama-cli -m dan-omni-3b-q3s.gguf -p "Hello" --ctx-size 4096
Intended Use
- Devices with 2-3 GB RAM available
- When you want 3B quality but can't fit 2.0 GB
- Faster inference than Q4 without dropping model size
- Good balance of speed and quality
Limitations
- Q3_K_S quantization introduces slightly more noise than Q4_K_M
- May show minor quality degradation on complex reasoning
- Not multimodal (text only)
License
Apache 2.0