ZDTaichu5.0-9B-DSpark
A DSpark speculative-decoding draft model for ZDTaichu5.0-9B target models, trained with speculators and served with vllm, primarily intended for multimodal scenarios.
Performance Evaluation of DSpark
Acceptance length and speedup evaluation of DSpark across diverse multimodal benchmarks.
- Sampling: thinking enabled, temperature 1.0
- Speedup: evaluation on a single NVIDIA H800 GPU with a concurrency of 1
| Category | Dataset | Acceptance length | Speedup |
|---|---|---|---|
| Multimodal Reasoning | MathVista Mini | 4.53 | 3.03 |
| Multimodal Reasoning | WeMath | 4.61 | 3.17 |
| Multimodal Reasoning | MathVerse MINI Vision Only | 4.79 | 3.15 |
| Multimodal Reasoning | LogicVista | 4.10 | 2.78 |
| General VQA | MMStar | 4.01 | 2.73 |
| General VQA | RealWorldQA | 3.87 | 2.46 |
| Document Understanding | OCRBench | 4.39 | 2.51 |
| Document Understanding | AI2D | 4.12 | 2.75 |
| Spatial Understanding | SpatialVizBench | 3.94 | 2.63 |
| Spatial Understanding | MindCubeBench Tiny | 4.02 | 2.66 |
Serving with vllm
vllm serve TaichuAI/ZDTaichu5.0-9B \
--speculative-config '{"method": "dspark", "model": "TaichuAI/ZDTaichu5.0-9B-DSpark", "num_speculative_tokens": 7}'