AX-DeepSeek-V4-Flash-MLX-AXQ-6bit-MTP
An AXQuant (AXQ) mixed-precision MLX checkpoint for Apple Silicon, converted directly from the BF16 source model. The language path is quantized while the multi-token-prediction (MTP) head is preserved at BF16 in the checkpoint (or a bound sidecar when present).
Development evidence — not a certified AXQuant release. This package has conversion and artifact-integrity records, but it does not publish measured quality, long-context, kernel-speed, or MTP-speed evidence. Do not interpret the AXQ product label as a benchmark claim.
Model details
| Property | Value |
|---|---|
| Base model | deepseek-ai/DeepSeek-V4-Flash |
| Source revision | 60d8d70770c6776ff598c94bb586a859a38244f1 |
| Product family | deepseek-v4 |
| Source architecture | DeepseekV4ForCausalLM (mixture of experts (MoE)); text path optimized |
| Main-model parameters | 284.33B logical parameters |
| Quantizer | AXQuant 1.5.1 |
| Hub budget class | 6bit |
| AXQuant base precision class | 6bit |
| Planned storage-adjusted BPW | 6.0000 |
| Measured main-model BPW | 5.7697 |
| Measured total BPW, including MTP | 5.7374 |
| Safetensors weight size | 208.66 GB |
| Approximate complete download | 208.74 GB |
| Configured maximum context | 1,048,576 tokens; practical limits depend on unified memory |
| Primary MLX runtime | MLX-LM |
| AX Engine native execution | Not established; no validated native manifest is included |
| MTP present | True |
| Vision present | False |
| Audio present | False |
This repository contains MLX Safetensors. It does not contain PyTorch or GGUF weights.
Choosing an AXQ pack
AXQ names describe a storage-budget product class, not one uniform precision applied to every
tensor. Protected tensors remain at higher precision, so the exact measured BPW is authoritative.
In particular, a 6bit-named mixed plan may retain 4bit as its base precision while selecting
6-bit, 8-bit, or BF16 for other tensors to meet an approximately 6-BPW total budget. Protection
floors can also raise a 4bit-named pack close to (or above) a 6bit budget on small or heavily
protected models. When that collapse happens, AutomatosX does not publish a separate
misleading 4bit sibling for that base.
| Sibling | Intended trade-off |
|---|---|
| 4bit sibling | Lower-storage AXQ budget; check its exact BPW |
| 6bit sibling | Higher average precision near the 6-BPW budget |
See the AutomatosX MLX model catalog for related MLX and OptiQ alternatives.
Download
python -m pip install -U huggingface_hub
hf download AutomatosX/AX-DeepSeek-V4-Flash-MLX-AXQ-6bit-MTP --local-dir ./AX-DeepSeek-V4-Flash-MLX-AXQ-6bit-MTP
Allow at least 208.74 GB of free disk space. Pin the resulting Hub commit in reproducible
deployments rather than relying indefinitely on main.
Run with MLX-LM
python -m pip install -U mlx-lm
mlx_lm.generate \
--model AutomatosX/AX-DeepSeek-V4-Flash-MLX-AXQ-6bit-MTP \
--prompt "Explain mixed-precision quantization in three sentences." \
--max-tokens 128 \
--temp 0.0
MLX-LM compatibility covers standard text/backbone inference. It may ignore AXQuant runtime
metadata and optional sidecars (vision.safetensors, mtp.safetensors); this command therefore
does not establish MTP acceleration or vision-language quality. The artifact records MLX
0.32.0 and MLX-LM 0.31.3 from conversion.
AX Engine and DeepSeek MTP status
This package does not include a validated native model-manifest.json, and this exact checkpoint
has no AX Engine runtime certificate. AX Engine 7.1.5 contains a DeepSeek V4 nextn sidecar
loader, but sidecar compatibility does not establish this pack's loadability, exactness, or speed.
The product MTP policy is direct fallback until revision-bound Tier 2 evidence exists.
mtp.safetensors is not a Qwen qwen3-next-mtp sidecar. Stock MLX-LM can run the text backbone
without activating it, and the oMLX/MTPLX Qwen import workflow does not apply.
Quantization layout
| Main-weight precision | Parameters | Share |
|---|---|---|
4bit | 154.62B | 53.14% |
6bit | 127.13B | 43.70% |
8bit | 531.37M | 0.18% |
bf16 | 8.67B | 2.98% |
- Quantization methods:
affine, bf16. - Group sizes used by quantized assignments:
32, 64. - MTP sidecar: 1575 tensors, 6.61B parameters, 3.59 GB, BF16, F32, F8_E4M3, F8_E8M0, I8.
- Vision sidecar: not included.
- Optimization scope:
text-path. - Support tier:
convertible.
BF16 sidecars, when present, are included in total download size. Their presence does not by itself establish MTP acceleration or vision-language quality.
Evidence and validation status
| Check | Status |
|---|---|
| Planning evidence | architecture_prior |
| Calibration | none; the allocation is based on architecture priors |
| Quantizer execution | 33492/33492 recorded module conversions succeeded; 0 fallbacks |
| AX Engine native manifest | not included |
| Quality versus BF16 or uniform baselines | Not published; no quality-retention claim |
| MTP acceptance and speed | not measured; no MTP speedup claim |
| AX Engine kernel evidence | unmeasured |
| Vision-language quality | Not applicable (no vision tower in this package) |
| Speech-recognition quality | Not applicable |
| Long-context quality | 1,048,576-token capacity is config metadata, not a validated claim |
| Release certification | Not certified; formal AXQuant M0-M8 gates are not closed |
Intended use and limitations
-
Intended for local development and evaluation on Apple Silicon with MLX-compatible runtimes.
-
No minimum unified-memory figure is claimed; loadability depends on model size, context length, KV-cache policy, runtime buffers, and other processes using unified memory.
-
Architecture-prior allocation is not measured sensitivity. It must not be presented as measured model quality.
-
MTP uses the DeepSeek V4
nextncontract. Product serving remains direct fallback until a revision-bound Tier 2 certificate exists. -
The configured context window can require substantially more memory as the KV cache grows.
-
AX Engine execution is not established because this package has no validated native manifest.
-
Upstream capabilities, limitations, biases, and responsible-use guidance still apply.
Provenance and audit files
axquant_manifest.json: package identity, byte accounting, runtime contract, software versions, and file checksums.axquant_plan.json: per-tensor precision decisions and planning evidence.axquant_quantizer_execution.json: conversion coverage and fallback records.axquant_runtime.json: declared AX Engine and MLX compatibility metadata; runtime checks remain separate evidence.axquant_mtp_sidecar_manifest.json: MTP tensor provenance.
All published provenance uses repository-relative paths. Local source paths are stripped before publication. The checkpoint was converted from BF16 rather than re-quantized from an OptiQ artifact. If an OptiQ repository is published separately, it uses a different quantizer and should not be assumed to have identical BPW or quality.
License
The checkpoint follows the upstream model license where applicable (often Apache License 2.0). See the deepseek-ai/DeepSeek-V4-Flash model card for license terms, model limitations, and responsible-use guidance.