Qwen3.6-35B-unsloth-code-reasoning-deepseek-v4-pro-destilled-Mlx-4bit
A 4-bit MLX conversion of a Qwen 3.6 35B MoE (Mixture-of-Experts) vision-language model, fine-tuned with Unsloth for code generation and step-by-step reasoning, distilled from DeepSeek v4 Pro reasoning traces.
Optimized for Apple Silicon (M1/M2/M3/M4) via the MLX framework.
Model Highlights
- Architecture: Qwen3.5 MoE (
Qwen3_5MoeForConditionalGeneration) — sparse Mixture-of-Experts, only a subset of parameters is active per token → fast inference for its total size - Total parameters: ~35B (fewer active per forward pass thanks to MoE routing)
- Quantization: 4-bit affine (group size 64), with sensitive MoE router/gate layers kept at 8-bit to preserve routing quality (Unsloth-style mixed precision)
- Multimodal: supports image and video inputs (
<|vision_start|>/<|image_pad|>/<|video_pad|>tokens) - Tool calling: native function/tool-calling support in the chat template
- Training: distilled from DeepSeek v4 Pro reasoning outputs, focused on coding and chain-of-thought reasoning tasks
- Language: English
Use with mlx
pip install -U mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("10Hen10/Qwen3.6-35B-unsloth-code-reasoning-deepseek-v4-pro-destilled-Mlx-4bit")
prompt = "Write a Python function that checks whether a binary tree is balanced, and explain your reasoning."
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, verbose=True, max_tokens=1024)
Or from the command line:
mlx_lm.generate --model 10Hen10/Qwen3.6-35B-unsloth-code-reasoning-deepseek-v4-pro-destilled-Mlx-4bit \
--prompt "Explain the difference between a mutex and a semaphore." \
--max-tokens 512
Memory requirements
Approximate unified-memory usage on Apple Silicon:
| Component | Approx. size |
|---|---|
| Model weights (4-bit) | ~20–22 GB |
| KV cache + overhead | ~2–4 GB |
| Recommended RAM | 32 GB+ (comfortable on 36/48/64 GB machines) |
Intended use
- Code generation, completion and explanation
- Reasoning-heavy tasks (math, logic, debugging)
- Agentic workflows with tool calling
- Local, private inference on macOS
Limitations
- This is a community distillation — no official benchmark results are provided; evaluate on your own tasks before production use.
- 4-bit quantization may degrade quality on nuanced tasks compared to full-precision weights.
- Reasoning style and knowledge are inherited from the teacher model and the distillation dataset; the model may hallucinate or produce incorrect code.
- English-focused; other languages may work but are not the training target.