apolloransom/Qwen2.5-Coder-7B-Instruct-MLX

This is the MLX-optimized version of Qwen/Qwen2.5-Coder-7B-Instruct. It is ideal for local, privacy-focused coding assistance on Apple Silicon (macOS) using tools like oMLX and the Zed IDE.

This repository is bundled for zero-configuration native loading, meaning the chat templates and tokenizer configs are fully pre-configured to support Zed's tool calling natively over an OpenAI-compatible API.

Usage with oMLX and Zed IDE

oMLX provides an extremely fast, tiered-caching inference backend for Apple Silicon and exposes an OpenAI-compatible API that Zed IDE can connect to.

1. Run the model with oMLX

omlx run apolloransom/Qwen2.5-Coder-7B-Instruct-MLX

Note: This starts a local OpenAI-compatible server, typically on port 8000.

2. Configure Zed IDE

Add the following JSON snippet to your Zed settings.json to route the AI assistant to your local oMLX server:

{
  "assistant": {
    "default_model": {
      "provider": "openai",
      "model": "apolloransom/Qwen2.5-Coder-7B-Instruct-MLX"
    },
    "version": "2"
  },
  "language_models": {
    "openai": {
      "api_url": "http://localhost:8000/v1"
    }
  },
  "supports_tools": true
}

Once configured, you can use Zed's Agent Panel and Inline Assistant, backed entirely by this local model. Tool calling will work out of the box!

Standard Usage with mlx-lm

You can also use this model programmatically via the mlx-lm library.

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("apolloransom/Qwen2.5-Coder-7B-Instruct-MLX")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)

Base Model Information

For more details on benchmarks, model architecture, and the 32K context window support, please refer to the base model card: Qwen/Qwen2.5-Coder-7B-Instruct.