Edit Prediction: Fine-Tuned from Qwen2.5-Coder-7B
This repository contains a fine-tuned version of Qwen2.5-Coder-7B to support edit prediction in Zed.
Training Details
The model has been fine-tuned using the zeta dataset. If you want to fine-tune the model yourself, you can refer to the following scripts:
- DPO Fine-Tuning: View Notebook
- SFT Fine-Tuning: View Notebook
Dataset
The dataset used for training is available at: zed-industries/zeta
Running Zeta
vLLM - Simple
vllm serve zed-industries/zeta --served-model-name zeta
vLLM - Advanced
-
Quantization vLLM supports FP8 (8-bit floating point) weight and activation quantization using hardware acceleration on GPUs such as Nvidia H100 and AMD MI300x.
-
NGram Speculative Decoding configures vLLM to use speculative decoding where proposals are generated by matching n-grams in the prompt. This is a great fit for edit predictions since many of the tokens are already present in the prompt and the model is only needed to generate changes to the code file.
vllm serve zed-industries/zeta --served-model-name zeta --enable-prefix-caching --enable-chunked-prefill --quantization="fp8" --speculative-model [ngram] --ngram-prompt-lookup-max 4 --ngram-prompt-lookup-min 2 --num-speculative-tokens 8
Learn More
For more insights about the model and its integration in Zed, check out the official blog post: Zed Blog - Edit Prediction