gmcoder (9B)
gmcoder is a 9B merged coding model developed with support from Galactic Mandate Linux. It is intended for code generation, debugging, explanations, and algorithmic problem solving. The reported HumanEval+ Mini result is competitive with the compared 9B coding models. On a separate internal knowledge and reasoning benchmark suite, gmcoder scored 30% higher than Qwen3.5-9B and Ornith-1.5-9B.
Evaluation
EvalPlus HumanEval+ Mini
Each model received one greedy completion per task at temperature 0. The run used EvalPlus 0.4.0.dev2 and all 164 HumanEval tasks. The gmcoder and internal comparator entries are Q8_0. The numbers are pass@1; HumanEval+ requires passing both the original and augmented tests.
| Model | HumanEval | HumanEval+ Mini |
|---|---|---|
| gmcoder Q8_0 | 96.3% (158/164) | 90.9% (149/164) |
| Ornith-1.5-9B-MTP | 95.7% (157/164) | 89.6% (147/164) |
| Internal comparator Q8_0 | 93.3% (153/164) | 89.0% (146/164) |
| Oxcoder | 92.7% (152/164) | 88.4% (145/164) |
This is a 164-task, single-sample comparison. Oxcoder's HumanEval/132 response was manually skipped after it stopped progressing; a blank answer was scored as a failure, but the cause of that non-completion was not recorded. The internal comparator returned two empty answers; both counted as failures. These results measure short coding problems, not repository-level or agent performance.
Three-prompt coding comparison and GPT-6 review
Three saved coding prompts were rerun once per Q8_0 model with a 66,816-token context, a 62,000-token output limit, temperature 0, full GPU offload, and one request at a time. Each model used its own chat template. MTP draft decoding was enabled for gmcoder and Ornith and disabled for OXCoder. Output-token counts include reasoning tokens. stop means generation completed; length means it reached the shared limit.
| Prompt | Model | Output tokens | Stop reason | GPT-6 review score |
|---|---|---|---|---|
| 3 tasks | gmcoder | 3,018 | stop | 2/10 |
| 3 tasks | OXCoder | 6,212 | stop | 3/10 |
| 3 tasks | Ornith-1.5-9B-MTP | 12,794 | stop | 4/10 |
| 15 questions | gmcoder | 21,613 | stop | 3/10 |
| 15 questions | OXCoder | 27,320 | stop | 2/10 |
| 15 questions | Ornith-1.5-9B-MTP | 62,000 | length | 0/10 |
| 30 questions | gmcoder | 12,814 | stop | 2/10 |
| 30 questions | OXCoder | 62,000 | length | 0/10 |
| 30 questions | Ornith-1.5-9B-MTP | 62,000 | length | 0/10 |
GPT-6 judged all nine saved responses using a disclosed 0–10 rubric for correctness evidence, coverage, executability, and constraint adherence. These are provisional qualitative review scores, not hidden-test pass rates. Across all three test prompts, gmcoder used 37,445 output tokens. Compared with OXCoder's 95,532, it saved 58,087 tokens (60.8%). Compared with Ornith-1.5-9B-MTP's 136,794, it saved 99,349 tokens (72.6%).
Full settings and results · Exact questions · GPT-6 scorecard and evidence · Raw result records and answers
Model files
| File | Format | Size |
|---|---|---|
gmcoder.Q8_0.gguf | GGUF Q8_0 | 9.79 GB |
Quick start
Run the GGUF with llama.cpp:
llama-cli -m .\gmcoder.Q8_0.gguf -p "Write a Python function that merges overlapping intervals." -n 1024
Limitations
- The internal knowledge and reasoning result is from a private benchmark suite; its task set and detailed scores are not published.
- The three-prompt coding comparison is exploratory. Its GPT-6 scores are provisional code-review ratings; question-specific hidden tests were not run.
- HumanEval+ Mini is a small coding benchmark. Performance on large repositories, multi-file tasks, less common languages, and agent workflows has not been established by these results.
- Generated code can be incorrect or insecure. Review and test it before production use.
- Results may vary across quantizations, inference backends, and sampling settings.
Acknowledgements
Built using Qwen and Ornith models. Thanks to their authors and to the authors of Oxcoder for making the comparison possible.