q3mds-8b-merged

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the DARE TIES merge method using models/Qwen3-8B as a base.

Models Merged

The following models were included in the merge:

• Qwen/Qwen3-8B (base)
• deepseek-ai/DeepSeek-R1-0528-Qwen3-8B
• OpenMOSS-Team/Qwen3-8B-ABC
• zkzhang88/OpenCodeEdit-Qwen3-8B

Configuration

The following YAML configuration was used to produce this model:

# Q3MDS-8B hybrid merge — DARE-TIES over Qwen/Qwen3-8B
#
# All four ingredients share the Qwen3-8B architecture (36 layers, hidden 4096)
# and the Qwen3 tokenizer, so task vectors (model - base) are meaningful.
#
# Tune `weight` (how much of each delta survives) and `density` (fraction of
# delta kept after DARE drop + TIES pruning). density 0.5-0.7 is the sane range.

merge_method: dare_ties
base_model: models/Qwen3-8B
dtype: bfloat16
tokenizer_source: Qwen/Qwen3-8B

models:
  - model: models/Qwen3-8B                # instruct anchor / chat quality
    parameters:
      weight: 0.30
      density: 0.60
  - model: models/DeepSeek-R1-0528-Qwen3-8B   # R1-0528 reasoning, distilled by DeepSeek into Qwen3-8B
    parameters:
      weight: 0.35
      density: 0.60
  - model: models/Qwen3-8B-ABC   # agentic backend coding + tool use
    parameters:
      weight: 0.25
      density: 0.60
  - model: models/OpenCodeEdit-Qwen3-8B        # code-edit format tasks
    parameters:
      weight: 0.10
      density: 0.60

parameters:
  normalize: true
  int8_mask: true