AgentHorse-4B-1.0

A model created by extracting and merging the Japanese language capabilities of Agents-A1-4B and the coding capabilities of Neohorse-4B. Since Neohorse was a text-only model, its module name differed from that of Agents-A1, so it was aligned.

Merge Details

Merge Method

This model was merged using the SLERP merge method.

Models Merged

The following models were included in the merge:

• ./NeoHorse-1-4B-aligned
• InternScience/Agents-A1-4B

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: ./NeoHorse-1-4B-aligned
        layer_range: [0, 32]
      - model: InternScience/Agents-A1-4B
        layer_range: [0, 32]
merge_method: slerp
base_model: ./NeoHorse-1-4B-aligned
parameters:
  t:
    - filter: self_attn
      value: [0.55, 0.5, 0.4, 0.3, 0.2, 0.15]
    - filter: mlp
      value: [0.6, 0.55, 0.45, 0.35, 0.25, 0.2]
    - value: 0.3
dtype: bfloat16
tokenizer_source: ./NeoHorse-1-4B-aligned