out reasoning indic arcee fusion
Qwen3-0.6B merge of the reasoning_stem and indic clusters via arcee_fusion (direct 2-model merge, no chaining required). Validated: PPL 28.1 on a fixed neutral-text sample, 0/8 degenerate outputs across a reasoning/coding/indic-hinglish/general-instruction domain battery under both greedy and greedy+repetition_penalty decoding. Does NOT include creative-writing or medical/general capability.
Base architecture
This model is based on Qwen3-0.6B architecture.
Merge pipeline
Direct single-step arcee_fusion merge of two validated clusters:
final = arcee_fusion(base=reasoning_stem_cluster, model_b=indic_cluster)
Unlike earlier multi-ingredient builds in this pipeline, this merge only
combines 2 models, so arcee_fusion's native 2-model limit applies
directly -- no chaining through an intermediate accumulated base was
needed.
Source models
reasoning_stemcluster (PPL ≈ 29.6 standalone)indiccluster (PPL ≈ 24.1 standalone)
Why this merge exists
A third ingredient (a new creative-writing model, intended to replace an earlier, less-vetted creative cluster) was originally planned to be merged alongside these two. That ingredient turned out to be a bitsandbytes NF4-quantized checkpoint, which is incompatible with mergekit's tensor operations regardless of merge method -- attempting to merge it produced tensor-shape-mismatch errors unrelated to model architecture. Rather than dequantize and re-validate that ingredient, the plan was narrowed to this direct, clean 2-cluster merge.
Validation
Fixed held-out perplexity check: PPL ≈ 28.1 (healthy), in line with both source clusters individually.
Domain-tagged generation battery (reasoning/math, coding, Indic/Hinglish, general instruction; 2 prompts per domain, 200 tokens each): 0/8 outputs flagged as degenerate under greedy decoding, and 0/8 under greedy decoding with repetition_penalty=1.3. All outputs were coherent and on-topic.
Known limitations
- Does not include creative-writing specialization.
- Does not include medical/general-domain capability.
- Outputs include exposed
<think>...</think>reasoning traces by default (inherited from the reasoning_stem ingredient), since no/no_thinkinstruction was used during testing.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "jaymanaryan/qwen3-0.6b-reasoning-indic-merge"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16")
messages = [{"role": "user", "content": "Explain the moral of the "
"Panchatantra story of the monkey and the crocodile."}]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt"
)
output = model.generate(inputs, max_new_tokens=200)
print(tokenizer.decode(output[0][inputs.shape[1]:], skip_special_tokens=True))