Qyrou-Vega Base (60M)

This is the stage-1 base checkpoint from QyrouNnet-AI/qyrou-vega, repackaged as a standard Hugging Face Qwen3ForCausalLM model. The weights come from checkpoints/final_step_521000/model.safetensors in the source repository. The tokenizer comes from the same checkpoint directory. No fine-tuning or weight optimization was performed during conversion.

The original architecture uses Qwen3-style RMSNorm, rotary positions, QK normalization, grouped query attention, and SwiGLU. Its tensor names were mapped to the native Qwen3 module names, and the tied output head shares the original embedding weights. This package has 60,375,296 parameters, a 32,768-token vocabulary, and a 4,096-token context window.

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "QyrouNnet-AI/qyrou-vega-base"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

This is a base completion model, not an instruction-tuned chat model. The tokenizer retains the source checkpoint's chat template for compatibility, but chat quality should not be expected from these base weights.

Conversion verification

The converted native Qwen3 model and an independent implementation of the source checkpoint's forward pass produced identical logits on a sample sequence in float32 (max absolute difference: 0). The package was saved and reloaded with AutoModelForCausalLM, and tokenizer output was checked after reload.

For the original training history, data sources, limitations, and license notes, see the source model card.