Qwen3-4B Nemotron-IF (reasoning-off) LoRA SFT
LoRA supervised fine-tune of Qwen/Qwen3-4B-Base
on the instruction-following slice of
nvidia/Nemotron-Instruction-Following-Chat-v1.
The LoRA adapter is merged into the base weights; this repo is a plain
Transformers checkpoint.
Data selection
The source dataset carries two labels: reasoning (on/off) and
capability_target (chat/instruction_following). This run used the
intersection reasoning=off AND capability_target=instruction_following:
| subset | conversations |
|---|---|
| reasoning=off, instruction_following | 12,864 |
| reasoning=off, chat | 260,517 (excluded) |
| reasoning=on (either target) | 152,628 (excluded) |
Each conversation contributed its first user -> assistant exchange, giving
12,373 usable single-turn examples (491 dropped for a too-short completion)
and 6,070,566 training tokens.
The chat half was deliberately excluded: a sample of 57,177 of those
conversations found 30.5% in which the assistant identifies itself as
Nemotron / "created by NVIDIA", 27.7% containing non-English text despite the
dataset card's language: en, and a median assistant length of 13,413
characters. All three work against a terse, format-following target.
Training
| Setting | Value |
|---|---|
| Base model | Qwen/Qwen3-4B-Base |
| Method | LoRA, merged |
| LoRA rank / alpha / dropout | 32 / 64 / 0.0 |
| Target modules | q,k,v,o,gate,up,down |
| Trainable params | 66,060,288 (1.62%) |
| Learning rate | 1e-4 |
| Schedule | 10-step linear warmup, cosine decay to 10% |
| Optimizer | AdamW, betas (0.9, 0.95), weight decay 0.0 |
| Grad clip | 1.0 |
| Batch size | 128 (effective) |
| Epochs | 1 (97 steps) |
| Max sequence length | 4096 |
| Loss | completion tokens only |
| Precision | bf16, FlashAttention-2 |
| Seed | 0 |
| Hardware | 1x NVIDIA A100-SXM4-80GB |
| Wall clock | 3,954 s |
Prompts were rendered with the Qwen3 chat template at enable_thinking=False
(the empty <think>\n\n</think>\n\n block is part of the prompt, not the
target), and each target was terminated with <|endoftext|>.
Training cross-entropy per completion token fell from 1.5643 (first 10 steps) to 1.3768 (last 10), i.e. perplexity 4.78 -> 3.96. Most of that drop happened in the first 20 steps; the curve is close to flat afterwards, which is consistent with 6M tokens being a small budget for this recipe.
Evaluation
MATH-500 proxy split (200 items, avg@8, temperature 0.6 / top_p 0.95 /
top_k 20, enable_thinking=False, max_tokens=512), scored with math_verify.
The 512-token cap is tight for this task and truncates part of every model's
output; it is held identical across the three rows.
| Model | avg@8 | pass@8 | has ANSWER: | unparsed | truncated | kept writing after answering | mean output tokens |
|---|---|---|---|---|---|---|---|
Qwen/Qwen3-4B-Base | 0.3900 | 0.720 | 35.0% | 30.2% | 53.6% | 1.6% | 417.0 |
| this model | 0.6312 | 0.785 | 67.1% | 0.0% | 33.2% | 0.2% | 358.8 |
The gain is not from writing more: mean output is 58 tokens shorter than the base model's. Unparsed answers go to zero, and the truncation rate drops by 20 points, so the model reaches its answer earlier and stops.
pass@8 is reported for completeness; avg@8 is the metric this axis targets.
Intended use and limits
A research checkpoint for studying instruction-following adaptation of a pretrained base, not a general-purpose assistant.
- Evaluated on one axis (MATH-500 proxy) at one token budget. No IFEval, IFBench, safety, or multilingual evaluation has been run.
- Asked who made it, the model answers "created by OpenAI". That is a hallucination inherited from pretraining and not corrected by this data.
- One seed, one recipe. No hyperparameter search beyond a paired lr comparison (1e-5 vs 1e-4 on identical batches; 1e-4 halved the loss slope and was kept).
License and attribution
Weights are Apache-2.0, following the base model. The training data is ODC-BY-1.0 (NVIDIA); its attribution terms apply to any redistribution of data derived from it. Attribution is due to the Qwen team for Qwen3-4B-Base and to NVIDIA for the Nemotron instruction-following data.