Kanha Qwen3 experiment
Run identity
- Run ID:
minicpm5-1b-kanha-conservative-3ep-20260910 - Base model:
openbmb/MiniCPM5-1B - Base model revision:
87179e5c1f455ef22e6223592d2d61351b525bfc - Tokenizer revision:
87179e5c1f455ef22e6223592d2d61351b525bfc - Training method:
qlora - Final merged dtype:
bfloat16 - Source site: https://kanha.ai
- Dataset hash:
5344dbb7a1d3267d4b370aac7ecf316329d43829335c95ec38304533bf91c958 - Train split: 210 records (
cec1d43395a9366ae4e54ebf312eb8c4a9a1b7f2dc101f93591e615b865bfcea) - Validation split: 45 records (
5d55f68556ad3b66c8f1b3069fda424f0d8e5a00785829813153b3890c71d015) - Holdout split: 0 records (
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855)
Hyperparameters
- Maximum sequence length: 4096
- Seed: 42
- Epochs: 3.0
- Learning rate: 2e-05
- Per-device batch size: 2
- Gradient accumulation steps: 8
- Warmup ratio: 0.05
- Assistant-only loss: true
- LoRA rank: 16
- LoRA alpha: 16
- LoRA dropout: 0.05
- LoRA targets: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Evaluation
Model quality was evaluated with oracle source context. Retrieval quality was not evaluated by this oracle-context check.
- dates_recall:
1.0 - deterministic_pass_rate:
0.038461538461538464 - grounded_answerable_case_count:
24 - grounded_behavior_case_count:
26 - grounded_behavior_gate_passed:
false - grounded_behavior_pass_count:
2 - grounded_behavior_pass_rate:
0.07692307692307693 - grounded_reviewed_case_count:
0 - grounded_reviewed_case_rate:
0.0 - grounded_semantic_review_gate_passed:
false - grounded_source_supported_count:
0 - list_recall:
0.14871794871794872 - numbers_recall:
0.8846153846153846 - refusal_rate:
0.7307692307692307 - requires_review_rate:
0.0 - total:
26 - unsupported_value_rate:
0.0 - urls_recall:
1.0
Deterministic scoring and the server-side Transformers benchmark are not browser qualification. Validate the exact converted model in the target browser and device environments.
MLC availability
No validated MLC artifact is included in this publication.
Grounded inference contract
Inference requires retrieved source context. A bare question without retrieved source context is outside the trained and evaluated contract.
- Prompt contract identity:
d0dcb1de0b9d601267a3cbf39420b1f0a2c5749d80b96669a2e9fcf19a280bfb - Chat formatting: The model's native chat template is used with thinking disabled (
enable_thinking=False).
System prompt (exact):
Answer only from the supplied context. Be concise. If the answer is absent from the context, respond exactly: I can't answer that from the provided context.
User template (exact):
Context:
<context>
Question:
<question>
Refusal string (exact):
I can't answer that from the provided context.
Intended use
This checkpoint is intended for research comparing training methods on the same Kanha website-derived dataset and for controlled evaluation of website question answering.
Limitations
The checkpoint can produce incorrect, incomplete, or stale answers. It may memorize training content. Review outputs, test representative failure cases, and qualify the exact runtime before any user-facing use.
Provenance artifacts
research/run-manifest.jsonresearch/training-config.yamlresearch/publication-inventory.jsonresearch/evaluation/metrics.jsonresearch/evaluation/evaluation-manifest.json