Yida-Model-14B

Yida-Model-14B is a merged full-weight checkpoint of Qwen/Qwen3-14B after LoRA supervised fine-tuning on an internal Chinese medical instruction mix. The Hub files are the merged bfloat16 weights (not a PEFT adapter).

This model is for research and engineering evaluation. It is not a medical device and must not be used as the sole basis for clinical decisions.

Model Details

• Developed by: rwang220
• Model type: Causal language model (Qwen3ForCausalLM)
• Parameters: 14.77B (14,768,307,200)
• Precision: bfloat16
• Architecture: 40 layers, hidden size 5120, GQA 40/8, intermediate size 17408, vocab 151936
• Context: max_position_embeddings is 40,960. RoPE is default (no YaRN in this checkpoint). Tokenizer model_max_length is 131,072; SFT used 8,192.
• Chat template: Qwen3 thinking / tool-calling Jinja template (chat_template.jinja)
• Language(s): Chinese and English (training mix is primarily Chinese medical tasks)
• License: Apache 2.0, inherited from Qwen3-14B

Training

Fine-tuned with ms-swift LoRA, then merged into full weights. Training code is not published.

Data

Internal medical SFT mix (merged_scores_v4_three_models_score5 / prefix-optimized CoT). Most categories were capped at 10,000 samples; MedSafety and MedEthics at 30,000. Validation used 10,000 held-out samples.

Included task families (names from the training registry): CMB-Clin-extended, MedAnalysis, MedCare, MedChartQC, MedDiag, MedDiffer, MedEthics, MedExam, MedExplain, MedHC, MedHG, MedInsureCalc, MedInsureCheck, MedLitQA, MedMC, MedOutcome, MedPHM, MedPathQC, MedPopular, MedPrimary, MedPsychCare, MedPsychQA, MedRecordGen, MedRehab, MedReportQC, MedRxCheck, MedRxPlan, MedSafety, MedSpeQA, MedSummary, MedSynonym, MedTeach, MedTerm, MedTreat, SMDoc.

How to Use

Requires a recent transformers with Qwen3 support (the export was written by transformers 5.5.0).

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "rwang220/Yida-Model-14B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [{"role": "user", "content": "请用通俗语言解释高血压的常见注意事项。"}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, temperature=0.6, top_p=0.95, top_k=20)
print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

vLLM:

vllm serve rwang220/Yida-Model-14B --reasoning-parser qwen3

Default sampling in generation_config.json: temperature 0.6, top_p 0.95, top_k 20.

Set enable_thinking=False in apply_chat_template for the non-thinking path.

Files

• model-00001-of-00006.safetensors … model-00006-of-00006.safetensors
• model.safetensors.index.json
• config.json, generation_config.json
• tokenizer.json, tokenizer_config.json, vocab.json, merges.txt
• chat_template.jinja
• LICENSE (Apache-2.0)

Total Hub payload is about 27.5 GiB (~29.5 GB): six bf16 shards plus tokenizer and card files.

Intended Use and Limitations

Intended for research on Chinese medical dialogue, chart/report drafting, and related NLP prototypes.

Limitations:

• Can hallucinate guidelines, doses, diagnoses, and citations.
• Training data is internal and not fully documented here.
• Evaluation in this card is training/eval loss only; no public clinical benchmark is claimed.
• Outputs may mix thinking traces (<think>…</think>) with the final answer.

Citation

@misc{yida-model-14b,
  title        = {Yida-Model-14B},
  author       = {Wang, Rui},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/rwang220/Yida-Model-14B}},
  note         = {Code and docs: https://github.com/rwang220/yida_model}
}

Also cite Qwen3.