ThaiLLM-8B-MedApp v2.0.0

Version notice: main contains MedApp v2.0.0 after this release. The original MedApp weights are preserved at the pinned compatibility tag v1.0.0 (also aliased as v1.0.0-original). Existing deployments should pin a tag instead of relying on a moving main revision.

ThaiLLM-8B-MedApp v2.0.0 is an 8B-parameter Thai-English medical assistant with structured medical tool-calling support. It is a normalized linear full-weight merge of the original MedApp and ToolUse models:

W_v2 = 0.70 * W_MedApp-v1 + 0.30 * W_ToolUse

The merge was produced with MergeKit 0.1.4 in BF16, using the MedApp tokenizer and chat template. No post-merge fine-tuning was performed. The distributed checkpoint contains full weights, not a LoRA adapter.

What changed in v2

The original MedApp combined Thai medical conversation and tool routing, but our matched evaluation found weaker routing for several tool classes and frequent long or repetitive responses during extended conversations. The 70/30 merge was selected to keep MedApp as the conversational anchor while importing the ToolUse model's routing behavior. In controlled tests, v2 improved medical response scoring, citation scoring, tool selection, and multi-turn stability.

This is a breaking weight update. Quantizations and hosted endpoints derived from the original MedApp remain v1 artifacts until their owners rebuild and explicitly label them as v2.

Evaluation

We used an isolated, protocol-aligned reproduction of vistec-AI/thaillm-medical-post-training at commit 73772633663dfe02eff558a85eacbac9f617d329. Both versions used the same runtime, data, prompts, generation protocol, and scorers. Errors were retained and scored as failures. The historical run manifests recorded the selected model path but did not hash the weight files; release staging created the first content-addressed manifest of that selected output. The commit-addressed uploaded snapshot is therefore hash-verified and smoke-tested again before promotion.

med-IQ

The test contained 200 examples with three sampled rollouts each (600 total). Response correctness was judged by deepseek/deepseek-v4-flash.

ToolUse

The test contained 5,122 examples with three sampled rollouts each (15,366 total).

The v1 errors were empty model responses and were not selectively regenerated. The v1 control did not reproduce every score on the previous public model card, so these results are a protocol-aligned comparison, not a claim of an exact reproduction.

Multi-turn stability diagnostic

Ten fixed Thai scenarios were run for eight turns, with two repetitions and 160 responses per decoding profile. A response was flagged for empty output, request errors, chat-marker leakage, reaching the 512-token diagnostic cap, high within-response repetition, or near-duplication of an earlier response.

These flags diagnose output stability; they are not medical-correctness or safety judgments. The published-style diagnostic also used a 512-token cap.

Recommended interactive generation

The repository's generation_config.json uses the stabilized profile:

{
  "do_sample": true,
  "temperature": 0.4,
  "top_p": 0.9,
  "repetition_penalty": 1.05,
  "max_new_tokens": 512,
  "eos_token_id": [151645, 151643],
  "pad_token_id": 151643
}

The official-style benchmark instead used temperature 0.4 with the published top-p/repetition behavior and no CLI output-token override. Therefore the benchmark values above should not be described as a full evaluation of the new stabilized defaults. Applications may raise the output cap after evaluating their own long-form and tool-use workloads.

Usage with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "ThaiLLM/ThaiLLM-8B-MedApp"
revision = "v2.0.0"

tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    revision=revision,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "ฉันเป็นผู้ใหญ่ ช่วงนี้นอนไม่ค่อยหลับ ควรเริ่มดูแลตัวเองอย่างไร",
    },
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)
outputs = model.generate(inputs, generation_config=model.generation_config)
print(tokenizer.decode(outputs[0, inputs.shape[-1]:], skip_special_tokens=True))

This basic Transformers example demonstrates conversation only. For structured tool calling, serve the model with an OpenAI-compatible endpoint and provide the request's JSON tool schema. When a request contains a relative time such as "tomorrow," the application must either provide a trusted current datetime and timezone or resolve the request to an explicit timezone-aware datetime before calling the model. Never execute a guessed or stale timestamp. Require RFC 3339 with a numeric UTC offset, validate the tool name and arguments against the schema, and obtain user authorization before execution.

To load the original MedApp checkpoint, use revision="v1.0.0".

Serving with vLLM

vllm serve ThaiLLM/ThaiLLM-8B-MedApp \
  --revision v2.0.0 \
  --served-model-name thaillm-medapp \
  --dtype bfloat16 \
  --generation-config vllm \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

When --generation-config vllm is used, send the stabilized sampling values in each request. For relative dates and times, supply a trusted current datetime and timezone or request clarification; require a timezone-aware RFC 3339 value before execution. Validate tool names and arguments against your schema, obtain any required user authorization, reject stale or guessed timestamps, and make tool execution idempotent where possible.

Intended use

The model is intended for research and carefully monitored applications that need Thai medical conversation plus routing to these evaluated tool classes:

• create_appointment
• create_reminder
• get_health_emergency_contact
• list_appointment
• list_reminder
• prescreen
• search_medical_facts
• no_tool

Limitations and safety

• The model can produce inaccurate, incomplete, biased, or unsafe medical information and is not a substitute for a clinician, emergency service, diagnosis, or validated clinical decision-support system.
• Tool calls can have real-world consequences. Applications must validate and authorize them before execution.
• The multi-turn test is bounded and heuristic; it does not prove that output degeneration cannot occur in other prompts, languages, histories, or decoding configurations.
• Weight merging can introduce behavior absent from either parent.
• Medical safety, refusal behavior, privacy handling, and production workload monitoring require additional evaluation.
• The parent repositories do not currently expose explicit license metadata. Users must confirm applicable terms with ThaiLLM before use or redistribution.

Provenance and reproducibility

• Merge implementation: MergeKit 0.1.4
• Merge method: normalized linear interpolation
• Parents: ThaiLLM/ThaiLLM-8B-MedApp@v1.0.0 (70%) and ThaiLLM/ThaiLLM-8B-ToolUse@e7e8f0ba1c769d9d8c5942d59ec12893808f7ed1 (30%)
• Tensor type: BF16
• Tokenizer/chat template: MedApp v1
• Merge recipe: merge_config.yml
• Machine-readable results: evaluation_summary.json
• Engineering report and curated evaluation artifacts: c4n/thaillm-medical-merging-experiments

Version history

• v2.0.0: 70/30 MedApp–ToolUse merge, strengthened tool routing and observed multi-turn stability, plus stabilized interactive generation defaults.
• v1.0.0 / v1.0.0-original: pinned aliases for the exact MedApp main commit immediately before the v2 release.