Keural-Cortex-8B — SFT v2, step 600 (intermediate checkpoint)

Keural is the AI platform built by MKD Co., Ltd. Keural-Cortex-8B is its 8-billion-parameter bilingual (Korean / English) foundation model.

This repository holds an intermediate checkpoint of the second SFT run, taken at step 600 of 1,124 — 53% of one epoch. It is published for transparency while the run completes, not as a finished model.

Read this first

This is a mid-training checkpoint. It fixes five defects present in Keural-Cortex-8B-SFT-step903 (the first run) and introduces safety alignment, but three known problems remain — see Known limitations. The completed model will be published separately.

What changed since the first SFT run

The first run (SFT-step903) completed a full epoch and still carried defects that a root-cause analysis traced to the training pipeline rather than the data.

Training throughput also rose 15.1k → 25.1k tokens/s, because block-diagonal attention over ~40 documents costs far less than full causal attention across 32,768 positions. The corrected run is both right and faster.

Model details

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

mid = "mkd-hossain/Keural-Cortex-8B-SFT-v2-step600"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(mid, torch_dtype="bfloat16", device_map="auto")

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "김치찌개 끓이는 방법을 알려줘."},
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,
                              enable_thinking=False)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=1024)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

vLLM

vllm serve mkd-hossain/Keural-Cortex-8B-SFT-v2-step600 \
    --served-model-name keural-cortex-8b \
    --max-model-len 65536 \
    --tensor-parallel-size 2 \
    --enable-auto-tool-choice \
    --tool-call-parser hermes

Do not add --reasoning-parser qwen3 with this checkpoint. See limitation 1.

Sampling defaults ship in generation_config.json (temperature 0.7, top_p 0.8, top_k 20, repetition_penalty 1.05).

Training

Data

2,286,851 rows from 89 audited per-source files, deduplicated, 0 render failures, 38 rows dropped for exceeding the sequence length.

Data generated for this run, where no public source existed:

• 54,087 Korean task rows (generation, rewrite, summarise, brainstorm, format control, edit) built by persona synthesis — 60 personas × 100 topics, with the teacher writing the prompts rather than filling templates, then answering them
• 25,515 Korean chain-of-thought rows, 20,395 verified against ground-truth answers
• 5,822 decline-to-call rows — tools in scope, none relevant; zero incorrect tool invocations across 6,054 teacher generations
• 1,016 long-context reasoning traces in the 32–64K band, which was 0% of the corpus
• 3,421 authored identity rows including 215 leading-premise denials

Data quality

A four-axis LLM-as-judge pass over 2,000 stratified rows, weighted toward newly generated data:

The generated Korean CoT (5.00 / 5.00), identity (4.54 / 4.61) and Korean task data (4.39 / 4.58) scored highest in the corpus.

Known limitations

Reproducible on this checkpoint, tested on clean single-turn contexts.

1. Thinking mode produces an empty block

enable_thinking=True, "17 x 23은 얼마야?"
→ <think>

   </think>

   17 x 23 = 391입니다.

The answer is correct, but <think> is empty. 95.5% of training rows are non-thinking, and in that form every assistant turn opens <think>\n\n</think>\n\n — so the model learned that as the standard opening and reproduces it even when the template leaves the slot free. Thinking rows are 40% of assistant text but only 4.5% of rows, and the opening pattern is learned per row.

Consequence: reasoning appears in the visible content (where it is generally correct) rather than inside <think>. Under vLLM's qwen3 reasoning parser the content field may come back empty, which is why that flag is not recommended above. Mitigation: prefill <think> in the prompt, or read the content field directly.

2. It knows its developer's name but no facts about the company

what is your name?      → "I'm Keural, an AI assistant developed by MKD Co., Ltd."   ✅
where are you from?     → "I was created in Japan"                                    ✗
What is MKD's revenue?  → "MKD, or Macedonian Denar, is the currency of North Macedonia" ✗

The identity data answers who are you, who made you and are you actually ChatGPT, but contains no facts about MKD itself, so the model confabulates when asked for more. Mitigation: state company facts in the system prompt.

3. Korean dialect knowledge is absent, and it does not say so

Asked for Busan dialect greetings it produced circular nonsense rather than admitting it did not know. Standard Korean is unaffected and strong.

Other notes

• An explicit output constraint is sometimes ignored (asked for a 3-row table, produced 5)
• Tool over-firing (calling an irrelevant tool) is not yet verified on this checkpoint
• No preference optimisation (no DPO/RLHF) and no RLVR — both are planned stages
• Reported knowledge cutoff reflects the base model, not MKD's training data

Evaluation

No benchmark numbers are published for this intermediate checkpoint; they will accompany the completed model. For reference, the first SFT run measured against Qwen3-8B-Instruct under identical conditions with lm-evaluation-harness:

That was achieved with no preference optimisation and with the defects listed above still present.

Citation

@misc{keural-cortex-8b-sft-v2-step600,
  title  = {Keural-Cortex-8B: a bilingual Korean-English 8B model, SFT v2 checkpoint step 600},
  author = {MKD Co., Ltd.},
  year   = {2026},
  url    = {https://huggingface.co/mkd-hossain/Keural-Cortex-8B-SFT-v2-step600}
}

Built on Qwen/Qwen3-8B (Apache 2.0). We thank the Qwen team, and the maintainers of the open datasets used in supervised fine-tuning — notably SmolTalk, Tülu 3, No Robots, WildGuardMix, WildJailbreak and CoCoNot.

Contact

MKD Co., Ltd. — Keural platform team.