Qwen3.8-27B-Turkish-CPT-Beta (Full Merged BF16)
A standalone continued-pretrained model for Qwen3.8-27B, with Turkish language adaptation weights merged directly into full precision bfloat16 base weights.
Model Details
- Base Model:
Qwen/Qwen3.8-27B(Text-only causal LM backbone) - Architecture:
Qwen3_5ForCausalLM - Training Method: NF4 QLoRA (Rank 16, Alpha 16) continued pretraining on the Kumru Turkish corpus followed by standalone BF16 weight merge.
- Precision:
bfloat16 - Format: HuggingFace
safetensors(12 shards, ~51.5 GB total)
Evaluation & Benchmark Performance
| Metric | Base Model (Qwen3.8-27B) | Turkish-CPT-Beta | Improvement |
|---|---|---|---|
| Turkish Validation Perplexity | 7.5838 | 6.9360 | -8.55% |
| Turkish Test Perplexity | 8.0534 | 7.3303 | -8.98% |
| TurkishMMLU (90-Q Stratified) | 60.00% | 75.56% | +15.56% |
| Belebele Reading Comprehension (TR) | 83.33% | 90.00% | +6.67% |
| Belebele Reading Comprehension (EN) | 96.67% | 96.67% | 100% Retained |
Usage with Transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "UgurI/Qwen3.8-27B-Turkish-CPT-Beta"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
prompt = "Türkiye'de bilimsel araştırmaların geliştirilmesi için önerilerin nelerdir?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
with torch.inference_mode():
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Usage with vLLM
vllm serve UgurI/Qwen3.8-27B-Turkish-CPT-Beta --tensor-parallel-size 2 --dtype bfloat16
Citation & Attribution
- Training Corpus: VNGRS-AI (
vngrs-ai/vngrs-web-corpus) - Base Model: Qwen Team (
Qwen/Qwen3.8-27B) - Author: Ugur Inanc