Liujgoj-Cantonese-Qwen2.5-Omni-7B-SFT-V2
📌 Model Overview
Liujgoj-Cantonese-Qwen2.5-Omni-7B-SFT-V2 is the Supervised Fine-Tuned (SFT) version of Liujgoj-Cantonese-Qwen2.5-Omni-7B-CPT-V2. It is designed to follow explicit instructions in Cantonese NLP and speech-related pipelines, with a focus on Liujgoj Cantonese orthography, romanization, syllable decomposition, and IPA transcription.
While the CPT stage taught the model the statistical structure and phonological distribution of Liujgoj Cantonese, this SFT stage teaches it to follow user instructions and perform discrete conversion tasks with high accuracy.
- Developer: Yvthyvq / Liujgoj Cantonese Project
- Base Model: Yvthyvq/Liujgoj-Cantonese-Qwen2.5-Omni-7B-CPT-V2
- Model Type: Instruction-Tuned / Supervised Fine-Tuned (SFT)
- Primary Language/Script: Cantonese (Liujgoj Latin Orthography & Traditional Chinese Characters)
- Model Size: 7B (full Omni architecture preserved)
🏗️ Model Architecture Note
This checkpoint is a complete Qwen2.5-Omni model, not a text-only derivative. The SFT LoRA adapter was merged back into the thinker submodule, while the original talker and token2wav components were preserved intact.
This design enables the following pipeline: Qwen2.5-Omni-7B (base) │ ▼ [CPT] Liujgoj Cantonese text continuation │ ▼ [SFT] Instruction tuning (this checkpoint) │ ▼ [ASR] Cantonese speech recognition fine-tuning (upcoming)
Because the full multimodal architecture is retained, this checkpoint can be directly used as the starting point for downstream speech tasks (ASR / TTS) without re-loading the original Omni weights.
🛠️ Training Details & Methodology
This checkpoint represents the SFT (Supervised Fine-Tuning) Stage, following the CPT stage.
- Training Method: LoRA (r=16, alpha=32, dropout=0.05), merged back to backbone
- Target Modules:
q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_projof the thinker language model - Training Data:
merged_sft_data.jsonl— a multi-task instruction corpus of 150,846 samples, covering:- Romanized Liujgoj → Traditional Chinese translation
- Traditional Chinese → Romanized Liujgoj translation
- Word → syllable decomposition
- Word / syllable → IPA transcription
- IPA → word / syllable reconstruction
- Cantonese short-phrase dialogue
- Training Configuration:
- Effective batch size: 16 (per_device 8 × grad_accum 2)
- Max sequence length: 2,048 tokens
- Learning rate: 1e-4 (cosine schedule, 100 warmup steps)
- Epochs: 1
- Total steps: 9,428
- Training time: ~1 hour 37 minutes on a single A800-80GB
- Objective: Instruction-following via autoregressive next-token prediction, with loss computed only on the assistant response tokens (user prompt tokens are masked to
-100). - Training Loss: Final average train loss ≈ 0.201 (from 2.99 at start)
💡 Capabilities & Intended Uses
Capabilities
- Instruction Following: Recognises ChatML (
<|im_start|>) dialogue boundaries and responds to explicit user requests rather than simply continuing text. - Romanization ⇄ Chinese Conversion: Accurate bidirectional conversion between Liujgoj Cantonese and Traditional Chinese Characters.
- Phonological Tasks: Syllable decomposition, IPA transcription, and IPA-to-syllable reconstruction.
- Foundation for ASR / TTS Pipelines: Because the full Omni architecture is preserved, this model is the correct starting point for audio-conditioned downstream tasks.
Intended Use
- Cantonese NLP Assistant: Front-end for Cantonese-language digital assistants and chat systems.
- Romanization Tool: Batch conversion of Cantonese text between Liujgoj script and Traditional Chinese.
- Phonological Analysis: Automated syllable segmentation and IPA transcription for Cantonese datasets.
- ASR / TTS Text Processing: Ideal text-side backbone for speech-to-text and text-to-speech tokenization workflows.
⚠️ Important Notice
This is an instruction-tuned model, NOT a chat model.
- It follows structured instructions well but does not have a general conversational persona.
- For open-ended chat, further fine-tuning on dialogue data is recommended.
- It may occasionally produce outputs with minor inaccuracies in rare or domain-specific romanization.
🚀 Quickstart & Inference Code
Ensure you have transformers, torch, and peft installed.
import torch
from transformers import AutoTokenizer, AutoConfig
try:
from transformers.models.qwen2_5_omni.modeling_qwen2_5_omni import Qwen2_5OmniForConditionalGeneration
except ImportError:
from transformers.models.qwen2_5_omni import Qwen2_5OmniForConditionalGeneration
MODEL_PATH = "Yvthyvq/Liujgoj-Cantonese-Qwen2.5-Omni-7B-SFT-V2"
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH, trust_remote_code=True)
config = AutoConfig.from_pretrained(MODEL_PATH, trust_remote_code=True)
model = Qwen2_5OmniForConditionalGeneration.from_pretrained(
MODEL_PATH,
config=config,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
model.eval()
# Example 1: Chinese -> Liujgoj
prompt = "請將以下粵語漢字轉寫為 Liujgoj 羅馬字：\n喺江南靠近海邊一條大路之上，有一隊清兵，手執刀槍，押住七架囚車，向住北方行緊。"
# Example 2: Liujgoj -> Chinese
# prompt = "請將以下 Liujgoj 羅馬字轉寫為粵語漢字：\nWaahsvt hair Cingjciux ge Hongjheij cojninx, dungjtinj ge mauq yatj yath.
batjfungj yvx douj, munq deih bingjsoengj."
# Example 3: lexicon -> syllable, ipa
prompt = "請將「waahsvt」拆分成音節，並標註音標。"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to("cuda")
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=256,
do_sample=False,
repetition_penalty=1.05,
)
# Qwen2.5-Omni returns a tuple; outputs[0] is a 2-D tensor
token_ids = outputs[0] if isinstance(outputs, tuple) else outputs
if isinstance(token_ids, torch.Tensor) and token_ids.dim() == 2:
token_ids = token_ids[0]
# Trim the prompt tokens
input_len = inputs["input_ids"].shape[1]
new_tokens = token_ids[input_len:]
response = tokenizer.decode(new_tokens, skip_special_tokens=True)
print("Model Output:\n", response)
【測試 1】 Prompt: 請將以下粵語漢字轉寫為 Liujgoj 羅馬字： 喺江南靠近海邊一條大路之上，有一隊清兵，手執刀槍，押住七架囚車，向住北方行緊。
Setting pad_token_id to eos_token_id:8292 for open-end generation.
--- 模型輸出 ---
Hair Gongjnaamx hauhcing heibinj yatj tiux daaihlouh zijsoengh, yauq yatj deoi cingj bingj, saur zapj doujcoengj, aat zvh catj gaa kauxcej, hoeng zvh bakjfungj haangxganr.
【測試 2】 Prompt: 請將以下 Liujgoj 羅馬字轉寫為粵語漢字： Waahsvt hair Cingjciux ge Hongjheij cojninx, dungjtinj ge mauq yatj yath. batjfungj yvx douj, munq deih bingjsoengj.
Setting pad_token_id to eos_token_id:8292 for open-end generation.
--- 模型輸出 ---
話説喺清朝嘅康熙初年，冬天嘅某一日。北風如刀，滿地冰霜。
【測試 3】 Prompt: 請將「waahsvt」拆分成音節，並標註音標。
Setting pad_token_id to eos_token_id:8292 for open-end generation.
--- 模型輸出 ---
waa + svt
🤝 Dataset Acknowledgements & Licenses
Yvthyvq/ZoengjYvthgaaij-MouxZaakhdungj-Liujgoj
Multi-film Cantonese dialogue corpus (cleaned and transliterated into Liujgoj orthography)
We thank the original authors and maintainers of these corpora for making Cantonese NLP research possible.
📝 Changelog
v2.0 (SFT release): Instruction tuning on 150,846 multi-task samples covering romanization, syllable decomposition, and IPA transcription. Full Omni architecture preserved for downstream ASR.