Xing4.0-29B-A4B-Finance-SFT
This model is a full-parameter supervised fine-tune (SFT) of XingChen-AGI/Xing4.0-29B-A4B, specialized for Chinese and English financial-domain tasks.
[!IMPORTANT] This repository requires
trust_remote_code=True. It ships custom modeling code (modeling_xing4_0.py,configuration_xing4_0.py,tokenization_xing4_0.py) and declares anauto_mapinconfig.json. Do not usetrust_remote_code=False.
Base Model
Xing4.0-29B-A4B is developed by China Telecom AI Technology Co., Ltd. (中电信人工智能科技有限公司). It is a Mixture-of-Experts model in the Xing series (formerly the TeleChat series) with 29B total parameters, of which only ~4B are activated per token. It uses the mHC + MLA + MTP architecture, natively supports a 256K context window (extensible to 512K), and was trained on Ascend NPU clusters with the MindSpore framework.
| Xing4.0-29B-A4B | |
|---|---|
| Total / active parameters | 29B / 4B |
| Layers | 40 |
| Hidden size | 3584 |
| Attention | MLA |
| Routed experts | 64 (4 active per token) + 1 shared |
| Context length | 256K |
Training Details
| Item | Value |
|---|---|
| Fine-tuning type | Full-parameter SFT |
| Training sequence length | 2048 |
| Learning rate | 7e-6 (AdamW, betas 0.9/0.95, weight decay 0.1) |
| LR schedule | Warmup + cosine decay, 30 warmup steps, cos_min_ratio 0.1 |
| Total steps | 578 (≈2 epochs) |
| Per-device batch size | 4 |
| Gradient accumulation | 4 |
| GPUs | 8 |
| Effective batch size | 128 |
| Precision | bfloat16 |
| Distributed strategy | DeepSpeed ZeRO-3 (optimizer CPU offload, overlap_comm) |
| Gradient clipping | 1.0 |
Training Data
Approximately 368K financial-domain multi-turn chat samples
(fin_sft_v1_368k_messages.jsonl, 368,415 records), aggregated from publicly
available financial NLP datasets and corpora, covering:
- Financial examinations / certification QA — accounting, securities, and finance exam questions with explanations (largest component).
- Financial sentiment classification — classifying financial news and statements as positive / negative / neutral.
- Financial question answering — QA over financial filings and reports.
- Financial evaluation sets — a small held-out-style evaluation portion.
Samples are formatted in the base model's native chat template
(<_system> / <_user> / <_bot> / <_end>), with <think>...</think>
reasoning blocks where present.
Quickstart
Because this is a custom_code model, set trust_remote_code=True:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "HITSZ-TMG/Xing4.0-29B-A4B-Finance-SFT"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
trust_remote_code=True,
device_map="auto",
dtype="bfloat16",
)
Chat
messages = [
{"role": "system", "content": "你是一名专业的金融领域助手。"},
{"role": "user", "content": "解释一下什么是久期，以及它对债券价格的影响。"},
]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=1024, temperature=1.0, top_p=0.95)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Thinking mode is controlled by the enable_thinking template flag
(apply_chat_template(..., enable_thinking=True/False)).
Recommended generation parameters: temperature=1.0, top_p=0.95,
repetition_penalty=1.05.
Intended Use and Limitations
- Intended for: financial-domain Chinese/English assistants, financial QA, financial sentiment analysis, and as a starting point for further domain adaptation in banking, securities, and accounting scenarios.
- Not intended for: use as a substitute for professional financial, legal, or investment advice; autonomous trading; or any high-stakes financial decision-making without human review.
- The model inherits the limitations and biases of its base model and of the underlying public financial datasets.
- Outputs may be factually incorrect ("hallucinated"), especially on numeric and regulatory questions. Always verify financial figures against primary sources.
- This model has not been safety-aligned or red-teamed beyond the base model's own training; apply your own guardrails in production.
License
This model is released under the Apache-2.0 license, inherited from XingChen-AGI/Xing4.0-29B-A4B. Please also comply with any terms attached to the upstream financial datasets used during fine-tuning.
Citation
If you use this model, please cite the base model:
@misc{xing4_0_29b_a4b,
title = {Xing4.0-29B-A4B},
author = {China Telecom Artificial Intelligence Technology Co., Ltd.},
year = {2026},
url = {https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B}
}