🧠 Zen Ingress 1-3B (DPO Merged)
Zen Ingress 1 is a custom 3-billion parameter causal language model engineered by ZenithLLM, built as a highly optimized fine-tune of Qwen 2.5 3B. It is specifically designed for rigorous step-by-step reasoning, clean algorithmic efficiency, and edge-device deployment.
🚀 Pipeline & Development Phases
The model was forged through a rigid 4-phase development cycle with a "No-Compromise" strictness rule:
Phase 1: Data Generation & Curation
- Generated 15,070 strictly filtered samples.
- Curated for zero fluff, high signal-to-noise ratio, and direct logical paths.
Phase 2: Supervised Fine-Tuning (SFT)
- Trained for 3 Epochs (5,451 steps).
- Checkpoint available at:
ZenithLLM/zen-ingress-1-3b-sft-merged.
Phase 3: Direct Preference Optimization (DPO) Alignment
Aligned using a highly curated Multi-Domain Mixture (~12,000 preference pairs):
- 4,000 pairs from
ibivibiv/cleaned_orca_math_dpo_pairs(Enhances step-by-step arithmetic and algebraic deduction). - 4,000 pairs from
coseal/CodeUltraFeedback_binarized(Enhances algorithmic efficiency, clean coding style, and edge-case safety). - 3,800 pairs from
Intel/orca_dpo_pairs(Suppresses conversational fluff, filler words, and hallucinations). - ~200 custom pairs: Zenith Identity & First-Principles Reasoning Anchors.
- Hardware: Trained on 2× Tesla T4 via DistributedDataParallel (1,493 steps).
Phase 4: Extreme Edge Optimization (Imatrix Quantization)
This repository contains the merged weights. If you are looking for the highly optimized GGUF versions targeting mobile edge bandwidth (Dimensity 7300, Cortex-A78), we generated the following through Importance Matrix (Imatrix) calibration (using 1024-token chunking):
IQ4_NL: The optimal balance for mobile bandwidth and generation speed.Q5_K_M: The high-fidelity quality baseline.
(Note: GGUF binaries are uploaded separately under this repo or a dedicated GGUF repo).
💻 Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "ZenithLLM/zen-ingress-1-3b-dpo-merged"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")
prompt = "Explain the concept of entropy using first principles."
messages = [
{"role": "system", "content": "You are Zen Ingress, a direct and highly logical assistant."},
{"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
📜 Identity & Persona
This model heavily suppresses "As an AI..." disclaimers and introductory fluff. It favors concise, strictly formatted, and algorithmically sound responses.
Engineered by ZenithLLM.