⚡ ZOZ-Reasoning-Master-3B

A Lightweight, Tri-Stage Agentic Foundation Model (3.09B Parameters)

📌 Model Overview

ZOZ-Reasoning-Master-3B is a compact, high-efficiency language model engineered specifically to operate as an Autonomous Edge Agent. It was developed using a sequential Tri-Stage Agentic Pipeline designed to bridge three core competencies rarely found together in 3B-class models: strict schema-compliant tool calling, resilient long-context retrieval without degradation, and structured multi-turn reasoning.

🔬 The Tri-Stage Training Pipeline

To prevent Catastrophic Forgetting, the model was trained through an incremental staging protocol:

[Qwen2.5-3B-Instruct]
│
▼ (Stage 1: Tool Use & Schema Discipline)
[ZOZ-Function-Master-3B] ──► NousResearch/hermes-function-calling-v1
│
▼ (Stage 2: Deep Needle Retrieval & Multi-Hop Reasoning)
[ZOZ-Function-Master-3B-LongContext] ──► zai-org/LongBench-v2
│
▼ (Stage 3: Conversational CoT & Logic Alignment)
[ZOZ-Reasoning-Master-3B] ──► Magpie-Align/Magpie-Pro-300K-Filtered (Curated)

1. Stage 1 (Function Calling & Tool Use): Fine-tuned on Hermes function-calling datasets to enforce strict JSON output syntax and eliminate parameter hallucination.
2. Stage 2 (In-Context Reasoning & Long Retrieval): Trained on hard reasoning tasks from LongBench-v2 to mitigate the Lost-in-the-Middle phenomenon and enable needle-in-a-haystack data extraction.
3. Stage 3 (Conversational Alignment): Aligned with a curated subset of Magpie-Pro to impart clean chain-of-thought (CoT) reasoning distilled from frontier models (Llama-3.1-70B) without diluting tool-calling precision.

🚀 Quickstart & Inference

1. Basic Reasoning & Chat

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "z51722369/ZOZ-Reasoning-Master-3B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are ZOZ-Reasoning-Master, a precise and logical AI agent."},
    {"role": "user", "content": "Analyze the potential bottlenecks of deploying an in-memory Redis cache versus a persistent disk store in a high-throughput microservices architecture."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.3,
    top_p=0.9,
    repetition_penalty=1.1,
    do_sample=True
)

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

2. Function Calling Example

tools_schema = """[
  {
    "type": "function",
    "function": {
      "name": "query_database",
      "description": "Execute a structured SQL query against the production telemetry database.",
      "parameters": {
        "type": "object",
        "properties": {
          "table": {"type": "string", "description": "Target database table"},
          "filter": {"type": "string", "description": "SQL WHERE clause filter"}
        },
        "required": ["table", "filter"]
      }
    }
  }
]"""

messages = [
    {"role": "system", "content": f"You are an agent with access to external tools. Available tools:\n{tools_schema}"},
    {"role": "user", "content": "Check all failed authentication events in the audit_logs table from the last 2 hours."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.1)

print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
# Expected Output: <tool_call> {"name": "query_database", "arguments": {"table": "audit_logs", "filter": "status = 'FAILED' AND timestamp >= NOW() - INTERVAL 2 HOUR"}} </tool_call>

⚙️ Hyperparameters & Training Specifications

• Adapter Method: LoRA ($r=16, \alpha=32$, Dropout = $0.05$)
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Optimizer: paged_adamw_8bit
• Precision: Mixed FP16 via PyTorch Distributed (torchrun DDP across 2x Tesla T4 GPUs)
• Gradient Clipping: max_grad_norm = 0.3

🎯 Intended Use Cases

• Local RAG Pipelines: High-speed document reasoning engine for locally indexed knowledge bases.
• Autonomous Micro-Agents: Routing, parameter extraction, and execution loops across webhooks and internal APIs.
• Edge & On-Device Deployment: Direct export to 4-bit GGUF or AWQ formats for low-latency execution on mobile devices, Apple Silicon, or embedded compute platforms (e.g., Raspberry Pi 5, Jetson Orin).

⚠️ Limitations & Boundary Conditions

1. World Knowledge Limits: As a 3B parameter model, it is not intended to replace 70B+ models for deep encyclopedic trivia. It delivers optimal accuracy when grounded with context or tools.
2. Context Window Best Practices: While supporting 32K context natively, maintaining inference sequences between 1,024 and 4,096 tokens yields the best balance between latency and memory footprint.

📜 Citation

If you use or build upon ZOZ-Reasoning-Master-3B, please cite:

@misc{zoz_reasoning_master_3b_2026,
  author = {Ziad Khodr (ZOZ AI)},
  title = {ZOZ-Reasoning-Master-3B: A Specialized Long-Context & Function-Calling 3B Agent},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{[https://huggingface.co/z51722369/ZOZ-Reasoning-Master-3B](https://huggingface.co/z51722369/ZOZ-Reasoning-Master-3B)}}
}