ClokCEM — Customer Executive Model

354M parameter language model trained from scratch for enterprise customer support, with native intent classification, chain-of-thought reasoning, and quality gating.

ClokCEM (Customer Executive Model) is a decoder-only transformer built specifically for customer care workflows. Unlike general-purpose chatbots fine-tuned for CS, ClokCEM was pre-trained from random initialization on 21 curated customer support datasets spanning multiple industries, languages, and interaction patterns. It learns to classify user intent across 27 categories, reason through multi-turn conversations, and self-assess response quality — all within a single forward pass.

Model Summary

What Makes ClokCEM Different

ClokCEM is trained from random initialization it learns customer support patterns natively rather than retrofitting general language understanding. This means:

• Architecture optimized specifically for CS workflows
• Intent classification built into the model head, not bolted on as a separate classifier
• Thinking capability emerges from training, not prompt engineering

Native Intent Classification

ClokCEM classifies user intent across 27 categories in a single forward pass, without requiring separate classifier models or prompt templates:

Thinking Head

ClokCEM includes a dedicated thinking module that processes intermediate hidden states (from layer 12) to produce:

• Thinking Steps: A scalar indicating how many reasoning steps the model believes are needed for the query
• Quality Score: A confidence measure (0–1) of the model's self-assessed response quality
• Intent Logits: 27-class probability distribution over customer care intents

These are computed in parallel with the main language modeling head, adding negligible overhead.

Training Details

Hardware

• Platform: NVIDIA T4
• Total Training Time: ~3 weeks
• Batch Size: 32 per GPU (64 effective with DDP)
• Mixed Precision: FP16 with gradient scaling
• Gradient Checkpointing: Enabled (reduces VRAM by ~40%)

Optimizer & Schedule

• Optimizer: AdamW (β₁=0.9, β₂=0.95, ε=1e-8)
• Learning Rate: 3e-4 → 1e-5 (cosine decay with warmup)
• Weight Decay: 0.1
• Label Smoothing: 0.05
• Gradient Clipping: Max norm 1.0

Training Progression

The loss curve shows a characteristic pattern: rapid descent from 8.2 → 2.0 in the first 5K steps as the model learns basic language patterns, followed by a steady decline to 0.765 as it masters intent classification and response generation. The final loss of 1.74 reflects the model's behavior at convergence — it has learned to produce shorter, more focused responses rather than maximizing likelihood on all tokens.

Training Data

ClokCEM was trained on 21 curated datasets covering multiple customer support domains, instruction following, and reasoning tasks.

Architecture

Key Components

Grouped Query Attention (GQA)

• 16 attention heads, 4 KV heads
• 4× memory compression vs standard MHA
• Enables 1,024 token context on T4 GPUs

SwiGLU Feed-Forward

• Gate-based activation (SiLU gating)
• 2,816 intermediate dimensions
• More expressive than standard ReLU FFN

RMSNorm

• Normalizes by root mean square (no mean centering)
• Faster training, more stable gradients
• Replaces LayerNorm in all positions

RoPE (Rotary Position Embeddings)

• Position-aware attention without learned embeddings
• θ=10,000 for long-range dependency modeling
• Applied to Q and K projections

Flash Attention

• Uses PyTorch's F.scaled_dot_product_attention
• Memory-efficient attention computation
• Automatic causal masking

Usage

Installation

pip install transformers torch safetensors

Loading the Model

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "clokai/CLOK-CEM",
    torch_dtype=torch.float16,
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("clokai/CLOK-CEM")

Text Generation

# Chat format with system prompt
prompt = "<|system|>You are a helpful customer care assistant for an e-commerce company.<|user|>Where is my order?<|assistant|>"
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.7,
        top_p=0.9,
        top_k=50,
        repetition_penalty=1.1,
        do_sample=True,
    )

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Intent Classification

with torch.no_grad():
    out = model(input_ids)

# Get predicted intent
intent_logits = out["intent_logits"]
intent_probs = torch.softmax(intent_logits, dim=-1)
predicted_intent_id = intent.argmax(-1).item()
confidence = intent_probs[0, predicted_intent_id].item()

# Map to intent name
intent_classes = [
    "order_status", "return_request", "refund_status", "product_info",
    "price_inquiry", "complaint", "technical_support", "account_help",
    "billing_query", "cancellation", "feedback", "escalation",
    "shipping_query", "payment_issue", "discount_coupon", "warranty_claim",
    "appointment_booking", "schedule_change", "prescription_query",
    "insurance_claim", "loan_inquiry", "balance_check", "plan_change",
    "network_issue", "new_connection", "porting_request", "general_query",
]

print(f"Intent: {intent_classes[predicted_intent_id]}")
print(f"Confidence: {confidence:.2%}")

Thinking Analysis

with torch.no_grad():
    out = model(input_ids)

thinking_steps = out["thinking_steps"].item()
quality_score = out["quality_score"].item()

print(f"Recommended thinking steps: {thinking_steps:.4f}")
print(f"Response quality confidence: {quality_score:.4f}")

Multi-Turn Conversation

def chat(model, tokenizer, history, user_message):
    # Format with chat template
    prompt = "<|system|>You are a helpful customer care assistant.<|user|>"
    for i, msg in enumerate(history):
        if i % 2 == 0:
            prompt += f"{msg}<|assistant|>"
        else:
            prompt += f"{msg}<|user|>"
    prompt += f"{user_message}<|assistant|>"
    
    inputs = tokenizer(prompt, return_tensors="pt")
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=256,
            temperature=0.7,
            top_p=0.9,
            repetition_penalty=1.1,
        )
    
    response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    history.append(user_message)
    history.append(response)
    return response

# Example conversation
history = []
response = chat(model, tokenizer, history, "Hi, I need help with my order")
print(response)
response = chat(model, tokenizer, history, "My order number is 12345")
print(response)

Special Tokens

Training Evaluation

Intent Classification Accuracy

Evaluated on 20 held-out queries across 7 intent categories:

Overall Accuracy: 80% (16/20 correct)

Response Quality Metrics

Model Architecture Details

Parameter Count Breakdown

Per-Layer Breakdown (×16 layers)

Thinking Head Architecture

The Thinking Head processes intermediate activations from layer 12 (not the final layer) to produce:

Layer 12 Output (1536)
    │
    ├─→ Intent Classifier
    │   ├─ Linear(1536 → 768) + ReLU
    │   ├─ Linear(768 → 384) + ReLU + Dropout(0.1)
    │   └─ Linear(384 → 27) → Intent Logits
    │
    ├─→ Thinking Steps
    │   ├─ Linear(1536 → 384) + ReLU
    │   └─ Linear(384 → 1) → Step Count
    │
    └─→ Quality Gate
        ├─ Linear(1536 → 384) + ReLU
        └─ Linear(384 → 1) + Sigmoid → Quality Score

Technical Specifications

Memory Requirements

Inference Speed

Supported Tasks

• ✅ Customer support chat
• ✅ Intent classification
• ✅ Multi-turn conversations
• ✅ Response quality assessment
• ✅ Chain-of-thought reasoning

Limitations

1. Context Length: Limited to 1,024 tokens. Longer conversations will be truncated.
2. No Real-Time Learning: The model does not learn from conversations at inference time.
3. Intent Overlap: Some queries may map to multiple intents (e.g., "I want to cancel and get a refund" → cancellation + refund_status).
4. Language Coverage: Primarily English with partial Indian language support. Not suitable for non-English-only deployments without further training.
5. No Hallucination Prevention: The model may generate plausible but incorrect information. Always verify facts in production.

Citation

@software{clokcem2026,
  title={ClokCEM: Customer Executive Model},
  author={Clok AI},
  year={2026},
  url={https://huggingface.co/clokai/CLOK-CEM},
  license={apache-2.0}
}

License

Apache 2.0 — see LICENSE for details.

Links

• Model: clokai/CLOK-CEM
• Paper: Coming soon
• Demo: Coming soon
• GitHub: Coming soon