captCHAD: Sub-100k Parameter Neural CAPTCHA OCR

🚀 v1.5 Update: Weights upgraded to epoch 83 convergence. Real-world challenge benchmark increased to 4/5 (80%) in the gallery and 5/6 (83.3%) overall, now solving PXAZGC and JQA3Lg with 100% exact match. ONNX (FP32/INT8) and Safetensors (FP32/FP16) packages updated.

captCHAD is a compact, 97,057-parameter optical character recognition (OCR) model engineered for low-power CPU, edge, and browser environments.

⚡ Try the In-Browser Demo: You can test captCHAD directly in your browser with zero server latency using WebAssembly at AndresDev/captCHAD-demo. Pick from preset challenge datasets or drop/paste your own CAPTCHAs.

Rather than relying on heavy Vision Transformers (TrOCR, 61M–330M parameters) or standard CRNN pipelines that degrade when encountering crossing strike-bars, distorted grid patterns, and high-contrast color shifts, captCHAD employs a lightweight sequence architecture combining Inverted Residual MobileNet blocks, a Dual Contrast Stem, Bidirectional GRUs, and Connectionist Temporal Classification (CTC) decoding.

Direct Challenge Evaluation Gallery

Evaluation against open-source models on real-world production CAPTCHAs collected from active websites:

Comprehensive Benchmark: captCHAD vs. Hugging Face Models

Benchmarked on identical hardware (CPU, 4 threads) across 50 unseen multi-archetype test captchas:

Architectural Observations

• TrOCR (61.6M params): Autoregressive Vision Transformers trained primarily on clean documents lack inductive edge bias. When facing crossing strike-bars, non-linear waves, or bulge grids, the attention mechanism loses positional alignment and hallucinates, resulting in 0% exact match and 564 ms latency.
• CRNN and Conv-Transformer (3.6M–12.3M params): Standard convolutional backbones without edge-separation stages fuse background grid lines and noise specks into character activations, resulting in 10%–12% exact match on multi-archetype noise.
• captCHAD (97k params): Uses a pre-convolutional Contrast Stem (normalized luminance and directional Sobel gradients) and Squeeze-and-Excitation channel gating to isolate glyph contours from background clutter, maintaining 80% exact match with significantly lower compute requirements.

Quantization Formats & Benchmark

To support different deployment environments (WASM browsers, microcontrollers, embedded Linux, server inference), captCHAD is provided in multiple precision formats:

Architecture Details

Input Image (3 x 64 x 192)
     │
     ▼
[Contrast Stem] ──> Extracts Normalized Luminance + Sobel Spatial Gradients (Sobel-X & Sobel-Y)
     │
     ▼
[MobileNet Inverted Residual Blocks] ──> Depthwise-Separable Convolutions + Squeeze-and-Excitation
     │
     ▼
[Height Pooling] ──> Vertical feature compression: (B, 52, 48)
     │
     ▼
[Bidirectional GRU] ──> Horizontal temporal sequence modeling
     │
     ▼
[CTC Sequence Decoder] ──> Arbitrary-length prediction without segmentation

• Input Dimensions: (B, 3, 64, 192)
• Output Dimensions: (48, B, 63) (Logits for CTC Loss, sequence length $T=48$)
• Output Character Length: 1 to 8 characters per image (optimized for 4–7 character CAPTCHAs)
• Character Vocabulary: 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ (62 alphanumeric classes + 1 CTC blank token at index 0)
• Trainable Parameters: 97,057

Quick Start

1. ONNX Runtime Inference

Requires onnxruntime, numpy, and pillow:

import numpy as np
from PIL import Image
import onnxruntime as ort

# Supports captchad.onnx (FP32), captchad_int8.onnx (INT8), captchad_fp16.onnx (FP16)
session = ort.InferenceSession("captchad.onnx")
input_name = session.get_inputs()[0].name

CHARSET = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
IDX2CHAR = {i + 1: ch for i, ch in enumerate(CHARSET)}

# Preprocess image to [1, 3, 64, 192] normalized in [-1.0, 1.0]
img = Image.open("sample.png").convert("RGB").resize((192, 64), Image.BILINEAR)
arr = (np.array(img, dtype=np.float32).transpose(2, 0, 1) - 127.5) / 127.5
inp = arr[np.newaxis, :, :, :]

# Run inference
logits = session.run(None, {input_name: inp})[0]
preds = np.argmax(logits[:, 0, :], axis=-1)

# CTC Collapse
decoded, prev = [], None
for t in preds:
    if t != prev and t != 0:
        decoded.append(IDX2CHAR[t])
    prev = t

print("Prediction:", "".join(decoded))

2. Safetensors and PyTorch Inference

import torch
from safetensors.torch import load_file
from model import captCHAD, decode_tokens
from PIL import Image
import numpy as np

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = captCHAD()

# Load weights (supports model.safetensors, model_fp16.safetensors, model_fp8.safetensors)
state_dict = load_file("model_fp16.safetensors")
state_dict = {k: v.to(torch.float32) if v.is_floating_point() else v for k, v in state_dict.items()}
model.load_state_dict(state_dict)
model.to(device)
model.eval()

img = Image.open("sample.png").convert("RGB").resize((192, 64), Image.BILINEAR)
arr = (np.array(img, dtype=np.float32).transpose(2, 0, 1) - 127.5) / 127.5
tensor = torch.from_numpy(arr).unsqueeze(0).to(device)

with torch.no_grad():
    logits = model(tensor)
    preds = logits.argmax(dim=-1)[:, 0].tolist()
    text = decode_tokens(preds)

print("Prediction:", text)

3. Command-Line Inference

Test different engines and quantization formats directly via inference.py:

# Fastest CPU production baseline (ONNX FP32)
python inference.py sample.png --engine onnx --quant fp32

# Balanced half-precision (ONNX FP16, 395 KB)
python inference.py sample.png --engine onnx --quant fp16

# Dynamic integer quantization (ONNX INT8, 412 KB)
python inference.py sample.png --engine onnx --quant int8

# Compact half-precision (Safetensors FP16, 206 KB)
python inference.py sample.png --engine safetensors --quant fp16

# Ultra-compact 8-bit float (Safetensors FP8, 110 KB)
python inference.py sample.png --engine safetensors --quant fp8

# Extreme 4-bit packed weights (Safetensors INT4, 72 KB)
python inference.py sample.png --engine safetensors --quant int4

License

This project is licensed under the Apache 2.0 License.