pico-type 🔍

A tiny byte-level multi-head content classifier — ~1.5M params, ~9MB single-file ONNX (FP32), ~18ms CPU inference.

Classifies any content from raw bytes: coarse type · modality · subtype · code language · text language · file MIME · risk flags

✨ Features

• No tokenizer — operates directly on raw UTF-8 bytes (supports all languages, no preprocessing)
• 7 heads, one forward pass — coarse type, modality, subtype, code language, text language, file MIME, risk flags
• 4 Matryoshka tiers — tiny (16d) → small (64d) → base (192d) → pro (576d) — same trunk, accuracy scales with dim
• ~9MB single-file ONNX (FP32) — deploy on edge devices, serverless, browser (WebAssembly/ONNX Runtime Web)
• ~18ms inference on CPU via ONNX Runtime
• CLI, Python API, Gradio Space, MCP server — ready to use

📊 Evaluation

Overall Accuracy (v2 — trained on real data)

v0.1 baseline (synthetic-only): code_lang 3%, text_lang 19%. Real-data training in v2 improves code by 57pp and text by 79pp.

Code Language — Per-Language Accuracy

Note: Low-accuracy languages have fewer real training samples. More data will improve them.

🚀 Quick Start

Install

pip install picotype

CLI

# Classify from stdin
echo "def hello(name):\n    return f'Hi {name}'" | picotype --pretty

# Classify a file
picotype --file document.txt

# Classify clipboard content
picotype --clip

# All 4 tiers available
echo "..." | picotype --tier pro

Python API

from picotype import load_onnx_model, run_onnx

session = load_onnx_model("base")
result = run_onnx(session, "def hello(): pass")
print(result)
# {
#   "coarse": "code",
#   "code_language": "python",
#   "modality": "textual",
#   "confidence": 0.98,
#   ...
# }

MCP Server (for Claude Desktop, Cursor, etc.)

pip install picotype
PICOTYPE_MODEL_DIR=./checkpoints python -m model.pico_type.mcp_server

Then add to your MCP config:

{
  "mcpServers": {
    "pico-type": {
      "command": "python",
      "args": ["-m", "model.pico_type.mcp_server"],
      "env": { "PICOTYPE_MODEL_DIR": "./checkpoints" }
    }
  }
}

Gradio Web UI

Try it live: huggingface.co/spaces/eulogik/pico-type

🏗 Architecture

Bytes ─▶ ByteEmbed(256→96d) ─▶ 3×Conv1D(k=3,5,7) ─▶ 2×BiAttention(RoPE) ─▶ Pool ─▶ 7×Matryoshka Heads

Total parameters: 1.43M (tiny) / 1.45M (small) / 1.48M (base) / 1.56M (pro)

🔧 Model Tiers

ONNX sizes are single-file FP32 exports (graph-only files are 203–206 KB).

All tiers share the same backbone; only the final linear projection layers differ. Higher-tier models use more dimensions for finer-grained classification.

🧪 Classification Heads

🌐 Deployment

📚 Resources

• Paper — Architecture, training, and evaluation details
• Model Card — Detailed architecture and training configuration
• Walkthrough — Development log and decisions
• Architecture Plan — Original design document

📄 License

Apache 2.0