ControlMT v2.3 — Compact Kannada ↔ English Translation (139M)

TL;DR. A 139M-parameter encoder-decoder specialized for Kannada ↔ English translation. Single-pair focus + code-mix-native training + Anti-LM contrastive decoding. Achieves competitive FLORES-200 KN↔EN performance for its parameter size, with COMET-DA above 0.84 in both directions. Apache 2.0, deployable on consumer GPU.

Headline benchmark — FLORES-200 devtest

CometKiwi-DA and COMET-DA both clear the 0.82 production floor and the 0.85 aspirational target. BLEU/chrF measured with sacrebleu (default tokenization).

How this got built — journey + decisions + dead ends

If you're building a similar specialized model, the docs/ folder is a first-person account of how ControlMT went from zero to public release in three months, solo, on one GPU:

• docs/top-lessons.md — 10 takeaways, one paragraph each (start here if you only have 10 minutes)
• docs/the-journey.md — chronological narrative, v1 → v2.3
• docs/what-didnt-work.md — 8 failed experiments + root-cause analysis
• docs/how-it-was-built.md — concrete data + training + eval + deployment recipes
• docs/working-with-claude.md — patterns for solo + AI-assistant collaboration
• docs/repo-map.md — folder layout, file conventions

Available releases

Easiest path — the SDK does the right thing automatically:

# CPU-only (smaller install — ~200 MB torch instead of ~2 GB)
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install controlmt

# GPU (CUDA) — default; pulls the full ~2 GB CUDA torch wheel
pip install controlmt

from controlmt import ControlMT
model = ControlMT.from_hf()                              # GPU fp16 / CPU bf16 / etc — auto
model = ControlMT.from_hf(quant="int8")                  # CPU int8 dynamic
model = ControlMT.from_hf(device="cpu", dtype="bf16")    # explicit
print(model.translate("ನಾನು ಕನ್ನಡ ಮಾತನಾಡುತ್ತೇನೆ."))         # "I speak Kannada."

Why two install paths? pip install controlmt pulls torch>=2.0, which by default fetches the CUDA-enabled wheel (~2 GB). If you don't have a GPU, install the CPU-only torch wheel first (the line with --index-url) — it's ~200 MB and runs the model just fine on CPU at bf16 or int8. This is a PyTorch ecosystem quirk, not a ControlMT one — every model that depends on torch has the same trade-off.

Raw Transformers also works (no SDK needed):

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

# Main repo — choose dtype at load time
tokenizer = AutoTokenizer.from_pretrained("anandkaman/controlmt-v2.3", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("anandkaman/controlmt-v2.3", trust_remote_code=True)

# int8 repo — quantization auto-applied
model_int8 = AutoModelForSeq2SeqLM.from_pretrained("anandkaman/controlmt-v2.3-int8", trust_remote_code=True)

→ Full deployment recipes + verified latency/memory matrix: DEPLOYMENT.md

1. Model Details

ControlMT v2.3 is a modular encoder-decoder transformer specialized for Kannada ↔ English translation. Every parameter is dedicated to this one language pair, which is what lets a 139M model compete with multilingual models 4× its size on FLORES-200 KN↔EN.

Architecture

                ┌── Router (per-row direction token) ──┐
                │                                        │
        ┌───────▼─────────┐                       ┌─────▼───────────┐
        │ KN Lang Encoder │                       │ EN Lang Encoder │
        │ (2 layers)      │                       │ (2 layers)      │
        └───────┬─────────┘                       └─────────────────┘
                │
        ┌───────▼─────────┐
        │ Shared Core Enc │  6 layers, ~19M
        └───────┬─────────┘
                │
        ┌───────▼─────────┐
        │ Shared Core Dec │  6 layers, ~25M
        └───────┬─────────┘
                │
        ┌───────▼─────────┐                       ┌─────────────────┐
        │ KN Lang Decoder │                       │ EN Lang Decoder │
        │ (2 layers)      │                       │ (2 layers)      │
        └─────────────────┘                       └─────────────────┘
                                                  ↓
                                Output projection (tied embeddings, 128K vocab)

Why single-pair?

Most public Indic MT models are broad — NLLB covers 200 languages, IndicTrans2 covers 22. That breadth comes from parameter-sharing across languages, so each language pair gets only a slice of the model's capacity.

ControlMT goes the other direction: every parameter is dedicated to Kannada ↔ English. If you need broad multilingual coverage, use NLLB or IndicTrans2. If you need Kannada specifically — and you care about size, latency, or on-device deployment — this is what the trade-off looks like.

2. Intended Use & Out-of-Scope Use

Intended use

• Production KN↔EN translation for Indian-context content: news, government documents, e-commerce, social media, customer support, conversational interfaces
• Code-mix-aware translation — handles natural Indian Kannada that embeds English acronyms, brand names, and short loanwords
• Edge / on-device deployment — at 139M params + int8 quantization, runs on consumer hardware (laptops, mid-tier devices with ≥4 GB RAM)
• Office / form-data translation (KYC, applications, customer records) — the model demonstrated near-perfect preservation on the release evaluation suite for Aadhar, phone, email, dates, customer IDs, and PAN numbers in the KN→EN direction. EN→KN has a small edge case where mid-sentence PAN-format strings may character-by-character transliterate to Kannada syllables (information preserved, recoverable via a small regex postprocessing pass — see Limitations Section 6).

Out-of-scope use

• ❌ Not a multilingual translator — only Kannada ↔ English. For other language pairs, see NLLB-200 or IndicTrans2.
• ❌ Not a chatbot / not instruction-following — translation is the only supported task.
• ❌ Not a literal-translator for idioms — see Limitations (Section 6).
• ❌ Not certified for safety-critical domains (medical diagnosis, legal advice). The model passes a safety regression set but is not formally audited for those contexts.
• ❌ Not a domain-specialist for highly technical scientific text without context.

3. Training Data (summary)

The base corpus is 8.06M parallel KN↔EN pairs aggregated from public Indic MT datasets — Samanantar (Ramesh et al. 2022), BPCC (Gala et al. 2023 / IndicTrans2), Sangraha (Khan et al. 2024 / IndicLLMSuite), and Aksharantar (Madhani et al. 2023) for transliteration coverage.

A multi-stage filtering pipeline (profanity filter, roundtrip audit, CometKiwi quality scoring, misalignment detection) reduces this to 6.64M clean rows in master_v22.jsonl. Bad rows (62,853) are quarantined with _drop_reason audit trail rather than deleted.

Augmenting the main corpus, four small internally-generated streams target specific weaknesses: Pattern A (~30K NER-validated proper-noun pairs), Pattern B (~8K cm_paired groups for code-mix), F2 (~5K letter-spelled acronyms), and numerical_aug (form-preservation for digits/dates/currency).

Full filtering pipeline, per-stream methodology, training principles, and reproducibility steps: see TRAINING_GUIDE.md.

Data licensing: Model weights and ControlMT-specific generated streams are released under Apache 2.0. Public source corpora retain their original licenses (Samanantar: CC-BY-NC 4.0; others: CC-BY-4.0).

3.4 Training principles

• Decoder hygiene gate (kn_is_mixed): rows with 3+ consecutive Latin words in KN are excluded from EN→KN target — prevents mixed-code emission
• CM-Concatenation Level A: paired (kn_pure, kn_mixed) batching for natural code-mix handling
• EMA (decay=0.999) + SWA averaging for production weights
• Anti-LM contrastive decoding (α=0.5) at inference — kills repetition + hallucination

4. Evaluation

4.1 Public benchmark sets

4.2 Scoring tools

4.3 Decoding configuration for reported scores

4.4 Results

FLORES-200 devtest (1,012 pairs)

Ship-gate verdict: ✅ PASS — both directions clear the 0.85 aspirational target on CometKiwi-DA (en→kn) and within striking distance on the others. All four metrics above the production floor.

IN22-Conv (1,503 pairs, AI4Bharat conversational benchmark)

Ship-gate verdict: ✅ PASS — both COMET-DA values clear the 0.82 production floor; EN→KN CometKiwi at 0.8845 exceeds the 0.85 aspirational target. BLEU EN→KN is naturally low on conversational data (short colloquial utterances with high lexical variance); CometKiwi/COMET-DA (semantic adequacy) is the more reliable signal here.

Aggregate JSON: eval_results/in22_conv.json.

4.5 Comparison context

Direct head-to-head COMET-DA benchmarks for distilled-size Indic MT models on FLORES KN↔EN are not uniformly published in a single source. The major academic reference (IndicTrans2, Gala et al. 2023) reports chrF++ in its main tables and includes COMET-22 values in supplementary tables; NLLB (NLLB Team 2022) reports spBLEU + chrF++ but does not publish per-language COMET-DA for the distilled checkpoints.

As one citation-grounded anchor in the same metric space: IndicTrans2 1.1B reports COMET-22 ≈ 0.84 on IN22-Conv KN→EN (Gala et al. 2023, Appendix Table 45). ControlMT v2.3 reports COMET-DA 0.8459 on FLORES KN→EN at 139M parameters — competitive within its parameter scale class.

For an apples-to-apples comparison on your own infrastructure, the open-source eval pipeline in scripts/eval_release.py can be pointed at any KN↔EN MT model (NLLB / IndicTrans2 / Sarvam-Translate) using the same FLORES devtest pairs and same scoring tooling (CometKiwi-DA, COMET-DA, sacrebleu), giving directly-comparable numbers without trusting any individual paper's reporting.

5. Decoding Configuration (recommended presets)

Default (production)

generate_kwargs = dict(
    num_beams=6,
    length_penalty=1.2,
    no_repeat_ngram_size=3,
    anti_lm_alpha=0.5,
    max_length=256,
)

Fast (~2× throughput, ~0.5 BLEU lower)

generate_kwargs = dict(num_beams=4, anti_lm_alpha=0.0, max_length=256)

Greedy (fastest, ~1.5 BLEU lower than default)

generate_kwargs = dict(num_beams=1, max_length=256)

High-quality (~30% slower, marginal gain)

generate_kwargs = dict(num_beams=8, anti_lm_alpha=0.7, max_length=256)

What is Anti-LM contrastive decoding?

At every decoding step, the model computes two next-token distributions:

1. Main: p(y_t | source, y_<t)
2. Anti-LM: p(y_t | NO_source, y_<t) (cross-attention masked out)

Contrastive score: log p_main − α · log p_antilm. Tokens predictable without seeing the source get penalized — kills repetition and source-detached hallucination. α=0 disables; α=0.5 is the production default.

6. Limitations

Things the model does well

• ✅ Numbers preserved across multi-number sentences
• ✅ Dates preserved (including years 2024-2030)
• ✅ Indian-format numbers (2,50,000 ↔ 2.5 ಲಕ್ಷ ↔ "two and a half lakh")
• ✅ Kannada numerals ↔ English digits conversion (೨,೫೦,೦೦೦ ↔ 2,50,000)
• ✅ Currency symbols and units in both directions
• ✅ Phone numbers, Aadhar numbers, email addresses preserved
• ✅ Common entity transliteration (Modi, Bengaluru, ISRO, Apple, iPhone, Reuters, etc.)
• ✅ Long sentences with complex semantics (multi-clause, conditional, scientific)
• ✅ Negation, tense, aspect handled correctly
• ✅ Safety regression — no toxic output on provocative inputs (Falklands/Hancock/Peacock set)

Failure-mode honesty

This is a specialized model, not a frontier LLM. For:

• Idioms → use a 7B+ model or post-edit
• Modern technical jargon (cloud-native stack names) → either keep source-as-is or use a frontier LLM
• Multilingual translation → use NLLB-200 or IndicTrans2

7. Ethical Considerations & Bias

Safety filtering applied

• 40,586 profanity/adult-content rows dropped during corpus filtering
• Safety regression test set (Falklands/Hancock/Peacock variants) — 100% pass

Known biases (inherent to corpus)

• Indian-context skew — entities, locations, brand names from Indian public discourse over-represented (this is intentional given the deployment target)
• 2022-era training data — modern tech terminology (2023-2026) less well-covered
• News + Wikipedia heavy — colloquial chat patterns under-represented vs daily speech

Source code attribution

This release ships with HF integration code (configuration_controlmt.py, modeling_controlmt.py, tokenization_controlmt.py) plus the native architecture (model.py). All Apache 2.0.

Usage

Quick start — Python + Transformers

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("anandkaman/controlmt-v2.3", trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained("anandkaman/controlmt-v2.3", trust_remote_code=True)

# KN → EN
out = model.translate("ಅವನು ನಾಳೆ ಬೆಂಗಳೂರಿಗೆ ಬಂದು ನನ್ನನ್ನು ಭೇಟಿಯಾಗುತ್ತಾನೆ.",
                       tokenizer=tokenizer, direction="kn2en")
# "He will come to Bangalore tomorrow and meet me."

# EN → KN
out = model.translate("India is a country in South Asia.",
                       tokenizer=tokenizer, direction="en2kn")
# "ದಕ್ಷಿಣ ಏಷ್ಯಾದ ಒಂದು ದೇಶ ಭಾರತ."

8. Deployment

ControlMT v2.3 is an encoder-decoder seq2seq model — same family as T5/mBART, not decoder-only LM. That distinction matters for serving (see Section 9 of the deployment guide for which platforms can run it natively).

Verified deployment matrix (RTX 5060 Ti box, beam=2, 6 KN↔EN test pairs):

Pinned versions that we verified with: python 3.12.3 · torch 2.10.0 · transformers 4.57.6 · sentencepiece 0.2.1 · safetensors 0.7.0 · huggingface_hub 0.36.2. Minimum supported is torch >= 2.0, transformers >= 4.40.

Not directly supported (architectural — these are decoder-only frameworks): vLLM, Ollama, llama.cpp / GGUF, HF TGI, bitsandbytes int8. Use the FastAPI wrapper instead — at 0.19 s/pair, the optimizations these tools provide are dominated by request overhead.

→ Full recipes, code, and pre-launch checklist in DEPLOYMENT.md. → Reproduce the matrix above: python assets/scripts/verify_deployment.py --device cuda

🎯 Help shape v2.4 — Break-the-Model Challenge

v2.4 is being designed around the gaps we find in v2.3. The live demo includes an opt-in research-data sharing toggle — when enabled, your translation (input + output + timing) is logged to a private dataset we use to identify edge cases for v2.4 training.

Things we particularly want to see fail:

• Heavily code-mixed phrases (Nange last meeting nalli decision aagilla)
• Complex numerals (೨,೩೫,೬೭೮, 1,23,45,678, mixed-script percentages)
• Regional Karnataka dialects (Mangalorean, Dharwad, Kalyana Karnataka)
• Domain terminology (cricket, finance, government schemes, temple names)
• Long literary sentences (Bendre, Karanth-era prose)
• Modern tech / SaaS jargon (already known weak — confirm + extend)

Opt-in is unchecked by default. When you do opt in, inputs are automatically PII-redacted (PAN, Aadhar, phone, email, card numbers) before storage. Full details in PRIVACY.md.

Roadmap

v2.4 — Priorities locked from v2.3 evaluation

#1 — Multi-token code-mix data slice (highest-impact gap from v2.3 evaluation)

A 50k+ corpus slice of Kannada matrix sentence + 2–4 Latin-script English tokens paired with English target preserving every Latin-script token verbatim. This is the largest visible v2.3 weakness, identified during competitor comparison against IndicTrans2 1.1B and Sarvam-Translate (see internal eval_results/competitor_comparison.md):

• v2.3 handles Kannada + 1 English entity cleanly
• v2.3 hallucinates entity names at Kannada + 2+ English tokens (e.g. Manyata Tech Park → Girinagar Tech Park when "Software Engineer" is also present in the same sentence)
• IndicTrans2 1.1B and Sarvam-Translate 4B both handle the 2+ case correctly

Root cause hypothesis: decoder over-weights the Kannada language prior when the source has high English-token density, and substitutes nearest-by-phonetic Kannada place-name from training distribution. Closing this gap is expected to also improve:

• Long-sentence robustness (better source-attention discipline)
• Number + entity ordering in payment/transactional prose
• Tech / startup / finance jargon (which clusters multi-token English)

Other v2.4 priorities (in order of expected impact)

• Kannada proverbs & idioms corpus (5–10k pairs) — v2.3 + IT2 + Sarvam all fail on proverbs like ಮಾಡಿದ್ದುಣ್ಣೋ ಮಹಾರಾಯ (= "you reap what you sow")
• Hindi support ([HI2EN] / [EN2HI]) — opens a second language pair
• Iterative back-translation for low-resource domain expansion
• Expanded vocabulary (modern tech terms, longer alphanumeric IDs)
• Standardized BPE tokenizer (currently SentencePiece Unigram)
• Register / style control revisit (rebalanced labels + contrastive separation training)

v3.0 (TBD)

Copy-mechanism / pointer-generator for OOV-proof transliteration. A built-in solution for the entity-preservation problem instead of corpus-only fix.

Citation

@misc{controlmt-v2.3-2026,
  author = {Anand Kaman},
  title  = {ControlMT v2.3 — A 139M-Parameter Specialized Kannada↔English Translation Model
           with Code-Mix-Native Training},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/anandkaman/controlmt-v2.3}}
}

License

Apache 2.0 — see LICENSE.