Ishara Soma v1 Preview

An early Kenyan Sign Language-to-English text model for supervised research and community testing.

Request access · Ishara AI

[!WARNING] Soma v1 Preview is experimental. It can return incorrect text with high confidence. It is an assistive recognition and captioning model, not a replacement for a qualified interpreter. Do not rely on it for emergency communication or use it on its own for medical or legal decisions.

What is Soma?

Soma v1 Preview converts a short video clip of Kenyan Sign Language (KSL) into written English. It is the sign-to-text engine behind the Ishara Translate demonstration.

The model does not read raw pixels directly during recognition. A preprocessing step extracts body and hand landmarks from the video. Soma then represents the movement across frames and uses two possible recognition paths:

1. Phrase retrieval matches the clip against phrases already represented in its index.
2. Text generation uses a T5-small decoder to attempt an English output.

This is a gloss-free approach: the system goes from landmarks to text without requiring an intermediate sign-by-sign gloss sequence.

Soma v1 Preview is being shared so that researchers, Deaf community members, universities and accessibility partners can test it in controlled settings, identify failures and help shape a stronger future release.

Preview status

This release demonstrates progress on short KSL clips, especially for phrases represented during training. It does not solve general KSL translation.

The most important limitations are:

• conversational training data comes primarily from one signer, so multi-signer robustness has not been established;
• unfamiliar phrases are not recognised reliably;
• performance can change with signer, framing, lighting, recording conditions and clip length;
• the retrieval path can only return phrases already present in its index;
• the system can confuse similar phrases and may produce a confident but incorrect result;
• continuous, unsegmented signing and arbitrary fingerspelling are outside the scope of this release.

Any evaluation should let the signer review and correct the returned text before it is treated as meaning.

Evaluation

Conversational KSL benchmark

The conversational expert was trained on 567 clips covering 314 phrases. It was evaluated on 140 held-out clips: 70 development clips and 70 test clips. The evaluation contains 123 clips of phrases represented during training and 17 clips of unfamiliar phrases.

These figures measure this specific held-out development/test pool. They are not an accuracy estimate for KSL generally or for broad signer coverage.

Signer-disjoint word benchmark

On a separate public 30-class KSL word dataset, with test signers excluded from training, the word-class expert achieved 21.60% top-1 accuracy (chance: 3.33%). This is a different task and should not be compared directly with the conversational retrieval result.

Intended uses

Soma v1 Preview is intended for:

• supervised research and evaluation;
• controlled demonstrations with Deaf participants;
• testing correction and feedback experiences;
• studying KSL landmark-to-text methods;
• accessibility research conducted with appropriate community participation and consent.

Uses outside this release

Do not use Soma v1 Preview:

• as a certified or qualified interpreter;
• for emergency communication;
• to make medical, legal, eligibility, employment or other high-impact decisions;
• as an unattended transcription system;
• to claim broad KSL fluency or support for other sign languages;
• for surveillance, identity inference or attempts to identify a signer;
• in a public-facing service without visible human review and a correction path.

Accessing the files

This repository uses manual gated access. Access is granted to individual Hugging Face accounts after review by Ishara AI.

After your request is approved, authenticate and download the repository:

pip install -U huggingface_hub
hf auth login

from huggingface_hub import snapshot_download

local_path = snapshot_download(
    repo_id="IsharaAI/ishara-soma-v1-preview-weights",
)
print(local_path)

This repository is a weights release, not a standard one-line transformers.pipeline() package. Running end-to-end inference also requires the compatible Ishara preprocessing and serving code, configuration, phrase index and menu files. Approved collaborators should contact Ishara for the integration package and current setup guidance.

Model details

The model uses MediaPipe Holistic for landmark extraction. Each routed expert contains a four-layer Transformer landmark encoder and a T5-small text decoder. Only one expert is active for a clip.

Training and evaluation data

The KSL data pool used in the research includes:

The Maseno contribution is separate from the 140-clip conversational benchmark. The 85.7% retrieval result should not be read as performance across the 40 Maseno signers.

Raw training datasets are not included in this weights repository. Access to model weights does not grant access to participant recordings or partner datasets.

Responsible testing and reporting

If you test Soma, please:

1. obtain appropriate consent before recording or sharing a participant's video;
2. explain what will be recorded, retained and used for further training;
3. keep a human correction step visible;
4. record the signed phrase, predicted phrase, alternatives and relevant recording conditions;
5. report both successful and failed examples;
6. avoid presenting model scores as calibrated certainty;
7. stop testing in situations where an error could cause harm.

Corrections do not improve the deployed model automatically. They must be reviewed, permissioned where necessary, incorporated into development, and followed by a fresh evaluation.

Known failure modes

Observed failure modes include:

• replacing a phrase with a semantically similar known phrase;
• returning sensitive or stigmatising text for an unrelated sign;
• returning harmless text for an emergency phrase;
• routing a clip to the wrong domain;
• degradation when signer or recording conditions differ from training;
• low performance for phrases outside the indexed vocabulary.

Confidence thresholds can flag some errors, but they do not catch every confident mistake. This is why supervised testing and signer confirmation are required.

Access review

Ishara reviews access requests individually. In your request, describe:

• your organisation or affiliation;
• your country;
• the purpose of the work;
• how Deaf people or KSL expertise will be involved, where relevant;
• how you will evaluate outputs and handle recordings;
• whether you need only the weights or also integration support.

Approval provides repository access to the requesting Hugging Face account. It is not approval for access to Ishara or partner datasets, and it does not remove your responsibility to comply with applicable law, consent requirements and the repository's licence terms.

Licence

Soma v1 Preview is provided under the Ishara Soma Research Preview Licence v1.0 for approved research, education and supervised evaluation.

Commercial use, production deployment, model hosting, sublicensing, and redistribution of the original or modified weights require separate written permission from Ishara AI.

Access approval does not grant access to training videos, participant data or partner datasets. Users must comply with applicable consent, privacy and data-protection requirements.

Acknowledgements

Soma v1 Preview was made possible through the work of many contributors:

• AISCA provided compute support over a two-month period;
• Qubit Hub provided compute and technical support;
• Maseno University contributed a KSL pose dataset for research;
• members of the Deaf community participated in data collection and validation;
• the Ishara team developed, evaluated and reviewed the system.

We are grateful for the time, resources and expertise that made this preview possible.

Research context

Google DeepMind introduced its multilingual sign-language-to-text system, SL2T, in August 2026. Its release supports the broader direction of landmark-based, gloss-free sign-language translation, but it does not imply equivalent performance, endorsement or a partnership with Ishara. Soma v1 Preview currently supports KSL-to-English; support for other African sign languages is a longer-term ambition.

• Google DeepMind: Putting sign language AI into users' hands
• MediaPipe Holistic
• T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Contact and collaboration

We welcome responsible collaboration from Deaf organisations, KSL researchers, universities, accessibility teams, funders and technology partners. Areas of support include community-led testing, responsibly sourced data, compute grants, technical expertise and research funding.

Use the repository's Community tab for questions that can be discussed publicly. For private collaboration or data-governance questions, contact Ishara through its approved organisational contact channel.

Citation

If you use the model in research, cite the repository and record the exact commit or revision used:

@software{ishara_soma_v1_preview_2026,
  author       = {{Ishara AI}},
  title        = {Ishara Soma v1 Preview},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/IsharaAI/ishara-soma-v1-preview-weights},
  note         = {Research preview; cite the accessed repository revision}
}

Version history

• Soma v1 Preview / external evidence version 2.1 - 27 August 2026: initial gated research-preview documentation.