Meivin MLM 0.1 | מייבין — מודל שפה לעברית תורנית וארמית

ההקשר התורני, ברמת הטוקן. Meivin MLM is a compact bidirectional BERT model for masked-token prediction in Hebrew and Aramaic Judaic text. It joins Meivin Embed 0.1 in the Meivin model family.

כל שימוש מסחרי, ישיר או עקיף, דורש אישור מפורש מראש ובכתב מאריאל דניאלי: arieldaniely@gmail.com. אישור גישה למאגר אינו אישור מסחרי. This release uses Meivin Non-Commercial License 1.0, reproduced in LICENSE.md. Third-party rights and valid permissions for previous releases remain unaffected.

The Meivin family | סדרת מייבין

Built for the language of Jewish texts, with compact models that run locally. The two products share a documented training foundation and serve complementary tasks. This release is available under Meivin Non-Commercial License 1.0; commercial licensing and collaboration: Ariel Daniely.

מה מקבלים?

מודל MLM ייעודי לעברית תורנית ולארמית, המציע טוקנים אפשריים במקום סימן המסכה לפי ההקשר משני הצדדים. מתאים למחקר לשוני, ניסויי השלמת טקסט והמשך אימון למשימות שפה. המאגר כולל משקלים רגילים בטוחים לטעינה, טוקנייזר, ONNX FP32 ו־ONNX INT8, דוחות בחירה ואימות וחתימות SHA-256.

זהו מודל להשלמת טוקנים חסרים, ולא צ'אט או מחולל תשובות. תחזית טוקן היא הצעה סטטיסטית ואינה מקור תורני, הכרעת הלכה או שחזור מוסמך של נוסח. מודל ההטמעה בסדרה מיועד לחיפוש סמנטי; המודל הזה אינו תחליף לו. מילה עברית או ארמית יכולה להתפצל לכמה טוקנים, ולכן מסכה אחת אינה בהכרח מילה שלמה.

במבט אחד

42.16 מיליון פרמטרים · 512 טוקנים · 3 פורמטים · 15,600 צעדי MLM מתועדים.

בבדיקת הטוקנים המוסתרים הקבועה, דיוק הניחוש הראשון הוא 30.47%, והטוקן הנכון מופיע בחמש ההצעות הראשונות ב־49.40% מהמקרים. ה־loss הממוצע לפי אורכי רצף השתפר ב־3.40% ביחס לבסיס. אלו תוצאות הפרוטוקול המתועד, ולא מדדי איכות אחזור או דיוק של תשובות.

Release formats

File sizes use decimal MB.

All formats derive from the same selected checkpoint. ONNX exports return full vocabulary logits with shape [batch, sequence, 32000]. Inputs are input_ids and attention_mask, both int64, with dynamic batch and sequence dimensions up to 512 tokens. Quantization is an inference variant, not a separately trained model. Performance depends on hardware and input shape; no universal speedup is claimed.

Architecture

42,162,432 parameters. BERT encoder with 8 layers, hidden width 512, 8 attention heads, intermediate width 2048, 32,000-token vocabulary, tied input/output embeddings and 512 position embeddings. The context limit includes special tokens. The repository uses standard BertForMaskedLM; no custom remote code is required. Tokenization uses the project's Hebrew/Aramaic Unigram/Metaspace tokenizer. Discover the mask spelling from tokenizer.mask_token instead of assuming it.

Quick start: Transformers

Request access on the model page, authenticate with hf auth login, and install:

pip install torch transformers huggingface_hub safetensors

import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM

repo = "ArieLLL123/Meivin-MLM-0.1"
tokenizer = AutoTokenizer.from_pretrained(repo, token=True)
model = AutoModelForMaskedLM.from_pretrained(repo, token=True).eval()
text = f"אמר רבי {tokenizer.mask_token} תורה"
inputs = tokenizer(text, return_tensors="pt", truncation=True,
                   max_length=512, return_token_type_ids=False)
positions = (inputs["input_ids"][0] == tokenizer.mask_token_id).nonzero().flatten()
with torch.no_grad():
    logits = model(**inputs).logits
for position in positions.tolist():
    scores, ids = logits[0, position].softmax(-1).topk(5)
    for score, token_id in zip(scores.tolist(), ids.tolist()):
        print(tokenizer.decode([token_id]), round(score, 4))

For GPU inference, move both model and tensor inputs to the same device. A multi-mask call predicts all masks jointly from the masked context; it does not implement autoregressive sentence generation. These probabilities are not calibrated confidence scores for factual correctness.

Quick start: ONNX / INT8

pip install onnxruntime transformers huggingface_hub numpy

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

repo = "ArieLLL123/Meivin-MLM-0.1"
tokenizer = AutoTokenizer.from_pretrained(repo, token=True)
# Switch to onnx/model.onnx for FP32.
import json
config_path = hf_hub_download(repo, "config.json", token=True)
config = json.load(open(config_path, encoding="utf-8"))
path = hf_hub_download(repo, "onnx/model_int8.onnx", token=True)
session = ort.InferenceSession(path, providers=["CPUExecutionProvider"])
inputs = tokenizer(f"בראשית ברא {tokenizer.mask_token} את השמים ואת הארץ",
                   return_tensors="np", return_token_type_ids=False,
                   truncation=True, max_length=config["max_position_embeddings"])
feed = {name: inputs[name].astype(np.int64) for name in ("input_ids", "attention_mask")}
logits = session.run(["logits"], feed)[0]
position = np.flatnonzero(inputs["input_ids"][0] == tokenizer.mask_token_id)[0]
top5 = np.argsort(logits[0, position])[-5:][::-1]
print([tokenizer.decode([int(i)]) for i in top5])

ONNX provides inference only; use the regular weights for gradient-based fine-tuning. Full logits can consume substantial memory for long sequences and large batches. Begin with a small batch.

Training lineage and checkpoint choice

This release packages selected/ from ArieLLL123/judaic-semantic-teacher-8x512-mlm-mixed-v2, pinned to the exact revision recorded in source_revision.json. The training decision selected step-2400, passing all recorded MLM guardrails.

Controlled continuation used sequence lengths 64, 96, 128, 160, 192, 224, 256, 320, 384, 448 and 512, averaging 234.88 tokens under the configured distribution. Mask probability decreased from 25% to 15%; learning rate was 2.5e-5, micro-batch 8, gradient accumulation 16, and the recorded run used FP16. Corpus buckets cover Tanakh, Mishnah, Bavli, Yerushalmi, Midrash, halakha, rishonim, acharonim, Kabbalah, later Judaica and modern text. These are training metadata, not a guarantee of equal quality for every genre.

Only the release artifacts and aggregate reports are distributed here. Corpus rows, private evaluation text, optimizer state and training credentials are excluded. The model release does not grant rights to underlying source texts.

Evaluation: measured results

One fixed mask-only test suite contained 1,408 exact-length packed examples (128 for each of 11 lengths) and 47,168 masked tokens, with 15% masking. The comparisons below use the identical protocol. Perplexity applies to masked tokens under this protocol, not autoregressive language modeling.

The recorded 3.398% relative improvement is for the macro loss across lengths, used by the selection gate; it must not be confused with token-weighted overall loss. Full numerical reports are in baseline_report.json, candidate_reports.json and model_decision.json.

The original decision explicitly required a separate controlled retrieval ablation before promotion as a retrieval backbone. This release claims MLM readiness and does not claim that this pending gate has been resolved, or that this model improves semantic search.

Export validation and limitations

export_validation.json records actual FP32/INT8 comparisons against PyTorch at sequence lengths 8, 64, 192 and 512, including a batch of two and masked padding, plus four Hebrew/Aramaic mask examples. FP32 export must meet maximum absolute error below 0.001; INT8 must exceed 80% top-1 agreement in each synthetic smoke case. Those are deployment smoke checks, not a full corpus quality evaluation of INT8. The original training benchmark describes the unquantized checkpoint.

The model may produce unsuitable, incorrect or biased completions, and quality may vary with spelling, abbreviations, genre, rare terminology and multi-token words. It is not validated for textual restoration, religious decisions, medical/legal advice, or factual question answering. No comparison with external commercial models is claimed. Use human review for displayed suggestions and evaluate task-specific quality before integration.

Reproducibility and integrity

manifest.json records SHA-256 and byte size for every release file except the manifest itself, plus immutable source provenance. RELEASE_NOTES.md identifies this packaging release. Training reports are preserved as source evidence, including their original run paths. The export validation records runtime versions. Dependency versions in requirements.txt describe the tested environment, without bundling third-party libraries.

License, access and contact

Meivin Non-Commercial License 1.0 applies to this release's model materials to the extent the licensor can license them. Commercial integration, internal business use, hosted services and commercial advantage require prior explicit written permission, even if the integrating feature is free. Redistribution and derivatives must preserve the license, notices, attribution and restrictions. Accompanying project-authored code remains governed by Personal Use License 1.0 reproduced in LICENSE.md; consult those distinct terms before organizational use of the sample code.

Manual access approval is not commercial authorization. Earlier versions retain valid rights already granted under their applicable terms; this release does not revoke them. Attribution: Ariel Daniely (אריאל דניאלי), Judaic Semantic Embedding project within Otzaria. Commercial permissions and collaboration: arieldaniely@gmail.com.

Citation

@misc{daniely2026meivinmlm,
  author = {Ariel Daniely},
  title = {Meivin MLM 0.1: Hebrew and Aramaic Masked Language Model},
  year = {2026},
  url = {https://huggingface.co/ArieLLL123/Meivin-MLM-0.1}
}

Detailed training report | תיעוד אימון מלא

TRAINING.md documents the 15,600 evidenced MLM training steps, corpus preparation, masking, sampling, hyperparameters, checkpoint selection, curves and gaps in the historical logs. Later retrieval experiments belong to the Embed branch and are not counted as MLM training.

Languages and textual coverage | שפות וניבי לשון

הסימון הידני he, tmr, jpa, arc מתאים להיקף המקורות. tmr הוא הארמית היהודית הבבלית, jpa הארמית היהודית הארץ־ישראלית, ו־arc משמש כאן גם לסימון הארמית המקראית. דניאל ועזרא כוללים עברית וארמית; בבלי וירושלמי הם שמות קורפוסים, לא שמות שפות. אין לסמן את כל התלמודים כשפה אחת או את כל ספר דניאל כארמית.

The arc ISO label is historical and is not a precise label for every Aramaic variety in the corpus. Targumic and Zoharic/literary Aramaic are also represented in the broader Judaic source collection; they are described here rather than assigned unrelated modern Aramaic codes. Presence of texts establishes training coverage, not independently benchmarked mastery of each dialect. No per-language quality scores or exact percentages of Aramaic are available, because corpus buckets are not language annotations. English instructions in this repository do not imply English model support.

See aggregate corpus checks. Language code references: Jewish Babylonian Aramaic, Jewish Palestinian Aramaic, and Biblical Hebrew/Aramaic language codes in BHSA.

Downloads and parameter display | הורדות ופרמטרים

The full FP32 checkpoint is distributed as root model.safetensors, enabling Hugging Face to inspect its tensor headers. The count includes stored training parameters; ONNX can omit unused components. INT8 is a numerical export of the same model, not a reduction in architectural parameter count.

Use the documented loader/configuration together with the weights. The Hub counts server-side requests to designated query files such as config.json; it does not count unique users or every arbitrary file download. ONNX integrations below load the real configuration as part of model initialization. Direct downloads of ONNX files alone may not be represented in this counter. Historical missed requests cannot be reconstructed from repository metadata.

The model page shows recent downloads; Hub API also exposes downloadsAllTime for cumulative downloads, when available. The release includes an example that reads both fields live, without hardcoding a launch-day number. See Hugging Face counting rules.