NOESIS / AMAImedia
Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO — Deterministic Hybrid Control Framework for Frozen Neural Operators).
- Founder: Ilia Bolotnikov
- Organization: AMAImedia.com
- X (Twitter): @AMAImediacom
- LinkedIn: Ilia Bolotnikov
- Telegram: @djbionicl
- NOESIS version: v16.1
- Release date: 2026-08-26
Emo-1T-14B-A1B-OLMoE-NOESIS-BF16
BF16 dtype-repack of
allenai/Emo_1b14b_1T— original FP32 floating-point weights losslessly cast tobfloat16for LoRA / DoRA / PEFT compatibility and reduced disk footprint. The model architecture, parameter values, tokenizer, and configuration are identical to upstream — only the IEEE-754 storage dtype was changed.
License preserved end-to-end — see
LICENSEin this repo for the full text and attribution chain.
Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO — Deterministic Hybrid Control Framework for Frozen Neural Operators).
- Repack date: 2026-05-19
Summary
Mixture-of-Experts language model trained with the Expert Modularity Objective (EMO) — documents constrained to route through shared expert pools during training, enabling domain-specific expert subsets to be deployed in isolation. Pretrained on 1T tokens (OLMoE pretraining mix) + 50B token annealing under the EMO objective.
Use case inside NOESIS
Research model for emergent modularity / domain-specific expert routing. English-only. Inside NOESIS this is out-of-scope for the dubbing pipeline (single-language, research-oriented) — kept as research reference.
What changed vs upstream
| Aspect | Upstream | This bundle |
|---|---|---|
| Floating-point storage dtype | FP32 | bfloat16 |
config.json torch_dtype | as-is | bfloat16 |
model.safetensors.index.json total_size | as-is | recomputed |
| Tokenizer / chat template / modeling code | as-is | unchanged |
| Number of parameters | as-is | unchanged |
| Value-level transformation beyond dtype cast | — | none |
| Disk size | 51 GB | 26 GB |
Architecture
| Property | Value |
|---|---|
| Immediate parent | allenai/Emo_1b14b_1T |
| Architecture | EmoForCausalLM |
| Architecture base / lineage | OLMoE-family MoE (128 experts = 127 routed + 1 shared, k=8 active per token, Expert-Modularity-Objective pretraining) |
| Parameters | 1B active / 14B total |
| Hidden size | 2048 |
| Num hidden layers | 16 |
| Attention heads / KV heads | 16 |
| Vocab size | 100352 |
| Max position embeddings | 4096 |
| Format | bfloat16 |
| Bundle size on disk | 26 GB |
| License | Other (see upstream — typically Apache 2.0 OLMoE family policy) |
| Project page | https://huggingface.co/allenai/Emo_1b14b_1T |
| Paper / arxiv | arxiv:2605.06663 |
Repack tooling
CPU-only sharded repack via
repack_fp32_to_bf16.py
— reads each shard with safetensors.safe_open, casts floating-point
tensors to torch.bfloat16, rewrites the shard, updates the index
manifest. No GPU involvement, no value-level transformation
beyond the IEEE-754 dtype cast.
Performance reference (RTX 3060 laptop, NVMe SSD):
- Single 5 GB FP32 shard cast → ~28-40 sec
- Full 51 GB → 26 GB in 1 pass, sharded
Use cases (for the BF16 bundle)
- ✅ LoRA / DoRA / IA³ fine-tuning that requires a
dtype=torch.bfloat16base - ✅ Bitsandbytes NF4 / AWQ-INT4 / GPTQ quantization (these tools prefer BF16 input)
- ✅ Inference on Ampere+ / MI200+ hardware with native BF16 support
- ✅ KD-teacher (forward-only) where BF16 storage saves bandwidth
- ❌ Full-parameter fine-tuning of weights — use FP32/BF16 master weights pattern; storage dtype alone is insufficient
Quick start
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
repo = "AMAImedia/Emo-1T-14B-A1B-OLMoE-NOESIS-BF16"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
repo,
dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
).eval()
Sealed rules (NOESIS DHCF-FNO)
R-DTYPE-REPACK-BF16— pure IEEE-754 dtype cast from FP32 to bfloat16. No value-level transformation, no LoRA merge, no architectural change. Equivalent to loading upstream withdtype=torch.bfloat16and saving, but materialised on disk.R-UPSTREAM-OTHER— upstream Other (see upstream — typically Apache 2.0 OLMoE family policy) preserved end-to-end via the LICENSE file in this repo. AMAImedia adds only a derivative-work notice for the repack step.R-NO-VALUE-TRANSFORM— no fine-tuning, no distillation, no merge has been applied between upstream and this repo. Outputs are bit-for-bit equivalent up to the precision difference of the dtype cast.
License & attribution
This bundle inherits Other (see upstream — typically Apache 2.0 OLMoE family policy) from
allenai/Emo_1b14b_1T. Original
model card, citation, and attribution from upstream apply without
modification. See LICENSE in this repo for the complete text plus the
NOESIS derivative-work NOTICE.
Citation
@misc{noesis2026emo1t14ba1bolmoenoesisbf16bf16,
title = {NOESIS DHCF-FNO :: Emo-1T-14B-A1B-OLMoE-NOESIS-BF16 — BF16 dtype-repack derivative},
author = {Bolotnikov, Ilia and AMAImedia},
year = {2026},
note = {BF16 dtype-repack of allenai/Emo_1b14b_1T for LoRA / PEFT
compatibility. 26 GB on disk, Other (see upstream — typically Apache 2.0 OLMoE family policy)
preserved end-to-end.},
url = {https://huggingface.co/AMAImedia/Emo-1T-14B-A1B-OLMoE-NOESIS-BF16}
}
Please also cite the upstream model when using this bundle. See the
upstream README and LICENSE in this repo for citation requirements.
- Repack date: 2026-05-19
- HF repo:
AMAImedia/Emo-1T-14B-A1B-OLMoE-NOESIS-BF16 - Upstream:
allenai/Emo_1b14b_1T
Produced 2026-08-26 by NOESIS DHCF-FNO v15.8 — AMAImedia.com