NOESIS-MiniCPM-V-4.6-SigLIP2-BF16 (inside NOESIS-3.5B-A0.5B-DUBBING-FILM)

Vision-only BF16 extraction derivative of openbmb/MiniCPM-V-4.6 — the SigLIP2 vision tower (453 tensors, 996 MB) extracted from MiniCPM-V-4.6 and rebuilt as a stand-alone HF SiglipVisionModel-compatible bundle. Integrated as the vision submodule for Stage 4 / 6 / 16 of the parent NOESIS-3.5B-A0.5B-DUBBING-FILM dubbing pipeline.

✅ APACHE 2.0 — COMMERCIAL USE PERMITTED. Both upstreams are Apache 2.0: OpenBMB MiniCPM-V-4.6 (vision_tower weights extracted here) and Google's SigLIP2 (the original vision-encoder design). This NOESIS extraction inherits the same Apache 2.0 terms and is commercial-safe inside the parent NOESIS-3.5B bundle.

NOESIS / AMAImedia

Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO — Deterministic Hybrid Control Framework for Frozen Neural Operators).

• Founder: Ilia Bolotnikov
• Organization: AMAImedia.com
• X (Twitter): @AMAImediacom
• LinkedIn: Ilia Bolotnikov
• Telegram: @djbionicl
• NOESIS version: v15.9
• Extraction date: 2026-05
• Last updated: 2026-08-28

Use case

Vision-only auxiliary encoder inside the NOESIS-3.5B-A0.5B-DUBBING-FILM dubbing pipeline. The encoder consumes face / mouth crops or scene frames and produces dense patch-level visual features [B, num_patches, 1152] consumed by three pipeline stages:

Production NOESIS routes vision through this Apache-2.0 stack (MiniCPM-V-4.6 SigLIP2 → AMAImedia extraction) — no CC-BY-NC or GPLv3 vision-encoder mixing is allowed by the parent NOESIS-3.5B sealed rule set (see R-APACHE-CLEAN, R-NO-VISION-NC below).

NOESIS-3.5B-A0.5B-DUBBING-FILM vision stack — where this fits

The parent NOESIS-3.5B bundle uses a vision-only auxiliary encoder for Stage 4/6/16 work. This SigLIP2 extraction is the canonical pick:

Why MiniCPM-V-4.6's SigLIP2 vs raw Google SigLIP2: MiniCPM-V fine-tunes its vision tower extensively on multi-image / dense-caption / video data during MLLM training. The resulting tower carries OCR + grounded detection + temporal cues that raw Google SigLIP2 lacks. For lip-motion forced alignment on movie-style close-up shots, the MiniCPM-tuned tower outperforms the raw base by a clear margin while staying Apache 2.0.

What's in this bundle

What was DISCARDED

• LLM backbone — Qwen3.5-0.8B hybrid inside MiniCPM-V-4.6. NOESIS uses its own Qwopus3.5-0.8B-Reasoning trunk for text/LM, so the embedded MiniCPM LM is redundant.
• Multimodal projection — the image_features → LLM embedding linear adapter. NOESIS connects raw SigLIP2 patch features through its own projection head (separate, trained for dubbing-specific tasks).
• Chat templates / tokenizer — vision-only inference does not use them.
• Generation config — vision-only inference has no autoregressive decoding loop.

The extraction reduces 4.5+ GB MiniCPM-V-4.6 weights down to 996 MB of pure vision-tower BF16 tensors.

Why BF16 (not NF4)

This bundle is intentionally kept BF16, not NF4:

If a strictly smaller footprint is needed for embedded inference, bitsandbytes 4-bit quantization can be applied post-load (see quantization_config=BitsAndBytesConfig(load_in_4bit=True, ...) in HF docs); this is not the default path inside NOESIS-3.5B.

Quick start

Loading the SigLIP2 encoder (NOESIS path)

import torch
from transformers import SiglipVisionModel, SiglipVisionConfig

bundle = r"B:\...\NOESIS-3.5B-A0.5B-DUBBING-FILM\NOESIS-MiniCPM-V-4.6-SigLIP2-BF16"

config = SiglipVisionConfig.from_pretrained(bundle)
model = SiglipVisionModel(config)

# Option A — load from safetensors (recommended, mmap-friendly)
from safetensors.torch import load_file
state = load_file(f"{bundle}/model.safetensors")
model.load_state_dict(state, strict=True)

# Option B — load from torch .pt state_dict (alternative)
# state = torch.load(f"{bundle}/siglip2_vision_encoder.pt", map_location="cpu")
# model.load_state_dict(state, strict=True)

model = model.to(torch.bfloat16).cuda().eval()

# Inference: pixel_values is [B, 3, 980, 980] BF16 on CUDA
with torch.no_grad():
    out = model(pixel_values)
    image_features = out.last_hidden_state          # [B, num_patches, 1152]
    pooled = out.pooler_output if out.pooler_output is not None else image_features.mean(1)

Using upstream preprocessor

from transformers import AutoImageProcessor

processor = AutoImageProcessor.from_pretrained(bundle)
inputs = processor(images=pil_image, return_tensors="pt").to("cuda", torch.bfloat16)
features = model(**inputs).last_hidden_state

Sealed rules (NOESIS DHCF-FNO)

• R-APACHE-CLEAN — Apache 2.0 preserved end-to-end. Vendored copy may be freely combined with other Apache-2.0 NOESIS components (Qwen3-ASR-0.6B / 1.7B, omniASR-CTC, Qwen3-ForcedAligner, parent NOESIS-3.5B trunk).
• R-VENDORED-INTERNAL — this folder is an internal vendor copy inside the parent NOESIS-3.5B-A0.5B-DUBBING-FILM bundle, not a separately distributed NOESIS publication. Distribution happens as part of the parent NOESIS-3.5B archive.
• R-VISION-ONLY-EXTRACT — only model.vision_tower.* was extracted. LLM backbone, multimodal projection, chat templates and tokenizer are intentionally discarded. Re-introducing them inside this bundle is forbidden — the parent NOESIS-3.5B trunk owns the language path (Qwopus3.5).
• R-BF16-NATIVE — keep as upstream BF16; do not NF4-quantize this bundle by default (quality regression on ±10 ms lip-sync timing outweighs the ~0.5 GB VRAM savings on 6 GB GPU).
• R-FROZEN-VISION-OPERATOR — SigLIP2 vision tower is treated as a frozen neural operator under DHCF-FNO. No gradient backprop through these weights inside Phase 1 NOESIS training (scope-locked to LoRA r≤8 on the parent NOESIS-3.5B trunk only).
• R-DUBBING-FILM-SCOPE — vision encoder serves film-dubbing tasks (Stage 4 active speaker + Stage 6 lip-motion forced alignment + Stage 16 lip-sync QC). Not for general image captioning, OCR, or generic VQA — those pathways are out of NOESIS Phase 1 scope (SCOPE LOCK 2026-05-15).
• R-NO-VISION-NC — production NOESIS vision must remain on Apache 2.0 (this bundle). Mixing CC-BY-NC / non-commercial vision encoders (e.g. MMS-style NC variants) into the dubbing pipeline is forbidden by R-APACHE-CLEAN propagation.
• R-NO-FROM-SCRATCH-VLM — per Phase 1 scope lock, no training of custom vision-language models from scratch. This SigLIP2 tower is used as-is.
• R-NEVER-DELETE-WITHOUT-EXPLICIT-CONSENT — MUST NOT be deleted without explicit operator instruction "удали " / "delete ". Sealed 2026-05-21.

Provenance

Citation

@proceedings{yu2025minicpmv45cookingefficient,
  title  = {MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture,
            Data, and Training Recipe},
  author = {Tianyu Yu and Zefan Wang and Chongyi Wang and Fuwei Huang and
            Wenshuo Ma and Zhihui He and Tianchi Cai and Weize Chen and
            Yuxiang Huang and Yuanqian Zhao and others},
  year   = {2025},
  url    = {https://arxiv.org/abs/2509.18154}
}

@article{yao2024minicpm,
  title  = {MiniCPM-V: A GPT-4V Level MLLM on Your Phone},
  author = {Yao, Yuan and Yu, Tianyu and Zhang, Ao and Wang, Chongyi and
            Cui, Junbo and Zhu, Hongji and Cai, Tianchi and Li, Haoyu and
            Zhao, Weilin and He, Zhihui and others},
  journal= {arXiv preprint arXiv:2408.01800},
  year   = {2024}
}

@misc{noesis2026minicpmsiglip2,
  title  = {NOESIS DHCF-FNO :: MiniCPM-V-4.6 SigLIP2 vision-only extraction
            (BF16, 996 MB) for film-dubbing Stage 4/6/16},
  author = {Bolotnikov, Ilia and AMAImedia},
  year   = {2026},
  note   = {Vision-only BF16 extraction derivative of openbmb/MiniCPM-V-4.6
            rebuilt as stand-alone SiglipVisionModel for active-speaker
            detection, lip-motion forced alignment, and lip-sync QC inside
            the NOESIS-3.5B-A0.5B-DUBBING-FILM dubbing pipeline.},
  url    = {https://amaimedia.com}
}

License

Apache License 2.0. Original MiniCPM-V-4.6 weights (including the extracted vision_tower) © OpenBMB / THUNLP (Tsinghua University) (2024-2026). SigLIP2 foundational vision-encoder design © Google LLC (2024-2026). NOESIS vision-only extraction, BF16 SiglipVisionModel rebuild, integration metadata, sealed-rule wiring: © AMAImedia (NOESIS DHCF-FNO project) 2026 (redistributed under Apache 2.0).

Commercial use is permitted subject to the standard Apache 2.0 preservation requirements (copyright + LICENSE + NOTICE-equivalent attribution must travel with redistributions). See LICENSE in this folder for the full Apache 2.0 text plus the NOESIS attribution banner and required-attribution block.

The upstream MiniCPM-V authors' disclaimer about MLLM content generation (see LICENSE § Statement) is inherited verbatim by this derivative.

Bundle identity

• Parent bundle: NOESIS-3.5B-A0.5B-DUBBING-FILM
• Vendored component: NOESIS-MiniCPM-V-4.6-SigLIP2-BF16 (SigLIP2 vision tower, BF16, 996 MB)
• Original extraction date: 2026-05-17
• Source card last updated: 2026-05-27

📚 Theorem reference layout (post 2026-05-20 split). This document references NOESIS Theorem entries (NT-N / R-X / B-N / Decision-N). The project-level theorem files and split tooling are not bundled in this model-only repository; the original path names are retained as plain code references for provenance without creating dead Hugging Face links.

Project references: docs/NOESIS_04_THEOREM.md, docs/theorem/00_INDEX.md, docs/theorem/ANCHOR_MAP.json, scripts/split_theorem.py, and scripts/migrate_theorem_refs.py.