SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

📣 Updated News

• [2026.06.29] Release SenseNova-U1-8B-MoT-Infographic-V2 📊, an upgraded infographic model with improved dense small-text rendering with sharper text edges, stronger complex dense-layout generation, and better overall visual aesthetics and harmony, plus a fix for the black-background issue. Model details and visual examples are available in ✨ U1 Infographic Model Series.

• [2026.06.12] Release SenseNova-U1-8B-MoT-Infographic-LoRA-8step-V1.0 for faster infographic generation. Please see the example script.

• [2026.06.11] Release SenseNova-U1-8B-MoT-Interleaved 📖, specially optimized for interleaved image-text generation, with notably improved narrative coherence, character and style consistency, and text-image alignment in multi-page content.

• [2026.05.21] Release the full-parameter fine-tuning training code for SenseNova-U1.

• [2026.05.15] Release SenseNova-U1-8B-MoT-Infographic 📊 model for improved infographic generation. See U1 Infographic Model Series for details, and ✨ Infographic Showcases for 100 generated examples.

• [2026.05.10] Release 🔥SenseNova-U1 Technical Report🔥 and the weights for SenseNova-U1-A3B-MoT-SFT & SenseNova-U1-A3B-MoT.

• [2026.05.08] Add GGUF quantized checkpoints and layer-offload VRAM modes for low-VRAM single-GPU inference. See Memory-efficient inference. GGUF weights for SenseNova-U1-8B-MoT-Merger are available at 🤗 smthem/SenseNova-U1-8B-MoT-Merger-gguf — many thanks to @smthem for contributing the quantized weights.

• [2026.05.06] Release SenseNova-U1-8B-MoT-LoRA-8step-V1.0. Please see the example script.

• [2026.04.30] Release the preview version of the 8-step inference model SenseNova-U1-8B-MoT-8step-preview. In most cases, the image generation quality of this model closely matches that of the base model (see comparison and existing issues). To test this model, you can use the inference scripts, but with the following parameters: --cfg_scale 1.0 --num_steps 8 .

• [2026.04.27] Initial release of the weights for SenseNova-U1-8B-MoT-SFT and SenseNova-U1-8B-MoT.

• [2026.04.27] Initial release of the inference code for SenseNova-U1.

🌟 Overview

🚀 SenseNova U1 is a new series of native multimodal models that unifies multimodal understanding, reasoning, and generation within a monolithic architecture. It marks a fundamental paradigm shift in multimodal AI: from modality integration to true unification. Rather than relying on adapters to translate between modalities, SenseNova U1 models think-and-act across language and vision natively.

Unifying visual understanding and generation in an end-to-end architecture from pixel to word opens tremendous possibilities, enabling highly efficient and strong understanding, generation, and interleaved reasoning in a natively multimodal manner.

🏗️ Key Pillars:

At the core of SenseNova U1 is NEO-unify, a novel architecture designed from the first principles for multimodal AI: It eliminates both Visual Encoder (VE) and Variational Auto-Encoder (VAE) where pixel-word information are inherently and deeply correlated. Several important features are as follows:

• 🔗 Model language and visual information end-to-end as a unified compound.
• 🖼️ Preserve semantic richness while maintaining pixel-level visual fidelity.
• 🧠 Reason across modalities with high efficiency & minimal conflict via native MoTs.

Powered by this new core architecture, SenseNova U1-8B-MoT-Infographic (infographic-specifically enhanced version of SenseNova U1-8B-MoT) delivers exceptional efficiency and state-of-the-art infographic performance:

• Benchmark Performance: Compared with SenseNova-U1-8B-MoT-Infographic, V2 improves BizGenEval hard/easy from 46.6 / 65.4 to 50.3 / 67.9 (+3.7 / +2.5), and IGenBench Q-ACC/I-ACC from 69.5 / 17.0 to 71.4 / 18.3 (+1.9 / +1.3). OneIG(EN/ZH) is 55.4 / 53.5, roughly on par with the previous version, indicating that infographic-specific capability improves while general generation ability remains stable.
• Generation Quality: V2 further improves small-text rendering, complex dense layout, and overall visual aesthetics: text edges are sharper, high-information-density layouts are more stable, and poster, dashboard, and report-style infographics look more polished. It also fixes the black-background issue, avoiding unintended black or overly dark backgrounds.

Note: IGenBench scores are reported as percentages. Commercial and open-source models are sorted separately by the average of BizGenEval hard/easy and IGenBench Q-ACC/I-ACC. OneIG is included as a general generation reference.

• 📰 High-density information rendering (Specialized): This specific model demonstrates strong capabilities in dense visual communication, generating richly structured layouts for knowledge illustrations, posters, presentations, comics, resumes, and other information-rich formats.

• 🏆 Open-source SoTA: SenseNova U1 sets a new standard for unified multimodal understanding and generation, achieving state-of-the-art infographic performance among open-source models.

🎨 Infographic Showcases

V2 Qualitative Comparison: SenseNova-U1-8B-MoT-Infographic vs. SenseNova-U1-8B-MoT-Infographic-V2

We present a qualitative comparison between the SenseNova-U1-8B-MoT-Infographic (V1) model and the upgraded SenseNova-U1-8B-MoT-Infographic-V2 model across three key dimensions: small-text rendering, complex dense-layout generation and overall aesthetics, and background stability. For the full comparison, please refer to ✨ U1 Infographic Model Series.

Small-Text Rendering

Complex Dense Layout and Overall Aesthetics

Background Stability

🛠️ Quick Start

🌐 Use with SenseNova-Studio

The fastest way to experience SenseNova-U1 is through SenseNova-Studio — a 🆓 free online playground where you can try the model directly in your browser, no installation or GPU required.

Note: To serve more users, U1-Fast has undergone step and CFG distillation, and is dedicated to infographic generation.

🦞 Use with SenseNova-Skills (OpenClaw)

The easiest way to integrate SenseNova-U1 into your own agent or application is through our companion repository SenseNova-Skills (OpenClaw) 🦞, which ships SenseNova-U1 as a ready-to-use skill with a unified tool-calling interface.

Refer to the SenseNova-Skills README for installation and usage details.

🤗 Run with transformers (Default)

Setup: Follow the Installation Guide to clone the repo and install dependencies with uv.

🌟 Generate High-Quality Infographics

For generating complex infographics, we highly recommend using the following parameters: --cfg_scale 4.0, --timestep_shift 3.0, and --num_steps 50.

python examples/t2i/inference.py \
  --model_path sensenova/SenseNova-U1-8B-MoT-Infographic-V2 \
  --prompt "该信息图以复古漫画/动漫风格为主题设计，整体背景为深蓝色星空背景（带有颗粒质感），由多个带有技术网格线和星座线图案装饰的模块围绕中心主题呈海报式上下左右分布布局。中心是一个由两位角色与驾驶舱构成的主体视觉区域，内部包含主标题“SENSETIME”（带星星装饰的黄色大号粗体字），副标题“2026 走向未来！”（蓝色文字），下方有左上角信息框和文字“任务编号：K-2026-ALPHA | 发布日期：2026.06.27 | 保密等级：公开”。中心上方有散落的黄色星星与星座线图案装饰。 信息图共分为六个主要部分，编号1至6，分布在中心周围： **1. 顶部区域（标题与信息框模块）** - **主标题**（黄色大号粗体字带星星装饰）：“SENSETIME”。 - **副标题**（蓝色文字）：“2026 走向未来！”。 - **左上角信息框**（信息标签）：包含“任务编号：K-2026-ALPHA”、“发布日期：2026.06.27”、“保密等级：公开”。 **2. 中心主视觉（角色与装备展示模块）** 以三位角色及驾驶舱按顺序排列，标签连接： - **左侧男性角色**（戴护目镜、红围巾、弹药带，手持武器，帅气姿态动作）：标签文字“左轮手枪”（指向枪套），描述文字“标准配备｜口径：.45｜弹容量：6发｜重量：1.1kg”。 - **中心女性角色**（戴护目镜、红围巾、条纹衫，弹奏白色空心电吉他，吉他上有“KATAO”字样，优美姿态）：标签文字“格雷奇”（指向角色），描述文字“装备型号：G-77电吉他｜频率范围：80Hz-12kHz｜输出功率：50W”。 - **背景右侧角色**（驾驶舱内两名角色，女性调整护目镜，男性戴护目镜看前方）：驾驶舱标签“驾驶舱控制面板｜导航系统：GPS-2025｜通讯频道：CH-77”。 **3. 左侧边栏（任务简报模块）** 包含一段说明文字：“任务简报” 下方配有五项核心指标列表： - 目标：探索未知领域 - 时间：2025年启动 - 地点：黑空 - 队员：精英组 - 装备：未来科技武装 **4. 右侧边栏（成员档案模块）** 包含一段说明文字：“探险队成员档案” 下方配有五名成员及技能列表： - 队长：经验丰富的领航员 - 副队长：技术专家 - 成员A：武器专家 - 成员B：通讯专员 - 特殊技能：时空导航、能量操控、战术分析 **5. 右下角区域（批次规格模块）** 包含一个金色警长星徽图案，内文：“活力 编号01 未来”。 - **大标题**（醒目文字）：“ETB-77”。 - **描述文字**：“这是将分配给未来探险任务的批次。” - **详细信息框**（下方数据框）：“批次规格｜成员数量：2人｜任务时长：365天｜装备清单：完整｜能源补给：充足｜目标区域：未定”。 **6. 底部横幅与背景装饰（标语与元素模块）** 包含一段说明文字：“KATAO探险队｜连接过去与未来｜探索无限可能｜官方网站：www.SENSETIME.com｜联系我们：SENSETIME” 下方配有背景元素装饰：散落的黄色星星、星座线图案、技术网格线装饰。 整体视觉风格复古、结构清晰，通过鲜艳色彩与颗粒质感增强视觉冲击力和未来感，强调在未来探险任务中遵循精英组队、装备未来科技武装，并通过时空导航与能量操控达成探索无限可能的目标。所有文本均为中文（保留特定英文标识），语言简洁硬核，适合用于科幻探险主题的复古漫画风格海报宣传场景。" \
  --width 2720 --height 1536 \
  --cfg_scale 4.0 --cfg_norm none --timestep_shift 3.0 --num_steps 50 \
  --output output.png --profile

Default resolution is 2048×2048 (1:1). See supported resolution buckets for other aspect ratios.

For high-quality infographic generation, it is recommended to apply prompt enhancement before generating images.

💾 Memory-efficient inference (GGUF + VRAM modes)

For users running on a single consumer GPU, two complementary features lower the VRAM footprint of the transformers path. They can be combined freely.

--vram_mode: single-GPU layer offload

Pass --vram_mode to keep the language-model layers resident on CPU pinned memory and stream them onto the GPU on-demand during forward, freeing weight VRAM while keeping activations on-device.

python examples/t2i/inference.py \
  --model_path sensenova/SenseNova-U1-8B-MoT-Infographic-V2 \
  --vram_mode balanced \
  --prompt "..." --output output.png

--gguf_checkpoint and --vram_mode compose: a Q4 GGUF + balanced is the recommended setup for ~10–12 GB consumer cards.

⚡ Run with LightLLM + LightX2V (Recommended)

For production serving, we co-design a dedicated inference stack on top of LightLLM (understanding) and LightX2V (generation). The two engines are disaggregated so that each path can use its own parallelism and resource budget, with a low-overhead transfer channel in between.

On a single node with TP2 + CFG2, this stack delivers roughly ~0.15 s/step and ~9 s end-to-end for a 2048×2048 image on H100 / H200, with a ~2.4–3.2× prefill speedup from our FA3-based hybrid-mask attention over the Triton baseline. Full per-GPU performance are reported in docs/inference_infra.md.

An official docker image is provided for one-command deployment:

docker pull lightx2v/lightllm_lightx2v:20260407

⚙️ Deployment guide (Docker, launch flags, modes, quantization, API test): see docs/deployment.md.

📖 Full design and performance profiling: see docs/inference_infra.md.

🌐 Join the Community!

Join our growing community to share feedback, get support, and stay updated on the latest SenseNova-U1 developments — we'd love to hear from you!

⚖️ License

This project is released under the Apache 2.0 License.