NSFW enabled capable model that still has base model capability and functions. Betas are all testing integrations and bases and are just samples ahead of inegrating sulphur h3 which will be the main data backbone and v1.0.

beta_6 is a built mostly-reference hybrid with a custom turbo meant to mimic how Fastv2 looks and sounds. Samples on 10/3 shift, fully dense or with Fastv2 VSA sparse node settings on multiple samplers like er_sde, euler, LCM, res_multistep, all having some different behaviors with prompt. It's a slow 8-12 step reference model in that configuration. 4step turbo loras can be run on top of it to reduce that. SLA sparsity seems to destroy it, avoid using that on any workflows. It's ideal configuration is with full gates and Fastv2 turbo and VSA gates when they have a reference solution figured out. The attempt to do that is the unstable version which has multiple issues but looks and sounds perfect at 8 steps.

beta_5 is built with a different normalization technique. It's built off 7 different concept-grouped grafts and consensus merges from over 20 loras, no direct lora merges. Lora credits are on the civit page. It allows for a non-turbo version which is avaliable. The files with TURBO have a hybrid turbo delta fusion baked in, saving 4.2 gigabytes of memory instead of having to load both ref/fl turbos. Other concept loras will also load on top more readily, usually needing lower strength 0.2-0.6.

Sampling and prompt are the key to quality and success. Non-turbo full step audio will always be better than audio on turbo versions.

Some working sampling setups:

er_sde/beta57 4-6 steps

res_multistep/simple 6-8 steps (my go-to for best motion quality)

LCM/simple 6-8 steps (best turbo audio)

LCM/beta 4-6 steps

Euler/simple 4-8 steps

Int8 versions https://huggingface.co/cicalooo/10Eros-Max-h3-int8-convrot

beta_4 and beta_3 are corrupted test versions. beta_5 is the only true functional one so far. No cache or spectrum if doing reference, they cause accuracy loss. Hybrid model is built on the delta1024 h3 merge.

Regarding the grafts:

Due to a ton of confusion I've let Claude compile a full MD on the grafting, including code and methodology as it was mostly used to create code, document, and manage the graft project while I tested, architected, and mixed. I was keeping that to myself since maybe it'd actually be nice to have something to myself but after a handful of ignorant comments I'm open sourcing everything from it except the scripts themselves. You can hand the md back to claude or any competent agent and graft in the same way or have them explain it until you understand. No more dumbassery allowed now.

This is an evolving project subject to future fixes in H3 training. Training is clearly problematic, and my branch will rely on Sulphur H3 tunes. But I didn't want to wait for all that so I took the data from older models, grafting it in where the model needs it, grafting it to attn layers at a low level that doesn't disturb H3's visual or audio output quality. That's essentialy it.

The standard H3 community license applies. Because this release now carries transferred character from LTX 2.3, Wan 2.2, and Krea 2, the community licenses for those source models apply as well to the portions of character that came from each.

10Eros Max H3 Transformer (Pruned Diffusers Format)

MiniMax-H3 transformer, converted from the 10Eros Max checkpoint

Loading

You MUST use AutoModel.from_pretrained with trust_remote_code=True:

from diffusers import AutoModel

transformer = AutoModel.from_pretrained(
    "ibyteohdear/10Eros-Max-Transformer_8stepTURBO_beta6_pruned",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)

Do NOT use `MiniMaxH3Transformer3DModel.from_pretrained` - it will fail because
this is a pruned checkpoint with compressed AdaLN (rank-8).

# this code need the provided differs from this repo, in root of app
import os
import sys 
import tempfile
import re
import random
BASE = tempfile.gettempdir()
IS_HF_SPACES = bool(os.getenv("SPACES_ZERO_GPU"))
HF_TOKEN = os.getenv("HF_TOKEN")
if IS_HF_SPACES:
    import spaces
    clip_dir=BASE
else:
    os.environ["HF_HOME"] = "/mnt/data/huggingface"
    os.environ["HF_HUB_DISABLE_XET"] = "1"
    os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "120"
    clip_dir="/mnt/data/huggingface"

import gradio as gr
import torch
import numpy as np
from PIL import Image
from huggingface_hub import snapshot_download, hf_hub_download

diffusers_path = snapshot_download(
    repo_id="ibyteohdear/10Eros-Max-Transformer_8stepTURBO_beta6_pruned",
    token=HF_TOKEN,
    allow_patterns=["diffusers/*"],
)
sys.path.insert(0, diffusers_path)

from diffusers import (
    TorchAoConfig,
    AutoencoderKLMiniMaxH3,
    AutoencoderKLMiniMaxH3Audio,
)
from diffusers.utils.export_utils import encode_video
from torchao.quantization import Int8WeightOnlyConfig
from transformers import Qwen3VLForConditionalGeneration, AutoTokenizer, Qwen3VLProcessor
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from diffusers.modular_pipelines.minimax_h3.before_denoise import (
    MiniMaxH3PrepareLatentsStep,
    MiniMaxH3PrepareLayoutStep,
    MiniMaxH3SetTimestepsStep,
)
from diffusers.modular_pipelines.minimax_h3.before_encoder import (
    MiniMaxH3SetupStep,
)
from diffusers.modular_pipelines.minimax_h3.denoise import (
    MiniMaxH3DenoiseStep,
)
from diffusers.modular_pipelines.minimax_h3.encoders import (
    MiniMaxH3TextEncoderStep,
)
from diffusers.modular_pipelines.minimax_h3.modular_blocks_minimax_h3 import (
    MiniMaxH3AutoKeyframeVaeEncoderStep,
    MiniMaxH3DecodeStep,
    _generation_outputs,
)
from diffusers.modular_pipelines.modular_pipeline import SequentialPipelineBlocks
from diffusers.modular_pipelines.modular_pipeline_utils import OutputParam
import torch.nn.functional as F
from transformers import TorchAoConfig as TransformersTorchAoConfig
from diffusers import AutoModel

# -----------------------------------------------------------------------------
# 1. Zero-GPU decorator
# -----------------------------------------------------------------------------
def zero_gpu_decorator(duration=1500):
    def decorator(func):
        if IS_HF_SPACES: 
            return spaces.GPU(duration=duration)(func)
        return func
    return decorator

#--------------------------------------------------------------------------
# 2. ClipProj
# -----------------------------------------------------------------------------
PROJ_FILENAME = "mmh3-4b-ClipProj-v3.1.safetensors"
MLP_FILENAME  = "mmh3-4b-ClipProj-v3.1-mlp.safetensors"

CLIPPROJ_REPO = "NicoLab28/ClipProj-MiniMax-H3"

clipproj_dir = os.path.join(clip_dir, "clipproj")
os.makedirs(clipproj_dir, exist_ok=True)

proj_path = hf_hub_download(
    repo_id=CLIPPROJ_REPO,
    filename=PROJ_FILENAME,
    local_dir=clipproj_dir,
)

mlp_path = hf_hub_download(
    repo_id=CLIPPROJ_REPO,
    filename=MLP_FILENAME,
    local_dir=clipproj_dir,
)

proj_weights = load_file(proj_path)
mlp_weights = load_file(mlp_path)

W = proj_weights["W"].to(torch.bfloat16)
mean_in = proj_weights["mean_in"].to(torch.bfloat16)
std_in = proj_weights["std_in"].to(torch.bfloat16)
mean_out = proj_weights["mean_out"].to(torch.bfloat16)
std_out = proj_weights["std_out"].to(torch.bfloat16)

sink_out = proj_weights.get("sink_out")

if sink_out is not None:
    sink_out = sink_out.to(torch.bfloat16)

# -----------------------------------------------------------------------------
# 3. Classes
# -----------------------------------------------------------------------------
class ClipProjModelWrapper(torch.nn.Module):
    def __init__(
        self,
        original_model,
        W,
        mean_in,
        std_in,
        mean_out,
        std_out,
        mlp_weights,
        sink_out=None,
    ):
        super().__init__()

        self.original_model = original_model

        self.register_buffer("W", W)
        self.register_buffer("mean_in", mean_in)
        self.register_buffer("std_in", std_in)
        self.register_buffer("mean_out", mean_out)
        self.register_buffer("std_out", std_out)

        if sink_out is not None:
            self.register_buffer("sink_out", sink_out)
        else:
            self.sink_out = None

        d_in = W.shape[0]
        out_dim = W.shape[1]

        mlp_in = mlp_weights["mlp.0.weight"].shape[1]
        hidden_dim = mlp_weights["mlp.0.weight"].shape[0]
        mlp_out = mlp_weights["mlp.2.weight"].shape[0]

        self.fc1 = torch.nn.Linear(
            d_in,
            hidden_dim,
            bias=True,
            dtype=torch.float16,
        )

        self.fc2 = torch.nn.Linear(
            hidden_dim,
            out_dim,
            bias=True,
            dtype=torch.float16,
        )

        with torch.no_grad():

            self.fc1.weight.copy_(
                mlp_weights["mlp.0.weight"]
            )

            self.fc1.bias.copy_(
                mlp_weights["mlp.0.bias"]
            )

            self.fc2.weight.copy_(
                mlp_weights["mlp.2.weight"]
            )

            self.fc2.bias.copy_(
                mlp_weights["mlp.2.bias"]
            )

        self.fc1.requires_grad_(False)
        self.fc2.requires_grad_(False)

        self.fc1.eval()
        self.fc2.eval()

    def forward(self, *args, **kwargs):
        kwargs["output_hidden_states"] = True
    
        outputs = self.original_model(
            *args,
            **kwargs,
        )
    
        h = outputs.hidden_states[24]
    
        h_norm = (
            h - self.mean_in
        ) / self.std_in
    
        projected = h_norm @ self.W
    
        with torch.no_grad():
            mlp_input = h_norm.to(self.fc1.weight.dtype)
    
            residual = self.fc2(
                F.gelu(
                    self.fc1(mlp_input)
                )
            ).to(projected.dtype)
    
        projected = projected + residual
    
        projected = (
            projected * self.std_out
            + self.mean_out
        )
    
        if self.sink_out is not None:
            projected = projected.clone()
    
            projected[:, 0, :] = self.sink_out.to(
                device=projected.device,
                dtype=projected.dtype,
            )
            
        hidden_states = list(outputs.hidden_states)
    
        while len(hidden_states) <= 50:
            hidden_states.append(projected)
    
        hidden_states[50] = projected
    
        outputs.hidden_states = tuple(hidden_states)
        outputs.last_hidden_state = projected
    
        return outputs
            
class MiniMaxH3ConditionerBlocks(SequentialPipelineBlocks):
    model_name = "minimax-h3"

    block_classes = [
        MiniMaxH3SetupStep,
        MiniMaxH3TextEncoderStep,
    ]

    block_names = [
        "setup",
        "text_encoder",
    ]

    @property
    def outputs(self):
        return [
            OutputParam.template("prompt_embeds"),
            OutputParam(
                "text_token_tags",
                description="The per-row modality tag of every row of prompt_embeds.",
            ),
            OutputParam(
                "height",
                type_hint=int,
                description="Resolved height.",
            ),
            OutputParam(
                "width",
                type_hint=int,
                description="Resolved width.",
            ),
            OutputParam(
                "num_frames",
                type_hint=int,
                description="Resolved number of frames.",
            ),
        ]

class MiniMaxH3GeneratorBlocks(SequentialPipelineBlocks):
    model_name = "minimax-h3"

    block_classes = [
        MiniMaxH3SetupStep,
        MiniMaxH3AutoKeyframeVaeEncoderStep,
        MiniMaxH3PrepareLayoutStep,
        MiniMaxH3PrepareLatentsStep,
        MiniMaxH3SetTimestepsStep,
        MiniMaxH3DenoiseStep,
        MiniMaxH3DecodeStep,
    ]

    block_names = [
        "setup",
        "vae_encoder",
        "prepare_layout",
        "prepare_latents",
        "set_timesteps",
        "denoise",
        "decode",
    ]

    @property
    def outputs(self):
        return _generation_outputs()

# -----------------------------------------------------------------------------
# 4. Load Qwen3-VL-4B
# -----------------------------------------------------------------------------
text_encoder_4b = Qwen3VLForConditionalGeneration.from_pretrained(
    "DreamFast/Qwen3-VL-4b-Heretic",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    quantization_config=TransformersTorchAoConfig(
        Int8WeightOnlyConfig(),
        modules_to_not_convert=[
             "model.visual", 
             "model.language_model.embed_tokens", 
             "model.language_model.norm", 
             "lm_head"
        ],
    ),
)

# -----------------------------------------------------------------------------
# 5. Wrap the model with ClipProj
# -----------------------------------------------------------------------------
original_model = text_encoder_4b.model

model_wrapper = ClipProjModelWrapper(
    original_model=original_model,
    W=W,
    mean_in=mean_in,
    std_in=std_in,
    mean_out=mean_out,
    std_out=std_out,
    mlp_weights=mlp_weights,
    sink_out=sink_out,
)

original_unwrapped_model = text_encoder_4b.model

text_encoder_4b.model = model_wrapper

text_encoder_4b.config.text_config.num_hidden_layers = 64

text_encoder_4b.requires_grad_(False)
text_encoder_4b.eval()

# -----------------------------------------------------------------------------
# 5.5 Prompt Enhancer (Vision-Aware)
# -----------------------------------------------------------------------------
ENHANCE_SYSTEM_PROMPT = """You are an expert video prompt engineer for MiniMax-H3, a state-of-the-art text-to-video AI model.
Your task: Expand short/rough user prompts into rich, detailed, cinematic descriptions optimized for video generation.

Rules:
- Describe visual details: lighting, camera movement, atmosphere, textures, colors
- Describe motion: how subjects move, camera pans/zooms, environmental dynamics
- Keep it flowing and temporal (beginning → middle → end)
- Avoid negations ("no", "without", "don't") - frame positively
- Aim for 40-80 words
- Output ONLY the enhanced prompt, no explanations, no quotes

Examples:
Input: "A cat sleeping"
Output: A fluffy orange cat curled up on a velvet windowsill, soft afternoon sunlight streaming through sheer curtains, dust particles drifting lazily in golden rays, the cat's chest gently rising and falling with peaceful breaths, occasional ear twitches as distant birds chirp outside.

Input: "They all drink a beer and cheer!"
Output: """

def enhance_prompt(raw_prompt: str, e_prompt_temp: float = 0.7, first_frame: Image.Image = None) -> str:
    if not raw_prompt:
        return raw_prompt.strip()
    
    if first_frame is not None:
        if isinstance(first_frame, np.ndarray):
            img_array = first_frame
        else:
            img_array = np.array(first_frame.convert("RGB"))
        
        content = [
            {"type": "image", "image": img_array},
            {"type": "text", "text": f"Based on this first frame image, enhance this video prompt: {raw_prompt}"}
        ]
    else:
        content = raw_prompt
    
    messages = [
        {"role": "system", "content": ENHANCE_SYSTEM_PROMPT},
        {"role": "user", "content": content}
    ]

    text = h3_processor.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )

    if first_frame is not None:
        inputs = h3_processor(
            text=[text],
            images=[img_array],
            return_tensors="pt",
            padding=True
        )
    else:
        inputs = h3_processor(
            text=[text],
            return_tensors="pt",
            padding=True
        )
    
    inputs = {k: v.to("cuda") if isinstance(v, torch.Tensor) else v for k, v in inputs.items()}
    
    text_encoder_4b.model = original_unwrapped_model
    text_encoder_4b.to("cuda")
    
    try:
        with torch.inference_mode():
            outputs = text_encoder_4b.generate(
                **inputs,
                max_new_tokens=150,
                do_sample=True,
                temperature=e_prompt_temp,
                top_p=0.9,
                pad_token_id=h3_processor.tokenizer.pad_token_id,
                eos_token_id=h3_processor.tokenizer.eos_token_id,
            )
        
        input_len = inputs["input_ids"].shape[1]
        generated_ids = outputs[:, input_len:]
        enhanced = h3_processor.batch_decode(
            generated_ids, 
            skip_special_tokens=True
        )[0].strip()
        
        enhanced = re.sub(r'^["\']|["\']$', '', enhanced)
        enhanced = re.sub(r'\n+', ' ', enhanced).strip()
        
        if not enhanced or len(enhanced) < 10:
            return raw_prompt.strip()
        
        return enhanced
    finally:
        text_encoder_4b.model = model_wrapper

# -----------------------------------------------------------------------------
# 6. Build pipeline components
# -----------------------------------------------------------------------------
transformer = AutoModel.from_pretrained(
    "ibyteohdear/10Eros-Max-Transformer_8stepTURBO_beta6_pruned",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    trust_remote_code=True,
    quantization_config=TorchAoConfig(
        Int8WeightOnlyConfig(),
        modules_to_not_convert=[
            "proj_in", "audio_proj_in", "context_embedder", 
            "token_refiner", "norm_out", "proj_out", "audio_proj_out",
        ],
    ),
)

transformer.enable_group_offload(
    onload_device=torch.device("cuda"),
    offload_device=torch.device("cpu"),
    offload_type="block_level",
    num_blocks_per_group=1,
    use_stream=False,
)

vae = AutoencoderKLMiniMaxH3.from_pretrained(
    "MiniMaxAI/MiniMax-H3",
    subfolder="vae",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
)

audio_vae = AutoencoderKLMiniMaxH3Audio.from_pretrained(
    "MiniMaxAI/MiniMax-H3",
    subfolder="audio_vae",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
)

h3_tokenizer = AutoTokenizer.from_pretrained(
    "MiniMaxAI/MiniMax-H3", 
    subfolder="text_encoder", 
    low_cpu_mem_usage=True,
)

h3_processor = Qwen3VLProcessor.from_pretrained(
    "MiniMaxAI/MiniMax-H3",
    subfolder="text_encoder",
    low_cpu_mem_usage=True,
)

conditioner_blocks = MiniMaxH3ConditionerBlocks()

conditioner = conditioner_blocks.init_pipeline(
    "MiniMaxAI/MiniMax-H3"
)

conditioner.update_components(
    text_encoder=text_encoder_4b,
    tokenizer=h3_tokenizer,
    processor=h3_processor,
)

generator_blocks = MiniMaxH3GeneratorBlocks()

generator = generator_blocks.init_pipeline(
    "MiniMaxAI/MiniMax-H3"
)

generator.update_components(
    transformer=transformer,
    vae=vae,
    audio_vae=audio_vae,
)

generator.load_components(dtype=torch.bfloat16)

# -----------------------------------------------------------------------------
# 7. Helpers
# -----------------------------------------------------------------------------
FPS = 24
FRAMES_PER_CHUNK = 17
LATENTS_PER_CHUNK = 5
RESOLUTION_PROFILES = {
    "ultra": { "16:9": (1472, 832), "9:16": (832, 1472), "1:1": (1024, 1024) },
    "high":   { "16:9": (1408, 768), "9:16": (768, 1408), "1:1": (960, 960) },
    "medium": { "16:9": (1344, 704), "9:16": (704, 1344), "1:1": (896, 896) },
    "low":    { "16:9": (1280, 640), "9:16": (640, 1280), "1:1": (832, 832) },
}
MAX_SEED = np.iinfo(np.int32).max

def snap_frames(seconds: float) -> int:
    frames = max(1, round(seconds * FPS))
    while frames % FRAMES_PER_CHUNK != LATENTS_PER_CHUNK:
        frames += 1
    return frames

def auto_crop_image(image_path: str, target_width: int, target_height: int, session_id: str = "default") -> str:
    print(f"[Auto-Crop] Resizing & Cropping to match target resolution: {target_width}x{target_height}")
    
    img = Image.open(image_path).convert("RGB")
    orig_w, orig_h = img.size
    
    target_aspect = target_width / target_height
    orig_aspect = orig_w / orig_h
    
    if abs(orig_aspect - target_aspect) < 0.01:
        print("[Auto-Crop] Image aspect ratio matches settings. Skipping modifications.")
        return image_path

    if orig_aspect > target_aspect:
        new_h = orig_h
        new_w = int(orig_h * target_aspect)
        left = (orig_w - new_w) // 2
        top = 0
        right = left + new_w
        bottom = orig_h
    else:
        new_w = orig_w
        new_h = int(orig_w / target_aspect)
        left = 0
        top = (orig_h - new_h) // 2
        right = orig_w
        bottom = top + new_h
        
    img_cropped = img.crop((left, top, right, bottom))
    print(f"[Auto-Crop] Centered crop applied from {orig_w}x{orig_h} to bounds: left={left}, top={top}, right={right}, bottom={bottom}")
    
    cropped_out_path = os.path.join(BASE, f"cropped_{session_id}.png")

    if os.path.exists(cropped_out_path):
        try:
            os.remove(cropped_out_path)
        except Exception:
            pass

    img_cropped.save(cropped_out_path)
    return cropped_out_path
    
# -----------------------------------------------------------------------------
# 8. Generation
# -----------------------------------------------------------------------------
@zero_gpu_decorator(duration=1500)
def generate_video(prompt, e_prompt_temp, image_pil, end_image_pil, quality, ratio, duration, seed, randomize_seed, use_enhancer, progress=gr.Progress(track_tqdm=True)):
    if image_pil is None:
        raise ValueError("A starting first frame image is required.")

    final_prompt = prompt
    if use_enhancer and prompt:
        final_prompt = enhance_prompt(prompt, e_prompt_temp, first_frame=image_pil)

    width, height = RESOLUTION_PROFILES[quality][ratio]
    num_frames = snap_frames(duration)
    current_seed = random.randint(0, MAX_SEED) if randomize_seed else int(seed)
    session_id = str(current_seed)
    
    if isinstance(image_pil, Image.Image):
        temp_first_path = os.path.join(BASE, f"temp_first_{session_id}.png")
        image_pil.save(temp_first_path)
        cropped_first_path = auto_crop_image(temp_first_path, width, height, session_id=f"first_{session_id}")
        image_pil = Image.open(cropped_first_path).convert("RGB")

    end_image = None
    if end_image_pil is not None and isinstance(end_image_pil, Image.Image):
        temp_end_path = os.path.join(BASE, f"temp_end_{session_id}.png")
        end_image_pil.save(temp_end_path)
        cropped_end_path = auto_crop_image(temp_end_path, width, height, session_id=f"end_{session_id}")
        end_image = Image.open(cropped_end_path).convert("RGB")

    text_encoder_4b.to("cuda")
    
    with torch.inference_mode():
        cond_result = conditioner(
            prompt=final_prompt,
            image=image_pil,
            last_image=end_image,
            height=height,
            width=width,
            num_frames=num_frames,
        )
    
        prompt_embeds = cond_result.prompt_embeds
        text_token_tags = cond_result.text_token_tags
    
    text_encoder_4b.to("cpu")
    
    del cond_result
    
    torch.cuda.empty_cache()
    torch.cuda.ipc_collect()
    
    vae.to("cuda")
    audio_vae.to("cuda")

    gen_kwargs = {
        "prompt_embeds": prompt_embeds, 
        "text_token_tags": text_token_tags, 
        "image": image_pil,
        "height": height,
        "width": width,
        "num_frames": num_frames,
        "generator": torch.Generator().manual_seed(int(current_seed)),
        "output": ["videos", "audio", "sampling_rate"],
        "num_inference_steps": 8 + 1,
    }
    if end_image is not None:
        gen_kwargs["last_image"] = end_image

    gen_result = generator(**gen_kwargs)

    out_path = os.path.join(BASE, f"minimax_h3_gradio_output_{session_id}.mp4")

    with torch.inference_mode():
        encode_video(
            gen_result["videos"][0],
            FPS,
            out_path,
            gen_result["audio"][0],
            gen_result["sampling_rate"]
        )
    
    return out_path, final_prompt

# -----------------------------------------------------------------------------
# 8.5 Standalone prompt enhancement
# -----------------------------------------------------------------------------
@zero_gpu_decorator(duration=120)
def enhance_prompt_standalone(raw_prompt, e_prompt_temp, image_pil):
    if not raw_prompt:
        return raw_prompt  
    
    enhanced = enhance_prompt(raw_prompt, e_prompt_temp, first_frame=image_pil)
    return enhanced

# -----------------------------------------------------------------------------
# 9. UI
# -----------------------------------------------------------------------------
with gr.Blocks(title="MiniMax-H3 FL2VA") as demo:
    gr.Markdown("# MiniMax-H3 FL2VA Generator")
    gr.Markdown("Using **Qwen3-VL-4B + ClipProj v3.1 + 8 step Lightning**")

    with gr.Row():
        with gr.Column():
            
            with gr.Row():
                prompt_input = gr.Textbox(
                    label="Prompt",
                    value="They all drink a beer and cheer!",
                    lines=2,
                )
            
            with gr.Row():  
                e_prompt_temp = gr.Slider(minimum=0.1, maximum=1.0, value=0.7, step=0.1, label="Enhance prompt temperature")
                
            with gr.Row():    
                enhance_btn = gr.Button("✨ Enhance Prompt", variant="secondary")    
            
            with gr.Row():
                image_input = gr.Image(
                    label="First Frame Image (Required)",
                    type="pil",
                )
                end_image_input = gr.Image(
                    label="Last Frame Image (Optional)",
                    type="pil",
                )
            
            with gr.Row():
                quality_tier = gr.Dropdown(
                    choices=["ultra", "high", "medium", "low"],
                    value="high",
                    label="Quality / Canvas Tier"
                )
                aspect_ratio = gr.Dropdown(
                    choices=["16:9", "9:16", "1:1"],
                    value="16:9",
                    label="Aspect Ratio"
                )

            with gr.Row():
                duration_slider = gr.Slider(
                    minimum=5.0,
                    maximum=10.0,
                    step=0.5,
                    value=5.0,
                    label="Duration (seconds)",
                )
            
            with gr.Row():
                seed_input = gr.Number(
                    label="Seed",
                    value=42,
                    precision=0,
                )
            
            with gr.Row():
                randomize_seed = gr.Checkbox(value=True, label="Randomize Seed")

            with gr.Row():
                enhance_toggle = gr.Checkbox(
                    label="Auto-Enhance on Generate",
                    value=False,
                    info="Automatically enhance prompt when clicking Generate"
                )

            with gr.Row():
                generate_btn = gr.Button("Generate", variant="primary")
    
        with gr.Column():
            
            with gr.Row():
                video_output = gr.Video(label="Generated Video")

    gr.Examples(
        examples=[
            "They all drink a beer and cheer!",
        ],
        inputs=[prompt_input],
        example_labels=[
            "They all drink a beer and cheer!",
        ],
        label="💡 Try these:"
    )

    enhance_btn.click(
        fn=enhance_prompt_standalone,
        inputs=[prompt_input, e_prompt_temp, image_input],
        outputs=[prompt_input],
    )

    generate_btn.click(
        fn=generate_video,
        inputs=[
            prompt_input,
            e_prompt_temp,
            image_input,
            end_image_input, 
            quality_tier, 
            aspect_ratio, 
            duration_slider,
            seed_input,
            randomize_seed,
            enhance_toggle
        ],
        outputs=[video_output, prompt_input],
    )

if __name__ == "__main__":
    if IS_HF_SPACES:
        demo.queue().launch(debug=False, share=False)
    else:
        demo.queue().launch(debug=True, share=True)

from huggingface_hub import HfApi, create_repo, hf_hub_download, snapshot_download
from diffusers import AutoModel
from safetensors.torch import load_file, save_file
import torch
import os
import json
import shutil

HF_TOKEN = os.environ["HF_TOKEN"]
LOCAL_DIR = "./10eros_h3_transformer_pruned"
REPO_ID = "ibyteohdear/10Eros-Max-Transformer_8stepTURBO_beta6_pruned"

NUM_ATTENTION_HEADS = 56
ATTENTION_HEAD_DIM = 128
INNER_DIM = NUM_ATTENTION_HEADS * ATTENTION_HEAD_DIM  # 7168

# Download checkpoint
checkpoint_path = hf_hub_download(
    repo_id="TenStrip/10Eros-Max",
    filename="10Eros_Max_h3_8stepTURBO_beta6_pruned.safetensors",
    token=HF_TOKEN,
)

# Load base pruned transformer
print("Loading base transformer...")
transformer = AutoModel.from_pretrained(
    "multimodalart/MiniMax-H3-Pruned",
    subfolder="transformer",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)

expected_keys = set(transformer.state_dict().keys())
print(f"Expected keys: {len(expected_keys)}")

# Load checkpoint
print("Loading checkpoint...")
src = load_file(checkpoint_path, device="cpu")
print(f"Source keys: {len(src)}")

# Convert keys
print("Converting...")
dst = {}

# --- CRITICAL FIX: AdaLN basis transpose validation ---
adaln_basis = src.get("adaln_basis")
if adaln_basis is not None:
    if adaln_basis.shape[0] == 2688 and adaln_basis.shape[1] == 8:
        print("Transposing adaln_basis from [2688, 8] -> [8, 2688]")
        adaln_basis = adaln_basis.T.contiguous()
    elif adaln_basis.shape[0] == 8 and adaln_basis.shape[1] == 2688:
        adaln_basis = adaln_basis.contiguous()
    else:
        raise ValueError(f"Unexpected adaln_basis shape: {adaln_basis.shape}")

adaln_mean = src.get("adaln_mean")

if "adaln_t_table" in src:
    dst["time_embedder.table"] = src["adaln_t_table"].contiguous()
if adaln_basis is not None:
    dst["adaln_basis"] = adaln_basis
if adaln_mean is not None:
    dst["adaln_mean"] = adaln_mean.contiguous()

for name, tensor in src.items():
    new_name = None
    
    # Skip already-handled AdaLN keys
    if name in ("adaln_t_table", "adaln_basis", "adaln_mean"):
        continue
    if name.startswith("time_embedder."):
        continue
    
    if name == "video_patch_proj.weight":
        new_name = "proj_in.weight"
    elif name == "video_patch_proj.bias":
        new_name = "proj_in.bias"
    elif name == "audio_patch_proj.weight":
        new_name = "audio_proj_in.weight"
    elif name == "audio_patch_proj.bias":
        new_name = "audio_proj_in.bias"
    elif name == "condition_proj.weight":
        new_name = "context_embedder.weight"
    elif name == "condition_proj.bias":
        new_name = "context_embedder.bias"
    
    # FIX: Skip rope.inv_freq instead of mapping to same name
    elif name == "rope.inv_freq":
        print(f"  SKIP  {name} (reconstructed from config)")
        continue
    
    elif name == "final_layer.adaln_proj.linear.weight":
        dst["norm_out.linear.weight"] = tensor.contiguous()
        bias = src.get("final_layer.adaln_proj.linear.bias")
        if bias is not None:
            # FIX: folded_bias must be float32 for numerical stability
            dst["norm_out.folded_bias"] = bias.to(torch.float32).contiguous()
        continue
    elif name == "final_layer.adaln_proj.linear.bias":
        continue
    elif name == "final_layer.norm.weight":
        new_name = "norm_out.norm.weight"
    elif name == "final_layer.video_out.weight":
        new_name = "proj_out.weight"
    elif name == "final_layer.video_out.bias":
        new_name = "proj_out.bias"
    elif name == "final_layer.audio_out.weight":
        new_name = "audio_proj_out.weight"
    elif name == "final_layer.audio_out.bias":
        new_name = "audio_proj_out.bias"
    elif name == "token_refiner.final_norm.weight":
        new_name = "token_refiner.final_norm.weight"
    elif name.startswith("token_refiner.blocks."):
        base_name = name.replace("token_refiner.blocks.", "token_refiner.refiner_blocks.", 1)
        
        if ".attn.qkv_proj.weight" in base_name:
            prefix = base_name.replace(".attn.qkv_proj.weight", ".attn.")
            q, k, v = tensor.split(INNER_DIM, dim=0)
            dst[f"{prefix}to_q.weight"] = q.contiguous()
            dst[f"{prefix}to_k.weight"] = k.contiguous()
            dst[f"{prefix}to_v.weight"] = v.contiguous()
            continue
        elif ".attn.qkv_proj.bias" in base_name:
            prefix = base_name.replace(".attn.qkv_proj.bias", ".attn.")
            q_b, k_b, v_b = tensor.split(INNER_DIM, dim=0)
            dst[f"{prefix}to_q.bias"] = q_b.contiguous()
            dst[f"{prefix}to_k.bias"] = k_b.contiguous()
            dst[f"{prefix}to_v.bias"] = v_b.contiguous()
            continue
        elif ".attn.out_proj.weight" in base_name:
            new_name = base_name.replace(".attn.out_proj.weight", ".attn.to_out.0.weight")
        elif ".attn.out_proj.bias" in base_name:
            new_name = base_name.replace(".attn.out_proj.bias", ".attn.to_out.0.bias")
        elif ".attn.q_norm.weight" in base_name:
            new_name = base_name.replace(".attn.q_norm.weight", ".attn.norm_q.weight")
        elif ".attn.k_norm.weight" in base_name:
            new_name = base_name.replace(".attn.k_norm.weight", ".attn.norm_k.weight")
        elif ".mlp.fc1.weight" in base_name:
            gate, value = tensor.chunk(2, dim=0)
            new_name = base_name.replace(".mlp.fc1.weight", ".ff.net.0.proj.weight")
            dst[new_name] = torch.cat([value, gate], dim=0).contiguous()
            continue
        elif ".mlp.fc1.bias" in base_name:
            gate_b, value_b = tensor.chunk(2, dim=0)
            new_name = base_name.replace(".mlp.fc1.bias", ".ff.net.0.proj.bias")
            dst[new_name] = torch.cat([value_b, gate_b], dim=0).contiguous()
            continue
        elif ".mlp.fc2.weight" in base_name:
            new_name = base_name.replace(".mlp.fc2.weight", ".ff.net.2.weight")
        elif ".mlp.fc2.bias" in base_name:
            new_name = base_name.replace(".mlp.fc2.bias", ".ff.net.2.bias")
        else:
            new_name = base_name
                
    elif name.startswith("blocks."):
        base_name = name.replace("blocks.", "transformer_blocks.", 1)
        
        if ".attn.qkv_proj.weight" in base_name:
            prefix = base_name.replace(".attn.qkv_proj.weight", ".attn.")
            q, k, v = tensor.split(INNER_DIM, dim=0)
            dst[f"{prefix}to_q.weight"] = q.contiguous()
            dst[f"{prefix}to_k.weight"] = k.contiguous()
            dst[f"{prefix}to_v.weight"] = v.contiguous()
            continue
        elif ".attn.qkv_proj.bias" in base_name:
            prefix = base_name.replace(".attn.qkv_proj.bias", ".attn.")
            q_b, k_b, v_b = tensor.split(INNER_DIM, dim=0)
            dst[f"{prefix}to_q.bias"] = q_b.contiguous()
            dst[f"{prefix}to_k.bias"] = k_b.contiguous()
            dst[f"{prefix}to_v.bias"] = v_b.contiguous()
            continue
        elif ".attn.out_proj.weight" in base_name:
            new_name = base_name.replace(".attn.out_proj.weight", ".attn.to_out.0.weight")
        elif ".attn.out_proj.bias" in base_name:
            new_name = base_name.replace(".attn.out_proj.bias", ".attn.to_out.0.bias")
        elif ".attn.q_norm.weight" in base_name:
            new_name = base_name.replace(".attn.q_norm.weight", ".attn.norm_q.weight")
        elif ".attn.k_norm.weight" in base_name:
            new_name = base_name.replace(".attn.k_norm.weight", ".attn.norm_k.weight")
        elif ".mlp.fc1.weight" in base_name:
            gate, value = tensor.chunk(2, dim=0)
            new_name = base_name.replace(".mlp.fc1.weight", ".ff.net.0.proj.weight")
            dst[new_name] = torch.cat([value, gate], dim=0).contiguous()
            continue
        elif ".mlp.fc1.bias" in base_name:
            gate_b, value_b = tensor.chunk(2, dim=0)
            new_name = base_name.replace(".mlp.fc1.bias", ".ff.net.0.proj.bias")
            dst[new_name] = torch.cat([value_b, gate_b], dim=0).contiguous()
            continue
        elif ".mlp.fc2.weight" in base_name:
            new_name = base_name.replace(".mlp.fc2.weight", ".ff.net.2.weight")
        elif ".mlp.fc2.bias" in base_name:
            new_name = base_name.replace(".mlp.fc2.bias", ".ff.net.2.bias")
        elif ".adaln_proj.linear.weight" in base_name:
            dst[base_name] = tensor.contiguous()
            bias_key = name.replace(".weight", ".bias")
            if bias_key in src:
                folded_bias_key = base_name.replace(".adaln_proj.linear.weight", ".adaln_proj.folded_bias")
                # FIX: folded_bias must be float32
                dst[folded_bias_key] = src[bias_key].to(torch.float32).contiguous()
            continue
        elif ".adaln_proj.linear.bias" in base_name:
            continue
        else:
            new_name = base_name
    else:
        print(f"  SKIP: {name}")
        continue
        
    if new_name is not None:
        dst[new_name] = tensor.contiguous()

# Validate
dst_keys = set(dst.keys())
matched = expected_keys & dst_keys
missing = expected_keys - dst_keys
unexpected = dst_keys - expected_keys

print(f"\nMatched: {len(matched)}/{len(expected_keys)}")
print(f"Missing: {len(missing)}")
print(f"Unexpected: {len(unexpected)}")

# Load into model
print("\nLoading into model...")
missing_loaded, unexpected_loaded = transformer.load_state_dict(dst, strict=False)
print(f"After load: missing={len(missing_loaded)}, unexpected={len(unexpected_loaded)}")

# Save
print(f"\nSaving to {LOCAL_DIR}...")
os.makedirs(LOCAL_DIR, exist_ok=True)
transformer.save_pretrained(LOCAL_DIR, safe_serialization=True)

# Update config
config_path = os.path.join(LOCAL_DIR, "config.json")
with open(config_path, "r") as f:
    config = json.load(f)

# Only set _class_name (this is always needed)
config["_class_name"] = "MiniMaxH3PrunedTransformer3DModel"

# auto_map is already in the base model config — don't modify it
# The warning comes from the modeling file, not from our code

with open(config_path, "w") as f:
    json.dump(config, f, indent=2)
print("Updated config.json (_class_name only)")

# Copy modeling file
base_repo_path = snapshot_download(
    "multimodalart/MiniMax-H3-Pruned",
    allow_patterns=["transformer/modeling_minimax_h3_pruned.py"],
    local_dir="./_temp_base_repo",
)
src_modeling = os.path.join(base_repo_path, "transformer", "modeling_minimax_h3_pruned.py")
if os.path.exists(src_modeling):
    shutil.copy2(src_modeling, os.path.join(LOCAL_DIR, "modeling_minimax_h3_pruned.py"))
if os.path.exists("./_temp_base_repo"):
    shutil.rmtree("./_temp_base_repo")

# Upload
print(f"\nUploading to {REPO_ID}...")
api = HfApi(token=HF_TOKEN)
create_repo(REPO_ID, exist_ok=True, private=True)
api.upload_folder(folder_path=LOCAL_DIR, repo_id=REPO_ID, repo_type="model")

print(f"\nDone! Load with:")
print(f'  AutoModel.from_pretrained("{REPO_ID}", trust_remote_code=True)')