ive just updated and fixed some issues

Now that minimax has clarified their license; This is, and was previously, meant to be an NSFW capable model that still has base model capability and functions. "Eros" is the greek god of sex, not sure why that didn't catch on or why there is confusion about what the point of this was. It started with literally extracting NSFW Wan2.2 model and Krea2 data to make those changes with grafts.

beta_5 is built with a different normalization technique. It's built off 7 different concept-grouped grafts and consensus merges from over 20 loras, no direct lora merges. Lora credits are on the civit page. It allows for a non-turbo version which is avaliable. The files with TURBO have a hybrid turbo delta fusion baked in, saving 4.2 gigabytes of memory instead of having to load both ref/fl turbos. Other concept loras will also load on top more readily, usually needing lower strength 0.2-0.6.

Sampling and prompt are the key to quality and success. Non-turbo full step audio will always be better than audio on turbo versions.

Some working sampling setups:

er_sde/beta57 4-6 steps

res_multistep/simple 6-8 steps (my go-to for best motion quality)

LCM/simple 6-8 steps (best turbo audio)

LCM/beta 4-6 steps

Euler/simple 4-8 steps

Int8 versions https://huggingface.co/cicalooo/10Eros-Max-h3-int8-convrot

beta_4 and beta_3 are corrupted test versions. beta_5 is the only true functional one so far. No cache or spectrum if doing reference, they cause accuracy loss. Hybrid model is built on the delta1024 h3 merge.

Regarding the grafts:

Due to a ton of confusion I've let Claude compile a full MD on the grafting, including code and methodology as it was mostly used to create code, document, and manage the graft project while I tested, architected, and mixed. I was keeping that to myself since maybe it'd actually be nice to have something to myself but after a handful of ignorant comments I'm open sourcing everything from it except the scripts themselves. You can hand the md back to claude or any competent agent and graft in the same way or have them explain it until you understand. No more dumbassery allowed now.

This is an evolving project subject to future fixes in H3 training. Training is clearly problematic, and my branch will rely on Sulphur H3 tunes. But I didn't want to wait for all that so I took the data from older models, grafting it in where the model needs it, grafting it to attn layers at a low level that doesn't disturb H3's visual or audio output quality. That's essentialy it.

The standard H3 community license applies. Because this release now carries transferred character from LTX 2.3, Wan 2.2, and Krea 2, the community licenses for those source models apply as well to the portions of character that came from each.

10Eros Max H3 Transformer (Diffusers Format)

MiniMax-H3 transformer, converted from the 10Eros Max checkpoint

Loading

You MUST use AutoModel.from_pretrained with trust_remote_code=True:

from diffusers import AutoModel

transformer = AutoModel.from_pretrained(
    "ibyteohdear/10Eros-Max-Transformer_TURBO-hybrid_beta5",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)

Do NOT use `MiniMaxH3Transformer3DModel.from_pretrained` - it will fail because
this is a pruned checkpoint with compressed AdaLN (rank-8).

# this code need the provided differs from this repo, in root of app
import os
import sys 
import tempfile
import re
import random
BASE = tempfile.gettempdir()
IS_HF_SPACES = bool(os.getenv("SPACES_ZERO_GPU"))
HF_TOKEN = os.getenv("HF_TOKEN")
if IS_HF_SPACES:
    import spaces
    clip_dir=BASE
else:
    os.environ["HF_HOME"] = "/mnt/data/huggingface"
    os.environ["HF_HUB_DISABLE_XET"] = "1"
    os.environ["HF_HUB_DOWNLOAD_TIMEOUT"] = "120"
    clip_dir="/mnt/data/huggingface"

import gradio as gr
import torch
import numpy as np
from PIL import Image
from huggingface_hub import snapshot_download, hf_hub_download

diffusers_path = snapshot_download(
    repo_id="ibyteohdear/10Eros-Max-Transformer_TURBO-hybrid_beta5",
    token=HF_TOKEN,
    allow_patterns=["diffusers/*"],
)
sys.path.insert(0, diffusers_path)

from diffusers import (
    TorchAoConfig,
    AutoencoderKLMiniMaxH3,
    AutoencoderKLMiniMaxH3Audio,
)
from diffusers.utils.export_utils import encode_video
from torchao.quantization import Int8WeightOnlyConfig
from transformers import Qwen3VLForConditionalGeneration, AutoTokenizer, Qwen3VLProcessor
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from diffusers.modular_pipelines.minimax_h3.before_denoise import (
    MiniMaxH3PrepareLatentsStep,
    MiniMaxH3PrepareLayoutStep,
    MiniMaxH3SetTimestepsStep,
)
from diffusers.modular_pipelines.minimax_h3.before_encoder import (
    MiniMaxH3SetupStep,
)
from diffusers.modular_pipelines.minimax_h3.denoise import (
    MiniMaxH3DenoiseStep,
)
from diffusers.modular_pipelines.minimax_h3.encoders import (
    MiniMaxH3TextEncoderStep,
)
from diffusers.modular_pipelines.minimax_h3.modular_blocks_minimax_h3 import (
    MiniMaxH3AutoKeyframeVaeEncoderStep,
    MiniMaxH3DecodeStep,
    _generation_outputs,
)
from diffusers.modular_pipelines.modular_pipeline import SequentialPipelineBlocks
from diffusers.modular_pipelines.modular_pipeline_utils import OutputParam
import torch.nn.functional as F
from transformers import TorchAoConfig as TransformersTorchAoConfig
from diffusers import AutoModel

# -----------------------------------------------------------------------------
# 1. Zero-GPU decorator
# -----------------------------------------------------------------------------
def zero_gpu_decorator(duration=1500):
    def decorator(func):
        if IS_HF_SPACES: 
            return spaces.GPU(duration=duration)(func)
        return func
    return decorator

#--------------------------------------------------------------------------
# 2. ClipProj
# -----------------------------------------------------------------------------
PROJ_FILENAME = "mmh3-4b-ClipProj-v3.1.safetensors"
MLP_FILENAME  = "mmh3-4b-ClipProj-v3.1-mlp.safetensors"

CLIPPROJ_REPO = "NicoLab28/ClipProj-MiniMax-H3"

clipproj_dir = os.path.join(clip_dir, "clipproj")
os.makedirs(clipproj_dir, exist_ok=True)

proj_path = hf_hub_download(
    repo_id=CLIPPROJ_REPO,
    filename=PROJ_FILENAME,
    local_dir=clipproj_dir,
)

mlp_path = hf_hub_download(
    repo_id=CLIPPROJ_REPO,
    filename=MLP_FILENAME,
    local_dir=clipproj_dir,
)

proj_weights = load_file(proj_path)
mlp_weights = load_file(mlp_path)

W = proj_weights["W"].to(torch.bfloat16)
mean_in = proj_weights["mean_in"].to(torch.bfloat16)
std_in = proj_weights["std_in"].to(torch.bfloat16)
mean_out = proj_weights["mean_out"].to(torch.bfloat16)
std_out = proj_weights["std_out"].to(torch.bfloat16)

sink_out = proj_weights.get("sink_out")

if sink_out is not None:
    sink_out = sink_out.to(torch.bfloat16)

# -----------------------------------------------------------------------------
# 3. Classes
# -----------------------------------------------------------------------------
class ClipProjModelWrapper(torch.nn.Module):
    def __init__(
        self,
        original_model,
        W,
        mean_in,
        std_in,
        mean_out,
        std_out,
        mlp_weights,
        sink_out=None,
    ):
        super().__init__()

        self.original_model = original_model

        self.register_buffer("W", W)
        self.register_buffer("mean_in", mean_in)
        self.register_buffer("std_in", std_in)
        self.register_buffer("mean_out", mean_out)
        self.register_buffer("std_out", std_out)

        if sink_out is not None:
            self.register_buffer("sink_out", sink_out)
        else:
            self.sink_out = None

        d_in = W.shape[0]
        out_dim = W.shape[1]

        mlp_in = mlp_weights["mlp.0.weight"].shape[1]
        hidden_dim = mlp_weights["mlp.0.weight"].shape[0]
        mlp_out = mlp_weights["mlp.2.weight"].shape[0]

        self.fc1 = torch.nn.Linear(
            d_in,
            hidden_dim,
            bias=True,
            dtype=torch.float16,
        )

        self.fc2 = torch.nn.Linear(
            hidden_dim,
            out_dim,
            bias=True,
            dtype=torch.float16,
        )

        with torch.no_grad():

            self.fc1.weight.copy_(
                mlp_weights["mlp.0.weight"]
            )

            self.fc1.bias.copy_(
                mlp_weights["mlp.0.bias"]
            )

            self.fc2.weight.copy_(
                mlp_weights["mlp.2.weight"]
            )

            self.fc2.bias.copy_(
                mlp_weights["mlp.2.bias"]
            )

        self.fc1.requires_grad_(False)
        self.fc2.requires_grad_(False)

        self.fc1.eval()
        self.fc2.eval()

    def forward(self, *args, **kwargs):
        kwargs["output_hidden_states"] = True
    
        outputs = self.original_model(
            *args,
            **kwargs,
        )
    
        h = outputs.hidden_states[24]
    
        h_norm = (
            h - self.mean_in
        ) / self.std_in
    
        projected = h_norm @ self.W
    
        with torch.no_grad():
            mlp_input = h_norm.to(self.fc1.weight.dtype)
    
            residual = self.fc2(
                F.gelu(
                    self.fc1(mlp_input)
                )
            ).to(projected.dtype)
    
        projected = projected + residual
    
        projected = (
            projected * self.std_out
            + self.mean_out
        )
    
        if self.sink_out is not None:
            projected = projected.clone()
    
            projected[:, 0, :] = self.sink_out.to(
                device=projected.device,
                dtype=projected.dtype,
            )
            
        hidden_states = list(outputs.hidden_states)
    
        while len(hidden_states) <= 50:
            hidden_states.append(projected)
    
        hidden_states[50] = projected
    
        outputs.hidden_states = tuple(hidden_states)
        outputs.last_hidden_state = projected
    
        return outputs
            
class MiniMaxH3ConditionerBlocks(SequentialPipelineBlocks):
    model_name = "minimax-h3"

    block_classes = [
        MiniMaxH3SetupStep,
        MiniMaxH3TextEncoderStep,
    ]

    block_names = [
        "setup",
        "text_encoder",
    ]

    @property
    def outputs(self):
        return [
            OutputParam.template("prompt_embeds"),
            OutputParam(
                "text_token_tags",
                description="The per-row modality tag of every row of prompt_embeds.",
            ),
            OutputParam(
                "height",
                type_hint=int,
                description="Resolved height.",
            ),
            OutputParam(
                "width",
                type_hint=int,
                description="Resolved width.",
            ),
            OutputParam(
                "num_frames",
                type_hint=int,
                description="Resolved number of frames.",
            ),
        ]

class MiniMaxH3GeneratorBlocks(SequentialPipelineBlocks):
    model_name = "minimax-h3"

    block_classes = [
        MiniMaxH3SetupStep,
        MiniMaxH3AutoKeyframeVaeEncoderStep,
        MiniMaxH3PrepareLayoutStep,
        MiniMaxH3PrepareLatentsStep,
        MiniMaxH3SetTimestepsStep,
        MiniMaxH3DenoiseStep,
        MiniMaxH3DecodeStep,
    ]

    block_names = [
        "setup",
        "vae_encoder",
        "prepare_layout",
        "prepare_latents",
        "set_timesteps",
        "denoise",
        "decode",
    ]

    @property
    def outputs(self):
        return _generation_outputs()

# -----------------------------------------------------------------------------
# 4. Load Qwen3-VL-4B
# -----------------------------------------------------------------------------
text_encoder_4b = Qwen3VLForConditionalGeneration.from_pretrained(
    "DreamFast/Qwen3-VL-4b-Heretic",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    quantization_config=TransformersTorchAoConfig(
        Int8WeightOnlyConfig(),
        modules_to_not_convert=[
             "model.visual", 
             "model.language_model.embed_tokens", 
             "model.language_model.norm", 
             "lm_head"
        ],
    ),
)

# -----------------------------------------------------------------------------
# 5. Wrap the model with ClipProj
# -----------------------------------------------------------------------------
original_model = text_encoder_4b.model

model_wrapper = ClipProjModelWrapper(
    original_model=original_model,
    W=W,
    mean_in=mean_in,
    std_in=std_in,
    mean_out=mean_out,
    std_out=std_out,
    mlp_weights=mlp_weights,
    sink_out=sink_out,
)

original_unwrapped_model = text_encoder_4b.model

text_encoder_4b.model = model_wrapper

text_encoder_4b.config.text_config.num_hidden_layers = 64

text_encoder_4b.requires_grad_(False)
text_encoder_4b.eval()

# -----------------------------------------------------------------------------
# 5.5 Prompt Enhancer (Vision-Aware)
# -----------------------------------------------------------------------------
ENHANCE_SYSTEM_PROMPT = """You are an expert video prompt engineer for MiniMax-H3, a state-of-the-art text-to-video AI model.
Your task: Expand short/rough user prompts into rich, detailed, cinematic descriptions optimized for video generation.

Rules:
- Describe visual details: lighting, camera movement, atmosphere, textures, colors
- Describe motion: how subjects move, camera pans/zooms, environmental dynamics
- Keep it flowing and temporal (beginning → middle → end)
- Avoid negations ("no", "without", "don't") - frame positively
- Aim for 40-80 words
- Output ONLY the enhanced prompt, no explanations, no quotes

Examples:
Input: "A cat sleeping"
Output: A fluffy orange cat curled up on a velvet windowsill, soft afternoon sunlight streaming through sheer curtains, dust particles drifting lazily in golden rays, the cat's chest gently rising and falling with peaceful breaths, occasional ear twitches as distant birds chirp outside.

Input: "They all drink a beer and cheer!"
Output: """

def enhance_prompt(raw_prompt: str, e_prompt_temp: float = 0.7, first_frame: Image.Image = None) -> str:
    if not raw_prompt:
        return raw_prompt.strip()
    
    if first_frame is not None:
        if isinstance(first_frame, np.ndarray):
            img_array = first_frame
        else:
            img_array = np.array(first_frame.convert("RGB"))
        
        content = [
            {"type": "image", "image": img_array},
            {"type": "text", "text": f"Based on this first frame image, enhance this video prompt: {raw_prompt}"}
        ]
    else:
        content = raw_prompt
    
    messages = [
        {"role": "system", "content": ENHANCE_SYSTEM_PROMPT},
        {"role": "user", "content": content}
    ]

    text = h3_processor.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )

    if first_frame is not None:
        inputs = h3_processor(
            text=[text],
            images=[img_array],
            return_tensors="pt",
            padding=True
        )
    else:
        inputs = h3_processor(
            text=[text],
            return_tensors="pt",
            padding=True
        )
    
    inputs = {k: v.to("cuda") if isinstance(v, torch.Tensor) else v for k, v in inputs.items()}
    
    text_encoder_4b.model = original_unwrapped_model
    text_encoder_4b.to("cuda")
    
    try:
        with torch.inference_mode():
            outputs = text_encoder_4b.generate(
                **inputs,
                max_new_tokens=150,
                do_sample=True,
                temperature=e_prompt_temp,
                top_p=0.9,
                pad_token_id=h3_processor.tokenizer.pad_token_id,
                eos_token_id=h3_processor.tokenizer.eos_token_id,
            )
        
        input_len = inputs["input_ids"].shape[1]
        generated_ids = outputs[:, input_len:]
        enhanced = h3_processor.batch_decode(
            generated_ids, 
            skip_special_tokens=True
        )[0].strip()
        
        enhanced = re.sub(r'^["\']|["\']$', '', enhanced)
        enhanced = re.sub(r'\n+', ' ', enhanced).strip()
        
        if not enhanced or len(enhanced) < 10:
            return raw_prompt.strip()
        
        return enhanced
    finally:
        text_encoder_4b.model = model_wrapper

# -----------------------------------------------------------------------------
# 6. Build pipeline components
# -----------------------------------------------------------------------------
transformer = AutoModel.from_pretrained(
    "ibyteohdear/10Eros-Max-Transformer_TURBO-hybrid_beta5",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    trust_remote_code=True,
    quantization_config=TorchAoConfig(
        Int8WeightOnlyConfig(),
        modules_to_not_convert=[
            "proj_in", "audio_proj_in", "context_embedder", 
            "token_refiner", "norm_out", "proj_out", "audio_proj_out",
        ],
    ),
)

transformer.enable_group_offload(
    onload_device=torch.device("cuda"),
    offload_device=torch.device("cpu"),
    offload_type="block_level",
    num_blocks_per_group=1,
    use_stream=False,
)

vae = AutoencoderKLMiniMaxH3.from_pretrained(
    "MiniMaxAI/MiniMax-H3",
    subfolder="vae",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
)

audio_vae = AutoencoderKLMiniMaxH3Audio.from_pretrained(
    "MiniMaxAI/MiniMax-H3",
    subfolder="audio_vae",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
)

h3_tokenizer = AutoTokenizer.from_pretrained(
    "MiniMaxAI/MiniMax-H3", 
    subfolder="text_encoder", 
    low_cpu_mem_usage=True,
)

h3_processor = Qwen3VLProcessor.from_pretrained(
    "MiniMaxAI/MiniMax-H3",
    subfolder="text_encoder",
    low_cpu_mem_usage=True,
)

conditioner_blocks = MiniMaxH3ConditionerBlocks()

conditioner = conditioner_blocks.init_pipeline(
    "MiniMaxAI/MiniMax-H3"
)

conditioner.update_components(
    text_encoder=text_encoder_4b,
    tokenizer=h3_tokenizer,
    processor=h3_processor,
)

generator_blocks = MiniMaxH3GeneratorBlocks()

generator = generator_blocks.init_pipeline(
    "MiniMaxAI/MiniMax-H3"
)

generator.update_components(
    transformer=transformer,
    vae=vae,
    audio_vae=audio_vae,
)

generator.load_components(dtype=torch.bfloat16)

# -----------------------------------------------------------------------------
# 7. Helpers
# -----------------------------------------------------------------------------
FPS = 24
FRAMES_PER_CHUNK = 17
LATENTS_PER_CHUNK = 5
RESOLUTION_PROFILES = {
    "ultra": { "16:9": (1472, 832), "9:16": (832, 1472), "1:1": (1024, 1024) },
    "high":   { "16:9": (1408, 768), "9:16": (768, 1408), "1:1": (960, 960) },
    "medium": { "16:9": (1344, 704), "9:16": (704, 1344), "1:1": (896, 896) },
    "low":    { "16:9": (1280, 640), "9:16": (640, 1280), "1:1": (832, 832) },
}
MAX_SEED = np.iinfo(np.int32).max

def snap_frames(seconds: float) -> int:
    frames = max(1, round(seconds * FPS))
    while frames % FRAMES_PER_CHUNK != LATENTS_PER_CHUNK:
        frames += 1
    return frames

def auto_crop_image(image_path: str, target_width: int, target_height: int, session_id: str = "default") -> str:
    print(f"[Auto-Crop] Resizing & Cropping to match target resolution: {target_width}x{target_height}")
    
    img = Image.open(image_path).convert("RGB")
    orig_w, orig_h = img.size
    
    target_aspect = target_width / target_height
    orig_aspect = orig_w / orig_h
    
    if abs(orig_aspect - target_aspect) < 0.01:
        print("[Auto-Crop] Image aspect ratio matches settings. Skipping modifications.")
        return image_path

    if orig_aspect > target_aspect:
        new_h = orig_h
        new_w = int(orig_h * target_aspect)
        left = (orig_w - new_w) // 2
        top = 0
        right = left + new_w
        bottom = orig_h
    else:
        new_w = orig_w
        new_h = int(orig_w / target_aspect)
        left = 0
        top = (orig_h - new_h) // 2
        right = orig_w
        bottom = top + new_h
        
    img_cropped = img.crop((left, top, right, bottom))
    print(f"[Auto-Crop] Centered crop applied from {orig_w}x{orig_h} to bounds: left={left}, top={top}, right={right}, bottom={bottom}")
    
    cropped_out_path = os.path.join(BASE, f"cropped_{session_id}.png")

    if os.path.exists(cropped_out_path):
        try:
            os.remove(cropped_out_path)
        except Exception:
            pass

    img_cropped.save(cropped_out_path)
    return cropped_out_path
    
# -----------------------------------------------------------------------------
# 8. Generation
# -----------------------------------------------------------------------------
@zero_gpu_decorator(duration=1500)
def generate_video(prompt, e_prompt_temp, image_pil, end_image_pil, quality, ratio, duration, seed, randomize_seed, use_enhancer, progress=gr.Progress(track_tqdm=True)):
    if image_pil is None:
        raise ValueError("A starting first frame image is required.")

    final_prompt = prompt
    if use_enhancer and prompt:
        final_prompt = enhance_prompt(prompt, e_prompt_temp, first_frame=image_pil)

    width, height = RESOLUTION_PROFILES[quality][ratio]
    num_frames = snap_frames(duration)
    current_seed = random.randint(0, MAX_SEED) if randomize_seed else int(seed)
    session_id = str(current_seed)
    
    if isinstance(image_pil, Image.Image):
        temp_first_path = os.path.join(BASE, f"temp_first_{session_id}.png")
        image_pil.save(temp_first_path)
        cropped_first_path = auto_crop_image(temp_first_path, width, height, session_id=f"first_{session_id}")
        image_pil = Image.open(cropped_first_path).convert("RGB")

    end_image = None
    if end_image_pil is not None and isinstance(end_image_pil, Image.Image):
        temp_end_path = os.path.join(BASE, f"temp_end_{session_id}.png")
        end_image_pil.save(temp_end_path)
        cropped_end_path = auto_crop_image(temp_end_path, width, height, session_id=f"end_{session_id}")
        end_image = Image.open(cropped_end_path).convert("RGB")

    text_encoder_4b.to("cuda")
    
    with torch.inference_mode():
        cond_result = conditioner(
            prompt=final_prompt,
            image=image_pil,
            last_image=end_image,
            height=height,
            width=width,
            num_frames=num_frames,
        )
    
        prompt_embeds = cond_result.prompt_embeds
        text_token_tags = cond_result.text_token_tags
    
    text_encoder_4b.to("cpu")
    
    del cond_result
    
    torch.cuda.empty_cache()
    torch.cuda.ipc_collect()
    
    vae.to("cuda")
    audio_vae.to("cuda")

    gen_kwargs = {
        "prompt_embeds": prompt_embeds, 
        "text_token_tags": text_token_tags, 
        "image": image_pil,
        "height": height,
        "width": width,
        "num_frames": num_frames,
        "generator": torch.Generator().manual_seed(int(current_seed)),
        "output": ["videos", "audio", "sampling_rate"],
        "num_inference_steps": 8 + 1,
    }
    if end_image is not None:
        gen_kwargs["last_image"] = end_image

    gen_result = generator(**gen_kwargs)

    out_path = os.path.join(BASE, f"minimax_h3_gradio_output_{session_id}.mp4")

    with torch.inference_mode():
        encode_video(
            gen_result["videos"][0],
            FPS,
            out_path,
            gen_result["audio"][0],
            gen_result["sampling_rate"]
        )
    
    return out_path, final_prompt

# -----------------------------------------------------------------------------
# 8.5 Standalone prompt enhancement
# -----------------------------------------------------------------------------
@zero_gpu_decorator(duration=120)
def enhance_prompt_standalone(raw_prompt, e_prompt_temp, image_pil):
    if not raw_prompt:
        return raw_prompt  
    
    enhanced = enhance_prompt(raw_prompt, e_prompt_temp, first_frame=image_pil)
    return enhanced

# -----------------------------------------------------------------------------
# 9. UI
# -----------------------------------------------------------------------------
with gr.Blocks(title="MiniMax-H3 FL2VA") as demo:
    gr.Markdown("# MiniMax-H3 FL2VA Generator")
    gr.Markdown("Using **Qwen3-VL-4B + ClipProj v3.1 + 8 step Lightning**")

    with gr.Row():
        with gr.Column():
            
            with gr.Row():
                prompt_input = gr.Textbox(
                    label="Prompt",
                    value="They all drink a beer and cheer!",
                    lines=2,
                )
            
            with gr.Row():  
                e_prompt_temp = gr.Slider(minimum=0.1, maximum=1.0, value=0.7, step=0.1, label="Enhance prompt temperature")
                
            with gr.Row():    
                enhance_btn = gr.Button("✨ Enhance Prompt", variant="secondary")    
            
            with gr.Row():
                image_input = gr.Image(
                    label="First Frame Image (Required)",
                    type="pil",
                )
                end_image_input = gr.Image(
                    label="Last Frame Image (Optional)",
                    type="pil",
                )
            
            with gr.Row():
                quality_tier = gr.Dropdown(
                    choices=["ultra", "high", "medium", "low"],
                    value="high",
                    label="Quality / Canvas Tier"
                )
                aspect_ratio = gr.Dropdown(
                    choices=["16:9", "9:16", "1:1"],
                    value="16:9",
                    label="Aspect Ratio"
                )

            with gr.Row():
                duration_slider = gr.Slider(
                    minimum=5.0,
                    maximum=10.0,
                    step=0.5,
                    value=5.0,
                    label="Duration (seconds)",
                )
            
            with gr.Row():
                seed_input = gr.Number(
                    label="Seed",
                    value=42,
                    precision=0,
                )
            
            with gr.Row():
                randomize_seed = gr.Checkbox(value=True, label="Randomize Seed")

            with gr.Row():
                enhance_toggle = gr.Checkbox(
                    label="Auto-Enhance on Generate",
                    value=False,
                    info="Automatically enhance prompt when clicking Generate"
                )

            with gr.Row():
                generate_btn = gr.Button("Generate", variant="primary")
    
        with gr.Column():
            
            with gr.Row():
                video_output = gr.Video(label="Generated Video")

    gr.Examples(
        examples=[
            "They all drink a beer and cheer!",
        ],
        inputs=[prompt_input],
        example_labels=[
            "They all drink a beer and cheer!",
        ],
        label="💡 Try these:"
    )

    enhance_btn.click(
        fn=enhance_prompt_standalone,
        inputs=[prompt_input, e_prompt_temp, image_input],
        outputs=[prompt_input],
    )

    generate_btn.click(
        fn=generate_video,
        inputs=[
            prompt_input,
            e_prompt_temp,
            image_input,
            end_image_input, 
            quality_tier, 
            aspect_ratio, 
            duration_slider,
            seed_input,
            randomize_seed,
            enhance_toggle
        ],
        outputs=[video_output, prompt_input],
    )

if __name__ == "__main__":
    if IS_HF_SPACES:
        demo.queue().launch(debug=False, share=False)
    else:
        demo.queue().launch(debug=True, share=True)

from huggingface_hub import HfApi, create_repo, hf_hub_download, snapshot_download
from diffusers import AutoModel
from safetensors.torch import load_file, save_file
import torch
import os
import json
import shutil

HF_TOKEN = os.environ["HF_TOKEN"]

HF_TOKEN = os.environ["HF_TOKEN"]
LOCAL_DIR = "./10eros_h3_transformer_pruned"
REPO_ID = "ibyteohdear/10Eros-Max-Transformer_TURBO-hybrid_beta5"

NUM_ATTENTION_HEADS = 56
ATTENTION_HEAD_DIM = 128
INNER_DIM = NUM_ATTENTION_HEADS * ATTENTION_HEAD_DIM  # 7168

# Download checkpoint
checkpoint_path = hf_hub_download(
    repo_id="TenStrip/10Eros-Max",
    filename="10Eros_Max_h3_TURBO-hybrid_beta5.safetensors",  # or whichever
    token=HF_TOKEN,
)

# Load source
print("Loading checkpoint...")
src = load_file(checkpoint_path, device="cpu")
print(f"Source keys: {len(src)}")

# Convert keys
print("Converting...")
dst = {}

# --- AdaLN handling ---
adaln_basis = src.get("adaln_basis")
if adaln_basis is not None:
    if adaln_basis.shape[0] == 2688 and adaln_basis.shape[1] == 8:
        print("Transposing adaln_basis from [2688, 8] -> [8, 2688]")
        adaln_basis = adaln_basis.T.contiguous()
    elif adaln_basis.shape[0] == 8 and adaln_basis.shape[1] == 2688:
        adaln_basis = adaln_basis.contiguous()
    else:
        raise ValueError(f"Unexpected adaln_basis shape: {adaln_basis.shape}")

adaln_mean = src.get("adaln_mean")

if "adaln_t_table" in src:
    dst["time_embedder.table"] = src["adaln_t_table"].contiguous()
if adaln_basis is not None:
    dst["adaln_basis"] = adaln_basis
if adaln_mean is not None:
    dst["adaln_mean"] = adaln_mean.contiguous()

for name, tensor in src.items():
    new_name = None
    
    if name in ("adaln_t_table", "adaln_basis", "adaln_mean"):
        continue
    if name.startswith("time_embedder."):
        continue
    
    if name == "video_patch_proj.weight":
        new_name = "proj_in.weight"
    elif name == "video_patch_proj.bias":
        new_name = "proj_in.bias"
    elif name == "audio_patch_proj.weight":
        new_name = "audio_proj_in.weight"
    elif name == "audio_patch_proj.bias":
        new_name = "audio_proj_in.bias"
    elif name == "condition_proj.weight":
        new_name = "context_embedder.weight"
    elif name == "condition_proj.bias":
        new_name = "context_embedder.bias"
    elif name == "rope.inv_freq":
        print(f"  SKIP  {name} (reconstructed from config)")
        continue
    elif name == "final_layer.adaln_proj.linear.weight":
        dst["norm_out.linear.weight"] = tensor.contiguous()
        bias = src.get("final_layer.adaln_proj.linear.bias")
        if bias is not None:
            dst["norm_out.folded_bias"] = bias.to(torch.float32).contiguous()
        continue
    elif name == "final_layer.adaln_proj.linear.bias":
        continue
    elif name == "final_layer.norm.weight":
        new_name = "norm_out.norm.weight"
    elif name == "final_layer.video_out.weight":
        new_name = "proj_out.weight"
    elif name == "final_layer.video_out.bias":
        new_name = "proj_out.bias"
    elif name == "final_layer.audio_out.weight":
        new_name = "audio_proj_out.weight"
    elif name == "final_layer.audio_out.bias":
        new_name = "audio_proj_out.bias"
    elif name == "token_refiner.final_norm.weight":
        new_name = "token_refiner.final_norm.weight"
    elif name.startswith("token_refiner.blocks."):
        base_name = name.replace("token_refiner.blocks.", "token_refiner.refiner_blocks.", 1)
        
        if ".attn.qkv_proj.weight" in base_name:
            prefix = base_name.replace(".attn.qkv_proj.weight", ".attn.")
            q, k, v = tensor.split(INNER_DIM, dim=0)
            dst[f"{prefix}to_q.weight"] = q.contiguous()
            dst[f"{prefix}to_k.weight"] = k.contiguous()
            dst[f"{prefix}to_v.weight"] = v.contiguous()
            continue
        elif ".attn.qkv_proj.bias" in base_name:
            prefix = base_name.replace(".attn.qkv_proj.bias", ".attn.")
            q_b, k_b, v_b = tensor.split(INNER_DIM, dim=0)
            dst[f"{prefix}to_q.bias"] = q_b.contiguous()
            dst[f"{prefix}to_k.bias"] = k_b.contiguous()
            dst[f"{prefix}to_v.bias"] = v_b.contiguous()
            continue
        elif ".attn.out_proj.weight" in base_name:
            new_name = base_name.replace(".attn.out_proj.weight", ".attn.to_out.0.weight")
        elif ".attn.out_proj.bias" in base_name:
            new_name = base_name.replace(".attn.out_proj.bias", ".attn.to_out.0.bias")
        elif ".attn.q_norm.weight" in base_name:
            new_name = base_name.replace(".attn.q_norm.weight", ".attn.norm_q.weight")
        elif ".attn.k_norm.weight" in base_name:
            new_name = base_name.replace(".attn.k_norm.weight", ".attn.norm_k.weight")
        elif ".mlp.fc1.weight" in base_name:
            gate, value = tensor.chunk(2, dim=0)
            new_name = base_name.replace(".mlp.fc1.weight", ".ff.net.0.proj.weight")
            dst[new_name] = torch.cat([value, gate], dim=0).contiguous()
            continue
        elif ".mlp.fc1.bias" in base_name:
            gate_b, value_b = tensor.chunk(2, dim=0)
            new_name = base_name.replace(".mlp.fc1.bias", ".ff.net.0.proj.bias")
            dst[new_name] = torch.cat([value_b, gate_b], dim=0).contiguous()
            continue
        elif ".mlp.fc2.weight" in base_name:
            new_name = base_name.replace(".mlp.fc2.weight", ".ff.net.2.weight")
        elif ".mlp.fc2.bias" in base_name:
            new_name = base_name.replace(".mlp.fc2.bias", ".ff.net.2.bias")
        else:
            new_name = base_name
                
    elif name.startswith("blocks."):
        base_name = name.replace("blocks.", "transformer_blocks.", 1)
        
        if ".attn.qkv_proj.weight" in base_name:
            prefix = base_name.replace(".attn.qkv_proj.weight", ".attn.")
            q, k, v = tensor.split(INNER_DIM, dim=0)
            dst[f"{prefix}to_q.weight"] = q.contiguous()
            dst[f"{prefix}to_k.weight"] = k.contiguous()
            dst[f"{prefix}to_v.weight"] = v.contiguous()
            continue
        elif ".attn.qkv_proj.bias" in base_name:
            prefix = base_name.replace(".attn.qkv_proj.bias", ".attn.")
            q_b, k_b, v_b = tensor.split(INNER_DIM, dim=0)
            dst[f"{prefix}to_q.bias"] = q_b.contiguous()
            dst[f"{prefix}to_k.bias"] = k_b.contiguous()
            dst[f"{prefix}to_v.bias"] = v_b.contiguous()
            continue
        elif ".attn.out_proj.weight" in base_name:
            new_name = base_name.replace(".attn.out_proj.weight", ".attn.to_out.0.weight")
        elif ".attn.out_proj.bias" in base_name:
            new_name = base_name.replace(".attn.out_proj.bias", ".attn.to_out.0.bias")
        elif ".attn.q_norm.weight" in base_name:
            new_name = base_name.replace(".attn.q_norm.weight", ".attn.norm_q.weight")
        elif ".attn.k_norm.weight" in base_name:
            new_name = base_name.replace(".attn.k_norm.weight", ".attn.norm_k.weight")
        elif ".mlp.fc1.weight" in base_name:
            gate, value = tensor.chunk(2, dim=0)
            new_name = base_name.replace(".mlp.fc1.weight", ".ff.net.0.proj.weight")
            dst[new_name] = torch.cat([value, gate], dim=0).contiguous()
            continue
        elif ".mlp.fc1.bias" in base_name:
            gate_b, value_b = tensor.chunk(2, dim=0)
            new_name = base_name.replace(".mlp.fc1.bias", ".ff.net.0.proj.bias")
            dst[new_name] = torch.cat([value_b, gate_b], dim=0).contiguous()
            continue
        elif ".mlp.fc2.weight" in base_name:
            new_name = base_name.replace(".mlp.fc2.weight", ".ff.net.2.weight")
        elif ".mlp.fc2.bias" in base_name:
            new_name = base_name.replace(".mlp.fc2.bias", ".ff.net.2.bias")
        elif ".adaln_proj.linear.weight" in base_name:
            dst[base_name] = tensor.contiguous()
            bias_key = name.replace(".weight", ".bias")
            if bias_key in src:
                folded_bias_key = base_name.replace(".adaln_proj.linear.weight", ".adaln_proj.folded_bias")
                dst[folded_bias_key] = src[bias_key].to(torch.float32).contiguous()
            continue
        elif ".adaln_proj.linear.bias" in base_name:
            continue
        else:
            new_name = base_name
    else:
        print(f"  SKIP: {name}")
        continue
        
    if new_name is not None:
        dst[new_name] = tensor.contiguous()

print(f"Converted keys: {len(dst)}")

# Save directly as safetensors (no model instantiation needed)
print(f"\nSaving to {LOCAL_DIR}...")
os.makedirs(LOCAL_DIR, exist_ok=True)

# Save the converted weights
save_file(dst, os.path.join(LOCAL_DIR, "diffusion_pytorch_model.safetensors"))

# Fetch and save config from base repo
print("Fetching config...")
base_repo_path = snapshot_download(
    "multimodalart/MiniMax-H3-Pruned",
    allow_patterns=["transformer/config.json", "transformer/modeling_minimax_h3_pruned.py"],
    local_dir="./_temp_base_repo",
)

# Copy config
src_config = os.path.join(base_repo_path, "transformer", "config.json")
with open(src_config, "r") as f:
    config = json.load(f)

config["_class_name"] = "MiniMaxH3PrunedTransformer3DModel"

with open(os.path.join(LOCAL_DIR, "config.json"), "w") as f:
    json.dump(config, f, indent=2)

# Copy modeling file
src_modeling = os.path.join(base_repo_path, "transformer", "modeling_minimax_h3_pruned.py")
if os.path.exists(src_modeling):
    shutil.copy2(src_modeling, os.path.join(LOCAL_DIR, "modeling_minimax_h3_pruned.py"))

if os.path.exists("./_temp_base_repo"):
    shutil.rmtree("./_temp_base_repo")

print(f"\nDone! Files saved to {LOCAL_DIR}")
print(f"Load with DiffusionPipeline or the custom model class")
# Upload
print(f"\nUploading to {REPO_ID}...")
api = HfApi(token=HF_TOKEN)
create_repo(REPO_ID, exist_ok=True, private=True)
api.upload_folder(folder_path=LOCAL_DIR, repo_id=REPO_ID, repo_type="model")

print(f"\nDone! Uploaded to {REPO_ID}")
print(f"Load with DiffusionPipeline or the custom model class")