The model weights and their respective implementation were taken from moonshotai/Kimi-K2.6, so all credit and thanks go to the Moonshot AI team.
This checkpoint is the vision tower of Kimi-K2.6 stored in the native Kimi_K25VisionModel format, so it works directly with transformers — no trust_remote_code needed.
Example usage:
from transformers import AutoImageProcessor, AutoModel
import torch
from PIL import Image
img = Image.open("a_red_car.png")
processor = AutoImageProcessor.from_pretrained("Aquiles-ai/MoonViT-3D")
model = AutoModel.from_pretrained("Aquiles-ai/MoonViT-3D", dtype=torch.bfloat16).to("cuda")
inputs = processor(images=[img], return_tensors="pt")
with torch.no_grad():
out = model(
pixel_values=inputs["pixel_values"].to(torch.bfloat16).to("cuda"),
grid_thw=inputs["image_grid_thw"].to("cuda"),
)
print(out.last_hidden_state.shape) # (num_patches, 1152)
print(out.pooler_output.shape) # (num_merged_blocks, 4, 1152)
Batch processing:
from transformers import AutoImageProcessor, AutoModel
import torch
from PIL import Image
img = Image.open("a_red_car.png")
img2 = Image.open("a_gray_cat.png")
processor = AutoImageProcessor.from_pretrained("Aquiles-ai/MoonViT-3D")
model = AutoModel.from_pretrained("Aquiles-ai/MoonViT-3D", dtype=torch.bfloat16).to("cuda")
inputs = processor(images=[img, img2], return_tensors="pt")
with torch.no_grad():
out = model(
pixel_values=inputs["pixel_values"].to(torch.bfloat16).to("cuda"),
grid_thw=inputs["image_grid_thw"].to("cuda"),
)
print(out.last_hidden_state.shape) # (total_patches, 1152) - flattened over the batch
print(out.pooler_output.shape) # (total_merged_blocks, 4, 1152)
Actual use cases for this model:
- Image embedding extraction for similarity search (image retrieval), dataset deduplication, or image clustering.
- Frozen backbone for downstream fine-tuning: attach a classification, detection, or regression head on top of the features and train only that part.
- Building block for your own VLM: connect it to an LLM (via a projection layer such as a Q-Former or MLP) if you want to replicate something similar to Kimi-VL but with your own text decoder.
- Multimodal RAG: generate image embeddings to index alongside text in a vector database and perform cross-modal text-to-image retrieval.
- Frame-by-frame video processing: since it supports
grid_thwwith a temporal dimension, in theory you can pass sequences of frames and get spatio-temporal features, although the modelcard doesn't include an example of this — you'd need to build it yourself.