MiniCPM-V-4.6-abliterated-MAX-GGUF

MiniCPM-V-4.6-abliterated-MAX is an abliterated evolution built on top of openbmb/MiniCPM-V-4.6. This model applies advanced refusal direction analysis and ablation-based optimization strategies to reduce internal refusal behaviors while preserving the multimodal reasoning and instruction-following strengths of the original architecture. The result is a highly capable and ultra-efficient multimodal language model optimized for image, video, and text understanding with improved instruction adherence.

[!IMPORTANT] This model is intended for research and learning purposes only. It reduces internal refusal behaviors, and any content generated by it is used at the user’s own risk. The authors and hosting page disclaim any liability for outputs produced by this model. Users are responsible for ensuring safe, ethical, and lawful usage.

Getting Started with llama.cpp Using Docker

FROM ghcr.io/ggml-org/llama.cpp:full

WORKDIR /app

# Install minimal dependencies required for creating a Python virtual environment
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3-pip python3-venv \
    && rm -rf /var/lib/apt/lists/*

# Create virtual environment
RUN python3 -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

# Install Python packages inside the virtual environment only
RUN pip install --no-cache-dir -U huggingface_hub

# Download model and mmproj
RUN python3 -c 'from huggingface_hub import hf_hub_download; \
    repo="prithivMLmods/MiniCPM-V-4.6-abliterated-MAX-GGUF"; \
    hf_hub_download(repo_id=repo, filename="MiniCPM-V-4.6-abliterated-MAX.Q4_K_M.gguf", local_dir="/app"); \
    hf_hub_download(repo_id=repo, filename="MiniCPM-V-4.6-abliterated-MAX.mmproj-f16.gguf", local_dir="/app")'

CMD ["--server", \
     "-m", "/app/MiniCPM-V-4.6-abliterated-MAX.Q4_K_M.gguf", \
     "--mmproj", "/app/MiniCPM-V-4.6-abliterated-MAX.mmproj-f16.gguf", \
     "--host", "0.0.0.0", \
     "--port", "7860", \
     "-t", "3", \
     "--mlock", \
     "--prio", "3", \
     "--swa-full", \
     "--no-slots", \
     "-ngl", "99", \
     "--mmap", \
     "--log-disable", \
     "--skip-chat-parsing", \
     "--no-cont-batching", \
     "--threads-http", "1", \
     "--direct-io", \
     "--no-repack", \
     "--flash-attn", "off", \
     "-c", "640000", \
     "-n", "389012"]

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):