Elliot OV2 SFT 9k

ElliotVL vision-language checkpoint, exported to standard Hugging Face format from the 9k-step Megatron checkpoint.

Benchmarks

All ElliotVL numbers below were measured on this checkpoint with VLMEvalKit in no-think mode.

* reproduced by us · (parentheses) value reported by the model's authors or a third party · – not measured.

Read the numbers with these caveats:

• † Math rows are not official scores. They come from a local exact-match scorer with the judge API disabled, so they are not comparable to published MathVista/MathVerse/MathVision figures, which use an LLM judge.
• ChartQA is reported twice. 68.84 is the raw VLMEvalKit relaxed-accuracy result. Its non-numeric answers require an exact string match, and a format audit found many otherwise-correct predictions carrying trailing punctuation or units; normalising those conservatively gives 83.72. The raw number is the one to compare against other raw numbers.
• MME is split into perception and cognition (VLMEvalKit labels the latter reasoning in its raw CSV). InternVL3.5 reports only a sum, so its cells are not split.
• OCRBench for Qwen3-VL is normalised from the reported 858 to 85.8.
• MMMU reruns use an answer-only prompt; earlier exact-match values were invalid because the prompt produced long truncated explanations.
• DocVQA official reports are on the test split; VAL and TEST are different rows and are not interchangeable.
• CharXiv RQ/DQ were not run — they require a judge API that was disabled.

Usage

Requires Transformers 5.8.0.dev0 (development build from GitHub), plus torch, pillow, accelerate, safetensors.

from transformers import AutoModelForCausalLM, AutoProcessor

model = AutoModelForCausalLM.from_pretrained(
    "elliot-mllm/elliot-ov2-sft-9k", trust_remote_code=True, torch_dtype="bfloat16")
processor = AutoProcessor.from_pretrained(
    "elliot-mllm/elliot-ov2-sft-9k", trust_remote_code=True)

Text-only prompt:

prompt = (
    "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n"
    "<|im_start|>user\nAnswer briefly: what is a checkpoint?<|im_end|>\n"
    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
)

For multimodal prompts, include one <image> placeholder in the user text per image passed to the processor.

Repository contents

• config.json with auto_map entries for AutoConfig, AutoModelForCausalLM and AutoProcessor
• Sharded safetensors weights with model.safetensors.index.json
• Tokenizer, processor, image processor, chat template and remote-code modules

Export validation

This is a research checkpoint.