Hermes 2 Pro BakLLaVA - Mistral 7B
Hermes 2 Pro's LLaMA weights + BakLLaVA's mm_projector & vision_tower weights.
Good QA + Function Calling + JSON Mode + Vision Multimodal
GGUFs:
- Hermes 2 pro: https://huggingface.co/NousResearch/Hermes-2-Pro-Mistral-7B-GGUF
- BakLLaVA-1: https://huggingface.co/mys/ggml_bakllava-1
Test code:
from llava.mm_utils import get_model_name_from_path
from llava.eval.run_llava import eval_model
model_path = "vonjack/Hermes-2-Pro-BakLLaVA-Mistral-7B"
prompt = "What's the content of the image?"
image_file = "https://www.ilankelman.org/stopsigns/australia.jpg"
args = type('Args', (), {
"model_path": model_path,
"model_base": None,
"model_name": get_model_name_from_path(model_path),
"query": prompt,
"conv_mode": None,
"image_file": image_file,
"sep": ",",
"temperature": 0,
"top_p": None,
"num_beams": 1,
"max_new_tokens": 512
})()
eval_model(args)
Example:
