GoblinV1

GoblinV1 is a fine-tuned vision-language model for Rune Goblin, an AI dungeon crawler where players draw spell glyphs and the model interprets those drawings as magic.

The model is fine-tuned from OpenBMB MiniCPM-V-4.6 to read hand-drawn RuneLang glyphs and produce structured JSON describing detected runes, ambiguity, spell metadata, and visual presentation hints.

It is not a generic chatbot.

It is a tiny cursed spell reader.

Draw better runes, get stronger magic.
Draw cursed doodles, suffer beautifully.

Model Summary

What This Model Does

GoblinV1 reads a player-drawn spell image and returns structured spell interpretation metadata.

The model is trained to identify:

• drawn RuneLang glyphs
• ambiguous or messy rune shapes
• confidence of the visual reading
• spell name and spell type
• colors, shape, motion, particles, and sound tags
• presentation hints for game VFX
• weird, funny, cursed spell flavor consistent with Rune Goblin

The model is designed to act as the rune reader and spell presentation planner.

The game engine should still own final combat balance, HP changes, boss rules, inventory state, and durable quest state.

Rune Goblin

Rune Goblin is a Gradio-based AI dungeon crawler where the player draws their own spells.

Instead of selecting “fireball” from a fixed menu, the player draws symbolic glyphs. GoblinV1 reads the glyphs and converts them into spell interpretation JSON.

Example RuneLang meanings:

Example combinations:

Expected Input

The model expects an image containing a hand-drawn RuneLang spell.

The image usually contains 1-4 glyphs drawn by the player on a canvas.

The prompt should instruct the model to return JSON only.

Example prompt:

Look at this drawn RuneLang spell. Identify the runes, ambiguity, confidence, and produce spell presentation metadata: name, type, colors, shape, motion, grandeur, particles, and sound tags. Return valid JSON only.

Expected Output

GoblinV1 should return JSON in this style:

{
  "visual_reading": {
    "detected_runes": ["flame", "circle"],
    "ambiguous_runes": [],
    "confidence": 0.91,
    "layout": "left_to_right"
  },
  "spell": {
    "spell_name": "Ember Lunchbox Ward",
    "spell_type": "fire_defense",
    "rune_combo": ["flame", "circle"],
    "summary": "A circular flame ward forms around the player.",
    "colors": ["orange", "red", "gold"],
    "shape": "burning circular shield",
    "motion": "slow clockwise rotation with pulsing embers",
    "grandeur": "medium",
    "particles": ["embers", "sparks", "heat shimmer"],
    "sound_tags": ["crackle", "whoosh", "low hum"]
  }
}

Important Design Boundary

GoblinV1 is intentionally not responsible for final game-state authority.

The model may suggest spell interpretation and presentation metadata, but the game engine should decide:

• final HP changes
• damage numbers
• cooldowns
• boss immunity rules
• quest state
• inventory changes
• progression unlocks
• anti-cheat and validation logic

Recommended architecture:

Player drawing
    ↓
GoblinV1 vision model
    ↓
Rune + spell JSON
    ↓
Game engine validates / clamps / balances
    ↓
Final spell effect is applied

This keeps Rune Goblin fun and expressive while preventing model hallucinations from breaking combat balance.

Training Data

GoblinV1 was fine-tuned on ASHu2/rune_goblin_visual_dataset, a custom visual instruction dataset for Rune Goblin.

The dataset contains hand-drawn or synthetic RuneLang spell images paired with structured chat-style targets.

Each example teaches the model to:

• read visual glyphs
• map glyphs to RuneLang symbols
• handle messy or ambiguous drawings
• follow RuneLang combination rules
• return valid JSON
• preserve the game’s cursed-comedic tone

The dataset uses image paths and conversational messages containing a system prompt, user prompt, and assistant JSON target.

Training Objective

The objective is not general image captioning.

The model is trained for:

drawn rune image + instruction → structured spell interpretation JSON

The model learns the custom RuneLang vocabulary and the relationship between visual glyphs and spell presentation.

Model Formats

This repository includes multiple usable formats.

Safetensors

Use this for normal Transformers-based inference or deployment.

GGUF

A quantized GGUF version is included for local inference experiments with llama.cpp-compatible runtimes.

Example GGUF file:

gguf/rune-goblin-v46-Q4_K_M.gguf

If using the GGUF model for vision tasks, make sure your runtime supports MiniCPM-V style multimodal inference and loads the required vision/projector files when needed.

Basic Transformers Usage

Install dependencies:

pip install torch torchvision pillow transformers accelerate

Example usage:

from PIL import Image
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "ASHu2/goblinV1"

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True
)

model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

image = Image.open("example_rune.png").convert("RGB")

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {
                "type": "text",
                "text": (
                    "Look at this drawn RuneLang spell. "
                    "Identify the runes, ambiguity, confidence, and produce spell presentation metadata. "
                    "Return valid JSON only."
                )
            }
        ]
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False
    )

generated = output_ids[0][inputs["input_ids"].shape[-1]:]
text = processor.decode(generated, skip_special_tokens=True)

print(text)

llama.cpp / GGUF Usage

If using a compatible llama.cpp build:

llama-server -hf ASHu2/goblinV1:Q4_K_M

Or run directly:

llama-cli -hf ASHu2/goblinV1:Q4_K_M

For multimodal inference, ensure your client/runtime supports MiniCPM-V and any required vision projector configuration.

Intended Use

GoblinV1 is intended for:

• Rune Goblin gameplay
• hand-drawn spell glyph interpretation
• visual rune recognition
• experimental AI game mechanics
• structured JSON generation from fantasy glyph images
• small-model multimodal game prototypes

Out-of-Scope Use

GoblinV1 is not intended for:

• safety-critical image understanding
• medical, legal, financial, or security decisions
• general OCR benchmarking
• real-world symbol recognition systems
• moderation or surveillance
• authoritative factual QA
• replacing deterministic game rules

Limitations

GoblinV1 may:

• misread very messy drawings
• confuse visually similar glyphs
• produce malformed JSON in some cases
• invent spell details outside the intended schema
• overfit to Rune Goblin-style symbols
• perform poorly on non-RuneLang images
• require game-engine validation before applying effects

Recommended production safeguards:

• validate output JSON
• retry once on invalid output
• clamp all numeric values in the game engine
• reject unknown runes
• keep final state transitions deterministic
• log ambiguous readings for future dataset improvement

Suggested Evaluation

Recommended metrics:

Suggested test cases:

flame + circle
spiral + eye
bone + dots
mirror + jagged
leaf + bone
broken + coin
messy flame vs leaf
partial / incomplete glyph
empty canvas

Example Output

{
  "visual_reading": {
    "detected_runes": ["spiral", "eye", "broken_mark"],
    "ambiguous_runes": [
      {
        "candidates": ["spiral", "wave"],
        "reason": "curved repeated stroke could indicate either looping time or water/emotion"
      }
    ],
    "confidence": 0.82,
    "layout": "clustered"
  },
  "spell": {
    "spell_name": "Cursed Foresight Loop",
    "spell_type": "prophecy_curse",
    "rune_combo": ["spiral", "eye", "broken_mark"],
    "summary": "The spell reveals a possible future, then immediately makes it worse.",
    "colors": ["violet", "black", "pale blue"],
    "shape": "floating eye inside a cracked spiral",
    "motion": "spiral contracts inward while the eye flickers",
    "grandeur": "high",
    "particles": ["purple sparks", "black motes", "thin time-rings"],
    "sound_tags": ["whisper", "glass-crack", "reverse-chime"]
  }
}

Deployment Notes

GoblinV1 can be used as the model backend for a Gradio game.

Recommended serving layout:

Gradio UI / Canvas
    ↓
Image preprocessing
    ↓
GoblinV1 inference
    ↓
JSON parsing + validation
    ↓
Rune Goblin game engine
    ↓
Updated battle state + animation

For GPU deployment, use the Safetensors model with Transformers.

For lightweight/local experiments, use the GGUF export with a compatible llama.cpp runtime.

Related Project

GoblinV1 powers the Rune Goblin dungeon crawler.

Rune Goblin is an AI game where players draw spells, explore maps, fight bosses, trigger cursed outcomes, and unlock stronger effects by drawing clearer runes.

Links:

• Game/demo: https://huggingface.co/spaces/build-small-hackathon/Rune-Goblin
• Model: https://huggingface.co/ASHu2/goblinV1
• Dataset: https://huggingface.co/datasets/ASHu2/rune_goblin_visual_dataset

Citation

If you use this model, please cite:

@misc{goblinv1_2026,
  title        = {GoblinV1: A Fine-Tuned MiniCPM-V Rune Reader for Rune Goblin},
  author       = {Ashutosh Mishra},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {https://huggingface.co/ASHu2/goblinV1}
}

Base model:

@misc{minicpmv46_2026,
  title        = {MiniCPM-V 4.6},
  author       = {OpenBMB},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {https://huggingface.co/openbmb/MiniCPM-V-4.6}
}

License

This model is released under the Apache-2.0 license.

Please also follow the license and usage terms of the base model openbmb/MiniCPM-V-4.6.

Acknowledgements

GoblinV1 is built on top of OpenBMB MiniCPM-V-4.6.

Thanks to the open-source multimodal model community, Hugging Face, Gradio, Modal, llama.cpp, and the cursed little goblin inside every ambiguous doodle.