Rarestry-Moondream2-Pokemon-Grading-v0.1.1

Rarestry-Moondream2-Pokemon-Grading-v0.1.1 is an edge-optimized, fine-tuned Vision-Language Model tailored specifically for physical Pokémon trading card analysis, metadata extraction, and automated grade prediction.

Trained on expanded multi-era trading card distributions, this checkpoint maps raw visual card crops directly into 100% schema-compliant JSON, evaluating card identity (card_name, set_name, card_number), sub-pixel centering ratios, corner wear, edge chipping/whitening, surface condition, and predicted PSA tiering.

The repository includes both the full merged Hugging Face safetensors weights and the quantized INT8 ONNX model artifact (Rarestry-Moondream2-Pokemon-Grading-v0.1.1-int8.onnx) engineered to run locally within high-performance browser, mobile, and desktop PWA runtimes.

Model Architecture & Mobile Strategy

Moondream2 utilizes a compact 0.5B-parameter dual architecture consisting of an image ViT projection network and an autoregressive language decoder. Because physical sub-millimeter measurements require extreme mathematical precision, Rarestry pairs this model with a hybrid pipeline:

                    [Raw Camera Frame / Scan]
                               │
                               ▼
                [OpenCV Perspective Warp & Era Anchor]
                               │
            ┌──────────────────┴──────────────────┐
            ▼                                     ▼
   [Deterministic Math]                  [Moondream2 VLM / ONNX]
   - Sub-pixel border offsets            - Card name, set, & number OCR
   - Corner radius detection             - Edge whitening & chipping
   - Ratio bounds (50/50, 60/40)         - Holographic silvering & scratches
            │                                     │
            └──────────────────┬──────────────────┘
                               │
                               ▼
               [Structured Grading & Metadata JSON]

Target Output Schema

The model reliably outputs valid JSON containing both identification metadata and condition metrics without prefix conversational text or markdown code blocks:

{
  "identity": {
    "card_name": "Charizard",
    "set_name": "Base Set",
    "card_number": "4/102"
  },
  "centering": {
    "score": 9.5,
    "horizontal_ratio": "53.6/46.4",
    "vertical_ratio": "51.8/48.2",
    "psa_tier": "Gem Mint (54/46)"
  },
  "corners": {
    "score": 10.0,
    "flaws": []
  },
  "edges": {
    "score": 10.0,
    "flaws": []
  },
  "surface": {
    "score": 10.0,
    "flaws": []
  },
  "overall_grade": 9.5
}

Multi-Era Coverage & Metadata Integration

Fine-tuned and verified against card borders and text layouts across Pokémon TCG history:

• Vintage Wizards of the Coast (Base Set – Gym Challenge): Yellow border symmetry, holo foil scratching, vintage cardstock whitening, and classic bottom-right card numbering.
• e-Reader Era (Expedition, Aquapolis, Skyridge): Wide asymmetric bottom/left borders and dot-matrix code bounds.
• EX, Diamond & Pearl, Platinum Era: Classic side numbering and layout variance.
• Modern Silver & Scarlet / Violet Era: Low-contrast silver border ratios, set symbol recognition, and modern numbering formats (e.g., 001/165).
• Full Art, Secret Rare, & Trainer Gallery: Borderless and ultra-rare texture grading with extended collector code extraction.

Note: For live market pricing and exact variant lookups, pair the extracted set_name and card_number directly with the JustTCG API.

Quickstart (Python / PyTorch)

import json
import torch
from PIL import Image
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "economyofdreams/Rarestry-Moondream2-Pokemon-Grading-v0.1.1"

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model.eval()

# Load and prepare cropped card image
image_path = "charizard_base_set.jpg"
image = Image.open(image_path).convert("RGB")

# Grading prompt
prompt = "Identify card name, set name, card number, and analyze centering and physical edges. Output structured JSON."

# Inference
with torch.no_grad():
    image_embeds = model.encode_image(image)
    raw_response = model.answer_question(image_embeds, prompt, tokenizer)

# Parse output
grading_result = json.loads(raw_response.strip())
print(json.dumps(grading_result, indent=2))

Training Details

• Base Model: vikhyatk/moondream2 (Revision 2024-08-26 / 0.5B parameters)
• Dataset Size: 420 multi-era synthetic training samples mapping perspective-corrected scans to rigorous identity and condition schemas.
• Hardware: NVIDIA L4 (Google Cloud)
• Epochs: 3
• Final Converged Loss: ~0.15–0.24
• Optimizer: AdamW (LR: $5\times 10^{-5}$, Linear Warmup)

Repository Files

Ecosystem Integration

Part of the Rarestry card digitization and grading suite. Developed by TehWiz Productions.