This model is used in OpenPlants (domai-tb/OpenPlants), an open-source and privacy-friendly companion app for plant care.
Inside the app, it powers local image classification so users can identify plants directly on-device from a photo.
Built on google/vit-base-patch16-224, the model was fine-tuned for species-level plant identification on the GBIF / iNaturalist plant image dataset.
Highlights
| Property | Value |
|---|---|
| Base model | google/vit-base-patch16-224 |
| Parameters | ~97.2M |
| Training samples | 2,000,000 curated plant occurrences |
| Species coverage | ~14,000 unique species |
| Source data | GBIF / iNaturalist |
| Training method | End-to-end supervised fine-tuning |
| Primary use | Fast plant species classification from a single image |
OpenPlants
OpenPlants is the companion app this model was built for.
- Runs plant identification locally
- Keeps inference on-device for a privacy-friendly experience
- Helps users identify plants in a fast, offline-friendly workflow
Example Usage
from transformers import AutoImageProcessor, AutoModelForImageClassification
from PIL import Image
import requests
import torch
model_id = "domai-tb/OpenPlants-Identification-ViT-Base-Patch16-224"
processor = AutoImageProcessor.from_pretrained(model_id)
model = AutoModelForImageClassification.from_pretrained(model_id)
url = "https://example.com/plant.jpg"
image = Image.open(requests.get(url, stream=True).raw)
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
probs = logits.softmax(dim=-1)[0]
topk = torch.topk(probs, k=5)
for prob, idx in zip(topk.values, topk.indices):
label = model.config.id2label[idx.item()]
print(f"{label}: {prob.item():.4f}")
Intended Applications
- Plant identification in mobile apps
- Ecological surveys
- Nursery and horticulture tools
- Restoration and revegetation workflows
- Field research and biodiversity monitoring
- Citizen science and educational platforms
- Image-based species tagging pipelines
Data and Training Details
Dataset Construction
- Sourced from GBIF / iNaturalist occurrences with valid
speciesand image metadata - Cleaned and deduplicated before training
- Species filtered to those with at least 20 images
- Maximum cap of 1,000 images per species to reduce class imbalance
- Final training dataset: 2,000,000 images across roughly 14,000 species
Training
- ViT-Base fine-tuned for 1 epoch over 2M samples
- AdamW optimizer with standard ViT augmentations
- Mixed-precision training on GPU
Limitations
- Some species are visually indistinguishable without context such as location or reproductive structures
- Performance varies for rare, morphologically similar, or poorly photographed species
- The model is purely image-based and does not use location metadata
Labels
Species names follow canonical GBIF taxonomy (species_name).
Each class maps directly to one species.
You can inspect all labels with:
from transformers import AutoConfig
cfg = AutoConfig.from_pretrained("domai-tb/OpenPlants-Identification-ViT-Base-Patch16-224")
labels = cfg.id2label
Fine-Tuning and Adapters
You can further specialize the model with LoRA adapters for:
- Regional subsets
- Functional groups
- Threatened species
- Agricultural crops
- Disease classification
The base model is broad enough to support domain-specific adapter tuning with relatively little compute.
License
Model weights follow the same license as the underlying ViT-Base model. Users are responsible for complying with GBIF and iNaturalist usage terms for any downstream dataset creation.