YOLOv8s - Handwritten Signature Detection

This repository presents a YOLOv8s-based model, fine-tuned to detect handwritten signatures in document images.

Resource	Links / Badges	Details
Article		A detailed community article covering the full development process of the project
Model Files		Available formats:
Dataset – Original		2,819 document images annotated with signature coordinates
Dataset – Processed		Augmented and pre-processed version (640px) for model training
Notebooks – Model Experiments		Complete training and evaluation pipeline with selection among different architectures (yolo, detr, rt-detr, conditional-detr, yolos)
Notebooks – HP Tuning		Optuna trials for optimizing the precision/recall balance
Inference Server		Complete deployment and inference pipeline with Triton Inference Server
Live Demo		Graphical interface with real-time inference

Dataset

The training utilized a dataset built from two public datasets: Tobacco800 and signatures-xc8up, unified and processed in Roboflow.

Dataset Summary:

• Training: 1,980 images (70%)
• Validation: 420 images (15%)
• Testing: 419 images (15%)
• Format: COCO JSON
• Resolution: 640x640 pixels

Training Process

The training process involved the following steps:

1. Model Selection:

Various object detection models were evaluated to identify the best balance between precision, recall, and inference time.

Highlights:

• Best mAP50: conditional-detr-resnet-50 (0.936524)
• Best mAP50-95: yolov8m (0.665495)
• Fastest Inference Time: yolov10n (73.8596 ms)

Detailed experiments are available on Weights & Biases.

2. Hyperparameter Tuning:

The YOLOv8s model, which demonstrated a good balance of inference time, precision, and recall, was selected for hyperparameter tuning.

Optuna was used for 20 optimization trials. The hyperparameter tuning used the following parameter configuration:

    dropout = trial.suggest_float("dropout", 0.0, 0.5, step=0.1)
    lr0 = trial.suggest_float("lr0", 1e-5, 1e-1, log=True)
    box = trial.suggest_float("box", 3.0, 7.0, step=1.0)
    cls = trial.suggest_float("cls", 0.5, 1.5, step=0.2)
    opt = trial.suggest_categorical("optimizer", ["AdamW", "RMSProp"])

Results can be visualized here: Hypertuning Experiment.

3. Evaluation:

The models were evaluated on the test set at the end of training in ONNX (CPU) and TensorRT (GPU - T4) formats. Performance metrics included precision, recall, mAP50, and mAP50-95.

Results Comparison:

Results

After hyperparameter tuning of the YOLOv8s model, the best model achieved the following results on the test set:

• Precision: 94.74%
• Recall: 89.72%
• mAP@50: 94.50%
• mAP@50-95: 67.35%
• Inference Time:
  • ONNX Runtime (CPU): 171.56 ms
  • TensorRT (GPU - T4): 7.657 ms

How to Use

The YOLOv8s model can be used via CLI or Python code using the Ultralytics library. Alternatively, it can be used directly with ONNX Runtime or TensorRT.

The final weights are available in the main directory of the repository:

• yolov8s.pt (PyTorch format)
• yolov8s.onnx (ONNX format)
• yolov8s.engine (TensorRT format)

Python Code

• Dependencies

pip install ultralytics supervision huggingface_hub

• Inference

import cv2
import supervision as sv

from huggingface_hub import hf_hub_download
from ultralytics import YOLO

model_path = hf_hub_download(
  repo_id="tech4humans/yolov8s-signature-detector", 
  filename="yolov8s.pt"
)

model = YOLO(model_path)

image_path = "/path/to/your/image.jpg"
image = cv2.imread(image_path)

results = model(image_path)

detections = sv.Detections.from_ultralytics(results[0])

box_annotator = sv.BoxAnnotator()
annotated_image = box_annotator.annotate(scene=image, detections=detections)

cv2.imshow("Detections", annotated_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

Ensure the paths to the image and model files are correct.

CLI

• Dependencies

pip install -U ultralytics "huggingface_hub[cli]"

• Inference

huggingface-cli download tech4humans/yolov8s-signature-detector yolov8s.pt

yolo predict model=yolov8s.pt source=caminho/para/imagem.jpg

Parameters:

• model: Path to the model weights file.
• source: Path to the image or directory of images for detection.

ONNX Runtime

For optimized inference, you can find the inference code using onnxruntime and OpenVINO Execution Provider in the handler.py file and on the Hugging Face Space here.

Demo

You can explore the model and test real-time inference in the Hugging Face Spaces demo, built with Gradio and ONNXRuntime.

🔗 Inference with Triton Server

If you want to deploy this signature detection model in a production environment, check out our inference server repository based on the NVIDIA Triton Inference Server.

Infrastructure

Software

The model was trained and tuned using a Jupyter Notebook environment.

• Operating System: Ubuntu 22.04
• Python: 3.10.12
• PyTorch: 2.5.1+cu121
• Ultralytics: 8.3.58
• Roboflow: 1.1.50
• Optuna: 4.1.0
• ONNX Runtime: 1.20.1
• TensorRT: 10.7.0

Hardware

Training was performed on a Google Cloud Platform n1-standard-8 instance with the following specifications:

• CPU: 8 vCPUs
• GPU: NVIDIA Tesla T4

License

Model Weights (Fine-Tuned Model) – AGPL-3.0

• License: GNU Affero General Public License v3.0 (AGPL-3.0)
• Usage: The fine-tuned model weights, derived from the YOLOv8 model by Ultralytics, are licensed under AGPL-3.0. This requires that any modifications or derivative works of these model weights also be distributed under AGPL-3.0, and if the model is used as part of a network service, the corresponding source must be made available.

Code, Training, Deployment, and Data – Apache 2.0

• License: Apache License 2.0
• Usage: All additional materials—including training scripts, deployment code, usage instructions, and associated data—are licensed under the Apache 2.0 license.

For more details, please refer to the full license texts:

• GNU AGPL-3.0 License
• Apache License 2.0

Contact and Information

For further information, questions, or contributions, contact us at iag@tech4h.com.br.

Author