Introduction
PP-DocLayoutV2 is a dedicated lightweight model for layout analysis, focusing specifically on element detection, classification, and reading order prediction.
Model Architecture
PP-DocLayoutV2 is composed of two sequentially connected networks. The first is an RT-DETR-based detection model that performs layout element detection and classification. The detected bounding boxes and class labels are then passed to a subsequent pointer network, which is responsible for ordering these layout elements.
Model Usage
import requests
from PIL import Image
from transformers import AutoImageProcessor, AutoModelForObjectDetection
model_path = "PaddlePaddle/PP-DocLayoutV2_safetensors"
model = AutoModelForObjectDetection.from_pretrained(model_path)
image_processor = AutoImageProcessor.from_pretrained(model_path)
image = Image.open(requests.get("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout_demo.jpg", stream=True).raw)
inputs = image_processor(images=image, return_tensors="pt")
outputs = model(**inputs)
results = image_processor.post_process_object_detection(outputs, target_sizes=[image.size[::-1]])
for result in results:
print(result["scores"])
print(result["labels"])
print(result["boxes"])
for idx, (score, label_id, box) in enumerate(zip(result["scores"], result["labels"], result["boxes"])):
score, label = score.item(), label_id.item()
box = [round(i, 2) for i in box.tolist()]
print(f"Order {idx + 1}: {model.config.id2label[label]}: {score:.2f} {box}")