Gliese-OCR-7B-Post2.0-final
The Gliese-OCR-7B-Post2.0-final model is a refined and optimized version of Gliese-OCR-7B-Post1.0, built upon the Qwen2.5-VL architecture. It represents the final iteration in the Gliese-OCR series, offering enhanced efficiency, precision, and visualization capabilities for document OCR, visual analysis, and information extraction.
Fine-tuned with extended document visualization data and OCR-focused objectives, this model delivers superior accuracy across a wide range of document types, including scanned PDFs, handwritten pages, structured forms, and analytical reports.
Key Enhancements
- Optimized Document Visualization and OCR Pipeline: Significantly improved recognition of text, layout, and embedded visuals for structured document understanding.
- Context-Aware Multimodal Linking: Enhanced understanding of document context with stronger alignment between text, images, and layout components.
- Refined Document Retrieval: Improved retrieval accuracy from complex layouts and multi-page documents.
- High-Fidelity Content Extraction: Precise extraction of structured, semi-structured, and unstructured information with advanced text normalization.
- Analytical Recognition: Superior reasoning over charts, graphs, tables, and mathematical equations.
- Improved Visual Reasoning and Layout Awareness: Trained on document visualization datasets for advanced spatial and semantic comprehension.
- State-of-the-Art Performance Across Resolutions: Achieves top results on benchmarks such as DocVQA, InfographicVQA, MathVista, and RealWorldQA.
- Extended Multimodal Duration Support: Handles long document sequences and extended videos (20+ minutes).
- Final Release Stability: Consolidates all prior improvements for stable and reliable performance.
Quick Start with Transformers
from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from qwen_vl_utils import process_vision_info
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
"prithivMLmods/Gliese-OCR-7B-Post2.0-final", torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("prithivMLmods/Gliese-OCR-7B-Post2.0-final")
messages = [
{
"role": "user",
"content": [
{"type": "image", "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"},
{"type": "text", "text": "Describe the document structure and extract key text content."},
],
}
]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
text=[text],
images=image_inputs,
videos=video_inputs,
padding=True,
return_tensors="pt",
).to("cuda")
generated_ids = model.generate(**inputs, max_new_tokens=256)
generated_ids_trimmed = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
output_text = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)
print(output_text)
Intended Use
- Document visualization and OCR extraction tasks.
- Context-aware document retrieval and multimodal linking.
- Extraction and LaTeX formatting of equations and structured content.
- Analytical document interpretation (charts, tables, graphs, and figures).
- Multilingual OCR for enterprise, academic, and research use cases.
- Summarization, question answering, and cross-modal reasoning over long documents.
- Intelligent robotic or mobile automation guided by visual document input.
Limitations
- Reduced accuracy on heavily degraded or occluded documents.
- High computational requirements for large-scale or real-time applications.
- Limited optimization for low-resource or edge devices.
- Occasional misalignment in text layout or minor hallucinations in outputs.
- Performance may vary depending on visual token configuration and context length settings.