This benchmark evaluates the text recognition performance of OCR and vision-language models (VLMs) using early 20th-century Persian Nastaliq lithographs. It provides a held-out evaluation set separate from the training corpus, expert-verified ground-truth transcriptions, and per-model error analysis data. Developers can use these resources to compare model outputs with verified transcriptions, measure recognition accuracy, and identify common recognition errors.
OCR/VLM Performance Evaluation on Complex Nastaliq Script: Evaluate character- and word-level recognition accuracy by comparing model outputs with expert-verified transcriptions of challenging Persian Nastaliq documents, where existing models have shown word-recognition discrepancies of up to 76%.
Cross-Model OCR/VLM Error Comparison: Use the provided per-model error analysis data to compare recognition errors across different OCR and VLM systems and identify weaknesses specific to each model.