An End-To-End Deep Learning Framework for Multi-Font Urdu Handwritten Character Detection and Recognition in Unconstrained Images

Abstract

Detection and recognition of handwritten characters is the process of finding the area in an image where text is written and drawing a bounding box around it, with the label containing the text within the bounding box. It has widespread applications, including postal address reading, recognizing Persian vehicle number plates as the Persian language has similar digits to the Urdu language, digitizing and preserving old manuscripts, making handwritten documents in digital format, automatically reading house numbers, and also providing a way for creating apps for kids which will help them in writing and pronouncing Urdu characters properly. Urdu is a cursive language, so it is difficult to apply state-of-the-art handwritten character detection and recognition models developed for other languages to Urdu. The current state-of-the-art techniques cannot handle font style variations, complex backgrounds, illumination, stroke variations, and multiple characters simultaneously due to handcrafted feature extraction. A dataset with such kinds of variations is rare. Very few have used deep learning techniques for Urdu due to the lack of rich datasets. Moreover, the detection and recognition of Urdu handwritten characters are challenging due to variations in stroke sequence, font size, color, style, illumination, background, overlapping, and orientation. YOLO v3 is used for the detection and recognition of Urdu handwritten characters after some fine- tuning. Furthermore, a rich dataset is developed for the training and validation of the proposed technique. The performance of the proposed model is compared with state-of-the-art Urdu handwritten character detection and recognition models. The results show that the proposed technique outperforms the state-of-the-art Urdu handwritten character detection and recognition techniques, both qualitatively and quantitatively.

REFERENCES

[1] M. Bhunia, A. Sain, A. Kumar, and A. Bhunia, “Text is text, no matter what: Unifying text recognition using knowledge distillation,” in Proc. IEEE/CVF Int. Conf. Computer Vision (ICCV), 2023, pp. 19353–19363.

[2] J. Long, Y. Xue, Z. Li, X. Shan, and Y. Yao, “Scene text detection and recognition: The deep learning era with dataset and benchmark survey,” Pattern Recognition, vol. 136, Art. no. 109292, Apr. 2023.

[3] S. Naz, S. B. Ahmed, M. I. Razzak, and I. Siddiqi, “Offline Urdu Nastaleeq OCR: Challenges and solutions,” IEEE Access, vol. 10, pp. 21975–21992, 2022.

[4] M. Shoaib, A. Mehmood, and T. Saba, “A comprehensive survey on Urdu OCR: Datasets, methods, and future directions,” Multimedia Tools and Applications, vol. 81, no. 11, pp. 15423–15458, May 2022.

[5] S. Ali, M. Shafique, and N. Akhtar, “UrduTransNet: A transformer-based approach for offline Urdu handwritten character recognition,” Expert Systems with Applications, vol. 215, Art. no. 119381, Apr. 2023.

[6] U. Imtiaz, “Investigating the impact of phishing attacks on organizational cybersecurity posture,” Spectrum of Engineering Sciences, pp. 1758–1780, 2025.

[7] G. F. Simons and C. D. Fennig, Eds., Ethnologue: Languages of Asia, 26th ed. Dallas, TX, USA: SIL International, 2023.

[8] S. Akter, “Exploring the role of artificial intelligence in food waste reduction: Implications for public health, social behavior, and sustainable communities,” Journal of Computational Systems & Engineering Insights, vol. 1, no. 1, pp. 25–36, 2023.

[9] F. Amin, N. Daudpota, and A. Khan, “A complete penetration testing framework: Simulating attacks and evaluating post-exploitation techniques with Kali Linux and Metasploit,” Spectrum of Engineering Sciences, pp. 386–407, 2025.

[10] S. T. Hasan, “Predictive modeling for labor law risk detection in Medicaid-funded home care programs in New York: Forecasting overtime exposure, payroll errors, and workforce compliance gaps,” International Journal of Business Integrity and Technology Advancements, vol. 1, no. 1, pp. 1–10, 2025.

[11] S. M. H. Shah, F. Amin, and A. Khan, “Cyber-resilient mobile edge computing: A deep neural approach for secure and efficient task offloading,” The Asian Bulletin of Big Data Management, vol. 5, no. 1, pp. 200–215, 2025.

[12] M. A. Alam and M. Ahmed, “Leveraging deep CNNs for efficient Urdu handwritten character and digit recognition,” in Proc. ICICC 2024, Mar. 2025. doi: 10.2139/ssrn.5191580.

[13] U. Imtiaz and H. Elbedour, “Cybersecurity risk management in the digital era: The strategic value of ethical hacking,” Spectrum of Engineering Sciences, pp. 1076–1086, 2025.

[14] S. Chowdhury, A. Pal, and P. Roy, “A survey of handwritten script datasets for South Asian languages: Gaps and opportunities,” ACM Computing Surveys, vol. 56, no. 2, pp. 1–38, Feb. 2023.

[15] S. T. Hasan, “Natural language processing for employee relations case management: Analyzing workplace complaints, grievances, and investigation narratives in healthcare organizations,” Journal of Computational Systems & Engineering Insights, vol. 1, no. 1, pp. 11–24, 2023.

[16] R. Girshick, “Fast R-CNN,” in Proc. IEEE Int. Conf. Computer Vision (ICCV), 2015, pp. 1440–1448.

[17] S. B. Naz, R. Mahdi, I. Siddiqi, and Y. Rauf, “Deep learning-based isolated Arabic scene character recognition,” in Proc. 1st IEEE Workshop ASAR, Nancy, France, 2017.

[18] I. A. Badhan, M. N. Hasnain, M. H. Rahman, I. Chowdhury, and M. A. Sayem, “Strategic deployment of advance surveillance ecosystems: An analytical study on mitigating unauthorized U.S. border entry,” Inverge Journal of Social Sciences, vol. 3, no. 4, pp. 82–94, 2024.

[19] U. Iqbal, “AI-driven predictive maintenance for U.S. smart manufacturing: Deep learning models for equipment failure prediction and operational resilience,” Journal of Engineering and Computational Intelligence Review, vol. 3, no. 1, pp. 114–138, 2025.

[20] G. Jocher, “YOLOv5 by Ultralytics,” 2020. [Online]. Available: https://github.com/ultralytics/yolov5

[21] I. Kate, “Marketing innovation in the US e-commerce sector: Strategies for success in the evolving digital environment,” International Journal of Business Integrity and Technology Advancements, vol. 1, no. 2, pp. 1–7, 2025.

[22] R. Parveen, “Exploring the role of data analytics through digital platforms and IT in promoting transparency and accountability in civil service delivery: Lessons from Bangladesh,” International Journal of Humanities & Social Science Studies (IJHSSS), vol. 11, no. 1, pp. 127–146, 2025, doi: 10.29032/ijhsss.vol.11.issue.01w.015.

[23] A. S. Anwar, S. K. Bhowmik, R. B. Kadir, M. U. Haque, S. Rahman, and I. A. Badhan, “Challenges in implementing machine learning-driven IoT solutions in semiconductor design and wireless communication system,” International Journal on Recent and Innovation Trends in Computing and Communication, vol. 12, no. 2, pp. 872–889, 2024.

[24] U. Iqbal, “AI-enhanced network optimization for electric vehicle charging infrastructure expansion in the United States using graph theory and demand analytics,” Journal of Engineering and Computational Intelligence Review, vol. 2, no. 2, pp. 112–129, 2024.

[25] S. Bhardwaj et al., “Bimetallic Co–Fe sulfide and phosphide as efficient electrode materials for overall water splitting and supercapacitor,” Discover Nano, vol. 18, no. 1, Art. no. 59, 2023.

[26] A. Gupta et al., “A facile approach to recycling used facemasks for high-performance energy-storage devices,” in Specialty Polymers. Boca Raton, FL, USA: CRC Press, 2023, pp. 427–443.

[27] Y. Saeed, K. Ahmed, M. Zareei, A. Zeb, C. Vargas-Rosales, and K. M. Awan, “In-vehicle cognitive route decision using fuzzy modeling and artificial neural network,” IEEE Access, vol. 7, pp. 20262–20272, 2019, doi: 10.1109/ACCESS.2019.2895832.

[28] T. Rahman, I. Ahmad, A. Zeb, I. Khan, G. Ali, and M. ElAffendi, “Performance evaluation of routing protocols for underwater wireless sensor networks,” Journal of Marine Science and Engineering, vol. 11, no. 1, Art. no. 38, 2023, doi: 10.3390/jmse11010038.

[29] I. Ahmed and M. Asif, “The Role of HR in Managing Quiet Quitting and Employee Disengagement in Gen Z Employees of Telecom Sector,” Policy Journal of Social Science Review, vol. 4, no. 6, pp. 118–151, 2026, doi: 10.5281/zenodo.20581688.

[30] M. Asif, M. Abid, and A. Riaz, “Psychological drivers of investment decision making: A multi bias analysis of an emerging market’s retail investors,” Contemporary Journal of Social Science Review, vol. 4, no. 2, pp. 677–688, 2026, doi: 10.63878/cjssr.v4i2.2608.