A custom-built deep learning approach for text extraction from identity card images

Abstract

Information found on an identity card is needed for different essential tasks and   manually   extracting  this  information  is  time   consuming,  resource exhaustive  and  may  be  prone  to  human  error.  In  this  study,  an  optical character  recognition  (OCR)  approach  using  deep  learning  techniques  is proposed to automatically extract text related information from the image of an  identity  card  in  view  of  developing  an  automated  client  onboarding system. The OCR problem is divided into two main parts. Firstly, a custom-built image segmentation model, based on the U-net architecture, is used to detect the location of the text to be extracted. Secondly, using the location of the identified text fields, a(CRNN) based on long short-term memory (LSTM) cells is  trained  to recognise  the  characters  and  build  words.  Experimental  results, based on the national identity card of the Republic of Mauritius, demonstrate that our  approach  achieves  higher  accuracy  compared  to  other  studies.  Our  text detection  module  has an  intersection overunion (IOU)  measure  of 0.70 with a pixel  accuracy  of  98%  for  text  detection  and  the  text  recognition  module achieved a mean word recognition accuracy of around 97% on main fields of the identity card.