A Dataset of Amazigh Printed Words Images

Transcription

1 A Dataset of Amazigh Printed Words Images N. AHARRANE 1, A. DAHMOUNI 2, K. EL MOUTAOUAKIL 3, K.SATORI 4 Abstarct. 1 Student PhD, University Sidi Mohammed Ben AbedAllah aharranenabil@gmail.com 2 Student PhD, University Sidi Mohammed Ben AbedAllah abdellatifdahmouni@gmail.com 3 PA, National school of applied sciences Al-Hoceima yassirkarimimane@gmail.com 4 PES, University Sidi Mohamed Ben AbedAllah khalidsatori@gmail.com In the absence of a public database for a wide-scale benchmarking of Amazigh Optical Character Recognition (OCR) systems, this paper aims to provide a new Amazigh Printed Word Images database (APWID). This database contains 1795 different Amazigh words rendered with an automated procedure using different Amazigh fonts, sizes and styles to generate word images with their ground truth xml files. The database can be used as a source of training and testing sets to evaluate Amazigh OCR systems and also in other applications such as text classification systems and Amazigh characters segmentation. 1. Introduction and motivations In recent years, the OCR remains one of the most popular research subjects due to its diverse applications such as indexing archives, documents analysis, robotics, address classification system, processing of bank check, etc. Therefore, much work has been achieved for many languages, an excellent and recent survey can be found in [Peng et al., 2013]. Recently, due to the introduction of new technologies in communication and after the official introduction of the Amazigh language s teaching in the Moroccan educational system in 2003, researchers have begun to give attention to the Amazigh language trying to provide OCR systems able to achieve better performance. Rachidi et al present an overview of some released works [Rachidi et al., 2014]. Unfortunately, there is no common public database that makes significant the comparison of the elaborated OCR systems.

2 N. AHARRANE, A. DAHMOUNI, K. EL MOUTAOUAKIL, K. SATORI This work has as objective to present a new Amazigh Printed Word Images database (APWID). This database will serve in our research and it is publicly available (see the link in section 3) for the scientific community to enable them to evaluate and compare their OCR systems. The variability present in the data of the APWID database allow a large-scale benchmarking of multifonts, multi-sizes and multi-styles recognition systems of Amazigh words The Amazigh language has existed since the earliest antiquity. It has an original writing system, Tifinagh, used and preserved to this day. In recent decades, all Amazigh groups have reclaimed this ancestral writing. Currently, the Amazigh language is spoken by about 30 million speakers in North Africa (from the oasis of Siwa in Egypt to Morocco passing through Libya, Tunisia, Algeria, Niger, Mali, Burkina Faso and Mauritania). In Morocco, where nearly 50% of people are amazigh, the Amazigh language is divided into three regional varieties with Tarifit in the North, Tamazight in Central Morocco and South-East and Tachelhit in South-West and the High [Ameur et al., 2004]. Figure 1 : Tifinagh characters adopted by the IRCAM. The official introduction of the Amazigh language s teaching in the Moroccan educational system in 2003 involves the selection of a standard common language to teach. This task was accomplished by Royal Institute of the Amazigh Culture (IRCAM) created in 2001 [Sadiqi, 2011]. Actually, the Tifinagh-IRCAM alphabet is based on 33 characters [IRCAM, 2003] as seen in Figure 1. In the Amazigh OCR field, the characters ⴳⴳand ⴳⴳdo not have Unicode codes, so we obtain them by a combination of characters ⴳ, ⴳ with the sign of labialization ⵯ that have Unicode codes. The IRCAM institute has produced a number of Unicode-encoded Tifinagh fonts which are available for free download for all platforms. This variety in fonts allows a large variability in the rendered word images. The rest of this paper is organized as follows: Section 2 delineates all Necessary steps (data collection, rendering procedure, images description) to build the APWID database. In section 3, we present the corresponding statistics of the database and information about storage and usefulness. Finally, we conclude the paper with Section The APWID database This section describes the generation procedure of the APWID database and its specifications.

3 2.1. Data collection A NEW AMAZIGH PRINTED WORD IMAGES DATABASE 3 The database contains 1795 Amazigh words created from decomposable and non-decomposable words. Decomposable words are those generated from Amazigh verbs while non-decomposable ones are formed by Amazigh proper names, days, months, animals, etc. The Amazigh words were extracted from some books such as a French-Amazigh-Arabic dictionary and a Media dictionary published by the IRCAM institute. The collected words were grouped in a text file containing one Amazigh word in each line Sources of variability The images of the APWID database were generated using the 16 different Amazigh fonts proposed by the IRCAM institute (Table 1). We use all these fonts to cover different complexity of shapes of Amazigh printed characters, going from simple fonts with no or few overlaps (Tifinaghe IRCAM STANDARD) to more complex fonts rich in overlaps (Tifinaghe Tazirit UNICODE). We used also different sizes for each font: 8, 9, 10, 11, 12, 14, 16, 18, 20 and 24 points. And for each font and size combination we used different styles: Plain, Bold, Italic and Bold-Italic combination. Table 1: Different Amazigh fonts proposed by IRCAM The used fonts, sizes and styles guaranty a wide variability of the image database.

4 N. AHARRANE, A. DAHMOUNI, K. EL MOUTAOUAKIL, K. SATORI 2.3. Rendering procedure The word images were generated automatically by rendering the word text in images using a java program, so, noise and artifacts present in scanned images are not present in the image database. For each word text, we used different combinations of fonts, sizes and styles and the word image was rendered with the text anti-aliasing filtering implemented in the java standard library by RenderingHints class. The algorithm presented in the Figure 2 describes the followed rendering procedure to generate the APWID database. Figure 2: Algorithm to generate the APWID database 2.4. Ground truth description A detailed description for each image word of the APWID is attached by an XML file reporting ground truth information about the word sequence of characters, as well as information about the image and the rendering settings. Figure 3 illustrates an image word and its attached XML file. Figure 3: Example of the ground truth XML file

5 A NEW AMAZIGH PRINTED WORD IMAGES DATABASE 5 This file is composed of 4 markups to provide all necessary information about the word image such as content, font, image specifications and the generation procedure: Content: this element provides the transcription of the Amazigh word, the number of pieces of Amazigh word (npaws), that are characters, and subelements for each Paw giving its correspondent utf-8 code and its appearance frequency in the word. Font: in this element, we have information about the font used (name, size and style) to generate the word image. Specs: this element indicates the encoding of image, its width and its height. Generation: in this element, we give some additional information about the rendering procedure. 3. Database statistics and utilization This section is devoted to present the APWID statistics and its corresponding information about storage and usefulness Statistics The APWID database contains 1795 words composed from characters and rendered in different combinations of 16 fonts, 10 sizes and 4 styles. Table 2 reports the APWID statistics. Table 2: The APWID statistics Number of words Number of characters fonts * 10 sizes * 4 styles Total As shown in Table 2 the APWID database is composed of word images files, each word image is described by a ground truth xml file. The database contains in total characters, their appearance frequencies in the database are distributed as seen in Table 3.

6 N. AHARRANE, A. DAHMOUNI, K. EL MOUTAOUAKIL, K. SATORI Table 3: Characters appearance frequencies in the APWID database 3.2. Storage The database takes about 900 Mo of disk space and is publicly available for free download via the following link: As shown in Figure 4, the database files are organized in 16 directories representing the 16 Amazigh fonts, each font directory contains 10 other ones for the 10 font sizes and each size directory contains 4 directories for the 4 different font styles. Figure 4: APWID Database structure in disk

7 3.3. Utilization A NEW AMAZIGH PRINTED WORD IMAGES DATABASE 7 The APWID database can be used as training/testing sets in many applications as OCR systems, text classification systems and characters segmentation algorithms. We proposed some protocols as summarized in Table 4 to test the impact of the variability of the data in the database on the tested systems. Table 4: Database testing protocols Protocol Training Set Train (font, size, style) Testing Set Test (font, size, style) APWID1 Train (F01, 14, P) Test (F01, 14, P) APWID2 Train ([F01,F03,F05,F06], [12,14], [P, I]) Test ([F02,F04,F08,F09], [12,14], [P,I]) APWID3 Train([F01,F02], [14,16], [P,I]) Test ([F01,F02], [14,16], [B,BI]) APWID4 Train ([F11-F15], [9,13,20], P) Test ([F11-F15], [9,13,20], P) APWID5 Train (All, All, All) Test (All, All, All) APWID6 Train ([F01, F05, F08, F09], [20,24], All) Test ([F01, F05, F08, F09], [20,24], All) These protocols use the notations Train (font, size, style) and Test (font, size, style) to define the training and testing conditions where: Font: the font ids as indicated in Table 1; Size : defines the sizes used in points ; Style: the style used where P, B, I and BI are for Plain, Bold, Italic and Bold & Italic. The defined protocols have well-defined objectives and are as follow: APTWID1: This is the basic one given that there are no mismatched between the training and testing sets conditions. The performance of the OCR systems should be the highest possible; APTWID2: This one is to test the ability of systems to recognize unseen fonts. APTWID3: This protocol aims to evaluate the capability of systems to treat unseen styles. APTWID4: in this protocol, we measure the systems capability to recognize unseen sizes;

8 N. AHARRANE, A. DAHMOUNI, K. EL MOUTAOUAKIL, K. SATORI APTWID5: This protocol is a global one where all data is used for experimentation; APTWID6: The last protocol is destined to the text classification systems to identify Amazigh text. The database can be used also to test different characters segmentation algorithms in overlapped fonts such as Tifinaghe-IRCAMtaromit2_unicode where the segmentation algorithm using the vertical projection histogram (Figure 5) cannot deal. Figure 5: Characters segmentation by histogram projection [ES Saady et al., 2011] The APWID database users are free to create their own combinations of training and testing sets according to their own needs by benefiting from the variability of the data. 4. Conclusion In this work, we have presented a new Amazigh Printed Word Images Database consisting of different word images and their attached ground truth XML files to provide a common database for a large-scale benchmarking of the OCR systems. The database can be used to create multiple combinations of training and testing sets while benefiting from the wide variability of the database data in term of fonts, sizes and styles. The APWID database is publicly available via Internet. Références Ameur M., Bouhjar A., Boukhris F., Boukouss A., Boumalk A., Elmedlaoui M., Iazzi E. and Souifi H. (2004): Initiation à la langue amazighe, Publications de l'institut Royal de la Culture Amazighe, Manuels N.1, pp. 9. Es Saady Y., Rachidi A., El Yassa M. and Mammass D. (2011): Amazigh Handwritten Character Recognition based on Horizontal and Vertical Centerline of Character, International Journal of Advanced Science and Technology, Vol. 33, pp

9 A NEW AMAZIGH PRINTED WORD IMAGES DATABASE 9 Institut Royal de la Culture Amazighe. (2003) : Proposition de codification des tifinaghes, Rabat, Morocco. Peng X., Cao H., Setlur S., Govindarju V. and Natarajan P. (2013): Multilingual OCR research and applications: An Overview, Proceedings of the 4th International Workshop on Multilingual OCR, ACM, New York, NY, USA. Article No.1. Rachidi A., Eddahibi M., Essaady Y. and Amrouch M. (2014): Amazigh Characters Automatic Recognition: Overview and Prospects, International Journal of Scientific & Engineering Research, Vol. 5, Issue 11, pp Sadiqi F. (2011): The Teaching of Tifinagh (Berber) in Morocco, Handbook of Language and Ethnic Identity: The Success-Failure Continuum in Language and Ethnic Identity Efforts, Vol. 2, Oxford University Press, pp:33-44.