Automatic road sign detection and recognition based on neural network

Road sign detection and recognition is an integral part of intelligent transportation systems. It increases protection by reminding the driver of the current condition of the route, such as notices, bans, limitations, and other valuable driving information. This paper describes a novel system for automatic detection and recognition of road signs, which is achieved in two main steps. First, the initial image is pre-processed using DBSCAN clustering algorithm. The clustering is performed based on color information, and the generated clusters are segmented using artificial neural networks (ANN) classifier. The resulting ROIs are then carried out based on their aspect ratio and size to retain only significant ones. Then, a shape-based classification is performed using ANN as classifier and HDSO as feature to detect the circular, rectangular and triangular shapes. Second, a hybrid feature is defined to recognize the ROIs detected from the first step. It involves a combination of the so-called GLBP-Color which is an extension of the classical gradient local binary patterns feature to the RGB color space and the local self-similarity feature. ANN, AdaBoost, and support vector machine have been tested with the introduced hybrid feature and the first one is selected as it outperforms the other two. The proposed method has been tested in outdoor scenes, using a collection of common databases, well known in the traffic sign community (GTSRB, GTSDB, and STS). The results demonstrate the effectiveness of our method when compared to recent state-of-the-art methods.


Introduction
Advanced driver assistance (ADAS) systems are designed to improve vehicle safety and driving comfort. One of the This research work was supported by the National Center for Scientific and Technical Research (CNRST), Research Grant No: 20UIZ2015. most significant difficulties facing ADAS is the perception of the landscape and guidance of the vehicles in actual outdoor scenes including pedestrian detection (Preethaa and Sabari 2020;Lahmyed et al. 2019;El Ansari et al. 2018;Lahmyed and El Ansari 2016), vehicle environment perception (Kerkaou and El Ansari 2020; Sudha and Priyadarshini 2020;Kerkaou et al. 2018;El Ansari et al. 2010, traffic sign detection (Ellahyani and El Ansari 2017;Ellahyani and El Ansari 2016;Ellahyani et al. 2016;Liu et al. 2016), and so on. Human driving is an activity that is almost exclusively dependent on visual knowledge, and one of the tasks involved in good driving is to recognize road signs. Otherwise, it can pose a threat of people's lives due to lack of concentration or ignorance. Road signs offer updates on the existing status of the route, limits, bans, alarms, and other important navigation information.
Over the last two decades, the area of road sign detection and recognition systems has attracted substantial research interest. A number of systems have been proposed and imple- Fig. 1 Examples of challenges facing road sign recognition systems mented not only for ADAS, but also for other real-world applications. We mention here automated driving, urban scene understanding, and sign monitoring for maintenance. For such applications, accuracy and fast response time are highly significant metrics. However, in certain realistic situations, the identification of traffic signs is challenging, if not difficult. Some of those situations are illustrated in Fig. 1 and listed below: -Obstacles, e.g., trees, cars, and people may affect the identification of traffic signs (Fig. 1a). -Weather conditions such as snow, rain, and fog and air pollution, make the detection and recognition phases very complex (Fig. 1b). -Color fading: The color of the sign fades with time as a result of long exposure to sunlight, and the reaction of paint to air (see Fig. 1c). -Changes in lighting conditions at various periods (day and night) ( Fig. 1d).
Although impressive research progress has been made in road sign detection and recognition, the problem caused by the diversity of the realistic circumstances listed above persists. Furthermore, the quality of each study's results in this field differs from one research group to another. It is extremely challenging to decide which method produces superior overall results, mainly due to the lack of a standard dataset of traffic sign images. For instance, it is quite impossible to know how effectively the systems adjust to changes in picture illumination since it is not always clear if images with low illumination were utilized in the studies and experiments. Another drawback of the lack of a standardized dataset is that some works are based on small image sets since the compilation of a set of road scene images is a time-consuming process. The disadvantage of using small datasets is that it is hard to assess the results' credibility.
The most significant contribution of this study is the developed framework for road sign detection and recognition that employs multiple techniques such as extraction of color and shape cues as well as applying machine learning algo-rithms. In this paper, a two-stage traffic sign detection and recognition approach is presented. The first stage consists of detecting the traffic signs from the input images, which is achieved in two sub-steps. The first sub-step segments the images to extract ROIs based on DBSCAN clustering and ANN. The DBSCAN clustering algorithm is used to partition the initial image into a set of connected components based on color information. Then, the segmentation stage is carried out by the ANN classifier. The second sub-step verifies if the ROIs provided by the previous sub-step represent traffic signs or not by performing classification on the basis of the HDSO descriptor, which is inspired by the silhouette pattern of the road sign, together with ANN classifier. The second stage is performed by developing an extension of the GLBP feature to RGB color images that we name GLBP-Color. It is combined with the LSS to define the hybrid feature that we propose to adopt for the traffic recognition method.
The structure of this paper is organized as follows. Section 2 presents state-of-the-art road sign detection and recognition. The proposed traffic sign detection and recognition is described precisely in Sect. 3. Experimental results to assess the performance of the proposed approach are shown in Sect. 4. Section 5 concludes the paper and presents an outlook on further possible improvements.

Related work
Several different approaches to traffic sign detection and recognition have been suggested in the literature. Thus, the result of the research obtained differs from one group to another. Comparing and determining the best among these systems require more effort since they are based on nonavailability of standard dataset, which makes the result less reliable. Among the dataset used in the field, there is German Traffic Signs Dataset ( Regarding the purpose property of the dataset, the traffic sign algorithms could be divided into three major categories of methods: (1) methods aimed to detect the road sign in the image (Ellahyani and El Ansari 2017;Lillo-Castellano et al. 2015;Ruta et al. 2010;Bouti et al. 2019), (2) methods intended to recognize the sign class (Salti et al. 2015;Zaklouta and Stanciulescu 2012;Miura et al. 2000), and (3) methods designed to both detect and recognize the traffic sign simultaneously (Ellahyani et al. 2016;Zhu et al. 2016;Yang 2013).
In the first category, different road sign detection approaches have been introduced where two directions are defined. Here, we refer to color-based and shape-based methods. Some relevant works in the first direction propose to carry out the color segmentation in various color spaces including RGB (Red Green Blue) (Ruta et al. 2010), HSV (Hue, Saturation, Value) (Lillo-Castellano et al. 2015), YUV (Luma, Blue projection and Red projection components) (Miura et al. 2000), and L-a-b (Lightness axis, a-axis "green to red," b-axis "blue to yellow") (Lillo-Castellano et al. 2015). Li et al. Ruta et al. (2010) adopt the color enhancement method to segment red, yellow, and blue colors to detect road signs using the RGB color space. The same space has been also used in the work (Benallal and Meunier 2003). The authors have employed the difference between red and blue, and the difference between red and green channels to build two stable features in road sign detection. Besides RGB space, other approaches choose to employ other color spaces. For example, Both L-a-b and HSI systems are considered in Lillo-Castellano et al. (2015). They are utilized to extract candidate blobs for chromatic signs. Miura et al. (2000) performed their system in YUV space to detect blue rectangular signs.
In the second direction, some works suggest utilizing geometric information to identify and detect traffic signs. Typically, these shape-based techniques are used either directly on road scene images or as a second phase after color segmentation. Bascón and Rodríguez Bascón et al. (2010) proposed a function of the angle defined as the distance from the blob center to its edge to classify the blobs as squares, triangles, or circles. In Ellahyani and El Ansari (2017), a method for road sign detection based on mean shift clustering algorithm, random forests classifier, and logpolar transform technique is described. Dariu M. Gavrilla et al. Gavrila (1998) employed the techniques of Distance Transform (DT) and Template Matching (TM) to localize triangular and circular signs.
In the second category, numerous published studies have taken road sign recognition as the main problem, where different features and classifiers have been exploited including HOG, LBP, Haar-like wavelet, local self-similarity (LSS), Gabor filters, SVM, and ANN classifiers. For instance, the authors in Zaklouta and Stanciulescu (2012) utilized features combination of various sized HOG together with Fisher Linear Discriminant feature space reduction algorithm to label the road sign according to the information included in its pictogram. For the same purpose, Salti et al. (2015) suggested using the HOG features with SVMs classifier in the classification phase. A normalized correlation-based pattern matching using a road sign dataset to determine the content of the traffic signs is proposed by Miura in Miura et al. (2000). However, these methods tend to perform poorly in real scenarios, since they are often applied after detecting and localizing the traffic signs.
For the sake of alleviating this problem, some relevant works suggested combining the two categories into one (The third category) to design systems for detecting and recognizing traffic sign in urban scenarios. On this basic concept, various approaches have been proposed such as the one in Ellahyani et al. (2016) that applies invariant geometric moments to classify shapes and HSI-HOG combined with LSS to recognize the traffic signs. The same authors proposed another system (Ellahyani and El Ansari 2016), in which the Distance to borders (DtBs), HOG, LSS and random forests classifier are used. In the same context, a method for road sign images is proposed using CNN in Yang (2013). However, a very high cost of computation (time and hardware) is required. In Zhu et al. (2016), the authors proposed two convolutional neural networks (CNNs) system in this field. The first one is used for the detection alone, while the second one is utilized for simultaneous detection and classification purposes. The proposed system shares the same difficulties that face the one published in Yang (2013).
In spite of the achievements obtained, it is an unfortunate duty to report that most of the above-motioned systems are inherently restricted by non-accurately predicted responses under certain circumstances such as weather conditions, disorientation of the sign, and different illumination levels. We alternatively present in this paper a novel method for road traffic detection and recognition. The proposed method is an amelioration of the methods presented recently in Refs (Ellahyani et al. 2016;Ellahyani and El Ansari 2017;Ellahyani et al. 2018). It is developed based on machine learning techniques to carry out the segmentation, and a silhouette pattern of road sign descriptor for shape classification. It gives better results compared to the ones proposed in Ellahyani et al. (2016), Ellahyani and El Ansari (2017). Moreover, the recognition phase is performed on the basis of integrating the color information into the GLBP features by using the RGB components to compute the descriptor instead of grayscale images. The computed GLBP-Color is combined with LSS features to create a novel descriptor. These features were provided to the ANN classifier to recognize the traffic sign.

Proposed method
In this section, a novel approach for detecting and recognizing traffic signs is presented. As schematized in Fig. 2, the proposed approach is achieved in two main stages. In the first stage, we aim to detect traffic signs by employing color, shape cues, and machine learning techniques. The second one also uses the color information along with texture, gradient and internal geometric layout of local self-similarities information for features computation to identify the information included in the provided traffic signs by the first step. Here, we detail each of the aforementioned steps of the proposed method.

Detection
The first stage of the proposed system consists of two substeps: (1) Segmentation where the locations of possible traffic signs (ROIs) in natural scene images are determined and (2) shape classification, where tests are performed to verify the presence of road signs in the generated ROIs. The details of each of these sub-steps are given in this section.

Segmentation
Despite the fact that the road signs are distinct from each other, there are several similar properties for signs under one target category. For instance, triangular red borders characterize danger signs, mandatory signs are known by white arrows and blue backgrounds, derestriction signs have backgrounds with white color, prohibitory signs have circular borders colorized by red. Therefore, color segmentation procedure is adopted since it aims at detecting white, red, and blue colors. It is only used for extracting the ROIs (candidate traffic signs) rather than performing road sign detection due to many factors such as the presence of some objects with the same color as traffic signs in the road scene and the change of weather that may affect the color segmentation process. The colorbased segmentation is done by performing machine learning techniques on the input image.  First, the image pixels are clustered into a set of groups. The clustering procedure is performed using the densitybased spatial clustering of applications with noise (DBSCAN) algorithm (Ester et al. 1996) that groups together pixels with many nearby neighbors and considers pixels whose nearest neighbors are too far away as outliers. It is one of the most efficient density-based clustering techniques, which can withstand noise, densities, and shapes as well. Its functionality entirely relies on two parameters: (epsilon) the radius from a corresponding pixel ( pi x corre ), which includes neighboring pixels, and Min pxls is the minimum number of pixels needed to form a cluster.
The main concept of the DBSCAN algorithm is based on the idea of creating a cluster if and only if a pixel ( pi x) has nearly enough neighbors within the radius . Otherwise, pi x is labeled as noise (outlier).
The procedure begins by choosing an arbitrary unvisited pixel in the image. Taking the pixel and the value of input , Min pxls within the region formed will be verified. If the total number of pixels in -neighborhood of pi x is equal to or exceeds the input Min pxls value, then it will build a cluster. Pixels lying outside the cluster form noise pixels. If a new pixel is added into a cluster, implies that its neighbors within -distance are all added to the same cluster as well.
We continue to build further clusters by the same manner. Once the clusters are obtained, the ANN classifier carries out the segmentation step. Here, all pixels of each cluster are fed to the ANN classifier to determine the color components (CC) that they represent (see Fig. 3). Once the process is done, the pixels will participate in the vote to identify the color of the cluster by choosing the CC with the highest value of vote. The above process is repeated until all clusters colors are identified. Figure 4 depicts an example of the segmentation results obtained when the proposed approach is applied on a road scene image. The segmentation is illustrated with a binary image where the ROIs are represented with white pixels. The ROIs we get from the segmentation phase are mapped into their corresponding regions in the initial image. Each ROI in the binary image generates a ROI in the corresponding visible image (Fig. 5).
To ameliorate the detection approach performance, we discard the insignificant ROIs based on their sizes and aspect ratio.
A ROI is taken in consideration if: -The aspect ratio is between 1/2.03 and 2.03.
-The size is between w × h/27 and w × h/2.6, where h and w are the height and the width of the image sample, respectively.
The aspect ratio and the size are selected empirically regarding the collected data from German Traffic Sign Detection Benchmark (GTSDB) and Swedish Traffic Signs (STS). Figure 6 represents an example of the ROIs obtained in the image before and after discarding the insignificant ROIs. We can remark clearly from Fig. 6b how the number of maintained ROIs is decreased.

Shape classification
Once the ROIs are generated (candidate road signs), they are provided to the classification module to classify them as road sings or non-road signs. In this section, the approach utilized to classify the provided ROIs according to their shapes is detailed. Generally, three shapes are assumed to be detected as traffic signs. The shapes are rectangle, triangle, and circle. In this work, we refer to histogram of dominant silhouette orientation (HDSO) descriptor together with ANN classifier to perform this classification.
Motivated by its success in the field of pedestrian detection (Lahmyed et al. 2018) and inspired by the silhouette pattern of road sign as well, HDSO features is adopted to recognize road signs shape. The descriptor procedure requires three primary phases: The first phase, silhouette edge extraction, defines the silhouette of the road sign from the input ROI. The second phase of the proposed algorithm, polar transform, uses the polar representation of the coordinates instead of the Cartesian one. The last step, histogram computation, handles the procedure for HDOS feature vector extraction. The main three subsequent phases of HDSO descriptor are illustrated in Fig. 7. More details are listed below.
a. Silhouette edge extraction To compute HDSO features and as mentioned above, we first start by silhouette edge extraction step using Canny operator (Canny 1987). Once it is done, the silhouette edge where N is the total number of silhouette pixels. Then, we calculate the orientation ω p at every edge point p(x, y) according to the following equation: where the gradient of the sample image along the directions x and y are utilized.

b. Polar transform
To explain efficiently the road sign form, polar representation is utilized. The Cartesian coordinate system is converted into the corresponding polar coordinate system using Eqs. 2 and 3 and where θ i represents the orientation relative to the silhouette centroid c(x c , y c ), and r i signifies the Euclidean distance between p i and c. Once the computation is done, three information must be assigned for each p i and they are recorded as follow: c. Histogram computation Descriptor computation begins with the creation of a log-polar histogram that represents dominant silhouette orientations distribution under consistent partitions around the silhouette's center c(x c , y c ) (Fig. 7c). We partition the silhouette edge area into K cells by uniformly partitioning r max (maximum radius) into n components that indicate the circles number, and angles into d directions such that K = n × d. Then, for each cell K i = e × f (where e = 0, 1, 2, ..., n − 1 and f = 0, 1, 2, ..., d − 1), we build a histogram H e× f with 12 orientations spaced over 0 • to 360 • as follows: (4) where the ranges of θ j and r i are defined by 2πv m ≤ θ j ≤ 2π(v+1) m and u n r max ≤ r i ≤ u+1 n r max , respectively. In order to globalize the description of HDSO, we select the dominant orientation β e× f of the histogram from each cell according to the following equation: The maximum value β e× f would be taken as the cell e × f feature value. The above process is straightforwardly repeated until the dominant orientation is computed for each cell K i of the built histogram. The resulting feature descriptor HDSO is constructed from a vector containing all cells values with a size of n × d.
After computing the descriptor, it is fed to the ANN classifier to classify the provided ROIs into appropriate shapes. An overview of ANN classifier is presented in subsection 3.2.2.

Recognition
Once the candidate blobs are classified into a shape class, the recognition procedure is taken place in order to identify the sign. In this section, we describe how the proposed hybrid feature is created. It is a combination of two features: The first one is proposed based on GLBP feature by involving RGB color information (named GLBP-Color) and the second one is picked among the features that have good performance in the road sign representation (e.g., HOG, LSS, and Gabor). Among the Gabor, LSS, and HOG, we look for the one that succeeds in improving the performance of the recognition results when it is combined with the GLBP-Color.

Features extraction a. GLBP feature
The first feature involved in our experiments is the GLBP (Jiang et al. 2012). It is a version of the LBP (Ojala et al. 1996) descriptor based on the idea of combining two types of information (texture and gradient) to construct powerful and even more discriminative attributes. Its main concept consists to exploit the uniform LBP patterns to compute the histogram of oriented gradients in order to reduce the effect of the noise on the recognition results. The GLBP histogram dimension is defined by all possible width and angle values. Precisely, eight potential angle values or Freeman directions are available; meanwhile, the value of "1" in the uniform patterns varies from one to seven. This yields a GLBP histogram of 7 × 8 in which gradient attributes are accumulated. After computation, the normalization (L2-norm) is taken place to derive the GLBP histogram within the image.
b. GLBP-Color feature Computation of the GLBP descriptor in the work of Ning Jiang. Jiang et al. (2012) begins by converting the input color image into a grayscale one for simplifying the computations. However, this preprocessing step affects badly the descriptor quality. It discards all color information and leaves only the luminance of each pixel. Hence, we are persuaded that computing the GLBP from color images instead of the grayscale ones could improve the quality of the recognition and classification as well.
In this paper, we propose a new approach to compute the color-based GLBP feature. The computed GLBP from the blue (B), red (R), green (G) components using our approach, so-called GLBP-Color, has the same size as the classical GLBP (Jiang et al. 2012), and has also better performance of the classification and recognition compared to the one introduced in Jiang et al. (2012).
At each given image pixel p with the coordinates (i, j), we refer to its B, G, and R value components by B(i, j), G(i, j), and R(i, j), respectively. We begin first by computing the bit binary code (B BC) of p by comparing its value with those of its 8 neighbor pixels one by one at each blue, green and red components. We denote the computed bit binary code of p from B, G and R components by B BC B ( p), B BC G ( p) and B BC R ( p), respectively. The three computed values are used to compute the final bit binary code of the pixel p according to the following formula: X O R in Eq. 6 signifies the exclusive disjunction logical operation.
Once B BC( p) is checked as a uniform pattern, we compute two parameters. The first parameter is the width (ω). It represents the number of occurrences of "1" in the binary code. The second parameter is angle (θ ). It is the direction code of the middle pixel in "1" area of its binary code. Both θ and ω parameters are employed for mapping the position of bin in GLBP-Color histogram.
We then directly compute the gradient components (G x , G y ) using the following equations: and and CC signifies the color component which could be either B, G or R. Once the G x and G y are identified using Eqs. 9 and 10, respectively, the magnitude values at the pixel p(i, j) can be computed as: The magnitude value is used as weights for voting by following the same steps as the classical GLBP.
c. Gabor feature The second one is the Gabor feature (Daugman 1985) which is a set of band-pass filters that have been employed in different computer vision, pattern recognition, and signal processing problems, including texture analysis, due to their d

. HOG feature
The third feature adopted in this work is the HOG feature. It was first introduced by Triggs and Dalal Dalal and Triggs (2005) for pedestrian detection. It is extensively adopted in various image processing and computer vision problems for detecting objects. The main concept of the descriptor is that the shape and the appearance of an object can overwhelmingly be described rather well by the distribution of the local intensity gradients or edge directions, even without having a precise knowledge of the corresponding edge or gradient positions. Its computation is done on an intensive grid of uniformly spaced cells and the accuracy is improved using overlapping local contract normalization.
e. LSS feature LSS is the last feature involved in this work. It presents a local self-similarity description operator for object detection, first proposed by Irani et al. in Shechtman and Irani (2007). The basic idea behind the LSS is to capture the internal geometric layout of local regions and compare it throughout the images. This indicates that the input image is partitioned into small cells of the same size. Then, they are compared to a patch located at the center of the sample. The resulted distance surface is normalized and projected into a log-polar representation divided by radial and angle intervals. The feature value is regarded as the extreme value in the interval space.
f. Hybrid-based feature model Newly, feature combination has been adopted exceedingly in different object detection and pattern recognition fields. It has become one of the most powerful alternatives in different complex systems including road sign recognition (Ellahyani et al. 2018(Ellahyani et al. , 2016. Thus, we believe that our road sign recognition system can be enhanced by taking advantage of different discriminant information such as local information (self-similarity of color, edges, repetitive patterns) and entire information (texture, edge direction, color, and shape) of images. In our paper, two combinations to build novel hybrid-based feature models are studied. These combinations are: GLBP-Color+Gabor, and GLBP-Color+LSS.

a. SVM
SVM is a supervised learning model designed by Vladimir N. Vapnik and Alexey Ya. Chervonenkis Vapnik and Vapnik (1998). The fundamental principle of this classifier is to convert the input vectors by a nonlinear transformation into a higher-dimensional space, and then find a hyperplane that separates the results. The found hyperplane (H ) should have the greatest potential for generalization and isolate the negative samples from the positive ones. As depicted in Fig. 8, the training dataset which belongs to two classes is presented by black and white circles. The hyperplane (H ) that separates the negative samples from the positive ones is formed, in which the margin between the closest negatives and positives is maximal. The data located on the boundaries (H 1 and H 2) of the two classes are named support vectors. SVM was first used to solve problems of binary classification. It is, moreover, often used to solve multi-class problems, such that it is done by combinations of the binary classification problems.
b. AdaBoost AdaBoost "Adaptive Boosting" was introduced by Freund Yoav et al. in Freund et al. (1999). It has been applied to different scientific fields by many researchers and has achieved good results. It consists of producing a strong classifier from c. ANN ANNs have attracted great attention in machine learning due to their efficiency in difficult, complicated, multivariate nonlinear fields, such as road sign detection and recognition (Islam and Raj 2017; Saha et al. 2012). It is a powerful and versatile tool that is capable of capturing and representing complex input/output relationships, and there is no need to assume an essential data allocation such as the ones usually done in statistical modeling. Broadly, the ANNs classifier consists of a series of simulated neurons operating in parallel, as one or multilayer that are often composed of three layers: input, hidden, and output layer (see Fig. 9).
The function of the second layer (hidden) is to interact in any suitable way between the external input (vector samples) and the network output. For ROIs classification in both detection and recognition phases, we used Feed forward multilayer neural network (F F N N ) model which is used mainly to classify inputs into a set of target categories based on feature selection parameters (Madadlou et al. 2009). Furthermore, two types of signals are defined in this network (function and error signals). The first one, which is also acknowledged as the input signals, that are fed to the input of the F F N N network, propagate forward through the network (neuron by neuron), and reach the network output end as output signals. The second signal originates at the output neuron of the network and propagates backward (layer by layer) via the network. The neural network's output can be expressed according to the following equation: where W 0 j is the synaptic weights from neuron in the second layer to the single output neuron. F h (resp. F o ) represents the activation function of the neurons from the hidden (resp. output) layer. X j signifies the ith element of the input vector, and W ji is the connection weights between the neurons of the second layer and the inputs.

Experimental results
To evaluate the efficiency of the proposed traffic sign detection and recognition method, we carry out a series of comparative experiments using the three public datasets (GTSDB, STS, GTSRB) presented in Sect. 4.1. The obtained results clarify the contribution of each component of the proposed approach along with the entire approach using a 2.40 GHz Intel i5 processor.

Datasets
As mentioned above, three publicly available datasets have been used to assess system performance. The datasets are German Traffic Sign Detection Benchmark (GTSDB), Swedish Traffic Signs (STS), and German Traffic Sign Recognition Benchmark (GTSRB). These datasets were col- The GTSDB dataset contains 900 full images. Those images are divided into two sets. The first one with 600 images is used for the training phase while the second one (300 images) is utilized for the testing phase.
The STS dataset provides 20000 images with 20% labeled. The images are captured from Swedish highways and cities, and they contain more than 3400 traffic signs.
The GTSRB dataset offers more than 50000 German traffic signs images in total, divided into 43 classes. The images format is 24-bit color PPM and their size is varying from 15 × 15 to 250 × 250 pixels. Figure 10 illustrates the GTSRB dataset classes, which have been partitioned into six subsets (speed limit, derestriction, mandatory, danger, other prohibitory, and unique signs).
Both first and second datasets are used to evaluate the performance of the detection phase once the images are normalized to 640 × 480 pixels using bilinear interpolation. In the recognition phase, we adopt the last dataset for evaluation, which will permit us to easily compare our approach with other state-of-the-art methods.

Parameters setting
In this section, the effect of the parameters involved in the different steps of the proposed system is investigated. The parameters are chosen empirically using some sample images from GTSDB, STS, and GTSRB datasets.
In the color segmentation process, two parameters of DBSCAN clustering technique were used: Min pxls and (epsilon) the radius from a corresponding pixel, which includes neighboring pixels. Figure 11 depicts the number of true positives (T Ps) obtained and the corresponding computational time while changing the value of these two parameters over more than 250 images chosen from the GTSDB and STS datasets. Note that a correct detected road sign is counted T P if its corresponding bounding box overlaps with at least 50% of the area covered by the road sign present in the image. As shown in Fig. 11, both parameters have a noteworthy influence on the obtained results. Min pxls = 300 and = 3 are chosen since they guarantee high accuracy (more than 1455 T Ps) while the consumed time is at its lowest value (less than 12 ms).
The ANN parameters are also obtained empirically using data from GTSRB, GTSDB, and STS datasets on both detection and recognition phase. To achieve the optimum parameters of the ANN used in the proposed method, a procedure based on cross-validation experiments has been used. The datasets mentioned above are partitioned into two subsets (training and validation). The ANN classifier on the two subsets is trained and evaluated using various features and the parameters that optimize the accuracy of validation are chosen. Using the selected parameters, the classifier was retrained one more time on the training dataset. We remark from Fig. 12 that the classification accuracy enhances with the number of nodes (N N odes ) and becomes constant once this number reaches a specific value. Here, detection and recognition stages hit the highest accuracy score when N N odes = 18 and N N odes = 9, respectively. Thus, the value N N odes = 18 has been adopted to be the number of nodes in ANN classifier. The ANN parameters used for the training in both detection and recognition stages are summarized in Table 1.
The parameters of the features utilized in the proposed system are also obtained from cross-validation experiments. Using the selected features parameters, the classifiers were retrained one more time on the training dataset. Here, the GLBP-Color, HOG, LBP, LSS, and HDSO features are computed as follows:  image region equal to 40 × 40 pixels have been adopted and the log-polar coordinates are divided into 4 radial intervals and 20 angles. -The number of angles (m) and the number of circles (n) are the parameters for HDSO features. The values of both parameters affect detection efficiency. As depicted in Fig. 13, the highest results scores are obtained when m = 36 and n = 4. Therefore, the values 36 and 4 are adopted for m and n, respectively. Figure 14 illustrates an example of the results provided by the proposed module at its main stages applied to a sample image captured by the visible camera. First, the input image (Fig. 14a) is segmented using the DBSCAN algorithm The bold values signify the results obtained by the proposed approach together with the color information (Fig. 14b). The resulting clusters are projected on the image to obtain their corresponding ROIs (Fig. 14c). Some of these resulting ROIs are eliminated according to their aspect ratio and size. This procedure speeds up the detection since the number of ROIs to be treated is reduced (Fig. 14d). The segmentation approach retrieves the road sign present in Fig. 14a along with some other undesirable ROIs that they do not represent any traffic signs. To validate the detected ROIs, a shape classification procedure should be applied. Figure 14e depicts the traffic sign detection results when the combination between HDSO and ANN are used as a feature and classifier, respectively, on the obtained ROIs. To identify the detected traffic sign, a green bounding box is used. Once the road sign is detected, it is provided to the recognition module to identify which class (a)  it belongs to. Figure 14f represents the recognition results using ANN as a classifier and a color-based GLBP together with LSS as features. The computational time of each of the stages involved in the detection and recognition process is depicted in Table 3.

Results
To demonstrate the effectiveness of the proposed approach, we analyze and compare the obtained results from each step.
We first evaluate the proposed detection module and compare the obtained results with other reported works. Second, we assess the performance of the proposed recognition approach and we perform a comparison with the state-of-theart approaches on the GTSRB dataset. The evaluation of the detection and recognition stages is carried out on the basis of recall, precision, F-measure, and accuracy, where they are computed as follows: As depicted in Tables 4 and 5, the proposed detection method yields the scores with a precision of 95.83% at a recall of 94.22%, and a precision of 96.07% at a recall of 94.89% in GTSDB and STS datasets, respectively. Figure 15 a and b shows the precision-recall curves of the proposed approach when applied to GTSDB and STS datasets, respectively. The AUC of the two ROC curves is 95.76% and 96.67%, respectively. The bold values signify the results obtained by the proposed approach The bold values signify the results obtained by the proposed approach The proposed detection approach has been compared to other reported methods in order to assess its performance using the GTSDB dataset. The methods as well as their results in terms of precision, recall, and F-measure are listed in Table 6. We notice from this table that our method outperforms the ones introduced in Ellahyani and El Ansari  Bascón et al. (2010) and Yang et al. (2013) for which the F-measure scores are 93.50, 90.97, 90.59, 88.73, 70.70, 65.28, 65.07, 62.66, 54.57, and 46.42, respectively. More detection results are depicted in Fig. 16. The clustering results we obtain when we apply the DBSCAN algorithm to the test image are illustrated in Fig. 16 b. The gener-ated ROIs after mapping the resulting clusters on the image (Fig. 16a) with and without taking into account the aspect ratio and size are shown in Figure 16d and c, respectively. To validate the detected ROIs, a classification procedure was performed on the basis of the HDSO features and ANN classifier. The classified ROI is illustrated in the test image by green bounding boxes in Fig. 16e.
To evaluate the recognition module, GTSRB dataset has been used. Here, five evaluations have been included to prove the relevance of the proposed GLBP-Color. We first evaluate the performance of GLBP-Color and compare it with the classical GLBP (Jiang et al. 2012) and some other LBP versions (RGB-LBP (Banerji et al. 2012) , ALBP (Liu et al. 2014), and tLBP (Trefnỳ and Matas 2010)). In the second evaluation, we compare the performance of GLBP-Color with some  In the fourth evaluation, we try to combine the proposed GLBP-Color together with Gabor and LSS features to look for possible improvements. The last evaluation compares the results provided by ANN and those obtained by the SVM with radial basis function (RBF) kernel and AdaBoost classifiers. Table 7 represents the accuracy values obtained while using the proposed GLBP-Color, grayscale, GLBP, LBP, RGB-LBP, ALBP, and tLBP to the GTSRB dataset which is composed of six subsets (see Fig. 10). It is clear from Table 7 that the GLBP-Color outperforms all the other descriptors in terms of accuracy in all subsets. Its corresponding accuracy of all traffic signs is 97.97%, which has been improved by 2.07%, 2.01%, 1.97%, 2.08%, and 0.74% when the new GLBP-Color is compared to tLBP, ALBP, RGB-LBP, LBP, grayscale GLBP, respectively. Table 8 lists the results obtained while using the proposed GLBP-Color, HOG, LSS, and Gabor features. We remark from the table that among the four features, the GLBP-Color feature still performs better than the HOG, LSS, and Gabor features.
The same remark could be seen on the basis of the results listed in Table 9, when we compare GLBP-Color with some other color descriptors using ANN classifier.
With the aim to enhance the performance of the proposed GLBP-Color, it has been combined with other features, i.e., Gabor and LSS. Table 10 indicates the results obtained by the GLBP-Color alone, the combination of GLBP-Color and Gabor and the combination of GLBP-Color and LSS. We remark that the combination of GLBP-Color and LSS succeeds in improving the accuracy in all GTSRB datasets subsets. It provides 98.22% in terms of accuracy of all subsets. Therefore, the combination GLBP-Color and LSS adopted as the main feature vector in the recognition stage of the proposed method.
In order to justify the use of the ANN as a classifier, a comparison with the SVM with basis function (RBF) kernel and AdaBoost classifiers is made. Table 11 indicates the results given by the three classifiers when they are feed with all features already used. To obtain optimal design parameters of SVM classifier, we also run some cross-validation experiments on the training dataset to select the setting of maximum validation accuracy. The SVM parameters used in this comparison are: C = 5 and G = 0.07. It is obvious from the results that the ANN outperforms the other classifiers. Thus, the ANN is chosen in the proposed recognition method as a classifier.
A comparison versus some state-of-the-art works is given to evaluate the performance of the proposed approach. The  The bold values signify the results obtained by the proposed approach The bold values signify the results obtained by the proposed approach The bold values signify the results obtained by the proposed approach The bold values signify the results obtained by the proposed approach    , respectively, and the computational time is very low when we compared to CNNs-based approaches To assess the performance of the entire traffic sign detection and recognition system, the GTSDB dataset will be adopted. Figure 17 shows the precision-recall curve obtained by the whole proposed method. As illustrated in this figure, the value of AUC, which is mathematically acknowledged as the integral of the curve over the precision and the recall labels, obtained is 95.17% on an average run time of 8-10 frames/second. Figures 18,19,and 20 present some examples of the detection and recognition results when the proposed approach is applied to sample images. As can be seen in Fig. 18, the system successfully detects recognizes the traffic signs included in the two images. The traffic signs presented in Fig. 19 have been successfully detected. However, they could not be recognized well by the reason of the motion blur in the signs. In Fig. 20, the road sings could not be detected due to different reasons. The ROIs corresponding to road sings presented in the images were not extracted by the segmentation approach

Conclusion and perspectives
This paper presents a two stages system for real-time road sign detection and recognition. The first step performs the detection on the basis of color and shape cues. A clustering technique is carried out on the initial image to form a set of connected components. The resulting clusters are provided to the ANN classifier for segmentation according to their color. The obtained ROIs (possible traffic signs) are then processed based on their size and aspect ratio to keep only significant ones. Then, we refer to HDSO feature and ANN classifier to detect circular, triangular, and rectangular shapes on the resulting ROIs. In the recognition step, we combine the so-called GLBP-Color with LSS feature to form a new descriptor. This descriptor is then used with the ANN classifier to identify the signs from the detected ROIs. Results obtained on the public GTSDB, STS, and GTSRB datasets justify the effectiveness and robustness of the proposed method.
In future work, we are intending to enhance the quality of the results obtained by the proposed method in both detection and recognition phases. We aim also to use other machine learning techniques to accelerate the classification procedure, and improve the robustness of the system.

Declarations
Conflict of Interest All the authors have no conflict of interest to declare.
Ethical approval This article does not contain any studies with human participants or animals performed by any of the authors.