Tracking Large Trucks in Real-Time with License Plate. Recognition and Text-Mining Techniques

Transcription

1 Tracking Large Trucks in Real-Time with License Plate Recognition and Text-Mining Techniques Original Manuscript Submitted: ormally Francisco Moraes Oliveira-Neto Ph.D. Student Department of Civil & Environmental Engineering University of Tennessee 223 Perkins Hall Knoxville, TN Tel: (865) Manuscript to be considered for the Tennessee Section Institute of Transportation Engineers Student Paper Competition

2 ABSTRACT Large-truck speeds on interstate highways are not only a safety concern, but have now also become a problem with respect to air quality and energy consumption. Many cities have reduced truck speed limits in order to curb emissions. As potentially key building blocks of a proposed automated large-truck speed monitoring and enforcement system, license-plate recognition (LPR) and plate-matching algorithms are at the heart of this study. Present LPR systems are not perfect and can fail to read 1/5 to 1/2 of the characters on license plates, depending on various factors. Fortunately, even when LPR fails to read all of the characters on license plates, it is still able to read most of the characters with an appreciable degree of accuracy. By employing a text-mining technique called Edit Distance (with judicious use of travel-time) this study demonstrates that it is possible to match 97% of license plates when only just over 60% of the same plates were read correctly by LPR units at two different locations. This high matching rate does not entail a high false-positive matching rate, either (about 2%). This paper presents the plate-matching algorithm in detail and provides statistics resulting from a field study conducted on I-40 in In the conclusion, several promising research directions for better matching efficiency and further reduction of the false-positive rate are identified. KEYWORDS: Truck Monitoring, License-Plate Recognition, Data Mining, Text Mining, Enforcement, Air Quality, Speed Limit, Weigh Station, Truck Inspection, Highway Patrol, Real-Time Operations PAPER CONTENTS: 1. INTRODUCTION 2. MATCHING PLATES WITH EDIT DISTANCE 3. MATCHING METHODOLOGY 4. CASE STUDY AND RESULTS 4.1. LPR Performance 4.2. Truck Speed 4.3. ED Performance Results 5. DISCUSSION 6. CONCLUSIONS 2

3 1. INTRODUCTION In April 2006, Knoxville, Tennessee joined an increasing number of cities in reducing speed limits for large trucks (with gross weights over 10,000 pounds) on the interstate highways in its metropolitan area. In recent years, reducing large-truck speed limits in urban areas has become one of the preferred countermeasures for combating urban air-quality problems. The rationale for this is supported by a 2003 Federal Highway Administration (FHWA) study, which found that reducing largetruck speed by 16 km (or 10 mph from 65 to 55 mph) can reduce emissions of NOx by 18% per large truck (1). While reducing truck speed limits is a relatively simple act for metropolitan planning agencies, the enforcement side of it often meets with more challenges. This is the case for Tennessee Highway Patrol (THP), which has jurisdiction over Interstates 40 and 75 (I-40 and I-75), which both pass through the Knoxville metropolitan area. After the new speed law was enacted, THP found itself facing 12 million large trucks, most of which were going faster than the 88 km/hour (55 mph) speed limit, on this stretch of interstate; a complicating issue for the THP was that it was provided no annual budget or manpower increases for the purpose of enforcement. The aforementioned FHWA report did not state this specifically, but it would be difficult to expect emission reductions if the new speed limit were not diligently enforced. To this end, the University of Tennessee proposed the use of license-plate recognition (LPR) equipment to automatically track large trucks as they cross through the metropolitan area. Taking advantage of the existence of a weigh station on the western end of the area on I-40 where all trucks are required to stop for inspection, an LPR speed-enforcement system, with equipment strategically located 3

4 along I-40, could issue warnings or citations even while the perpetrating trucks stop on the weigh scale, with a THP officer stationed in the weigh house. This system would function in real time without the need for mailing out speeding tickets after the fact or pulling trucks over after dangerous high-speed pursuit, both of which alternatives are resource- and labor-intensive. License-plate recognition technology was originally developed to read licenseplate characters on moving vehicles. The process of capturing a plate image and recognizing the characters involves vehicle detection, image processing, and optical character recognition, which have all been documented in detail in past studies (2, 3). As Han et al. have pointed out, all LPR systems take advantage of basic patternrecognition technology to identify the alphanumerical characters on license plates (4). The concept of an LPR-based speed enforcement system is alluring: with simple installation of LPR units, real-time truck monitoring seems easily attainable. The reality is not so simple, and, hence, the potentialities of LPR are not quickly realizable. Depending on the type of internal technology, the installation, the onsite calibration, the weather, the lighting, the plate configuration, and a host of other conditions (5), LPR rarely recognizes more than 80% of the plates and often does worse than 60%. Fortunately, all is not lost; even when LPR fails to read a plate, meaning that not every single character is recognized correctly, the system usually returns very valuable and mostly correct individual character information. By comparing the imperfectly read plate against another such plate, one may still be able to render reasonable judgment in terms of whether the two plates are actually a match. For instance, if two strings (sequence of characters) differ from each other by only one character, they may well have originated from the same plate. Therfore, 4

5 a measure of similarity between two strings can be established to indicate the likelihood of a match. To this end, this paper reports the first attempt to employ a text-mining technique called Edit Distance (ED) to improve the matching efficiency of imperfectly read plates. In the following sections, this paper presents the fundamentals of the Edit Distance technique and follows with how it is applied to license-plate matching. A case study and its results are also presented, followed by discussion and conclusions. 2. MATCHING PLATES WITH EDIT DISTANCE The process of matching two strings involves a sequence of comparisons of individual characters to determine the degree of similarity between the two. Consider, for example, a license plate with the string 4455HZ which is read by two LPR machines at two different locations. Suppose that at the first location, the plate was read as 4455IIZ and at the second, 4455HZ. Note that neither LPR unit knows whether it has read the plate correctly. By looking at the two reports, one can either declare no match, or one may perhaps speculate a potential match since the two strings differ only by two pairs of characters: I - H and I - (where represents a null or empty character). If there were another plate that was read as 445OHZ earlier at the first location, one may speculate that it is less likely that the O - 5 pair is a match. The task here is to teach the computer to make such speculations. Techniques for measuring the similarity or dissimilarity between two strings have been developed in the past and have found application in areas such as handwritten character recognition and computation biology (6). The pioneer in this field is Vladimir Iosifovich Levenshtein, who developed Edit Distance (ED) (also 5

6 known as Levenshtein distance), which is a metric that computes the distance between two strings as measured by the minimum-cost sequence of edit operations (7). Given two strings x and y, their Edit Distance describes how many fundamental operations are required to transform x into y. These fundamental operations are termed as follows: Substitutions: A character in x is replaced by the corresponding character in y. Insertions: A character in y is inserted into x, thereby increasing the length of x by one character. Deletions: A character in x is deleted, thereby decreasing the length of x by one character. To relate the definition of Edit Distance to the problem presented in this paper, we will return to the example of the plate "4455HZ" being captured by two LPR stations. Let x = "4455IIZ" and y = "4455HZ"; the task is to compute the number of fundamental operations to transform x into y. (Note that x and y could have been assigned in reverse order since the true plate number is not known.) In this case, it can be established that the minimum number of operations is 2, which corresponds to the substitution of the first I in x by H and the deletion of the second I in x. Therefore, the Edit Distance d(x,y) between x and y is 2. To determine the minimum d(x,y) for any pair of strings x and y there efficient computational procedure called dynamic programming, proposed by Wagner and Fisher (8). A detailed description of this procedure can be found in the book Pattern Classification by Duda et al. (9). In many applications, the string y is provided by a list of words that has the maximum likelihood of containing the true value of the given string, x. This prespecified list of words is called a lexicon or reference for matching. Using this list of 6

7 words, it is possible to detect errors, generate candidate corrections, and rank these candidates. However, in the problem presented in this paper, x and y represent strings read from license plates and either one can be used as a reference for matching since there is no knowledge about the true string. 3. MATCHING METHODOLOGY Since this study considered plates read at two different locations, in the remaining text they will be referred to as LPR Station 1 and LPR Station 2. LPR Station 2 is downstream of LPR Station 1. As such, for a given plate read at LPR Station 2, there are a number of candidate plates already read, correctly or not, at LPR Station 1 for matching purposes. The number of candidate plates can be constrained by an imposed threshold value, τ, for the edit distance. There can also be a time window constraining the allowable travel time between the two stations. Therefore, the number of candidate plates read at LPR Station 1 for each plate at LPR Station 2 is defined by the following constraints. Fig. 1 illustrates this. ED τ tt i [tt lower, tt upper ] Using these constraints, all string candidates read from LPR Station 1 are ranked, and the candidate with the least ED not exceeding τ is selected. In case there are multiple candidates tied with the least ED value, the first appearing in chronological order is selected. The time window initially includes the entire period of data collection. In the end, time intervals were used to reduce comparison operations with limits defined as follows: 7

8 tt uper = tt_mean + z tt_std tt lower = tt_mean z tt_std where, tt_mean = sample mean travel time; tt_std = sample standard deviation of travel time; z = the number of standard deviations above or below the mean. t 1i = Passage time at LPR tt t= upper -tt 2 lower t 2i = Passage time at LPR Station 2 t 1i t 1 +tt i i - t t =t +tt 2i 1 i i t 1 +tt i i + t FIGURE 1 Time window constraint where, tt i = travel time observed for a potential match i; [tt lower, tt upper ] = lower and upper limits of the time window. 4. CASE STUDY AND RESULTS The LPR equipment used in this study was manufactured by PIPS Technology. Two versions of the equipment were used to capture license plates of westbound trucks on I-40, one at Campbell Station Road (LPR Station 1) and the other downstream at the weigh station (LPR Station 2). Both units used internal detection (plate-finder) software to trigger the camera and an infra-red-based illuminator, which was 8

9 activated when a vehicle was within the camera's field of view. The two cameras were set up to capture plates in the rightmost lane of the road. Data were collected on weekdays, between 1:00 PM and 4:00 PM, excluding days of abnormal traffic patterns. The distance between the two stations was about 1.4 miles. During five days of data collection in 2007, 2,671 plates were captured at the first station and 1,530 were captured at the second station. Among these, a total of 787 were manually verified as identical. In addition to reading plates, the equipment also stamped each plate image with time information, which was useful for later comparisons LPR Performance The raw images stored in the LPR system database were viewed manually to compare with the detection reports. The results show an average accuracy of 61% for Station 1 and 63% for Station 2. Since the cameras were not permanently mounted (they were mounted on heavy tripods), the accuracies could potentially be higher Truck Speed Figure 2 shows truck-speed comparisons before (measured by tube in 1997) and after (measured in 2007 using radar gun and LPR) the speed limit changed. After the new speed limit went into effect, truck speed ranged from 40 mph to 75 mph, and most of the speed values (the 20th percentile was approximately 55 mph) were higher than the actual speed limit of 55 mph. In Figure 2, comparing the before- and afterspeed distributions, a shift of only about 8 mph in the average speed was observed, with no change in the variance. 9

10 Cumulative Percentage 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Truck Speed Comparison over Years Speed (mph) 1997 Tube 2007 Radar 2007 LPR FIGURE 2 Histogram and cumulative distribution of the sample truck's speed 4.3. ED Performance Results To assess the performance of the matching methods implemented, modules in the MATLAB programming language were written to perform the calculations automatically. The number and percentage of positive matches, the number of false-positive matches, and the average number of candidates per plate were used as performance measures. Four different threshold values, 0 to 3, were used to constrain edit distance. Table 1 shows the results obtained when both the top-rank and the first chronological candidates are selected. As can be seen, although smaller threshold values result in fewer false-positive matches, they also result in fewer positive matches. Moreover, without considering the travel time information, false-positive matches are very likely to occur for threshold values of 2 and 3. 10

11 SIMILARITY MEASURE USED TABLE 1 Performance of ED without travel time information # OF # FALSE AVERAGE # OF PERCENTAGE THRESHOLD POSITIVE POSITIVE NUMBER OF MATCHES DETECTED MATCHES MATCHES CANDIDATES ED % ED % ED % ED % Table 2 presents the results of combining the two constraints, Edit Distance and Travel Time, with z = 4, for each day of data collection. Note that the time constraint (time window) was different for each day, as it depends on the mean and standard deviation observed on each day sampled. TABLE 2 Performance of ED for Time Windows Determined by the Observable Range of Travel Times SIMILARITY MEASURE USED THRESHOLD # OF MATCHES # OF POSITIVE MATCHES # FALSE POSITIVE MATCHES AVERAGE NUMBER OF CANDIDATES PERCENTAGE MATCHED ED + tt % ED + tt % ED + tt % ED + tt % In Table 2, for all threshold values tested it seems very unlikely (with the highest average number of candidates equaling 1.03) to have more than one candidate per plate and most candidates tend to be the true values (with about 2% of mismatches in the worst case, but with 97% of vehicles detected). It is worth noting that all cases of false matches in Table 2 came from pairs of plates in which either one or the two plates were missed, or not detected (the truck did not have its plate read by one of the station) by one of the LPR units. This can happen, for example, when a truck changes lane between the two LPR stations, either entering or exiting the monitored lane. This means that any plate captured by both stations has a high chance of being indentified and form a positive match, whereas whenever a plate is not detected by one of the stations it has a quite chance of forming a false match. Therefore, if the LPR technology were set up (aiming at multiple lanes) to 11

12 capture most of the plates through the stations, the false matches would be far less likely. As an illustration, Table 3 presents the five false-positive matches for the case of ED 2 and the time window constraint, z = 4. TABLE 3 False Positive Matches for ED 2 + tt 2 LPR Station 2 LPR Station 1 Hypothetical Recognized Plate Recognized Plate Time Time Speed (mph) Characters Number Characters Number 13:30:26 "1561" "1561" 13:28:46 "15S7" "15157" :46:37 "1297D" "12990" 14:43:37 "12905" "12905" :19:36 "1234" "1234" 15:16:57 "9214" "9214" :14:12 "2JZ294" "2JZ294" 15:13:21 "2JF204" "2JF204" :14:39 "9713" "9713" 14:13:13 "9214" "9214" DISCUSSION Although the algorithm is not expected to reach perfect vehicle detection with none false matches, further work is necessary to improve its reliability and make false matches a very rare event. To this end, it is believed that improvements in the similarity measure used and in how the travel time constraint was used, as well as a permanent setup of the LPR units certainly will increase the algorithm accuracy. As for the similarity measure used, the most frequent mistakes made by the LPR equipments can be compensated using weight functions into the ED formulation. For example, there is a relatively high chance of the characters "1," "0," and "B" being misread as "I," "O," and "8," respectively, by the LPR machine. The probabilities of such incidences were not considered in this paper. However, this information is available and can be obtained by constructing a matrix of error probabilities taken by each LPR unit used. Once the matrix is constructed, the challenge becomes how to design the weights (or the cost function) to be used in the edit distance calculation. For example, what is the cost for transforming "0" into "O" given that the odds that "O" is misread as "0" are, for example, 50%. Some initial 12

13 work by the author suggests that using a cost function would increase the number of positive matches and reduce false-positive matches. For example, in the first row of Table 3, "1561" and "15S7" would not have been falsely matched if it were known that the character "6" is very unlikely to be recognized as "S," or vice versa. As for the travel time constraint used in this study, a simple and deterministic method was used. However, a more suitable way of using travel-time information would take advantage of travel time distributions, which are believed to be a Gaussian function. This way, the deterministic and chronological method used herein for selecting candidate plates would be replaced by a probabilistic method, in which a candidate would be selected if it had both the least ED value and the highest travel time probability. As for the equipment setup on the roadside, it is believed that a permanent rather than a mobile setup would result on a better accuracy of the equipments in reading plate numbers. Thus in the next phase of the study LPR machines will be setup in fixed poles on the roadside. Therefore, adequate camera settings such as camera angle, height and direction will be more achievable under this permanent setup. Another issue of interest is the effect of the distance between LPR units. The closer two LPR units are located, the smaller in general the time window, and the less the chance of the presence of trucks with similar license plate numbers during the same time frame. On the other hand, very closely located LPR units tend to capture more of the instantaneous speeds and not the average trip speed. Trucks can simply slow down for the speed trap and then get back to the preferred cruising speed. In this regard, LPR units spaced farther apart could be more effective 13

14 as a speed deterrent. Therefore, this distance should be one that neither compromises the desirable reliability of the algorithm nor affects traffic behavior. 6. CONCLUSIONS This paper assessed the performance and feasibility of using license-plate recognition for large-truck speed monitoring and enforcement purposes. A field study using two LPR units was conducted on I-40 in the vicinity of Knoxville, Tennessee in A text-mining technique called Edit Distance was used to match license plates not correctly recognized by the LPR units. While the accuracy of the LPR units was less than perfect, most of the plate characters were recognized, even if incorrectly sometimes; the use of edit distance in combination with travel time constraint increased the number of positive matches (at 97% of positive detection) between the two LPR stations with very few false-positive matches (about 2%). Research is still needed to improve the reliability of LPRs system to further reduce false-positive matches. One of the main ideas the authors are working on is the use of a probability matrix based on the odds of one character being read, or misread, as another. This matrix can then be used to develop a weighted cost function, instead of the unitary one used in this paper, for the edit distance calculation. This increased sophistication, it is believed, will further increase the percentage of positive matches and reduce the likelihood of false-positive matches. ACKNOWLEDGMENT The author is grateful for financial support from the National Transportation Research Center. Valuable assistance from the Tennessee Department of 14

15 Transportation and the Tennessee Department of Safety is also hereby acknowledged. REFERENCES 1. Tang, T, M. Roberts and Cecilia Ho. Sensitivity analysis of MOBILE6 motor vehicle emission factor model. Federal Highway Administration, FHWA Resource Center, Atlanta, Rossetti, M. D. and J. Baker. Applications and Evaluation of Automated License Plate Reading Systems. In 11 th ITS America Meeting. CD-ROM. Conference proceedings. Miami Beach, Fla., Wiggins, A. ANPR Technology and Applications in ITS. Research into practice: 22 nd ARRB Conference Proceedings. CD-ROM. Australia Road Research Board, ARRB, Canberra, Australia, October of Han, L.D., F.J. Wegmann, and A. Chatterjee. Using License Plate Recognition (LPR) Technology for Transportation Study Data Collection. Report submitted to the Tennessee Department of Transportation, TDOT Nakanishi, Y. J. and J. Western. Ensuring the Security of Transportation Facilities: Evaluation of Advanced Vehicle Identification Technologies. In Transportation Research Record: Journal of the Transportation Research Board, No. 1938, Transportation Research Board of National Academies, 2005, pp Mei J. Markov Edit Distance. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(3), 2004, pp Levenshtein V.I. Binary Codes Capable of Correcting Deletions, Insertions and Reversals. Soviet phys. Dokl, 10(8), 1966, pp Wagner, R. A. and M. J. Fischer. The String-To-String Correction Problem. J. Association Computer Machinery, Vol. 21, No. 1, 1974, pp Duda, R. O., P. E. Hart and D. G. Stork. Recognition with strings. Pattern Classification. 2 nd Ed. Wiley Inter science. Ch. 8, 2000, p