PILL-ID: Matching and Retrieval of Drug Pill Imprint Images

Transcription

1 PILL-ID: Matching and Retrieval of Drug Pill Imprint Images Young-Beom Lee 1, Unsang Park 2, and Anil K. Jain 1,2 1 Brain and Cognitive Engineering Korea University, Korea 2 Computer Science and Engineering Michigan State University, USA

2 Legal drug pill or illicit drug pill? If illicit pill, which cartel manufactured it? What is the effective way to identify illicit drug?

3 ~35M in the U.S. used illicit or abused prescription drugs; $14B spent for drug treatment & prevention (2007) Prescription pills must be identifiable (by color, shape, and imprints) per FDA regulations Illicit pills (e.g., narcotics) also contain imprints to identify the cartel or distributor

4 Databases of prescription pills and illegal pills are available (pharmaceutical companies, FBI) Query Rank Imprint : 5883 Shape : round Color : brown Ingredient : MDMA, BZP, TFMPP Cartel : Gulf contents

5 Imprint is an indented or printed mark on a pill, tablet or capsule Symbol, text, digits or their combination Legal drug pills Illicit drug pills

6 Sobel operator to obtain gradient magnitude image Segmentation, scale normalization Rotation normalization Original Image Gradient magnitude Image Primary & Secondary Dominant Orientations Multiple template with Rotation variation Landmarks (key-points) are selected within a preset radius (SIFT descriptor)

7 Gradient magnitude images have smaller intra-class variations Original image Gray image Gradient Magnitude image Method Rank-1 accuracy (%) Gradient magnitude Grayscale Optimized SIFT descriptor (using 602 query-gallery dataset)

8 Images that did not match at rank-1 using SIFT but matched using the proposed method (fixed key points + SIFT descriptor) Method Number of key-points Original SIFT Min Max Avg Our method (SIFT descriptor) Rank-1 accuracy (%) Red dots: SIFT key points, Blue dots: preset key points

9 Select a set of key-points Collect gradient magnitude and orientation with Gaussian weighting and tri-linear interpolation Truncation Length of feature vector: = = 3712 Gaussian window centered at a key point Gaussian weighting Tri-linear interpolation Truncation

10 LBP histograms with multiple neighborhood parameters (P,R) are created and concatenated P=8, R=1.0 P=4, R=1.0 P=12, R=2.0 Feature vectors are constructed with the following parameters (P, R) Window size Shift value U(8, 1) 20 X 20 4 U(4, 1) 10 X 10 2 U(12, 2) 30 X 30 6 Length of feature vector: U(8,1) = 59, (4,1) = 16, U(12,2) = X(13 X 13)+16 X(31 X 31)+135 X(7 X 7) = 31,962

11 Given a query image (q) and N gallery images (g), the K feature vectors of the query are compared with the L n feature vectors of the n th gallery images (n = 1 to N, L 2 norm). L n is different for each gallery image The ID of the closest match in the gallery is selected as the ID Feature vectors of a query image, i q m Feature vectors of gallery images, j g n L n (=j) K m (=i) i j ID arg min d( q, g ) m m n n N

12 822 illicit drug pill images from the Australian Federal Police; 138 illicit drug pill images and 14,003 legal pill images from the U.S. DEA website, Drug information online and pharmer.org Image size: from 48 X 42 to 2,088 X 1,550 pixels; 96 dpi Query set: 602 illicit drug pills with duplicate images of the same imprint pattern (88 distinctive patterns) Gallery set: 960 (illicit drug pill images) + 14,003 (legal drug pill images) = 14,963 images Leave-one-out method to match each of the 602 query to all the 14,962 gallery images

13 SIFT descriptor parameters are optimized for pill imprint matching 1. Smoothing 2. Gradient orientation & magnitude 3. Gaussian weighting 4. Trilinear interpolation 5. Truncation with threshold values of 0.2, 0.5 and 1 Method Rank-1 accuracy (%) Truncation value Rotation Normalization Edge image Grayscale image SIFT with 1, 2, 3, 4, 5 (Original sift) 0.2 No SIFT with 2, 3, 4, No SIFT with 2, 4, No SIFT with 2, No SIFT with 2, 4, No SIFT with 2, 4, No SIFT with 2, 4, Yes

14 602 query and 14,962 gallery images Method Rank 1 (%) Rank 20 (%) MLBP SIFT descriptor SIFT (0.7)+MLBP (0.3)

15 Query Top-6 retrievals

16 Queries that were not correctly retrieved in top 20 matches Rank of Query Top-6 retrievals true mate Illumination noise in the background Similar shape and imprints Very similar pattern between query and top retrieved images 1897

17 Numeric or text information in imprints can be used for matching/filtering 5883 Imprint : 5883 Shape : round Color : brown Ingredient : MDMA, BZP, TFMPP Cartel : Gulf

18 Query Shape : Round Color: Pink Text: no Numbers: no Rank Using only imprints Rank Using imprint shape and color Content based matching can reduce retrieval errors

19 Proposed an image retrieval system for identifying illicit drugs 84.4% rank-1 (91.53% rank-20) accuracy with ~600 query and ~15K gallery images Evaluated two image descriptors (SIFT and MLBP) & their fusion; rotation invariant matching scheme was used Computation time: 2.3 (0.5) sec/image for feature extraction and 13.0 (4.0) sec for each query with ~15K gallery for SIFT (MLBP); code in MATLAB running on 2.8 GHz CPU, 8 GB RAM Future work Content based matching/filtering Evaluation on a larger database; collaboration with AFP More efficient matching scheme

20

21 If we can identify numbers or texts in imprints, content based methods can be used. Number : 5883 Text : WYETH Examples of the number and text imprint

22 MLBP is also evaluated with a various parameters using 602 querygallery dataset to optimize it for pill imprint matching 1. Number of LBPs 2. Sub-region (window size, shift value) 3. Input image size Method LBP Sub-region Image size Rank-1 accuracy (%) LBP 8,1+4,1 No u2 LBP 8,1+4,1+12,2 No u2 LBP 8,1+4,1+12,2 No u2 u2 LBP 8,1+4,1+12,2 (32, 8)(16, 4)(48, 12) u2 LBP 8,1+4,1+12,2 (16, 4)(8, 2)(24, 6) u2 u2 u2 u2 u2 u2 LBP 8,1+4,1+12,2 (20, 4)(10, 2)(30, 6)

23 15 Gradient magnitude image Orientation histogram Multiple Templates