White Paper Predictive Modeling for True-Name Fraud An Equifax Analytical Services Research Paper Dave Whitin, Consultant Michiko Wolcott, Statistician September 2006
Table of contents Executive summary................................... 1 Introduction.......................................... 2 Legal responses to identity theft......................... 3 Challenges in modeling for fraud........................ 3 Project goals......................................... 4 Data sources......................................... 5 Performance......................................... 6 Conclusion.......................................... 8 About Equifax........................................ 9 About the authors..................................... 9
Executive summary Identity theft continues to be a major white-collar crime in the United States. Its consequences are borne evenly by consumers, credit grantors and credit bureaus. To complicate matters, identity thieves are constantly changing their techniques, so their patterns of behavior are difficult to predict and monitor. Identity theft can be defined as any act in which someone uses the personal information of another (name, address, Social Security number, date of birth, account numbers) without the knowledge or consent of that individual. In the world of consumer credit lending, this often leads to the fraudster opening a line of credit in the victim s name. The perpetrator then takes the fraudulent card on a spending spree. A common term for this type of identity theft fraud is true-name fraud. Optimum performance is achieved when the solution draws upon a variety of data types, including credit-file data, identification match quality data and applicationspecific data. The risk and exposure associated with true-name fraud has sparked demand for advanced solutions that can identify this type of activity at the source; the time of application. By implementing real-time, cost-effective solutions with a low false-positive rate, lenders will increase compliance with legislation. Perhaps more importantly, lenders can bolster consumer confidence, deter fraud attempts, and cut their financial losses. In 2003, Equifax Analytical Services completed a detailed study for a national card issuer to analyze how to best achieve predictive modeling for true-name fraud. The study, which was revalidated in 2005 with a newly refined model, found that the most effective true-name-fraud solution needs to incorporate multiple types of data and should be customized to a specific lender. Optimum performance is achieved when the solution draws upon a variety of data types, including credit-file data, identification match quality data and application-specific data. This paper provides a background on identity theft and summarizes the performance of both the original and the refined models. The paper s goal is to help the reader learn more about the challenges of risk modeling for fraud. 1
Introduction The Federal Trade Commission (FTC) tracks the number of fraud and identity theft complaints it receives. As Figure 1 shows, the number of complaints soared from 2001 (the first year tracked) through 2003. Since then, identity theft complaints have climbed at a slower rate. Identity theft complaints as a percentage of total complaints have actually declined slightly (from 40% in 2003 to 37% in 2005). 300,000 Number of identity theft complaints 250,000 200,000 150,000 100,000 50,000 0 2001 2002 2003 2004 2005 Figure 1. Number of identity theft complaints received by the FTC According to the Identity Theft Resource Center, in 2005 each new identity theft victim spent an average of 60 hours resolving the resulting problems. Identity theft is carried out when the perpetrator hijacks someone s identity information, such as name, address or social security information, to commit fraud. This type of crime can affect anyone, anywhere, at any time. According to the Identity Theft Resource Center, in 2005 each new identity theft victim spent an average of 60 hours resolving the resulting problems. It is fairly easy for a criminal to obtain personal information to use in a fraudulent manner. Some websites facilitate internet fraud, either by revealing private information or by enabling amateur fraudsters to create an extremely legitimate-looking fake ID using a victim s name, address and date of birth information. 2
Legal responses to identity theft The surge of consumer fraud over the last decade prompted a variety of legislative, regulatory and litigious actions. Section 326 of the USA PATRIOT Act specifically requires financial institutions to implement a Customer Identification Program (CIP). The program establishes procedures for collecting ID information, maintaining ID records, verifying customers identity and determining if a customer is on any list of suspected terrorists. At least 25 states have passed security freeze laws, which lock a consumer s credit report and prevent anyone from opening a credit account or loan. The FTC, by law, must log all identity theft complaints in its data repository, provide victim assistance and consumer education, and refer complaints to appropriate reporting agencies and law enforcement agencies. In 2004, penalties for aggravated identity theft were increased. Many states have laws specifying criminal penalties for identity theft or laws aiding in the victim s recovery. Most states have rules or policies to protect Social Security numbers and personal financial information in court records. Versions of California s data theft notification law are being considered by many states and at the federal level. At least 25 states have passed security freeze laws, which lock a consumer s credit report and prevent anyone from opening a credit account or loan. The Federal Financial Institutions Examination Council is prodding financial institutions to take stronger measures for risk assessments, customer authentication and verification of new customers. Organizations that have experienced significant data losses are being hit with fear factor lawsuits, where courts must decide if merely the fear of identity theft warrants damages to plaintiffs who have not yet been harmed. Challenges in modeling for fraud Lenders attempting to manage the problem of identity theft and fraud face a multitude of challenges. Some of the toughest are: Varying definitions. Different lenders have different opinions on what defines fraud, and therefore losses are difficult to estimate. A common debate is whether or not to classify first-payment 3
defaults as fraud. A first-payment default is defined as an account that opens, charges up a balance and never makes a single payment. All first-payment defaults, however, are not necessarily fraudulent. Moving target. Fraud patterns and behavior are constantly changing. Fraudsters find a weakness in a lender s system and exploit it until it no longer works. Once the lender makes an adjustment, the fraudsters move on to another strategy. This changing nature of fraudulent behavior makes it extremely difficult to model. While a standard consumer risk model can remain predictive for many years, a fraud model will have to be regularly validated in order to ensure that it stays effective. High false-positive rates. These occur when applications are incorrectly classified as highly likely to be fraudulent based on some score or criteria. When a score yields a high false-positive rate, lenders must either decline or manually review a large portion of the through-the-door population. Both options are unattractive; declining too many good applications erodes profitability. Manually reviewing a high percentage of applicants is inefficient, time consuming and costly. Data validity. The relatively low rates of true-name fraud, combined with the issue of correctly identifying fraudulent activity, makes it difficult to create a valid sample for predictive modeling. The card issuer needed to maintain a reasonable falsepositive rate to minimize the inefficiency of manual reviews and the revenue lost when declining the false-positive cases. Project goals In 2003, Equifax Analytical Services was engaged by a national card issuer to prepare an in-depth modeling study for predicting fraudulent activity. The remainder of this paper describes the background and results of this project, and the subsequent engagement in 2005 to refine the model. The original study used two months of application data from a portfolio of revolving cards. The lender had identified transactions that were confirmed as true-name fraud. The applications came through multiple channels, including point of sale (register), Internet, phone, fax and mail. The lender wanted a predictive tool that would identify fraudulent applications in a real-time environment. They needed to maintain a reasonable false-positive rate to minimize the inefficiency of manual reviews and the revenue lost when declining the false-positive cases. 4
Data sources Equifax gathered data to be tested as independent predictive variables, or attributes, in the fraud solution. Each type of data was pulled from a point in time that corresponds with the application dates. The following data sources were used to create the independent attribute set: 1. Credit file attributes: These are characteristics from the individual credit file, such as balance, high credit, utilization, delinquency and inquiry information. Some credit attributes enter the model because they indicate deviations from the consumer s normal credit activities. Other credit file characteristics help describe the profile of a fraud victim. The model captured 37% of true-name frauds in the bottom-scoring 1% of applications and over 63% captured in the bottomscoring 5%. 2. Identification match quality attributes: A fraudulent application often has identification information that is slightly different from the victim s true information. Match attributes measure the quality of the identification information on the application. Examples of situations that generate flags: Zip code fails to match city Application address and phone number fail to match those on the credit file The Social Security number is invalid The driver s license number doesn t match the format of the issuing state 3. Application-specific attributes: Any information on the application that could potentially be used as an indicator for fraud was also tested as an independent attribute. This included, among other factors, the channel that the application came through, the state of residence on the application, and the day and time of the application. As previously discussed defining fraud is a challenge when building a predictive model. For this study, true-name fraud was defined as any application in which the individual submitting the information uses someone else s identification. In other words, the applicants are not who they say they are. The data set used for this model development exercise contained over 3,500 verified true-name frauds. Based on thorough follow-up research that the lender conducted on each of these complaints, they were verified as frauds. 5
Performance Original model s results (2003) The final model scorecard proved to be extremely effective in identifying true-name fraud on the development data sample. The model captured 37% of true-name frauds in the bottomscoring 1% of applications and over 63% captured in the bottom-scoring 5%. For the lender, this means the majority of cases can be handled without manual review or additional processing. The model s stability was validated on two separate holdout samples, both from different points in time than the development sample. Each of the validation samples contained verified truename frauds. The validation results also proved to be strong: Both validation samples had more than 37% of true-name frauds captured in the bottom scoring 1% of applications, mirroring the performance of the development sample. The summary of these results can be found in Table 1 below. Table 1. 2003 Data set KS value TNF bottom 1% TNF bottom 5% TNF bottom 10% Development 67 36.8% 63.4% 74.6% Validation 1 62 38.0% 56.0% 68.1% Validation 1 61 37.2% 60.0% 69.7% The model s predictive power was initially proven on the development sample, with a KS1 value of 67. More importantly, the model performance remained very strong and consistent on the two separate validation holdout samples, with KS values of 62 and 61. Given the ever-changing nature of fraudulent activity, these validations results speak well for the stability of the model. The contributions of the various types of data are graphically depicted in Figure 2 below. Application 8% Although credit-file data made up nearly two thirds of the variable set, the other sources added strongly to the overall performance of the model. Identification match quality 27% Credit file 65% 1 The Kolmogorov-Smirnov(KS) test is a method of determining if two datasets differ significantly. Figure 2. True-name fraud data sources 6
The composition of the final model included data from the sources described earlier. Although credit-file data made up nearly two-thirds of the variable set, the other sources added strongly to the overall performance of the model. Equifax tested several models without such variety, but the alternative models were not able to match the high standard of overall performance achieved when including the supplementary data sources. We therefore conclude that each data source added significant power to the model. Updated model s results (2005) Ever-changing patterns of identity theft necessitate periodic refinements to any predictive model. In 2005, Equifax updated the true-name fraud model for this lender. In two years, the model s capture rate for the bottom 1% had declined from 36.8% to 25.0%. Equifax believed that modest refinements could produce significant performance improvements. Table 2 compares the models, which have similar combinations of data types. The new model was able to identify 37.7% of true-name fraud cases in the bottom 1% of the population, exceeding the performance of the original model at its inception. The results verified that (1) the combination of data sources used in 2003 remains effective in predicting true-name fraud, and (2) fraud tools should be refreshed frequently for maximum effectiveness. Table 2. Model KS value TNF bottom 1% TNF bottom 5% TNF bottom 10% Original (2003) 54 25.0% 49.5% 61.4% Original (2005) 63 37.7% 61.4% 71.2% 7
Conclusion Identify theft continues to be a major problem for lenders, consumers, credit reporting agencies and government. Increased public awareness of the problem and attention from lawmakers, however, is helping to combat fraud losses. Lenders hold another key; predictive modeling gives lenders an effective tool to scrutinize applications and demonstrably reduce losses from true-name identity theft. Predictive modeling gives lenders an effective tool to scrutinize applications and demonstrably reduce losses from true-name identity theft. Fraud is a difficult behavior to define, and even more difficult to predict. The criminals that steal identities are constantly changing their methods, making patterns hard to establish. To maintain high-performance levels, therefore, predictive solutions must be continually updated with the latest data. As demonstrated by the study discussed in this paper, predictive modeling for fraud can produce highly effective results. Credit data alone can be very predictive; however, for optimum performance, credit data should be augmented with other data sources. In this study, some of the data sources that enhanced performance of credit-data-only models included: The quality of the address that is on the application The quality of the Social Security Number and phone number The quality of the application address and phone number match to the credit file Custom, lender-specific attributes from the application Fraud rates can vary drastically by application channel and by merchant or branch. Every consumer company has a different procedure for processing an application, and accordingly, fraud patterns are going to be different for each lender. This clearly calls for customized strategies for predicting fraud. Consumer companies today need to build fraud solutions that maintain consistent performance over time. The solutions should include multiple types of data, including credit file characteristics, and a variety of consistency checks that can be run on the address and other various application data elements. Equifax offers a powerful, cost-effective solution by leveraging the company s experience in true-name fraud detection and the data many lenders already use in their current bureau application processes. 8
About Equifax Equifax Inc. is a global leader in information technology that enables and secures global commerce with consumers and businesses. We are one of the largest sources of consumer and commercial data. Utilizing our databases, advanced analytics and proprietary enabling technology, we provide real-time answers for our customers. This innovative ability to transform information into intelligence is valued by customers across a wide range of industries and markets. Headquartered in Atlanta, Georgia, Equifax Inc. operates in the U.S. and 14 other countries throughout North America, Latin America and Europe. Equifax is a member of Standard & Poor s (S&P) 500 Index. Our common stock is traded on the New York Stock Exchange under the symbol EFX. About the authors Dave Whitin, Consultant, joined Equifax Analytical Services in 1998. He has experience providing scoring and analytical consulting to customers in a wide variety of industries. Dave holds a B.S. in Statistics from the University of South Carolina and an M.S. in Statistics from the University of Georgia. Michiko I. Wolcott, Statistician, joined Equifax Analytical Services in 2002. Working in the Custom Modeling group, she builds credit-risk and marketing scorecards. Michiko has an M.S. in Statistics from Florida State University and degrees in Music from Florida State University and the Peabody Conservatory. 9
Equifax is a registered trademark of Equifax Inc. Inform, Enrich, Empower is a trademark of Equifax Inc. Copyright 2009, Equifax Inc., Atlanta, Georgia. All rights reserved. Printed in the U.S.A. EFS-714-ADV 7/09