Business Case Development for Credit and Debit Card Fraud Re- Scoring Models

Transcription

1 Business Case Development for Credit and Debit Card Fraud Re- Scoring Models Kurt Gutzmann Managing Director & Chief ScienAst GCX Advanced Analy.cs LLC October 20,

2 IntroducAon & Topic List We will discuss how to develop a business case for payment card fraud detecaon analyacs, illustrated by a case studies with re- scoring models Topics Part I Performance Measures & Business Case Principles 1. Fraud detecaon as a binary classifier problem 2. Performance measures of classifiers 3. The Receiver OperaAng CharacterisAc Curve (ROC) and Area Under the Curve 4. Cost / Benefit FuncAon and the Payoff Matrix of a Fraud Detector Part II Two Case Studies of Re- Scoring Model Performance 1. Re- Scoring Models for Credit and Debit Card transacaons 2. Case Study 1: Endogenous Debit Card Re- Scoring Model Performance and Benefits 3. Case Study 2: Exogenous / MulA- Channel Credit Card Re- Scoring Model Performance and Benefits 4. OperaAonal Deployment of Re- Scoring Models in Card Fraud Scoring System Architectures 5. QuesAons & Answers October 20,

3 Fraud detection as a binary classifier problem There are two major types of fraud VicAm Fraud, a 3 rd party takes money or assets from a bank customer or the bank Stolen credit card usage Internet banking account takeovers First Party or Swindler Fraud, a criminal directly interacts with the bank to extract assets or money Fraudulent loan applicaaons Worthless items deposited at ATMs (empty envelopes) In this presentaaon, we are detecang vicam fraud in consumer credit card and debit card transacaons A transacaon must be classified into one of two categories, Fraud or Not Fraud This is a binary classifier problem October 20,

4 ConAngency Tables of Classifiers The operaaon of the binary classifier produces four possible outcomes A transacaon classified as fraud generates a fraud alert (possibly declining the transacaon) A transacaon classified as good is accepted as business as usual A conangency table represents the outcomes ConAngency table notaaon: TP : True PosiAve, a fraudulent transacaon correctly classified as a fraud FP : False PosiAve, a good transacaon incorrectly classified as a fraud FN : False NegaAve, a fraudulent transacaon incorrectly classified as a good one TN : True NegaAve, a good transacaon correctly classified as a good one Various Sums, TPFP = TP + FP = P etc. Frauds Not Totals Frauds Alerts TP FP TP+FP = P Not Alerted FN TN FN+TN = N Totals TP+FN FP+TN TP+FP+TN+FN October 20,

5 Performance Measures of Classifiers True posiave rate (TPR) is also called the detec.on rate If measured in dollars, it is the dollar detec.on rate Other performance measures are available, e.g. accuracy, misclassificaaon rate, posiave predicave value, negaave predicave value, and specificity, but are not generally meaningful in the fraud detecaon domain October 20,

6 Receiver OperaAng CharacterisAc (ROC) Curves The ROC is a plot in unit space [0,1], [0,1] of the true posiave rate (detecaon rate) on the y- axis versus the false posiave rate on the x- axis Scoring classifiers generate ROCs Rule- based classifiers do not have a ROC per se, but only a single operaang point The ROC generates a curve that for any reasonable classifier will capture an area of at least 0.5 This is the area under curve (AUC) metric of a ROC curve A classifier is considered predicave if AUC > 0.75 All else equal, a classifier with a greater AUC is generally preferred to another For fraud detectors, the AUC below an FPR of 0.05 is most important October 20,

7 Cost / Benefit FuncAon and the Payoff Matrix of a Fraud Detector $1, ($9.12) ($1,400.00) $ $207, ($13,935.36) $193, ($96,600.00) $249, $153, $110, $235, $346, Outcome Costs or Benefits Unit Payoff Matrix ConAngency Table Example $303, of apempted fraud Payoff matrix as shown at leq DetecAon rate is 149/ (149+69) = 68% OperaAonal cost of the detector is $110K Benefit is $207K Net Benefit is $346K Do Nothing alternaave net benefit is - $69K Net Benefit October 20,

8 Economic OpAmizaAon of the Alert Score Threshold Various alert score thresholds allocate TP, FP, FN, TN in differing proporaons to the conangency table Combined with the payoff matrix, this generates a cost/benefit curve that can be opamized with respect to the alert threshold score OperaAonal constraints must also be considered, however October 20,

9 Determining the Best Business Case by Net Present Value 1. Develop the ROC of the detector 2. IdenAfy the op.mal score threshold by inspecaon 3. Establish the payoff matrix for outcomes 4. Tabulate the conangency table at the opamal threshold 5. Compute the periodic net economic benefit (e.g. monthly) 6. Establish the planning horizon (e.g. 3 years) 7. Establish the cash flow discount rate (e.g. 10%) 8. EsAmate system implementaaon one- Ame costs 9. EsAmate other system operaaonal recurring costs 10. Develop the cash flow series 11. Compute the NPV of the cash flow series 12. Perform sensiavity analyses to determine robustness of the business case Example Cash Flow Profile October 20,

10 CASE STUDIES 1. Debit Card Rescoring 2. Credit Card MulAchannel Data October 20, Kurt Gutzmann Managing Director GCX Advanced Analy.cs LLC 10

11 TransacAon Classifiers, Authorizers, and Re- Scoring Models Basic Scoring System TransacAon Primary Fraud Scoring Engine TransacAon Advice Accept Reject Other Re- Scoring System Improved Score TransacAon Primary Fraud Scoring Engine Improved TransacAon Advice Accept Addi.onal Data Re- Scoring Model Reject Other October 20,

12 Case 1: Re- Scoring Model Performance Improvement versus ExisAng Debit Card ExisAng Dollar DetecAon Rate was 44%, Improved to 60% at the same False PosiAve Rate Re- Scoring model provided a wide range of operaaonal points to trade off alert volume vs detecaon rate Re- Scoring reduced fraud dollar losses by 32% versus the previous model Savings are proporaonal to business volume; for this regional bank this model generated a net benefit of $6M over a two year planning horizon October 20,

13 Case 2: MulAchannel Credit Card Residual Model Performance Model uses enterprise customer data in addiaon to credit card transacaon data DetecAon performance is on the residual fraud, so all detecaon is incremental improvement Model detects 78% of fraud otherwise undetected by exisang system at FPR of 3%, AUC = 0.92 Other risk score vectors for debit card and DDA arise from this mulachannel approach Annual increase in fraud dollars detected across debit card, credit card, and DDA was $48M October 20,

14 Simple OperaAonal Deployment of Rescoring Models Most vendor soluaons for card fraud detecaon have a business rules engine, business rules management system, or policy management component for ad hoc applicaaon of rules aqer the iniaal score is generated The BRMS usually has the capability to call a web service, a library funcaon, or shared object, as well The rescoring model is placed into the BRMS as a final rule that adjusts the score of the transacaon and provides the advice Deployment is simple and straighvorward Leverages exisang risk scoring applicaaon features BRMS October 20,

15 Summary 1. The ROC shows the performance envelope of a fraud detector 2. The payoff matrix and ROC generate the business case for the detector 3. Rescoring model deployment is straighvorward, leveraging the exisang fraud plavorm 4. Rescoring models for card payments have economically apracave business cases Q&A October 20,