Predictive Coding: Understanding the Wows & Weaknesses

Size: px

Start display at page:

Download "Predictive Coding: Understanding the Wows & Weaknesses"

Clare McCoy
8 years ago
Views:

1 Predictive Coding: Understanding the Wows & Weaknesses Bryan Callahan, CPA, CFF, CFE, CVA Managing Consultant Forensics & Valuation Services Lanny Morrow, EnCE Supervising Consultant Forensics & Valuation Services experience expertise // Elevate your understanding of key forensics concepts with guidance from BKD Forensics Institute.

2 Definitions & Aliases Predictive coding is the electronic coding, organization, and prioritization of entire sets of electronically stored information ( ESI ) according to their relation to discovery responsiveness, privilege, and designated issues before and during the legal discovery process. Lawyers control this process by specifying relevant criteria. --Forbes The analysis and identification of relevant ESI, assisted by artificial intelligence. Some also call it technology-assisted review. 2

3 The Reason Explosion of data Big Data 95% of data never leaves digital domain Considered greatest ediscovery challenge by 2015 Predictive coding has greatest potential to improve ediscovery by

4 What Is Unstructured Data? 4

5 Overview of the Process 1. Predictive coding tool selects random sample of documents (called seed set ) 2. Senior attorney(s) review this set & assign each item as relevant or not relevant 3. Artificial Intelligence (AI) component performs find more like this search 4. Subsequently retrieved documents are again reviewed & assigned a category 5. Process continues until no more items are found to be relevant 5

6 Under the Hood Seed set Relevance is sometimes moving target in a case. Random sampling protects against bias AI learns from initial items marked relevant, then assigns probability to every single item Applies true latent semantic analysis AI compares attorney decision to its own & continuously retrains itself Better systems will have game component Process is keyword independent 6

The update was easy to follow and was very well received by the group.

7 Latent Semantics From: To: The Boss Everyone I wanted to congratulate Lisa on her really outstanding presentation at the meeting! The update was easy to follow and was very well received by the group. Great job, Lisa! Kim also gave a nice presentation. We appreciate you both. Lisa Kim nice

8 Real Example To: Briber From: Employee taking bribe Thank you for the little token I park it in the driveway so my neighbors can be jealous! It screams on the open road (wow...!) PS-I promise it s just between you and me! Token Thank you park driveway road gift vehicle Between you and me promise conspire!,!,! Wow Thank you excitement

9 Benefits & Best Uses Forget smoking gun 85 to 90% reduction in review is primary benefit Every part of process is documented, repeatable & can be statistically validated Excellent in early case assessment & investigations for rapid identification of relevant materials Best use is in tandem with traditional keyword, concept & similarity searches already established Reduces inherent human error 9

10 2011 RAND Study on Human Review Taken together, this body of research shows that groups of human reviewers exhibit significant inconsistency when examining the same set of documents for responsiveness under conditions similar to those in large-scale reviews Human error in applying the criteria for inclusion appears to be the primary culprit [regarding the lack of accuracy], not a lack of clarity in the document s meaning or ambiguity in how the scope of the production demand should be interpreted. In other words, people make mistakes, and based on the evidence, they make them regularly when it comes to judging relevancy or responsiveness. 10

11 Weaknesses, Myths & Concerns Not man vs. machine it s more about augmented intelligence than artificial intelligence This is not new technology, just really improved Vendors jockeying for market position, so lots of proprietary flavors of technology Still under consideration by courts (when both parties are not in agreement) AI is still far from perfect proceed with caution 11

12 Da Silva Moore Decision Da Silva Moore decision in Southern District of New York appropriateness of predictive coding 1. Parties agreement 2. Vast amount of ESI to be reviewed 3. Superiority of computer-assisted review to available alternatives 4. Need for cost effectiveness & proportionality 5. Transparent process proposed by Defendant Da Silva Moore v. Publicis Groupe, 287 F.R.D. 102 (SDNY 2012) 12

13 Current Stance by Courts July 2012 Nat l Day Laborer Org. Network v United States ICE Agency, 877 F. Supp. 2d 87, 109 (SDNY) December 2012 Robacast, Inc. v. Apple, Inc., No (D. Del.) January 2013 EORHB, Inc. v. HOA Holdings, Inc., No VCL (Del. Ch. Ct.) March 2013 Chevron Corp. v. Donziger, No. 11 Civ. 0691, 2013 U.S. Dist. LEXIS (SDNY) and Harris v. Subcontracting Concepts, LLC, No. 1:12-MC-82, 2013 U.S. Dist. LEXIS (SDNY) 13

14 The Future of Predictive Coding It s hard to predict things, especially the future Yogi Berra Rise of evidentiary expert? Can attorneys cooperate enough? It s not perfect, is that ok? 14

15 Predictive Coding: Understanding the Wows & Weaknesses Bryan Callahan, CPA, CFF, CFE, CVA Managing Consultant Forensics & Valuation Services Lanny Morrow, EnCE Supervising Consultant Forensics & Valuation Services experience expertise // Elevate your understanding of key forensics concepts with guidance from BKD Forensics Institute.

16 BKD Forensics Institute Upcoming Schedule Valuation Strategies for 2013 & Beyond: Tax & Business Succession Planning After the Fiscal Cliff o Wednesday, May 1, 2013 o o 11:30 a.m. 12:30 p.m. CST Presented by Teal Dakan, CPA & Carol Lewis, CPA, ABV Whistle-Blowers & Fraud Hotlines: Reporting & Preventing Fraud o Wednesday, May 8, 2013 o o 11:30 a.m. 12:30 p.m. CST Presented by Julia Swafford, CPA, CFE bkd.com/fi 16

17 Starbucks Gift Card & Certificate To help promote the event as associate training tool, we are offering a bonus o Associates who attend all three sessions will receive a Starbucks gift card & Certificate of Participation from BKD 17

18 Slides & Webinar Archive Today s presentation slides are available at bkd.com/fi. Recorded copy of webinar will be available at the same location at conclusion of webinar series If you have any questions, please contact Dane dryals@bkd.com 18

19 Thank you for attending. Learn more at bkd.com/fi.

Predictive Coding: How to Cut Through the Hype and Determine Whether It s Right for Your Review

Predictive Coding: How to Cut Through the Hype and Determine Whether It s Right for Your Review ACEDS Webinar April 23, 2014 Sponsored by Robert Half Legal 1 2014 Robert Half Legal. An Equal Opportunity