Predictive Coding: How to Cut Through the Hype and Determine Whether It s Right for Your Review ACEDS Webinar April 23, 2014 Sponsored by Robert Half Legal 1 2014 Robert Half Legal. An Equal Opportunity Employer. All rights reserved.
ACEDS Membership Benefits Training, Resources and Networking for the E-Discovery Community! Exclusive News and Analysis! Weekly Web Seminars! Podcasts! On-Demand Training! Networking! Resources! Jobs Board & Career Center! bits + bytes Newsletter! CEDS Certification! And Much More! ACEDS provides an excellent, much needed forum to train, network and stay current on critical information. Kimarie Stratos, General Counsel, Memorial Health Systems, Ft. Lauderdale Join Today! aceds.org/join or Call ACEDS Member Services 786-517-2701 2
Federal Rules amendments, judicial guidance, taboos & more!! 14 panels, 41 speakers, 14 networking events! Presenters include:! Federal judges including Hon. David Waxse, Rules Advisory Committee member, government litigators, corporate attorneys! Register or learn more at EDiscoveryConference.com! Use code RHL20 to save 20%
What We ll Cover Predictive coding: Why all the fuss? Pros and cons of technology-assisted review, including: Understanding the process Recent court rulings and developments Risks, benefits and advantages Evaluating predictive coding for your case Balancing key elements and considerations Emerging best practices Q&A 4
Our Speakers Amy Jane Longo, senior trial counsel, Securities and Exchange Commission, Division of Enforcement, Los Angeles Regional Office Dera J. Nevin, managing director, re:discovery Law PC David S. Panzer, senior litigation counsel, DynCorp International Joel Wuesthoff, Esq., director, Robert Half Legal ediscovery Services Charles A. Volkert, Esq., executive managing director, Robert Half Legal ediscovery Services 5
e-discovery Shows No Signs of Slowing Down One-quarter of lawyers plan to increase spending on e-discovery and managed review services in the next two years Strategies and tools for reducing data management costs are being sought Courts, defendants, plaintiffs and counsel demanding alternative methods to perform reasonable, efficient and defensible data 6
Predictive Coding Defined Predictive coding involves training a computer (by way of an electronic discovery software or review tool) to recognize and identify the documents in a review set that are relevant and/or responsive to discovery requests. The software learns how to code from the case team's attorneys, as it tracks their review decisions (i.e., which tags are checked) and uses mathematical algorithms to predict based on the contents and characteristics of documents which tags the attorneys would have checked. Source: Predictive Coding: A Primer by Amy Jane Longo, Usama Kahf, Mealey s Litigation Report, Discovery, 2013. 7
Predictive Coding: Risks and Considerations Perceived Risk Response It hasn t been ruled on in court It s not defensible Multiple U.S. courts have approved and even actively encouraged the use of predictive coding I won t use what I can t explain We can be attacked over its use Predictive coding has been used in hundreds of cases Keyword search has known issues and technology-assisted review is It s too new considered equally/more accurate All search engines require proactive I don t want to be on the bleeding testing and validation edge 8
Artificial Intelligence (AI): Common Questions and Misconceptions Could AI replace first- or even second-tier review in the nation s largest litigations in the near future? How far into the future do we go? Has AI achieved the same or a higher-level of reasoning and legal capability to stand in for lawyers and contract attorneys in a meaningful and defensible manner? Is the market even asking the right questions about the capabilities of these new technologies? 9
Benefits Predictive coding can be beneficial if it can: Prioritize relevant documents for review Significantly reduce the volume Minimize the risks of inconsistent interpretations 10
ESI Problems Can Affect a Ruling More than one in 10 (12 percent) lawyers we surveyed said in the last three years, issues or problems with collecting or reviewing ESI negatively affected a case or ruling for their firm. Source: Survey of 350 lawyers among the largest law firms and corporations in the U.S. and Canada. The survey was commissioned by Robert Half Legal and conducted by an independent research firm. 11
Challenges Proper testing and deployment No one-size-fits-all tool Continuously evolving tools Need for continuous training 12
Is Predictive Coding Right for Your Document Review? Ask these questions: 13 To what extent will it support the legal review process? Is there an auditable process documentation trail? Does it provide an improved workflow that increases the accuracy and expedites the decision-making process? Does it require proper implementation throughout an enterprise?
Relevant Court Decisions Da Silva Moore v. Publicis Groupe SA, No. 11 Civ. 1279 (ALC) (AJP), 2012 U.S. Dist. LEXIS 58742 (S.D.N.Y. Apr. 26, 2012) Global Aerospace Inc. v. Landow Aviation, L.P., No. CL 61040 (Va. Cir. Ct. Apr. 23, 2012) In re Actos (Pioglitazone) Products Liability Litigation, No. 6:11-md-2299, (W.D. La. July 27, 2012) In re: Biomet M2a Magnum Hip Implant Products Liability Litigation, No. 3:12-MD-2391 (N.D. Ind. Apr. 18, 2013) 14
Proposed Changes to the Federal Rules of Procedure and Evidence 15 Proposed amendments adopted by the Civil Rules Advisory Committee; to be considered by the Standing Committee on May 29-30, 2014 If approved, would be submitted to the U.S. Judicial Conference for possible enactment in Dec. 2015 Changes that could affect the landscape for predictive coding: Rule 1, encouraging greater cooperation by parties Rule 16, shortening time for scheduling orders Rule 26(b)(2)(1), relocating the proportionality standard Rule 37(e), articulating bases for sanctions for failure to preserve discoverable information, including factors based on culpability and prejudice (Source: "The 2013 Civil Rules Package As Adopted," Thomas Y. Allman, former general counsel, current adjunct professor, University of Cincinnati College of Law)
Impact on the Practice of Law Foundations of search technology Inadequacies of keyword searches Support Vector Machines and Latent Semantic Indexing Visualization Determining when to utilize 16
Establishing Processes Who should be involved? Determining testing type for accuracy Methodology design and testing Information-sharing with opposing counsel When to use Defensibility Reports 17
Implement Appropriately Understand limits and possibilities Workflow and process Rely on specialists Results-driven validation Coordinate effort internally 18
Key Takeaways Ø Understand appropriate times to use predictive coding Ø Educate yourself on expanding case law on predictive coding Ø Recognize that it s just a tool and requires appropriate people and processes to work effectively Ø Solicit technology options from providers with financial modeling for each option 19 Ø cc
Questions? Need additional resources? Download our latest research report at futurelawoffice.com. 20 2014 Robert Half Legal. An Equal Opportunity Employer. All rights reserved. Name, title of presenter
Appendix Case Studies and Additional Court Decisions 21
Case Study: Da Silva Moore v. Publicis Group Judge Peck determined: The use of predictive coding is appropriate considering: The parties agreement [to use the process] The vast amount of ESI to be reviewed (over three million documents) The superiority of computer-assisted review to the available alternatives (i.e., linear manual review or keyboard searches), The need for cost effectiveness and proportionality under Rule 26(b)(2)(C), and The transparent process proposed by [defendant] MSL. Id. at *35-36. U.S. District Judge Andrew Carter subsequently adopted Judge Peck's opinion on April 26, 2012 See Da Silva Moore v. Publicis Groupe SA, No. 11- CV-1279, 2012 U.S. Dist. LEXIS 58742 (S.D.N.Y. April 26, 2012) 22
Case Study: Global Aerospace Inc. v. Landow Aviation 23 Virginia Circuit Court Judge James H. Chamblin issued the first known state court order approving the use of predictive coding for electronic discovery Defendants requested the use of predictive coding when the parties could not agree on how to review the large volume of documents in defendants possession Defendants argued that a traditional manual review of the documents would cost an estimated $2 million Defendants also argued that using predictive coding would lead to the location of more responsive documents (75%) than a traditional manual review (60%) Plaintiffs objected to the use of predictive coding Not as effective as traditional manual review and that, regardless of the process implemented Defendants should produce all responsive documents Judge Chamblin approved the use of predictive coding despite the plaintiffs objection that traditional manual review of electronic discovery would yield more accurate results and noted that plaintiffs are not without recourse, as they are still permitted to raise with the Court an issue as to completeness or the contents of the production or the ongoing use of predictive coding See Global Aerospace Inc. v. Landow Aviation, L.P., No. CL 61040 (Va. Cir. Ct. Apr. 23, 2012)
Case Study: In Re Actos Cooperative Case Management Order Parties agreed on the methodology of the review, the vendors and software Three experts from each side to review sample documents to ultimately train the system Defendants allowed to pre-screen sample documents for privilege Bi partisan to test documents deemed non-responsive Parties agreed to review all documents above an agreed upon relevant score See In re Actos (Pioglitazone) Products Liability Litigation, No. 6:11-md-2299, (W.D. La. July 27, 2012) 24
Case Study: Biomet Background Initial Total = 19.5 million documents Keyword culling reduced corpus to 3.9 million Deduplication further reduced it to 2.5 Statistical sampling suggested that between.55 and 1.33 percent of the unselected documents would be responsive Defendants loaded 2.5 million documents into predictive coding engine using team of 8 contract attorneys to train tool and assess the results 25
Case Study: Biomet Plaintiffs argue it was inappropriate to cull as performed by defendant and that keywords did more harm than good Court determined process used by defendant satisfied its FRCP duties and the Seventh Circuit Principles Relating to the Discovery of Electronically-Stored Information Update: August 13, 2013 - Court ruled on whether Defendant must identify discoverable documents in the seed set used to train the predictive coding algorithm See In re: Biomet M2a Magnum Hip Implant Products Liability Litigation, No. 3:12-MD-2391 (N.D. Ind. Apr. 18, 2013) 26
Case Study: Biomet Defendant declined to provide specifics on which documents were used to train Plaintiff argued that Sedona Cooperation Proclamation mandated sharing such information, court disagreed 27
Additional Court Decisions 28 National Day Laborer Organizing Network v. United States Immigration and Customs Enforcement Agency, No. 10 Civ. 3488 (SAS), 2012 U.S. Dist. LEXIS 97863 (S.D.N.Y. July 13, 2012) EORHB, Inc. v. HOA Holdings LLC, C.A. No. 7409- VCL (Del. Ch. Oct. 15, 2012) Chevron Corp. v. Donziger, 11 Civ. 0691 (LAK), 2013 U.S. Dist. LEXIS 36353 (S.D.N.Y. Mar. 15, 2013) Gabriel Technologies Corp. v. Qualcomm Inc., Civ. No. 08cv1992 (MDD), 2013 U.S. Dist. LEXIS 14105 (S.D. Cal. Feb. 1, 2013)