The State Of Predictive Coding

Transcription

1 MEALEY S TM LITIGATION REPORT Discovery The State Of Predictive Coding by Royce F. Cohen and Derek I.A. Silverman Stroock & Stroock & Lavan LLP New York A commentary article reprinted from the September 2014 issue of Mealey s Litigation Report: Discovery

2

3 MEALEY S LITIGATION REPORT: Discovery Vol. 11, #12 September 2014 Commentary The State Of Predictive Coding By Royce F. Cohen and Derek I.A. Silverman [Editor s Note: Royce F. Cohen is a Special Counsel in the Litigation Practice Group of Stroock & Stroock & Lavan LLP, and co-chair of the firm s ediscovery and Information Governance Group. Derek I.A. Silverman is an associate in Stroock s Litigation Practice Group, and a member of the firm s ediscovery and Information Governance Group. Both are in Stroock s New York office. Copyright # 2014 by Royce F. Cohen and Derek I.A. Silverman. Responses are welcome.] As the volume of electronically stored information ( ESI ) continues to soar, parties have increasingly turned to predictive coding to reduce the costs of reviewing ESI when disputes arise. Courts are increasingly accepting of the use of this discovery tool; however parties should be aware that it is not a fix-all solution appropriate for every case, and an ediscovery professional should be contacted to assess whether predictive coding should be considered for a specific matter. This article outlines the current state of the law with respect to predictive coding, and offers some guidelines on its use. What Is Predictive Coding? Predictive coding, alternatively referred to as computer-assisted review or technology-assisted review, 1 involves an attorney training a software system, often known as an analytics engine, by tagging a subset of documents, known as a seed set. By coding these documents as either responsive or non-responsive to a document request or issue, the analytics engine can then apply an algorithm which uses the responsive or non-responsive concepts tagged in the seed set to similarly identify the remaining documents in a productionaseitherresponsiveornon-responsive.by training multiple seed sets, and by using statistics and sampling to verify the accuracy of the computer coding, the majority of non-responsive documents can be segregated without necessitating attorney review. A document review project can thus be completed by reviewing only a fraction of the total number of documents, resulting in substantial cost savings over traditional keyword 2 searching, as well as providing a more accurate review, according to recent studies. 3 Da Silva Moore Paves The Way The first (and most often cited) case approving the use of predictive coding is Da Silva Moore v. Publicis Groupe. 4 Da Silva Moore involved a gender discrimination lawsuit in which a defendant was faced with a universe of over three million documents to potentially review. 5 The parties agreed to use predictive coding to attempt to cull down these documents, yet disagreed about how best to implement such review. 6 The parties were then instructed by Magistrate Judge Andrew Peck of the Southern District of New York to submit a joint ESI protocol that would outline the process by which the defendant would use predictive coding in the case. 7 This protocol was so-ordered by Magistrate Judge Peck; however the plaintiffs, whose own proposed protocol had been rejected, reserved their rights to object to its use. 8 When the plaintiffs later exercised their objections to the ESI protocol, District Judge Andrew Carter upheld Judge Peck s rulings, because they are well reasoned and they consider the potential advantages and pitfalls of the predictive coding software. 9 JudgePeck sanalysisshowedthat,inthis case, the approval of predictive coding was easy, because of (1) the parties agreement, (2) the vast 1

4 Vol. 11, #12 September 2014 MEALEY S LITIGATION REPORT: Discovery amount of ESI to be reviewed (over three million documents), (3) the superiority of computer-assisted review to the available alternatives (i.e., linear manual review or keyword searches), (4) the need for cost effectiveness and proportionality under Rule 26(b)(2)(C), and (5) the transparent process proposed by [the defendant]. 10 An important point to note concerning Da Silva Moore, however, is that the court did not order the parties to use predictive coding, as has been suggested on several blogs that have summarized the holding of this case. 11 Rather, here, the parties had agreed on the use of this technology. 12 Issues Left Unresolved By Da Silva Moore While Da Silva Moore was the first case to give judicial imprimatur to the use of predictive coding, it left two key issues unresolved. First, as stated explicitly in the opinion itself, Da Silva Moore did not address what would happen if the producing party wanted to use predictive coding over the objections of the requesting party. 13 Second, the court did not address whether the producing party was required to disclose the documents it had used for its seed set, as the parties again had agreed, as part of their stipulated ESI protocol, to disclose the seed set documents What happens when the requesting party objects to the producing party s use of predictive coding? This question has since been addressed by two courts, one in a county court in Virginia, and another, more recently, by the Southern District of New York. In the first case, Global Aerospace Inc. v. Landow Aviation, the Virginia court, in a summary order, granted the defendants request (over the plaintiff s objections), to proceed with the use of predictive coding for the processing and production of ESI. 15 While the court s decision was issued after briefing, and after 40 minutes of oral argument, unlike Da Silva Moore, it did not provide the parties with guidance concerning the specific protocol for the use of predictive coding (for example, the number of rounds of seed sets), or provide steps to ensure quality control. Following Global Aerospace, Judge Denise Cote of the Southern District of New York, in Federal Housing Finance Agency v. HSBC North America Holdings Inc., likewise permitted JPMorgan Chase, over the objection of the plaintiff, to produce its documents through the use of predictive coding, relying on literature that indicated that predictive coding had a better track record in the production of responsive documents than human review Are seed sets discoverable? As part of the parties stipulated ESI order in Da Silva Moore, the defendant agreed to provide the plaintiffs with the documents that the defendant used for its seed sets, and agreed to disclose the coding that it applied to these documents. 17 The court thus did not have to address the scenario that would arise had the parties refused to come to an agreement regarding seed set disclosure. This precise issue arose in the Northern District of Indiana in In re Biomet M2a Magnum Hip Implant Products Liability Litigation. 18 In this case, after Biomet had performed keyword searching to reduce the number of potentially discoverable documents from 19.5 million to 2.9 million, it turned to predictive coding to further cull the universe of documents. 19 After production, the receiving parties requested that Biomet identify the documents that it had used as its seed set, irrespective of whether they were identified as responsive or nonresponsive to the document requests. 20 Judge Robert Miller, Jr., denied the receiving parties request, and noted that Federal Rule of Civil Procedure 26(b)(1), which makes discoverable any nonprivileged matter that is relevant to any party s claim or defense, provided no authority for the receiving parties to discover how Biomet had used certain documents in the seed set prior to their disclosure. 21 Indeed, were the full disclosure of seed set documents to be the norm when using predictive coding, it would go far beyond what is normally required of parties that perform either a keyword search, or a linear review, as it would force parties to turn over documents that are not even responsive or relevant to an opposing party s document requests. As some commentators have rightfully noted, the selection and tagging of documents in the seed set could also constitute protected attorney work product, and thus a court should not force the disclosure of this process. 22 Predictive Coding Gains Further Traction With The Courts After the Da Silva Moore decision, other courts have become increasingly receptive to predictive coding as an alternative to keyword searching. In a more remarkable move towards the acceptance of this technology, in EORHB, Inc. v. HOA Holdings LLC, Vice Chancellor J. Travis Laster of the Delaware Chancery Court surprised both parties by requiring that they (i) retain a single discovery vendor to be used by both sides, and 2

5 MEALEY S LITIGATION REPORT: Discovery Vol. 11, #12 September 2014 (ii) conduct document review with the assistance of predictive coding, despite the fact that neither party had requested the use of predictive coding in the case. 23 Following objections by the parties, Vice Chancellor Laster amended his order to allow the parties to use different discovery vendors, and based on the low volume of relevant documents expected to be produced by EORHB, agreed that the cost of predictive coding would be outweighed by any practical benefit, and thus allowed EORHB to conduct document review using traditional methods. 24 Despite the court s subsequent about-face, EORHB,aswellasGlobal Aerospace Inc. v. Landow Aviation, andfederal Housing Finance Agency v. HSBC North America Holdings Inc., supra, show that courts across the country are beginning to accept predictive coding as a viable alternative to keyword searching and linear document review. 25 Instances In Which Courts Have Refused To Accept Parties Attempts To Use Predictive Coding While courts are increasingly accepting of the use of predictive coding, in certain cases, when other issues have been present, courts have denied parties requests to use this technology. In one case, Kleen Products LLC v. Packaging Corporation of America, afterthe defendants had conducted keyword searches of their ESI, the plaintiffs argued that these searches would be insufficient to retrieve responsive documents, and requested that the court order the defendants to use predictive coding. 26 The court denied the plaintiffs request, relying on Principle 6 of The Sedona Principles, 27 which provides that [r]esponding parties are best situated to evaluate the procedures, methodologies and technologies appropriate for preserving and producing their own electronically stored information. 28 Recently, in Progressive Casualty Insurance Co. v. Delaney, Magistrate Judge Peggy Leen of the U.S. District Court for the District of Nevada also denied a party s request for the use of predictive coding. 29 In this instance, Progressive had agreed to apply keyword search terms to the approximately 1.8 million documents it had collected. 30 After applying the search terms, Progressive s potentially responsive documents were culled to approximately 565,000 documents. 31 Progressive s counsel began manually reviewing these 565,000 documents, as required by the parties stipulated ESI order, but determined that the manual review was too time intensive and expensive. 32 Progressive then unilaterally began using predictive coding to review its ESI, 33 without the defendants consent to deviate from the stipulated ESI order, and without seeking leave of the court to amend the order to include the use of predictive coding. 34 Finding that Progressive was propos[ing] a do-over of its own invention that lacks transparency and cooperation regarding the search methodologies applied, the court required that Progressive turn over the 565,000 documents that it had originally retrieved by the use of the search terms, without any further review to segregate nonresponsive documents. 35 While the courts in both Kleen Products and Progressive denied parties attempts to use predictive coding, it is important to note that these cases involved very specific factual scenarios. In Kleen Products, a requesting party attempted to force a producing party, over its objections, to use predictive coding in lieu of traditional keyword searching. In Progressive, thepartieshad already stipulated to proceed using keyword searches, when a party, in violation of this court-approved stipulation attempted to use predictive coding. Kleen Products and Progressive should thus not be viewed as a rejection by the courts of predictive coding, but rather a recognition that this technology cannot be forced on an unwilling party, nor can it be used in the dark in clear violation of a court order. Recommendations While courts are increasingly likely to approve a party s request to use predictive coding, a party should assess whether the use of this technology is appropriate as early as possible in the discovery process, and, as can be seen from the recent Progressive case, certainly prior to entering into an ESI stipulation. While disclosure of the use of predictive coding to an opposing party is not a predicate to its use, as with all aspects of discovery, to the extent it is possible, transparency and communication with opposing counsel at the early stages of discovery can be key to avoiding costly later disputes. As Magistrate Judge Peck (the author of the Da Silva Moore opinion) noted in an earlier opinion, [e]lectronic discovery requires cooperation between opposing counsel and transparency in all aspects of preservation and production of ESI. 36 When considering the use of predictive coding, parties should be aware that not all cases lend themselves to this relatively newly-accepted technology. The following 3

6 Vol. 11, #12 September 2014 MEALEY S LITIGATION REPORT: Discovery factors are a non-exclusive list to consider when evaluating predictive coding for a case: Size of the document collection: Predictive coding can be a useful tool to avoid a manual review of hundreds of thousands, or even millions of documents. However, as seen in EORHB, Inc. v. HOA Holdings LLC, when faced with small sets of data, the cost of implementing predictive coding may not be a cost-effective solution over traditional keyword searching. Types of documents in the collection: Parties should survey the types of documents to be reviewed to determine if the documents lend themselves to the concept-based searching offered by predictive coding. For example, if the document universe contains a high volume of spreadsheets consisting largely of numerical figures, the predictive coding analytics engine may have difficulty identifying responsive and nonresponsive concepts from these spreadsheets. Likewise, if the document universe contains a large number of hard-copy documents, errors in converting these documents into machinereadable text 37 could prevent an analytics engine from accurately evaluating the responsiveness of a document. Knowledge of the facts of the case: If the attorney reviewing and tagging the seed set has a comprehensive understanding of the facts of the case, and can thus accurately and quickly determine which documents are relevant to the document requests, predictive coding can be an effective tool to reduce the time and cost of a review. However, often attorneys are only able to learn about the relevant issues of a case, particularly when dealing with a new client, by performing a substantive review of the documents. In this latter scenario, the reviewing attorney may not be able to accurately and quickly code the seed set, thus greatly undermining the efficiencies that would otherwise be gained from predictive coding. Review timeline: If the review has a short deadline, predictive coding, when done correctly, can be a relatively faster process than traditional review. However, a longer deadline allows parties the flexibility to consider other options for review, or to consider using a combination of predictive coding with traditional keyword reviews. Cost: Vendors and law firms can offer different pricing solutions for both predictive coding, and other traditional forms of review. Generally, when using predictive coding, there will be a higher cost for both use of the software, and for the increased time required for technical support from a firm or vendor s litigation support professionals. However, this cost can be offset by a reduction in the cost of the attorney time that would otherwise be required for a manual review of the documents. Above all, parties should consider consulting with an ediscovery attorney at the outset of discovery to evaluate whether predictive coding would be beneficial for a case, to help balance the benefit of predictive coding with the cost involved in using this technology, and to help develop a defensible ESI plan going forward. Endnotes 1. Predictive coding is a subset of computer-assisted or technology-assisted review ( TAR ), however the terms have largely been used interchangeably. 2. Keyword searching involves applying a single search term, or a combination of multiple search terms, to a set of documents, and then sequentially reviewing the documents containing these terms for responsiveness to document requests. 3. See Maura R. Grossman & Gordon V. Cormack, Technology Assisted Review in E Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, XVII RICH. J.L. & TECH. 11, 37 (2011) (reporting that manual reviewers identified between 25% and 80% of relevant documents, while technology-assisted review returned between 67% and 86%) F.R.D. 182 (S.D.N.Y. 2012) (Peck, M.J.), adopted sub nom., Da Silva Moore v. Publicis Groupe SA, No. 11 CIV. 1279(ALC)(AJP), 2012 WL (S.D.N.Y. Apr. 26, 2012). 5. Id. at

7 MEALEY S LITIGATION REPORT: Discovery Vol. 11, #12 September Id. at Id. at Id. at 187 n Da Silva Moore v. Publicis Groupe SA, No. 11 CIV (ALC)(AJP), 2012 WL , at *2 (S.D.N.Y. Apr. 26, 2012). 10. Da Silva Moore, 287 F.R.D. at Id. at 183 n Id. 13. Id. at Id. at 186 ( MSL...had agreed to provide all 2,399 documents (and MSL s coding of them) to plaintiffs for their review ) (emphasis added). Note, however, that the parties agreement came after the urging of Judge Peck during a discovery conference, who informed the defendants that: If [they] do predictive coding, [they] are going to have to give [their] seed set, including the seed documents marked as nonresponsive to the plaintiff s counsel so they can say, well, of course you are not getting any [relevant] documents, you re not appropriately training the computer. Id. at 185. Because of Judge Peck s statements to the parties, some litigants continue to disagree as to whether the court s opinion stands for the proposition that the contents of a party s seed set must be disclosed to the opposing party. See Gordon v. Kaleida Health, No. 08-CV-378S F, 2013 WL , at *2-*3 (W.D.N.Y. May 21, 2013); Hinterberger v. Catholic Health Sys., Inc., No. 08-CV-380S F, 2013 WL , at *2 (W.D.N.Y. May 21, 2013). 15. No. CL 61040, 2012 WL (Va. Cir. Ct., Loudoun Cnty., Apr. 23, 2012). 16. No. 11 CIV. 6189(DLC), 2014 WL , at *3 (S.D.N.Y. Feb. 14, 2014). 17. Da Silva Moore, 287 F.R.D. at Cause No. 3:12 MD 2391, 2013 WL , at *2 (N.D. Ind. Aug. 21, 2013). 19. Id. at * Id. 21. Id. at *2. The court noted, however, that since Biomet had not identified any harm that would result from identifying which of the responsive documents that Biomet had already produced were in Biomet s seed set, Biomet s cooperation with its opposing counsel fell below the level of what has been endorsed by the Sedona Conference1. Id. The court thus urged Biomet to reconsider its refusal to identify its seed set documents, but came short of ordering Biomet to do so. Id. 22. H. Christopher Boehning & Daniel K. Toal, Seed Set Documents Should Not Be Discoverable,N.Y.L.J.Vol. 251, No. 2 (Feb. 3, 2014) (citing Karl Schieneman & Thomas C. Gricks III, The Implications of Rule 26(G) on the Use of Technology-Assisted Review, 7 FED. CTS. L. REV. 239, 262 (2013) ( To the extent that development of the seed set reflects attorney work product, the certification obligations of Rule 26(g) clearly do not require disclosure. )). 23. C.A. No VCL, 2012 WL (Del. Ch. Oct. 15, 2012). 24. EORHB, Inc. v. HOA Holdings LLC, C.A. No VCL, 2013 WL , at *1 (Del. Ch. May 6, 2013). 25. See also F.D.I.C. v. Bowden, No. CV , 2014 WL , at *13 (S.D. Ga. June 6, 2014) (ordering that if there are further disagreements concerning an ESI-protocol, the parties should consider the use of predictive coding). 26. No. 10 C 5711, 2012 WL , at *5 (N.D. Ill. Sept. 28, 2012). 27. The Sedona Principles represent a best-practice guideline for ediscovery that evolved out of the discussions of The Sedona Conference1, a nonpartisan law and policy think tank. 28. EORHB, Inc., 2013 WL , at *5 (quoting The Sedona Conference1 Best Practices Commentary on the Use of Search and Information Retrieval Methods in E Discovery, 8 SEDONA CONF. J. 189, 193 (Fall 2007)). 5

8 Vol. 11, #12 September 2014 MEALEY S LITIGATION REPORT: Discovery 29. No. 2:11-CV LRH-PA, 2014 WL (D. Nev. July 18, 2014). 30. Id. at * Id. 32. Id. 33. Progressive applied predictive coding only to the 565,000 documents that were returned after a keyword search, rather than to the universe of documents that it had collected, contradicting its own vendor s recommended best practices for the use of predictive coding. Id. at * Id. at * Id. at *10-*11. In denying Progressive s attempt to use predictive coding, Magistrate Judge Leen s opinion misstated the amount of disclosure required from a party concerning a seed set. Specifically, Judge Leen stated that courts have required the producing party to provide the requesting party with full disclosure about the technology used, the process, and the methodology, including the documents used to train the computer. Id. at *10 (emphasis added) (citing Da Silva Moore, 287 F.R.D. at 191; In re Actos (Piglitazone Prods. Liab. Lit.), No. 6:11 md 2299, 2012 WL (W.D. La. July 27, 2012)). However, the courts in Da Silva Moore and In re Actos, to which Magistrate Judge Leen cited for this proposition, did not require the parties to disclose which documents they had used as their seed set. Rather, the parties had agreed, as part of their stipulated ESI protocols, to turn over these documents. Da Silva Moore, 287 F.R.D. at 186 ( MSL... had agreed to provide all 2,399 documents (and MSL s coding of them) to plaintiffs for their review ) (emphasis added); In re Actos, 2012 WL , at *3 ( The Parties have discussed the methodologies or protocols for the search and review of ESI... and the following is a summary of the Parties agreement... ) (emphasis added). 36. William A. Gross Const. Associates, Inc. v. Am. Mfrs. Mut. Ins. Co., 256 F.R.D. 134, 136 (S.D.N.Y. 2009). 37. The conversion of images to text is accomplished through Optical Character Recognition ( OCR ) software. 6

9

10 MEALEY S LITIGATION REPORT: DISCOVERY edited by Mark Rogers The Report is produced monthly by 1600 John F. Kennedy Blvd., Suite 1655, Philadelphia, PA 19103, USA Telephone: (215) MEALEYS ( ) mealeyinfo@lexisnexis.com Web site: ISSN