Making reviews more consistent and efficient.

Transcription

1 Making reviews more consistent and efficient. PREDICTIVE CODING AND ADVANCED ANALYTICS Predictive coding although yet to take hold with the enthusiasm initially anticipated is still considered by many the next revolution in the practice of law. One reason could simply be the fact that those in the legal industry have generally never been early adopters of technology. However, the explosion of ESI in recent years may indicate that it s only a matter of time before the pure financial benefits combined with the accelerated discovery timelines will necessitate the increased use of predictive coding. Understanding how predictive coding works, and what it can and cannot do, is an important part of evaluating when it may be the right time to jump in.

2 WHAT IS PREDICTIVE CODING? In the most general sense, predictive coding is a machine-learning technology that utilizes assessments provided by reviewers to organize the data from likely responsive to likely non-responsive. The machine will be trained to code the documents based on the initial determinations of key subject matter experts, who will help identify responsive or important documents. It truly is a learning technology, which means it will take time to train the system. The primary benefits of predictive coding are cost savings, greater efficiency and increased consistency. Predictive coding enables the relevant documents to surface faster and eliminates the need for the ofteninefficient linear review. Predictive coding is much more complex than earlier tools which primarily employed culling techniques such as keyword searches, date searches, and phrase matching combined with wild cards. These techniques were used to help minimize the overall amount of data, as well as identify responsive data. Predictive coding allows for a more targeted and analytical review of the documents with substantially less guesswork. While different software platforms utilize varying combinations of underlying technology, they are generally all a means to the same end. The merits of one technology vs. another can be debated forever, but the major players in the market all essentially provide proper workflow and the processes are built around the tool.

3 HOW PREDICTIVE CODING WORKS Understanding how predictive coding works is critical in achieving the best possible results. The predictive coding solution you select may impact the specifics of how the process is done, but the general path is outlined as follows: 1. A subset or seed set of the documents pertaining to the case is provided to the human reviewers. Human reviewers can preselect these documents or a random sample can be selected, generally some combination of the two work best. The application may specify which methodology is most appropriate for a specific vendor tool. 2. Human reviewers will review the individual documents and will tag the document responsive or non-responsive along with more detailed tags if necessary. The tool will then use the input from the document reviewers to process the documents in its database, assigning them predictions based on the reviewers selections. 3. Depending on the software and configuration, the process of fine-tuning the predictive coding is generally an iterative process requiring multiple passes in a waterfall rather than a linear approach. The application may also prompt the human reviewer for additional information so that it may accurately process the documents in the database. As information is gathered and the application builds a sufficient database, the reliability of the coding predictions will become greater this is often referred to as confidence or precision. Precision refers to the percentage of responsive documents found in the set identified by the predictive coding tool. For example, if the computer identifies a set of 100,000 documents as responsive and, after human review, 85,000 of them are deemed relevant, the Precision rate is 85%. By comparison, a traditional keyword search may yield a responsive rate of somewhere around 15%-25%. Recall, on the other hand, points to the number of relevant documents ultimately discovered from the entire dataset. 4. The entire database of documents for this subject will then be processed and the results ranked or scored based on their responsiveness as defined in the initial dataset. Barring differences in predictive coding software, the ultimate number of rounds of documents that need to be reviewed in order to achieve the necessary confidence and recall desired will be largely dependent on the quality and consistency of the initial tagging. It is extremely important that those who are familiar with the details of the matter are making the responsive/non-responsive calls or, at the very least, validating the decisions being made by others.

4 Once the documents for the case have been processed and ranked, they go back to the human reviewers. STATISTICAL SAMPLING IS YOUR FRIEND. Precision and recall rates can be validated through statistical sampling. Let s say that you ve gone through the likely relevant documents that were initially identified by your chosen tool, but you re still left with a few hundred thousand documents. If you were to take a randomized sampling of 10% of those documents that reveals less than 1% of the remaining documents are responsive, it is likely that counsel may end the review, as the likelihood of discoverable evidence yielding anything of substance is minimal. Besides, you do have a claw-back agreement, right? As technology progresses, the necessary reliance on the select few subject matter experts will likely diminish. There is one predictive coding vendor in particular who has minimized the need for one subject matter expert to look at the documents by using a crowdsourcing approach. The same document is distributed to three people if they all agree on the relevancy, then that document is weighted substantially heavier than others. Once the documents for the case have been processed and ranked, they go back to the human reviewers. Predictive coding does not replace the human mind for the final evaluative passes; instead it saves time by getting to the most likely relevant documents quicker. This also speeds up investigations of fact because it is highly probable that documents with the highest rankings contain the most important information. HOW SHOULD YOU USE PREDICTIVE CODING? It is important to remember that predictive coding is not a universal solution. Predictive coding may be highly beneficial in some cases, and not be the best tool to use in others. After becoming familiar with the technology, it will be easier to distinguish which cases will benefit from the use of predictive coding. Generally speaking, those cases that are heavy in , word, and text-based files such as:.doc,.txt. and.pst are all excellent candidates. Those that are heavily financial or excel-based, image-based, or small data sets of below 25,000 documents are generally not considered ideal candidates for the use of predictive coding. Although predictive coding will work on data sets below 25,000, the cost benefit ROI is generally not as great, if it exists at all. For these data sets to be statistically significant, there must be enough documents available to the tool in order to create a mathematically valid sample.

5 ADAPT PREDICTIVE CODING TO YOUR SPECIFIC SITUATION Many law firms and corporations are not ready to turn over documents to opposing counsel or an investigative entity without human eyes looking at each document that has hit on a particular search term. Does this mean that predictive coding cannot be used and the cost benefits of utilizing advanced technology abandoned? Not at all. It just means the workflow needs to be varied. Once the predictive analysis has been done, rather than have documents produced without being examined by counsel, a workflow similar to a linear model can be established that requires review of all documents. This accomplishes a few things: Since the computer has already coded the documents, human eyes are simply validating the decisions made by the system. The documents are often presented in a topically related nature following a streamof-subject model that allows attorneys to move through the material in a logical progression, eliminating the need to constantly jump from topic to topic. If you are reviewing all of the documents, the process is much less likely to receive any pushback from opposing counsel who is unfamiliar with the technological advances of predictive coding. Utilizing the method above can still result in significant cost savings over a traditional linear review as much as 40% in some cases while still providing the overall peace of mind that many feel exists with a more traditional linear review. There are a number of different ways in which you can deploy predictive coding. Standalone applications, service providers, and cloud-based services are the most common ways to deploy predictive coding. The vendor should be able to walk you completely through the process and demonstrate the efficiency of their product. Consulting services specializing in predictive coding, large-scale top-tier vendors, and commodity software shops are all making predictive coding applications and services available. Remember that it is not all about the cost of the service or application, but is primarily about their ability to deliver the service you need in a timely and effective manner, when you need it.

6 ADVANCED ANALYTICS Not ready for full predictive coding? That s okay. Using Advanced Analytics can give you similar results with less of the stigma. Near-Duplicate Detection Standard deduplication applied at the processing phase only suppresses redundant data that is a perfect 100% match. In any universe of culled data there are a large number of records that may be a 95% match that do not get suppressed. Multiple solutions can be created leveraging near-duplicate detection that allow for prioritized review batching, mass tagging, and quality control that increases the efficiency and quality of any review, and makes this one of the most powerful analytic tools available. By grouping like documents together, a single attorney can maintain a stream of consciousness review while comparing textually similar documents side by side. In addition, a reviewer can see in real time if any similar documents have already been coded, making inconsistent coding calls on nearly identical documents a thing of the past. Project managers now have the ability to quickly double-check to see if any documents that are slated for production may be privileged by comparing the subset to previous production decisions on similar documents. Thread Analysis thread analysis allows you to group and view documents together based on chains, bulk tag an entire discussion thread, and quickly identify forks in a conversation. volumes by suppressing all but the unique documents in an reply chain so that a reviewer is looking at the single that contains all previous s inline. Coding decisions can then be rolled up to the entire conversation, eliminating the need to review the same content over and over again. Relationship Analyzer Relationship analysis indexing allows an end-user to visually identify high-volume communications between custodians and unknown parties before ever reviewing a single document. A relationship analyzer is often leveraged as an early case assessment medium to identify aliases of custodians, hone in on date ranges that contain heavy communication, and locate information that would have otherwise gone unnoticed in a traditional review environment. Concept Clustering Concept clustering has made keyword searching a thing of the past by allowing you to search and visually cluster documents by concepts, not keywords, for one or more sources. These clustered concepts can quickly identify important topics and groupings of related documents across and between concepts, allowing you to prioritize your review early on in a case and focus on what matters most right from the start. Entire conversations can be batched together to further eliminate the possibility of inconsistent coding and defensibly reduce data

7 Predictive coding: an emerging necessity. The days when a complex case consisted of tens of thousands of documents have been replaced by cases that can potentially have millions of documents that require review. Courts are becoming more and more aggressive at independently mandating the use of predictive coding in their rulings. Vice Chancellor J. Travis Laster in Delaware Chancery Court recently required both sides to use predictive coding as well as the same vendor (EORHB, Inc., et al v. HOA Holdings, LLC, C.A. No VCL Del. Ch. Oct. 15, 2012). Aggressive adoption of the technology along with the commitment to learn its intricacies makes it easier to manage the large amounts of data, allowing for cost savings and a competitive advantage. For more insight on predictive coding and consultative support on your next ediscovery project, contact an eq office near you today.