An Open Look at Keyword Search vs. Predictive Analytics

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "An Open Look at Keyword Search vs. Predictive Analytics"

Transcription

1 catalystsecure.com ARTICLE An Open Look at Keyword Search vs. Can Keyword Search Be As Effective as TAR? John Tredennick, Esq. Founder and CEO, Catalyst Repository Systems 2015 Catalyst Repository Systems. All Rights Reserved.

2 Can Keyword Search Be As or More Effective Than Technology Assisted Review at Finding Relevant Documents? A client recently asked me this question and it is one I frequently hear from lawyers. The issue underlying the question is whether a TAR platform such as our Insight Predict is worth the fee we charge for it. The question is a fair one and it can apply to a range of cases. The short answer, drawing on my 20-plus years of experience as a lawyer, is unequivocally, It depends. Let me walk through my reasoning. The Question The question really is twofold: Can you get equal or better results with keyword search than with a product such as Predict? If so, can you do it at lower cost considering the hourly costs to develop and run the searches? To answer these questions, we have to consider the following: 1. How many hours will it take to develop the searches and test them, and at what cost? The question is twofold: can you get equal or better results with keyword search than with a product such as Predict? If so, can you do it at lower cost? 2. How many documents do you end up reviewing to complete the process (precision)? 3. What level of recall was obtained? 4. The goal in all of this is to reduce the total cost of review across your cases. If it were my case, I wouldn t care how that got done, just that it got done reliably. Measuring Success To start the inquiry, we need a reliable measure for success. Several years ago, a different client challenged Predict because we ran a parallel process to keyword search that suggested the team review more documents than hit on the keyword searches. You can read that 2013 article at catalystsecure.com/blog, My Key Word Searches are Better than Your Predictive Ranking Technology! 2015 Catalyst Repository Systems. All Rights Reserved. 2

3 To explain why Predict s results made sense, I created this chart to show the four possible states of the search and Predict results. As shown in the diagram, both approaches agreed on the documents in the top right and bottom left quadrants (specifically, on documents likely relevant and likely not relevant). The disagreement came in the top left and bottom right quadrants. In brief, Predict found a lot of documents missed by the keyword search and a good number of false hits from the keyword search. While keyword search can be effective in finding relevant documents, research has shown that it can suffer from both low recall and poor precision. Information retrieval scientists would couch this discussion in terms of precision and recall. The suggestion from the chart above is that keyword searches might have failed in terms of recall, at least if the goal was to reach a level of recall above 50%. However, precision is equally important in calculating the total cost of review. If the method does well with recall but retrieves a lot of irrelevant documents in the process, you meet the recall goal but at the expense of increased review costs. So Which Approach is Better? Here is where the lawyer answer comes in: It depends. While keyword search can be effective in finding relevant documents, research has shown that it can suffer from both low recall and poor precision. The oft-touted 1985 David Blair and M.W. Maron study, 2015 Catalyst Repository Systems. All Rights Reserved. 3

4 An Evaluation of Retrieval Effectiveness for a Full-Text Document- Retrieval System, sets a framework for the discussion. In the case they studied, lawyers felt their keyword searches were quite effective, finding 75% of the relevant documents. In fact, according to the scientists, the team had found (on average) just 20% of the relevant documents. 1 They were swayed by the seeming precision of the searches (seeing a lot of relevant documents) but didn t realize they were missing a lot of other relevant documents which didn t hit on the searches. You can read more about the study in my November 2015 blog post, Revisiting the Blair and Maron Study: Evaluation of Retrieval Effectiveness for a Full-Text Document-Retrieval System. The antidote to this problem is to use broader searches, but that comes at the cost of lower precision. That means while recall can be improved over the reported average of 20%, the gain would likely come at a reduction in precision. Specifically, the team would be required to look at a lot more irrelevant documents. Total review costs would go up accordingly. Some litigation support teams address this by using a sophisticated process of iterating keywords over a series of searches, sampling results and then refining the keyword searches. That can certainly improve results, but how much and under what circumstances are questions that require measurement. And, in doing so, you would want to include the cost of developing, refining and running those searches as well. In contrast, most predictive analytics tools (and certainly Insight Predict) are designed algorithmically to provide the best bang for the buck across most cases. The goal is to bring a desired level of recall (e.g. 75%, 80%, 90%, 95%) while reviewing the fewest possible documents. Predict uses tens of thousands of features with both positive and negative weighting during the process, and it refines the training literally hundreds often thousands of times during the review. While I can t explain how our proprietary algorithm works, it uses a sophisticated weighing of document features found through a continuous ranking process. The main difference here is that Predict uses tens of thousands of features with both positive and negative [1] It is also important to note that this 20% recall figure was an average across many different trials in the study. In some trials, keywords performed much better, but they performed much worse in others. This wide variance in search term performance is another reason to be cautious when comparing techniques from only a few trials Catalyst Repository Systems. All Rights Reserved. 4

5 weighting during the process, and it refines the training literally hundreds often thousands of times during the review. To be sure, humans don t build the searches (nor could they). But humans still control the process in that they identify the documents from which the searches are built. Comparing the Two Approaches So, which approach is better? My first answer is that either approach could be better in a specific case, depending on facts and circumstances. For example, say you were requested to produce all documents with the company name Acme in them. In that case, keyword search would be the quickest and most effective way to go. We recently had a chance to participate in the Total Recall track of the TREC information retrieval program, which is sponsored by the National Institute of Standards and Technology (NIST). The Total Recall track was led by Gordon Cormack and Maura Grossman, with help from many others. Its goal was to use machine learning to find relevant documents quickly, and with the least review possible. TREC has prohibitions on releasing results so I won t go into detail at this point. However, I can say that a number of the topics lent themselves to keyword search as readily as machine learning. For example, one topic was affirmative action, which is not a typical e-discovery subject. 2 We used Predict to find relevant topics but later learned through examination of the answers that a simple keyword search for affirmative action or one world would have returned more than 90% of the relevant documents with 63% precision. Several other topics out of the 30 total also could have been handled through the right combination of keywords. Research suggests that predictive analytics and a continuous active learning process will outperform keyword search both in terms of effectiveness and cost in most cases and is a much safer bet as a standard practice. How often is that the case in the real world of litigation and regulatory investigations? If your answer is often or always, then keyword search might be the answer for you. In my experience, and that of many others, keyword searches are useful but not effective or efficient at finding a high percentage of relevant documents with high precision. [2] I should point out that the Total Recall track was not aimed at e-discoverists. Rather, several e-discovery focused teams participated because it was as close to e-discovery as the tracks got. It was fun to participate, but the topics were not set up to match what we are used to in the legal realm, nor did they purport to be Catalyst Repository Systems. All Rights Reserved. 5

6 Ultimately, in each case you have to measure not only results, but the cost of getting those results. Our experience and the research out there suggests that predictive analytics, and particularly a continuous active learning process will outperform keyword search both in terms of effectiveness and cost in most cases and is a much safer bet as a standard practice. Keyword vs. Predict To be clear, we believe in the power of human-generated keyword searches. In fact, we encourage clients to use them for finding seed documents (as opposed to random selection used by some vendors) and for chasing down low-hanging fruit throughout the review process. For us, the question is not whether to use keyword searches. Rather, the question is how to use them most effectively during the review. Different cases will likely suggest different approaches. If I were in a corporate counsel s shoes, I would be asking how we might go about determining which cases might be better handled by keyword search rather than Predict. Independent research strongly suggests that keyword search can be less than optimal in many cases, missing large numbers of relevant documents (low recall) while bringing back too many non-relevant documents (low precision). Without doubt, smart searchers running an iterative process can improve results but how much and at what cost? I am not aware of any research suggesting that keyword search can match the results of a good predictive analytics process across the board. In particular, it is rare to hear about keyword searches getting much above 75 or 80% recall. Perhaps others experiences are different but any conclusions should be backed by rigorous analysis (across a wide variety of cases) of the documents that did not return as well as those that did. An added benefit of CAL is its ability to incorporate new documents into the collection at virtually any point in the review without sacrificing previous effort. Given that you can rarely tell in advance which case is right for a keyword-only approach, it is hard to see why a legal team wouldn t include predictive analytics as a core strategy. At the least, we need to measure the cost of Predict against the hourly cost of developing and running a keyword process. To that we would add the results differential (recall and precision) before making a comparison Catalyst Repository Systems. All Rights Reserved. 6

7 We recognize that TAR is a new process and that some clients may be leery about paying an extra charge for Predict without being certain of the results. For that reason, I initiated an unconditional guarantee for Predict costs. Simply put, if you use Predict and are unhappy with the results for any reason in the first 90 days, just say so. We will turn it off and refund the Predict fees without question. The bottom line is that we believe Predict with keyword search will be more effective than keyword search alone in almost every case. We believe that with such certainty that we are willing to put our proverbial money where our mouth is. Learn More catalystsecure.com Follow About the Author John Tredennick is the founder and CEO of Catalyst. John is passionate about the role of search in e-discovery. Before founding Catalyst in 2000, he was a trial lawyer and litigation partner. John has been a frequent speaker on legal and technology issues for more than 30 years. He s also written and edited five best-selling books and countless articles on litigation and technology issues Catalyst Repository Systems. All Rights Reserved. 7

How It Works and Why It Matters for E-Discovery

How It Works and Why It Matters for E-Discovery Continuous Active Learning for Technology Assisted Review How It Works and Why It Matters for E-Discovery John Tredennick, Esq. Founder and CEO, Catalyst Repository Systems Peer-Reviewed Study Compares

More information

Predictive Coding Helps Companies Reduce Discovery Costs

Predictive Coding Helps Companies Reduce Discovery Costs Predictive Coding Helps Companies Reduce Discovery Costs Recent Court Decisions Open Door to Wider Use by Businesses to Cut Costs in Document Discovery By John Tredennick As companies struggle to manage

More information

Five Reasons the Cloud Beats an Appliance for Big Data E-Discovery

Five Reasons the Cloud Beats an Appliance for Big Data E-Discovery 877.557.4273 catalystsecure.com ARTICLE Five Reasons the Cloud Beats an Appliance for Big Data E-Discovery John Tredennick, Esq. Founder and CEO, Catalyst Repository Systems Big data can mean big headaches

More information

E-discovery Taking Predictive Coding Out of the Black Box

E-discovery Taking Predictive Coding Out of the Black Box E-discovery Taking Predictive Coding Out of the Black Box Joseph H. Looby Senior Managing Director FTI TECHNOLOGY IN CASES OF COMMERCIAL LITIGATION, the process of discovery can place a huge burden on

More information

Judge Peck Provides a Primer on Computer-Assisted Review By John Tredennick

Judge Peck Provides a Primer on Computer-Assisted Review By John Tredennick By John Tredennick CEO Catalyst Repository Systems Magistrate Judge Andrew J. Peck issued a landmark decision in Da Silva Moore v. Publicis and MSL Group, filed on Feb. 24, 2012. This decision made headlines

More information

DSi Pilot Program: Comparing Catalyst Insight Predict with Linear Review

DSi Pilot Program: Comparing Catalyst Insight Predict with Linear Review case study DSi Pilot Program: Comparing Catalyst Insight Predict with Linear Review www.dsicovery.com 877-797-4771 414 Union St., Suite 1210 Nashville, TN 37219 (615) 255-5343 Catalyst Insight Predict

More information

Predictive Coding Defensibility and the Transparent Predictive Coding Workflow

Predictive Coding Defensibility and the Transparent Predictive Coding Workflow Predictive Coding Defensibility and the Transparent Predictive Coding Workflow Who should read this paper Predictive coding is one of the most promising technologies to reduce the high cost of review by

More information

Predictive Coding Defensibility and the Transparent Predictive Coding Workflow

Predictive Coding Defensibility and the Transparent Predictive Coding Workflow WHITE PAPER: PREDICTIVE CODING DEFENSIBILITY........................................ Predictive Coding Defensibility and the Transparent Predictive Coding Workflow Who should read this paper Predictive

More information

Three Methods for ediscovery Document Prioritization:

Three Methods for ediscovery Document Prioritization: Three Methods for ediscovery Document Prioritization: Comparing and Contrasting Keyword Search with Concept Based and Support Vector Based "Technology Assisted Review-Predictive Coding" Platforms Tom Groom,

More information

A Practitioner s Guide to Statistical Sampling in E-Discovery. October 16, 2012

A Practitioner s Guide to Statistical Sampling in E-Discovery. October 16, 2012 A Practitioner s Guide to Statistical Sampling in E-Discovery October 16, 2012 1 Meet the Panelists Maura R. Grossman, Counsel at Wachtell, Lipton, Rosen & Katz Gordon V. Cormack, Professor at the David

More information

Measuring Recall in E-Discovery, Part Two: No Easy Answers

Measuring Recall in E-Discovery, Part Two: No Easy Answers Measuring Recall in E-Discovery, Part Two: No Easy Answers John Tredennick In Part One of this article, I introduced readers to statistical problems inherent in proving the level of recall reached in a

More information

E-Discovery Tip Sheet

E-Discovery Tip Sheet E-Discovery Tip Sheet LegalTech 2015 Some Panels and Briefings Last month I took you on a select tour of the vendor exhibits and products from LegalTech 2015. This month I want to provide a small brief

More information

Mastering Predictive Coding: The Ultimate Guide

Mastering Predictive Coding: The Ultimate Guide Mastering Predictive Coding: The Ultimate Guide Key considerations and best practices to help you increase ediscovery efficiencies and save money with predictive coding 4.5 Validating the Results and Producing

More information

Background. The team. National Institute for Standards and Technology (NIST)

Background. The team. National Institute for Standards and Technology (NIST) This paper presents the results of the collaborative entry of Backstop LLP and Cleary Gottlieb Steen & Hamilton LLP in the Legal Track of the 2009 Text Retrieval Conference (TREC) sponsored by the National

More information

Take an Enterprise Approach to E-Discovery. Streamline Discovery and Control Review Cost Using a Central, Secure E-Discovery Cloud Platform

Take an Enterprise Approach to E-Discovery. Streamline Discovery and Control Review Cost Using a Central, Secure E-Discovery Cloud Platform Take an Enterprise Approach to E-Discovery Streamline Discovery and Control Review Cost Using a Central, Secure E-Discovery Cloud Platform A Smarter Approach Catalyst s e-discovery cloud platform provides

More information

Introduction to Predictive Coding

Introduction to Predictive Coding Introduction to Predictive Coding Herbert L. Roitblat, Ph.D. CTO, Chief Scientist, OrcaTec Predictive coding uses computers and machine learning to reduce the number of documents in large document sets

More information

Challenges in Legal Electronic Discovery CISML 2011

Challenges in Legal Electronic Discovery CISML 2011 Challenges in Legal Electronic Discovery CISML 2011 this presentation is available: http://tinyurl.com/cat-cisml2011 Dr Jeremy Pickens Sr Applied Research Scientist Likes: Information Retrieval, collaborative

More information

PICTERA. What Is Intell1gent One? Created by the clients, for the clients SOLUTIONS

PICTERA. What Is Intell1gent One? Created by the clients, for the clients SOLUTIONS PICTERA SOLUTIONS An What Is Intell1gent One? Created by the clients, for the clients This white paper discusses: Understanding How Intell1gent One Saves Time and Money Using Intell1gent One to Save Money

More information

Fill Your Sales Pipeline With Business That You Don t Have to Work For

Fill Your Sales Pipeline With Business That You Don t Have to Work For Sparta Success Systems Presents: Turn Your Website Into A Lead Generation Machine! Fill Your Sales Pipeline With Business That You Don t Have to Work For 508-243-6257 www.spartasuccess.com http://activerain.com/blogs/spartasuccess

More information

The Benefits of. in E-Discovery. How Smart Sampling Can Help Attorneys Reduce Document Review Costs. A white paper from

The Benefits of. in E-Discovery. How Smart Sampling Can Help Attorneys Reduce Document Review Costs. A white paper from The Benefits of Sampling in E-Discovery How Smart Sampling Can Help Attorneys Reduce Document Review Costs A white paper from 615.255.5343 dsi.co 414 Union Street, Suite 1210 Nashville, TN 37219-1771 Table

More information

Christina Wojcik, VP Legal Services, Seal Software Steven Toole, VP Marketing, Content Analyst Company Jason Voss, Senior Product Manager, TCDi

Christina Wojcik, VP Legal Services, Seal Software Steven Toole, VP Marketing, Content Analyst Company Jason Voss, Senior Product Manager, TCDi FEBRUARY 3 5, 2015 / THE HILTON NEW YORK ML1: Machine Learning Powered Rapid Insight into Big Content: Discovery from Contracts to Patents to Litigation Panelists Christina Wojcik, VP Legal Services, Seal

More information

Understand how PPC can help you achieve your marketing objectives at every stage of the sales funnel.

Understand how PPC can help you achieve your marketing objectives at every stage of the sales funnel. 1 Understand how PPC can help you achieve your marketing objectives at every stage of the sales funnel. 2 WHO IS THIS GUIDE FOR? This guide is written for marketers who want to get more from their digital

More information

Making The Most Of Document Analytics

Making The Most Of Document Analytics Portfolio Media. Inc. 860 Broadway, 6th Floor New York, NY 10003 www.law360.com Phone: +1 646 783 7100 Fax: +1 646 783 7161 customerservice@law360.com Making The Most Of Document Analytics Law360, New

More information

How To Choose A Search Engine Marketing (SEM) Agency

How To Choose A Search Engine Marketing (SEM) Agency How To Choose A Search Engine Marketing (SEM) Agency Introduction During the last four years, in both good and bad economies, Search Engine Marketing (SEM) has continued to grow. According to MarketingSherpa

More information

7 Secrets To Websites That Sell. By Alex Nelson

7 Secrets To Websites That Sell. By Alex Nelson 7 Secrets To Websites That Sell By Alex Nelson Website Secret #1 Create a Direct Response Website Did you know there are two different types of websites? It s true. There are branding websites and there

More information

Beyond the Polyester Veil : A Personal Injury Negotiations Case Study

Beyond the Polyester Veil : A Personal Injury Negotiations Case Study Beyond the Polyester Veil : A Personal Injury Negotiations Case Study I m going to tell you something else almost as useful.i m going to tell you the big secret of handling your personal injury case. For

More information

Industry Leading Solutions: Innovative Technology. Quality Results.

Industry Leading Solutions: Innovative Technology. Quality Results. Industry Leading Solutions: Innovative Technology. Quality Results. April 10, 2013 emagsolutions.com Agenda Speaker Introduction A Quick Word about emag Importance of Technology Assisted Review (TAR) Key

More information

An Exploratory Analysis of Control Sets for Measuring E-Discovery Progress

An Exploratory Analysis of Control Sets for Measuring E-Discovery Progress An Exploratory Analysis of Control Sets for Measuring E-Discovery Progress Jeremy Pickens Catalyst Repository Systems 8 Blake Street, 7th Floor Denver, CO 8 jpickens@catalystsecure.com ABSTRACT Control

More information

White Paper Technology Assisted Review. Allison Stanfield and Jeff Jarrett 25 February 2015. 1300 136 993 www.elaw.com.au

White Paper Technology Assisted Review. Allison Stanfield and Jeff Jarrett 25 February 2015. 1300 136 993 www.elaw.com.au White Paper Technology Assisted Review Allison Stanfield and Jeff Jarrett 25 February 2015 1300 136 993 www.elaw.com.au Table of Contents 1. INTRODUCTION 3 2. KEYWORD SEARCHING 3 3. KEYWORD SEARCHES: THE

More information

Designing and Implementing Your Communication s Dashboard: Lessons Learned

Designing and Implementing Your Communication s Dashboard: Lessons Learned Designing and Implementing Your Communication s Dashboard: Lessons Learned By Katie Delahaye Paine President, Paine & Partners Contact Information: Katie Delahaye Paine CEO KDPaine & Partners Durham, NH

More information

Pros And Cons Of Computer-Assisted Review

Pros And Cons Of Computer-Assisted Review Portfolio Media. Inc. 860 Broadway, 6th Floor New York, NY 10003 www.law360.com Phone: +1 646 783 7100 Fax: +1 646 783 7161 customerservice@law360.com Pros And Cons Of Computer-Assisted Review Law360,

More information

How to Select a Virtualization Management Tool

How to Select a Virtualization Management Tool www.virtualizationsoftware.com How to Select a Virtualization Management Tool By David Davis, vexpert, VCP, VCAP, CCIE Introduction While VMware provides an excellent management tool for your virtual infrastructure

More information

Examining the Evolving Cyber Insurance Marketplace

Examining the Evolving Cyber Insurance Marketplace Prepared Testimony and Statement for the Record of Ola Sage Founder and CEO e-management Hearing on Examining the Evolving Cyber Insurance Marketplace Before the Senate Committee on Commerce, Science,

More information

Seven Things You Must Know Before Hiring a Real Estate Agent

Seven Things You Must Know Before Hiring a Real Estate Agent Seven Things You Must Know Before Hiring a Real Estate Agent Seven Things To Know Before Hiring a Real Estate Agent Copyright All Rights Reserved 1 Introduction Selling a home can be one of the most stressful

More information

SEO Facts. Why do I need Search Engine Optimization (SEO)?... 2. How do you maintain quality and excellence in the delivery of your product?...

SEO Facts. Why do I need Search Engine Optimization (SEO)?... 2. How do you maintain quality and excellence in the delivery of your product?... Why do I need Search Engine Optimization (SEO)?... 2 Why SEO takes time and requires an on-going investment?... 2 Why is there such a price difference between SEO companies?... 4 Any lock-in Contracts!?...

More information

REDUCING COSTS WITH ADVANCED REVIEW STRATEGIES - PRIORITIZATION FOR 100% REVIEW. Bill Tolson Sr. Product Marketing Manager Recommind Inc.

REDUCING COSTS WITH ADVANCED REVIEW STRATEGIES - PRIORITIZATION FOR 100% REVIEW. Bill Tolson Sr. Product Marketing Manager Recommind Inc. REDUCING COSTS WITH ADVANCED REVIEW STRATEGIES - Bill Tolson Sr. Product Marketing Manager Recommind Inc. Introduction... 3 Traditional Linear Review... 3 Advanced Review Strategies: A Typical Predictive

More information

56 Key Profit Building Lessons I Learned from Jay Abraham s MasterMind Marketing Training.

56 Key Profit Building Lessons I Learned from Jay Abraham s MasterMind Marketing Training. 56 Key Profit Building Lessons I Learned from Jay Abraham s MasterMind Marketing Training. Jay Abraham is a man you should all know. If you do not - go to the library and start learning - just do it! He

More information

Framing Requirements for Predictive Analytic Projects with Decision Modeling

Framing Requirements for Predictive Analytic Projects with Decision Modeling Research Brief Framing Requirements for Predictive Analytic Projects with Decision Modeling August 2015 Written by: James Taylor Key Takeaways 1. Organizations are struggling to create a scalable, sustainable

More information

Business Mastery: Smart Strategies Every Business Owner Needs to Promote, Protect and Prosper

Business Mastery: Smart Strategies Every Business Owner Needs to Promote, Protect and Prosper Business Mastery: Smart Strategies Every Business Owner Needs to Promote, Protect and Prosper Susan Wilson Solovic Sponsored by: AT&T 2012 Susan Wilson Solovic, all rights reserved Business Mastery: Smart

More information

SEO OVERVIEW. We want you educated about SEO before we start! By Mary Cary NewMediaLegalMarketing / Video Blog Marketing / Video Studio Sausalito

SEO OVERVIEW. We want you educated about SEO before we start! By Mary Cary NewMediaLegalMarketing / Video Blog Marketing / Video Studio Sausalito SEO OVERVIEW We want you educated about SEO before we start! By Mary Cary NewMediaLegalMarketing / Video Blog Marketing / Video Studio Sausalito (415) 690 7112 Table of Contents Knowledge is Power What

More information

Cost-Effective and Defensible Technology Assisted Review

Cost-Effective and Defensible Technology Assisted Review WHITE PAPER: SYMANTEC TRANSPARENT PREDICTIVE CODING Symantec Transparent Predictive Coding Cost-Effective and Defensible Technology Assisted Review Who should read this paper Predictive coding is one of

More information

Making reviews more consistent and efficient.

Making reviews more consistent and efficient. Making reviews more consistent and efficient. PREDICTIVE CODING AND ADVANCED ANALYTICS Predictive coding although yet to take hold with the enthusiasm initially anticipated is still considered by many

More information

COMPANY INTRODUCTION. 201-33119 South Fraser Way Abbotsford, BC V2S 2B1. 888.262.6687 contact@1stonthelist.ca www.1stonthelist.ca

COMPANY INTRODUCTION. 201-33119 South Fraser Way Abbotsford, BC V2S 2B1. 888.262.6687 contact@1stonthelist.ca www.1stonthelist.ca COMPANY INTRODUCTION 201-33119 South Fraser Way Abbotsford, BC V2S 2B1 888.262.6687 contact@1stonthelist.ca www.1stonthelist.ca INTRODUCTION RE: Partnering with 1st on the List Dear Reader: I would like

More information

IBM Social Media Analytics

IBM Social Media Analytics IBM Social Media Analytics Analyze social media data to better understand your customers and markets Highlights Understand consumer sentiment and optimize marketing campaigns. Improve the customer experience

More information

The United States Law Week

The United States Law Week The United States Law Week Source: U.S. Law Week: News Archive > 2012 > 04/24/2012 > BNA Insights > Under Fire: A Closer Look at Technology- Assisted Document Review E-DISCOVERY Under Fire: A Closer Look

More information

Predictive Coding, TAR, CAR NOT Just for Litigation

Predictive Coding, TAR, CAR NOT Just for Litigation Predictive Coding, TAR, CAR NOT Just for Litigation February 26, 2015 Olivia Gerroll VP Professional Services, D4 Agenda Drivers The Evolution of Discovery Technology Definitions & Benefits How Predictive

More information

The Case for Technology Assisted Review and Statistical Sampling in Discovery

The Case for Technology Assisted Review and Statistical Sampling in Discovery The Case for Technology Assisted Review and Statistical Sampling in Discovery Position Paper for DESI VI Workshop, June 8, 2015, ICAIL Conference, San Diego, CA Christopher H Paskach The Claro Group, LLC

More information

EMAIL ARCHIVING. What it is, what it isn t, and how it can improve your business operations

EMAIL ARCHIVING. What it is, what it isn t, and how it can improve your business operations EMAIL ARCHIVING What it is, what it isn t, and how it can improve your business operations OVERVIEW: Why businesses are turning to email archiving As your business grows, communication between you and

More information

DESIGN THINKING IS THE COMPETITIVE ADVANTAGE THAT BUSINESS LEGAL SERVICES PROVIDERS NEED

DESIGN THINKING IS THE COMPETITIVE ADVANTAGE THAT BUSINESS LEGAL SERVICES PROVIDERS NEED DESIGN THINKING IS THE COMPETITIVE ADVANTAGE THAT BUSINESS LEGAL SERVICES PROVIDERS NEED The US legal market is roughly just over $300 billion per year. Over one million juried lawyers compete alongside

More information

Seven Things You Must Know Before Hiring a Real Estate Agent

Seven Things You Must Know Before Hiring a Real Estate Agent Seven Things You Must Know Before Hiring a Real Estate Agent 1 Introduction Selling a home can be one of the most stressful situations of your life. Whether you re upsizing, downsizing, moving across the

More information

TOP 10 MOST COMMON MISTAKES MADE IN HANDLING YOUR OWN INJURY CLAIM

TOP 10 MOST COMMON MISTAKES MADE IN HANDLING YOUR OWN INJURY CLAIM TOP 10 MOST COMMON MISTAKES MADE IN HANDLING YOUR OWN INJURY CLAIM More times than not, your personal injury claim will be a battle with an insurance company. A highly trained adjuster will be assigned

More information

From Web Analytics to Engagement Analytics

From Web Analytics to Engagement Analytics white paper From Web Analytics to Engagement Analytics Table of Contents Executive Summary 1 Engagement Analytics 2 Measuring Quality with Engagement Value Points 2 Engagement Analytics Examples 5 Overall

More information

Planet Data parlays software and services into e-discovery's new legal model

Planet Data parlays software and services into e-discovery's new legal model Planet Data parlays software and services into e-discovery's new legal model Analyst: David Horrigan 19 Jun, 2012 Not that long ago, the components of e-discovery came in separate silos. E-discovery service

More information

SMARTER. Jason R. Baron. Revolutionizing how the world handles information

SMARTER. Jason R. Baron. Revolutionizing how the world handles information COVER STORY ] THINKING SMARTER Jason R. Baron Revolutionizing how the world handles information It is common knowledge that we are living in what has been termed The Information Age. With the advent of

More information

Increasing Marketing ROI with Optimized Prediction

Increasing Marketing ROI with Optimized Prediction Increasing Marketing ROI with Optimized Prediction Yottamine s Unique and Powerful Solution Smart marketers are using predictive analytics to make the best offer to the best customer for the least cost.

More information

Predictive Coding: E-Discovery Game Changer?

Predictive Coding: E-Discovery Game Changer? PAGE 11 Predictive Coding: E-Discovery Game Changer? By Melissa Whittingham, Edward H. Rippey and Skye L. Perryman Predictive coding promises more efficient e- discovery reviews, with significant cost

More information

E-Discovery Getting a Handle on Predictive Coding

E-Discovery Getting a Handle on Predictive Coding E-Discovery Getting a Handle on Predictive Coding John J. Jablonski Goldberg Segalla LLP 665 Main St Ste 400 Buffalo, NY 14203-1425 (716) 566-5400 jjablonski@goldbergsegalla.com Drew Lewis Recommind 7028

More information

Overcoming HR Spreadsheet Hurdles

Overcoming HR Spreadsheet Hurdles Overcoming HR Spreadsheet Hurdles Table of Contents Introduction 3 Excuse 1: I m not ready to make a decision. 4 Excuse 2: But spreadsheets are free. 5 Excuse 3: I m afraid to make such a big change. 7

More information

7 Deadly Sins of the DIY Cloud

7 Deadly Sins of the DIY Cloud 7 Deadly Sins of the DIY Cloud Uncovering the Hidden Impact of Custom App Development in the Cloud The Do-It-Yourself Cloud Revolution Cloud computing has brought a revolution to application development.

More information

One Decision Document Review Accelerator. Orange Legal Technologies. OrangeLT.com Info@OrangeLT.com

One Decision Document Review Accelerator. Orange Legal Technologies. OrangeLT.com Info@OrangeLT.com One Decision Document Review Accelerator Orange Legal Technologies OrangeLT.com Info@OrangeLT.com By the Numbers: The Need for Technology in Attorney Review Seventy. Integrated near- duplicate detection

More information

The power of money management

The power of money management The power of money management One trader lost ($3000) during the course of a year trading one contract of system A. Another trader makes $25,000 trading the same system that year. One trader makes $24,000

More information

Comprehending the Challenges of Technology Assisted Document Review

Comprehending the Challenges of Technology Assisted Document Review Comprehending the Challenges of Technology Assisted Document Review Predictive Coding in Multi-Language E-Discovery 3 Lagoon Dr., Ste.180, Redwood UBIC North City, America, CA 94065 Inc. +1-650-654-7664

More information

community for use in e-discovery. It is an iterative process involving relevance feedback and

community for use in e-discovery. It is an iterative process involving relevance feedback and Survey of the Use of Predictive Coding in E-Discovery Julie King CSC 570 May 4, 2014 ABSTRACT Predictive coding is the latest and most advanced technology to be accepted by the legal community for use

More information

Predictive Coding in Multi-Language E-Discovery

Predictive Coding in Multi-Language E-Discovery Comprehending the Challenges of Technology Assisted Document Review Predictive Coding in Multi-Language E-Discovery UBIC North America, Inc. 3 Lagoon Dr., Ste. 180, Redwood City, CA 94065 877-321-8242

More information

Selecting the Right ediscovery Solution for Your Company

Selecting the Right ediscovery Solution for Your Company Selecting the Right ediscovery Solution for Your Company Speakers: George Socha, Co-founder, Apersee, EDRM Johannes Scholtes, Chief Strategy Officer, ZyLAB Moderated by Mary Mack, Enterprise Technology

More information

Result Categories of Electronic Discovery Searches

Result Categories of Electronic Discovery Searches May 2004 Volume 2, Number 2 Result Categories of Electronic Discovery Searches Table of Contents The Ethical Implications of Your Computer Result Categories of Electronic Discovery Searches Lawyers often

More information

HOW TO SET UP A MERCHANT ACCOUNT WITHOUT GETTING RIPPED-OFF

HOW TO SET UP A MERCHANT ACCOUNT WITHOUT GETTING RIPPED-OFF HOW TO SET UP A MERCHANT ACCOUNT WITHOUT GETTING RIPPED-OFF Discover Insider Information Sales Reps Do NOT Want You to Know: 3 Big Mistakes Business Owners Make When Setting Up a Merchant Account By Michael

More information

Money Math for Teens. Credit Score

Money Math for Teens. Credit Score Money Math for Teens This Money Math for Teens lesson is part of a series created by Generation Money, a multimedia financial literacy initiative of the FINRA Investor Education Foundation, Channel One

More information

The Basics of Automated Litigation Support

The Basics of Automated Litigation Support The Basics of Automated Litigation Support The attorney calls you in to let you know that he has just heard from the firms client. In response to the discovery request, your client has gathered over 100

More information

Read this guide and you ll discover:

Read this guide and you ll discover: BUSINESS ADVISOR REPORT Provided as an educational service by: Rick Reynolds, General Manager Read this guide and you ll discover: What remote, offsite, or managed backups are, and why EVERY business should

More information

Social Media in the 2009 Inc. 500: New Tools & New Trends

Social Media in the 2009 Inc. 500: New Tools & New Trends Social Media in the Inc. 500: New Tools & New Trends Conducted by: Nora Ganim Barnes, Ph.D. (nbarnes@umassd.edu) Eric Mattson (eric@ericmattson.com) The Center for Marketing Research at the University

More information

Louis Gudema: Founder and President of Revenue + Associates

Louis Gudema: Founder and President of Revenue + Associates The Interview Series - Presented by SmartFunnel Interviews of Sales + Marketing Industry Leaders Louis Gudema: Founder and President of Revenue + Associates PETER: Hello folks this is Peter Fillmore speaking.

More information

Azure Machine Learning, SQL Data Mining and R

Azure Machine Learning, SQL Data Mining and R Azure Machine Learning, SQL Data Mining and R Day-by-day Agenda Prerequisites No formal prerequisites. Basic knowledge of SQL Server Data Tools, Excel and any analytical experience helps. Best of all:

More information

It is clear the postal mail is still very relevant in today's marketing environment.

It is clear the postal mail is still very relevant in today's marketing environment. Email and Mobile Digital channels have many strengths, but they also have weaknesses. For example, many companies routinely send out emails as a part of their marketing campaigns. But people receive hundreds

More information

Integrating Routing Software With Your Business Process Workflow

Integrating Routing Software With Your Business Process Workflow Integrating Routing Software With Your Business Process Workflow Appian Software for Routing and Logistics from TMW Systems 2012 TMW Systems Inc. Cleveland Dallas Raleigh Indianapolis Oklahoma City Vancouver

More information

Software-assisted document review: An ROI your GC can appreciate. kpmg.com

Software-assisted document review: An ROI your GC can appreciate. kpmg.com Software-assisted document review: An ROI your GC can appreciate kpmg.com b Section or Brochure name Contents Introduction 4 Approach 6 Metrics to compare quality and effectiveness 7 Results 8 Matter 1

More information

Application of Simple Random Sampling 1 (SRS) in ediscovery

Application of Simple Random Sampling 1 (SRS) in ediscovery Manuscript submitted to the Organizing Committee of the Fourth DESI Workshop on Setting Standards for Electronically Stored Information in Discovery Proceedings on April 20, 2011. Updated May 18, 2011.

More information

Google Lead Generation For Attorneys - Leverage The Power Of Adwords To Grow Your Law Business FAST. The Foundation of Google AdWords

Google Lead Generation For Attorneys - Leverage The Power Of Adwords To Grow Your Law Business FAST. The Foundation of Google AdWords Google Lead Generation For Attorneys - Leverage The Power Of Adwords To Grow Your Law Business FAST You re about to discover the secrets of fast legal practice success with Google AdWords. Google AdWords

More information

The Evolution, Uses, and Case Studies of Technology Assisted Review

The Evolution, Uses, and Case Studies of Technology Assisted Review FEBRUARY 4 6, 2014 / THE HILTON NEW YORK The Evolution, Uses, and Case Studies of Technology Assisted Review One Size Does Not Fit All #LTNY Meet Our Panelists The Honorable Dave Waxse U.S. Magistrate

More information

The Truth About Predictive Coding: Getting Beyond The Hype

The Truth About Predictive Coding: Getting Beyond The Hype www.encase.com/ceic The Truth About Predictive Coding: Getting Beyond The Hype David R. Cohen Reed Smith LLP Records & E-Discovery Practice Group Leader David leads a group of more than 100 lawyers in

More information

CLOUD TECHNOLOGY FOR NONPROFITS. How the cloud carries your mission forward

CLOUD TECHNOLOGY FOR NONPROFITS. How the cloud carries your mission forward How the cloud carries your mission forward $ $ $ 2 Contents 01 02 What is the Cloud? 3 Introduction The Changing Landscape of Technology On Premise vs. the Cloud Why is the Cloud Right for Me? 7 Only Pay

More information

Applying Machine Learning to Stock Market Trading Bryce Taylor

Applying Machine Learning to Stock Market Trading Bryce Taylor Applying Machine Learning to Stock Market Trading Bryce Taylor Abstract: In an effort to emulate human investors who read publicly available materials in order to make decisions about their investments,

More information

THE SEO MANIFESTO One Method To Outrank Them All. Proven To Work With

THE SEO MANIFESTO One Method To Outrank Them All. Proven To Work With THE SEO MANIFESTO One Method To Outrank Them All Proven To Work With Are You At The Top Of Google When Your Clients Look For You? The fact is, your clients are on Google looking for your products and services

More information

Evolutionary Virtual Infrastructure Monitoring with Predictive Analytics. By David Davis, vexpert Co-Founder, ActualTech Media October 2014

Evolutionary Virtual Infrastructure Monitoring with Predictive Analytics. By David Davis, vexpert Co-Founder, ActualTech Media October 2014 Evolutionary Virtual Infrastructure Monitoring with Predictive Analytics By David Davis, vexpert Co-Founder, ActualTech Media October 2014 Introduction As more physical servers are moved into the virtual

More information

The Value of Information Security Certifications

The Value of Information Security Certifications The Value of Information Security Certifications Trends in information security certification as reported by the information security professionals themselves July 2011 InfoSecLeaders.com Mike Murray /

More information

MA 1125 Lecture 14 - Expected Values. Friday, February 28, 2014. Objectives: Introduce expected values.

MA 1125 Lecture 14 - Expected Values. Friday, February 28, 2014. Objectives: Introduce expected values. MA 5 Lecture 4 - Expected Values Friday, February 2, 24. Objectives: Introduce expected values.. Means, Variances, and Standard Deviations of Probability Distributions Two classes ago, we computed the

More information

SHUN Live Support. Are Today s Top OrganizationsTurning their Backs on Customers?

SHUN Live Support. Are Today s Top OrganizationsTurning their Backs on Customers? WHY LEADING COMPANIES SHUN Live Support Are Today s Top OrganizationsTurning their Backs on Customers? One need not look far to find examples of industry-leading companies that strategically obscure customer

More information

Copyright 2010 You have giveaway rights to this report. Feel free to share.

Copyright 2010 You have giveaway rights to this report. Feel free to share. Article Marketing Magic Copyright 2010 You have giveaway rights to this report. Feel free to share. Table of Contents What Is Article Marketing?...3 The History of Article Marketing...7 Article Marketing

More information

Online Reputation Management (ORM) Guide Engage with Customers Convert them to your Brand Ambassadors

Online Reputation Management (ORM) Guide Engage with Customers Convert them to your Brand Ambassadors Online Reputation Management (ORM) Guide Engage with Customers Convert them to your Brand Ambassadors Author: Pradeep Chopra (CEO, Digital Vidya) Serious about Digital Marketing? Join our Upcoming Training!

More information

E-Discovery in Mass Torts:

E-Discovery in Mass Torts: E-Discovery in Mass Torts: Predictive Coding Friend or Foe? Sherry A. Knutson Sidley Austin One S Dearborn St 32nd Fl Chicago, IL 60603 (312) 853-4710 sknutson@sidley.com Sherry A. Knutson is a partner

More information

This Method will show you exactly how you can profit from this specific online casino and beat them at their own game.

This Method will show you exactly how you can profit from this specific online casino and beat them at their own game. This Method will show you exactly how you can profit from this specific online casino and beat them at their own game. It s NOT complicated, and you DON T need a degree in mathematics or statistics to

More information

Argyle Conversations. by Argyle Executive Forum SM Streamlining the. ediscovery Process

Argyle Conversations. by Argyle Executive Forum SM Streamlining the. ediscovery Process Argyle Conversations by Argyle Executive Forum SM Streamlining the ediscovery Process Joe Garber, VP, information governance, HP Autonomy, on the cost- and time-saving benefits of information governance

More information

Law firms embrace marketing geared toward the female client

Law firms embrace marketing geared toward the female client Page 1 of 5 Law firms embrace marketing geared toward the female client Kelly Lucas July 18, 2012 For years, marketing research has indicated that when it comes to the purchase of products and services

More information

From Web Analytics to Engagement Analytics. Quality over Quantity

From Web Analytics to Engagement Analytics. Quality over Quantity From Web Analytics to Engagement Analytics Quality over Quantity Table of Contents Executive Summary...3 Engagement Analytics...4 Measuring Quality with Engagement Value Points...5 Engagement Analytics

More information

MrMajik s Money Management Strategy Copyright MrMajik.com 2003 All rights reserved.

MrMajik s Money Management Strategy Copyright MrMajik.com 2003 All rights reserved. You are about to learn the very best method there is to beat an even-money bet ever devised. This works on almost any game that pays you an equal amount of your wager every time you win. Casino games are

More information

The One Key Thing You Need to Be Successful In Prospecting and In Sales

The One Key Thing You Need to Be Successful In Prospecting and In Sales The One Key Thing You Need to Be Successful In Prospecting and In Sales The secret key to being successful in prospecting and in sales found in this report will work in ANY INDUSTRY for any product or

More information

Achieving the Gold Standard in Paid-Search Bid Optimization

Achieving the Gold Standard in Paid-Search Bid Optimization MORE MATH. BETTER MATH. BEST ROI. Achieving the Gold Standard in Paid-Search Bid Optimization Understanding the benefits of global keyword-level modeling white paper: The Gold Standard info@optimine.com

More information

Why Nuix doesn t believe in Magic

Why Nuix doesn t believe in Magic Why Nuix doesn t believe in Magic Nuix has participated in the Gartner Magic Quadrant for ediscovery Software for the past four years. Over that time, the ediscovery market has changed considerably. Unfortunately,

More information

The State of Internet Lead Generation

The State of Internet Lead Generation TM The State of Internet Lead Generation By Cheryl Hentz with Mitch Talenfeld, CEO of MDT Direct and MDT Tracking, Inc. Published by: workforce communications 2012 The Baxandall Company, Inc. 627 Bay Shore

More information