Why Nuix doesn t believe in Magic Nuix has participated in the Gartner Magic Quadrant for ediscovery Software for the past four years. Over that time, the ediscovery market has changed considerably. Unfortunately, Nuix does not believe Gartner s analysis of software providers has kept pace. Gartner says its Magic Quadrant evaluations are based on rigorous criteria, deep industry knowledge and thorough research. The famous two-axis diagram is designed to give the appearance of mathematical precision. But from our experience in the ediscovery Magic Quadrant process, it is much more pseudo than science. Specifically, Gartner analysts: Undervalue the impact that exploding data volumes have had on the cost and burden of ediscovery projects Fail to appreciate that rapidly processing data at scale is the crucial difference between successful and unsuccessful ediscovery Refuse to acknowledge that vendors they classify as Leaders have underperformed in the competitive marketplace. We think these oversights have caused the Gartner Magic Quadrant for ediscovery Software to break down. Compounding the error is lack of transparency in Gartner s research methodology for example, how do they evaluate vendors technical abilities and who are the ediscovery practitioners that contribute to Gartner s findings? As a result of Gartner s distorted market appraisal, enterprise customers have spent millions of dollars on software and services that didn t serve their needs. This has given the entire industry a bad name. All we ask for is a proper evaluation of ediscovery software with rigorous methodology and a realistic approach. But Gartner thinks it s good enough to throw some secret ediscovery ingredients into a cauldron, say hocus pocus and kaboom! a Magic Quadrant appears. An ongoing dialogue Nuix maintains ongoing dialogue with Gartner s ediscovery analysts. Earlier this year we spoke to Gartner s analysts about our concerns with their research methodology, stemming from the results of previous years reports. When Gartner showed us draft sections of this year s report, which were almost identical to the previous year s, we escalated our concerns through Gartner s Ombudsman. After numerous exchanges and discussions, the Ombudsman concluded that Gartner and Nuix had different views of the market. Specifically, the Ombudsman said: The Ombudsman maintains the basis of Nuix s escalation has been a fundamental difference of opinion with Gartner analysts and the way that they have chosen to evaluate vendors in the ediscovery Software market. While we can understand Nuix s position and its passion for its view, our investigation shows that the analysts followed proper methodologies and processes in this instance. The analysts approach to this market was vetted and supported by an extended Research team. We have created this document and distributed it to our employees and customers to give our side of the story. We believe it doesn t matter how accurately Gartner s analysts follow their methodology because the process itself is broken. We shared a draft of this document with Gartner and offered to include or link to their response. They declined a right of reply.
Gartner s magical thinking Gartner told us... We say... ediscovery in corporations doesn t involve large volumes of data. Many vendors offer end-to-end ediscovery. We have strict criteria and scoring methodologies to decide which vendors make it into the Magic Quadrant and are classified as Leaders. We evaluate vendors technical capabilities. We won t provide details of our research methodology or demographic information about the people we interview. We evaluate the ediscovery market using the EDRM model. Nuix is a left-hand side technology because its key strength is processing. We don t pay attention to the litigation service provider market, which is shrinking as corporations bring their ediscovery functions in-house. Our customers tell us that cases involving more than a handful of custodians can swell to 100 gigabytes or larger. Most vendors products can t process large volumes of data. Without robust and scalable processing, timely and effective ediscovery isn t possible, no matter what other functionality the software offers. This year Gartner has listed as Leaders vendors that have: Left the market entirely Shifted the majority of their effort from software development to services Laid off or lost almost all their staff and customers Stated publicly the reason for their substantial financial losses was the success of service providers Infuriated customers some to the point of litigation by claiming to provide functionality they simply couldn t deliver. To our knowledge, Gartner s analysts have no practical experience as ediscovery practitioners and do not perform any technical evaluation of the products. They tell us they assess technical abilities by interviewing ediscovery practitioners. However, they refuse to provide any details about who they speak to or what questions they ask. Without those details, it s impossible to know how accurate or realistic Gartner s research is. Why does Gartner keep this information secret? Gartner has invented its own framework that splits early data assessment, the left-hand side of the EDRM, from analysis, review and production, the right-hand side. This model doesn t reflect the EDRM or the way most organizations conduct ediscovery today. This doesn t even make sense given Gartner says processing is primarily a right-hand side activity. We don t understand why Gartner ignores Nuix s abilities across the EDRM while giving other vendors credit for capabilities that don t work or that customers only use in very limited ways. Where is the evidence? Our insights into the market come from working with almost all the major service providers and advisory firms worldwide. They tell us business is booming. They are buying more software from Nuix and many have grown their volume of business five or six times over the past few years. WHY NUIX DOESN T BELIEVE IN MAGIC page 2
Our argument in detail Gartner thinks ediscovery involves small volumes of data Gartner analysts believe that day-to-day ediscovery does not involve large volumes of data. Our experience with corporate and service provider customers shows that in the age of big data, there is no longer such a thing as a small case. The average enterprise email mailbox is multiple gigabytes for each custodian. ediscovery typically also involves data in file shares, SharePoint, archives, laptops, cloud email, cloud storage, smartphones and bring your own devices. As a result, any case that involves more than five or ten people can quickly swell to hundreds of gigabytes. Nuix technology is proven to work against very large volumes of data faster than any other technology. Gartner says corporate customers want an end-to-end ediscovery solution from a single vendor. It claims some vendors in the Leaders quadrant provide this end-to-end capability. But we know that in many cases, these vendors fail at the critical point: Processing more than 100 or 200 or 500 gigabytes at a time. Gartner can only claim these vendors provide end-to-end functionality if it ignores whether or not their technology works at scale. Gartner doesn t know its left hand from its right Gartner analysts claim their market evaluation is based on industry-standard EDRM model. In reality, they have invented their own model which splits the left-hand side of the EDRM from the righthand side. They say the left-hand side is about early case assessment, or what EDRM co-founder George Socha says should be more accurately called early data assessment. This typically comprises identification, collection and preservation, and sometimes processing. They believe these tasks are performed either internally or by litigation service providers. The right-hand side, Gartner analysts believe, is processing, analyzing, reviewing and producing documents. This is a function primarily performed by in-house lawyers or law firms, they say. The analysts say a vendor provides end-to-end ediscovery capability if it offers some functionality from the left-hand and right-hand sides. This division may have made sense four years ago when Gartner created the first Magic Quadrant but it is meaningless today. Many corporate and government customers seek to collect, process and analyze data internally to evaluate their risk exposure and whether or not they have a chance of winning. Some aim to conduct the entire discovery process in-house. Others engage service providers to do the heavy lifting collection and processing in some specialized circumstances, but more often analysis, review (human and technology-assisted) and production. Nuix has dozens of corporate customers who use our software internally and dozens of service provider customers who use it on behalf of clients. Some use Nuix on its own, some combine our software with other applications such as review platforms. Gartner analysts maintain that Nuix is a left-hand side technology because its key strength is processing, even though they also say processing is usually conducted on the right-hand side. This doesn t make sense to us and we can t convince the analysts to change their opinion. They really don t know their left hand from their right. WHY NUIX DOESN T BELIEVE IN MAGIC page 3
What does being a leader mean? Gartner s ediscovery Magic Quadrant reports include long blocks of text to explain how it decides which vendors to include and which vendors are Leaders. Gartner has very strict rules about who is allowed to quote from these explanations, so we can t include them here. But anyone who takes the time to read them will see they re a jumble of laundry lists, contradictions and opinions dressed up as facts. For example, Gartner analysts say Leader vendors must offer functionality from the left-hand and right-hand sides of the EDRM. This is one of the main reasons they give for excluding Nuix from the Leaders Quadrant. However, in their description of Nuix, they include a list of capabilities that go from the left-hand side to the right-hand side. By contrast, some of the vendors listed do not offer a credible functionality on both sides of the line, according to the Gartner analysts themselves, but still qualify as Leaders. Gartner analysts also say vendors in the Magic Quadrant should be focused on developing and selling software rather than services. Yet it prominently includes vendors that earn much more revenue from services than software. Some of them started out as software developers but are now almost entirely service providers. The Magic Quadrant even includes some service providers that use Nuix as their primary ediscovery engine. On the other hand, it leaves out some of the world s largest service providers because they don t make software. We don t mind if Gartner includes or excludes service providers, but it would be nice if they were consistent! Gartner s evaluation criteria also require that vendors continue to develop their ediscovery software and remain financially viable. What must be most embarrassing for Gartner is many of the vendors in the Leaders Quadrant have gone significantly and publicly backwards. They have dropped market share, laid off large numbers of staff and suffered plummeting revenues and significant financial losses. Some have abandoned ediscovery almost entirely. Gartner doesn t allow us to quote the Magic Quadrant report directly, but its descriptions for Leaders in this year s report say things like: Customers say this product is hard to install and upgrade Customers must be careful not to spend a lot on services with this product Customers are not satisfied with this vendor s ECA, visualization and processing capabilities Customers say this vendors products are expensive, complex and hard to use Customers identify problems with this product s performance, scalability and production functionality The departure of the company s CEO may cause disruption. We re not saying this to rejoice in our competitors misfortunes. But we wonder how bad things have to get before Gartner stops listing vendors as Leaders. Gartner can t (or won t) evaluate technical capabilities A fundamental question to ask when evaluating technology products is Do they work? Specifically, do they deliver the functionality that most customers need today? Gartner s Magic Quadrant includes many ediscovery software companies whose software cannot process the volumes or types of data that are now routinely involved in litigation. Without this capability, timely and effective ediscovery is not possible. The Magic Quadrant report never says explicitly that Gartner evaluates vendors capabilities. To our knowledge, Gartner analysts do not install, run through, benchmark or otherwise technically evaluate any software packages. The analysts have told us verbally that they ask the interviewees about technical performance during their research process. We ve asked them directly who they speak to, what questions they ask and what answers they receive. They won t tell us. WHY NUIX DOESN T BELIEVE IN MAGIC page 4
With Gartner, everything s a secret Any credible research report provides a thorough explanation of its methodology and demographic information about the people interviewed. What weighting does it give to different capabilities? What are its evaluation criteria and where does it draw the line? How does it validate vendors claims? What size organizations are the interviewees from? What roles do they perform? What is their involvement in day-to-day operations? Which countries are they from? Do they change from year to year? Gartner doesn t share this information with its customers or the industry. Without this information, it is impossible for customers to evaluate if the research was sound and applicable to their situation. For instance, if the interviewees were mainly lawyers from top law firms, the results would have little bearing on corporate IT departments. Gartner ignores litigation service providers and believes they are doomed Gartner analysts claim the service provider market is becoming commoditized and will shrink as more corporations bring their ediscovery functions in-house. They have maintained this position since the Magic Quadrant began but there is little evidence to support it. At Nuix, we deal with enterprises that insource their ediscovery work, some that outsource everything to service providers and many that insource and outsource depending on the circumstances. As far as we can tell, Gartner does not analyze the volume of data or the number of ediscovery cases undertaken by service providers. They tell us We don t cover that market. But service providers and advisory firms conduct the majority of corporate and government ediscovery work in the world today We concede there has been some consolidation of service providers, such as DTI s acquisition of Applied Discovery in January, but the rate of market consolidation is not extraordinary. We also agree that some enterprise customers are bringing their ediscovery functions in-house indeed we have helped some very large organizations do this. But there is also still movement in the opposite direction: Organizations abandoning in-house ediscovery and giving their business to service providers. For example, in its most recent 10-K filing, Guidance Software disclosed that the shift toward service providers had limited its success in the ediscovery market. The feedback we have from our many top- and mid-tier service provider customers is that business is booming. Their purchases of software from Nuix reflects this. For example, one mid-western US firm has grown into a leading international provider of ediscovery services since starting out in 2007. It has been included in the Inc. 500 list of the fastest growing private companies in the United States thanks to three-year revenue growth of 139%. This provider uses Nuix extensively for ediscovery processing, search, analysis and production. In the past three years, this company has increased its Nuix licensing capacity more than five-fold, reflecting the massive growth in its business and the volume of data it handles. Our advisory firm clients tell us they are taking on more staff and growing revenue in ediscovery and related fields such as digital forensics and information governance. We are selling them more software, not less. One firm has increased its ediscovery staffing around the world five-fold in just three years and we believe this is representative of all the Big Four. We invite our ediscovery service provider customers to share their growth stories with Gartner analysts. We also encourage the Gartner analysts to explain what evidence they have to support their conclusion that the litigation support industry is shrinking. Apac North America EMEA Australia: +61 2 9280 0699 USA: +1 877 470 6849 UK: +44 207 877 0300» Email: WHY NUIX sales@nuix.com DOESN T BELIEVE IN» Web: MAGIC nuix.com» Twitter: @nuix page 5 Copyright 2014 Nuix. All rights reserved.