BETTER THAN BIG DATA Why Smart Data is Superior for Most Business Decisions

Transcription

1 Why Smart Data is Superior for Most Business Decisions written by Tyler G Page The big data phenomenon and how you can get better ROI out of Smart Data.

2 Table of Contents Introduction 3 A Business Rule 3 Defining Smart Data 4 A Needle in a Haystack 4 Cost-Benefit Analysis 5 Big Data 5 Smart Data 6 An Example 7 Conclusion 8 How Can You Get Smart Data? 8 2

3 Introduction Nothing excites MBA classes more these days than spreadsheets brimming with numbers. That excitement, derived from Moneyball and stories of success in virtually every industry, has rushed nearly every business in America toward Big Data. There is no doubt that Big Data is BIG. It can be powerful and can drive real change. Businesses have jumped in with both feet by aggregating and mining every possible piece of data and putting it into complex spreadsheets for statistical analysis. But is Big Data always the right approach? In the midst of this rush to collect and analyze every piece of information on the planet, Big Data proponents often leave out critical questions: How quickly can I get data I can use? Is this data coming from reliable sources? Are the results even actionable? Will conducting this research yield a fast ROI? In the worlds of academia and government policy where the goal is to create and determine universal trends that can be applied everywhere in society more data is a good thing. But business is quite a bit different. A business doesn t have unlimited funds or time to make decisions, nor is a business trying to ascertain generic, universal trends that apply across industries. Rather, businesses are focused solely on improving themselves by positively impacting the customer experience... and the sooner, the better. That s a good thing, as every reader of Adam Smith will attest. But it means the calculus is different for a business when it comes to data. A Business Rule If two distinct paths of fact-finding result in the same decision, then the quicker path is better in the business world. That s not always true in academia or government. But results are king in business. In business, any data that doesn t help you make the right decision is a distraction not a bonus. So more is not better, and Big Data may not be the right approach. 3

4 Defining Smart Data Some businesses have rebranded big data solutions as Smart Data to reflect the intelligent solutions they provide. In their view, Smart Data is simply a buzzword that sounds different from Big Data... so they use it. That s not what we re talking about here. Here is what I mean by Big Data and Smart Data: Big Data: The lengthy process of aggregating as much existing, historical data as possible before scrutinizing it with complex and costly analytical tools (i.e., business intelligence) in order to ascertain statistically significant trends to guide business decisions. Smart Data: A method to get very specific data fast. Such data is so fast and specific that it makes decisions and next steps obvious. A Needle in a Haystack For a moment, imagine that a needle is the last item you need to solve your biggest problem. Does it matter whether that needle is placed gently on a pillow in a clean room or hidden inside a 12-foot tall haystack? Of course it does. You don t have all day to waste hunting for a needle in a haystack. The sooner you can make a business decision, the better off you will be. When a business uses a Big Data approach and gets every possible piece of information, it is spinning its own straw around the needle. If it is lucky, the business will only waste time and money finding the keys to improve its business. In a worst case scenario, that business will either lose the needle altogether or even pick up something other than a needle by mistake. The smart method says that you shouldn t spin the straw at all. Rather, you should start by focusing on what matters and you can control, gather data accordingly, and act quickly to solve your problem. 4

5 Cost-Benefit Analysis Which model should you use? It depends on a lot of things. Big Data Big data is more exhaustive, but it takes a long time, a lot of personnel, and a truckload of money. To do it well, you need to acquire gigantic storage systems (in-house or out in the cloud), powerful analytical tools, and extensive IT staff to manage them. Once you do that, you also need a really good in-house statistician who knows how to cut through meaningless data. Statistical significance tests are valuable but they can mislead in a number of ways. The most critical is that the statistician chooses the level of confidence in the results. In most cases, the 95% confidence level is standard. This means that 1 in 20 results will be off- base. So if you collect thousands of variables and run every statistical test possible, you are likely to find some false correlations that may not really exist. A good statistician knows this, but good statisticians don t come cheap. In business, you don t have the luxury of two years and millions of dollars to make decisions. You need to know now. The big data solutions you often hear about, like the two- year big data study described in this TED talk, take a long, long time to yield meaningful results. In some businesses, waiting that long for a solution is the difference between success and bankruptcy. In other cases, the data is so outdated by the time it is analyzed that the decisions it indicates are no longer useful. If you haven t already, go listen to that TED talk. While you listen, notice the criteria that they put into the software program. Does a really take a genius (or 2 years / millions of dollars) to figure out those criteria are the ones that matter? Don t you think a parole officer in the area could have come up with a similar result? Now go ahead and check out that video. It s ok, I ll wait. See what I mean? In business, you don t have the luxury of two years and millions of dollars to make decisions. You need to know now. For business, the ideal method is to find results in a day or two, make a decision, and evaluate its results. Then refine according to the results and repeat the process again. You could repeat that process a dozen or more times in the time it took to conduct this one big data study. All the while, you d be getting better results that you can constantly refine. 5

6 Now, let s talk about money. Consider that among the most cost-efficient Big Data storage frameworks is Hadoop, for which a one petabyte cluster requiring anywhere between nodes can cost about $1 million. Of course, that s the low end. A company like Yahoo! manages at least 200 petabytes across some 50,000 nodes. You can do the math. The price goes up exponentially. How many organizations can justify that sort of investment? Not many. And even fewer have a clear idea of how to deploy and, more importantly, effectively manage (all of which adds to cost) such a complex and ambiguous project. Darin Bartik, a Dell executive, recently wrote of this problem in an InformationWeek article. I often hear from customers who invest in Hadoop, load their data, and then say now what? Bartik wrote. They've invested in a new technology -- because the hype led them to believe that's what they're supposed to do -- without knowing if it can actually help them. In other words, it is certain that a Big Data approach will take a lot of time, a lot of personnel, and a truckload of money, but it may not yield an actionable result. Smart Data It is certain that a Big Data approach will take a lot of time, a lot of personnel, and a truckload of money, but it may not yield an actionable result. Smart Data uses many of the methods and tools of Big Data, but it emphasizes getting actionable results. That means getting results quickly and focusing research on topic areas where decisions can make a difference. This highlights another major difference between Smart Data and Big Data. Big Data solutions are macro in nature, meaning they can yield universal trends, but they are very weak on the micro the ability to see what each individual data point means and how it can be corrected. In contrast, Smart Data solutions are set up perfectly to deal with individual concerns. If one customer service representative is upsetting clients, Smart Data allows you to mend relationships before clients are lost. Big Data can only tell you what happened after the fact. Smart Data isn t as exhaustive, and therefore, isn t as suited for government and non- profit type research. Nor is it ideal for academic studies that seek to determine universal, generalizable 6

7 trends. Instead, Smart Data is focused on getting you actionable results right now. Because it focuses in on what s important, it is exponentially faster and cheaper, so as to give a very rapid ROI. Simply put, Smart Data is built for business An Example So imagine you re running a SaaS firm and you want to improve your sales. A Big Data approach would be to pull every piece of information possible: sales numbers by location, political orientation of the city/state government in target markets, customer satisfaction surveys, gender of customer service employees compared with retention rates, weather patterns during the time period specified, sales at competing software firms, race and age demographics in the target markets, and on, and on, and on. A Smart Data approach would start with what you know about the industry and focus in on things you can change. For instance, weather may very well impact the overall sales numbers, but there isn t anything you can do about the weather. A business could conduct a Smart Data study, change its business operations for a quarter, evaluate growth, and repeat the process before Big Data had even given a preliminary answer. Instead, a Smart Data approach would recognize most enterprise software success is based upon renewed contracts. That means customer satisfaction is probably pre-eminent. A Smart Data approach would ascertain customer satisfaction on a few key subjects: product quality, customer service experience, and price. It would then compare these to overall sales, find a correlation, and focus results toward solutions. The Big Data study would take months or years to get a result, while the Smart Data approach could take anywhere from 2-5 days. In other words, a business could conduct a Smart Data study, change its business operations for a quarter according to the results, evaluate growth, and 7

8 repeat the process 2-3 times before the Big Data approach had even given a preliminary answer. Oh, and based upon the low end cost estimates mentioned earlier (not including the cost of collecting all that extra data), a premier Smart Data solution will cost 37 times less than a cheap Big Data solution. Conclusion In the hyper-competitive world of business, Smart Data offers better solutions. While Big Data certainly has a place in academia, government, and gigantic enterprises, Smart Data is the best solution for most business decisions. Smart Data doesn t replace Big Data Smart Data merely puts Big Data in its place. The key is to find quality data quickly so that business decisions can easily follow. How Can You Get Smart Data? Peer60 has assisted companies all over the world create their Smart Data programs. By deploying our mobile solutions and supportive services, global brands have gathered unprecedented levels of actionable business intelligence in time to make big business decisions. If you want a little more information about Smart Data you can the big cheese himself at jbikman@peer60.com. Tyler G. Page is the Director of Marketing and Public Relations at peer60. He also teaches communication research methods at Brigham Young University. 8