Debugging the Hype about Big Data and Business Service Metrics

Transcription

1 Once you have defined Business Services, successful cost and performance management hinges on tracking the right metrics. While simple unit metrics are a start, the most effective way to gain insights about your Services is to combine and analyze data from multiple systems, building multi-metrics. The complexity and size of these data sources can sometimes drive companies into looking for Big Data tools and solutions for this analysis. The new era of Big Data is about how things relate in unstructured ways, both for finding new relationships and in proving them. This could be the set of metrics that provide a critical market differentiator, or it could be your next money pit. Before committing, you should understand the Hype and the Reality around Big Data Although the buzz about Big Data is relatively new, the process of crunching through large amounts of data for complex insights is not. Companies involved with Bio-Informatics, Energy, and the Financial/Trade sectors have been doing Big Data for decades. They would invest large amounts of money in very large hardware and very smart people hoping for results that would give them a competitive edge. It made good business sense for them because the potential returns on the investment were very large. Think of these businesses as the first gold miners on the scene. The cost was high, but they were positioned to find large nuggets of gold. Why then has this become a generalized hype? Why is every analyst and trade journal in existence insisting that if you are not doing Big Data, then you are missing the boat? Although social media has added a new and vital source of data, the truth is that many companies have had big or complex data that they could have been analyzing for quite some time, but the costs were too high for them to make the leap. Over the last decade, companies like Netflix and Amazon have proven that there is significant value to be found in the long tail market. This is the gold mining equivalent of going after pounds of flakes versus a single 6 oz. nugget. Just as it took improvements to mining technology to enable companies to find value in gold flakes versus nuggets, IT technology improvements make mining for critical mass of data insight flakes possible as well. 1

2 There have been significant developments in Infrastructure, hardware and software over the last decade which have lowered the entry point for companies wanting to find the flakes of IT gold. We have included a table that lists out some of the specific changes with impact to cost, but the take away is simple: Moore s law resulted in an overall reduction in the cost of computational power. The changes in hardware design and architecture allowed for cheaper overall computational cost and data storage. Increased bandwidth and lower network charges allowed for the birth of Infrastructure as a Service (IaaS). Hand in hand with this were advances in software that allowed novel ways of processing and analyzing data. Changes that Have Lowered the Barriers to Big Data: Infrastructure Changes Hardware Changes Software Changes Increase in computational density IAAS vendors Simplified Cluster Computing Reduction in cost of computational power- Moore s law Network technology increased speed/ability to connect Storage density decreased, prices dropped both online and off Introduction of multi core processing Display technology can display higher resolution images off the shelf to display information in greater detail Introduction of NOSQL databases and New class of algorithmscausality and correlation. Graph algorithms with 6 degrees of freedom. Hadoop and map reduce computing Parallel databases Growth and improved developer support for parallel computing ( s Improvements to visualization techniques and software Let s be honest- this is still not a free process. Storage and compute costs might be reduced from 10 years ago, but due to geometrically increased storage amounts, they still take an investment that is not trivial. If you are going to mine for data gold, you have to have a plan to get value out of your efforts. 2

3 The Mining analogy is actually useful when considering the steps needed to make decisions about where Big Data analytics will bring valid ROI in your Business Service Metrics and Management processes. Sometimes, looking at new data opens up new questions that would not have been possible before. This is more of an R&D approach- and should be funded by invest, or R&D dollars. From an operational perspective, Big Data can play a role in Data Driven Decision Making allowing you to measure things that were not possible before for both internal and external services. However, there is a critical risk in over measuring and investing in bad metrics. It is also important to choose the right metrics- making sure you have the critical metrics included, but not metrics which will cause the business to worry about things that are not important, rather than focusing on needed change. Here is the general process when deciding to mine for gold: 1. Complete a Geographic Survey. This survey gives you an indication of the most reasonable places gold might be found. 2. Take Core Samples to get detailed data on the actual soil composition. 3. Choose/design the processing equipment and layout. Will you be panning, cracking and extracting hard rock gold, or building wash stations for gold flakes? When deciding on how/if you should leverage Big Data techniques to manage your services better, you need to follow the same basic steps Data Landscape Surveys, Pilots and Studies, and Operationalized Process Design. 3

4 1. Geographic Surveys: Now is the time to take an inventory of your Data Sources across the organization. How many do you have? Do they have similar structures or are large amounts of data cleansing needed? Where does the data landscape best match the driving business needs? 2. Core Samples: Do some Pilot studies. Look to leverage statistical techniques in your analytics, rather than doing brute force computations in order to save cost and time. 3. Design the processing: This is where algorithm, infrastructure, and hardware decisions are important. Based off of the pilot(s) you completed, you should be able to gauge the size of ROI expected. You should also keep the following items in mind as you design your processes. i. Are you becoming a gold miner for life, or is this a one-time exploration? (Operational Analysis versus completing a Study) ii. Many tools are available as a SaaS solution. You can Rent/Lease as you need. You are no longer required to invest large capital to move forward. iii. If you do studies, how will you operationalize the results without requiring ongoing daily big data analysis? iv. Remember that if you leverage a data scientist who is a data expert but does not understand the business, you can easily end up with false positive/negatives. Simply mining all the data you have, just because it is possible, can throw the business into a tailspin. v. Make sure that the process you are designing matches your gold form- don t build a rock cracker when you are trying to wash for flakes. A Reality Check The reality is that there is No silver bullet. The Truth- Big Data is NOT always the best answer. It is worth the investment in getting expert advice in decision making, especially as you design and plan, to avoid expense of wrong decision. Although Big Data can, in some case, provide Big results, it is Not a low budget activity. 4

5 The Data experts at Thavron Solutions work in teams with our Business Analysts to understand what your top 5 business questions/problems are around managing your services and to sidestep the potential false positives and negatives that can arise when your analysis in not grounded in business process. Thavron can help you understand the potential ROI if you leverage Big/Complex Data in your Business Service Management Processes. We can give you a full report on the metrics where this approach makes good business sense, and where you can leverage less expensive methods to get the data you need in order to manage your services intelligently. For more information, please contact Thavron Solutions LLC: [email protected] / / thavron.com 5