!!!!! BIG DATA IN A DAY!

Transcription

2 EXECUTIVE SUMMARY Big Data is no longer just about analyzing data it s about the time value of data. Just like a dollar today is worth more than a dollar tomorrow, an actionable business insight today is worth more than a theoretical insight in the future. In the past, getting value from data was a long and arduous process. That is no longer the case. With the right software, it s now possible to get up and running in as little as a day to start gaining actionable insights from your data. Once deployed and in production, it s possible to answer business questions as quickly as they arise. In other words, getting value from data is no longer a multi-month or multi-year process. Audience Big Data in a Day is intended for C-level executives, general managers and business line managers who are learning about the new world of Big Data. We discuss real-world examples from a large online travel site and a major North American telecom company. We illustrate how social media data from sites like Twitter and Instagram is being put to immediate use. We show how organizations like yours are moving away from the traditional model of getting value from data in years to getting that value in a day or less. The New Era of Big Data The new era of Big Data is about: Rapid time to insight. Big Data today is about fast iteration. It s about how quickly you can get up and running and answer questions as business leaders bring them up. The new era of Big Data is about getting answers faster than ever Exploring data. In the new era of Big Data, you don t need to try to understand your data before you have it. When it comes to getting value from data, questions beget insights, which beget more questions. In the past, answering new questions meant a complicated and lengthy re-implementation cycle. In Big Data in a Day, we explore the ways in which some of today s most successful companies are getting rapid value from Big Data day after day. Rich context. By integrating structured and unstructured data, today s leading enterprises are able to understand extremely diverse data sets and how all their data connects. Historically, data has lived in silos. Many companies have accumulated different data repositories. These are a result of mergers and acquisitions, a wide variety of datarelated development projects and a diverse set of departmental needs. In Big Data in a Day, we ll look at how organizations are bridging the structured and unstructured data worlds to gain new value from the rich context that results. PAGE 2

3 WHAT DOES BIG DATA MEAN TO YOUR BUSINESS? Data doesn t live in a vacuum. Virtually no one wakes up one day and suddenly realizes their organization has a Big Data problem or opportunity. Rather, organizations build systems to meet customer and operational requirements. The volume of data in those systems grows and the diversity of that data increases. Ultimately, such systems become unmanageable. That s exactly what happened at a large online travel site. At the travel site, one department started out with a few SQL servers, but over a period of years expanded that to more than 50 servers used for reporting. That was simply for generating on-going, pre-defined reports, not even for performing new, on-the-fly queries to answer pressing business questions. It also did not include the data associated with clickstream analysis, the digital breadcrumbs that follow the path users take through e-commerce and other websites. Such data is critical to understanding on-site consumer behavior and delivering a personalized experience. The right Big Data solution can deliver $10 million or more in ROI annually. By adopting Splunk software, the company has realized an estimated annual ROI of more than $14 million. The Data Deluge Consumers want responsive websites, while business leaders want immediate answers to questions that depend on complex analysis of lots of data. Frequently, much of that data is stored in data silos, disparate databases and data warehouses that contain all the necessary data but are difficult to pull together into a comprehensive analysis. This is the data deluge. It often seems that data is everywhere, but actionable data is nowhere to be found. Data Inefficiency Analyzed properly, the data within the deluge can bring insights that result in significantly more efficient underlying systems and a better experience for customers. When it comes to travel, for example, consumers perform a lot of searches before they book. The vast majority of hotel queries some 70 percent do not convert to bookings. Systems grow and expand organically. This makes it increasingly hard to meet the reporting and analytics needs of business unit managers and the responsiveness PAGE 3

4 requirements of customers, while performing the business of the company in this case executing travel e-commerce transactions. The good news is there is a better way. Just Big Data accumulates over years but it need not take years to act on it. because Big Data accumulates over years doesn t mean it has to take years to act on it. The online travel site used Splunk Enterprise to detect repeated searches and an open source distributed database management system to cache them. This resulted in both more efficient searches for consumers and more efficient use of valuable system resources. With the right tools, it s possible to take operational data, such as web analytics data collected for operational purposes like measuring server performance, throughput and responsiveness, and turn that into actionable data for other parts of the organization, such as marketing or sales operations. Unlock the Insights in Your Data Consider all the potential insights locked in data like web log files. With better access to data you can benefit from: Improved customer analytics. For example, you can identify when the same customer returns to your site from a different device. You can get an early read on purchase patterns, product usage and customer churn. You can use real-time analytics to correlate transaction data with web and mobile data to gain insights into business performance. Effective product analytics. You can understand feature adoption, usage and the effectiveness of changes. Product managers can drill down, segment and correlate various datasets and derive actionable insights for their website or mobile apps. Improved user experience. You can get meaningful insights into which features contribute to effective user engagement and conversion. Think how valuable it would be for a sales executive to know which pages on your website a potential or existing customer had already visited when speaking with that contact. Many companies are already logging such data but often it is inaccessible to marketing and sales. It s locked away as unstructured operational data, unconnected with the structured customer data found in CRM systems. Connecting operational and CRM data may appear to be a daunting task. On the one hand, there are unstructured data sources such as server log files. On the other hand PAGE 4

5 there is structured data like customer records and sales contacts in a CRM system. The data that identifies a customer in one system is often different from the data that identifies them in the other. In reality, it s easy to unlock the value of your data. Simply connect the operational data with CRM customer records and put it into dashboards that marketing and sales managers can leverage. By doing so you can turn data into insights overnight. Today s advanced Big Data analytics platforms can identify the unique signatures of similar kinds of data across diverse systems even if that data looks somewhat different on each system. Splunk software, for example, can connect with both operational and marketing systems to match the same customers and their associated data, unlocking operational data that would otherwise remain in its own silo. The right Big Data tools can create opportunities to understand and serve new and existing customers in more personal and responsive ways. The Hidden Costs of Big Data When it comes to Big Data, there is a common misperception that storage is cheap. The reality is that data stored but not analyzed is incredibly expensive. While the price of storage continues to fall, the cost of The biggest Big Data cost of all is the opportunity cost of inaccessible business insights locked away in data silos. administrators and data analysts to manage complex structured data repositories continues to rise. The biggest cost of all is the opportunity cost of data stored in a silo. Such data could generate actionable business insights if only you could quickly find it without timeconsuming, expensive and manual processes. Traditional database servers also consume power and space in datacenters and require experienced personnel to manage them. As a result, many resources end up being applied to maintaining existing infrastructure rather than working on much needed features and future projects. These hidden costs are often overlooked when evaluating the move from legacy to modern Big Data systems. Software like Splunk Enterprise not only can store many terabytes of data a day on commodity hardware, it can also provide extensible, customizable application-level dashboards that give managers ready access to actionable business insights. Such flexibility frees up talented personnel to work on new projects. PAGE 5

6 With the right insights, you can deliver better, more personalized user experiences that ultimately result in greater margins and increased revenue. By tying user behavior on an e-commerce site to a company s CRM system, for example, it becomes far easier to deliver the most optimal offers and content to any given visitor to a website. With the right software, you can gain significant competitive advantage from your data, rather than just maintaining the systems you ve accumulated. THE MODERN DATA EXPLORATION MODEL Historically, obtaining insights from large quantities of data has meant defining business questions up front, analyzing that data in a data warehouse and generating reports with the results. This is the traditional data reporting model. For many years, it has been the only approach available to organizations when it comes to understanding their data. However, this approach is also fraught with challenges in many cases, the reports are static and have to be re-generated to answer new questions. Traditional Data Reporting Model Static Report-based Slow Modern Data Exploration Model Iterative Interactive Real-time Figure 1. Traditional and Modern Data Presentation Models. The process is also an incredibly slow one. Each of these stages of the data analysis pipeline introduces latency into the process of turning data into actionable business insights. It also means that once the data is analyzed and a report is available, if business users have new questions, they have to start the process over: define new questions, re-analyze the data and interpret new reports. In contrast to the traditional data reporting model, the modern data exploration model provides users with an approach that is iterative, interactive and real-time in nature. When it comes to understanding your data, answers often generate more questions. With the modern data exploration model, new questions can be answered as rapidly as the originals. PAGE 6

7 The Traditional Data Reporting Cycle Data analysts spend a lot of time simply trying to access and format data. Whether you re dealing with system log files, billing information or customer records, data comes in many different forms and formats, and resides in many different locations across the organization. In the traditional data reporting model, this creates a series of challenges and associated requirements: Capturing the data Setting up the software and systems necessary to analyze the data Moving all the data from different systems into one place so it can be analyzed Getting the data into compatible formats suitable for analytics software Cleaning the data so insights are accurate Finally, running the analysis necessary to get the insights This cycle is shown in Figure 2, below. Figure 2. The Traditional Data Reporting Cycle. PAGE 7

8 Benefits of the Modern Data Exploration Model In contrast to the traditional data reporting model, the modern data exploration model makes things a lot easier. Instead of going through the painstaking process of preparing data and forming all their business questions up front, business leaders can use a much more iterative and interactive process to gain insights, as shown in Figure 3 below. Figure 3. The Modern Data Exploration Cycle. Once they have initial answers, business leaders can continue to ask more questions and get those questions answered right away. Real-World Use Case: A Major North American Telecom Company One example that highlights the immense benefits of the modern data exploration model comes from the world of customer service at a major North American telecom company. Like many large corporations, this telecom provider had grown as a result of mergers and acquisitions. As the company grew, so to did its data infrastructure, leading to the use of many different data storage systems and analytics tools. Having data in many different systems can make understanding and serving customers difficult. In the traditional model, business leaders would work with IT to define the questions they wanted answered. IT would turn those questions into queries over a period of weeks or months, move the necessary data into a data warehouse to be analyzed, run the necessary queries and generate the reports containing answers to the initial questions. PAGE 8

9 In that model, IT becomes frustrated with the on-going development of queries and generation of reports. Business users get frustrated waiting for IT to generate answers to their questions. Both IT and business users become progressively less satisfied as the cycle repeats. In the modern data exploration model, business users can perform ad hoc queries, rather than having to formulate all of their questions up front. They get their questions answered quickly and can rapidly evolve their question set based on new answers. IT is freed up to work on important new initiatives rather than spending valuable time setting up new databases, designing schemas, writing queries and generating reports. In the case of this telecom provider, by leveraging the modern data exploration model, the company was able to predict customer churn and free up IT resources to work on other important new initiatives. From Data Silos to Actionable Insights In particular, this telecom and mobile communications provider combined multiple data sources, including customer information and dropped call records to make their data actionable. Through analytics, they were able to predict customer churn a critical capability in an industry known for high churn rates. They used Splunk Enterprise to receive, index and analyze streams of dropped call data in real time, and enriched this data stored natively in Splunk Enterprise with historical customer social media communications data stored in Hadoop. The telecom provider used this 360 degree view of the customer to assign a customer influencer score that indicated which customers had a high probability to churn. It also highlighted the customers that were connected with many friends, family, colleagues or other social media followers who therefore might influence other customers to churn. To make this integrated, real-time and historical data actionable, the telecom and mobile communications provider reported the customer influencer scores in the secure web dashboards used by call center and retail store employees to inform them which offers to give to which customers. For the customers with the highest influencer scores, the customer service team proactively called, ed or texted the customer to offer a new mobile phone or home femtocell device to reduce dropped calls. The result is effective predictive analytics that reduce churn. This directly addresses one of the top C-level business priorities for telecom and mobile communications providers. Churn is often the first question that financial analysts ask about in the quarterly earnings calls. The actionable insights came from the ability to combine multiple data sources and explore them. Having information about dropped calls at a particular cell tower, for example, didn t help the company serve an individual customer more effectively. But combining the knowledge that a customer was frequently affected by this cell tower at PAGE 9

10 home, the office or on the road with call data records and billing information led to powerful insights. Today, the company is able to address churn before it becomes an issue, experiment with new regional marketing programs and deliver bold new customer offers. More recently, the company has introduced major new initiatives to provide access to more frequent phone upgrades and more flexible voice and data billing plans. Business leaders at the company no longer need to pose their questions up front and wait for the answers. Instead, they re able to use the modern data exploration model to combine multiple data sources into real-time, actionable insights. TIME TO INSIGHT The power of the data exploration model isn't simply that it enables a more creative, more iterative data analytics process. It also enables business leaders to leverage data to take action a lot faster than they could in the past. In an era in which product reviews spread online like wildfire and news is distributed and re-distributed to hundreds of millions of viewers on every social media outlet in a matter of minutes, what matters most is the speed at which business leaders must make decisions. The Time Value of Data The sooner you analyze and act on your data, the more value it has. Questions of inventory, pricing, delivery, marketing and customer service are all impacted by a focus on velocity. Speed has always been a factor in business, but now, more often than ever, speed is the factor that matters m o s t w h e n i t c o m e s t o b u s i n e s s competitiveness. In retail and travel, online shoppers can compare prices faster than the systems of many merchants can update their prices to remain competitive. Those merchants that can t respond to real-time changes like these won t last long in highly competitive markets. In customer service, those companies that can t understand their customers and address their needs experience high churn rates. Just as in the financial world the concept of the time value of money is well-understood, when it comes to data, time is equally valuable. The sooner you analyze and act on PAGE 10

11 your data, the more value it has. Perhaps nowhere is acting on data quickly more critical than in disaster response. Real-World Use Case: Leveraging Social Media for Disaster Response In the fall of 2012, Hurricane Sandy hit the East Coast of the United States, causing an incredible loss of life and immense financial damage. Members of the Splunk4Good and Geeks Without Bounds teams partnered to perform a post-storm analysis of social media data related to the storm, to see if they could rapidly analyze a diverse data set to spot trends that could help future disaster relief efforts. Source: Wikipedia Splunk4Good and Geeks Without Bounds were able to demonstrate that large-scale social media data, including tweets from Twitter and images from Instagram, combined with the right software, could be very powerful in disaster situations. In particular, such a combination could be used to pinpoint where to direct relief aid in real time. It could enable agencies like the Federal Emergency Management Agency (FEMA) to perform sophisticated damage analysis all with publicly available social media data. Instagram photos from Hurricane Sandy. Source: PAGE 11

12 These are just a few of the many ways that diverse data sources in combination with software that can analyze unstructured and semi-structured data in real time can bring powerful benefits to public agencies and the people they serve. CONCLUSION Business users often associate Big Data with something that seems imposing, slow moving and difficult to work with. Meanwhile, IT frequently views Big Data as a burden, one that involves time-consuming and painstaking query development and reporting that takes valuable time away from critical new initiatives. A data exploration approach built on modern Big Data software can empower business leaders like you to make decisions more rapidly, creatively and effectively than you could in the past. The modern Data Exploration model: Is iterative and interactive Produces answers quickly Efficiently brings together numerous data sources From customer service to disaster recovery, the social and business impact of analyzing Big Data should not be discounted. PAGE 12

13 FOR MORE INFORMATION The Big Data Group The Big Data Group provides strategy consulting, market research and advisory services to technology buyers and vendors. The Big Data Group produces The Big Data Landscape and The Big Data Trends presentation, which together have been viewed more than 150,000 times. To find out more, please contact: David Feinleib Managing Director SPLUNK Download Splunk Enterprise for free. You ll get a Splunk Enterprise license for 60 days and you can index up to 500 megabytes of data per day. Download Hunk : Splunk Analytics for Hadoop for free for a 60-day trial to connect to any size Hadoop cluster. You can convert to a Splunk Enterprise perpetual Free license or purchase an Enterprise license for Splunk Enterprise or Hunk by contacting sales@splunk.com. PAGE 13